We were working with a team who were uploading data into HBASE using the doBulkLoad, also we had moved the /hbase to encryption zone.
The issue started after we completed the encryption for hbase directory /hbase.
The files which need to be loaded was created successfully but the load was failing.
Further looking into the issue we found that the staging directory (hbase.bulkload.staging.dir) which hbase doBulkLoad uses was outside the encryption zone, this needed to be inside.
ERROR
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles: Attempt to bulk load region containing into table ... failed. This is recoverable and they will be retired.
SOLUTION
We can get the configuration for this in hbase-site.xml. By default this is how the configuration looks like.
<property>
<name>hbase.bulkload.staging.dir</name>
<value>/tmp/hbase-staging</value>
</property>
We changed the configuration to make sure the staging directory was inside the encryption zone.
<property>
<name>hbase.bulkload.staging.dir</name>
<value>/hbase/hbase-staging</value>
</property>