Checking HDFS health using fsck.
When we have large data sets on the cluster, there will be corruptions of blocks. This could be due to disk or any other.
Recent Posts
Checking
HDFS Block Corrupted - HBase Error Could not seek StoreFileScanner Invalid HFile block magic
We ran into an issue which was causing HBase to fail. When the job was rerun then it would work again. In the logs we got 2 errors.
MongoDB to Neo4j Using neo4j_doc_manager.
We had a requirement where we wanted to have all the data which is in mongodb to be replicated on neo4j to show few graphs. Here is quick way to demonstrate ...
Variable Substitution in YAML Config pyyaml [Python]
PyYAML - The next generation YAML parser and emitter for Python.
Chef Kitchen Test Deployment on Multiple Nodes with Private Interface.
What is Kitchen? Kitchen provides a test harness to execute infrastructure code on one or more platforms in isolation.
Cropping Bulk Images Using Python.
I was working on getting post headers for my post on this blog. I had couple of images from unsplash. But the header for the post need to be a little more ho...
Installing Ruby2.4 on Ubuntu Docker Container.
Was setting up jekyll using docker, but was not able to install ruby on the docker machine as apt-add-repository was not working. Below is my Dockerfile when...
Kafka Kerberos Enable and Testing.
Apache Kafka is a distributed streaming platform. Kafka 2.0 supports Kerberos authentication, Enabling Kerberos Authentication Using the Wizard on cloudera m...
Cloudera Manager - Duplicate entry ‘zookeeper’ for key ‘NAME’.
Deployment error during setting zookeeper on a new cluster using Cloudera Manager UI, when an earlier cluster containing zookeeper earlier was built using Cl...
Parcel Not Distributing Cloudera CDH.
We were deploying one of the cluster on our lab environment which is used by everyone. So the lab has it own share of stale information on it.