Python Getting Started - Learning Notes.
Python is a clear and powerful object-oriented programming language, comparable to Perl, Ruby, Scheme, or Java. This is basic documentation for getting star...
Recent Posts
Cloud Monitoring - Stackdriver - GCP Learning Notes.
Stackdriver driver provides monitoring, logging, error reporting, tracing and debugging services. Currently known as Google Cloud’s operations suite.
Cloud Databases - SQL/Spanner/Firestore/BigTable/Memorystore - GCP Learning Notes.
Cloud SQL is a fully managed service which includes MySQL, PostgreSQL, Microsoft SQL Server. And for horizontal scalability, consider using Cloud Spanner.
getmerge Operation not Permitted
getmerge command takes a source directory and a destination file as input and concatenates files in src into the destination local file.
Cloud IAM - GCP Learning Notes.
Google Cloud offers IAM, which lets you give more granular access to specific Google Cloud resources and prevents unwanted access to other resources.
Cloud Storge Bucket - GCP Learning Notes.
Cloud Storage is Google Cloud’s object storage service, and it allows worldwide storage and retrieval of any amount of data at anytime.
Long Running Jobs in YARN distcp.
Usually distcp and other batch jobs run for a very long time. This is fine if you are running a HADOOP environment without kerberos. When we kerberize a clus...
ERROR - java heap space caused using distcp.
When you see this error default thing to do is the set the HADOOP_CLIENT_OPTS config on your env. In My case did a typo HADDOP_CLIENT_OPTS which caused us sp...
Bulk Upload Hfiles doBulkLoad - LoadIncrementalHfiles failed. This is recoverable. Hbase Encryption Zone.
We were working with a team who were uploading data into HBASE using the doBulkLoad, also we had moved the /hbase to encryption zone. The issue started after...
Checking HDFS health using fsck.
When we have large data sets on the cluster, there will be corruptions of blocks. This could be due to disk or any other.