Securely Manage Secrets with Google Cloud Secret Manager and Python
Managing secrets securely is crucial for any application, especially in cloud environments. Google Cloud Secret Manager provides a centralized and secure way...
Posts by Tag
API
AWS
Ansible and Terraform - A Powerful DevOps Combination for Cloud and On-Premises Infrastructure
Ansible and Terraform: The Perfect Pair for DevOps and Hybrid Cloud
Ad Blocking
Setting Up Pi-hole on a Raspberry Pi Zero 2W for Network-Wide Ad Blocking
Tired of intrusive ads slowing down your browsing and compromising your privacy? Want a single solution to block ads across all devices on your network? This...
Ansible
Event-Driven Ansible - Automating Responses to Real-Time Events: A Deep Dive
Introduction to Event-Driven Automation with Ansible
Ansible and Terraform - A Powerful DevOps Combination for Cloud and On-Premises Infrastructure
Ansible and Terraform: The Perfect Pair for DevOps and Hybrid Cloud
Ansible EDA
Event-Driven Ansible - Automating Responses to Real-Time Events: A Deep Dive
Introduction to Event-Driven Automation with Ansible
Ansible Rulebook
Event-Driven Ansible - Automating Responses to Real-Time Events: A Deep Dive
Introduction to Event-Driven Automation with Ansible
Automation
Event-Driven Ansible - Automating Responses to Real-Time Events: A Deep Dive
Introduction to Event-Driven Automation with Ansible
Ansible and Terraform - A Powerful DevOps Combination for Cloud and On-Premises Infrastructure
Ansible and Terraform: The Perfect Pair for DevOps and Hybrid Cloud
Automating Secret Management with Secret Manager and Cloud Build
Handling secrets like API keys and database passwords is a critical aspect of any CI/CD pipeline. Storing these secrets as plaintext in your code or configur...
Azure
Ansible and Terraform - A Powerful DevOps Combination for Cloud and On-Premises Infrastructure
Ansible and Terraform: The Perfect Pair for DevOps and Hybrid Cloud
BigQuery
CI/CD
Automating Secret Management with Secret Manager and Cloud Build
Handling secrets like API keys and database passwords is a critical aspect of any CI/CD pipeline. Storing these secrets as plaintext in your code or configur...
Cloud
Ansible and Terraform - A Powerful DevOps Combination for Cloud and On-Premises Infrastructure
Ansible and Terraform: The Perfect Pair for DevOps and Hybrid Cloud
Cloud Build
Automating Secret Management with Secret Manager and Cloud Build
Handling secrets like API keys and database passwords is a critical aspect of any CI/CD pipeline. Storing these secrets as plaintext in your code or configur...
Cloud Deployment Manager
Terraform vs Pulumi vs Cloud Deployment Manager - Choosing Your GCP IaC Tool
Infrastructure as Code (IaC) has become essential for managing cloud resources efficiently and consistently. Google Cloud Platform (GCP) offers several IaC t...
Compliance
Policy as Code - Enforcing Guardrails with Organization Policy Service
In today’s cloud environments, maintaining consistent security and configuration across a large organization can be a daunting task. Human error and unintent...
Configuration Management
Ansible and Terraform - A Powerful DevOps Combination for Cloud and On-Premises Infrastructure
Ansible and Terraform: The Perfect Pair for DevOps and Hybrid Cloud
Policy as Code - Enforcing Guardrails with Organization Policy Service
In today’s cloud environments, maintaining consistent security and configuration across a large organization can be a daunting task. Human error and unintent...
DNS
Setting Up Pi-hole on a Raspberry Pi Zero 2W for Network-Wide Ad Blocking
Tired of intrusive ads slowing down your browsing and compromising your privacy? Want a single solution to block ads across all devices on your network? This...
Event-Driven Automation
Event-Driven Ansible - Automating Responses to Real-Time Events: A Deep Dive
Introduction to Event-Driven Automation with Ansible
GCP
Ansible and Terraform - A Powerful DevOps Combination for Cloud and On-Premises Infrastructure
Ansible and Terraform: The Perfect Pair for DevOps and Hybrid Cloud
Automating Secret Management with Secret Manager and Cloud Build
Handling secrets like API keys and database passwords is a critical aspect of any CI/CD pipeline. Storing these secrets as plaintext in your code or configur...
Terraform vs Pulumi vs Cloud Deployment Manager - Choosing Your GCP IaC Tool
Infrastructure as Code (IaC) has become essential for managing cloud resources efficiently and consistently. Google Cloud Platform (GCP) offers several IaC t...
Securely Manage Secrets with Google Cloud Secret Manager and Python
Managing secrets securely is crucial for any application, especially in cloud environments. Google Cloud Secret Manager provides a centralized and secure way...
Google Cloud Operations Suite - A Deep Dive into Monitoring, Logging, Tracing, and Profiling
In today’s complex cloud environments, gaining a holistic view of your application’s health is crucial for ensuring performance, reliability, and cost-effect...
Policy as Code - Enforcing Guardrails with Organization Policy Service
In today’s cloud environments, maintaining consistent security and configuration across a large organization can be a daunting task. Human error and unintent...
Google Cloud
Automating Secret Management with Secret Manager and Cloud Build
Handling secrets like API keys and database passwords is a critical aspect of any CI/CD pipeline. Storing these secrets as plaintext in your code or configur...
Google Cloud Operations Suite - A Deep Dive into Monitoring, Logging, Tracing, and Profiling
In today’s complex cloud environments, gaining a holistic view of your application’s health is crucial for ensuring performance, reliability, and cost-effect...
Guardrails
Policy as Code - Enforcing Guardrails with Organization Policy Service
In today’s cloud environments, maintaining consistent security and configuration across a large organization can be a daunting task. Human error and unintent...
HADOOP
Kafka Kerberos Enable and Testing.
Apache Kafka is a distributed streaming platform. Kafka 2.0 supports Kerberos authentication, Enabling Kerberos Authentication Using the Wizard on cloudera m...
HCL
Terraform vs Pulumi vs Cloud Deployment Manager - Choosing Your GCP IaC Tool
Infrastructure as Code (IaC) has become essential for managing cloud resources efficiently and consistently. Google Cloud Platform (GCP) offers several IaC t...
IAM
Securely Manage Secrets with Google Cloud Secret Manager and Python
Managing secrets securely is crucial for any application, especially in cloud environments. Google Cloud Secret Manager provides a centralized and secure way...
IP aliasing
Configuring Multiple IP Addresses on a Single Network Interface in Linux
This comprehensive guide explains how to assign multiple IP addresses to a single network interface in Linux. This technique, often called IP aliasing or cre...
IT Automation
Event-Driven Ansible - Automating Responses to Real-Time Events: A Deep Dive
Introduction to Event-Driven Automation with Ansible
IaC
Ansible and Terraform - A Powerful DevOps Combination for Cloud and On-Premises Infrastructure
Ansible and Terraform: The Perfect Pair for DevOps and Hybrid Cloud
Terraform vs Pulumi vs Cloud Deployment Manager - Choosing Your GCP IaC Tool
Infrastructure as Code (IaC) has become essential for managing cloud resources efficiently and consistently. Google Cloud Platform (GCP) offers several IaC t...
JSON
Sending JSON from Browser/cURL to NodeJS to Kafka
This post outlines how to send JSON data from a client (browser or curl) through a NodeJS server and ultimately into a Kafka topic. This architecture is usef...
KB2995388
Fixing Not Enough Physical Memory Error in VMware Workstation 10 on Windows 8.1
Are you encountering the frustrating “Not enough physical memory is available to power this virtual machine” error in VMware Workstation 10 after a Windows u...
Logging
Google Cloud Operations Suite - A Deep Dive into Monitoring, Logging, Tracing, and Profiling
In today’s complex cloud environments, gaining a holistic view of your application’s health is crucial for ensuring performance, reliability, and cost-effect...
Metrics
Monitoring
Google Cloud Operations Suite - A Deep Dive into Monitoring, Logging, Tracing, and Profiling
In today’s complex cloud environments, gaining a holistic view of your application’s health is crucial for ensuring performance, reliability, and cost-effect...
NAT
Setting Up Port Forwarding in VirtualBox for Enhanced Network Performance
This blog post will guide you through configuring port forwarding in Oracle VirtualBox. We’ll address a common performance bottleneck experienced with bridge...
Network Security
Setting Up Pi-hole on a Raspberry Pi Zero 2W for Network-Wide Ad Blocking
Tired of intrusive ads slowing down your browsing and compromising your privacy? Want a single solution to block ads across all devices on your network? This...
Networking
Setting Up Port Forwarding in VirtualBox for Enhanced Network Performance
This blog post will guide you through configuring port forwarding in Oracle VirtualBox. We’ll address a common performance bottleneck experienced with bridge...
NoSQL
Setting up an HBase Cluster on Hadoop YARN - A Detailed Guide for Ubuntu 12.04 LTS
HBase is a powerful open-source, non-relational, distributed database designed for handling massive datasets. Inspired by Google’s BigTable, HBase is written...
Observability
On-Premises
Ansible and Terraform - A Powerful DevOps Combination for Cloud and On-Premises Infrastructure
Ansible and Terraform: The Perfect Pair for DevOps and Hybrid Cloud
Operations Suite
Google Cloud Operations Suite - A Deep Dive into Monitoring, Logging, Tracing, and Profiling
In today’s complex cloud environments, gaining a holistic view of your application’s health is crucial for ensuring performance, reliability, and cost-effect...
Organization Policy
Policy as Code - Enforcing Guardrails with Organization Policy Service
In today’s cloud environments, maintaining consistent security and configuration across a large organization can be a daunting task. Human error and unintent...
Pi-hole
Setting Up Pi-hole on a Raspberry Pi Zero 2W for Network-Wide Ad Blocking
Tired of intrusive ads slowing down your browsing and compromising your privacy? Want a single solution to block ads across all devices on your network? This...
Policy as Code
Policy as Code - Enforcing Guardrails with Organization Policy Service
In today’s cloud environments, maintaining consistent security and configuration across a large organization can be a daunting task. Human error and unintent...
Port Forwarding
Setting Up Port Forwarding in VirtualBox for Enhanced Network Performance
This blog post will guide you through configuring port forwarding in Oracle VirtualBox. We’ll address a common performance bottleneck experienced with bridge...
Privacy
Setting Up Pi-hole on a Raspberry Pi Zero 2W for Network-Wide Ad Blocking
Tired of intrusive ads slowing down your browsing and compromising your privacy? Want a single solution to block ads across all devices on your network? This...
Profiler
Google Cloud Operations Suite - A Deep Dive into Monitoring, Logging, Tracing, and Profiling
In today’s complex cloud environments, gaining a holistic view of your application’s health is crucial for ensuring performance, reliability, and cost-effect...
PromQL
Provisioning
Ansible and Terraform - A Powerful DevOps Combination for Cloud and On-Premises Infrastructure
Ansible and Terraform: The Perfect Pair for DevOps and Hybrid Cloud
Pulumi
Terraform vs Pulumi vs Cloud Deployment Manager - Choosing Your GCP IaC Tool
Infrastructure as Code (IaC) has become essential for managing cloud resources efficiently and consistently. Google Cloud Platform (GCP) offers several IaC t...
Python
Terraform vs Pulumi vs Cloud Deployment Manager - Choosing Your GCP IaC Tool
Infrastructure as Code (IaC) has become essential for managing cloud resources efficiently and consistently. Google Cloud Platform (GCP) offers several IaC t...
Securely Manage Secrets with Google Cloud Secret Manager and Python
Managing secrets securely is crucial for any application, especially in cloud environments. Google Cloud Secret Manager provides a centralized and secure way...
Raspberry Pi
Setting Up Pi-hole on a Raspberry Pi Zero 2W for Network-Wide Ad Blocking
Tired of intrusive ads slowing down your browsing and compromising your privacy? Want a single solution to block ads across all devices on your network? This...
Raspberry Pi Zero 2W
Setting Up Pi-hole on a Raspberry Pi Zero 2W for Network-Wide Ad Blocking
Tired of intrusive ads slowing down your browsing and compromising your privacy? Want a single solution to block ads across all devices on your network? This...
Remediation
Event-Driven Ansible - Automating Responses to Real-Time Events: A Deep Dive
Introduction to Event-Driven Automation with Ansible
Rulebooks
Event-Driven Ansible - Automating Responses to Real-Time Events: A Deep Dive
Introduction to Event-Driven Automation with Ansible
SLI
SSH
Setting Up Port Forwarding in VirtualBox for Enhanced Network Performance
This blog post will guide you through configuring port forwarding in Oracle VirtualBox. We’ll address a common performance bottleneck experienced with bridge...
Secret Manager
Automating Secret Management with Secret Manager and Cloud Build
Handling secrets like API keys and database passwords is a critical aspect of any CI/CD pipeline. Storing these secrets as plaintext in your code or configur...
Securely Manage Secrets with Google Cloud Secret Manager and Python
Managing secrets securely is crucial for any application, especially in cloud environments. Google Cloud Secret Manager provides a centralized and secure way...
Secrets Management
Securely Manage Secrets with Google Cloud Secret Manager and Python
Managing secrets securely is crucial for any application, especially in cloud environments. Google Cloud Secret Manager provides a centralized and secure way...
Security
Automating Secret Management with Secret Manager and Cloud Build
Handling secrets like API keys and database passwords is a critical aspect of any CI/CD pipeline. Storing these secrets as plaintext in your code or configur...
Securely Manage Secrets with Google Cloud Secret Manager and Python
Managing secrets securely is crucial for any application, especially in cloud environments. Google Cloud Secret Manager provides a centralized and secure way...
Policy as Code - Enforcing Guardrails with Organization Policy Service
In today’s cloud environments, maintaining consistent security and configuration across a large organization can be a daunting task. Human error and unintent...
Terraform
Ansible and Terraform - A Powerful DevOps Combination for Cloud and On-Premises Infrastructure
Ansible and Terraform: The Perfect Pair for DevOps and Hybrid Cloud
Terraform vs Pulumi vs Cloud Deployment Manager - Choosing Your GCP IaC Tool
Infrastructure as Code (IaC) has become essential for managing cloud resources efficiently and consistently. Google Cloud Platform (GCP) offers several IaC t...
Tracing
Google Cloud Operations Suite - A Deep Dive into Monitoring, Logging, Tracing, and Profiling
In today’s complex cloud environments, gaining a holistic view of your application’s health is crucial for ensuring performance, reliability, and cost-effect...
Ubuntu
Setting Up Port Forwarding in VirtualBox for Enhanced Network Performance
This blog post will guide you through configuring port forwarding in Oracle VirtualBox. We’ll address a common performance bottleneck experienced with bridge...
VirtualBox
Setting Up Port Forwarding in VirtualBox for Enhanced Network Performance
This blog post will guide you through configuring port forwarding in Oracle VirtualBox. We’ll address a common performance bottleneck experienced with bridge...
Webhooks
Event-Driven Ansible - Automating Responses to Real-Time Events: A Deep Dive
Introduction to Event-Driven Automation with Ansible
Windows 11
Setting Up Port Forwarding in VirtualBox for Enhanced Network Performance
This blog post will guide you through configuring port forwarding in Oracle VirtualBox. We’ll address a common performance bottleneck experienced with bridge...
YAML
Terraform vs Pulumi vs Cloud Deployment Manager - Choosing Your GCP IaC Tool
Infrastructure as Code (IaC) has become essential for managing cloud resources efficiently and consistently. Google Cloud Platform (GCP) offers several IaC t...
access control
Securing Your Data: A Step-by-Step Guide to Enabling HBase Authorization
Apache HBase is a powerful NoSQL database, but out-of-the-box, it’s a bit like a house with no locks on the doors. Any user who can connect to the cluster ca...
access-filter
Access Filter Setup with SSSD
If using access_provider = ldap, this option is mandatory. It specifies an LDAP search filter criteria that must be met for the user to be granted access on ...
active-directory
Redhat Intergration with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
Simple Steps to Intergrate RHEL with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
ad
Getting started with Hive with Kerberos.
Apache Hive is a powerful data warehousing application built on top of Hadoop; it enables you to access your data using Hive QL, a language that is similar t...
Redhat Intergration with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
Simple Steps to Intergrate RHEL with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
ad-hoc
Ad-Hoc Commands - The Unsung Hero of Ansible
Ansible is a powerful automation tool that allows you to manage and configure infrastructure as code. Back when we were running 100s of hadoop nodes these we...
alerting
Automating Firestore Health Checks with Terraform Cloud Functions Cloud Monitoring.
This blog post demonstrates how to automate Firestore health checks using Terraform to deploy a Cloud Function (Gen 1) and set up a complete monitoring and a...
alerts
Log-Based Metrics - Unlocking Insights from Your Application Logs
Application logs are a treasure trove of information. Beyond debugging and troubleshooting, they can provide valuable insights into your application’s perfor...
ansible
5 Common Ansible Mistakes and How to Avoid Them
Ansible is a powerful automation tool that simplifies infrastructure management and application deployment. However, like any technology, it’s easy to make m...
Ad-Hoc Commands - The Unsung Hero of Ansible
Ansible is a powerful automation tool that allows you to manage and configure infrastructure as code. Back when we were running 100s of hadoop nodes these we...
Creating an Ansible Collection - A Detailed Step-by-Step Guide
Creating an Ansible Collection: A Comprehensive Guide
Ansible 101- Your First Playbook to Automate Web Server Setup
Are you ready to dive into the world of automation? Ansible is a powerful tool that allows you to manage and configure systems with ease. This blog post will...
Google Compute Engine (GCE) Instance with Ansible
Using Ansible within the initialization script of a Google Compute Engine (GCE) instance can streamline and automate the configuration and setup of your virt...
Installing ansible on RHEL 6.6.
Ansible is a radically simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchest...
Ansible Playbook - Setup Zookeeper Using tarball.
This is a simple zookeeper playbook, to quickly start zookeeper running on a single or more nodes, in a clustered mode.
Ansible Playbook - Setup Storm Cluster.
This is a simple Storm Cluster Setup. We are using a dedicated Zookeeper Cluster/Node, instead of the standalone zkserver. Below is how we will deploy our c...
Ansible Playbook - Setup Kafka Cluster.
This is a simple Kafka setup. In this setup we are running kafka over a dedicated zookeeper service. (NOT the standalone zookeeper which comes with kafka)
Ansible Playbook - Setup Hadoop CDH5 Using tarball.
Setting up Hadoop using Ansible, we will be using cdh5 tarball for installation of the cluster.
apache
Ansible 101- Your First Playbook to Automate Web Server Setup
Are you ready to dive into the world of automation? Ansible is a powerful tool that allows you to manage and configure systems with ease. This blog post will...
apache kafka
Routing JSON Payloads to Multiple Kafka Topics with Node.js and Kafka-node
This guide demonstrates how to build a Node.js application that receives JSON data via HTTP and routes it to different Apache Kafka topics based on the URL p...
api
Creating an Ansible Collection - A Detailed Step-by-Step Guide
Creating an Ansible Collection: A Comprehensive Guide
apt-get
Fixing Unable to locate package oracle-java7-installer on Ubuntu
I recently ran into a frustrating issue while trying to install Java 7 on an older Ubuntu system (Ubuntu 13, in this case). The seemingly simple command sud...
auth
Explain - System Security Services Daemon (SSSD) Config.
The System Security Services Daemon (SSSD) provides a set of daemons to manage access to remote directories and authentication mechanisms. It provides Name S...
authentication
Explain - System Security Services Daemon (SSSD) Config.
The System Security Services Daemon (SSSD) provides a set of daemons to manage access to remote directories and authentication mechanisms. It provides Name S...
authorization
Securing Your Data: A Step-by-Step Guide to Enabling HBase Authorization
Apache HBase is a powerful NoSQL database, but out-of-the-box, it’s a bit like a house with no locks on the doors. Any user who can connect to the cluster ca...
automation
5 Common Ansible Mistakes and How to Avoid Them
Ansible is a powerful automation tool that simplifies infrastructure management and application deployment. However, like any technology, it’s easy to make m...
Ad-Hoc Commands - The Unsung Hero of Ansible
Ansible is a powerful automation tool that allows you to manage and configure infrastructure as code. Back when we were running 100s of hadoop nodes these we...
Creating an Ansible Collection - A Detailed Step-by-Step Guide
Creating an Ansible Collection: A Comprehensive Guide
Ansible 101- Your First Playbook to Automate Web Server Setup
Are you ready to dive into the world of automation? Ansible is a powerful tool that allows you to manage and configure systems with ease. This blog post will...
Testing Your Infrastructure Code - A Guide to Terratest on GCP
Infrastructure as Code (IaC) has revolutionized how we manage and provision infrastructure. Tools like Terraform allow us to define our infrastructure in cod...
Simple Script to Create Users on CentOS/RHEL 6.5
This script provides a simple way to create users on CentOS/RHEL 6.5 systems from the command line. It allows you to create both normal and sudo (administrat...
backup
Zabbix Database Backup Script for postgres and mysql.
After the zabbix setup is done, we would like to take backup of the database. Here we are doing a complete backup. But if we are looking for only configurati...
bash
Simple Script to Create Users on CentOS/RHEL 6.5
This script provides a simple way to create users on CentOS/RHEL 6.5 systems from the command line. It allows you to create both normal and sudo (administrat...
benchmarking
Load Testing with Siege on CentOS 6.5 - Kernel 2.6, x86_64 - A Comprehensive Guide
Siege is a powerful, open-source HTTP load testing and benchmarking utility. It’s designed to help web developers and system administrators measure the perfo...
best-practices
5 Common Ansible Mistakes and How to Avoid Them
Ansible is a powerful automation tool that simplifies infrastructure management and application deployment. However, like any technology, it’s easy to make m...
bi
Installing SpagoBI 5.1 on Centos 6.5 -Tomcat 7 with MySQL 5.6.
The SpagoBI project is a free software/open source initiative by the SpagoBI Labs of Engineering Group. It aims to realize the most complete 100% open source...
SpagoBI Tomcat Clustering with mod_jk and httpd on CentOS - In-Memory Session Replication - A Deep Dive
In a previous post, we explored a basic session-based replication setup for SpagoBI. However, that configuration lacked true session redundancy.
Setting up Tomcat Cluster for SpagoBI 5.1.
This post details how to set up a Tomcat cluster for SpagoBI 5.1, building upon a previous guide for installing SpagoBI with MySQL (post). This configuration...
big data
Setting up an HBase Cluster on Hadoop YARN - A Detailed Guide for Ubuntu 12.04 LTS
HBase is a powerful open-source, non-relational, distributed database designed for handling massive datasets. Inspired by Google’s BigTable, HBase is written...
bigquery
Exploring Data Extraction from BigQuery
BigQuery, Google’s fully-managed and serverless data warehouse, empowers organizations to analyze massive datasets with remarkable speed and efficiency. But ...
10 Tips for Optimizing Your Queries and Reducing Costs in BigQuery
Google BigQuery is a powerful and cost-effective data warehouse that enables you to analyze massive datasets quickly. However, to get the most out of BigQuer...
bridge
Comprehensive Guide to KVM Installation and Configuration on CentOS 6.x
Kernel-based Virtual Machine (KVM) has become a cornerstone of Linux server virtualization, known for its maturity and performance. Red Hat’s decision to fav...
cassandra
Creating a Multi-node Cassandra Cluster on Centos 6.5.
The Apache Cassandra database is the right choice when you need scalability and high availability without compromising performance. Linear scalability and pr...
cdh
Cloudera Hadoop Deployment using Cloudera Manager Cookbook [CentOS 6.6]
This cookbook [currently as of now] can be used to setup a Cloudera Manager Server (Management Server) running on MySQL/Postgres database. But the intended u...
centos
Kafka Kerberos Enable and Testing.
Apache Kafka is a distributed streaming platform. Kafka 2.0 supports Kerberos authentication, Enabling Kerberos Authentication Using the Wizard on cloudera m...
Cloudera Manager - Duplicate entry ‘zookeeper’ for key ‘NAME’.
Deployment error during setting zookeeper on a new cluster using Cloudera Manager UI, when an earlier cluster containing zookeeper earlier was built using Cl...
Parcel Not Distributing Cloudera CDH.
We were deploying one of the cluster on our lab environment which is used by everyone. So the lab has it own share of stale information on it.
Creating /etc/hosts file in Chef.
We had a cluster environment which we needed to update the /etc/hosts file. Which would help communicate between the server over a private network. Our serve...
Encrypted Data Bags - Chef
Data Bags are a way to store information on the chef-server which all the cookbooks can access. Few more additional advantages are that we can encrypt the da...
knife Quick Reference
This is a quick reference for few of the commands which I use often.
Remove Old Files using find Command
GNU find searches the directory tree rooted at each given file name by evaluating the given expression from left to right, according to the rules of preceden...
Moving RRD file from 32bit to 64bit Architecture
When we were working on a nagios monitoring system we were migrating from a 32bit nagios to a 64bit Architecture. Most of the graphs are not working as the R...
Migrating Nagios XI to a New Server on Centos6.6
Migrating an old Nagios backup to a new nagiosxi server. Migration is simple in Nagios XI, you a backup and restore it. Once that is done we need to update/a...
Setting up SSL https On Nagios XI Server
HTTPS is a protocol for secure communication over a computer network which is widely used on the Internet. HTTPS consists of communication over Hypertext Tra...
Standalone Chef Server / Workstation / Client Setup on CentOS 6
The standalone installation of Chef server creates a working installation on a single server. This installation is also useful when you are installing Chef s...
Creating Documents Using pandoc
Pandoc is an opensource utility to create documents from markdown. We can create PDF, Doc, doc, html and other formats. And can be also used to convert html ...
Setup/Configuration Nagios XI on Centos6.6
Nagios monitors your entire IT infrastructure to ensure systems, applications, services, and business processes are functioning properly. In the event of a f...
RPM Command Cheat Sheet
RPM (Redhat Package Manager) is the most popular package utility and is used mostly on RHEL, Centos and Fedora. RPM helps user/admins to build, install, que...
Creating a large file in Windows / Linux
Was working on a monitoring project, need to create a large file to test notifications. Here is how we can do that.
Cloudera Hadoop Deployment using Cloudera Manager Cookbook [CentOS 6.6]
This cookbook [currently as of now] can be used to setup a Cloudera Manager Server (Management Server) running on MySQL/Postgres database. But the intended u...
Check Port on Remote Server CentOS 6.6/RHEL 6
Checking port available on a remote machine using nc command instead of telnet. Same command can be used to check on a remote server as well, change the 127....
LUKS Disk encryption for CentOS 6.6/RHEL 6
Linux Unified Key Setup-on-disk-format (or LUKS) allows you to encrypt partitions on your Linux computer. This is particularly important when it comes to mo...
Mysql Database Moving Data Directory to New Location.
How to move an existing data directory in mysql to a new location. We were running out of space and had to move the existing data directory to a new drive. B...
Mysql Database Disk Usage.
We were running out of disk space on one of the databases server, we need to get information on what the current table/database usage was. Below are few comm...
Installing python 2.7.x on Centos 6.5/6.6
By default centos comes with python 2.6. In most of the cases we might need python 2.7 or later to be installed. Below are few ways to install python 2.7 on ...
Zabbix Database Backup Script for postgres and mysql.
After the zabbix setup is done, we would like to take backup of the database. Here we are doing a complete backup. But if we are looking for only configurati...
Installing Zabbix Version 2.4 Offline (Zabbix Server without Internet).
There might be situations where you have a remote/zabbix server which does not have internet connectivity, due to security or other reasons. So we create a ...
Zabbix Template Creation using CSV file.
In zabbix we dont have a better way to capture snmptraps. we have to manually create an item and corresponfding trigger to handle a trap arriving from the de...
Installing SNMP Builder using zabbix-extra on Zabbix Version 2.4.
SNMP Builder/Extra is an add-in for Zabbix. It provides new web interface components to browse MIB trees and values. SNMP OIDs can then be automatically conv...
Setting up SNMP Trapper for Zabbix.
Receiving SNMP traps is the opposite to querying SNMP-enabled devices. In this case the information is sent from a SNMP-enabled device and is collected or “t...
YUM Repository Creation on HTTPD Web Server.
Setting up yum repos on RHEL using httpd. We will be setting up httpd and yum repo on top of it. So that we can access yum over http.
Access Filter Setup with SSSD
If using access_provider = ldap, this option is mandatory. It specifies an LDAP search filter criteria that must be met for the user to be granted access on ...
Redhat Intergration with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
Simple Steps to Intergrate RHEL with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
Installing ansible on RHEL 6.6.
Ansible is a radically simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchest...
Creating a passwordless entry on Centos 6.5.
You can enter a linux system without entring a password using below steps. We will be creating a ssh jey which will be share between the servers, which will ...
Creating a Multi-node Cassandra Cluster on Centos 6.5.
The Apache Cassandra database is the right choice when you need scalability and high availability without compromising performance. Linear scalability and pr...
Installing SpagoBI 5.1 on Centos 6.5 -Tomcat 7 with MySQL 5.6.
The SpagoBI project is a free software/open source initiative by the SpagoBI Labs of Engineering Group. It aims to realize the most complete 100% open source...
Comprehensive Guide to KVM Installation and Configuration on CentOS 6.x
Kernel-based Virtual Machine (KVM) has become a cornerstone of Linux server virtualization, known for its maturity and performance. Red Hat’s decision to fav...
Installing zabbix-java-gateway on Centos 6.5
Zabbix 2.0 introduced native support for monitoring JMX applications through the Zabbix Java gateway. This Java-based daemon acts as an intermediary, allowin...
Setting Up HAProxy for High Availability and Load Balancing on CentOS
HAProxy is a robust, high-performance TCP/HTTP load balancer, perfectly suited for ensuring high availability and efficient traffic distribution across your ...
Installing NodeJS on CentOS 6.6 - A Comprehensive Guide
Node.js is an open-source, cross-platform JavaScript runtime environment that executes JavaScript code server-side. It’s incredibly popular for building scal...
Simple Script to Create Users on CentOS/RHEL 6.5
This script provides a simple way to create users on CentOS/RHEL 6.5 systems from the command line. It allows you to create both normal and sudo (administrat...
Setting up Highly Available Apache (httpd) with Heartbeat on CentOS/RHEL 6
This guide walks you through configuring a High Availability (HA) Apache web server using Heartbeat on CentOS/RHEL 6. HA ensures your web server remains acce...
Zabbix Installation 2.4 - CentOS 6.5
Zabbix is the ultimate enterprise-level software designed for monitoring availability and performance of IT infrastructure components. Zabbix is open source ...
chef
Chef Kitchen Test Deployment on Multiple Nodes with Private Interface.
What is Kitchen? Kitchen provides a test harness to execute infrastructure code on one or more platforms in isolation.
Creating /etc/hosts file in Chef.
We had a cluster environment which we needed to update the /etc/hosts file. Which would help communicate between the server over a private network. Our serve...
Windows Testing Using Kitchen Chef
Kitchen-Vagrant has the capability to spin up a windows instance for testing. To make it work you will need the vagrant-winrm to be installted on the worksta...
Encrypted Data Bags - Chef
Data Bags are a way to store information on the chef-server which all the cookbooks can access. Few more additional advantages are that we can encrypt the da...
Bootstrap Windows - knife-windows - Chef
knife-windows plugin adds additional functionality to the Chef Knife CLI tool for configuring / interacting with nodes running Microsoft Windows.
Standalone Chef Server / Workstation / Client Setup on CentOS 6
The standalone installation of Chef server creates a working installation on a single server. This installation is also useful when you are installing Chef s...
Cloudera Hadoop Deployment using Cloudera Manager Cookbook [CentOS 6.6]
This cookbook [currently as of now] can be used to setup a Cloudera Manager Server (Management Server) running on MySQL/Postgres database. But the intended u...
Chef Workstation Setup on Windows Machine.
The Chef Development Kit (ChefDK) brings the best-of-breed development tools built by the awesome Chef community to your workstation with just a few clicks. ...
chef-client
Bootstrap Windows - knife-windows - Chef
knife-windows plugin adds additional functionality to the Chef Knife CLI tool for configuring / interacting with nodes running Microsoft Windows.
chefdk
Standalone Chef Server / Workstation / Client Setup on CentOS 6
The standalone installation of Chef server creates a working installation on a single server. This installation is also useful when you are installing Chef s...
Cloudera Hadoop Deployment using Cloudera Manager Cookbook [CentOS 6.6]
This cookbook [currently as of now] can be used to setup a Cloudera Manager Server (Management Server) running on MySQL/Postgres database. But the intended u...
Chef Workstation Setup on Windows Machine.
The Chef Development Kit (ChefDK) brings the best-of-breed development tools built by the awesome Chef community to your workstation with just a few clicks. ...
cloud
Cloud VPN - GCP Learning Notes.
Cloud VPN securely connects your peer network to your Virtual Private Cloud (VPC) network through an IPsec VPN connection.
Cloud Monitoring - Stackdriver - GCP Learning Notes.
Stackdriver driver provides monitoring, logging, error reporting, tracing and debugging services. Currently known as Google Cloud’s operations suite.
Cloud Databases - SQL/Spanner/Firestore/BigTable/Memorystore - GCP Learning Notes.
Cloud SQL is a fully managed service which includes MySQL, PostgreSQL, Microsoft SQL Server. And for horizontal scalability, consider using Cloud Spanner.
Cloud IAM - GCP Learning Notes.
Google Cloud offers IAM, which lets you give more granular access to specific Google Cloud resources and prevents unwanted access to other resources.
Cloud Storge Bucket - GCP Learning Notes.
Cloud Storage is Google Cloud’s object storage service, and it allows worldwide storage and retrieval of any amount of data at anytime.
cloud datastore
Monitoring Firestore Health - A Python-Based Uptime Check
Ensuring the health and availability of your Firestore database is crucial for maintaining a reliable application. This post provides a Python script that se...
cloud-logging
Log-Based Metrics - Unlocking Insights from Your Application Logs
Application logs are a treasure trove of information. Beyond debugging and troubleshooting, they can provide valuable insights into your application’s perfor...
cloud-monitoring
cloudera
Long Running Jobs in YARN distcp.
Usually distcp and other batch jobs run for a very long time. This is fine if you are running a HADOOP environment without kerberos. When we kerberize a clus...
Bulk Upload Hfiles doBulkLoad - LoadIncrementalHfiles failed. This is recoverable. Hbase Encryption Zone.
We were working with a team who were uploading data into HBASE using the doBulkLoad, also we had moved the /hbase to encryption zone. The issue started after...
Checking HDFS health using fsck.
When we have large data sets on the cluster, there will be corruptions of blocks. This could be due to disk or any other.
HDFS Block Corrupted - HBase Error Could not seek StoreFileScanner Invalid HFile block magic
We ran into an issue which was causing HBase to fail. When the job was rerun then it would work again. In the logs we got 2 errors.
Kafka Kerberos Enable and Testing.
Apache Kafka is a distributed streaming platform. Kafka 2.0 supports Kerberos authentication, Enabling Kerberos Authentication Using the Wizard on cloudera m...
Cloudera Manager - Duplicate entry ‘zookeeper’ for key ‘NAME’.
Deployment error during setting zookeeper on a new cluster using Cloudera Manager UI, when an earlier cluster containing zookeeper earlier was built using Cl...
Parcel Not Distributing Cloudera CDH.
We were deploying one of the cluster on our lab environment which is used by everyone. So the lab has it own share of stale information on it.
Enable Kerberos Using Cloudera API.
Python API for cloudera is really nice, apart from getting the cluster setup, we can also do configuration and automation. We use a lot of automation using C...
Setting Up HDFS Services Using Cloudera API [Part 3]
This is the second follow up post. In the earlier post
Setting Up Zookeeper Services Using Cloudera API [Part 2]
This is the second follow up post. In the earlier post Setting Up Cloudera Manager Services Using Cloudera API [Part 1] we install the cloudera management se...
Setting Up Cloudera Manager Services Using Cloudera API [Part 1]
Cloudera API is a very convenient way to setup a cluster and do more.
Getting Started with Cloudera API
This is a basic steps to get connected with cloudera manager.
Basic Testing On Hadoop Environment [Cloudera]
These are a set of testing which we can do on a Hadoop environment. These are basic testing to make sure the environment is setup correctly.
Setting Hue to Listen on 0.0.0.0 [Cloudera]
We were working on setting up a cluster, but the Hue URL was set to a private IP of the server. As we had setup all the nodes to access each other using a pr...
Update Cloudera Manager to specific version [5.4.5]
Upgrading Cloudera Manager 5 to the Latest Cloudera Manager, In most cases it is possible to complete the following upgrade without shutting down most CDH se...
Getting started with Hive with Kerberos.
Apache Hive is a powerful data warehousing application built on top of Hadoop; it enables you to access your data using Hive QL, a language that is similar t...
Setting up Pentaho Data Integration 5.4.1 with Hadoop Cluster (Clouder Manager)
Pentaho Data Integration (PDI) is a suite of open source Business Intelligence (BI) products which provide data integration, OLAP services, reporting, dashb...
cloudera-api
Enable Kerberos Using Cloudera API.
Python API for cloudera is really nice, apart from getting the cluster setup, we can also do configuration and automation. We use a lot of automation using C...
Setting Up HDFS Services Using Cloudera API [Part 3]
This is the second follow up post. In the earlier post
Setting Up Zookeeper Services Using Cloudera API [Part 2]
This is the second follow up post. In the earlier post Setting Up Cloudera Manager Services Using Cloudera API [Part 1] we install the cloudera management se...
Setting Up Cloudera Manager Services Using Cloudera API [Part 1]
Cloudera API is a very convenient way to setup a cluster and do more.
Getting Started with Cloudera API
This is a basic steps to get connected with cloudera manager.
cloudera-manager
Cloudera Hadoop Deployment using Cloudera Manager Cookbook [CentOS 6.6]
This cookbook [currently as of now] can be used to setup a Cloudera Manager Server (Management Server) running on MySQL/Postgres database. But the intended u...
cloudfunctions
Automating Firestore Health Checks with Terraform Cloud Functions Cloud Monitoring.
This blog post demonstrates how to automate Firestore health checks using Terraform to deploy a Cloud Function (Gen 1) and set up a complete monitoring and a...
cloudscheduler
Automating Firestore Health Checks with Terraform Cloud Functions Cloud Monitoring.
This blog post demonstrates how to automate Firestore health checks using Terraform to deploy a Cloud Function (Gen 1) and set up a complete monitoring and a...
cluster
Kafka Kerberos Enable and Testing.
Apache Kafka is a distributed streaming platform. Kafka 2.0 supports Kerberos authentication, Enabling Kerberos Authentication Using the Wizard on cloudera m...
Cloudera Manager - Duplicate entry ‘zookeeper’ for key ‘NAME’.
Deployment error during setting zookeeper on a new cluster using Cloudera Manager UI, when an earlier cluster containing zookeeper earlier was built using Cl...
Parcel Not Distributing Cloudera CDH.
We were deploying one of the cluster on our lab environment which is used by everyone. So the lab has it own share of stale information on it.
Creating /etc/hosts file in Chef.
We had a cluster environment which we needed to update the /etc/hosts file. Which would help communicate between the server over a private network. Our serve...
High Availability Nginx with Heartbeat - A Step-by-Step Guide
In today’s digital landscape, ensuring high availability for your web applications is paramount. Downtime can lead to lost revenue, damaged reputation, and ...
clustering
SpagoBI Tomcat Clustering with mod_jk and httpd on CentOS - In-Memory Session Replication - A Deep Dive
In a previous post, we explored a basic session-based replication setup for SpagoBI. However, that configuration lacked true session redundancy.
cman
Creating a two-node RHEL cluster with Virtual IP using CMAN and Pacemaker.
CMAN v3 is a Corsync plugin that monitors the names and number of active cluster nodes in order to deliver membership and quorum information to clients (such...
collection
Creating an Ansible Collection - A Detailed Step-by-Step Guide
Creating an Ansible Collection: A Comprehensive Guide
command-line
Ad-Hoc Commands - The Unsung Hero of Ansible
Ansible is a powerful automation tool that allows you to manage and configure infrastructure as code. Back when we were running 100s of hadoop nodes these we...
commands
Remove Old Files using find Command
GNU find searches the directory tree rooted at each given file name by evaluating the given expression from left to right, according to the rules of preceden...
config
How To Configure Swappiness
Swappiness is a crucial Linux kernel parameter that dictates how aggressively the system uses swap space. Understanding and configuring swappiness can signif...
configuration
Navigating npm Behind a Corporate Proxy - A Comprehensive Guide
npm, short for Node Package Manager, is the cornerstone of JavaScript development within the Node.js ecosystem. It simplifies the process of managing projec...
configuration-management
5 Common Ansible Mistakes and How to Avoid Them
Ansible is a powerful automation tool that simplifies infrastructure management and application deployment. However, like any technology, it’s easy to make m...
coprocessor
Securing Your Data: A Step-by-Step Guide to Enabling HBase Authorization
Apache HBase is a powerful NoSQL database, but out-of-the-box, it’s a bit like a house with no locks on the doors. Any user who can connect to the cluster ca...
couchdb
Installing CouchDB on Ubuntu 14 LTS.
CouchDB is a database that completely embraces the web. Store your data with JSON documents. Access your documents and query your indexes with your web brows...
create
Simple Script to Create Users on CentOS/RHEL 6.5
This script provides a simple way to create users on CentOS/RHEL 6.5 systems from the command line. It allows you to create both normal and sudo (administrat...
csv-processing
Zabbix Hadoop Monitoring.
This script can be used to monitor Namenode Parameters. This script can be used to Generate Zabbix Import XML or Send monitoring data to Zabbix server.
Fetch Monitoring Data from Item to CSV file.
This section describes how to get monitoring data from Zabbix server through the command line and export these to a CSV formatted file.
Zabbix Item Creation using OID and Name CSV files.
This script is to create a xml import file from a csv file. We use 2 csv file for this.
Zabbix Import File from iReasoning Export CSV
Module used to generate Zabbix import xml for snmp traps, Below are steps to create a Zabbix Import for Traps. These traps are captured from the snmptrap fun...
cygwin
Package Installer for Cygwin [apt-cyg].
After a longtime I was on my windows machine and had to make it feel more like my linux machine. So install the thing what everyone else does cygwin. Surpise...
dashboards
Log-Based Metrics - Unlocking Insights from Your Application Logs
Application logs are a treasure trove of information. Beyond debugging and troubleshooting, they can provide valuable insights into your application’s perfor...
data-collector
SFTP Data Collector
Easy way to collect files recursively over a sftp server is to connect to the server over scp and do scp -r. Problem was that the device we were connecting ...
data-integration
Setting up Pentaho Data Integration 5.4.1 with Hadoop Cluster (Clouder Manager)
Pentaho Data Integration (PDI) is a suite of open source Business Intelligence (BI) products which provide data integration, OLAP services, reporting, dashb...
data-processing
Streaming Data Processing - Storm Vs Spark.
Apache Spark is an in-memory distributed data analysis platform– primarily targeted at speeding up batch analysis jobs, iterative machine learning jobs, inte...
dataflow
Setting up Scheduled dataflow backups using Batch templates.
TL;DR In this post we will be setting up a scheduled job to take backup for Bigtable table in avro format.
dataproc
Setting Up Kerberos on a Dataproc Cluster A Step-by-Step Guide
Kerberos authentication in Google Dataproc clusters offers enhanced security and user isolation. It’s a crucial step for multi-tenancy, encryption, and user ...
datastream
Datastream Workflow - Cloud SQL (MySQL) to BigQuery via Cloud Auth Proxy
Welcome back to our series on datastream workflows! In our previous article, we explored the basics of setting up a datastream workflow.
Establishing a Datastream from Cloud SQL (MySQL) to BigQuery
This guide provides detailed instructions on setting up a datastream connection from Cloud SQL (MySQL) to BigQuery. We aim to simplify the process by using d...
devops
5 Common Ansible Mistakes and How to Avoid Them
Ansible is a powerful automation tool that simplifies infrastructure management and application deployment. However, like any technology, it’s easy to make m...
Ad-Hoc Commands - The Unsung Hero of Ansible
Ansible is a powerful automation tool that allows you to manage and configure infrastructure as code. Back when we were running 100s of hadoop nodes these we...
Creating an Ansible Collection - A Detailed Step-by-Step Guide
Creating an Ansible Collection: A Comprehensive Guide
Ansible 101- Your First Playbook to Automate Web Server Setup
Are you ready to dive into the world of automation? Ansible is a powerful tool that allows you to manage and configure systems with ease. This blog post will...
disk
LUKS Disk encryption for CentOS 6.6/RHEL 6
Linux Unified Key Setup-on-disk-format (or LUKS) allows you to encrypt partitions on your Linux computer. This is particularly important when it comes to mo...
distcp
Long Running Jobs in YARN distcp.
Usually distcp and other batch jobs run for a very long time. This is fine if you are running a HADOOP environment without kerberos. When we kerberize a clus...
ERROR - java heap space caused using distcp.
When you see this error default thing to do is the set the HADOOP_CLIENT_OPTS config on your env. In My case did a typo HADDOP_CLIENT_OPTS which caused us sp...
docker
Installing Ruby2.4 on Ubuntu Docker Container.
Was setting up jekyll using docker, but was not able to install ruby on the docker machine as apt-add-repository was not working. Below is my Dockerfile when...
documents
Creating Documents Using pandoc
Pandoc is an opensource utility to create documents from markdown. We can create PDF, Doc, doc, html and other formats. And can be also used to convert html ...
dpkg-reconfigure
Changing Timezone in Ubuntu Server - A Comprehensive Guide
Changing the timezone on your Ubuntu server is a crucial task for ensuring accurate timekeeping, especially when dealing with scheduled tasks, logs, and appl...
encryption
LUKS Disk encryption for CentOS 6.6/RHEL 6
Linux Unified Key Setup-on-disk-format (or LUKS) allows you to encrypt partitions on your Linux computer. This is particularly important when it comes to mo...
error
Nagios - Service Group Summary ERROR
We were working on nagios and found that after our migration, service group summary was not working. You might get below error on the screen and the solution...
No valid credentials provided Mechanism level Failed to find any Kerberos tgt
We have 2 domains forests in our environment, ABC and XYZ. We were not able to authenticate normal users from either of the domains. Most of the information ...
Fixing Not Enough Physical Memory Error in VMware Workstation 10 on Windows 8.1
Are you encountering the frustrating “Not enough physical memory is available to power this virtual machine” error in VMware Workstation 10 after a Windows u...
exec
Harnessing the Power of null_resource with local-exec and remote-exec in Terraform
Terraform, a widely adopted Infrastructure as Code (IaC) tool, offers a rich array of resources and provisioners for efficiently managing and configuring inf...
export
Data Export from Datastore and Firestore
In this blog post, we will venture into exporting data from Firestore and Datastore modes. Find the code on Github.
failover
High Availability Nginx with Heartbeat - A Step-by-Step Guide
In today’s digital landscape, ensuring high availability for your web applications is paramount. Downtime can lead to lost revenue, damaged reputation, and ...
firestore
Automating Firestore Health Checks with Terraform Cloud Functions Cloud Monitoring.
This blog post demonstrates how to automate Firestore health checks using Terraform to deploy a Cloud Function (Gen 1) and set up a complete monitoring and a...
Monitoring Firestore Health - A Python-Based Uptime Check
Ensuring the health and availability of your Firestore database is crucial for maintaining a reliable application. This post provides a Python script that se...
ftp
SFTP Data Collector
Easy way to collect files recursively over a sftp server is to connect to the server over scp and do scp -r. Problem was that the device we were connecting ...
ftp-server
Fine-Tuning Linux Kernel Parameters with sysctl.conf for Web and FTP Servers
The sysctl interface provides a powerful way to modify the behavior of a running Linux kernel. By editing the /etc/sysctl.conf file, you can persistently co...
galaxy
Creating an Ansible Collection - A Detailed Step-by-Step Guide
Creating an Ansible Collection: A Comprehensive Guide
gce
Automating GCE Image Creation with Packer
Ensuring consistency and streamlining infrastructure provisioning is crucial for effective cloud management. Creating custom virtual machine (VM) images on G...
Google Compute Engine (GCE) Instance with Ansible
Using Ansible within the initialization script of a Google Compute Engine (GCE) instance can streamline and automate the configuration and setup of your virt...
gcp
Testing Your Infrastructure Code - A Guide to Terratest on GCP
Infrastructure as Code (IaC) has revolutionized how we manage and provision infrastructure. Tools like Terraform allow us to define our infrastructure in cod...
Automating Firestore Health Checks with Terraform Cloud Functions Cloud Monitoring.
This blog post demonstrates how to automate Firestore health checks using Terraform to deploy a Cloud Function (Gen 1) and set up a complete monitoring and a...
Cloud VPN - GCP Learning Notes.
Cloud VPN securely connects your peer network to your Virtual Private Cloud (VPC) network through an IPsec VPN connection.
Cloud Monitoring - Stackdriver - GCP Learning Notes.
Stackdriver driver provides monitoring, logging, error reporting, tracing and debugging services. Currently known as Google Cloud’s operations suite.
Cloud Databases - SQL/Spanner/Firestore/BigTable/Memorystore - GCP Learning Notes.
Cloud SQL is a fully managed service which includes MySQL, PostgreSQL, Microsoft SQL Server. And for horizontal scalability, consider using Cloud Spanner.
Cloud IAM - GCP Learning Notes.
Google Cloud offers IAM, which lets you give more granular access to specific Google Cloud resources and prevents unwanted access to other resources.
Cloud Storge Bucket - GCP Learning Notes.
Cloud Storage is Google Cloud’s object storage service, and it allows worldwide storage and retrieval of any amount of data at anytime.
gcs
Design IAM on Google Cloud Platform
Security is paramount when setting up resources in the cloud or on-premises. It encompasses various layers of protection, including network security, encrypt...
Harnessing the Power of null_resource with local-exec and remote-exec in Terraform
Terraform, a widely adopted Infrastructure as Code (IaC) tool, offers a rich array of resources and provisioners for efficiently managing and configuring inf...
getmerge
getmerge Operation not Permitted
getmerge command takes a source directory and a destination file as input and concatenates files in src into the destination local file.
github
SFTP Data Collector
Easy way to collect files recursively over a sftp server is to connect to the server over scp and do scp -r. Problem was that the device we were connecting ...
go
Testing Your Infrastructure Code - A Guide to Terratest on GCP
Infrastructure as Code (IaC) has revolutionized how we manage and provision infrastructure. Tools like Terraform allow us to define our infrastructure in cod...
google cloud
Monitoring Firestore Health - A Python-Based Uptime Check
Ensuring the health and availability of your Firestore database is crucial for maintaining a reliable application. This post provides a Python script that se...
google-cloud
gparted
Mounting RAID10 using parted.
GUID Partition Table (GPT) is a standard for the layout of the partition table on a physical hard disk, using globally unique identifiers (GUID). Although it...
ha
Creating a two-node RHEL cluster with Virtual IP using CMAN and Pacemaker.
CMAN v3 is a Corsync plugin that monitors the names and number of active cluster nodes in order to deliver membership and quorum information to clients (such...
Setting Up HAProxy for High Availability and Load Balancing on CentOS
HAProxy is a robust, high-performance TCP/HTTP load balancer, perfectly suited for ensuring high availability and efficient traffic distribution across your ...
hadoop
getmerge Operation not Permitted
getmerge command takes a source directory and a destination file as input and concatenates files in src into the destination local file.
Long Running Jobs in YARN distcp.
Usually distcp and other batch jobs run for a very long time. This is fine if you are running a HADOOP environment without kerberos. When we kerberize a clus...
ERROR - java heap space caused using distcp.
When you see this error default thing to do is the set the HADOOP_CLIENT_OPTS config on your env. In My case did a typo HADDOP_CLIENT_OPTS which caused us sp...
Bulk Upload Hfiles doBulkLoad - LoadIncrementalHfiles failed. This is recoverable. Hbase Encryption Zone.
We were working with a team who were uploading data into HBASE using the doBulkLoad, also we had moved the /hbase to encryption zone. The issue started after...
Checking HDFS health using fsck.
When we have large data sets on the cluster, there will be corruptions of blocks. This could be due to disk or any other.
HDFS Block Corrupted - HBase Error Could not seek StoreFileScanner Invalid HFile block magic
We ran into an issue which was causing HBase to fail. When the job was rerun then it would work again. In the logs we got 2 errors.
Parcel Not Distributing Cloudera CDH.
We were deploying one of the cluster on our lab environment which is used by everyone. So the lab has it own share of stale information on it.
Creating /etc/hosts file in Chef.
We had a cluster environment which we needed to update the /etc/hosts file. Which would help communicate between the server over a private network. Our serve...
Enable Kerberos Using Cloudera API.
Python API for cloudera is really nice, apart from getting the cluster setup, we can also do configuration and automation. We use a lot of automation using C...
Setting Up HDFS Services Using Cloudera API [Part 3]
This is the second follow up post. In the earlier post
Setting Up Zookeeper Services Using Cloudera API [Part 2]
This is the second follow up post. In the earlier post Setting Up Cloudera Manager Services Using Cloudera API [Part 1] we install the cloudera management se...
Setting Up Cloudera Manager Services Using Cloudera API [Part 1]
Cloudera API is a very convenient way to setup a cluster and do more.
Getting Started with Cloudera API
This is a basic steps to get connected with cloudera manager.
Basic Testing On Hadoop Environment [Cloudera]
These are a set of testing which we can do on a Hadoop environment. These are basic testing to make sure the environment is setup correctly.
Setting Hue to Listen on 0.0.0.0 [Cloudera]
We were working on setting up a cluster, but the Hue URL was set to a private IP of the server. As we had setup all the nodes to access each other using a pr...
YUM Repository Creation on HTTPD Web Server.
Setting up yum repos on RHEL using httpd. We will be setting up httpd and yum repo on top of it. So that we can access yum over http.
Access Filter Setup with SSSD
If using access_provider = ldap, this option is mandatory. It specifies an LDAP search filter criteria that must be met for the user to be granted access on ...
Update Cloudera Manager to specific version [5.4.5]
Upgrading Cloudera Manager 5 to the Latest Cloudera Manager, In most cases it is possible to complete the following upgrade without shutting down most CDH se...
Getting started with Hive with Kerberos.
Apache Hive is a powerful data warehousing application built on top of Hadoop; it enables you to access your data using Hive QL, a language that is similar t...
Redhat Intergration with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
Simple Steps to Intergrate RHEL with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
Setting up Pentaho Data Integration 5.4.1 with Hadoop Cluster (Clouder Manager)
Pentaho Data Integration (PDI) is a suite of open source Business Intelligence (BI) products which provide data integration, OLAP services, reporting, dashb...
No valid credentials provided Mechanism level Failed to find any Kerberos tgt
We have 2 domains forests in our environment, ABC and XYZ. We were not able to authenticate normal users from either of the domains. Most of the information ...
Interface Forwarding - from eth1 to eth0 on EDGE node.
Adding route to all the slaves which reside on a private network to communicate with External Server directly using an EDGE node using Interface Forwarding.
Installing ansible on RHEL 6.6.
Ansible is a radically simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchest...
Ansible Playbook - Setup Zookeeper Using tarball.
This is a simple zookeeper playbook, to quickly start zookeeper running on a single or more nodes, in a clustered mode.
Ansible Playbook - Setup Storm Cluster.
This is a simple Storm Cluster Setup. We are using a dedicated Zookeeper Cluster/Node, instead of the standalone zkserver. Below is how we will deploy our c...
Ansible Playbook - Setup Kafka Cluster.
This is a simple Kafka setup. In this setup we are running kafka over a dedicated zookeeper service. (NOT the standalone zookeeper which comes with kafka)
Ansible Playbook - Setup Hadoop CDH5 Using tarball.
Setting up Hadoop using Ansible, we will be using cdh5 tarball for installation of the cluster.
Creating a Multi-node Cassandra Cluster on Centos 6.5.
The Apache Cassandra database is the right choice when you need scalability and high availability without compromising performance. Linear scalability and pr...
Performance Tuning HBase and Hadoop for Optimal Performance
Getting HBase to perform well in a production environment is an iterative process that involves tweaking various configuration parameters. This post dives de...
Setting up an HBase Cluster on Hadoop YARN - A Detailed Guide for Ubuntu 12.04 LTS
HBase is a powerful open-source, non-relational, distributed database designed for handling massive datasets. Inspired by Google’s BigTable, HBase is written...
Installing KAFKA Single Node - Quick Start Guide
Apache Kafka is often described as a distributed, fault-tolerant, high-throughput publish-subscribe messaging system. Think of it as a central nervous system...
Tuning Hadoop Performance with sysctl.conf
This post delves into optimizing Hadoop performance at the kernel level using sysctl. The sysctl interface provides a way to dynamically modify a running Lin...
How To Configure Swappiness
Swappiness is a crucial Linux kernel parameter that dictates how aggressively the system uses swap space. Understanding and configuring swappiness can signif...
Setting up Local HBase on Top of HDFS.
First lets setup the Hbase Configuration Files. For pseudo-distributed replace with 'localhost'
Securing Your Data: A Step-by-Step Guide to Enabling HBase Authorization
Apache HBase is a powerful NoSQL database, but out-of-the-box, it’s a bit like a house with no locks on the doors. Any user who can connect to the cluster ca...
Zabbix Hadoop Monitoring.
This script can be used to monitor Namenode Parameters. This script can be used to Generate Zabbix Import XML or Send monitoring data to Zabbix server.
hadoop-config
Setting up Local HBase on Top of HDFS.
First lets setup the Hbase Configuration Files. For pseudo-distributed replace with 'localhost'
haproxy
Creating a two-node RHEL cluster with Virtual IP using CMAN and Pacemaker.
CMAN v3 is a Corsync plugin that monitors the names and number of active cluster nodes in order to deliver membership and quorum information to clients (such...
Setting Up HAProxy for High Availability and Load Balancing on CentOS
HAProxy is a robust, high-performance TCP/HTTP load balancer, perfectly suited for ensuring high availability and efficient traffic distribution across your ...
hashicorp
Setting Up HashiCorp Vault for Secret Management
We will walk you through the process of setting up HashiCorp Vault to effectively manage your secrets. This guide is an gettinf started with vault with terra...
hbase
Bulk Upload Hfiles doBulkLoad - LoadIncrementalHfiles failed. This is recoverable. Hbase Encryption Zone.
We were working with a team who were uploading data into HBASE using the doBulkLoad, also we had moved the /hbase to encryption zone. The issue started after...
HDFS Block Corrupted - HBase Error Could not seek StoreFileScanner Invalid HFile block magic
We ran into an issue which was causing HBase to fail. When the job was rerun then it would work again. In the logs we got 2 errors.
Performance Tuning HBase and Hadoop for Optimal Performance
Getting HBase to perform well in a production environment is an iterative process that involves tweaking various configuration parameters. This post dives de...
Setting up an HBase Cluster on Hadoop YARN - A Detailed Guide for Ubuntu 12.04 LTS
HBase is a powerful open-source, non-relational, distributed database designed for handling massive datasets. Inspired by Google’s BigTable, HBase is written...
Setting up Local HBase on Top of HDFS.
First lets setup the Hbase Configuration Files. For pseudo-distributed replace with 'localhost'
Securing Your Data: A Step-by-Step Guide to Enabling HBase Authorization
Apache HBase is a powerful NoSQL database, but out-of-the-box, it’s a bit like a house with no locks on the doors. Any user who can connect to the cluster ca...
hbase-config
Setting up Local HBase on Top of HDFS.
First lets setup the Hbase Configuration Files. For pseudo-distributed replace with 'localhost'
hdfs
getmerge Operation not Permitted
getmerge command takes a source directory and a destination file as input and concatenates files in src into the destination local file.
Checking HDFS health using fsck.
When we have large data sets on the cluster, there will be corruptions of blocks. This could be due to disk or any other.
HDFS Block Corrupted - HBase Error Could not seek StoreFileScanner Invalid HFile block magic
We ran into an issue which was causing HBase to fail. When the job was rerun then it would work again. In the logs we got 2 errors.
Setting Up HDFS Services Using Cloudera API [Part 3]
This is the second follow up post. In the earlier post
Setting up Local HBase on Top of HDFS.
First lets setup the Hbase Configuration Files. For pseudo-distributed replace with 'localhost'
health check
Monitoring Firestore Health - A Python-Based Uptime Check
Ensuring the health and availability of your Firestore database is crucial for maintaining a reliable application. This post provides a Python script that se...
heartbeat
Creating a two-node RHEL cluster with Virtual IP using CMAN and Pacemaker.
CMAN v3 is a Corsync plugin that monitors the names and number of active cluster nodes in order to deliver membership and quorum information to clients (such...
High Availability Nginx with Heartbeat - A Step-by-Step Guide
In today’s digital landscape, ensuring high availability for your web applications is paramount. Downtime can lead to lost revenue, damaged reputation, and ...
Setting up Highly Available Apache (httpd) with Heartbeat on CentOS/RHEL 6
This guide walks you through configuring a High Availability (HA) Apache web server using Heartbeat on CentOS/RHEL 6. HA ensures your web server remains acce...
high-availability
Creating a two-node RHEL cluster with Virtual IP using CMAN and Pacemaker.
CMAN v3 is a Corsync plugin that monitors the names and number of active cluster nodes in order to deliver membership and quorum information to clients (such...
Setting Up HAProxy for High Availability and Load Balancing on CentOS
HAProxy is a robust, high-performance TCP/HTTP load balancer, perfectly suited for ensuring high availability and efficient traffic distribution across your ...
High Availability Nginx with Heartbeat - A Step-by-Step Guide
In today’s digital landscape, ensuring high availability for your web applications is paramount. Downtime can lead to lost revenue, damaged reputation, and ...
SpagoBI Tomcat Clustering with mod_jk and httpd on CentOS - In-Memory Session Replication - A Deep Dive
In a previous post, we explored a basic session-based replication setup for SpagoBI. However, that configuration lacked true session redundancy.
Setting up NGINX for HTTP Load Balancing - A Comprehensive Guide
NGINX is a powerful and versatile open-source software that excels as an HTTP server, reverse proxy, and IMAP/POP3 proxy server. Renowned for its high perfo...
Setting up Highly Available Apache (httpd) with Heartbeat on CentOS/RHEL 6
This guide walks you through configuring a High Availability (HA) Apache web server using Heartbeat on CentOS/RHEL 6. HA ensures your web server remains acce...
hive
Getting started with Hive with Kerberos.
Apache Hive is a powerful data warehousing application built on top of Hadoop; it enables you to access your data using Hive QL, a language that is similar t...
http
YUM Repository Creation on HTTPD Web Server.
Setting up yum repos on RHEL using httpd. We will be setting up httpd and yum repo on top of it. So that we can access yum over http.
Load Testing with Siege on CentOS 6.5 - Kernel 2.6, x86_64 - A Comprehensive Guide
Siege is a powerful, open-source HTTP load testing and benchmarking utility. It’s designed to help web developers and system administrators measure the perfo...
High Availability Nginx with Heartbeat - A Step-by-Step Guide
In today’s digital landscape, ensuring high availability for your web applications is paramount. Downtime can lead to lost revenue, damaged reputation, and ...
SpagoBI Tomcat Clustering with mod_jk and httpd on CentOS - In-Memory Session Replication - A Deep Dive
In a previous post, we explored a basic session-based replication setup for SpagoBI. However, that configuration lacked true session redundancy.
Setting up Tomcat Cluster for SpagoBI 5.1.
This post details how to set up a Tomcat cluster for SpagoBI 5.1, building upon a previous guide for installing SpagoBI with MySQL (post). This configuration...
Setting up NGINX for HTTP Load Balancing - A Comprehensive Guide
NGINX is a powerful and versatile open-source software that excels as an HTTP server, reverse proxy, and IMAP/POP3 proxy server. Renowned for its high perfo...
Setting up Highly Available Apache (httpd) with Heartbeat on CentOS/RHEL 6
This guide walks you through configuring a High Availability (HA) Apache web server using Heartbeat on CentOS/RHEL 6. HA ensures your web server remains acce...
httpd
YUM Repository Creation on HTTPD Web Server.
Setting up yum repos on RHEL using httpd. We will be setting up httpd and yum repo on top of it. So that we can access yum over http.
High Availability Nginx with Heartbeat - A Step-by-Step Guide
In today’s digital landscape, ensuring high availability for your web applications is paramount. Downtime can lead to lost revenue, damaged reputation, and ...
SpagoBI Tomcat Clustering with mod_jk and httpd on CentOS - In-Memory Session Replication - A Deep Dive
In a previous post, we explored a basic session-based replication setup for SpagoBI. However, that configuration lacked true session redundancy.
Setting up NGINX for HTTP Load Balancing - A Comprehensive Guide
NGINX is a powerful and versatile open-source software that excels as an HTTP server, reverse proxy, and IMAP/POP3 proxy server. Renowned for its high perfo...
Setting up Highly Available Apache (httpd) with Heartbeat on CentOS/RHEL 6
This guide walks you through configuring a High Availability (HA) Apache web server using Heartbeat on CentOS/RHEL 6. HA ensures your web server remains acce...
https
Setting up SSL https On Nagios XI Server
HTTPS is a protocol for secure communication over a computer network which is widely used on the Internet. HTTPS consists of communication over Hypertext Tra...
Load Testing with Siege on CentOS 6.5 - Kernel 2.6, x86_64 - A Comprehensive Guide
Siege is a powerful, open-source HTTP load testing and benchmarking utility. It’s designed to help web developers and system administrators measure the perfo...
iac
Testing Your Infrastructure Code - A Guide to Terratest on GCP
Infrastructure as Code (IaC) has revolutionized how we manage and provision infrastructure. Tools like Terraform allow us to define our infrastructure in cod...
iam
Design IAM on Google Cloud Platform
Security is paramount when setting up resources in the cloud or on-premises. It encompasses various layers of protection, including network security, encrypt...
Cloud IAM - GCP Learning Notes.
Google Cloud offers IAM, which lets you give more granular access to specific Google Cloud resources and prevents unwanted access to other resources.
ifcfg-eth0
Configuring Multiple IP Addresses on a Single Network Interface in Linux
This comprehensive guide explains how to assign multiple IP addresses to a single network interface in Linux. This technique, often called IP aliasing or cre...
image-processing
Cropping Bulk Images Using Python.
I was working on getting post headers for my post on this blog. I had couple of images from unsplash. But the header for the post need to be a little more ho...
infrastructure-as-code
5 Common Ansible Mistakes and How to Avoid Them
Ansible is a powerful automation tool that simplifies infrastructure management and application deployment. However, like any technology, it’s easy to make m...
install
Installing python 2.7.x on Centos 6.5/6.6
By default centos comes with python 2.6. In most of the cases we might need python 2.7 or later to be installed. Below are few ways to install python 2.7 on ...
install-java
Setting SUN Java for Ubuntu - A Step-by-Step Guide
The Java Development Kit (JDK) is an essential tool for Java developers, providing the necessary components to compile, debug, and run Java applications. Whi...
installation
Installing NodeJS on CentOS 6.6 - A Comprehensive Guide
Node.js is an open-source, cross-platform JavaScript runtime environment that executes JavaScript code server-side. It’s incredibly popular for building scal...
Installing KAFKA Single Node - Quick Start Guide
Apache Kafka is often described as a distributed, fault-tolerant, high-throughput publish-subscribe messaging system. Think of it as a central nervous system...
integrate
Integrating NodeJS and Kafka - A Practical Guide
This blog post provides a comprehensive, step-by-step guide to integrating NodeJS with Kafka. Whether you’re building real-time data pipelines, implementing ...
interface
Setting iptables - Port forwarding from one interface to another.
iptables is a user-space application program that allows a system administrator to configure the tables provided by the Linux kernel firewall (implemented as...
Interface Forwarding - from eth1 to eth0 on EDGE node.
Adding route to all the slaves which reside on a private network to communicate with External Server directly using an EDGE node using Interface Forwarding.
iptables
Setting iptables - Port forwarding from one interface to another.
iptables is a user-space application program that allows a system administrator to configure the tables provided by the Linux kernel firewall (implemented as...
ireasoning
Fetch Monitoring Data from Item to CSV file.
This section describes how to get monitoring data from Zabbix server through the command line and export these to a CSV formatted file.
Zabbix Item Creation using OID and Name CSV files.
This script is to create a xml import file from a csv file. We use 2 csv file for this.
Zabbix Import File from iReasoning Export CSV
Module used to generate Zabbix import xml for snmp traps, Below are steps to create a Zabbix Import for Traps. These traps are captured from the snmptrap fun...
java
Performance Tuning HBase and Hadoop for Optimal Performance
Getting HBase to perform well in a production environment is an iterative process that involves tweaking various configuration parameters. This post dives de...
Fixing Unable to locate package oracle-java7-installer on Ubuntu
I recently ran into a frustrating issue while trying to install Java 7 on an older Ubuntu system (Ubuntu 13, in this case). The seemingly simple command sud...
Installing zabbix-java-gateway on Centos 6.5
Zabbix 2.0 introduced native support for monitoring JMX applications through the Zabbix Java gateway. This Java-based daemon acts as an intermediary, allowin...
Setting SUN Java for Ubuntu - A Step-by-Step Guide
The Java Development Kit (JDK) is an essential tool for Java developers, providing the necessary components to compile, debug, and run Java applications. Whi...
jdk
Fixing Unable to locate package oracle-java7-installer on Ubuntu
I recently ran into a frustrating issue while trying to install Java 7 on an older Ubuntu system (Ubuntu 13, in this case). The seemingly simple command sud...
Setting SUN Java for Ubuntu - A Step-by-Step Guide
The Java Development Kit (JDK) is an essential tool for Java developers, providing the necessary components to compile, debug, and run Java applications. Whi...
jekyll
Installing Ruby2.4 on Ubuntu Docker Container.
Was setting up jekyll using docker, but was not able to install ruby on the docker machine as apt-add-repository was not working. Below is my Dockerfile when...
json
Routing JSON Payloads to Multiple Kafka Topics with Node.js and Kafka-node
This guide demonstrates how to build a Node.js application that receives JSON data via HTTP and routes it to different Apache Kafka topics based on the URL p...
jvm
Performance Tuning HBase and Hadoop for Optimal Performance
Getting HBase to perform well in a production environment is an iterative process that involves tweaking various configuration parameters. This post dives de...
kafka
Kafka Kerberos Enable and Testing.
Apache Kafka is a distributed streaming platform. Kafka 2.0 supports Kerberos authentication, Enabling Kerberos Authentication Using the Wizard on cloudera m...
Cloudera Manager - Duplicate entry ‘zookeeper’ for key ‘NAME’.
Deployment error during setting zookeeper on a new cluster using Cloudera Manager UI, when an earlier cluster containing zookeeper earlier was built using Cl...
Ansible Playbook - Setup Kafka Cluster.
This is a simple Kafka setup. In this setup we are running kafka over a dedicated zookeeper service. (NOT the standalone zookeeper which comes with kafka)
Integrating NodeJS and Kafka - A Practical Guide
This blog post provides a comprehensive, step-by-step guide to integrating NodeJS with Kafka. Whether you’re building real-time data pipelines, implementing ...
Routing JSON Payloads to Multiple Kafka Topics with Node.js and Kafka-node
This guide demonstrates how to build a Node.js application that receives JSON data via HTTP and routes it to different Apache Kafka topics based on the URL p...
NodeJS Kafka Producer - A Deep Dive with kafka-node
Ready to integrate your NodeJS application with Kafka? This post provides a comprehensive guide to setting up a Kafka producer using the kafka-node library. ...
Sending JSON from Browser/cURL to NodeJS to Kafka
This post outlines how to send JSON data from a client (browser or curl) through a NodeJS server and ultimately into a Kafka topic. This architecture is usef...
Installing KAFKA Single Node - Quick Start Guide
Apache Kafka is often described as a distributed, fault-tolerant, high-throughput publish-subscribe messaging system. Think of it as a central nervous system...
kafka-node
Integrating NodeJS and Kafka - A Practical Guide
This blog post provides a comprehensive, step-by-step guide to integrating NodeJS with Kafka. Whether you’re building real-time data pipelines, implementing ...
Routing JSON Payloads to Multiple Kafka Topics with Node.js and Kafka-node
This guide demonstrates how to build a Node.js application that receives JSON data via HTTP and routes it to different Apache Kafka topics based on the URL p...
NodeJS Kafka Producer - A Deep Dive with kafka-node
Ready to integrate your NodeJS application with Kafka? This post provides a comprehensive guide to setting up a Kafka producer using the kafka-node library. ...
Sending JSON from Browser/cURL to NodeJS to Kafka
This post outlines how to send JSON data from a client (browser or curl) through a NodeJS server and ultimately into a Kafka topic. This architecture is usef...
kerberos
Setting Up Kerberos on a Dataproc Cluster A Step-by-Step Guide
Kerberos authentication in Google Dataproc clusters offers enhanced security and user isolation. It’s a crucial step for multi-tenancy, encryption, and user ...
Kafka Kerberos Enable and Testing.
Apache Kafka is a distributed streaming platform. Kafka 2.0 supports Kerberos authentication, Enabling Kerberos Authentication Using the Wizard on cloudera m...
Enable Kerberos Using Cloudera API.
Python API for cloudera is really nice, apart from getting the cluster setup, we can also do configuration and automation. We use a lot of automation using C...
Getting started with Hive with Kerberos.
Apache Hive is a powerful data warehousing application built on top of Hadoop; it enables you to access your data using Hive QL, a language that is similar t...
No valid credentials provided Mechanism level Failed to find any Kerberos tgt
We have 2 domains forests in our environment, ABC and XYZ. We were not able to authenticate normal users from either of the domains. Most of the information ...
kernel
Optimizing sysctl.conf for High-Performance Web Servers - Node.js, Nginx
This post provides a detailed sysctl.conf file, optimized for Linux systems running high-performance web servers like Nginx and Node.js applications. The sy...
Fine-Tuning Linux Kernel Parameters with sysctl.conf for Web and FTP Servers
The sysctl interface provides a powerful way to modify the behavior of a running Linux kernel. By editing the /etc/sysctl.conf file, you can persistently co...
Tuning Hadoop Performance with sysctl.conf
This post delves into optimizing Hadoop performance at the kernel level using sysctl. The sysctl interface provides a way to dynamically modify a running Lin...
kitchen
Chef Kitchen Test Deployment on Multiple Nodes with Private Interface.
What is Kitchen? Kitchen provides a test harness to execute infrastructure code on one or more platforms in isolation.
Windows Testing Using Kitchen Chef
Kitchen-Vagrant has the capability to spin up a windows instance for testing. To make it work you will need the vagrant-winrm to be installted on the worksta...
Encrypted Data Bags - Chef
Data Bags are a way to store information on the chef-server which all the cookbooks can access. Few more additional advantages are that we can encrypt the da...
krb5
No valid credentials provided Mechanism level Failed to find any Kerberos tgt
We have 2 domains forests in our environment, ABC and XYZ. We were not able to authenticate normal users from either of the domains. Most of the information ...
kvm
Comprehensive Guide to KVM Installation and Configuration on CentOS 6.x
Kernel-based Virtual Machine (KVM) has become a cornerstone of Linux server virtualization, known for its maturity and performance. Red Hat’s decision to fav...
ldap
Getting started with Hive with Kerberos.
Apache Hive is a powerful data warehousing application built on top of Hadoop; it enables you to access your data using Hive QL, a language that is similar t...
Redhat Intergration with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
learning
Setting up Postgres using Dockerfile.
In this blog post we will be setting up a postgres database on docker using Dockerfile.
Python - Virtual Environment.
Python virtual environment creates a isoloated workspace of python work. This helps in creating project specific virtual environment without worrying about ...
Setting up Postgres on Docker.
In this blog post we will be setting up a postgres database on docker.
Python - List Comprehensions.
List Comprehensions provides easy and functional way to create list in python. We could make a single line of code which otherwise would take a few lines. l...
Python Getting Started - Learning Notes.
Python is a clear and powerful object-oriented programming language, comparable to Perl, Ruby, Scheme, or Java. This is basic documentation for getting star...
libvirt
Comprehensive Guide to KVM Installation and Configuration on CentOS 6.x
Kernel-based Virtual Machine (KVM) has become a cornerstone of Linux server virtualization, known for its maturity and performance. Red Hat’s decision to fav...
linux
Cloudera Manager - Duplicate entry ‘zookeeper’ for key ‘NAME’.
Deployment error during setting zookeeper on a new cluster using Cloudera Manager UI, when an earlier cluster containing zookeeper earlier was built using Cl...
Parcel Not Distributing Cloudera CDH.
We were deploying one of the cluster on our lab environment which is used by everyone. So the lab has it own share of stale information on it.
Creating /etc/hosts file in Chef.
We had a cluster environment which we needed to update the /etc/hosts file. Which would help communicate between the server over a private network. Our serve...
Enable Kerberos Using Cloudera API.
Python API for cloudera is really nice, apart from getting the cluster setup, we can also do configuration and automation. We use a lot of automation using C...
Setting Up HDFS Services Using Cloudera API [Part 3]
This is the second follow up post. In the earlier post
Setting Up Zookeeper Services Using Cloudera API [Part 2]
This is the second follow up post. In the earlier post Setting Up Cloudera Manager Services Using Cloudera API [Part 1] we install the cloudera management se...
Setting Up Cloudera Manager Services Using Cloudera API [Part 1]
Cloudera API is a very convenient way to setup a cluster and do more.
Getting Started with Cloudera API
This is a basic steps to get connected with cloudera manager.
Basic Testing On Hadoop Environment [Cloudera]
These are a set of testing which we can do on a Hadoop environment. These are basic testing to make sure the environment is setup correctly.
Setting Hue to Listen on 0.0.0.0 [Cloudera]
We were working on setting up a cluster, but the Hue URL was set to a private IP of the server. As we had setup all the nodes to access each other using a pr...
Nagios - Service Group Summary ERROR
We were working on nagios and found that after our migration, service group summary was not working. You might get below error on the screen and the solution...
Zabbix History Table Clean Up
Zabbix history table gets really big, and if you are in a situation where you want to clean it up. Then we can do so, using the below steps.
Package Installer for Cygwin [apt-cyg].
After a longtime I was on my windows machine and had to make it feel more like my linux machine. So install the thing what everyone else does cygwin. Surpise...
Issues - Monitoring MongoDB using Nagios XI.
Monitoring for mongodb using nagiosxi is straight forword but you might have some issues when we are setting up. Here are few issues which might come up usin...
Installing CouchDB on Ubuntu 14 LTS.
CouchDB is a database that completely embraces the web. Store your data with JSON documents. Access your documents and query your indexes with your web brows...
Installing MongoDB on Ubuntu 14 LTS.
MongoDB is an open-source document database, and leading NoSQL database. MongoDB is written in c++. Below is a brief document about installing a mongodb on a...
Encrypted Data Bags - Chef
Data Bags are a way to store information on the chef-server which all the cookbooks can access. Few more additional advantages are that we can encrypt the da...
Remove Old Files using find Command
GNU find searches the directory tree rooted at each given file name by evaluating the given expression from left to right, according to the rules of preceden...
Moving RRD file from 32bit to 64bit Architecture
When we were working on a nagios monitoring system we were migrating from a 32bit nagios to a 64bit Architecture. Most of the graphs are not working as the R...
Setting up SSL https On Nagios XI Server
HTTPS is a protocol for secure communication over a computer network which is widely used on the Internet. HTTPS consists of communication over Hypertext Tra...
Standalone Chef Server / Workstation / Client Setup on CentOS 6
The standalone installation of Chef server creates a working installation on a single server. This installation is also useful when you are installing Chef s...
Creating Documents Using pandoc
Pandoc is an opensource utility to create documents from markdown. We can create PDF, Doc, doc, html and other formats. And can be also used to convert html ...
Check Port on Remote Server CentOS 6.6/RHEL 6
Checking port available on a remote machine using nc command instead of telnet. Same command can be used to check on a remote server as well, change the 127....
LUKS Disk encryption for CentOS 6.6/RHEL 6
Linux Unified Key Setup-on-disk-format (or LUKS) allows you to encrypt partitions on your Linux computer. This is particularly important when it comes to mo...
Mysql Database Moving Data Directory to New Location.
How to move an existing data directory in mysql to a new location. We were running out of space and had to move the existing data directory to a new drive. B...
Mysql Database Disk Usage.
We were running out of disk space on one of the databases server, we need to get information on what the current table/database usage was. Below are few comm...
Installing python 2.7.x on Centos 6.5/6.6
By default centos comes with python 2.6. In most of the cases we might need python 2.7 or later to be installed. Below are few ways to install python 2.7 on ...
Zabbix Database Backup Script for postgres and mysql.
After the zabbix setup is done, we would like to take backup of the database. Here we are doing a complete backup. But if we are looking for only configurati...
Installing Zabbix Version 2.4 Offline (Zabbix Server without Internet).
There might be situations where you have a remote/zabbix server which does not have internet connectivity, due to security or other reasons. So we create a ...
Zabbix Template Creation using CSV file.
In zabbix we dont have a better way to capture snmptraps. we have to manually create an item and corresponfding trigger to handle a trap arriving from the de...
Installing SNMP Builder using zabbix-extra on Zabbix Version 2.4.
SNMP Builder/Extra is an add-in for Zabbix. It provides new web interface components to browse MIB trees and values. SNMP OIDs can then be automatically conv...
Setting up SNMP Trapper for Zabbix.
Receiving SNMP traps is the opposite to querying SNMP-enabled devices. In this case the information is sent from a SNMP-enabled device and is collected or “t...
Explain - System Security Services Daemon (SSSD) Config.
The System Security Services Daemon (SSSD) provides a set of daemons to manage access to remote directories and authentication mechanisms. It provides Name S...
YUM Repository Creation on HTTPD Web Server.
Setting up yum repos on RHEL using httpd. We will be setting up httpd and yum repo on top of it. So that we can access yum over http.
Access Filter Setup with SSSD
If using access_provider = ldap, this option is mandatory. It specifies an LDAP search filter criteria that must be met for the user to be granted access on ...
Update Cloudera Manager to specific version [5.4.5]
Upgrading Cloudera Manager 5 to the Latest Cloudera Manager, In most cases it is possible to complete the following upgrade without shutting down most CDH se...
Getting started with Hive with Kerberos.
Apache Hive is a powerful data warehousing application built on top of Hadoop; it enables you to access your data using Hive QL, a language that is similar t...
Redhat Intergration with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
Simple Steps to Intergrate RHEL with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
Setting iptables - Port forwarding from one interface to another.
iptables is a user-space application program that allows a system administrator to configure the tables provided by the Linux kernel firewall (implemented as...
Setting up Pentaho Data Integration 5.4.1 with Hadoop Cluster (Clouder Manager)
Pentaho Data Integration (PDI) is a suite of open source Business Intelligence (BI) products which provide data integration, OLAP services, reporting, dashb...
Installing squid as a sibling to an already existing Parent squid.
Squid is a caching and forwarding web proxy. It has a wide variety of uses, from speeding up a web server by caching repeated requests; to caching web, DNS a...
Creating a two-node RHEL cluster with Virtual IP using CMAN and Pacemaker.
CMAN v3 is a Corsync plugin that monitors the names and number of active cluster nodes in order to deliver membership and quorum information to clients (such...
Interface Forwarding - from eth1 to eth0 on EDGE node.
Adding route to all the slaves which reside on a private network to communicate with External Server directly using an EDGE node using Interface Forwarding.
Installing ansible on RHEL 6.6.
Ansible is a radically simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchest...
Mounting RAID10 using parted.
GUID Partition Table (GPT) is a standard for the layout of the partition table on a physical hard disk, using globally unique identifiers (GUID). Although it...
Setup and Configure NFS Mounts on Linux Server
A Network File System (NFS) allows remote hosts to mount file systems over a network and interact with those file systems as though they are mounted locally....
Ansible Playbook - Setup Zookeeper Using tarball.
This is a simple zookeeper playbook, to quickly start zookeeper running on a single or more nodes, in a clustered mode.
Ansible Playbook - Setup Storm Cluster.
This is a simple Storm Cluster Setup. We are using a dedicated Zookeeper Cluster/Node, instead of the standalone zkserver. Below is how we will deploy our c...
Ansible Playbook - Setup Kafka Cluster.
This is a simple Kafka setup. In this setup we are running kafka over a dedicated zookeeper service. (NOT the standalone zookeeper which comes with kafka)
Ansible Playbook - Setup Hadoop CDH5 Using tarball.
Setting up Hadoop using Ansible, we will be using cdh5 tarball for installation of the cluster.
Streaming Data Processing - Storm Vs Spark.
Apache Spark is an in-memory distributed data analysis platform– primarily targeted at speeding up batch analysis jobs, iterative machine learning jobs, inte...
Creating a passwordless entry on Centos 6.5.
You can enter a linux system without entring a password using below steps. We will be creating a ssh jey which will be share between the servers, which will ...
Creating a Multi-node Cassandra Cluster on Centos 6.5.
The Apache Cassandra database is the right choice when you need scalability and high availability without compromising performance. Linear scalability and pr...
Installing SpagoBI 5.1 on Centos 6.5 -Tomcat 7 with MySQL 5.6.
The SpagoBI project is a free software/open source initiative by the SpagoBI Labs of Engineering Group. It aims to realize the most complete 100% open source...
Comprehensive Guide to KVM Installation and Configuration on CentOS 6.x
Kernel-based Virtual Machine (KVM) has become a cornerstone of Linux server virtualization, known for its maturity and performance. Red Hat’s decision to fav...
Performance Tuning HBase and Hadoop for Optimal Performance
Getting HBase to perform well in a production environment is an iterative process that involves tweaking various configuration parameters. This post dives de...
Setting up an HBase Cluster on Hadoop YARN - A Detailed Guide for Ubuntu 12.04 LTS
HBase is a powerful open-source, non-relational, distributed database designed for handling massive datasets. Inspired by Google’s BigTable, HBase is written...
Fixing Unable to locate package oracle-java7-installer on Ubuntu
I recently ran into a frustrating issue while trying to install Java 7 on an older Ubuntu system (Ubuntu 13, in this case). The seemingly simple command sud...
Installing zabbix-java-gateway on Centos 6.5
Zabbix 2.0 introduced native support for monitoring JMX applications through the Zabbix Java gateway. This Java-based daemon acts as an intermediary, allowin...
Setting Up HAProxy for High Availability and Load Balancing on CentOS
HAProxy is a robust, high-performance TCP/HTTP load balancer, perfectly suited for ensuring high availability and efficient traffic distribution across your ...
Navigating npm Behind a Corporate Proxy - A Comprehensive Guide
npm, short for Node Package Manager, is the cornerstone of JavaScript development within the Node.js ecosystem. It simplifies the process of managing projec...
Integrating NodeJS and Kafka - A Practical Guide
This blog post provides a comprehensive, step-by-step guide to integrating NodeJS with Kafka. Whether you’re building real-time data pipelines, implementing ...
Routing JSON Payloads to Multiple Kafka Topics with Node.js and Kafka-node
This guide demonstrates how to build a Node.js application that receives JSON data via HTTP and routes it to different Apache Kafka topics based on the URL p...
NodeJS Kafka Producer - A Deep Dive with kafka-node
Ready to integrate your NodeJS application with Kafka? This post provides a comprehensive guide to setting up a Kafka producer using the kafka-node library. ...
Sending JSON from Browser/cURL to NodeJS to Kafka
This post outlines how to send JSON data from a client (browser or curl) through a NodeJS server and ultimately into a Kafka topic. This architecture is usef...
Installing NodeJS on CentOS 6.6 - A Comprehensive Guide
Node.js is an open-source, cross-platform JavaScript runtime environment that executes JavaScript code server-side. It’s incredibly popular for building scal...
Installing KAFKA Single Node - Quick Start Guide
Apache Kafka is often described as a distributed, fault-tolerant, high-throughput publish-subscribe messaging system. Think of it as a central nervous system...
Fixing sudo effective uid is not 0 Error on Linux
Encountered a frustrating issue on a Linux server while trying to use sudo? Specifically, the error message: sudo: effective uid is not 0, is sudo installed ...
Optimizing sysctl.conf for High-Performance Web Servers - Node.js, Nginx
This post provides a detailed sysctl.conf file, optimized for Linux systems running high-performance web servers like Nginx and Node.js applications. The sy...
Fine-Tuning Linux Kernel Parameters with sysctl.conf for Web and FTP Servers
The sysctl interface provides a powerful way to modify the behavior of a running Linux kernel. By editing the /etc/sysctl.conf file, you can persistently co...
Configuring Multiple IP Addresses on a Single Network Interface in Linux
This comprehensive guide explains how to assign multiple IP addresses to a single network interface in Linux. This technique, often called IP aliasing or cre...
Changing Timezone in Ubuntu Server - A Comprehensive Guide
Changing the timezone on your Ubuntu server is a crucial task for ensuring accurate timekeeping, especially when dealing with scheduled tasks, logs, and appl...
Tuning Hadoop Performance with sysctl.conf
This post delves into optimizing Hadoop performance at the kernel level using sysctl. The sysctl interface provides a way to dynamically modify a running Lin...
Setting SUN Java for Ubuntu - A Step-by-Step Guide
The Java Development Kit (JDK) is an essential tool for Java developers, providing the necessary components to compile, debug, and run Java applications. Whi...
Simple Script to Create Users on CentOS/RHEL 6.5
This script provides a simple way to create users on CentOS/RHEL 6.5 systems from the command line. It allows you to create both normal and sudo (administrat...
Performance Tuning for nginx
Nginx (pronounced “engine x”) is a powerful and versatile web server renowned for its high concurrency, exceptional performance, and efficient memory utiliza...
How To Configure Swappiness
Swappiness is a crucial Linux kernel parameter that dictates how aggressively the system uses swap space. Understanding and configuring swappiness can signif...
High Availability Nginx with Heartbeat - A Step-by-Step Guide
In today’s digital landscape, ensuring high availability for your web applications is paramount. Downtime can lead to lost revenue, damaged reputation, and ...
SpagoBI Tomcat Clustering with mod_jk and httpd on CentOS - In-Memory Session Replication - A Deep Dive
In a previous post, we explored a basic session-based replication setup for SpagoBI. However, that configuration lacked true session redundancy.
Setting up Tomcat Cluster for SpagoBI 5.1.
This post details how to set up a Tomcat cluster for SpagoBI 5.1, building upon a previous guide for installing SpagoBI with MySQL (post). This configuration...
Setting up NGINX for HTTP Load Balancing - A Comprehensive Guide
NGINX is a powerful and versatile open-source software that excels as an HTTP server, reverse proxy, and IMAP/POP3 proxy server. Renowned for its high perfo...
Setting up Highly Available Apache (httpd) with Heartbeat on CentOS/RHEL 6
This guide walks you through configuring a High Availability (HA) Apache web server using Heartbeat on CentOS/RHEL 6. HA ensures your web server remains acce...
Zabbix Hadoop Monitoring.
This script can be used to monitor Namenode Parameters. This script can be used to Generate Zabbix Import XML or Send monitoring data to Zabbix server.
Fetch Monitoring Data from Item to CSV file.
This section describes how to get monitoring data from Zabbix server through the command line and export these to a CSV formatted file.
Zabbix Item Creation using OID and Name CSV files.
This script is to create a xml import file from a csv file. We use 2 csv file for this.
Send KPI Data to Zabbix from XML KPI File
Send Zabbix metric by reading from a XML file. This needs package zbxsend from : https://github.com/ahmedzbyr/zbxsend
Zabbix Import File from iReasoning Export CSV
Module used to generate Zabbix import xml for snmp traps, Below are steps to create a Zabbix Import for Traps. These traps are captured from the snmptrap fun...
SFTP Data Collector
Easy way to collect files recursively over a sftp server is to connect to the server over scp and do scp -r. Problem was that the device we were connecting ...
load-balancing
Setting Up HAProxy for High Availability and Load Balancing on CentOS
HAProxy is a robust, high-performance TCP/HTTP load balancer, perfectly suited for ensuring high availability and efficient traffic distribution across your ...
Performance Tuning for nginx
Nginx (pronounced “engine x”) is a powerful and versatile web server renowned for its high concurrency, exceptional performance, and efficient memory utiliza...
High Availability Nginx with Heartbeat - A Step-by-Step Guide
In today’s digital landscape, ensuring high availability for your web applications is paramount. Downtime can lead to lost revenue, damaged reputation, and ...
SpagoBI Tomcat Clustering with mod_jk and httpd on CentOS - In-Memory Session Replication - A Deep Dive
In a previous post, we explored a basic session-based replication setup for SpagoBI. However, that configuration lacked true session redundancy.
Setting up NGINX for HTTP Load Balancing - A Comprehensive Guide
NGINX is a powerful and versatile open-source software that excels as an HTTP server, reverse proxy, and IMAP/POP3 proxy server. Renowned for its high perfo...
load-testing
Installing and Initial Setup of Tsung Load Testing on CentOS
Tsung (formerly known as idx-Tsunami) is a powerful, open-source, multi-protocol distributed load testing tool. Written in Erlang, it’s licensed under the GP...
Load Testing with Siege on CentOS 6.5 - Kernel 2.6, x86_64 - A Comprehensive Guide
Siege is a powerful, open-source HTTP load testing and benchmarking utility. It’s designed to help web developers and system administrators measure the perfo...
logging
Log-Based Metrics - Unlocking Insights from Your Application Logs
Application logs are a treasure trove of information. Beyond debugging and troubleshooting, they can provide valuable insights into your application’s perfor...
luks
LUKS Disk encryption for CentOS 6.6/RHEL 6
Linux Unified Key Setup-on-disk-format (or LUKS) allows you to encrypt partitions on your Linux computer. This is particularly important when it comes to mo...
manager
Update Cloudera Manager to specific version [5.4.5]
Upgrading Cloudera Manager 5 to the Latest Cloudera Manager, In most cases it is possible to complete the following upgrade without shutting down most CDH se...
Setting up Pentaho Data Integration 5.4.1 with Hadoop Cluster (Clouder Manager)
Pentaho Data Integration (PDI) is a suite of open source Business Intelligence (BI) products which provide data integration, OLAP services, reporting, dashb...
memory
Fixing Not Enough Physical Memory Error in VMware Workstation 10 on Windows 8.1
Are you encountering the frustrating “Not enough physical memory is available to power this virtual machine” error in VMware Workstation 10 after a Windows u...
message queue
NodeJS Kafka Producer - A Deep Dive with kafka-node
Ready to integrate your NodeJS application with Kafka? This post provides a comprehensive guide to setting up a Kafka producer using the kafka-node library. ...
messaging
Installing KAFKA Single Node - Quick Start Guide
Apache Kafka is often described as a distributed, fault-tolerant, high-throughput publish-subscribe messaging system. Think of it as a central nervous system...
metrics
Log-Based Metrics - Unlocking Insights from Your Application Logs
Application logs are a treasure trove of information. Beyond debugging and troubleshooting, they can provide valuable insights into your application’s perfor...
mib
Zabbix Hadoop Monitoring.
This script can be used to monitor Namenode Parameters. This script can be used to Generate Zabbix Import XML or Send monitoring data to Zabbix server.
Fetch Monitoring Data from Item to CSV file.
This section describes how to get monitoring data from Zabbix server through the command line and export these to a CSV formatted file.
Zabbix Item Creation using OID and Name CSV files.
This script is to create a xml import file from a csv file. We use 2 csv file for this.
Send KPI Data to Zabbix from XML KPI File
Send Zabbix metric by reading from a XML file. This needs package zbxsend from : https://github.com/ahmedzbyr/zbxsend
Zabbix Import File from iReasoning Export CSV
Module used to generate Zabbix import xml for snmp traps, Below are steps to create a Zabbix Import for Traps. These traps are captured from the snmptrap fun...
mod_jk
SpagoBI Tomcat Clustering with mod_jk and httpd on CentOS - In-Memory Session Replication - A Deep Dive
In a previous post, we explored a basic session-based replication setup for SpagoBI. However, that configuration lacked true session redundancy.
module_utils
Creating an Ansible Collection - A Detailed Step-by-Step Guide
Creating an Ansible Collection: A Comprehensive Guide
modules
Creating an Ansible Collection - A Detailed Step-by-Step Guide
Creating an Ansible Collection: A Comprehensive Guide
mongo
Issues - Monitoring MongoDB using Nagios XI.
Monitoring for mongodb using nagiosxi is straight forword but you might have some issues when we are setting up. Here are few issues which might come up usin...
Installing MongoDB on Ubuntu 14 LTS.
MongoDB is an open-source document database, and leading NoSQL database. MongoDB is written in c++. Below is a brief document about installing a mongodb on a...
mongodb
MongoDB to Neo4j Using neo4j_doc_manager.
We had a requirement where we wanted to have all the data which is in mongodb to be replicated on neo4j to show few graphs. Here is quick way to demonstrate ...
Issues - Monitoring MongoDB using Nagios XI.
Monitoring for mongodb using nagiosxi is straight forword but you might have some issues when we are setting up. Here are few issues which might come up usin...
Installing MongoDB on Ubuntu 14 LTS.
MongoDB is an open-source document database, and leading NoSQL database. MongoDB is written in c++. Below is a brief document about installing a mongodb on a...
monitoring
Automating Firestore Health Checks with Terraform Cloud Functions Cloud Monitoring.
This blog post demonstrates how to automate Firestore health checks using Terraform to deploy a Cloud Function (Gen 1) and set up a complete monitoring and a...
Monitoring Firestore Health - A Python-Based Uptime Check
Ensuring the health and availability of your Firestore database is crucial for maintaining a reliable application. This post provides a Python script that se...
Log-Based Metrics - Unlocking Insights from Your Application Logs
Application logs are a treasure trove of information. Beyond debugging and troubleshooting, they can provide valuable insights into your application’s perfor...
Cloud VPN - GCP Learning Notes.
Cloud VPN securely connects your peer network to your Virtual Private Cloud (VPC) network through an IPsec VPN connection.
Cloud Monitoring - Stackdriver - GCP Learning Notes.
Stackdriver driver provides monitoring, logging, error reporting, tracing and debugging services. Currently known as Google Cloud’s operations suite.
Nagios - Service Group Summary ERROR
We were working on nagios and found that after our migration, service group summary was not working. You might get below error on the screen and the solution...
Zabbix History Table Clean Up
Zabbix history table gets really big, and if you are in a situation where you want to clean it up. Then we can do so, using the below steps.
knife Quick Reference
This is a quick reference for few of the commands which I use often.
Migrating Nagios XI to a New Server on Centos6.6
Migrating an old Nagios backup to a new nagiosxi server. Migration is simple in Nagios XI, you a backup and restore it. Once that is done we need to update/a...
Setting up SSL https On Nagios XI Server
HTTPS is a protocol for secure communication over a computer network which is widely used on the Internet. HTTPS consists of communication over Hypertext Tra...
Setup/Configuration Nagios XI on Centos6.6
Nagios monitors your entire IT infrastructure to ensure systems, applications, services, and business processes are functioning properly. In the event of a f...
Zabbix Database Backup Script for postgres and mysql.
After the zabbix setup is done, we would like to take backup of the database. Here we are doing a complete backup. But if we are looking for only configurati...
Installing Zabbix Version 2.4 Offline (Zabbix Server without Internet).
There might be situations where you have a remote/zabbix server which does not have internet connectivity, due to security or other reasons. So we create a ...
Zabbix Template Creation using CSV file.
In zabbix we dont have a better way to capture snmptraps. we have to manually create an item and corresponfding trigger to handle a trap arriving from the de...
Installing SNMP Builder using zabbix-extra on Zabbix Version 2.4.
SNMP Builder/Extra is an add-in for Zabbix. It provides new web interface components to browse MIB trees and values. SNMP OIDs can then be automatically conv...
Installing zabbix-java-gateway on Centos 6.5
Zabbix 2.0 introduced native support for monitoring JMX applications through the Zabbix Java gateway. This Java-based daemon acts as an intermediary, allowin...
Zabbix Installation 2.4 - CentOS 6.5
Zabbix is the ultimate enterprise-level software designed for monitoring availability and performance of IT infrastructure components. Zabbix is open source ...
Zabbix Hadoop Monitoring.
This script can be used to monitor Namenode Parameters. This script can be used to Generate Zabbix Import XML or Send monitoring data to Zabbix server.
Fetch Monitoring Data from Item to CSV file.
This section describes how to get monitoring data from Zabbix server through the command line and export these to a CSV formatted file.
Zabbix Item Creation using OID and Name CSV files.
This script is to create a xml import file from a csv file. We use 2 csv file for this.
Send KPI Data to Zabbix from XML KPI File
Send Zabbix metric by reading from a XML file. This needs package zbxsend from : https://github.com/ahmedzbyr/zbxsend
Zabbix Import File from iReasoning Export CSV
Module used to generate Zabbix import xml for snmp traps, Below are steps to create a Zabbix Import for Traps. These traps are captured from the snmptrap fun...
monitoring-tools
mount
Mounting RAID10 using parted.
GUID Partition Table (GPT) is a standard for the layout of the partition table on a physical hard disk, using globally unique identifiers (GUID). Although it...
Setup and Configure NFS Mounts on Linux Server
A Network File System (NFS) allows remote hosts to mount file systems over a network and interact with those file systems as though they are mounted locally....
multiple IPs
Configuring Multiple IP Addresses on a Single Network Interface in Linux
This comprehensive guide explains how to assign multiple IP addresses to a single network interface in Linux. This technique, often called IP aliasing or cre...
mysql
Mysql Database Moving Data Directory to New Location.
How to move an existing data directory in mysql to a new location. We were running out of space and had to move the existing data directory to a new drive. B...
Mysql Database Disk Usage.
We were running out of disk space on one of the databases server, we need to get information on what the current table/database usage was. Below are few comm...
Zabbix Database Backup Script for postgres and mysql.
After the zabbix setup is done, we would like to take backup of the database. Here we are doing a complete backup. But if we are looking for only configurati...
Installing SpagoBI 5.1 on Centos 6.5 -Tomcat 7 with MySQL 5.6.
The SpagoBI project is a free software/open source initiative by the SpagoBI Labs of Engineering Group. It aims to realize the most complete 100% open source...
nagios
Nagios - Service Group Summary ERROR
We were working on nagios and found that after our migration, service group summary was not working. You might get below error on the screen and the solution...
Zabbix History Table Clean Up
Zabbix history table gets really big, and if you are in a situation where you want to clean it up. Then we can do so, using the below steps.
Issues - Monitoring MongoDB using Nagios XI.
Monitoring for mongodb using nagiosxi is straight forword but you might have some issues when we are setting up. Here are few issues which might come up usin...
Installing CouchDB on Ubuntu 14 LTS.
CouchDB is a database that completely embraces the web. Store your data with JSON documents. Access your documents and query your indexes with your web brows...
Installing MongoDB on Ubuntu 14 LTS.
MongoDB is an open-source document database, and leading NoSQL database. MongoDB is written in c++. Below is a brief document about installing a mongodb on a...
knife Quick Reference
This is a quick reference for few of the commands which I use often.
Moving RRD file from 32bit to 64bit Architecture
When we were working on a nagios monitoring system we were migrating from a 32bit nagios to a 64bit Architecture. Most of the graphs are not working as the R...
Migrating Nagios XI to a New Server on Centos6.6
Migrating an old Nagios backup to a new nagiosxi server. Migration is simple in Nagios XI, you a backup and restore it. Once that is done we need to update/a...
Setting up SSL https On Nagios XI Server
HTTPS is a protocol for secure communication over a computer network which is widely used on the Internet. HTTPS consists of communication over Hypertext Tra...
Setup/Configuration Nagios XI on Centos6.6
Nagios monitors your entire IT infrastructure to ensure systems, applications, services, and business processes are functioning properly. In the event of a f...
Zabbix Database Backup Script for postgres and mysql.
After the zabbix setup is done, we would like to take backup of the database. Here we are doing a complete backup. But if we are looking for only configurati...
Installing Zabbix Version 2.4 Offline (Zabbix Server without Internet).
There might be situations where you have a remote/zabbix server which does not have internet connectivity, due to security or other reasons. So we create a ...
Zabbix Template Creation using CSV file.
In zabbix we dont have a better way to capture snmptraps. we have to manually create an item and corresponfding trigger to handle a trap arriving from the de...
Installing SNMP Builder using zabbix-extra on Zabbix Version 2.4.
SNMP Builder/Extra is an add-in for Zabbix. It provides new web interface components to browse MIB trees and values. SNMP OIDs can then be automatically conv...
Installing zabbix-java-gateway on Centos 6.5
Zabbix 2.0 introduced native support for monitoring JMX applications through the Zabbix Java gateway. This Java-based daemon acts as an intermediary, allowin...
Zabbix Installation 2.4 - CentOS 6.5
Zabbix is the ultimate enterprise-level software designed for monitoring availability and performance of IT infrastructure components. Zabbix is open source ...
Zabbix Hadoop Monitoring.
This script can be used to monitor Namenode Parameters. This script can be used to Generate Zabbix Import XML or Send monitoring data to Zabbix server.
Fetch Monitoring Data from Item to CSV file.
This section describes how to get monitoring data from Zabbix server through the command line and export these to a CSV formatted file.
Zabbix Item Creation using OID and Name CSV files.
This script is to create a xml import file from a csv file. We use 2 csv file for this.
Send KPI Data to Zabbix from XML KPI File
Send Zabbix metric by reading from a XML file. This needs package zbxsend from : https://github.com/ahmedzbyr/zbxsend
Zabbix Import File from iReasoning Export CSV
Module used to generate Zabbix import xml for snmp traps, Below are steps to create a Zabbix Import for Traps. These traps are captured from the snmptrap fun...
nagiosxi
Nagios - Service Group Summary ERROR
We were working on nagios and found that after our migration, service group summary was not working. You might get below error on the screen and the solution...
Zabbix History Table Clean Up
Zabbix history table gets really big, and if you are in a situation where you want to clean it up. Then we can do so, using the below steps.
Issues - Monitoring MongoDB using Nagios XI.
Monitoring for mongodb using nagiosxi is straight forword but you might have some issues when we are setting up. Here are few issues which might come up usin...
Migrating Nagios XI to a New Server on Centos6.6
Migrating an old Nagios backup to a new nagiosxi server. Migration is simple in Nagios XI, you a backup and restore it. Once that is done we need to update/a...
Setting up SSL https On Nagios XI Server
HTTPS is a protocol for secure communication over a computer network which is widely used on the Internet. HTTPS consists of communication over Hypertext Tra...
Setup/Configuration Nagios XI on Centos6.6
Nagios monitors your entire IT infrastructure to ensure systems, applications, services, and business processes are functioning properly. In the event of a f...
nc
Check Port on Remote Server CentOS 6.6/RHEL 6
Checking port available on a remote machine using nc command instead of telnet. Same command can be used to check on a remote server as well, change the 127....
neo4j
MongoDB to Neo4j Using neo4j_doc_manager.
We had a requirement where we wanted to have all the data which is in mongodb to be replicated on neo4j to show few graphs. Here is quick way to demonstrate ...
network
Setting iptables - Port forwarding from one interface to another.
iptables is a user-space application program that allows a system administrator to configure the tables provided by the Linux kernel firewall (implemented as...
Interface Forwarding - from eth1 to eth0 on EDGE node.
Adding route to all the slaves which reside on a private network to communicate with External Server directly using an EDGE node using Interface Forwarding.
network configuration
Configuring Multiple IP Addresses on a Single Network Interface in Linux
This comprehensive guide explains how to assign multiple IP addresses to a single network interface in Linux. This technique, often called IP aliasing or cre...
networking
Comprehensive Guide to KVM Installation and Configuration on CentOS 6.x
Kernel-based Virtual Machine (KVM) has become a cornerstone of Linux server virtualization, known for its maturity and performance. Red Hat’s decision to fav...
Navigating npm Behind a Corporate Proxy - A Comprehensive Guide
npm, short for Node Package Manager, is the cornerstone of JavaScript development within the Node.js ecosystem. It simplifies the process of managing projec...
nfs
Setup and Configure NFS Mounts on Linux Server
A Network File System (NFS) allows remote hosts to mount file systems over a network and interact with those file systems as though they are mounted locally....
nginx
Ansible 101- Your First Playbook to Automate Web Server Setup
Are you ready to dive into the world of automation? Ansible is a powerful tool that allows you to manage and configure systems with ease. This blog post will...
Setting Up HAProxy for High Availability and Load Balancing on CentOS
HAProxy is a robust, high-performance TCP/HTTP load balancer, perfectly suited for ensuring high availability and efficient traffic distribution across your ...
Optimizing sysctl.conf for High-Performance Web Servers - Node.js, Nginx
This post provides a detailed sysctl.conf file, optimized for Linux systems running high-performance web servers like Nginx and Node.js applications. The sy...
Performance Tuning for nginx
Nginx (pronounced “engine x”) is a powerful and versatile web server renowned for its high concurrency, exceptional performance, and efficient memory utiliza...
High Availability Nginx with Heartbeat - A Step-by-Step Guide
In today’s digital landscape, ensuring high availability for your web applications is paramount. Downtime can lead to lost revenue, damaged reputation, and ...
Setting up NGINX for HTTP Load Balancing - A Comprehensive Guide
NGINX is a powerful and versatile open-source software that excels as an HTTP server, reverse proxy, and IMAP/POP3 proxy server. Renowned for its high perfo...
nodejs
Navigating npm Behind a Corporate Proxy - A Comprehensive Guide
npm, short for Node Package Manager, is the cornerstone of JavaScript development within the Node.js ecosystem. It simplifies the process of managing projec...
Integrating NodeJS and Kafka - A Practical Guide
This blog post provides a comprehensive, step-by-step guide to integrating NodeJS with Kafka. Whether you’re building real-time data pipelines, implementing ...
Routing JSON Payloads to Multiple Kafka Topics with Node.js and Kafka-node
This guide demonstrates how to build a Node.js application that receives JSON data via HTTP and routes it to different Apache Kafka topics based on the URL p...
NodeJS Kafka Producer - A Deep Dive with kafka-node
Ready to integrate your NodeJS application with Kafka? This post provides a comprehensive guide to setting up a Kafka producer using the kafka-node library. ...
Sending JSON from Browser/cURL to NodeJS to Kafka
This post outlines how to send JSON data from a client (browser or curl) through a NodeJS server and ultimately into a Kafka topic. This architecture is usef...
Installing NodeJS on CentOS 6.6 - A Comprehensive Guide
Node.js is an open-source, cross-platform JavaScript runtime environment that executes JavaScript code server-side. It’s incredibly popular for building scal...
Optimizing sysctl.conf for High-Performance Web Servers - Node.js, Nginx
This post provides a detailed sysctl.conf file, optimized for Linux systems running high-performance web servers like Nginx and Node.js applications. The sy...
npm
Navigating npm Behind a Corporate Proxy - A Comprehensive Guide
npm, short for Node Package Manager, is the cornerstone of JavaScript development within the Node.js ecosystem. It simplifies the process of managing projec...
null_resource
Harnessing the Power of null_resource with local-exec and remote-exec in Terraform
Terraform, a widely adopted Infrastructure as Code (IaC) tool, offers a rich array of resources and provisioners for efficiently managing and configuring inf...
observability
Log-Based Metrics - Unlocking Insights from Your Application Logs
Application logs are a treasure trove of information. Beyond debugging and troubleshooting, they can provide valuable insights into your application’s perfor...
offline
Installing Zabbix Version 2.4 Offline (Zabbix Server without Internet).
There might be situations where you have a remote/zabbix server which does not have internet connectivity, due to security or other reasons. So we create a ...
Zabbix Template Creation using CSV file.
In zabbix we dont have a better way to capture snmptraps. we have to manually create an item and corresponfding trigger to handle a trap arriving from the de...
one-liner
Ad-Hoc Commands - The Unsung Hero of Ansible
Ansible is a powerful automation tool that allows you to manage and configure infrastructure as code. Back when we were running 100s of hadoop nodes these we...
opensource
knife Quick Reference
This is a quick reference for few of the commands which I use often.
Migrating Nagios XI to a New Server on Centos6.6
Migrating an old Nagios backup to a new nagiosxi server. Migration is simple in Nagios XI, you a backup and restore it. Once that is done we need to update/a...
optimization
Optimizing sysctl.conf for High-Performance Web Servers - Node.js, Nginx
This post provides a detailed sysctl.conf file, optimized for Linux systems running high-performance web servers like Nginx and Node.js applications. The sy...
Tuning Hadoop Performance with sysctl.conf
This post delves into optimizing Hadoop performance at the kernel level using sysctl. The sysctl interface provides a way to dynamically modify a running Lin...
pacemaker
Creating a two-node RHEL cluster with Virtual IP using CMAN and Pacemaker.
CMAN v3 is a Corsync plugin that monitors the names and number of active cluster nodes in order to deliver membership and quorum information to clients (such...
package manager
Fixing Unable to locate package oracle-java7-installer on Ubuntu
I recently ran into a frustrating issue while trying to install Java 7 on an older Ubuntu system (Ubuntu 13, in this case). The seemingly simple command sud...
packer
Automating GCE Image Creation with Packer
Ensuring consistency and streamlining infrastructure provisioning is crucial for effective cloud management. Creating custom virtual machine (VM) images on G...
pam
Explain - System Security Services Daemon (SSSD) Config.
The System Security Services Daemon (SSSD) provides a set of daemons to manage access to remote directories and authentication mechanisms. It provides Name S...
parted
Mounting RAID10 using parted.
GUID Partition Table (GPT) is a standard for the layout of the partition table on a physical hard disk, using globally unique identifiers (GUID). Although it...
pentaho
Setting up Pentaho Data Integration 5.4.1 with Hadoop Cluster (Clouder Manager)
Pentaho Data Integration (PDI) is a suite of open source Business Intelligence (BI) products which provide data integration, OLAP services, reporting, dashb...
performance
Optimizing sysctl.conf for High-Performance Web Servers - Node.js, Nginx
This post provides a detailed sysctl.conf file, optimized for Linux systems running high-performance web servers like Nginx and Node.js applications. The sy...
Fine-Tuning Linux Kernel Parameters with sysctl.conf for Web and FTP Servers
The sysctl interface provides a powerful way to modify the behavior of a running Linux kernel. By editing the /etc/sysctl.conf file, you can persistently co...
Tuning Hadoop Performance with sysctl.conf
This post delves into optimizing Hadoop performance at the kernel level using sysctl. The sysctl interface provides a way to dynamically modify a running Lin...
Performance Tuning for nginx
Nginx (pronounced “engine x”) is a powerful and versatile web server renowned for its high concurrency, exceptional performance, and efficient memory utiliza...
Installing and Initial Setup of Tsung Load Testing on CentOS
Tsung (formerly known as idx-Tsunami) is a powerful, open-source, multi-protocol distributed load testing tool. Written in Erlang, it’s licensed under the GP...
Load Testing with Siege on CentOS 6.5 - Kernel 2.6, x86_64 - A Comprehensive Guide
Siege is a powerful, open-source HTTP load testing and benchmarking utility. It’s designed to help web developers and system administrators measure the perfo...
performance-tuning
Performance Tuning HBase and Hadoop for Optimal Performance
Getting HBase to perform well in a production environment is an iterative process that involves tweaking various configuration parameters. This post dives de...
permissions
Fixing sudo effective uid is not 0 Error on Linux
Encountered a frustrating issue on a Linux server while trying to use sudo? Specifically, the error message: sudo: effective uid is not 0, is sudo installed ...
pitr
Firestore Native Mode - Point in Time Recovery (PITR)
Data is invaluable, and as any developer would testify, ensuring its security and availability in the event of unintended deletion or modification is paramou...
playbook
Ansible 101- Your First Playbook to Automate Web Server Setup
Are you ready to dive into the world of automation? Ansible is a powerful tool that allows you to manage and configure systems with ease. This blog post will...
port-forwarding
Setting iptables - Port forwarding from one interface to another.
iptables is a user-space application program that allows a system administrator to configure the tables provided by the Linux kernel firewall (implemented as...
postgres
Setting up Postgres using Dockerfile.
In this blog post we will be setting up a postgres database on docker using Dockerfile.
Setting up Postgres on Docker.
In this blog post we will be setting up a postgres database on docker.
Zabbix Database Backup Script for postgres and mysql.
After the zabbix setup is done, we would like to take backup of the database. Here we are doing a complete backup. But if we are looking for only configurati...
producer
NodeJS Kafka Producer - A Deep Dive with kafka-node
Ready to integrate your NodeJS application with Kafka? This post provides a comprehensive guide to setting up a Kafka producer using the kafka-node library. ...
proxy
Datastream Workflow - Cloud SQL (MySQL) to BigQuery via Cloud Auth Proxy
Welcome back to our series on datastream workflows! In our previous article, we explored the basics of setting up a datastream workflow.
Installing squid as a sibling to an already existing Parent squid.
Squid is a caching and forwarding web proxy. It has a wide variety of uses, from speeding up a web server by caching repeated requests; to caching web, DNS a...
Setting Up HAProxy for High Availability and Load Balancing on CentOS
HAProxy is a robust, high-performance TCP/HTTP load balancer, perfectly suited for ensuring high availability and efficient traffic distribution across your ...
Navigating npm Behind a Corporate Proxy - A Comprehensive Guide
npm, short for Node Package Manager, is the cornerstone of JavaScript development within the Node.js ecosystem. It simplifies the process of managing projec...
python
Creating an Ansible Collection - A Detailed Step-by-Step Guide
Creating an Ansible Collection: A Comprehensive Guide
Monitoring Firestore Health - A Python-Based Uptime Check
Ensuring the health and availability of your Firestore database is crucial for maintaining a reliable application. This post provides a Python script that se...
Data Export from Datastore and Firestore
In this blog post, we will venture into exporting data from Firestore and Datastore modes. Find the code on Github.
Firestore Native Mode - Point in Time Recovery (PITR)
Data is invaluable, and as any developer would testify, ensuring its security and availability in the event of unintended deletion or modification is paramou...
Design IAM on Google Cloud Platform
Security is paramount when setting up resources in the cloud or on-premises. It encompasses various layers of protection, including network security, encrypt...
Harnessing the Power of null_resource with local-exec and remote-exec in Terraform
Terraform, a widely adopted Infrastructure as Code (IaC) tool, offers a rich array of resources and provisioners for efficiently managing and configuring inf...
Exploring Data Extraction from BigQuery
BigQuery, Google’s fully-managed and serverless data warehouse, empowers organizations to analyze massive datasets with remarkable speed and efficiency. But ...
10 Tips for Optimizing Your Queries and Reducing Costs in BigQuery
Google BigQuery is a powerful and cost-effective data warehouse that enables you to analyze massive datasets quickly. However, to get the most out of BigQuer...
Unleash the Power of Google Cloud with the Cloud Asset Inventory Python API
Managing resources in your Google Cloud Platform (GCP) projects is a complex task, especially when dealing with multiple projects. Fortunately, Google offers...
Setting up Postgres using Dockerfile.
In this blog post we will be setting up a postgres database on docker using Dockerfile.
Python - Virtual Environment.
Python virtual environment creates a isoloated workspace of python work. This helps in creating project specific virtual environment without worrying about ...
Setting up Postgres on Docker.
In this blog post we will be setting up a postgres database on docker.
Python - List Comprehensions.
List Comprehensions provides easy and functional way to create list in python. We could make a single line of code which otherwise would take a few lines. l...
Python Getting Started - Learning Notes.
Python is a clear and powerful object-oriented programming language, comparable to Perl, Ruby, Scheme, or Java. This is basic documentation for getting star...
MongoDB to Neo4j Using neo4j_doc_manager.
We had a requirement where we wanted to have all the data which is in mongodb to be replicated on neo4j to show few graphs. Here is quick way to demonstrate ...
Variable Substitution in YAML Config pyyaml [Python]
PyYAML - The next generation YAML parser and emitter for Python.
Cropping Bulk Images Using Python.
I was working on getting post headers for my post on this blog. I had couple of images from unsplash. But the header for the post need to be a little more ho...
Installing python 2.7.x on Centos 6.5/6.6
By default centos comes with python 2.6. In most of the cases we might need python 2.7 or later to be installed. Below are few ways to install python 2.7 on ...
Zabbix Hadoop Monitoring.
This script can be used to monitor Namenode Parameters. This script can be used to Generate Zabbix Import XML or Send monitoring data to Zabbix server.
Fetch Monitoring Data from Item to CSV file.
This section describes how to get monitoring data from Zabbix server through the command line and export these to a CSV formatted file.
Zabbix Item Creation using OID and Name CSV files.
This script is to create a xml import file from a csv file. We use 2 csv file for this.
Send KPI Data to Zabbix from XML KPI File
Send Zabbix metric by reading from a XML file. This needs package zbxsend from : https://github.com/ahmedzbyr/zbxsend
SFTP Data Collector
Easy way to collect files recursively over a sftp server is to connect to the server over scp and do scp -r. Problem was that the device we were connecting ...
quick-start
Installing KAFKA Single Node - Quick Start Guide
Apache Kafka is often described as a distributed, fault-tolerant, high-throughput publish-subscribe messaging system. Think of it as a central nervous system...
raid
Mounting RAID10 using parted.
GUID Partition Table (GPT) is a standard for the layout of the partition table on a physical hard disk, using globally unique identifiers (GUID). Although it...
recovery
Data Export from Datastore and Firestore
In this blog post, we will venture into exporting data from Firestore and Datastore modes. Find the code on Github.
Firestore Native Mode - Point in Time Recovery (PITR)
Data is invaluable, and as any developer would testify, ensuring its security and availability in the event of unintended deletion or modification is paramou...
redhat
Kafka Kerberos Enable and Testing.
Apache Kafka is a distributed streaming platform. Kafka 2.0 supports Kerberos authentication, Enabling Kerberos Authentication Using the Wizard on cloudera m...
Cloudera Manager - Duplicate entry ‘zookeeper’ for key ‘NAME’.
Deployment error during setting zookeeper on a new cluster using Cloudera Manager UI, when an earlier cluster containing zookeeper earlier was built using Cl...
Parcel Not Distributing Cloudera CDH.
We were deploying one of the cluster on our lab environment which is used by everyone. So the lab has it own share of stale information on it.
Creating /etc/hosts file in Chef.
We had a cluster environment which we needed to update the /etc/hosts file. Which would help communicate between the server over a private network. Our serve...
replication
High Availability Nginx with Heartbeat - A Step-by-Step Guide
In today’s digital landscape, ensuring high availability for your web applications is paramount. Downtime can lead to lost revenue, damaged reputation, and ...
SpagoBI Tomcat Clustering with mod_jk and httpd on CentOS - In-Memory Session Replication - A Deep Dive
In a previous post, we explored a basic session-based replication setup for SpagoBI. However, that configuration lacked true session redundancy.
reverse-proxy
Setting up NGINX for HTTP Load Balancing - A Comprehensive Guide
NGINX is a powerful and versatile open-source software that excels as an HTTP server, reverse proxy, and IMAP/POP3 proxy server. Renowned for its high perfo...
rhel
knife Quick Reference
This is a quick reference for few of the commands which I use often.
Migrating Nagios XI to a New Server on Centos6.6
Migrating an old Nagios backup to a new nagiosxi server. Migration is simple in Nagios XI, you a backup and restore it. Once that is done we need to update/a...
Setup/Configuration Nagios XI on Centos6.6
Nagios monitors your entire IT infrastructure to ensure systems, applications, services, and business processes are functioning properly. In the event of a f...
RPM Command Cheat Sheet
RPM (Redhat Package Manager) is the most popular package utility and is used mostly on RHEL, Centos and Fedora. RPM helps user/admins to build, install, que...
YUM Repository Creation on HTTPD Web Server.
Setting up yum repos on RHEL using httpd. We will be setting up httpd and yum repo on top of it. So that we can access yum over http.
Access Filter Setup with SSSD
If using access_provider = ldap, this option is mandatory. It specifies an LDAP search filter criteria that must be met for the user to be granted access on ...
Redhat Intergration with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
Simple Steps to Intergrate RHEL with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
Installing ansible on RHEL 6.6.
Ansible is a radically simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchest...
Creating a passwordless entry on Centos 6.5.
You can enter a linux system without entring a password using below steps. We will be creating a ssh jey which will be share between the servers, which will ...
Creating a Multi-node Cassandra Cluster on Centos 6.5.
The Apache Cassandra database is the right choice when you need scalability and high availability without compromising performance. Linear scalability and pr...
Comprehensive Guide to KVM Installation and Configuration on CentOS 6.x
Kernel-based Virtual Machine (KVM) has become a cornerstone of Linux server virtualization, known for its maturity and performance. Red Hat’s decision to fav...
Installing zabbix-java-gateway on Centos 6.5
Zabbix 2.0 introduced native support for monitoring JMX applications through the Zabbix Java gateway. This Java-based daemon acts as an intermediary, allowin...
Installing NodeJS on CentOS 6.6 - A Comprehensive Guide
Node.js is an open-source, cross-platform JavaScript runtime environment that executes JavaScript code server-side. It’s incredibly popular for building scal...
Simple Script to Create Users on CentOS/RHEL 6.5
This script provides a simple way to create users on CentOS/RHEL 6.5 systems from the command line. It allows you to create both normal and sudo (administrat...
Setting up Highly Available Apache (httpd) with Heartbeat on CentOS/RHEL 6
This guide walks you through configuring a High Availability (HA) Apache web server using Heartbeat on CentOS/RHEL 6. HA ensures your web server remains acce...
root
Fixing sudo effective uid is not 0 Error on Linux
Encountered a frustrating issue on a Linux server while trying to use sudo? Specifically, the error message: sudo: effective uid is not 0, is sudo installed ...
routing
Routing JSON Payloads to Multiple Kafka Topics with Node.js and Kafka-node
This guide demonstrates how to build a Node.js application that receives JSON data via HTTP and routes it to different Apache Kafka topics based on the URL p...
rpms
RPM Command Cheat Sheet
RPM (Redhat Package Manager) is the most popular package utility and is used mostly on RHEL, Centos and Fedora. RPM helps user/admins to build, install, que...
rrd
Moving RRD file from 32bit to 64bit Architecture
When we were working on a nagios monitoring system we were migrating from a 32bit nagios to a 64bit Architecture. Most of the graphs are not working as the R...
ruby
Installing Ruby2.4 on Ubuntu Docker Container.
Was setting up jekyll using docker, but was not able to install ruby on the docker machine as apt-add-repository was not working. Below is my Dockerfile when...
scheduler
Setting up Scheduled dataflow backups using Batch templates.
TL;DR In this post we will be setting up a scheduled job to take backup for Bigtable table in avro format.
script
Simple Script to Create Users on CentOS/RHEL 6.5
This script provides a simple way to create users on CentOS/RHEL 6.5 systems from the command line. It allows you to create both normal and sudo (administrat...
SFTP Data Collector
Easy way to collect files recursively over a sftp server is to connect to the server over scp and do scp -r. Problem was that the device we were connecting ...
security
Explain - System Security Services Daemon (SSSD) Config.
The System Security Services Daemon (SSSD) provides a set of daemons to manage access to remote directories and authentication mechanisms. It provides Name S...
Access Filter Setup with SSSD
If using access_provider = ldap, this option is mandatory. It specifies an LDAP search filter criteria that must be met for the user to be granted access on ...
Getting started with Hive with Kerberos.
Apache Hive is a powerful data warehousing application built on top of Hadoop; it enables you to access your data using Hive QL, a language that is similar t...
Redhat Intergration with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
Simple Steps to Intergrate RHEL with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
Fixing sudo effective uid is not 0 Error on Linux
Encountered a frustrating issue on a Linux server while trying to use sudo? Specifically, the error message: sudo: effective uid is not 0, is sudo installed ...
Fine-Tuning Linux Kernel Parameters with sysctl.conf for Web and FTP Servers
The sysctl interface provides a powerful way to modify the behavior of a running Linux kernel. By editing the /etc/sysctl.conf file, you can persistently co...
Securing Your Data: A Step-by-Step Guide to Enabling HBase Authorization
Apache HBase is a powerful NoSQL database, but out-of-the-box, it’s a bit like a house with no locks on the doors. Any user who can connect to the cluster ca...
server
Changing Timezone in Ubuntu Server - A Comprehensive Guide
Changing the timezone on your Ubuntu server is a crucial task for ensuring accurate timekeeping, especially when dealing with scheduled tasks, logs, and appl...
setuid
Fixing sudo effective uid is not 0 Error on Linux
Encountered a frustrating issue on a Linux server while trying to use sudo? Specifically, the error message: sudo: effective uid is not 0, is sudo installed ...
sftp
SFTP Data Collector
Easy way to collect files recursively over a sftp server is to connect to the server over scp and do scp -r. Problem was that the device we were connecting ...
siege
Installing and Initial Setup of Tsung Load Testing on CentOS
Tsung (formerly known as idx-Tsunami) is a powerful, open-source, multi-protocol distributed load testing tool. Written in Erlang, it’s licensed under the GP...
Load Testing with Siege on CentOS 6.5 - Kernel 2.6, x86_64 - A Comprehensive Guide
Siege is a powerful, open-source HTTP load testing and benchmarking utility. It’s designed to help web developers and system administrators measure the perfo...
snmp
Zabbix Database Backup Script for postgres and mysql.
After the zabbix setup is done, we would like to take backup of the database. Here we are doing a complete backup. But if we are looking for only configurati...
Installing Zabbix Version 2.4 Offline (Zabbix Server without Internet).
There might be situations where you have a remote/zabbix server which does not have internet connectivity, due to security or other reasons. So we create a ...
Zabbix Template Creation using CSV file.
In zabbix we dont have a better way to capture snmptraps. we have to manually create an item and corresponfding trigger to handle a trap arriving from the de...
Installing SNMP Builder using zabbix-extra on Zabbix Version 2.4.
SNMP Builder/Extra is an add-in for Zabbix. It provides new web interface components to browse MIB trees and values. SNMP OIDs can then be automatically conv...
Setting up SNMP Trapper for Zabbix.
Receiving SNMP traps is the opposite to querying SNMP-enabled devices. In this case the information is sent from a SNMP-enabled device and is collected or “t...
spago
Installing SpagoBI 5.1 on Centos 6.5 -Tomcat 7 with MySQL 5.6.
The SpagoBI project is a free software/open source initiative by the SpagoBI Labs of Engineering Group. It aims to realize the most complete 100% open source...
Setting up Tomcat Cluster for SpagoBI 5.1.
This post details how to set up a Tomcat cluster for SpagoBI 5.1, building upon a previous guide for installing SpagoBI with MySQL (post). This configuration...
spago-bi
SpagoBI Tomcat Clustering with mod_jk and httpd on CentOS - In-Memory Session Replication - A Deep Dive
In a previous post, we explored a basic session-based replication setup for SpagoBI. However, that configuration lacked true session redundancy.
Setting up Tomcat Cluster for SpagoBI 5.1.
This post details how to set up a Tomcat cluster for SpagoBI 5.1, building upon a previous guide for installing SpagoBI with MySQL (post). This configuration...
spagobi
Installing SpagoBI 5.1 on Centos 6.5 -Tomcat 7 with MySQL 5.6.
The SpagoBI project is a free software/open source initiative by the SpagoBI Labs of Engineering Group. It aims to realize the most complete 100% open source...
spark
Streaming Data Processing - Storm Vs Spark.
Apache Spark is an in-memory distributed data analysis platform– primarily targeted at speeding up batch analysis jobs, iterative machine learning jobs, inte...
sql
Cloud Databases - SQL/Spanner/Firestore/BigTable/Memorystore - GCP Learning Notes.
Cloud SQL is a fully managed service which includes MySQL, PostgreSQL, Microsoft SQL Server. And for horizontal scalability, consider using Cloud Spanner.
squid
Installing squid as a sibling to an already existing Parent squid.
Squid is a caching and forwarding web proxy. It has a wide variety of uses, from speeding up a web server by caching repeated requests; to caching web, DNS a...
ssl
Setting up SSL https On Nagios XI Server
HTTPS is a protocol for secure communication over a computer network which is widely used on the Internet. HTTPS consists of communication over Hypertext Tra...
sssd
Explain - System Security Services Daemon (SSSD) Config.
The System Security Services Daemon (SSSD) provides a set of daemons to manage access to remote directories and authentication mechanisms. It provides Name S...
Access Filter Setup with SSSD
If using access_provider = ldap, this option is mandatory. It specifies an LDAP search filter criteria that must be met for the user to be granted access on ...
Redhat Intergration with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
Simple Steps to Intergrate RHEL with Active Directory using SSSD.
There are inherent structural differences between how Windows and Linux handle system users. The user schemas used in Active Directory and standard LDAPv3 di...
sssd-configuration
Explain - System Security Services Daemon (SSSD) Config.
The System Security Services Daemon (SSSD) provides a set of daemons to manage access to remote directories and authentication mechanisms. It provides Name S...
static IP
Configuring Multiple IP Addresses on a Single Network Interface in Linux
This comprehensive guide explains how to assign multiple IP addresses to a single network interface in Linux. This technique, often called IP aliasing or cre...
storage
Cloud Storge Bucket - GCP Learning Notes.
Cloud Storage is Google Cloud’s object storage service, and it allows worldwide storage and retrieval of any amount of data at anytime.
storm
Ansible Playbook - Setup Storm Cluster.
This is a simple Storm Cluster Setup. We are using a dedicated Zookeeper Cluster/Node, instead of the standalone zkserver. Below is how we will deploy our c...
Streaming Data Processing - Storm Vs Spark.
Apache Spark is an in-memory distributed data analysis platform– primarily targeted at speeding up batch analysis jobs, iterative machine learning jobs, inte...
sudo
Fixing sudo effective uid is not 0 Error on Linux
Encountered a frustrating issue on a Linux server while trying to use sudo? Specifically, the error message: sudo: effective uid is not 0, is sudo installed ...
swap
How To Configure Swappiness
Swappiness is a crucial Linux kernel parameter that dictates how aggressively the system uses swap space. Understanding and configuring swappiness can signif...
swappiness
How To Configure Swappiness
Swappiness is a crucial Linux kernel parameter that dictates how aggressively the system uses swap space. Understanding and configuring swappiness can signif...
sysctl
Performance Tuning HBase and Hadoop for Optimal Performance
Getting HBase to perform well in a production environment is an iterative process that involves tweaking various configuration parameters. This post dives de...
Optimizing sysctl.conf for High-Performance Web Servers - Node.js, Nginx
This post provides a detailed sysctl.conf file, optimized for Linux systems running high-performance web servers like Nginx and Node.js applications. The sy...
Fine-Tuning Linux Kernel Parameters with sysctl.conf for Web and FTP Servers
The sysctl interface provides a powerful way to modify the behavior of a running Linux kernel. By editing the /etc/sysctl.conf file, you can persistently co...
Tuning Hadoop Performance with sysctl.conf
This post delves into optimizing Hadoop performance at the kernel level using sysctl. The sysctl interface provides a way to dynamically modify a running Lin...
How To Configure Swappiness
Swappiness is a crucial Linux kernel parameter that dictates how aggressively the system uses swap space. Understanding and configuring swappiness can signif...
system-administration
Ad-Hoc Commands - The Unsung Hero of Ansible
Ansible is a powerful automation tool that allows you to manage and configure infrastructure as code. Back when we were running 100s of hadoop nodes these we...
systemctl
Changing Timezone in Ubuntu Server - A Comprehensive Guide
Changing the timezone on your Ubuntu server is a crucial task for ensuring accurate timekeeping, especially when dealing with scheduled tasks, logs, and appl...
tarball
Ansible Playbook - Setup Hadoop CDH5 Using tarball.
Setting up Hadoop using Ansible, we will be using cdh5 tarball for installation of the cluster.
terraform
Testing Your Infrastructure Code - A Guide to Terratest on GCP
Infrastructure as Code (IaC) has revolutionized how we manage and provision infrastructure. Tools like Terraform allow us to define our infrastructure in cod...
Automating Firestore Health Checks with Terraform Cloud Functions Cloud Monitoring.
This blog post demonstrates how to automate Firestore health checks using Terraform to deploy a Cloud Function (Gen 1) and set up a complete monitoring and a...
Datastream Workflow - Cloud SQL (MySQL) to BigQuery via Cloud Auth Proxy
Welcome back to our series on datastream workflows! In our previous article, we explored the basics of setting up a datastream workflow.
Establishing a Datastream from Cloud SQL (MySQL) to BigQuery
This guide provides detailed instructions on setting up a datastream connection from Cloud SQL (MySQL) to BigQuery. We aim to simplify the process by using d...
Setting Up HashiCorp Vault for Secret Management
We will walk you through the process of setting up HashiCorp Vault to effectively manage your secrets. This guide is an gettinf started with vault with terra...
Firestore Native Mode - Point in Time Recovery (PITR)
Data is invaluable, and as any developer would testify, ensuring its security and availability in the event of unintended deletion or modification is paramou...
Automating GCE Image Creation with Packer
Ensuring consistency and streamlining infrastructure provisioning is crucial for effective cloud management. Creating custom virtual machine (VM) images on G...
Google Compute Engine (GCE) Instance with Ansible
Using Ansible within the initialization script of a Google Compute Engine (GCE) instance can streamline and automate the configuration and setup of your virt...
Design IAM on Google Cloud Platform
Security is paramount when setting up resources in the cloud or on-premises. It encompasses various layers of protection, including network security, encrypt...
Harnessing the Power of null_resource with local-exec and remote-exec in Terraform
Terraform, a widely adopted Infrastructure as Code (IaC) tool, offers a rich array of resources and provisioners for efficiently managing and configuring inf...
Exploring Data Extraction from BigQuery
BigQuery, Google’s fully-managed and serverless data warehouse, empowers organizations to analyze massive datasets with remarkable speed and efficiency. But ...
10 Tips for Optimizing Your Queries and Reducing Costs in BigQuery
Google BigQuery is a powerful and cost-effective data warehouse that enables you to analyze massive datasets quickly. However, to get the most out of BigQuer...
Unleash the Power of Google Cloud with the Cloud Asset Inventory Python API
Managing resources in your Google Cloud Platform (GCP) projects is a complex task, especially when dealing with multiple projects. Fortunately, Google offers...
Terraform Workflow Validation with null_resource
When working with Terraform, it’s crucial to ensure that your infrastructure follows certain conditions or constraints. While Terraform provides validation m...
Effective Terraform Validation Techniques
Validation in Terraform is an essential practice to detect and prevent errors early in the infrastructure provisioning process. By incorporating robust valid...
Setting Up Kerberos on a Dataproc Cluster A Step-by-Step Guide
Kerberos authentication in Google Dataproc clusters offers enhanced security and user isolation. It’s a crucial step for multi-tenancy, encryption, and user ...
Setting up Scheduled dataflow backups using Batch templates.
TL;DR In this post we will be setting up a scheduled job to take backup for Bigtable table in avro format.
terratest
Testing Your Infrastructure Code - A Guide to Terratest on GCP
Infrastructure as Code (IaC) has revolutionized how we manage and provision infrastructure. Tools like Terraform allow us to define our infrastructure in cod...
testing
Testing Your Infrastructure Code - A Guide to Terratest on GCP
Infrastructure as Code (IaC) has revolutionized how we manage and provision infrastructure. Tools like Terraform allow us to define our infrastructure in cod...
Basic Testing On Hadoop Environment [Cloudera]
These are a set of testing which we can do on a Hadoop environment. These are basic testing to make sure the environment is setup correctly.
Windows Testing Using Kitchen Chef
Kitchen-Vagrant has the capability to spin up a windows instance for testing. To make it work you will need the vagrant-winrm to be installted on the worksta...
Installing and Initial Setup of Tsung Load Testing on CentOS
Tsung (formerly known as idx-Tsunami) is a powerful, open-source, multi-protocol distributed load testing tool. Written in Erlang, it’s licensed under the GP...
Load Testing with Siege on CentOS 6.5 - Kernel 2.6, x86_64 - A Comprehensive Guide
Siege is a powerful, open-source HTTP load testing and benchmarking utility. It’s designed to help web developers and system administrators measure the perfo...
timezone
Changing Timezone in Ubuntu Server - A Comprehensive Guide
Changing the timezone on your Ubuntu server is a crucial task for ensuring accurate timekeeping, especially when dealing with scheduled tasks, logs, and appl...
tomcat
Installing SpagoBI 5.1 on Centos 6.5 -Tomcat 7 with MySQL 5.6.
The SpagoBI project is a free software/open source initiative by the SpagoBI Labs of Engineering Group. It aims to realize the most complete 100% open source...
SpagoBI Tomcat Clustering with mod_jk and httpd on CentOS - In-Memory Session Replication - A Deep Dive
In a previous post, we explored a basic session-based replication setup for SpagoBI. However, that configuration lacked true session redundancy.
Setting up Tomcat Cluster for SpagoBI 5.1.
This post details how to set up a Tomcat cluster for SpagoBI 5.1, building upon a previous guide for installing SpagoBI with MySQL (post). This configuration...
trap
Zabbix Database Backup Script for postgres and mysql.
After the zabbix setup is done, we would like to take backup of the database. Here we are doing a complete backup. But if we are looking for only configurati...
Installing Zabbix Version 2.4 Offline (Zabbix Server without Internet).
There might be situations where you have a remote/zabbix server which does not have internet connectivity, due to security or other reasons. So we create a ...
Zabbix Template Creation using CSV file.
In zabbix we dont have a better way to capture snmptraps. we have to manually create an item and corresponfding trigger to handle a trap arriving from the de...
Installing SNMP Builder using zabbix-extra on Zabbix Version 2.4.
SNMP Builder/Extra is an add-in for Zabbix. It provides new web interface components to browse MIB trees and values. SNMP OIDs can then be automatically conv...
Setting up SNMP Trapper for Zabbix.
Receiving SNMP traps is the opposite to querying SNMP-enabled devices. In this case the information is sent from a SNMP-enabled device and is collected or “t...
troubleshooting
5 Common Ansible Mistakes and How to Avoid Them
Ansible is a powerful automation tool that simplifies infrastructure management and application deployment. However, like any technology, it’s easy to make m...
Fixing Not Enough Physical Memory Error in VMware Workstation 10 on Windows 8.1
Are you encountering the frustrating “Not enough physical memory is available to power this virtual machine” error in VMware Workstation 10 after a Windows u...
Fixing Unable to locate package oracle-java7-installer on Ubuntu
I recently ran into a frustrating issue while trying to install Java 7 on an older Ubuntu system (Ubuntu 13, in this case). The seemingly simple command sud...
Fixing sudo effective uid is not 0 Error on Linux
Encountered a frustrating issue on a Linux server while trying to use sudo? Specifically, the error message: sudo: effective uid is not 0, is sudo installed ...
tsung
Installing and Initial Setup of Tsung Load Testing on CentOS
Tsung (formerly known as idx-Tsunami) is a powerful, open-source, multi-protocol distributed load testing tool. Written in Erlang, it’s licensed under the GP...
tuning
Optimizing sysctl.conf for High-Performance Web Servers - Node.js, Nginx
This post provides a detailed sysctl.conf file, optimized for Linux systems running high-performance web servers like Nginx and Node.js applications. The sy...
Fine-Tuning Linux Kernel Parameters with sysctl.conf for Web and FTP Servers
The sysctl interface provides a powerful way to modify the behavior of a running Linux kernel. By editing the /etc/sysctl.conf file, you can persistently co...
Tuning Hadoop Performance with sysctl.conf
This post delves into optimizing Hadoop performance at the kernel level using sysctl. The sysctl interface provides a way to dynamically modify a running Lin...
Performance Tuning for nginx
Nginx (pronounced “engine x”) is a powerful and versatile web server renowned for its high concurrency, exceptional performance, and efficient memory utiliza...
tutorial
Installing NodeJS on CentOS 6.6 - A Comprehensive Guide
Node.js is an open-source, cross-platform JavaScript runtime environment that executes JavaScript code server-side. It’s incredibly popular for building scal...
ubuntu
Windows Testing Using Kitchen Chef
Kitchen-Vagrant has the capability to spin up a windows instance for testing. To make it work you will need the vagrant-winrm to be installted on the worksta...
Issues - Monitoring MongoDB using Nagios XI.
Monitoring for mongodb using nagiosxi is straight forword but you might have some issues when we are setting up. Here are few issues which might come up usin...
Installing CouchDB on Ubuntu 14 LTS.
CouchDB is a database that completely embraces the web. Store your data with JSON documents. Access your documents and query your indexes with your web brows...
Installing MongoDB on Ubuntu 14 LTS.
MongoDB is an open-source document database, and leading NoSQL database. MongoDB is written in c++. Below is a brief document about installing a mongodb on a...
Encrypted Data Bags - Chef
Data Bags are a way to store information on the chef-server which all the cookbooks can access. Few more additional advantages are that we can encrypt the da...
Remove Old Files using find Command
GNU find searches the directory tree rooted at each given file name by evaluating the given expression from left to right, according to the rules of preceden...
Moving RRD file from 32bit to 64bit Architecture
When we were working on a nagios monitoring system we were migrating from a 32bit nagios to a 64bit Architecture. Most of the graphs are not working as the R...
Setting up an HBase Cluster on Hadoop YARN - A Detailed Guide for Ubuntu 12.04 LTS
HBase is a powerful open-source, non-relational, distributed database designed for handling massive datasets. Inspired by Google’s BigTable, HBase is written...
Fixing Unable to locate package oracle-java7-installer on Ubuntu
I recently ran into a frustrating issue while trying to install Java 7 on an older Ubuntu system (Ubuntu 13, in this case). The seemingly simple command sud...
Changing Timezone in Ubuntu Server - A Comprehensive Guide
Changing the timezone on your Ubuntu server is a crucial task for ensuring accurate timekeeping, especially when dealing with scheduled tasks, logs, and appl...
Setting SUN Java for Ubuntu - A Step-by-Step Guide
The Java Development Kit (JDK) is an essential tool for Java developers, providing the necessary components to compile, debug, and run Java applications. Whi...
upgrade
Update Cloudera Manager to specific version [5.4.5]
Upgrading Cloudera Manager 5 to the Latest Cloudera Manager, In most cases it is possible to complete the following upgrade without shutting down most CDH se...
uptime
Monitoring Firestore Health - A Python-Based Uptime Check
Ensuring the health and availability of your Firestore database is crucial for maintaining a reliable application. This post provides a Python script that se...
user
Simple Script to Create Users on CentOS/RHEL 6.5
This script provides a simple way to create users on CentOS/RHEL 6.5 systems from the command line. It allows you to create both normal and sudo (administrat...
vault
Setting Up HashiCorp Vault for Secret Management
We will walk you through the process of setting up HashiCorp Vault to effectively manage your secrets. This guide is an gettinf started with vault with terra...
virt-manager
Comprehensive Guide to KVM Installation and Configuration on CentOS 6.x
Kernel-based Virtual Machine (KVM) has become a cornerstone of Linux server virtualization, known for its maturity and performance. Red Hat’s decision to fav...
virtual interface
Configuring Multiple IP Addresses on a Single Network Interface in Linux
This comprehensive guide explains how to assign multiple IP addresses to a single network interface in Linux. This technique, often called IP aliasing or cre...
virtual-machine
Comprehensive Guide to KVM Installation and Configuration on CentOS 6.x
Kernel-based Virtual Machine (KVM) has become a cornerstone of Linux server virtualization, known for its maturity and performance. Red Hat’s decision to fav...
Fixing Not Enough Physical Memory Error in VMware Workstation 10 on Windows 8.1
Are you encountering the frustrating “Not enough physical memory is available to power this virtual machine” error in VMware Workstation 10 after a Windows u...
vmware
Comprehensive Guide to KVM Installation and Configuration on CentOS 6.x
Kernel-based Virtual Machine (KVM) has become a cornerstone of Linux server virtualization, known for its maturity and performance. Red Hat’s decision to fav...
Fixing Not Enough Physical Memory Error in VMware Workstation 10 on Windows 8.1
Are you encountering the frustrating “Not enough physical memory is available to power this virtual machine” error in VMware Workstation 10 after a Windows u...
web-server
Setting up NGINX for HTTP Load Balancing - A Comprehensive Guide
NGINX is a powerful and versatile open-source software that excels as an HTTP server, reverse proxy, and IMAP/POP3 proxy server. Renowned for its high perfo...
webserver
Ansible 101- Your First Playbook to Automate Web Server Setup
Are you ready to dive into the world of automation? Ansible is a powerful tool that allows you to manage and configure systems with ease. This blog post will...
YUM Repository Creation on HTTPD Web Server.
Setting up yum repos on RHEL using httpd. We will be setting up httpd and yum repo on top of it. So that we can access yum over http.
Integrating NodeJS and Kafka - A Practical Guide
This blog post provides a comprehensive, step-by-step guide to integrating NodeJS with Kafka. Whether you’re building real-time data pipelines, implementing ...
Routing JSON Payloads to Multiple Kafka Topics with Node.js and Kafka-node
This guide demonstrates how to build a Node.js application that receives JSON data via HTTP and routes it to different Apache Kafka topics based on the URL p...
NodeJS Kafka Producer - A Deep Dive with kafka-node
Ready to integrate your NodeJS application with Kafka? This post provides a comprehensive guide to setting up a Kafka producer using the kafka-node library. ...
Sending JSON from Browser/cURL to NodeJS to Kafka
This post outlines how to send JSON data from a client (browser or curl) through a NodeJS server and ultimately into a Kafka topic. This architecture is usef...
Installing NodeJS on CentOS 6.6 - A Comprehensive Guide
Node.js is an open-source, cross-platform JavaScript runtime environment that executes JavaScript code server-side. It’s incredibly popular for building scal...
Optimizing sysctl.conf for High-Performance Web Servers - Node.js, Nginx
This post provides a detailed sysctl.conf file, optimized for Linux systems running high-performance web servers like Nginx and Node.js applications. The sy...
Fine-Tuning Linux Kernel Parameters with sysctl.conf for Web and FTP Servers
The sysctl interface provides a powerful way to modify the behavior of a running Linux kernel. By editing the /etc/sysctl.conf file, you can persistently co...
win
Creating Documents Using pandoc
Pandoc is an opensource utility to create documents from markdown. We can create PDF, Doc, doc, html and other formats. And can be also used to convert html ...
windows
Windows Testing Using Kitchen Chef
Kitchen-Vagrant has the capability to spin up a windows instance for testing. To make it work you will need the vagrant-winrm to be installted on the worksta...
Package Installer for Cygwin [apt-cyg].
After a longtime I was on my windows machine and had to make it feel more like my linux machine. So install the thing what everyone else does cygwin. Surpise...
Bootstrap Windows - knife-windows - Chef
knife-windows plugin adds additional functionality to the Chef Knife CLI tool for configuring / interacting with nodes running Microsoft Windows.
Update hosts file in Windows 8
Host file contains IP followed by the FQDN which can be used to reach that IP address. Host file takes precedence over your DNS servers. In Microsoft operati...
Creating a large file in Windows / Linux
Was working on a monitoring project, need to create a large file to test notifications. Here is how we can do that.
Chef Workstation Setup on Windows Machine.
The Chef Development Kit (ChefDK) brings the best-of-breed development tools built by the awesome Chef community to your workstation with just a few clicks. ...
Fixing Not Enough Physical Memory Error in VMware Workstation 10 on Windows 8.1
Are you encountering the frustrating “Not enough physical memory is available to power this virtual machine” error in VMware Workstation 10 after a Windows u...
xml
Zabbix Hadoop Monitoring.
This script can be used to monitor Namenode Parameters. This script can be used to Generate Zabbix Import XML or Send monitoring data to Zabbix server.
xml-processing
Send KPI Data to Zabbix from XML KPI File
Send Zabbix metric by reading from a XML file. This needs package zbxsend from : https://github.com/ahmedzbyr/zbxsend
yarn
Long Running Jobs in YARN distcp.
Usually distcp and other batch jobs run for a very long time. This is fine if you are running a HADOOP environment without kerberos. When we kerberize a clus...
ERROR - java heap space caused using distcp.
When you see this error default thing to do is the set the HADOOP_CLIENT_OPTS config on your env. In My case did a typo HADDOP_CLIENT_OPTS which caused us sp...
Performance Tuning HBase and Hadoop for Optimal Performance
Getting HBase to perform well in a production environment is an iterative process that involves tweaking various configuration parameters. This post dives de...
Setting up an HBase Cluster on Hadoop YARN - A Detailed Guide for Ubuntu 12.04 LTS
HBase is a powerful open-source, non-relational, distributed database designed for handling massive datasets. Inspired by Google’s BigTable, HBase is written...
yum
YUM Repository Creation on HTTPD Web Server.
Setting up yum repos on RHEL using httpd. We will be setting up httpd and yum repo on top of it. So that we can access yum over http.
zabbix
Nagios - Service Group Summary ERROR
We were working on nagios and found that after our migration, service group summary was not working. You might get below error on the screen and the solution...
Zabbix History Table Clean Up
Zabbix history table gets really big, and if you are in a situation where you want to clean it up. Then we can do so, using the below steps.
Zabbix Database Backup Script for postgres and mysql.
After the zabbix setup is done, we would like to take backup of the database. Here we are doing a complete backup. But if we are looking for only configurati...
Installing Zabbix Version 2.4 Offline (Zabbix Server without Internet).
There might be situations where you have a remote/zabbix server which does not have internet connectivity, due to security or other reasons. So we create a ...
Zabbix Template Creation using CSV file.
In zabbix we dont have a better way to capture snmptraps. we have to manually create an item and corresponfding trigger to handle a trap arriving from the de...
Installing SNMP Builder using zabbix-extra on Zabbix Version 2.4.
SNMP Builder/Extra is an add-in for Zabbix. It provides new web interface components to browse MIB trees and values. SNMP OIDs can then be automatically conv...
Setting up SNMP Trapper for Zabbix.
Receiving SNMP traps is the opposite to querying SNMP-enabled devices. In this case the information is sent from a SNMP-enabled device and is collected or “t...
Installing zabbix-java-gateway on Centos 6.5
Zabbix 2.0 introduced native support for monitoring JMX applications through the Zabbix Java gateway. This Java-based daemon acts as an intermediary, allowin...
Zabbix Installation 2.4 - CentOS 6.5
Zabbix is the ultimate enterprise-level software designed for monitoring availability and performance of IT infrastructure components. Zabbix is open source ...
Zabbix Hadoop Monitoring.
This script can be used to monitor Namenode Parameters. This script can be used to Generate Zabbix Import XML or Send monitoring data to Zabbix server.
Fetch Monitoring Data from Item to CSV file.
This section describes how to get monitoring data from Zabbix server through the command line and export these to a CSV formatted file.
Zabbix Item Creation using OID and Name CSV files.
This script is to create a xml import file from a csv file. We use 2 csv file for this.
Send KPI Data to Zabbix from XML KPI File
Send Zabbix metric by reading from a XML file. This needs package zbxsend from : https://github.com/ahmedzbyr/zbxsend
Zabbix Import File from iReasoning Export CSV
Module used to generate Zabbix import xml for snmp traps, Below are steps to create a Zabbix Import for Traps. These traps are captured from the snmptrap fun...
zabbix-java-gateway
Installing zabbix-java-gateway on Centos 6.5
Zabbix 2.0 introduced native support for monitoring JMX applications through the Zabbix Java gateway. This Java-based daemon acts as an intermediary, allowin...
zbxsend
Send KPI Data to Zabbix from XML KPI File
Send Zabbix metric by reading from a XML file. This needs package zbxsend from : https://github.com/ahmedzbyr/zbxsend
zookeeper
Cloudera Manager - Duplicate entry ‘zookeeper’ for key ‘NAME’.
Deployment error during setting zookeeper on a new cluster using Cloudera Manager UI, when an earlier cluster containing zookeeper earlier was built using Cl...
Setting Up HDFS Services Using Cloudera API [Part 3]
This is the second follow up post. In the earlier post
Setting Up Zookeeper Services Using Cloudera API [Part 2]
This is the second follow up post. In the earlier post Setting Up Cloudera Manager Services Using Cloudera API [Part 1] we install the cloudera management se...
Ansible Playbook - Setup Zookeeper Using tarball.
This is a simple zookeeper playbook, to quickly start zookeeper running on a single or more nodes, in a clustered mode.
Ansible Playbook - Setup Storm Cluster.
This is a simple Storm Cluster Setup. We are using a dedicated Zookeeper Cluster/Node, instead of the standalone zkserver. Below is how we will deploy our c...
Ansible Playbook - Setup Kafka Cluster.
This is a simple Kafka setup. In this setup we are running kafka over a dedicated zookeeper service. (NOT the standalone zookeeper which comes with kafka)