AHMED ZBYR

AHMED ZBYR

Biker, EV Enthusiast

In today’s digital landscape, ensuring high availability for your web applications is paramount. Downtime can lead to lost revenue, damaged reputation, and frustrated users. This guide details how to set up a High Availability (HA) Nginx web server using heartbeat on CentOS/RHEL 6. Heartbeat acts as a cluster resource manager, monitoring the health of your primary server and automatically failing over to a secondary server in case of failure. NGINX will be configured as a highly available web server.

Heartbeat is a daemon providing cluster infrastructure services, enabling nodes to communicate and be aware of each other’s status. NGINX is a powerful and versatile open-source HTTP server, reverse proxy, and load balancer, known for its performance and stability. Together, they provide a robust foundation for a highly available web application.

Why Heartbeat and Nginx for HA?

  • Automated Failover: Heartbeat automatically detects failures and switches to the backup server, minimizing downtime.
  • High Performance: Nginx is known for its efficient handling of web traffic, ensuring optimal performance even during failover.
  • Cost-Effective: Both Heartbeat and Nginx are open-source, reducing licensing costs.
  • Relatively Simple Setup: While requiring careful configuration, the setup is straightforward compared to more complex clustering solutions.

Step 1: Pre-Configuration Requirements

Before you begin, ensure you have two servers running CentOS/RHEL 6. Assign hostnames and IP addresses as follows:

  • Primary Node: nginx-primary - IP address: 192.168.230.138 (assigned to eth0)
  • Secondary Node: nginx-slave - IP address: 192.168.230.139 (assigned to eth0)
  • Virtual IP Address: 192.168.230.150 - This IP will be assigned to the active server. Clients will access the web server through this IP.

Configure your /etc/hosts file on both servers:

[root@nginx-primary ~]# cat /etc/hosts
127.0.0.1   localhost
192.168.230.138         nginx-primary
192.168.230.139         nginx-slave

Verify the hostnames:

Primary Server: nginx-primary

[root@nginx-primary ~]# uname -n
nginx-primary

Secondary Server: nginx-slave

[root@nginx-slave ~]# uname -n
nginx-slave

Important: Ensure Nginx is installed and configured on both nginx-primary and nginx-slave servers before proceeding with the Heartbeat installation. A basic Nginx configuration is sufficient for this guide. You can test the initial Nginx setup by accessing each server’s IP address directly in your browser.

Now, let’s install Heartbeat on both servers.

Step 2: Install EPEL Repository and Heartbeat

First, install the EPEL (Extra Packages for Enterprise Linux) repository, which contains the Heartbeat package.

[root@nginx-primary ~]# wget http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
[root@nginx-primary ~]# rpm -ivh epel-release-6-8.noarch.rpm

Troubleshooting: If you encounter errors during the yum install heartbeat process, you might need to change https to http in the /etc/yum.repos.d/epel.repo file.

[root@nginx-primary ~]# vim /etc/yum.repos.d/epel.repo

Then, install Heartbeat:

[root@nginx-primary ~]# yum install heartbeat

Repeat these steps on nginx-slave.

Step 3: Copy Heartbeat Configuration Files

Copy the sample configuration files to the /etc/ha.d/ directory:

[root@nginx-primary ~]# cp /usr/share/doc/heartbeat-3.0.4/authkeys /etc/ha.d/
[root@nginx-primary ~]# cp /usr/share/doc/heartbeat-3.0.4/ha.cf /etc/ha.d/
[root@nginx-primary ~]# cp /usr/share/doc/heartbeat-3.0.4/haresources /etc/ha.d/

The heartbeat-3.0.4 directory might differ based on your Heartbeat version. Adjust the command accordingly.

Step 4: Configure Authentication (authkeys)

Edit the authkeys file to configure authentication between the Heartbeat nodes:

[root@nginx-primary ~]# vi /etc/ha.d/authkeys

Add the following lines:

auth 2
2 sha1 anyRandomKeyHere

Replace anyRandomKeyHere with a strong, randomly generated key. This key must be identical on both servers.

Step 5: Secure the Authentication Key File

Restrict access to the authkeys file for security reasons:

[root@nginx-primary ~]# chmod 600 /etc/ha.d/authkeys

Step 6: Configure Heartbeat (ha.cf)

The ha.cf file is the central configuration file for Heartbeat. Edit it using vi:

[root@nginx-primary ~]# vi /etc/ha.d/ha.cf

Add the following lines to the ha.cf file. These settings are crucial for proper Heartbeat operation. Adjust eth0 if your network interface is different.

logfile /var/log/ha-log
logfacility local0
keepalive 2
deadtime 30
initdead 120
bcast eth0
udpport 694
auto_failback on
node nginx-primary
node nginx-slave

Explanation of settings:

  • logfile: Specifies the log file for Heartbeat messages.
  • logfacility: Specifies the syslog facility to use for logging.
  • keepalive: Sets the interval (in seconds) between heartbeat signals.
  • deadtime: Specifies the time (in seconds) after which a node is considered dead if no heartbeat signals are received.
  • initdead: Sets the initial dead time (in seconds) at startup.
  • bcast: Specifies the network interface for broadcasting heartbeat signals.
  • udpport: Specifies the UDP port used for heartbeat communication.
  • auto_failback: Enables automatic failback to the primary node when it recovers.
  • node: Lists the names of the nodes in the cluster.

Step 7: Configure Resource Management (haresources)

The haresources file defines the resources that Heartbeat will manage and the order in which they will be started. Edit the file:

vi /etc/ha.d/haresources

Add the following line:

nginx-primary 192.168.230.150 nginx

Explanation:

  • nginx-primary: The primary node where the resources will initially run.
  • 192.168.230.150: The virtual IP address that will be assigned to the active server.
  • nginx: The name of the service (in this case, the Nginx web server). Heartbeat will use the nginx script in /etc/init.d/ to start and stop the Nginx service.

Step 8: Copy Configuration to the Secondary Node

Copy the entire /etc/ha.d/ directory from nginx-primary to nginx-slave:

scp -r /etc/ha.d/ root@nginx-slave:/etc/

Ensure you can SSH to nginx-slave as root without a password prompt. You might need to configure SSH key-based authentication.

Step 9: Configure Nginx to Listen on the Virtual IP

Edit the Nginx configuration file on both servers to listen on the virtual IP address:

vi /etc/nginx/nginx.conf

Within the http block, add or modify the listen directive in the server block to include the virtual IP:

http {
    server {
        listen 192.168.230.150:80; # Listen on the virtual IP address
        server_name yourdomain.com; # Replace with your domain name

        # ... other Nginx configuration ...
    }
}

Replace yourdomain.com with your actual domain name. If you don’t have a domain name, you can use the virtual IP address in your browser.

Step 10: Copy the Nginx Configuration

Copy the modified Nginx configuration file from nginx-primary to nginx-slave:

scp /etc/nginx/nginx.conf root@nginx-slave:/etc/nginx/

Step 11: Start Heartbeat

Start the Heartbeat service on both nginx-primary and nginx-slave:

/etc/init.d/heartbeat start

Check the Heartbeat log file (/var/log/ha-log) for any errors. It may take a few moments for Heartbeat to establish communication and assign the virtual IP.

Step 12: Test the Setup

Open a web browser and navigate to the virtual IP address:

http://192.168.230.150

You should see your Nginx web page. This confirms that the virtual IP has been successfully assigned to the primary server.

Step 13: Simulate Failover

To test the failover mechanism, stop the Heartbeat daemon on nginx-primary:

/etc/init.d/heartbeat stop

Wait for a few seconds (longer than the deadtime configured in ha.cf). Then, refresh your browser pointing to the virtual IP address (http://192.168.230.150). You should still see your Nginx web page, indicating that the virtual IP and Nginx service have been automatically taken over by nginx-slave.

Important Notes:

  • Virtual IP Assignment: Heartbeat automatically manages the virtual IP address. You do not need to manually configure a virtual network interface.
  • Firewall (iptables): Firewall rules can interfere with Heartbeat communication. For testing purposes, you may temporarily disable iptables (service iptables stop). For a production environment, configure iptables to allow UDP traffic on port 694 (Heartbeat’s default port) between the cluster nodes, and allow TCP traffic on port 80 to both nodes. The example iptables rules provided in the original extract are a good starting point. Remember to save the iptables rules after adding them (service iptables save).

Troubleshooting and Common Issues

  • Heartbeat Not Starting: Check the /var/log/ha-log file for errors. Common causes include incorrect configuration files, authentication problems, or firewall restrictions.
  • Virtual IP Not Assigned: Ensure the ha.cf and haresources files are correctly configured and identical on both nodes. Verify that Heartbeat is running on both servers and that they can communicate with each other.
  • Failover Not Occurring: Check the deadtime setting in ha.cf. Make sure the secondary node waits long enough before assuming the primary node is down. Verify firewall rules are not blocking Heartbeat communication.

Advanced Configuration

  • STONITH (Shoot The Other Node In The Head): For more robust failover, consider implementing STONITH, which physically disables the failed node to prevent data corruption. This typically involves using a power switch or other hardware-based mechanism.
  • Load Balancing: While this guide focuses on failover, Nginx can also be configured as a load balancer in front of multiple backend servers for increased performance and scalability.
  • Monitoring: Implement monitoring tools to track the health of your Heartbeat cluster and Nginx servers.

Conclusion

This guide provides a comprehensive overview of setting up a High Availability Nginx web server using Heartbeat. By following these steps, you can significantly improve the reliability and availability of your web applications. Remember to thoroughly test your setup and adapt the configuration to your specific environment.

  • https://wiki.archlinux.org/index.php/Simple_IP_Failover_with_Heartbeat
  • http://www.linuxnix.com/2010/01/heartbeat-clustering.html
  • https://www.howtoforge.com/high_availability_heartbeat_centos
  • http://blog.iwayvietnam.com/tuanta/2010/05/19/configuring-a-high-availability-cluster-on-rhel-centos/
  • http://www.howtogeek.com/177621/the-beginners-guide-to-iptables-the-linux-firewall/
  • http://www.cyberciti.biz/tips/linux-cluster-building-firewall-rules.html
  • http://www.linux-ha.org/doc/users-guide/_creating_an_initial_heartbeat_configuration.html

You May Also Enjoy