Sunday 28 April 2013

Setup Load-Balancing Cluster with LVS and Piranha on Centos 6

This setup guide is about building web cluster with a pair of Linux loadbalancers as frontend. It uses Linux Virtual Server(LVS) and Piranha. Piranha is a web-based gui installed in LVS Routers primarily to generating a valid /etc/lvs.cf file. Start by installing LVS on LVS Router.
[root@lvsrouter ~]# yum groupinstall "Load Balancer"
================================================================================
 Package         Arch        Version                         Repository    Size
================================================================================
Installing:
 ipvsadm         x86_64      1.25-10.el6                     base          41 k
 piranha         x86_64      0.8.6-2.el6_4.1                 updates      623 k
Installing for dependencies:
 libedit         x86_64      2.11-4.20080712cvs.1.el6        base          74 k
 libnl           x86_64      1.1-14.el6                      base         121 k
 php             x86_64      5.3.3-22.el6                    base         1.1 M
 php-cli         x86_64      5.3.3-22.el6                    base         2.2 M
 php-common      x86_64      5.3.3-22.el6                    base         524 k

Transaction Summary
================================================================================
Install       7 Package(s)

Total download size: 4.7 M
Installed size: 18 M
Is this ok [y/N]: y
Start piranha and pulse services on reboot.
chkconfig piranha-gui on
chkconfig pulse on
Set a password for piranha web
/usr/sbin/piranha-passwd
Allow ports in iptables
vi /etc/sysconfig/iptables
-A INPUT -m state --state NEW -m tcp -p tcp --dport 3636 -j ACCEPT
Edit piranha config
vi  /etc/sysconfig/ha/conf/httpd.conf
Start the piranha gui service
service piranha-gui start
For LVS router to forward network packets properly to real servers, each LVS router node must have IP forwarding turned on. Turn on packet forwarding by editing sysctl.conf:
vi /etc/sysctl.conf
net.ipv4.ip_forward = 1

Reload sysctl
sysctl -p /etc/sysctl.conf
Start http services on the Real Servers
service httpd start
On the LVS Router, log in to the Piranha web ui to begin configuration. (eg http://(loadbalancer-ip):3636). Using "piranha" as user name, and key in the newly setup password. Enter Primary Router's IP address in "Global Settings"->"Primary server public IP:".
CENTOS Piranha Global Settings
Enter Redundant Router's IP address in "Redundancy"->"Redundant server public IP:".
CENTOS Piranha Redundancy
Enter Virtual Server settings in "Virtual Servers"->"Virtual Server".
CENTOS Piranha Virtual Servers
Include Real Server settings in "Virtual Servers"->"Real Server".
CENTOS Piranha Real Server
On each time the Piranha Gui was changed, have to sync the settings to Backup Routers, and restart the pulse service on both routers.
service pulse restart
To see the virtual server's statistics, use "watch ipvsadm".
[root@lvsrouter ~]# watch ipvsadm 
Create arptables entry for each Virtual IP address on each Real Server(eg Webserver). You can add below command to /etc/rc.local to start on every reboot. If the network adaptor on Real Server is eth0, refer to below, if not change as according:
ip addr add (virtual ip) dev eth0:1
Direct Routing with arptables_jf. To configure each real server to ignore ARP requests for each of the virtual IP addresses the Piranha cluster services:
yum install arptables_jf
arptables -A IN -d (virtual_ip) -j DROP
arptables -A OUT -d (virtual_ip) -j mangle --mangle-ip-s (real_ip)
chkconfig arptables_jf on
service arptables_jf save
service arptables_jf restart
Create a loopback on each Real Server for monitoring Virtual IP.
vi /etc/sysconfig/network-scripts/ifcfg-lo:0

DEVICE=lo:0
IPADDR=(Virtual IP)
NETMASK=255.255.255.255
NETWORK=192.168.0.0
ONBOOT=yes
NAME=loopback

16 comments :

  1. Hello. First of all, thank you very much for tutorial! After I start pulse with 'pulse start', it crashes in several seconds.
    $ tail -f /var/log/message says the following:
    Dec 21 02:30:38 server3 pulse[4425]: STARTING PULSE AS MASTER
    Dec 21 02:30:56 server3 pulse[4425]: partner dead: activating lvs
    Dec 21 02:30:56 server3 pulse[4425]: Failed to open semaphore: Permission denied
    What kind of permissions pulse refers to?
    It’s interesting to note that “partner dead: activating lvs” message persists when I disasble redundancy in Piranha.

    ReplyDelete
  2. it is cause your selinux bro

    ReplyDelete
  3. Will the Virtual IP go on eth0:1 on LVS Router Machine or the will it go on the real server.. Please explain. I have 4 machines altogether 2 of which are my webserver and the other two my router..

    I understand I have to create a virtual IP on eth0:1 device but on which oth the machine with I configure the eth0:1

    ReplyDelete
    Replies
    1. Do we need to configure virtual interface on LB hosts also? apart from web?

      Delete
    2. Virtual IP is setup at the LB webgui. If all else fail, try rebooting webserver, and maybe LB.

      Delete
  4. May 28 01:07:46 server nanny[86525]: CONNECT to 66.45.250.91:80 timed out how can i fix this?

    ReplyDelete
  5. Do we need to configure virtual IP on LB hosts also?

    ReplyDelete
  6. Thanks a lot very much for the high quality and results-oriented help. I won’t think twice to endorse your blog post to anybody who wants and needs support about this area. data science training in Hyderabad

    ReplyDelete
  7. Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. data scientist courses

    ReplyDelete
  8. very well explained. I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
    Logistic Regression explained
    Correlation vs Covariance
    Simple Linear Regression
    KNN Algorithm

    ReplyDelete
  9. Through this post, I know that your good knowledge in playing with all the pieces was very helpful. I notify that this is the first place where I find issues I've been searching for. You have a clever yet attractive way of writing.
    machine learning courses in bangalore

    ReplyDelete
  10. Thanks for posting the best information and the blog is very good.,data science training in ranchi

    ReplyDelete

search iomeweekly