FreeBSD & Carp – failover and load balancing

FreeBSD supports CARP (Common Address Redundancy Protocol) and has done for many years now. We’ve been using it here at Gconnect as part of our ‘Mailout’ system with no issues at all and it’s served use well since FreeBSD version 6.’something’. As part of the Mailout system upgrade i’ll be using CARP again, and heres how to make it work..

Firstly the requirements:

1. The two servers should load balance
2. A failure of one server should not be noticed by the users
3. When a failed server recovers, load balancing should resume

Our plan is to create 2 servers, both with a normal ip address on the interface. These server will share 2 more ‘virtual ip’ addresses using carp. Each server will be the ‘master’ for one of the virtual ip addresses and the ‘backup’ for the other. Our DNS record will have both ip addresses and will do the basic round robin load balancing. When a server fails, the remaining server will become the ‘master’ for both of the virtual ip addresses. When the server recovers, the system will return to normal. For this system to work we need a method of forcing one server to become the master and another to become the backup. To do this we use the ‘advskew’ setting, which is like a weighting. Then we need to enable ‘pre-emtion’ to ensure that once a server is master for a virtual ip address, it will give it up again if another server which a higher advskew value joins the group.

The initial task is to ensure that the kernel has carp enabled, your kernel file needs to have:

device     carp

If not, add it, rebuild the kernel and off you go. Now requirement number 3 was that upon recovery the servers should resume load balancing. Pre-emtion requires an explicit setting via sysctl. You can get a list of carp based values like this:

# sysctl net.inet.carp 
net.inet.carp.allow: 1
net.inet.carp.preempt: 0
net.inet.carp.log: 1
net.inet.carp.arpbalance: 0
net.inet.carp.suppress_preempt: 0

We need to make ‘net.inet.carp.preempt’ = 1, to do that, type in:

# sysctl net.inet.carp.preempt=1
net.inet.carp.preempt: 0 -> 1

To make the change permanent, i.e. happen when the system boots, add the following line to /etc/sysctl.conf

net.inet.carp.preempt=1

Now we can put the config into /etc/rc.conf (server A):

cloned_interfaces="carp0 carp1"
ifconfig_carp0="vhid 1 pass my_password1 172.16.2.9 255.255.255.0"
ifconfig_carp1="vhid 2 advskew 100 pass my_password2 172.16.2.10 255.255.255.0"

And in server B

cloned_interfaces="carp0 carp1"
ifconfig_carp0="vhid 1 advskew pass my_password1 172.16.2.9 255.255.255.0"
ifconfig_carp1="vhid 2 pass my_password2 172.16.2.10 255.255.255.0"

Notice the subtle difference? The advskew value is a mirror of the first server. After a reboot we can see the CARP by typing ‘ifconfig’

...
carp0: flags=49 metric 0 mtu 1500
	inet 172.16.2.9 netmask 0xffff0000 
	carp: MASTER vhid 1 advbase 1 advskew 0
carp1: flags=49 metric 0 mtu 1500
	inet 172.16.2.10 netmask 0xffff0000 
	carp: BACKUP vhid 2 advbase 1 advskew 100

Now if you reboot a server you will see that the reaming server will become master for both carp0 and carp1 and when its finish the situation returns as we wanted!