CARP oddity

[Stephan Tesch <stephan_(_at_)_tesch_(_dot_)_cx>]

Hi,

I tried to setup a simple CARP cluster using two old PCs. They share two 
private subnets (192.168.10.0/24 and 192.168.30.0/24). One machine is 
configured with a default advskew (0), the other has an advskew of 100.

If I boot both nodes, they behave very nicely and one becomes master 
(advskew=0), the other one is backup (advskew=100). That is what one would 
expect from them. 

As soon as I issue a 'sh /etc/netstart' on the backup node, things are 
getting ugly. Sometimes I get this behaviour:

# sh /etc/netstart
Feb  5 17:29:00 faramir /bsd: Data modified on freelist: word 3 of object 0xd08f5dc0 size 0x40 previous type UVM amap (0xdeadbeee != 0xdeadbeef)
Feb  5 17:29:00 faramir /bsd: Data modified on freelist: word 3 of object 0xd08f5dc0 size 0x40 previous type UVM amap (0xdeadbeee != 0xdeadbeef)

Both nodes assume they are master, even though one of them has a higher 
advskew. Both send out carp advertisements:

15:58:08.003079 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 34450)
15:58:09.013079 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 60693)
15:58:09.554305 arp who-has 192.168.30.3 tell 192.168.30.3
15:58:10.023076 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 65437)
15:58:11.033079 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 60068)
15:58:12.043078 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 53588)
15:58:13.053078 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 37036)
15:58:14.063079 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 33829)
15:58:14.168367 carp 192.168.30.3 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=240 (DF) [tos 0x10] (ttl 255, id 45711)
15:58:14.168430 arp who-has 192.168.30.1 tell 192.168.30.1 (0:0:5e:0:1:1)
15:58:15.073077 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 60419)
15:58:15.577576 carp 192.168.30.3 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=240 (DF) [tos 0x10] (ttl 255, id 57012)
15:58:16.083079 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 55772)
15:58:17.093079 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 54002)
15:58:17.526505 carp 192.168.30.3 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=240 (DF) [tos 0x10] (ttl 255, id 60581)
15:58:18.103077 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 47312)
15:58:19.113094 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 52638)
15:58:19.475426 carp 192.168.30.3 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=240 (DF) [tos 0x10] (ttl 255, id 46159)
15:58:20.123110 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 60655)
15:58:21.133080 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 40992)
15:58:21.424350 carp 192.168.30.3 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=100 (DF) [tos 0x10] (ttl 255, id 48554)
15:58:22.143079 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 61169)
15:58:22.833572 carp 192.168.30.3 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=100 (DF) [tos 0x10] (ttl 255, id 39388)
15:58:23.153080 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 33405)
15:58:24.163079 carp 192.168.30.2 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=0 (DF) [tos 0x10] (ttl 255, id 59246)
15:58:24.242781 carp 192.168.30.3 > 224.0.0.18: CARPv2-advertise 36: vhid=1 advbase=1 advskew=100 (DF) [tos 0x10] (ttl 255, id 48801)

Is this issue known? Am I just too stupid to configure something as easy as
 carp? I guess this has nothing to do with the bug Steven Marley pointed out a 
couple of days ago...

Here's the config for both nodes:

primary node:

/etc/hostname.fxp1:
  inet 192.168.30.2 255.255.255.0 192.168.30.255
/etc/hostname.carp1:
  inet 192.168.30.1 255.255.255.0 192.168.30.255 vhid 2 pass bar

secondary node:

/etc/hostname.fxp1:
  inet 192.168.30.3 255.255.255.0 192.168.30.255
/etc/hostname.carp1:
  inet 192.168.30.1 255.255.255.0 192.168.30.255 vhid 2 advskew 100 pass bar

Preemption is enabled for both nodes. PF is disabled. The PCs are both running 
3.6-RELEASE.

-- 
PGP key: http://www.tesch.cx/stephan.asc
Fingerprint: 9CF9 0D64 2957 B44D A0C8
             35FE 0382 AE49 DFAB 9CAF