IPMP with "local-mac-address?=false"

Network interfaces in an IPMP group must have unique MAC addresses. What happens if network interfaces in an IPMP group do not have unique MAC addresses, and network interfaces failover and failback?

This happened on one of our Sun V890 systems running Solaris 10. The system has two gigabit NICs in an active/standby IPMP configuration.

When the primary interface failed over to the secondary interface, and then failed back to the primary interface, we experienced what appeared to be a corrupt ARP cache, causing significant network problems.

Nov 14 17:24:37 host in.mpathd[131]: [ID 215189 daemon.error] The link has gone down on ce0
Nov 14 17:24:37 host in.mpathd[131]: [ID 594170 daemon.error] NIC failure detected on ce0 of group ce
Nov 14 17:24:37 host in.mpathd[131]: [ID 832587 daemon.error] Successfully failed over from NIC ce0 to NIC ce1
Nov 14 17:24:41 host in.mpathd[131]: [ID 820239 daemon.error] The link has come up on ce0
Nov 14 17:24:41 host in.mpathd[131]: [ID 299542 daemon.error] NIC repair detected on ce0 of group ce
Nov 14 17:24:41 host in.mpathd[131]: [ID 620804 daemon.error] Successfully failed back to NIC ce0

# ifconfig ce0
ce0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet 10.205.0.104 netmask fffff000 broadcast 10.205.15.255
        groupname ce
        ether 0:3:ba:db:cf:15

# ifconfig ce1
ce1: flags=69000842<BROADCAST,RUNNING,MULTICAST,IPv4,NOFAILOVER,STANDBY,INACTIVE> mtu 0 index 3
        inet 0.0.0.0 netmask 0
        groupname ce
        ether 0:3:ba:db:cf:15

At this point, arp -a would hang, the ssh service was unreachable remotely, and the LDAP client binding was lost.

The solution was to run eeprom 'local-mac-address?=true' and reboot the system.

Back to brandonhutchinson.com.
Last modified: 2006/11/15