Just hit this issue, although – not entirely un-triggered by myself.
My situation started with a vSphere HA of a v5.1.0 host and a v5.5 host. I removed the host from the vSphere not by feature but by function of the halt command
After powering off the host, reinstalling the OS drive with v5.5, rebuilding the RAID array and restoring to the network – I noticed I had a MAC collision taking place between the two IPs associated with the two hosts above.
Host #1 – 172.16.0.21 – was the v5.1.0 host (now upgraded).
Host #2 – 172.16.0.23 – is the v5.5 host.
The network was seeing a broadcast response for both IPs with the same MAC address. This was the MAC address of host #1! Not only did this confuse the hell out of the network causing disruption for anything management, it also confused the hell out of me!
Wasn’t until I started looking into the vSphere setup that I realised it does ARP takeover/spoofing in order to service requests from the network.
Quick look at the ‘esxcfg-vmknic -l’ showed my offending MAC address. Running ‘esxcfg-advcfg -s 1 /Net/FollowHardwareMac’ and rebooting fixes the issue and I’m back to one MAC per host.
Not exactly the same issue as yours but related none the less. Maybe this comes in useful for anyone else having a similar amount of fun
This was my comment on http://www.vhersey.com/2012/08/cloned-esxi-duplicate-vmk0-mac-addresses/#comment-87085 after resolving the issue on my system.