How to migrate vSphere Management vmk ports from a ghosted dvSwitch


In my haste to rebuild a lab environment,  I ungracefully disconnected a host whose management vmknic was connected to a dvSwitch.   The plan was to add the host to a new vCenter….  a new vCenter that did not know of the previous dvSwitch associated with the host's management vmknic,  not smart on my part!

Up until this point my mistake was still not apparent to me.   I was able to add the host to the new vCenter without issue.  However,  when I went into the vSphere Client to view the host's network configuration I noticed something odd:  no vSwitches!   I clicked on the vSphere Distributed Switch button (below image) and saw that the host still was referencing the DWsitch1 dvSwitch.   I thought, OK not a problem I could still use the Manage Physical Adapters link to migrate a vmnic to a local vSwitch right?  Nope.  Since the DSwitch1 dvSwitch was recognized as a dvSwitch,  but not managed by the new vCenter,  I wasn't permitted to move either of its NICs to a local vSwitch.  

Since I could not migrate the physical NICs associated with DSwitch1 to vSwitch0 from the GUI,  I had to figure out how to do it from the command-line.  The ESXCLI command is the preferred on host CLI utility allowing you to modify MOST ESXi configuration settings.  However,  ESXCLI still does not allow you to modify dvSwitches.   While ESXCLI doesn't allow you to modify DV Switches,  it does allow you to list the DV Switch configuration.  I executed the following command to view the DSwitch1 configuration:
esxcli network vswitch dvs vmware list



Now that I could see how the orphaned DSwitch1's configuration, I needed to figure out if there was a way to disassociate the vmnic's,  one at a time, from DSwitch1 so that I could add them to vSwitch0 without losing my management connection to the host.

I Googled on how to remove a vmnic from a dvSwitch and found this Vmware KB:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1008127

I executed the following esxcfg-vswitch command to remove one of the redundant vmnics from DSwitch1:
esxcfg-vswitch -Q vmnic0 -V 16 DSwitch1
(NOTE: 16 is the Port ID gathered using the ESXCLI command)

Once I did this I went into the vSphere Client and created vSwitch0.  Since vmnic0 was successfully removed from DSwitch1,  I was able to add it to vSwitch0 from the vSphere Client GUI.

Now that I had a vSwitch with a vmnic on it that contained the vSphere Management VLAN on it,  I still needed to migrate the vmkernel port to vSwitch0.   From Host -> Configuration -> Networking I clicked on vSphere Distributed Switch and then choose Manage Virtual Adapters.  I clicked on vmk0 and chose Migrate -> vSwitch0 and saved the changes.   Vmk0 migrated to vSwitch0 without any downtime.  To remove the 2nd physical NIC from the orphaned DSwitch1 I then executed:
esxcfg-vswitch -Q vmnic1 -V 17 DSwitch1

Once the 2nd NIC was removed I was able to add it to vSwitch0 for redundancy.

Crisis averted.

Labels: , , , , , , , , , , ,