Fixing Emulex 10GbE on ESXi 5.5

20150715 - Man downLast year at VMworld in San Francisco I bought two Emulex OCe10102 10GbE cards for my home lab, and used two Dell/Force10 cables to connect them together.  One card went into my Dell PowerEdge T710, and one in my HP Microserver Gen7.

However, things did not start well.  I downloaded the driver and firmware from Emulex, and then flashed each card.  I installed the Emulex driver on my ESXi host, but for some reason it continued to run dog slow.  Like 10mb files taking an hour to copy across for example.

I changed the OS on the NAS from FreeNAS to Windows Server 2012 R2, and then to Windows Server 2016 Technical Preview 2.  Performance was still terrible.

In the end I decided to take a look at the ESXi box.  Using:

esxcli network nic list

I could see that the cards were in half-duplex. I tried to force them into full-duplex using ethtool but that didn’t work.

After seeing lots of errors in vmkernel.log, I concluded the driver must be the issue. Using:

esxcli software vib list

I could see that I’d previous installed two Emulex drivers, elxnet and lpfc. It was time for these to go.

After shutting down the entire lab, I put the host into maintenance mode:

vim-cmd hostsvc/maintenance_mode_enter

I then removed both drivers:

esxcli software vib remove --vibname elxnet
esxcli software vib remove --vibname lpfc

And then rebooted the host. When it came back up, I listed the network adapters again, and this time saw that the be2net driver had kicked in:

20150826 - 1
Everything was fixed. A simple copy of a 1GB ISO took forty seconds using the datastore browser. Still not blistering, but that is down to the NAS (slow disks and a Celeron processor).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s