Poor network performance of VMware ESX Server 3i on a gigabit link

I had a very strange experience with VMware ESX Server 3i. It’s basically a virtual machine container which installs pretty much as an OS on a given hardware and let’s you run VMs inside. It’s a free product (registration needed) from VMware, so I decided to give it a shot.

I installed in on whatever I had at hand, which happened to be a Lenovo X60 convertible tablet. I know that this is not an obvious choice for a test server, but It has Core Duo 1.6GHz, 2GB of RAM, 80GB 7200 RPM SATA drive and a Gigabit Ethernet NIC, and I wasn’t using it at the moment.

The connection to the graphical VM consoles seemed very laggy and the overall experience was not very good. I checked whether any VMs were intensively swapping, but they weren’t. The CPU utilization was also modest. Pings were really strange:

64 bytes from icmp_seq=1 ttl=64 time=334 ms
64 bytes from icmp_seq=2 ttl=64 time=1.16 ms
64 bytes from icmp_seq=3 ttl=64 time=1005 ms
64 bytes from icmp_seq=4 ttl=64 time=41.8 ms
64 bytes from icmp_seq=5 ttl=64 time=299 ms

I stopped all the VMs but nothing changed. I rebooted the whole box, same thing. Then after a while I was enlightened and connected the X60 to a 100Mbps switch instead of the 1Gbps one, and surprise surprise:

64 bytes from icmp_seq=1 ttl=64 time=1.04 ms
64 bytes from icmp_seq=2 ttl=64 time=0.235 ms
64 bytes from icmp_seq=3 ttl=64 time=0.229 ms
64 bytes from icmp_seq=4 ttl=64 time=0.278 ms

I took the setup down to the office, and same thing happened with 1Gbps HP switch. However, there was no problem with the gigabit link when I had ubuntu installed on the device, so I would blame it on the drivers contained in the ESXi install.

VMware is not recommending the ESX 3i version for new setups, since it only runs on 32-bits. For now I gave up deeper investigation of the problem. With this problem down, I’m pretty satisfied with the performance and ease of setup. Perhaps this will help someone with a similar symptoms.