VMware and Windows poor TCP performance

I have had a strange issue lately with Windows servers on VMware vSphere 5.1 hosts. Throughput of TCP connections between some virtual machines were very very slow, barely 10 mbit/s .

The behavior was easily reproducible : just start an iperf connection between a Windows Server 2008 and a Windows 2012 server, and you get 10 mbit/s .

Moving both virtual machines on the same ESXi host, or on different hosts, did not change anything.
Changing from E1000 to vmxnet3 or vice-versa did not make any difference.
Disabling some or all network offloading (Large Send Offload, TCP Checksum Offloading, etc.) did not make any difference either.

Out of despair, I tried to add the “-N” to my iperf command. That flag disables Nagle’s algorithm.
Instantly, the throughput went from 10 mbit/s to over 800 mbit/s .

Finally, after a bit of Googling, I ended up disabling Nagle’s algorithm and delayed TCP Acknowledgment in Windows registry. I will continue to monitor the behavior of the servers, but it fixed the throughput issues and latency is not affected.

You can do this by editing your registry HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces and, in each interface folder, add the two following 32bit D-WORD:
Name:TCPNoDelay Value:1
Name:TcpAckFrequency Value:1

This entry was posted in Computer, Microsoft, Virtualization, VMware. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *