How to fix Slow File Systems on VMs running under ESX

Author: NetworkAdminKB.com
Created: 2009-11-09
Modified: 2011-03-07

Issue:

Accessing files in a Virtual Machine is slow.  This can happen on any Guest OS, but here are Windows OS examples include

 

“Slow” is approximately 3 seconds or longer or compared to physical equivalents

Changing directories in Windows Explorer is slow

Opening files is slow

Saving files is slow

 

Cause:

Because VMWare ESX hosts many guest OS’s it should be considered a high performance OS and as such will need to have its disk IO path settings increased to allow better performance of the Disk Subsystem to the Guest OS.

 

By default, the disk IO path is not optimized as a high performance disk subsystem.  You will need to increase the allowed IO queue settings in the IO path to the disk.  The IO path consists of the following components each with their own settings.  Ideally these IO queue settings should all be the same.

 

HBA Settings ßà Driver Settings ßà ESX OS Settings

 

Solution:

QLogic HBA Settings

Press CTRL-Q during the HBA boot and configure the Advanced Adapter Setting of Execution Throttle to be between 64 and 256.  The default value is 16.

 

 

QLogic Driver Settings in ESX 3.5

1)      Go to the ESX console

2)      Determine which QLogic HBA module is currently loaded:

vmkload_mod -l | grep qla2300

 

                                                               i.      qla2300_707 (ESX Server 3.0.x)

                                                             ii.      qla2300_707_vmw (ESX Server 3.5)

                                                            iii.      qla2xxx.o (ESXi Server 4.x)

3)      Determine current HBA settings

esxcfg-module -g qla2300_707_vmw

 

a.       Example output

qla2300_707_vmw enabled = 1 options = 'qlport_down_retry=60'

b.      Any options currently set should be maintained

4)      Configure the QLogic Driver ql2xmaxqdepth option, and any previously set options.

a.       Configure the ql2xmaxqdepth to be between 64 and 256.

b.      To configure multiple options use quotes and separate options with spaces

c.       Example:

esxcfg-module -s "qlport_down_retry=60 ql2xmaxqdepth=64" qla2300_707_vmw

 

ESX/EXi 3.5 / 4.x Settings

1)      Open the VI Client

2)      Go to the Configuration tab of the ESX server

a.       Click Advance Settings

                                                               i.      Click Disk

                                                             ii.      Configure the Disk.SchedNumReqOutstanding to be between 64 and 256

3)      Reboot Server

 

More Information:

According to ESX 3.5 documentation settings above 64 (anywhere in the IO path) have very little performance benefit because of SCSI parallelism.  I have not validated or tested this in ESX.  If true this appears to be a limit of the ESX implementation of SCSI.

 

Various other factors can affect disk performance.  Speed of drives and the bandwidth available to access the drives are other key factors to take into consideration when trying to increase disk performance.

 

Performance Tuning the Execution Throttle of a QLogic HBA

Article ID: 343, Created On: 9/19/2011, Modified: 9/19/2011