scsi_cable

Introduction

Hopefully you have read my previous blog posts on pvSCSI.  It describes what the driver is, how it works, and how it can positively impact your performance and workloads.  Part two covers the process of installing the pvSCSI driver on an existing system and a new system.  Both can be found here on the site and you might find them useful:

http://www.virtualinsanity.com/index.php/2009/11/21/more-bang-for-your-buck-with-pvscsi-part-1/

http://www.virtualinsanity.com/index.php/2009/12/01/more-bang-for-your-buck-with-pvscsi-part-2/

Interrupt Coalescing

VMware recently published a KB article that answers a question that has been floating around the community for a while.  The pvSCSI driver sounds superior to the LSI driver with direct I/O access to the hypervisor so why not use it in all cases?  The article states that you should only use the newer driver when driving higher workloads, those that are typically 2000 IOPS or greater.  For those that don’t know 2000 IOPS is a pretty big workload.  Consider this, a standard fiber channel 10,000 RPM drive averages around 125 IOPS per disk.

I didn’t really understand this and the knowledge base article is lacking any detail on the rational behind the statement.  I reached out to VMware performance engineer Scott Drummonds to see if he had anything he could publish to help clarify the KB article.  Scott was nice enough to research this and posted his findings here.

So it appears that the technical explanation is interrupt coalescing or buffering.  The paravirtual SCSI driver was designed to handle receiving multiple requests at a high rate and then “batching” the requests together for better efficiencies in throughput.  If you aren’t generating high enough workloads on the virtual machine, the I/O request could unnecessarily sit in the queue while the “batch" waits to be filled up for the next transaction.  This could cause storage performance problems which would typically be seen as higher latency and would negatively impact the virtual machine.

Now and Then

The great news is the current release of the driver is optimized for heavy workloads.  If you are starting to virtualize SQL/Oracle systems and need the performance, go for the pvSCSI driver and get better throughput.  If your deploying standard virtual machines that are doing lower workloads, continue to embrace the existing LSI Logic driver.

If you are new to vSphere 4, or have just upgraded from 3.5 and are starting to rebuild your templates to embrace virtual hardware version 7, don’t use the pvSCSI driver as part of your standard template.  VMware is working on the driver and will be introducing advanced coalescing functionality.  When this is built into the driver stack pvSCSI will then be able to be utilized for all workloads as it will understand when it needs to ramp up for higher workloads.

Thanks again to Scott Drummonds for taking the time out of his busy schedule to track this one down.

Post to Twitter Post to Delicious Post to Digg Post to StumbleUpon

  • Pingback: uberVU - social comments

  • Tim Oudin

    Great, and quick, follow up Scott!

  • Pingback: Most Tweeted Articles by Virtualization Experts: MrTweet

  • http://deinoscloud.wordpress.com/ PiroNet

    Hi Scott,
    2000 IOPS it's definitely a lot for a single VM!

    >Consider this, a standard fiber channel 10,000 RPM drive averages around 125 IOPS per disk.
    Here is a formula to easily calculate the IOPS of any disk: 1/(average latency in ms + average seek time for read or write in ms)

    More at http://deinoscloud.wordpress.com/2009/09/11/und…

    Cheers,
    Didier

  • scottsauer

    Thanks for the link Didier! Yes this breaks the traditional expected IOPS for any virtual machine, I would say once you get into several hundred iops per vm, you need to shift your thinking about your storage configuration. You could easily begin to impact your other virtual machines that share that same data store.

  • Matt Ray

    Hi, note that this issue is now fixed in vSphere 4.1, so you can use PVSCSI for any workload.

    • Section_32

      HI
      Now a year has moved on, do we know if 4.0 U4 has the same fix as 4.1?