Archive for 2009

First things first

Thanks to Scott Sauer (@ssauer) and John Blessing (@vTrooper) for holding down the fort here at Virtual Insanity while I’ve been finishing up some unfinished projects and preparing for the VCDX Design Exam (which I take later this month).  One of Scott’s posts actually won a vSphere blog contest.  Nice work Scott!  These two guys are becoming pretty good friends of mine here in the Cincinnati area, so hopefully I can convince them to keep the content flowin’.

An itch I couldn’t scratch

I’ve mentioned here on this blog, at least once or twice, that I “eat the dog food” and actually run my primary XP desktop as a VMware View image.  Since the conversion almost a year ago, everything has been running pretty well with only a few minor bumps along the way.  And with the recent addition of PCoIP, I can’t imaging ever going back.

But there was one little reoccurring problem I was having for which I couldn’t seem to find an answer.  It wasn’t a show stopper of an issue, but it was just an “itch I couldn’t scratch,” if you know what I mean.  And the problem went something like this …

1. Inside my desktop VM I have a Cisco VPN client, necessary for a secure connection back to corporate HQ in Palo Alto, CA. 
2. When connecting to my desktop with the VPN client inside the VM inactive, I had no issue.
3. However, if I disconnect from my desktop while the VPN session was active, then I couldn’t reconnect to my desktop via VMware View. 

The reason?  The broker was sending me the new IP address of the Cisco VPN Adapter, which is an IP address on the VPN, and an IP address my local computer didn’t know about. 

Now, if I were to log off instead of disconnect from my desktop, this would terminate the VPN session and therefore wouldn’t be a problem.  But who wants to log off every time?  More often than not, I have things open on my desktop (e.g. half written emails, documents, browsers with many many open tabs, etc.) that I don’t want to bother saving and closing every time I step away from the computer.  And really the bigger issue is with unintentional disconnects that result from local power/network/OS issues.

I tried all sorts of things to fix this.  Among other thins, I tried …

1. Reordering the NICs, hoping the broker was just grabbing the first NIC. 
2. Poking around the broker and agent install files, hoping to find a way to force the IP address. 
3. I even tried uninstalling and reinstalling the View agent and the Cisco client, hoping the order of installation might do the trick (admittedly, this was a random shot in the dark)

But nothing seemed to work.  So until recently, to reconnect I would have to connect directly to my desktop via RDP, or connect to the console via the VMware Infrastructure Client, then disconnect the Cisco VPN and then reconnect via the View client. 

See what I mean?  Not a show stopper, but man what a pain in the butt! 

The solution

Well I found a way around this with a handy new addition to the Command Line Tool in View4.  Check out page 12 of the Command Line Tool for View Manager titled “Override IP Address.”  On the broker from a DOS prompt, in the c:\Program Files\VMware\VMware View\Server\bin directory, execute the following …

vdmadmin.exe –A –d <desktop name> –m <machine name> –override –i hostname

The “desktop name” is the name of the VM in the broker.  The “machine name” is the name of the VM in vCenter.  It’s likely they’ll be the same, but they don’t have to be and in fact, in my case they weren’t the same.  The “hostname” can be either a FQDN or an IP address.  Oh, and I can tell you that all parameters must be present or the command won’t execute. 

But that was all there was too it.  Now I can disconnect and reconnect to my desktop, regardless of the state of my VPN client.

Part 2 Doing the work

As you might have noticed, this blog post is a continuation to my first post about PVSCSI, you can access Part 1 here.

Hopefully now you have a better understanding of what the Paravirtual SCSI driver is all about, and we can prove there are tangible reasons to move in this direction.  Let’s get on with the important part, the implementation phase.

(I need to finish off this blog post, I am running out of pictures of SCSI cables)

There are some caveats I need to start out with.  In case you missed it, PVSCSI drivers on virtual machines aren’t supported on operating system disks unless you are running vSphere 4 update 1.  You can use the driver on a secondary data disk if you so desire, but for this post I am going to assume you are running vSphere 4 update 1 (Virtual Center and ESX Hosts) and want to know how to get the driver working on all disks.

In most cases, it’s always easier to build new.  You know you have a clean install, the drivers are updated, the configuration is solid.  I would suggest updating your templates to include the new paravirtual scsi driver.  Your existing virtual machines run fine with their existing configurations, and depending on your environment, it might be a lot of work to go back and target all of your virtual machines.  For an upgrade path, my personal opinion would be to target your heavy I/O virtual machines.  Upgrade the VM’s that will make a difference, and you will see some immediate benefits.  Reducing the I/O on the disk subsystem will only benefit the other virtual machines that might share those same physical disk spindles.

Clean install

This section will walk you through the process of installing the driver with a Microsoft Windows 2003/2008 operating system.  Currently these two operating systems are the only ones supported.  Hopefully we will see some added support for the various Linux operating systems down the road.

Walk through the “New virtual machine Wizard” as you normally would.  On step 9, ensure you select the “VMware Paravirtual” option as seen below.

Before powering your new VM up, you need to connect the virtual floppy image file that has the driver for your desired guest operating system.  This is not on the VMware.com website under downloads, it already exists on your ESX host.  You will need to browse to the following location on your ESX host. [Datastores]\vmimages\floppies I would wait to connect your floppy disk image after you boot off the Windows CD-ROM so it doesn’t try to boot off the floppy drive.

When you power up your new virtual machine, select the F6 option to tell the operating system you need to use a third party SCSI driver:

Now connect your floppy disk image to your virtual machine under the “edit settings” option.  You should now be able to point to operating system to the driver as seen below:

Continue on with your normal installation, and you are complete.  Your new virtual machine is now utilizing the Paravirtual SCSI drivers.  I suggest now converting this image you created to a template for future deployments with this configuration.

Upgrading and Existing Virtual Machine

To upgrade an existing virtual machine, the process is pretty straight forward.  Assuming you have already upgraded to the latest virtual hardware (Version 7), make sure your VMtools are upgraded post Update 1.  Shut down the VM, and edit the settings “Change Type” as shown below:

You will get another window that will alllow you to change the type of controller as seen below:

Select the “VMware Paravirtual” and then select ok.  Boot up your virtual machine and you are all set.  Your system is now running with the updated drivers and you can take advantage of the newer drivers that provide better throughput and less latency!

Hope you found this post useful.  Good luck!

Scott

One of the new features that was added to the release of VMWare vSphere 4.0 was a new SCSI subsystem driver that allows more I/O and less latency per virtual machine.  What the heck is PVSCSI?  Here is the technical definition stripped right from the vSphere storage guide. (RTFM).

VMware Paravirtualized SCSI (PVSCSI) is a special purpose driver for high-performance storage adapters that offer greater throughput and lower CPU utilization for virtual machines. They are best suited for environments in which guest applications are very I/O intensive. VMware requires that you create a primary adapter for use with a disk that will host the system software (boot disk) and a separate PVSCSI adapter for the disk that will store user data, such as a database.  The primary adapter will be the default for the guest operating system on the virtual machine. For example, a virtual machine with Microsoft Windows 2008 guest operating systems, LSI Logic is the default primary adapter. The PVSCSI driver is similar to vmxnet in that it is an enhanced and optimized special purpose driver for VM traffic and works with only certain Guest OS verision that currently include Windows Server 2003, 2008 and RHEl 5. It can also be shared by multiple VMs running on a single ESX, unlike the VMDirectPath I/O which will dedicate a single adaptor to a single VM.”

So what does all that mean for you?  Better disk performance and less CPU cycles spent on processing these disk requests.  I took some notes at VMWorld 2009 during a few different sessions that discussed PVSCSI.  Here is my logical diagram of what PVSCSI is.  Download the PDF version here so you can print it out and frame it on your cube wall!

Sauer_PVSCSI.pdf

With the release of vSphere 4.0 PVSCSI was only supported on disks other than the operating system (secondary data drives).  For more information on this, reference KB Article: 1010398.

vSphere 4 update 1 is now released and it’s exciting news for those looking at utilizing PVSCSI.  Support for boot disk devices attached to a Paravirtualized SCSI ( PVSCSI) adapter has been added for Windows 2003 and 2008 guest operating systems.

So let’s first find out if it’s all that.  We need to do some testing to validate the hype.  I created two virtual machines, one with the traditional LSI Logic SCSI driver, and one with the new PVSCSI driver.  The host is the same for each VM, 4 socket Intel Xeon system with 64 GB of RAM, connected to EMC Clariion CX3-80 storage.  The Raid configuration is a 4+1 RAID 5 set (10K spindles), with the default Clariion Active/Passive MRU setup (No PPVE).  Each VM has 2 vCPU’s and 4 GB of RAM and both are running 32 bit Microsoft Windows 2003 R2.  Both Virtual Machines data disks were formatted using diskpart and the tracks were correctly aligned.  Anti-virus real time scanning was disabled on both systems.  This test is meant to get as close as possible to a standard configuration that we can benchmark from.

I used IOMETER as my testing engine.  I didn’t go too deep on the various settings.  The first run is 32K 50%R 0%W.

Non-PVSCSI

With-PVSCSI

Quite the difference, no?  To be honest, I was seeing a lot of fluctuation while doing my tests.  I probably should have segregated things out a little more, but the screen captures were the average of the results.  I was thinking maybe I should use the built-in random IOmeter combined results.  So here you go.

Non-PVSCSI

With-PVSCSI

I believe the results speak for themselves.  I need to do a little more testing for my own personal preferences.  I want to get a more insight on what the differences are on the reads/writes and the various sizes.  I am certain cache has a lot to do with the results, but I think IOmeter can bypass cache since you force the randomizer.  I’m also curious about the sweet spot on the block sizes and how that plays out with read vs write.

Conclusion

PVSCSI is a technology worth moving towards.  There is no cost involved, and it can deliver better disk performance across your ESX environment.  It also can bring your host CPU utilization down, which can provide you with better consolidation ratios across your clusters.  Stay tuned for part 2, when I am going to provide the “how to do it” aspect so you can begin to leverage this technology you are already paying for!

Hope you found this information helpful.  Thanks goes out to Aaron Sweemer (@asweemer) for allowing me to abuse his website and not having to deal with bringing up my own site.

Thanks!

Scott Sauer

A few months ago I posted an update, with a section near the bottom titled “RoR (Ruby on Rails) and other next generations frameworks."  And in that section I made the statement …

 

About two years ago I was introduced to Ruby on Rails and since then, most of my development work has been with RoR.  Thus far, however, I haven’t posted anything on this blog about RoR.  Why?  Two reasons.  The apps I’ve written to date have absolutely nothing to do with VMware.  And second, like I said, I’m an amateur.  Anyone looking for RoR help and advice can probably find better info on actual RoR blogs. … But I’ve decided that this is about to change.

 

But since that post, I have yet to write anything about RoR. Why? Believe it or not, I’ve got a really good reason.  You may have recently heard that VMware has acquired a company called SpringSource.  And SpringSource offers support for a similar type of language and framework which has deep roots in Java, called Goovy and Grails (in addition to a slew of other Java related products and services).  From the SpringSource website …

 

Grails is an advanced and innovative open source web application platform that delivers new levels of developer productivity by applying principles like Convention over Configuration.

 

Groovy is the leading open source dynamic language for the Java Virtual Machine that offers a flexible Java-like syntax that most Java developers can learn in matter of hours.

 

Once I learned about the acquisition, I had to make a decision. Do I continue down the RoR path? Or do I switch gears and go in the direction that VMware’s going? Not that it’s impossible to be good at both — or even difficult for a true developer — but I’m not a developer by trade and I’ve got too much going on in my life and with VMware to focus on more than one language and framework at at time.

Now, this may sound like an easy decision, as it would naturally make sense to follow my employer’s lead. But while I’ve played with may different languages in my past (e.g. C++, Visual Basic, Perl and Ruby) and even become fairly proficient in one or two, the one language I’ve avoided has been Java. Frankly, Java just isn’t fun for the amateur developer, in my humble opinion. But after doing my homework and reading numerous blog posts such as Bye bye Ruby, hello Groovy I decided to make the switch.

And so far I’m pretty happy with my decision. Groovy may have Java-like syntax, but it is a dynamic language that is a lot of fun to code in and it’s and pretty darn powerful. I’ve already finished my first web app written in Groovy and Grails (a reporting and graphing tool that the local OHV rep’s and SE’s will use) and it’s about to go live.  So right now, I’m feeling pretty Groovy.

Actually, as I type this, I’m at the New Orleans airport after three days at SpringOne 2GX where I’ve been immersed in all things Groovy, Grails, Spring, etc. It’s been a great event, where I sat in on many fantastic sessions and got to meet super crazy smart people. That’s always fun for me. But right now I’m in information overload. I need to compile my notes (which I had to take by hand because my laptop battery decided to reduce it’s charge life to about 5min.  Grrrrr.) and put them into something meaningful for the audience of this blog, which for the most part are not developers.

Until next time, check out Groovy and Grails and read the good tutorials out there like …

• Mastering Grails by Scott Davis
• The Official Grails Quick Start Guide
• Grails and Google AppEngine Beginners Guide by Morten Nielsen
• The Groovy Getting Started Guide

I’m in New Orleans.  This is arguably one of the funnest cities on earth (or so I’m told, this is my first time here).  But despite the crazy night life and all the energy outside, I type this blog post sitting in a dark hotel room with a stiff drink in my hand.  I’m not in the mood to venture outside right now.  The vodka tonic helps blur a reality I keep avoiding.

Just a few short hours ago, while sitting in one of the sessions at the SpringOne 2GX event, I received an email from one of my oldest and dearest friends.  His mother, Martha Speck, lost her battle with cancer.  A battle that began a mere two weeks ago and frankly, a battle that I didn’t even know was being fought until today.  While reading his message, I found myself trying not to break down in tears in front of 50 of my peers in the session.  And my emotional response came as somewhat of a surprise to me because it had been a number of years since I’ve spoken to Mrs. Speck.  But when someone touches your heart and life, time is irrelevant.

Do you have a teacher that you identify as the one person who really inspired you?  I do.  It was Mrs. Speck.  She was my English and Creative Writing teacher in high school.  Anyone who had Mrs. Speck for Creative Writing at Liberty High School knows what an incredible teacher she was.  Steve and I actually had her class together at the same time.  I’m sure that must’ve been a bit strange for her.  Can you imagine?  Her son and her son’s best friend sitting in the back of the classroom thinking they could get away with murder.   Oh and we tried!  We were young and stupid and tried to take advantage of the situation with all sorts of crazy nonsense.  But she handled our antics with the perfect blend of class, humor and discipline.  We got away with more than we probably should have, but not nearly as much as we wanted to!  And somehow, through it all, I came out of her class with a passion for writing … something I certainly didn’t have going in to her class.  She exposed and cultivated a latent passion within me and therefore she, in no small way, had a hand in shaping my future.  If you think about it, this very blog is the result of her inspirational teaching.

But her inspirational teaching, in and of itself, wouldn’t make me well up with tears.  You see, Mrs. Speck was once known to me as my “second mom.”   During my high school years, her son Steve and I were the best of friends and we shared most of our free time together.  This, of course, means that I had the fortuitous opportunity to spend a lot of time with Steve’s family.  His mother, father and sister became my second family as I shared countless evenings and weekends with them.   It was a great time in my life filled with so many wonderful memories.

But despite the wonderful memories I have of my time with Steve and his family, right now as I sit here in this dark hotel room, I am overwhelmed with deep sadness and regret.  Deep sadness because an amazing, brilliant woman who played a significant role in shaping my future, and more importantly, a woman who I once called “mom,” is gone.  And I feel deep regret because as my life has taken me all over the world, it has been years since I’ve seen or spoken to her … something I will never be able to rectify.

At the end of the day, I believe the only thing we can hope for (as far as this earthly life is concerned) is to leave the world a little bit better than we found it.  In fact, I believe there is no higher compliment than to simply say, “my life is better because of you.”  This is a compliment I would pay to more than one person in my life for sure, but also to be sure, the list would be extremely short.  So let me say, with a tear on my cheek and with all the sincerity and love in my heart …

Farewell Mrs. Speck.  My life is better because of you.

 

– The vTrooper Report –

 

This is a continuation of the Capacity Conundrum, if you missed the first part start here.

$ per Compute VM

So let’s cut to the chase.  In the case of the compute tiles of our Quad we have a price per vCPU and $ per GB of RAM to settle. Keeping our example 2U server in play we could expect to spend approximately $15,000 for a 2U fully loaded with 4GB DIMMs.   Well unfortunately a small part of that 15K  is consumed in I/O cards and maintenance which needs to be pulled out to get the compute number.  For our argument we will use $10K for the compute system without the I/O cards and maint. costs;  This is the CAPEX we will offset in our $/per values.

vCOMPUTE – FIREPOWAH!

We know how a CPU works right? Move process into memory , execute CPU cycles, churn, churn more, back to the I/O guys, rinse and repeat.  Basically, this is where the hardware container happens in our data centers.   I say container because it’s easy to show it as a box; It’s hard to define what it will always be in physical form.  1U, 2U, 4U, half blade, Full Blade, appliance , PC ; you name it, it is probably in some one’s ‘datacenter’.  The lowest common denominator I have been able to settle on for a common form factor is Cores per Ram.  Grouping per socket fits because you are measuring the type of memory that is close to the CPU socket.  The NUMA architectures of AMD and Intel with memory controllers on-board and transports to the memory DIMMs without access through the I/O  controllers (eg. Northbridge) help define the grouping.

TECHNOTE:  Every core has associated memory banks it will use and every container(physical server) has a series of sockets that it controls.   A hypervisor has a limit to how well it can control the associated memory space to the nearest vCPU.   Generally the hypervisor will always schedule available vCPU’s from the same socket and swap the corresponding memory for those processes to the memory banks of the corresponding socket.  It does this is for efficiencies of the x86 architectures.  It can move the vm to another socket and readdress the memory but it has a ‘cost’ associated with such a move.  Path of least resistance is to stay in the same socket.

If you create a 4 vCPU VM and run it on a 1 core  processor it gets bogged down.   If you have the same VM on a two socket Quad Core (8 Cores)  the four cores utilized by the VM are likely to be on socket 1 or socket 2 .  The cost of splitting the vCPU between the two physical sockets by the scheduler is greater than running the vCPU in the same socket.    AMD delivered this earlier than Intel and sustained higher levels of virtualization consolidation “Per Host” than similar class systems of Intel could provide through the Northbridge.   Core i7 is a new game for Intel and the results of Nehalem show the improvements.

For more indepth information here is a good read:  CPU Scheduler in VMware ESX

We have a host of $10K  CapX charge that has two sockets at a 4/45GB Socket Ratio with approx $5k spend in each socket.  Looking at our Hardware invoice the CPU Cores are about 25% of the cost of a socket so we can assume that our per socket cost is broken down into 25% Core and 75% Memory.   So our Socket Ratio yields a $1250 cost for 4 cores and $3750 for 45GB of memory:

Per Core CapX = $312.50;  Per GB RAM CapX = $83.33

That gives us a bare metal cost without a hypervisor charge on top, but we need a hypervisor to get a VM running.  Adding in the ESX cost for a per socket license of ESX Enterprise Plus (worst case) you can add $3500 each socket.

ESX Lic. Cost per socket CapX = $3500

Raw burn rate of the host would be $8500 per socket if we never loaded a VM on the Host.  Well, we did it for a reason, so let’s get our money back. If we target the standard allocation for this host (4/45GB socket ratio) we get our target VM count of 16 per socket(1 vCPU/2.8 GB RAM).  Also, keep in mind that we broke the socket cost down by 25%  to CPU and 75% to Memory so we will keep that  same  split here.  If we don’t do the split, then any VM that is deployed to the socket will bear the same cost regardless of its size.

ESX Lic. Cost per VM= $218.75   ( 3500 / 16 )

-Or-

Split by the 25/75 % we did previously for the cost of the CPU and Memory and you get a little different calculation.

3500 * .25 = 875 / 16  = $55 AND     3500 *.75 = 2625 / 45  = $58

per vCPU=$55

per vMEM=$58

Adding it up with our target ratios in tow we get the burn rate of the $ per Compute on a VM basis.

($312/4  = 4:1 ratio) + (83*2.8) + {(55*1) + (58 * 2.8) } = $530

Or Summarized: (vCPU = $78)+(vMEM = $233)+(Hyper$=219)=(vCOMPUTE = $530)

Assuming 8760 Hours (1 year) this VM would cost $.06/hr in vCOMPUTE.

Lets apply that to some other VM systems and see if it sticks.  If we plan for the following VM deployment on our socket:

The costs would spit out as such:

Or slice it up into a per hour number:

So based on this analysis some of my VM’s probably only cost $.05 per hour for vCompute.  Interesting. What is more interesting is the fact that the memory cost associated with a VM scales more accurately to the consumption.  You can have as much memory you like for your new 4 and 8 GB aspirations; (eg. memory leaks) you just need to pay for it accordingly.

Too bad that only pays for the top part of my total cost model.  That said, the benefit here is that this model can span across hypervisors and any market hypervisor can be split up to show the cost of a VM consumed on a Xen , KVM, VirtualIron, Parallels’, or Hyper-V infrastructure.

I will be working on a few powershell scripts and excel calculators that one can use to make this model more repeatable. At the very least, it is a model that I will use to consider CapacityIQ and third party products like the offering from VKernel; and the output they measure.  Especially if they consume additional costs on a per socket basis.  Which I can now calculate as Overhead.

Alas there is more to consider, stay tuned for Part III – “the I/O that binds”

Going Thin and not looking back.

Yes, I am slowly losing my hair like many other aging men out there, but it wouldn’t be virtual insanity if I were blogging about my personal male pattern baldness issues.  With the latest release of VMware vSphere comes a lot of new features and functionality that can be leveraged to make our lives easier.  One of these features, that I personally have been looking forward to for a while, is Thin Provisioning.  If you aren’t familiar with this technology, jump over to Gestalt IT for a great explanation of what it is and how it works.

One of the exciting promises of thin provisioning, is getting more “bang for your buck” out of the expensive enterprise storage you have been investing in for your ESX environment.  But, as Bret Michael’s once said, “Every rose has its thorn” and there are some things to look out for and considerations to make, before implementing thin disk technologies.

Efficiencies are great if they work right and don’t over

complicate the environment.

Do your homework and make sure you understand the characteristics of the virtual machine that you are considering migrating into a thin disk configuration.  The last thing you want to do is convert every VM to thin disk, and four months down the road all of your data stores are filling up and you’re scrambling for a storage CAPEX.  Some people are of the opinion to do thin provisioning either on the host side (VMware) or on the storage array side, but not both.  Take a gander at Chad Sakac’s blog that discusses thin on thin and some thoughts around each of these approaches.  I’m not going to go into all of the pluses and minuses of thin provisioning but rather focus on how to make it work for you.

Coffee Talk

So now that we have some of the basics out of the way, I wanted to share my thoughts on thin provisioning.  Like many organizations, we get requests from our customers that err on the side of caution.  They want to plan for the worse case and ensure that their project and/or application isn’t setup for failure.  I don’t blame them really, I do it myself all the time when I make coffee at home.  I always end up making more coffee than I typically drink, just in case I might need that extra charge.  The best way to do that is pad it, request more than what you might really need, just in case something comes up down the road.  Virtual machine disk storage in some cases fits this same profile.  If my coffee maker granted me access to hot coffee on demand, I would stop making extra coffee.  Thin disks can give your end users that capacity on demand so you can gain control of the padding effect that typically takes place in most corporate organizations.

Take it back…

So now you have done your research, you’re starting to get a feel for what this thin stuff is and how it might play out in your shop.  It’s go time.  If you’re a smaller VMware customer, you probably already have an idea of what are good target disks to convert.  If you’re a larger environment, it might be a little more difficult to gauge where the bloated pigs are hiding.

I worked at GE for a couple of years and was exposed to some of the Six Sigma methodologies they preach as well as practice.  Sounds boring, right?  Not really.  You can really leverage DMAIC for a lot of IT related problems/issues/projects.  You don’t have to take it to the extreme, use the framework to help guide you on your quest:

DMAIC

The DMAIC project methodology has five phases:

• Define high-level project goals and the current process.
• Measure key aspects of the current process and collect relevant data.
• Analyze the data to verify cause-and-effect relationships. Determine what the relationships are and attempt to ensure that all factors have been considered.
• Improve or optimize the process based upon data analysis using techniques like Design of experiments.
• Control to ensure that any deviations from target are corrected before they result in defects. Set up pilot runs to establish process capability, move on to production, set up control mechanisms and continuously monitor the process.

We have already defined our project goals and what we are trying to accomplish.  We need a good “Measure” tool to really find where we might benefit from thin provisioning.  Powershell is a great tool that most VMware administrators use, or have at least heard of.  So this was the first place I turned to for assistance.

Alan Renouf of “Virtu-AL” http://www.virtu-al.net/ gave me a hand in writing the powershell script needed.  (Thanks again, Alan!).  Alan already had a one liner script to produce a list of vm’s, their disks assigned, and how much data each disk was consuming.  I needed the ability to see this data outside a powershell window and be able to analyze it in a better format.  We have a decent-sized VMware environment and exporting this out to a .csv for analysis is extremely helpful.  Here is the script!

************************************************************************

# Set the Filename for the exported data
$Filename = “C:\VMDisks.csv”

Connect-VIServer MYVIServer

$AllVMs = Get-View -ViewType VirtualMachine
$SortedVMs = $AllVMs | Select *, @{N=”NumDisks”;E={@($_.Guest.Disk.Length)}} | Sort NumDisks -Descending

$VMDisks = @()
ForEach ($VM in $SortedVMs){
$Details = New-object PSObject
$Details | Add-Member -Name Name -Value $VM.name -Membertype NoteProperty
$DiskNum = 0
Foreach ($disk in $VM.Guest.Disk){
$Details | Add-Member -Name “Disk$($DiskNum)path” -MemberType NoteProperty -Value $Disk.DiskPath
$Details | Add-Member -Name “Disk$($DiskNum)Capacity(MB)” -MemberType NoteProperty -Value ([math]::Round($disk.Capacity/ 1MB))
$Details | Add-Member -Name “Disk$($DiskNum)FreeSpace(MB)” -MemberType NoteProperty -Value ([math]::Round($disk.FreeSpace / 1MB))
$DiskNum++
}
$VMDisks += $Details
Remove-Variable Details
}
$VMDisks | Export-Csv -NoTypeInformation $Filename

***********************************************************************

So now that you have this great spreadsheet, you can do all sorts of crazy sorting and reporting, within Excel.  Take some time on phase 3, “Analyze” what you’re seeing.  Talk to your VM stakeholders to see how things might be changing from their perspective.  Try to plan for the surprises and position yourself accordingly.

Next is the “Improve” phase of DMAIC (see it’s easy!).  This is the part where you actually do the work.  It’s time to start leveraging the storage VMotion API’s, and reclaim some of that unused disk.

1. Select the target VM in the VC client.
2. Right click on the VM and select the option “Migrate”.
3. Select the option “Change Datastore”.
4. Select the destination, or click advanced if you are targeting one particular disk.
5. Select “Thin provisioned format”.
6. Select Finish.

Rinse and Repeat for the rest of that spreadsheet you have worked so hard on.

The last phase of DMAIC is “Control”.  This is one of the most important pieces to thin provisioning in my opinion.  At the minimum you need to setup Virtual Center alerts to monitor when your datastores are approaching critical levels.  You can’t implement thin disks in your vSphere environment and walk away.  The smart people over at VMware have given us the ability to monitor datastore disk space usage and over-allocation with the latest release of Virtual Center.  Setup your monitors so you are e-mailed when some of these thin disks begin to grow and you need to take some action.

Eric Gray of VMware takes this to the next level, check out his blog post on utilizing powershell to prevent datastore emergencies.  My personal approach to this concept is to setup a “hotspare” datastore for your environment.  A good practice to implement here would be to try reclaiming enough storage from your migrations to thin disks to free-up a “hot spare datastore”.  Implementing an automated recovery solution like Eric’s will help you sleep easier at night.  Worried about what might happen if your script doesn’t work or you do hit the perfect storm and end up with a full VMFS volume?  Intelligence has been built into vSphere to automatically pause the virtual machines, impressive.  Check out Eric’s video:

Wrapping it all up

Thin disk provisioning is a great feature that you should consider leveraging in your environment.  With some forward thinking and best practices you can achieve higher ROI for your ESX storage.  VMware vSphere offers the ability for you to migrate from thick to think with no downtime, so you can begin reclaiming storage on the fly.  Keep it simple, start out with a high level analysis of your infrastructure.  Identify the candidates that are a good fit and worth focusing on.  Setup your alerts on the datastores as soon as you migrate your first virtual machine so you are protecting yourself from problems down the road.  Consider taking automated actions if your datastores are reaching critical thresholds.

I hope you found this article helpful, good luck!

Scott Sauer

A colleague of mine, Mark Medovich, turned me on to an interesting solution for Mac users, like me. I am a recently converted Windows user to the MAC. (now for a little over a year) I have to admit, I love my MAC. I still use many Windows applications. I like outlook better than entourage, and the powerpoint presentation still works better with the version in which it was created. Hurray for Fusion… But I digress…. This solution is about running a MAC OS on top of vsphere. What? Really? I thought that wasn’t allowed? Actually, the great community over at DiscCloud have created a way to do so, without actually “running” the MAC OS on vsphere. http://disccloud.ning.com/

You do still have to run the Mac OS on your Mac and you can’t run the MAC OS on a non MAC Hardware. Pretty cool stuff. The way this works, is that it mounts the MAC OS instance as another instance on your mac client. Everything you work on, is stored and secured on the vsphere server. Backups can happne in the cloud.

All of the FAQs are here, including how it doesn’t affect the Apple EULA. http://disccloud.ning.com/page/disccloud-faq

This is a real world example of how the cloud can benefit the average MAC user, like me. This example brings home the how the cloud will eventually be a natural way of how we all will do our computing, in the future. Apple, as we know, has had a resurgence in popularity in the last several years with the iPhone being one of the major drivers. But us with MacBook Pros, have also helped. This is all conusmer/end user driven growth. Not Enterprise driven growth. Actually the MAC causes traditional enterprise IT shops difficutly. This is not unlike of how vmware got its momentum through individual server administrators that were tired of doing work at 2am on Sunday morning. And now, DiscCloud, a group of passionate MAC users and developers have developed a very inovative solution using vSphere and a cloud computing mindset. Here’s to the community, individual and innovation. Now its up to companies like vmware to take those inovations and enhance their ability to be managed within the enterprise.

 

–The vTrooper Report –

 

In an effort to gel up an internal billing and allocation model (GaaS – Goughing as a Service) I’ve been struggling with the concept of cost per vm.  I was asking a simple question in the twittersphere about that idea and it turned into a discussion and well…got out of hand.  I apologize for that, as this is a better format to explain. (Special thanks to @asweemer for a dumping ground)

If I had a Nickel for each VM…

At VMWorld 2009 there was a presentation in the keynote that showed the price of a vm hosted with Terramark that was $.05 per hour.  I thought  wow.   A nickel per hour.  If I had a nickel per vm/hour; How much would I have available to spend on coffee?

Then I thought wait.  I have VM’s. How much do they really cost me per hour?  Well the answer is … it depends.  Old servers with high power consumption and low density versus a new system with Intel 5500’s and packed in blades have different burn rates visible to different systems(power, cooling, depreciation).  I haven’t found a great model to break those units down to my satisfaction yet.  I need another way.

As a general practice I create in my mind some maxims that I follow in the creation of a VM.

1. S  – 1vCPU, 1GB Ram, 1GB Net, 10GB Disk
2. M -2vCPU, 2GB Ram, 1GB Net, 20GB Disk
3. L – 4vCPU, 4GB Ram, 2GB Net, 40GB Disk

Seems simple enough, but it doesn’ t really generate a cost model on a consistant basis.   Hardware continues to change and each VM that consumes resources does it at different rates and times of the day.   A VM that isn’t doing anything isn’t really ‘consuming’ anything, right?  I thought I would try break it down further by creating a 4 quadrant block with two macro categories:  Compute (CPU and Memory) and I/O (Network and Disk)

Each resource area could increase\decrease for a reason without changing the size of the original maxim it was created under.  This allows for small variations of size without having a customer yell that their bill when up by $2 this month.

The Measureable Unit

Use a unit of measure to identify the four quadrants:   vCPU : vMEM : vNET : vDISK  or C:M:N:D  .   Then overlay the VM creation to count up the units.  This way the growth of a ‘VM’ during its lifecycle can be adequately allocated back into the proper IT metric.  Using the VM creation maxims up above this may be:

1. S  – 1:1:1:1
2. M -2:2:1:2
3. L – 4:4:2:4

This isn’t perfect but it at least allows for the average cpu cost to be allocated seperately from a memory, network, and disk cost.  Afterall, you don’t get to upgrade all four parts of the quadrant in the same fiscal year usually.  This also allows a way to trend an average of your cost rate per unit over a period of months and years to see which cost areas are improving.  It is an interesting metric for the business and IT.  Win-Win in my book.  Even if no-one internally ever has to pay the values back (Showback).  It also helps police which VM is consuming too much of a specific value which would skew the numbers if you simply took the cost of the esx hosts and divide by the number of VM’s.

Apples , Oranges, Lemons, and Grapes = Frutti Results

So you have a unit of measure and a type of system to match the measurement up towards over a period of time.  Here’s where the fruit cart and the horse get hooked up.

This is all very complex, why can’t I just buy the same server I have purchased for the last 5 years?

Sorry Kids. They don’t build’em like they used to.  But in todays market, the UCS system from Cisco has a new buzz to the original players of IBM, HP, and Dell.   How do you sort any of that out among the offerings, and how do you select the right platform for your new ESX System?     By the Socket !   Every system of the x86 family has them from both the Intel and AMD families.  And now that you have to pay for your hypervisor and additional tools (Capacity IQ, AppSpeed,  Nexus1000v) per socket  it matters more.  I need to squeeze the value out of those sockets.

Still staying in the upper half of the Quad;  lets measure cores and RAM as a ratio assuming dual rank 4GB Dimms and measure them to some of the standard 2 socket servers.

Standard Intel x5450

2 Socket – 4 Core – 16 Dimms (8 per socket) produces 4 cores/ 32 GB Ram

Standard Intel Nehalem  x5500

2 Socket – 4 Core – 18 Dimms (9 per socket)  produces 4 cores/ 45 GB Ram

Cisco UCS extention  on x5500

2 Socket – 4 Core – 48 Dimms (24 per socket)   produces 4 cores/ 96 GB ram

What this shows is that for every license of ESX consumed in the environment there are different amounts of memory available for a VM to use.  The approach by the UCS system allows for a much higher allowance of memory to a VM at the same licensing cost.   Sure you could buy 4 way servers and claim that the 256 GB of RAM gives the VM more allowance but in reality the vm will have ratios of contention to the vCPU and Memory within each of the 4 sockets. You can change the size of the container by moving to a 4 way,  but it won’t change the value of the ratio  for that container in regards to the cores and memory.

CPU Contention

The idea of CPU contention is becoming more visible to most administrators of virtualized environments because the desire to pack the vm’s onto a host is so strong.  If I can get 10 VM’s on a host for $5000 then getting 25 VM’s on the same host is lowering my cost per vm.  It could also be cheating your customers of the performance they paid. Especially if you have multiple vCPU’s assigned to those 25 VM’s.    This is where the ratio of VM per host becomes obsolete and vCPU/core  makes more sense.

Using the example containers above you can generate an expected number of VM’s per socket.  There is no reason to do a 1:1 ratio of cores to VM because the point of virtualization is to run more with less.  I think a good ratio to start with is 4:1 for a production VM and 16:1 for a VDI implementation:

Standard Intel x5450  -  (4 /32 GB SocketRatio)   yields 16  VM’s with a 1 vCPU/ 2GB ram configuration per socket

Standard Intel Nehalem  x5500  -  (4 /45GB SocketRatio) yields 16  VM’s with a 1 vCPU/ 2.8GB ram configuration per socket

Cisco UCS  -   (4 /96GB SocketRatio) yields 16  VM’s with a 1 vCPU/ 6GB ram configuration per socket

You can always adjust your actual deployment if these ratios don’t match up for your environment.   The expected deployment number helps determine how large the pizza slices are for the team.  Not how many slices each of them consume.  In these configurations you can see where the density of the RAM per socket (SocketRatio) of the UCS allows for much larger VM configurations before overcommitment. A nice fit for the new 64bit installations. These expected numbers of VM per socket help determine what the burn rate of a C:M:N:D value is for the CapX  spend you made.

BurnRate

To fully understand how much a VM costs, one has to look at what was spent in the CapX of the host and agree on the measuring stick to measure the C:M:N:D value of the created VM.  If a series of hosts are in service from different families and are at different parts of lifecycle there may have to be some averaging.  The SocketRatio of Cores/RAM is a consistent way to measure systems from different form factors and families and levelset the expected allocation of VM’s.  The expected allocation of VM’s for a host helps determine what density ratio is desired for vCPU:vMEM.

This is the end of Part 1 –  In Part Deux I will take a deeper dive into the Compute and I/O areas and assign a more detail cost per VM model.

I ran into an issue the other day as I was trying to deploy a VM from from a template using the vCenter Customization Specification Manager.  I was trying to use a custom Sysprep.inf file that would automatically have the newly created VM join my AD domain and placed in a specific OU. 

Now, when I Sysprep’d the VM normally with my custom .inf file, everything worked fine.  But importing that exact .inf file into vCenter and deploying the VM from a template, the Sysprep failed.  So I had some digging to do, and here’s what I found out.

First, vCenter doesn’t actually store Sysprep.inf files.  Rather, vCenter stores the configuration parameters in XML and then generates the Sysprep.inf file on the fly during the deployment process.  (This part I actually already knew, the next part I didn’t).

Second, and most importantly, when importing a customized Sysprep.inf file, vCenter does not store each parameter as a separate XML element.  So for example, given the following custom text …

[Identification]
JoinDomain=mydomain.com

DomainAdmin=administrator

DomainPassword=1234

MachineObjectOU="OU = MyOU,DC = mydomain,DC = com"

 
I thought this would be stored in the normal, expected format, like this …

<JoinDomain>mydomain.com</JoinDomain>

<DomainAdmin>administrator</DomainAdmin>

<DomainPassword>1234</DomainPassword>

<MachineObjectOU>"OU = MyOU,DC = mydomain,DC = com" </MachineObejctOU>

But it turns out, when importing sysprep.inf files, vCenter stores the parameters as a single XML element with a modified <_type> element like this …

<_type>vim.vm.customization.SysprepText</_type>

<value> [Identification] JoinDomain=mydomain.com DomainAdmin=administrator DomainAdminPassword=1234 MachineObjectOU="OU = MyOU,DC = mydomain,DC = com"</value>

There’s a couple important points to note here:

1. vCenter only stores the XML this way when importing a sysprep.inf file.  When using the customization wizard, vCenter generates XML which is formatted the normal way. 
2. The first element, <_type>, contains the value vim.vm.customization.SysprepText.  When using the wizard, the value for this element is vim.vm.customization.Sysprep (without the trailing “Text”).
3. When the XML is stored this way, whitespace matters!  Notice how whitespace is the delimiter in the <value> element?  And notice the spaces in the MachineObjectOU parameter?  Removing the spaces did the trick.

Mr. Dudley Smith has updated his PDF diagram with some minor corrections and additions.  Get the latest, most up-to-date version here (click the graphic) …

He also updated “the brain” which can be found at it’s new home http://webbrain.com/brainpage/brain/89EFA582-2C35-F6A2-9ED1-7AD4810266C2/.  Make sure you update your bookmark accordingly.

Chris Everette is a colleague of mine, a Sr. Systems Engineer out of Detroit.  He is a very sharp, seasoned, virtualization industry pro.  In working with him over the past two years, I’ve noticed that he writes very well.  So of course, I’ve been trying to get him to start a blog, or contribute to this one, for quite a while. 

My hard work has paid off   It looks like for now he’ll be a guest blogger here on Virtual Insanity.  Whether or not he starts his own blog remains to be seen.  Please welcome Chris Everette to Virtual Insanity!

During VMworld last week, he sent me his first post, which I’m just now getting a chance to post …

What will your role be in the cloud?

Well being at my 3^rd vmworld gets me thinking.  I am wondering about this “cloud thing” just like everyone else who has been in IT for a long time.  I sometimes follow other blogger’s articles, and I like Chris Wolf’s writing.  He got me thinking, as well.  His article, titled the “Cloud and the Wal-martification of IT”  struck a chord.  If this Cloud thing really takes off, which by all indications it has and will continue to gain momentum, companies may scratch their heads and wonder if it makes sense to have their own IT assets and resources and IT professionals.  Especially companies where their core business is not IT.  So, what does that mean for IT professionals and particularly my customers?  Are we really going to all get our computing from several large cloud providers and many smaller ones?  Does anyone remember mainframe time sharing? 

Do I think that companies will outsource all of their IT to the cloud in the next year?  Probably not.  Chris Wolf uses the timeframe 5-10 years.  However, will portions be moved to the cloud?  I was speaking with one of my customers and he reminded me that he is already “outsourcing” web filtering and spam filtering to two different providers.  Many companies have their web  presences already “in the cloud”.  Software as a Service has had some bumpy starts and stops, but is now a reality for many types of applications.  It will be an evolution.  Security is still of concern.  There will be companies that will try something, not like it, pull back, only to move again to a model that provides them more flexibility and reduces costs. 

What does this mean for the IT professional?  I believe that if you are working for a company that is not an IT company, that you will want to manage your companies migration to the cloud.  Get out in front of it.  Volunteer to do the research.  Use it to further your own knowledge and career.  Can you convert your IT department from a cost center to a profit center and be a cloud provider for other types of similar businesses?   If all else fails, take your expertise to a cloud company.  The exciting thing is in the future you can work for a company that will service many different types of customers and provide you many interesting job challenges.  I believe cloud providers (since their business is IT) will be on the forefront of exciting and new technology as well as need the best and brightest to operate them.  Do an inventory of your skills.  Do you understand databases?  Great, cloud providers need to manage many databases (if not for their customers) for their own internal systems for billing, monitoring, reporting, etc.  Do you understand networking and security?  Cloud providers are going to need to guarantee that data cannot bleed from one customer to another.  Same for storage.  And obviously, also for virtualization. 

We call this concept of being able to support multiple customers on the same infrastructure as multi-tenancy.  Since a cloud provider’s model is multiple customers on the same infrastructure (“cloud”), it better be secure.    Is your expertise Enterprise Applications, messaging, development?  Guess what, cloud providers need application expertise as well so they can meet the business needs of their customers.  If you are a developer you need to be able to write applications that are “cloud aware”.  Applications that can ask for more resources if they need them.  Applications that may service more than one customer.  Do you understand service management?  Do you work with infrastructure services such as backup and recovery, business continuity or data center design?  Do you work for an IT provider, already?  Your customers of the future may change to include a mix of large “cloud providers”.    They exist, today and are called many things.  They may provide Hosting (infrastructure as a service), application services (Software as a service) or multiple platforms including items such as  storage (Platforms as a service) and even voice services.   I am sure there are some cloud “thinkers” that may challenge my simple definitions of these terms, but you get the idea.

I believe the security concern is the single largest inhibitor to companies running full tilt into cloud computing.  However, just like we use VPNs, SSL web services, and other forms of digital encryption of data on the public internet, these security problems will get solved and enforced.

IT is an interesting business.  Sometimes things build up until there is a tidal wave and things change rapidly.  Sometimes things move more slowly and there is an evolution.  However, if you hear the term “cloud” and think it is only a marketing term, you may want to think again.  It is already affecting and will affect all of us in IT sooner than we may think.

What do you think?

I’ve been here at VMworld2009 in San Francisco since Sunday.  Monday was Partner Day and marked the unofficial first day of the event.  Yesterday, however, was the actual first day, open to all attendees.  There is much coverage of the event by numerous bloggers, so I won’t reinvent the wheel and bore you with duplicate content.  Instead, here are a few of my favorite things, so far (we’ve to two more days).  Oh, and this is by no means a complete list.  there are a LOT of cool things happening here and I don’t have the time and/or energy to write about all of them.

John Troyer Streaming Live from the Solutions Exchange

First, I often find myself watching John Troyer’s live coverage from the Solutions Exchange.  Which is weird because I could literally walk there in about 2 minutes.  But when I’m in my room in between meetings, it’s nice to have it on in the background so I can listen in on all the stuff I’m missing.  And John has been interviewing some very cool people.  

Check it out here … http://www.ustream.tv/channel/vmworld

 

vCloud Express

The vCloud Express is …

The VMware vCloud™ Express service delivers the ability to provision infrastructure on-demand, via credit card, and pay for use by the hour. As a VMware Virtualized ™ service, it ensures compatibility with other VMware environments both internally and with external services.

(Taken from http://www.boche.net/blog/index.php/2009/09/01/vmware-announces-vcloud-express/)

VMware actually demoed vCloud Express with Terramark, one of the service providers in the program.  It was pretty slick to see them simply add some user and credit card information and then spin up a VM quickly and easily on stage. 

Now that I’m having serious power problems in my house because of my home lab (hence the reason this blog keeps going up and down), I really think I’ll be using vCloud Express very soon.

 

SpringSource

VMware’s acquisition of SpringSource was actually announced weeks ago, but this was the first time there was really any lengthy discussion about it.  Frankly, the SpringSource acquisition is probably the thing that I am most excited about.  And I personally believe it will play a significant role in VMware’s future.  There is actually a lot of things I’d like to say about this, but will save it for a later post.

 

Running VMware View / RDP sessions on your iPhone with the Wyse Pocket Cloud client

Given the fact that I “eat the dog food” and actually run my VMware corporate desktop as a VMware View image, and I am also an iPhone user, I think this is super slick and something I know I will use …

 

A Shameless Plug

The folks over at Virtual Strategy Magazine have asked me to do a video blog of the event.  Our first recording was last night and I would guess they’ll have it posted sometime today.  When it’s up on their site, you can find it here … http://www.virtual-strategy.com/VMworld-2009.html

Also, if you’re here at VMworld and undecided about your Thursday schedule, why not come to my session?    I’ll be presenting at 10AM in room 135.  The topic?  How to convert old PCs to thin clients using a Linux OS and the VMware View Open Client.  Hope to see you there!

The editor of Virtual Strategy Magazine recently asked me if I would be interested in committing to a regular monthly column for them.  After thinking about it for a few days, I agreed and my first article in the series was published on Monday.  The title of the column is going to be “Confessions of a Virtualization-aholic” where I’ll talk about real world experiences, with plenty of exaggeration and embellishment for effect.

You can follow my new column by visiting Virtual Strategy Magazine and this month’s article is here http://www.virtual-strategy.com/Features/Sweemer-20090825.html.  I’ve also republished the article below.

 

The data center lights are brighter than the sun. The air is particularly stale and warm today. Beads of sweat are forming on my forehead. One bead grows too large and trickles down my face into my right eye. The salty liquid forces me to squint and temporarily blurs my vision. I hear Jamiroquai’s "Virtual Insanity" playing in the background. Sweet.

My vision returns and I notice a lone leaf of paper, like tumbleweed, dancing across the raised tile floor. It’s caught in the airstream of the temporary fans brought in to combat the blistering heat pouring off the mountains of servers. Off in the distance, I see an Oompa-Loompa doing a perfect pirouette. That’s weird. He’s kinda freakin’ me out. But nevermind. Back to the heat. It’s bad and it’s getting worse.

There are just too many servers. Some renegade, old-school wahoo added more hardware when the existing hardware is way underutilized. Brainwashed products of an era dominated by inefficient operating systems and incompatible applications, you can’t blame them anymore than you can blame a dog for being a dog. But placing blame aside, they have an incredible knack for making a horrible mess of things. I need to put an end to this.

I can feel the temperature rising. The data center technicians have all stripped down to their knickers. Long hair and pot bellies abound. It ain’t pretty. Except for Megan Fox. She’s hot. Evidently, in between filming scenes for her next Transformers movie, she moonlights as a server admin. Who knew? She turns to me and says "Aaron, you have to help us!" There’s an air of desperation in her voice.

"Don’t worry, miss…everything’s going to be alright."

In super slow motion, she flicks back her hair, gives me a sexy smile and a wink. I can’t disappoint Megan! Now I’m on a mission, the heat must come down. Servers must be eliminated. Everything must be virtualized!

It’s show time. Without hesitation I start to P2V everything in sight. The growing crowd starts to cheer and I feed off their excitement. Web, database, middleware, you name it … nothing is safe, nothing is sacred. I’m a P2V monkey, but instead of flinging poo, I’m flinging servers. One by one, each server meets its timely end. The heat is retreating. Trees are being saved! Energy execs are sobbing as their profits diminish with every dead server!

The pile of lifeless hardware is growing quite large. Each server bears an official death certificate which reads "Virtualized by VMware." Most of us cheer. But a few of the old timers hover over the steaming pile of scrap metal screaming "NOOOOOOOOOOOOO!" One of the poor pathetic souls shakes his fist at me and calls me a murderer. I can’t help but snicker.

Just then, Megan breaks through the crowd and comes running toward me. Oddly enough, her knickers are gone and she’s covered by nothing more than a recent issue of Virtual Strategy Magazine. Carryl Roy, Editor in Chief of VSM, yells "We’re only digital, not print!" To which I exclaim, "I really think you’re missing the point!" And at that very moment, Megan leaps toward me and lands with her lips in perfect alignment with mine.

"My hero!" says Megan.

"Sweemer’s the name, virtualization’s my game."

She moves in for a long, wet kiss. But before our lips touch, she pauses, gives me a funny look and says, "BEEP." Huh? That was weird. Well, it’s Megan Fox, she can say whatever she wants ‘cause she’s so hot! But let’s try that again.

"BEEP"

Okay, Megan sweetheart, you really need to stop that. It’s distracting.

"BEEP BEEP BEEEEEEEEEEEEEEEEEEEEEEEEEEP"

My eyelids open and I’m staring at the rear end of my 10-year-old boxer, Lucy, who apparently managed to crawl into bed in the middle of the night. I reach over and grab my alarm clock and mid BEEP, I throw it against the wall. I’m pissed. I lost Megan and woke up to old stinky dog butt. But then I roll over and see my amazing, beautiful wife. Megan’s got nothing on her! And life is good again.

My wife opens her eyes and says "You were dreaming about Megan Fox again, weren’t you?"

"Why do you say that?"

"Because you woke me up 15 times last night screaming, ‘P2V me, Megan! P2V me, baby!’"

I try to conceal my embarrassment as my wife just giggles. But she does a great job of comforting me when she says "Don’t worry, baby. I dream about Megan Fox, too." Sweeeeeet. I love my wife.

Well, time to get up and get ready, I’ve got a long day, which begins with my Virtualization-aholic’s Anonymous meeting. It’s been rumored that Simon Crosby and Steve Ballmer will be there. Care to join us? We meet right here, once a month, at Virtual Strategy Magazine. You’ll laugh, you’ll cry, you may even hurl, as you hear weird, wacky and sometimes seriously disturbing Confessions of a Virtualization-aholic.

I wish I could take credit for the following work, but everything below is brought to you by Michael White.  Michael is a co-worker of mine, an SE out of Canada who we often refer to as the “SRM King.”  He continually impresses me with his ability to crank out a weekly news letter loaded full of great content.  Well last night, he happened to mention I could republish his work on my blog.  Shoot, you don’t have ask me twice!

Keep in mind as you’re reading, everything is a direct cut and paste.  So anything written in the first person (e.g. “I have found …” or “I have decided”) would be referring to him, not me.  I certainly don’t want to take credit for all his hard work!  

If you have any questions or comments for Michael, feel free to leave a message for him.

 

Notes from VMware:

Cluster BP, FT and Issue, HA Issue, vDS Cheat Sheet, vDR Issue, YAPOTAV, vSphere Reference Card, View Design BP, SRM FAQ, and really a LOT more!

 

vSphere Cluster – ESX or ESXi or Mixed – suggestion / recommended best practice

We say that one day that ESX will not exist, and that ESX and ESXi are the same.  Or almost the same.  However, I have found in Host Profiles and FT there is very good reason to not mix ESX and ESXi in the same cluster.  As soon as VMworld is over, I am redoing my mixed cluster to all ESXi (instead of mixed).  First, we all know of the problem I reported some time ago that the 8/6/09 patches for vSphere would break FT in a mixed ESX / ESXi cluster.  There is no short term solution for that. The workaround is to have a cluster that is all ESX or all ESXi.  Second, host profiles have a problem dealing with service console / management network ports.  In theory you can manage that by using a reference server that is ESX and it will translate as necessary for ESXi.  It doesn’t do so well at that.  So using Host Profiles to do a push of a distributed virtual switch (only) ends up causing issues in ESXi consoles.  I ended up doing the ESXi hosts manually.  The real solution to the FT and HP type issues is to have a cluster all ESX or ESXi.  And I am voting for ESXi in my lab.  Make no mistake, if you don’t listen to this you will have some issues that are not pleasant.

 

Using ESXi and ESX and FT in same cluster?  And FT broke with the 8/6/09 patches?

The only solution to this at this time is to separate your ESXi and ESX servers into their own cluster, or upgrade one or the other to be the same as the other – meaning all ESXi or ESX and your problem should go away.  If you have not installed the 8/6/09 patches yet, and you are using FT, and you have ESXi and ESX in your cluster than either change your cluster to be all ESXi or ESX and than install the patches.  Not installing the patches until we fix this is NOT an option.  I have decided, and as mentioned somewhere else in here, to redo my cluster as all ESXi.  It won’t take much time.  Some background on this issue can be found at http://communities.vmware.com/message/1335428#1335428.

Update on odd issue with HA not working if the vSphere ESX console was using certain IP addresses

I hope everyone has already heard that the vSphere bug talked about in http://kb.vmware.com/kb/1013013 and something I mentioned, I think in my last newsletter now has a patch. This is the bug that when a very specific IP address scheme is in use on management ports / service console with no other IP schemes in use and a host crashed, the VM’s that should have been started by HA would in fact not be started at all.  I have not tested the fix, as I am wrestling with SRM and trying to get ready for VMworld.  To avoid this bug, only one of the addresses on your service console or management ports need to be using something outside of the ‘special’ scheme.

vDS Implementation Cheat Sheet

I worked with the distributed switches in the past in a lab sense, but recently. For my future SRM testing, I got it going for real in my lab.  And it was hard, confusing, and not intuitive at all.  So I wrote a cheat sheet so you would not have to suffer.  It is attached.  I have used it a few times and am happy with it so hopefully it will make things quicker and easier.  Let me know if you need improvements or changes in it.  http://www.virtualinsanity.com/wp-content/uploads/vDS-Implementation-Cheat-Sheet-b.pdf

Data Recovery Issue – which stops backups from happening

If you ever have an issue with writing to your destination when doing backups, you may see the restore point in red with a (Damaged) beside it.  This can cause your backup to not work again.   The events part of the Reports will show file access errors – 3902.  The solution to this is not in the documentation for vDR but it is here. Expand the display of restore points to be bigger than the default 5.  I used 25 when I had this issue.  Now click all of the restore points that show as damaged.  Then select the Mark for Deletion button in the top right of the screen.  Now change to the Configuration \ Destinations screen and select the destination that is associated with your backup, and use the Integrity Check option near the top right of the screen.  It will take a while.  Once it is complete with no errors – check the Events view of Reports – you need to restart the appliance.  Now your backups should work!

YAPOTAV – Yet another post on why to attend VMworld

Find this at http://blogs.vmware.com/vmtn/2009/08/yapowtav-yet-another-post-on-why-to-attend-vmworld.html.

New vSphere document reference card

Forbes Guthrie has done a wonderful job on a reference card for vSphere documentation stuff.  It pulls stuff out of the documentation and highlights it as a result.  Very handy and well done.  Find it at http://www.vreference.com/public/vsphere4-notes1.0.pdf

View Design Best Practices training

Would you like to learn more about designing a View infrastructure?  The more people you have that depend on it the more important training and experience becomes.  Get some ideas on design at http://mylearn.vmware.com/descriptions/EDU_DATASHEET_ViewDesignBestPractices_V3.pdf

SRM FAQ online now thanks to Duncan at Yellow-Bricks

This is from information I have shared with Duncan but it is great information and I appreciate him sharing with everyone.  Find it at http://www.yellow-bricks.com/srm-faq/.  Duncan’s web site is one of the few you should read frequently. He is a PSO guy in Europe and is very smart, and knows what to communicate – does it real well and I appreciate it.

 

vSphere and VM snapshots and block size

This is something else that Duncan has done.  There is a behavior difference between 3.5x and 4.0 that could catch someone.  Find out more from Duncan at http://www.yellow-bricks.com/2009/08/24/vsphere-vm-snapshots-and-block-size/.

VMware View Cheat Sheet

I have had some help to update my VMware View Cheat Sheet and it has gone very well.  Our next update of this will have a lot more but this is a good document to get you going with View.  www.virtualinsanity.com/wp-content/uploads/VMware-View-Cheat-Sheet-a.pdf

 

Important patch for Celerra when using NFS with VMware

You can find more information about this at Virtual Geek, but it is important to understand that you need to upgrade your Celerra DART OS before you enable NFS datastores with VMware.  Find out more at http://virtualgeek.typepad.com/virtual_geek/2009/08/important-patch-for-celerranfsvmware.html

Lab Manager 4 Upgrade issue

The installer during an upgrade of LM4 assumes all the default roles are present and unmodified.  If the customer removes or changes any the upgrade installer will fail.

FT – Architecture and Performance

Do you know how to determine how many FT enabled VM’s your vSphere server can support?  Do you know how to design your FT environment for the best performance?  In fact, do you know what the performance overhead for FT is?  All of this and more is answered in http://www.vmware.com/resources/techresources/10058.

How can I determine the exact build number for my ESX 4.0.x hosts?

You can find out the way to determine the build numbers for components of ESX 4.0 hosts at http://kb.vmware.com/kb/1012514

VMware Data Recovery Evaluator’s guide

This is a very nice document for someone who needs some guidance for testing VDR.  It is a quite way to get started.  http://www.vmware.com/resources/techresources/10055.  My preso on VDR at VMworld is a combination of install / config / best practices and it will be very useful.  Look for the session, or the preso after VMworld.  It will fit with this eval guide nicely and is known as BC2142.

 

AppSpeed and Maintenance Mode

Currently AppSpeed has no when to listen to the ESX host it is working on, so when the host tries to enter Maintenance mode it will not be able to since the AppSpeed sensor VM will not listen to it and it will not VMotion off the host.  This is a very high priority for us to fix. You will need to manually turn off this sensor before trying to do maintenance mode.
Need some help searching the VMware KB?  Find it at http://xtravirt.com/xd10112 – some interesting info.

NFS Storage Configuration Help

Do you need some help configuring NFS support for your ESX servers.  There is some help at
http://communities.vmware.com/docs/DOC-7900.  This link has only a little info but it does include some troubleshooting info.

VUM and Cisco – conflict message

I got a conflict message from VUM when I tried to patch recently.  It was a conflict with the Cisco Nexus stuff which I do not have installed.  It turns out that I could just ignore it but it was a little bothersome.  We are going to change that message in the near future to be more informative.  That way if you know you don’t have Cisco (or whatever) installed you can just install with no issues.  The issue is we download all the meta data or patches for ESX without any granularity. So the Cisco patches come done too.  More info can be found at http://kb.vmware.com/kb/1013068.

Suggested VMware Employee Sessions at VMworld

This is a list that one of my co-workers put together. It might give you some ideas of what to look for. 

• Michael White – BC2142 – Data Recovery intro and best practices
• Tiffany To – DV1790 – View TCO-ROI expert
• Mahesh Ramachandran – VM1724 – Capacity IQ Tech Preview
• Chris Rimer – EA2342 – Oracle sessions (especially around questions of Support and Licensing)
• Richard McDougall – TA3438 – vSphere Performance Guru
• Jacob Jensen – TA2103 – Virtual Networking guru (especially around the Cisco v1000)
• Andy Banta – TA3264 – iSCSI Best Practices (THE iSCSI Engineer/Expert at VMware!)
• Kaushik Banerjee – TA2942 – Performance Best Practices (This guys is a genius in performance and on the Perf. core team!)
• Paul Manning – VM3566 – Storage Best Practices (Many of you have been on calls with Paul for storage related topics!)
• Brian CS, Charu Charubal, and Rob Randell – VM2847, TA2544, DV2626, – Security Team extraordinaire
• Mostafa Khalil – TA2509 – Storage Best Practices (Mostafa is one of the first VCDX members!)
• Amir Sharif – TA3195, V13226 – ESXi PM – ESXi sessions
• Monica Sharma – VM2408 – ConfigControl Tech Preview
• Bill Call – VM2657 – LifeCycle Manager Uber-Guru!
• Dean Flaming and Travis Sales – DV2478 – ThinApp (These are some of the best sessions I have ever seen historically from these guys!)
• Gaetan Castelein – EA3605, EA 3606 – Virtualizing Tier 1 applications –
• Srinivas Krishnamurti – VM2280 – Managing VI from your mobile phone!
• Duncan Epping – TA2259 – Expert VI Design (Duncan runs the #1 Virtualization blog “Yellow-Bricks”)
• Dean Yao – BC3369 – FT Real World design
• Howie Xu – TA3521 – vNetwork Troubleshooting (Howie invented the vSwitch! – and wrote one of our TCP/IP stacks)
• Banjot Chanana – BC3425 – High Availability Futures
• Nicholas Jacques – PA4694 – AppSpeed PM
• Eric Horschmann – TA3880 – vSphere vs Hyper-V/XenServer
• Warren Ponder – DV2697 – View /VDI PM
• Mike DiPetrillo – TA3326 – Cloud (Mike is another uber-rock star and talks all things Cloud!)
• Rahul Ravulur- -VM4380 – vCenter PM covering future of vCenter
• Naeem Malik – VM3609 – Capacity Planner expert
• Aaron Sweemer – DV3567 – How to convert old PCs to Thin Clients using a thin Linux OS and VMware View Open Client.

**** Reminders ******

• vSphere Upgrade Center – http://www.vmware.com/products/vsphere/upgrade-center/
• vSphere Support Center – http://www.vmware.com/support/vsphere.html
• vSphere-land documentation links – http://vsphere-land.com/vsphere-links/documentation-links-2.html
• Connections and Ports in ESX / ESXi – http://www.virtualinsanity.com/esx-connections-and-ports/
• vSphere-land download links – http://vsphere-land.com/vsphere-links/download-links-2.html
• vSphere and other VMware products compatibility matrix – http://www.vmware.com/resources/compatibility/docs/vSphere_Comp_Matrix.pdf
• VMware TCO / ROI Calculator (for VI 3 consolidation, virtual lab automation, and VDI savings) – http://www.vmware.com/products/vi/calculator.html
• VMware Cost per application calculator – http://www.vmware.com/technology/calculator/costperapp.html
• Why choose VMware? – http://www.vmware.com/technology/whyvmware/
• Server Virtualization Validation Program – you can check the status of servers or virtualization technology approved by Windows in the SVVP program – http://windowsservercatalog.com/svvp.aspx?svvppage=svvp.htm
• Support policy – http://support.microsoft.com/kb/957006
• VMware Visio shapes at http://www.veeam.com/vmware-esx-stencils.html and http://engineering.xtravirt.com/products/vi3-visio-action-pack.html
• VMware graphics for PowerPoint – http://viops.vmware.com/home/docs/DOC-1338
• License issues – call 877-486-9273 and select option 1 – or vi-hotline@vmware.com
• Interesting SRM links – http://tendam.wordpress.com/srm-links/
• Practical DR (before SRM) – http://www.vmware.com/files/pdf/practical_guide_bcdr_vmb.pdf
• VMware Security Center – blogs, RSS feeds, and more – http://www.vmware.com/security
• Enhanced VMotion Compatibility (EVC) processor support – http://kb.vmware.com/kb/1003212 
• Patch Notice (via email) is available at http://www.vmware.com/support
• Reporting bugs to VMware – http://www.vmware.com/support/policies/defect.html
• Requesting a specific feature – http://www.vmware.com/support/policies/feature.html
• License File checker – http://www.vmware.com/checklicense/  – doesn’t work with SRM license codes yet!
• Dell and VMware alliance – includes white papers and VMotion compatibility info – http://www.dell.com/content/topics/global.aspx/alliances/en/vmware_resources?c=us&cs=555&l=en&s=biz&~section=004  
• HP Certification matrix for VMware on Proliant – including VMotion compatibility info – http://h18004.www1.hp.com/products/servers/software/vmware/hpvmotion-compatibility-matrix.html
• Emma Explains Microsoft Licensing in Depth – A well done UK site that actually seems to make sense when it comes to MS license info – http://ladylicensing.spaces.live.com/default.aspx