Author Archive
April 21st, 2009 | 
Author: I was at the Louisville VMUG on Friday talking about Troubleshooting ESX. In my preparation for the event, I was looking for a good PowerPoint presentation I could reuse and I stumbled across a sweet little gem of a document. Dudley Smith, a VMware Technical Account Manager (TAM) out of Virginia, created a cool one page Mind Map for Troubleshooting ESX. Does it address every potential issue you’ll come across? No, of course not. But it’s a heck of a good place to start. One look at his Mind Map and I thought to myself, “that would be a great thing to have printed out and hanging over every VMware admin’s desk.”
Well, long story short, I snagged it and threw it up on the big screen behind me as I was presenting. During the presentation (and many times since the presentation) I had many requests to post the PDF for download.
But since I couldn’t just start passing out someone else’s work as my own, I sent Dudley a quick email asking for permission to distribute. He responded by saying, “Sure, publish away! You might enjoy this too… ” Attached was another one page document that visually shows the TCP/UDP ports leveraged in VI3.5. Nice! Again, another great document to have printed out and hanging over your desk, IMHO.
So, courtesy of the author, Dudley Smith, here are two documents that I would recommend you add to your tool belt. (click the images to download the PDFs)
If you like them, leave a comment for Dudley.
Everywhere I turn, everybody’s talking about cloud computing. And I agree with Mike DiPetrillo, very few people understand what the cloud is today and what it could and/or should be tomorrow. I’ve kept silent on the topic of cloud computing on this blog until now, mainly because I prefer to know what I’m talking about before I put something out there for the world to see 
But now that I believe I’ve got a good grasp on it, I’d like to share a few of my thoughts …
What’s in a name?
First of all, I don’t like the name "cloud." I think it’s a stupid name. Let me explain why. I was with a customer the other day talking about the future of cloud computing and he said, "man, what a horrible name, it just sounds like the most insecure, undefined, unmanageable place … why would I ever want to put my apps in a cloud?" I couldn’t agree more. For years we’ve been preaching about putting applications in data centers, and the importance of securing data with things like firewalls and intrusion detection. You don’t even have to know what a firewall is, what it does or how it works, but the name just sounds safe. And in my opinion, referring to the next generation of computing as the "cloud" would be like if someone called the first firewall "come on in" and the industry adopted it.
To me (and many of the customer I’ve spoken to), the word cloud conjures up images of dancing ferries, unicorns and other mythical creatures all prancing around in some fluffy place somewhere northeast of never-never-land. But alas, cloud is the name the industry seems to have settled on. And since, at this point, I don’t have a better name to offer, I too will refer to it as the cloud. But for the record, I think the name should be something strong and manly, like Spike, or Butch, or Krull the Warrior King!
By the way, I like the name vShield Zones, a new VMware offering that will logically partition a cloud. I think a name like this conveys a much better image about where an organization’s apps and data “live.”
What is the cloud? An explanation for the business owner (IT people, you may want to stop reading now)
Most people still think about IT as servers and networks and storage, all powered by a bunch of computer geeks that hibernate in a data center or crawl around under desks when a computer breaks. Actually, they probably still think about IT this way because this is by and large the reality for almost all organizations.
But let me ask you this … why are thinking about IT at all? Unless you’re in the IT / Hosting business, shouldn’t you be thinking about, um, your business? You don’t think about electricity or the plumbing do you? No, you don’t. Unless of course, the electricity goes out or the toilets backup … then you can’t stop thinking about them! But I’m pretty sure an advertising company, for example, doesn’t have an electrician or a plumber on staff.
When cloud computing is fully realized, IT should be very much the same thing. It is a tool that should serve you and your business, not the other way around. Now, it’s not like servers and networks and storage and IT geeks like myself will cease to exist. No, we’ll still be here, but you won’t think or care about us anymore. *sniffle*
To understand the cloud, you need to STOP thinking about the plumbing behind the applications (i.e. servers / networks / storage), and you need to START thinking about what matters most, the applications and data you need to run your business.
When the cloud is fully realized, your applications will be always on, extremely reliable, accessible anytime and from anywhere, and they will “live” in a cloud. Now that cloud might be external to your organization, or it might be an internal cloud, built on your existing infrastructure. Either way, you’ll be able to self-provision new applications with a few clicks of a mouse and pay only for what you use.
Sounds pretty good, yes? Don’t go beating up your IT department just yet. While many pieces of cloud computing are in place, the cloud is still forming. Standards are being hammered out, committees are being formed, and it seems like everyone has a SOAP box, apparently even me (yes, I meant to capitalize SOAP, it’s a little joke for the developers)
I have more thoughts. More to come.
Me too. Actually, I’ve been studying for a few weeks now. A while back, a friend of mine and fellow VMware SE (and SRM super freak), Michael White, turned me on to Evernote, an awesome tool for capturing my web research and lab notes (among other other things). So, as I’ve been studying for the exam, I’ve been saving everything in Evernote, which I have installed on my iPhone and my XP desktop. (Now, if only they made a version for Linux, I’d be all set. *sigh* Yet another company with a great app that chooses to ignore the Linux community).
Anyway, now I just need to go back and do some simple formatting and VOILA!, I’ve got quite a few ready-made blog posts. Which is handy, given my recent commitment to a post a day.
So if you too are studying for the exam (or planning to) and looking for study material, feel free to check back here from time to time over the next few weeks. In addition, you might want to check out the following list, which contains everything I’m using to prepare.
- Hands down, the best “cheat sheet” for VI3 is the vmreference vi3 card, by Forbes Guthrie
- Duncan Epping over at Yellow Bricks has put together a great list of documents in his blog post VCDX Design Exam, how to prepare?
- Looks like Jon Owings is also studying for the exam and has a number of good posts on his blog, 2 VCP’s and a Truck
- The best, and most comprehensive study guide I’ve found is the VMware Enterprise Administration Exam study guide 3.5 by Peter van den Bosch.
- Oh, and don’t forget the official VMware Enterprise Administration Exam Blueprint v3.5
Also, send me an email if you’d like to be part of a weekly online VCDX study group I’m trying to put together.
Wow! I can’t believe it’s been almost two months since my last post! Sorry for my extended absence. I’ve been super busy with VMware events, customer presentations and meetings … oh yeah, and there was a nice ski trip to Vail too. Time flies when you’re having fun
Over the past few weeks, with the little spare time I had, I actually completed my conversion to a virtual desktop. So now, my corporate VMware desktop is 100% in a VM, always on and "lives" on the virtual infrastructure in my home lab. And with my shiny new AT&T 3G wireless laptop card, I can access it anytime, anywhere (though admittedly, this is a last resort).
You might have guessed by the title, for this post I’ll focus on my network setup. I stress the word “my” because when I first started to write this section almost a month ago, I had a lot of trouble trying to address all possible network configurations (or at least, a good majority of them). Finally I gave up, realizing this was an impossible task. There are just way too many options. So, I’m simply going to document my network. If you have VDI network configuration questions that aren’t answered in this section, email me directly (aaron at sweemer dot com) and I’ll be happy to help out.
I think the best place to start would be showing you a high level diagram of my network (click the graphic to see the full image).
As you’re reviewing the diagram, here are a couple things to keep in mind:
- All IP addresses have been changed and domain names have been removed for security purposes. Hostnames, however, remain unchanged.
- You do not need to have a similar setup. In fact, you can have as little as a single physical server with local storage. You might not be able to get the full benefits that a fully loaded virtual infrastructure can provide (e.g. VMotion, DRS, HA). But if you’re just looking to test out virtual desktops with VMware View, you can certainly go with a slimmed down environment.
- The configuration of the physical servers (cincylab-esx1, 2 and 3) as well as the iSCSI SAN (cincylab-ts1) will be addressed in the next section.
- The ISP router is a fairly unintelligent device which I’ve configured to simply forward all network traffic to cincylab-rtr1. As such, I won’t address the configuration here.
I love Visio diagrams, they make everything look so pretty and shiny! What does this actually look like? Here’s a photo of my lab …
Notice the PC on the right (cincylab-rtr1)? That’s an old Gateway I had lying around, which has a single 2.2GHz processor, 1Gig of RAM and a single 100Mbps NIC. I installed Ubuntu server 8.04.1 (kernel 2.6.24-19-server) on it and made it the gateway between my lab and the DMZ (aka, my home network). It’s on this PC that I route between VLANs, terminate external VPN connections and run my DHCP server. Additionally, it’s where I run scripts that continually scan for changes to my public IP address and when necessary, automatically updates my dynamic DNS provider.
Since this post is getting long, I decided to break this section into two parts. In part two, I’ll walk through the all the configurations of cincylab-rtr1 and cincylab-sw1. And before you say anything … no, it won’t be another two months before I post part two
I finally got the VMware Infrastructure Client running directly on my Ubuntu Linux. I use the word running, because I’m not prepared to use the word working just yet. Though the application seems fairly stable and most feature / functionality I’ve tested thus has worked, I’ve already found two bugs, one of which will hang your entire terminal. And believe me, I’ve only just begun to test, so who knows how many more bugs I’ll find?
But until today, I couldn’t even get the client to launch and I haven’t been able to find anyone else that has been able to do so either. So I’m going to consider this a minor success and a step in the right direction. But just to be clear, I would NOT advise you start managing your environment with the VIC on wine just yet!
OK, here’s the proceedure I used get this running.
- My environment (I haven’t yet tried this proceedure on any other combination of Linux and Wine):
- ubuntu 8.10 (kernel 2.6.27-11)
- wine-1.1.14
- Download and install the latest version of winetricks.
- Run winetricks and select ONLY the following options: dotnet20, ie6 and winxp. Now, for future reference, I believe the bugs I’ve already found can be cured with a few more options. But, one step at a time. I also know that a couple options will crash the application.
- Download and install the VMware Infrastructure Client. You can get this by going to http://<ip of your vCenter server>.
- Here’s the critical part, you need to modify your vCenter (or ESX) server to accept both HTTP and HTTPS. By defult, vCenter and ESX will accept HTTP requests, but they are immediately redirected to HTTPS. And currently, this will break the VIC on wine. Do NOT do this on a production vCenter server!! To modify your vCenter server, do the following:
- On your vCenter server, go to C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter
- Copy proxy.xml to proxy.xml.bak
- Open proxy.xml with a text editor
- Find the lines with httpsWithRedirect and replace with httpAndHttps
- Restart the “VMware VirtualCenter Server” Service
- Back on your linux workstation, go to ~/.wine/drive_c/Program Files/VMware/Infrastructure/Virtual Infrastructure Client/Launcher/
- Execute wine VpxClient.exe and you should see the following:
- Make sure you put an http:// in front of the IP or DNS name of your vCenter server. Otherwise, it will try to connect via HTTPS and again, this is currently problematic.
- That should do it.
OK, here are the bugs that I’ve found so far (other than SSL, which I’ve already mentioned):
- Using the right click menu will freeze your screen about 50% of the time. When it freezes, you’ll have to connect to another TTY, find the process and kill it. But the alternative menus seem to work. For example, if you right click on a host and click “New Virtual Machine … ” your screen will likely freeze. However, if instead you click the “New Virtual Machine” link on the Summary tab, the New Virtual Machine Wizard will properly launch.
- The New Virtual Machine Wizard will not advance past the Virtual Disk Capacity step. It produces the error “The disk capacity entered was not a properly formatted number or was out of range …” It gives this same error no matter what value I enter. Actually, simply clicking Cancel will produce the error. Weird.
- The Getting Started tab correctly renders the proper HTML, but the viewing area is about 100 x 100 pixels and not adjustable. (This is nothing more than an annoyance).
- You can create a folder in a datastore, but you can’t delete one. Deleting files on a datastore seems to work fine.
Most of the navigation (other than the right click menu I mentioned above), seems to work well. VMotion worked fine. Configuring HA and DRS worked fine. Performance stats rendered fine. But I’ve got a lot more to test and I’ll update again with my test results as I progress.
Thanks to Dan Kegel (www.kegel.com) and Jeff Warnica (don’t know his website/blog) for your help and pointing me in the right direction!
Here are a few more screenshots of the client in action. Click on each image for the full scale picture.
Performance stats, and completed VMotion (in Recent Tasks at bottom)
Datastore browser
This is the first post in my e.t.d.f (eating the dog food) series. I had hoped to get this first one typed up quickly. But instead, my week was consumed with customer meetings and the logistics around rescheduling a big event I am putting on for the Commonwealth of Kentucky. (Thanks to a ton of snow and ice, we had to postpone it a few weeks.) So now that I’ve got a little free time again, let’s get started.
In case you’re just joining us, as a quick recap, this series will document my conversion to a virtual desktop. Meaning, when this series is over, I will no longer be tied to any physical laptop, PC, mobile phone or whatever. My dedicated VMware corporate desktop will live full-time on the virtual infrastructure in my house. I will then connect to my desktop remotely (whether I’m a few rooms away, or a few hundred miles away) via a VMware client or a web interface. Sounds easy, right? Well it really is, though we will have some challenges around multimedia, working offline, and accessing some local devices … all of which will be addressed as we progress.
But for now, first things first. I want to establish some requirements and goals.
- The desktop needs to be always on. When connecting to my desktop, I don’t want to wait for it to power on. I want to simply launch the client and connect.
- I want to be able to securely connect to it anytime, anywhere. This might sound obvious, but my home Internet connection has a dynamic IP address. How do I connect remotely when my IP address changes? And what about security? Will other people be able to access my desktop from the web too?
- My desktop has to be at least as performant as my current local desktop. The only exception I’ll make here is for high end mulitmedia.
- I want my desktop completely maintenance and worry free. To me, that means:
- My data is always backed up.
- My desktop can be destroyed by a hacker or virus (or my own stupidity) and restored to its previous state with little effort and under 30 min.
- Updating and patching my desktop has to be done automatically, or at least, a fairly painless process.
- I want to be able to carry my desktop on a USB stick or LiveCD. (If you don’t know what this means, I’ll be covering this in more detail when we discuss options for connecting to the desktop)
- Finally, I want this to be scalable. Keeping in mind that this series aimed to also serve as a loose guide to a Proof of Concept, I need to be able to add users and deploy desktops quickly and with minimal effort.
If at the end of this series I have met these goals, then I will consider my conversion a success and my desktop will permanently remain virtual.
So now let’s discuss what we’ll need to make this all a reality.
Server Requirements
VMware virtual desktops run on ESX, so ideally you would want at least one server that is currently on the HCL (Hardware Compatability List). If you have a server but can’t find it in the “Systems” section of the HCL, then search for the components of your server in “I/O Devices.” If your components are listed, there’s a good chance that ESX will run just fine.
If you don’t have a server and don’t have a big budget to buy a new one, then have a look at Mike D’s Building a $500 ESXi Host. Or another great resource is VM-Help.com, which maintains the Unofficial ESX Whitebox HCL. Keep in mind that anything not on the official HCL won’t be supported by VMware.
As for me, I have three servers in my lab, all of which are HP ML150′s with 8GB of RAM and 300GB of local storage. Each server is connected to a Buffalo TeraStation iSCSI SAN with close to 1TB of storage.
Network Requirements
At a bare minimum, we’ll need a dedicated Internet connection. Mine is a business grade, cable modem service provided by Cincinnati Bell with 5Mbps down, 1Mbps up, and a single dynamic external IP address. We will also need an internal DHCP server with a range of IP addresses set aside for our desktops. If you’re setting this up as a Proof of Concept for your company, you probably already have a solid Internet connection. So make sure you’ve got DHCP enabled with enough available IPs for the number of desktops you plan to deploy.
VMware Software
If you haven’t already done so, go sign up for the free 60 day evaluation of VMware View and download the software bundle. It will contain everything we’ll need from VMware for this project.
Desktop Operating System
My corporate desktop is Windows XP, so I’ll stick with that. Make sure you’ve got the proper Microsoft licenses secured before deploying desktops.
That’s about it for our planning session. Now for our homework assignment. In the next day or two, please be sure to do the following:
- Identify at least one server upon which VMware ESX can be installed. Two servers would be better, if possible.
- Make sure you’ve got an internal DHCP server set up.
- If you have a separte network team, try to secure a dedicated external IP address.
- Download the 60 day evaluation of VMware View.
- Download the VMware View Manager Administration Guide. It’s close to 200 pages long, so don’t worry about reading it now. And really, following this series will cover most of what’s in the guide anyway. But it’s nice to have handy as we move along.
On a final note, I’ll be making a separate page next to the “About the Author(s)” page at the top, for quicker access. See ya next time
I got a comment on yesterday’s post from Rodney Haywood, a.k.a. Rodos, a fellow virtualization (or, as he would probably type it, virtualisation) blogger, that said …
Cool. Maybe you were inspired by my post http://rodos.haywood.org/2009/01/life-without-v…
Be great to hear about your progress and journey. It can be done. I miss my VDI!
Now, before I officially started the Eating the dog food series, I was going to type up a post on all the reasons why I want to run a virtual desktop in the first place (other than “practicing what I preach”). But after reading Rodos’ post last night, I realized I could save myself the time by just pointing everyone to his blog. He pretty much sums up everything I was going to say anyway So check out his post Life without VDI sucks if you want to know why I’m eager to run a virtual desktop.
Rodos, great work and thanks for the comment. And where did you get that sticker?! I want one!!
I’m a big believer in “practice what you preach” and I try to do my best to live by this motto. Now, let me be the first to say that I also believe that by being human, almost by definition, makes me a hypocrite and I am certainly not saying that I’ve never failed at this. All I’m saying is that I try my best.
All day long, as a VMware Systems Engineer, I spend my time evangelizing the wonderful things that virtualization and VMware software can do for companies. I can do this because I’ve been working with VMware software almost since VMware began. And I’ve helped implement virtual infrastructures for many different clients, long before actually being employed at VMware. I even helped build VDI’s (Virtual Desktop Infrastructures) before VMware had a VDI product offering.
VDI is really starting to take off this year and most of my conversations these days are about giving corporate users a virtual desktop. And while I’ve helped build VDI solutions before, am I actually using a virtual desktop myself? No! Well technically I do, given that I run my corporate destkop in VMware Workstation on top of Ubuntu Linux. But this is not the same solution I’m preaching to my customers. A virtual desktop running locally is a very different beast than running a virtual desktop remotely on VI3 via a desktop broker. Normally, I would be able to rationalize this and say “I’m not a corporation that has the virtual infrastructure to run a virtual destktop.” But this is no longer true. I have a dedicated Internet connection and at least three good servers always running in my lab.
So, I’m about to start practicing what I preach and “eating my own dog food.” I’m going to begin a series of blog posts that document my conversion to a dedicated virtual desktop. I hope that this can also serve as inspiration, and possibly even a guide to a proof of concept, for those who are interested in virtual desktops. I’m sure this blog series evolve as I progress, but I envision the upcoming posts will look something like this …
- Planning the environment
- Setting up the network and dedicated remote access
- Getting the virtual infrastructure set up correctly
- Setting up VMware View (the broker)
- Preparing the desktops
- ThinApp’ing my applications
- Troubleshooting, tweaking and optimizing
Again, as I progress, this could all change. But for now, this is the blog series I’ll begin in the next day or two and hope to complete over the next few weeks. I hope you join me. If you plan to follow along and, better yet, get involved, please sign up for a Disqus account. It’s the system I use for commenting and discussions.
By the way, do you know who originally coined the term “eating our own dog food?” VMware’s CEO, Paul Maritz. But he didn’t coin the term while on duty at VMware. From Wikipedia …
In 1988, Microsoft manager Paul Maritz sent Brian Valentine, test manager for Microsoft LAN Manager, an email titled “Eating our own Dogfood” challenging him to increase internal usage of the product; from there, the usage of the term spread through Microsoft, as chronicled in the book Inside Out: Microsoft—In Our Own Words (ISBN 0446527394)
The irony is so thick, you can cut it with a knife!!
It’s no secret that I’m a big Linux fan. And us Linux weenies frequently like to point and snicker whenever we see the infamous BSOD or any other Microsoft flaw we stumble across. But in the spirit of fairness, I thought I’d share this picture I took on a recent flight …
Evidently, the entertainment systems on Delta airlines are powered by Linux (very cool). BUT, on this particular flight, my screen — and only my screen — was in a never ending reboot!! That made for a fun 4 hour flight and I can tell you was NOT happy with the Penguin at the and of that trip. Of course, I think it’s funny now, so I thought I’d share.
It depends But to help you better guesstimate how much storage you’ll save with View Composer, let’s take a look at the actual storage savings in my home lab. Keep in mind that VMware View isn’t just something I’m playing around with. There are currenlty 5 other VMware Systems Engineers using my lab on a regular basis. And I use VMware View to give each lab user a dedicated, persistent desktop VM which they connect to remotely via the View Client. And from their dedicated lab desktop, they have full access to the lab environment I’ve built. So, while my home lab environment may be small and by no means a production enviornment, I’m still using VMware View as a “production” application, so to speak. It’s the one application in my lab that needs to be up, and it gets heavily used.
Let’s first look at how much storage my virtual desktop infrastructure would require without the use of View Composer. As I said, each lab user has a dedicated desktop, which has a 12G partition for OS and a 4G partition for user data. Also, each desktop has 1G of memory with zero memory reservation, which transltates into a 1G swap file. That’s a total of 17G per user. Right now there are 5 other remote users, plus I have a desktop for myself. And I have 2 desktops on standby for new users (so they don’t have to wait for a new VM to be built upon first login). So, that brings the total virtual desktops to 8. Therefore the total storage requirement for virtual desktops in my lab without View Composer would be 136G (17G * 8 desktops).
Now let’s look and see how much space is actually being used in my lab. Here’s a cut and paste showing the disk usage of the volume holding the desktop VMs …
[root@cincylab-esx1 cincylab-vol1]# du -sh *
3.1G labuser-01
2.9G labuser-02
3.6G labuser-03
2.6G labuser-04
3.1G labuser-05
1.9G labuser-06
1.5G labuser-07
1.5G labuser-08
2.0G replica-774c678b-e6b5-495a-b8ff-
448K source-lc-8ae5400e-4af3-4a09-8d8
13G parent_desktop
[root@cincylab-esx1 cincylab-vol1]#
The grand total here is 33.2G, which is about a 75% reduction in storage. Not bad. The storage reduction is achieved through two technologies, Linked Clones and Thin Disks.
Linked Clones
In my environment, the virtual disks for the desktops are located in the labuser-0* directories (this is done automatically for me during the deployment process). These virtual disks are not thick, monolithic disks like they used to be in the previous version. Rather, they are delta disks, which only store data differences between the desktop OS and the OS of the parent VM. In the cut and paste above, notice the 13G parent_desktop? That’s my starting point and contains my golden image. That direcotry also contains a snapshot, which I took when I was ready to being deploying desktops. The replica-774c678b-e6b5-495a-b8ff- VM is derived from this snapshot and ultimately serves as the parent VM (for now, this can change over time) of the labuser-0* desktops. Linked Clones have other benefits too, especially around patching and updateing. But that’s a topic for a later post.
Thin Disks
Remember how I said the users need a 17G? (12G for OS, 4G for user data and 1G for swap) But did you notice that the parent_desktop directory is only 13G in size (12G OS plus 1G swap)? From the administration guide “Thin provisioned disks (thin disks) are used by the linked clones to store user data, and are not linked to the Parent VM.” So I didn’t need to include the user partition in the parent VM. The user partition is handled for me when a desktop is deployed and included with each of the user desktops. User data disks are persistent and they are thin, meaning they occupy no more space than the data requires. In my environment, looking at the virtual disks for labuser-03:
[root@cincylab-esx1 labuser-03]# ls -lah
total 3.6G
drwxr-xr-x 1 root root 1.8K Dec 27 09:23 .
drwxr-xr-t 1 root root 3.7K Jan 5 06:10 ..
-rw——- 1 root root 1.0G Dec 27 09:23 labuser-03-a2263868.vswp
-rw——- 1 root root 8.5K Dec 27 09:23 labuser-03.nvram
-rw——- 1 root root 4.0G Jan 5 08:07 labuser-03-vdm-user-disk-D-flat.vmdk
-rw——- 1 root root 443 Dec 27 09:24 labuser-03-vdm-user-disk-D.vmdk
-rw——- 1 root root 75 Dec 27 09:21 labuser-03.vmsd
-rwxr-xr-x 1 root root 3.9K Dec 31 06:58 labuser-03.vmx
-rw——- 1 root root 265 Dec 31 06:58 labuser-03.vmxf
-rw——- 1 root root 2.5G Jan 5 08:07 replica-774c678b-e6b5-495a-b8ff–cl1-delta.vmdk
-rw——- 1 root root 379 Dec 27 09:24 replica-774c678b-e6b5-495a-b8ff–cl1.vmdk
-rw-r–r– 1 root root 62K Dec 27 09:21 vmware-1.log
-rw-r–r– 1 root root 36K Jan 5 06:21 vmware.log
[root@cincylab-esx1 labuser-03]#
Look at the size of labuser-03-vdm-user-disk-D-flat.vmdk, see how the file system “thinks” it’s 4.0G? It’s really not. A closer look at disk usage reaveals:
[root@cincylab-esx1 labuser-03]# du -sh *
1.0G labuser-03-a2263868.vswp
64K labuser-03.nvram
43M labuser-03-vdm-user-disk-D-flat.vmdk
64K labuser-03-vdm-user-disk-D.vmdk
64K labuser-03.vmsd
64K labuser-03.vmx
64K labuser-03.vmxf
2.6G replica-774c678b-e6b5-495a-b8ff–cl1-delta.vmdk
64K replica-774c678b-e6b5-495a-b8ff–cl1.vmdk
64K vmware-1.log
64K vmware.log
[root@cincylab-esx1 labuser-03]#
It’s actually only taking up 43M, not 4G. And a quick look inside the VM confirms it too “thinks” it has 4G:
So that’s my real world example of how much storage I’m saving with View Composer. How much will you save? Again, it depends There are things that I haven’t discussed in this post (e.g. desktop refresh, desktop recomposition and desktop rebalance), which will also affect storage savings. Also, using ThinApp’d applications that live on a file share (as opposed to putting them in the OS) could have a big positive impact on storage savings. In my lab, with only linked clones and thin disks, I’m getting a 75% reduction. With a little more effort and planning on my part, I’m confident I could achieve 85% or better.
I’ve been trying to get the VMware Infrastructure Client to run on my Ubuntu laptop via Wine. I’ve been making progress and getting closer, but it’s still not working. After playing around with native vs. builtin DLL’s and a couple of other configuration items, I can successfully install and launch the VIC. I have no issue getting the familiar VIC login prompt. But after entering the IP address of my Virtual Center server, user name and password, I get the error “The client could not validate the server’s SSL certificate,” as you can see here:
I can get farther by putting an http:// in front of the IP address, forcing the VIC to bypass SSL, and it appears the client is attempting to connect to my Virtual Center server. However, ultimately the application crashes with the following trail of console messages.
fixme:shell:URL_ParseUrl failed to parse L”"
fixme:shell:URL_ParseUrl failed to parse L”VirtualInfrastructure.25u2″
fixme:shell:URL_ParseUrl failed to parse L”System.Windows.Forms”
fixme:shell:URL_ParseUrl failed to parse L”System”
fixme:shell:URL_ParseUrl failed to parse L”VpxClientCommon.25u2″
fixme:shell:URL_ParseUrl failed to parse L”VMware.CustomControls.25u2″
fixme:shell:URL_ParseUrl failed to parse L”System.Drawing”
fixme:shell:URL_ParseUrl failed to parse L”VimSoapService.25u2″
fixme:exec:SHELL_execute flags ignored: 0×00000500
fixme:ole:Context_QueryInterface interface not implemented {000001da-0000-0000-c000-000000000046}
fixme:advapi:RegisterEventSourceW ((null),L”.NET Runtime”): stub
fixme:advapi:ReportEventW (0xcafe4242,0×0001,0×0000,0x000003ff,(nil),0×0001,0×00000000,0x7d94d124,(nil)): stub
err:eventlog:ReportEventW L”.NET Runtime version 2.0.50727.42 – Fatal Execution Engine Error (7A05E2B3) (80131506)”
fixme:advapi:DeregisterEventSource (0xcafe4242) stub
fixme:advapi:CheckTokenMembership (0x27c 0x1cef20 0x7d94c778) stub!
fixme:shell:URL_ParseUrl failed to parse L”System.Web.Services”
fixme:shell:URL_ParseUrl failed to parse L”TransportInterfaces.25u2″
fixme:shell:URL_ParseUrl failed to parse L”System.Xml”
fixme:imm:ImmDisableIME (-1): stub
fixme:thread:NtQueryInformationThread Cannot get kerneltime or usertime of other threads
fixme:thread:NtQueryInformationThread info class 9 not supported yet
(repeated 10 times)
fixme:advapi:RegisterEventSourceW ((null),L”.NET Runtime 2.0 Error Reporting”): stub
fixme:advapi:ReportEventW (0xcafe4242,0×0001,0×0000,0x000003e8,(nil),0×0008,0x000000f6,0x3009a1b4,0x7e1b55e0): stub
err:eventlog:ReportEventW L”vpxclient.exe”err:eventlog:ReportEventW L”2.5.0.64227″
err:eventlog:ReportEventW L”48db0525″
err:eventlog:ReportEventW L”mscorwks.dll”
err:eventlog:ReportEventW L”2.0.50727.42″
err:eventlog:ReportEventW L”4333e7ec”
err:eventlog:ReportEventW L”0″
err:eventlog:ReportEventW L”000b333f”
fixme:advapi:DeregisterEventSource (0xcafe4242) stub
fixme:thread:NtQueryInformationThread info class 9 not supported yet
(repeated 10 times)
As you can see here:
I’ve also tried this with a ThinApp’d version of the VIC and basically get the same results. I realize I’m probably beating my head against a wall, looking for a solution that probably just doesn’t exist. But in the off chance someone has figured this out, I want to see if anyone has progressed farther than I. Here’s my setup …
Version of OS: Ubuntu 8.10 (kernel 2.6.27-9)
Version of Wine: wine-1.1.10
Anyone have any ideas?
Let me show you a slick application I stumbled across about 6 months ago. My HP all-in-one USB printer is certainly a handy device, but being bound to the thing via a USB cable was driving me nuts. I wanted to have full control (which meant a print server wasn’t going to cut it) and I wanted that control from anywhere in my house over my wireless network. Now, there are a number of products on the market that can do this. As an example, AnywhereUSB from Digi is one such product which seems to have a good reputation and from what I’ve seen, it works well. But, I was in one of my moods and I was bound and determined to find a free solution. In my searches I found a few software products and tried the 30 day demos. They all worked well but they weren’t free and they were all products for Windows. And since my WindowsXP desktop is actually a VMware Workstation virtual machine running on top of Ubuntu Linux, what I really needed was a solution for Linux.
I knew that if I could find something for Linux I would kill two birds with one stone. Because not only would I be able to connect USB devices to my Linux OS, but as an added benefit, my WindowsXP virtual machine would see the connected device just as if it were actually connected … no additional software needed for Windows! Eventually I stumbled across USB Server (+ USB Client) for Linux Beta which is offered as a freeware product from IncentivesPro (http://www.incentivespro.com). After playing with the product a bit and creating a few custom scripts to further automate the connecting and disconnecting of USB devices, I found the solution I was looking for! And I can tell you, I use the product on a daily basis. Here’s a quick look at my setup.
1) I have an HP all-in-one USB printer connected to a Linux server running USB Server for Linux.
2) On my laptop, running Ubuntu 8.04 I have the USB Client for Linux installed.
3) To automate the process of connecting to the server and attaching the USB devices, I created a Bash script called connect_usb. Simply running this script produces the following output …
asweemer@cowbuntu:~$ connect_usb
Restarting USB Server on sweemserv … Success!
Restarting local USB Server … Success!
Connecting to USB Server on sweemserv … Success!
Looking for USB Devices on sweemserv … Success!
Found the following USB Devices on sweemserv:
1: USB Server on sweemserv:32032 status: [connected]
‘–> 5: USB Device: Officejet 5600 series HP – Composite USB Device
busid: 1-1 hwid: 03f0-4f11
speed: [full] status: [device is connected]
Connecting to the USB Devices on sweemsrv … Success!
asweemer@cowbuntu:~$
After I see this, I have full control of the device just as if I had the USB cable plugged directly into my laptop.
4) I already mentioned that I run my VMware corporate XP desktop as a Workstation 6.5 instance. The USB Client presents the USB devices as local, so I connect to them in Workstation as if they were directly connected devices. Check out the following screenshot …
See the last line “Hewlett-Packard Officejet 5600 series”? That’s the printer. And believe me, it’s not local like the other devices in the list, but VMware Workstation doesn’t know the difference. And when I connect the device to the virtual machine, WindowsXP doesn’t know the difference either.
6) That’s it! Works like a charm everytime
If you’d like a copy of the Bash script, let me know and I’ll update the post. Also, I’d be interested in any other unique ways to handle USB Redirection, so please comment if you have a solution.
I’ve been using Perl for close to 10 years now, so my natural tendency has been to use Perl when creating automation scripts for virtual infrastructure. Now for some reason, there has been a significant increase in the amount of questions I get about scripting repetitive virtual infrastructure tasks. In fact, just yesterday a customer asked me “is there an easy way to rescan the HBA’s in all off my (many) hosts?” With the GUI, this could translate into 100s of clicks and a very boring hour or more of work. Of course, a properly written script could easily handle this, requiring a mere 30 seconds of his time. And again, a few months ago I would have answered the question by pointing him towards Perl.
But recently there has been a lot of buzz around Powershell and the VI Toolkit for Windows. So I started checking it out a few months ago and thus far, I’m very impressed. Check this out, a script that would solve the HBA rescan problem I just described could be as simple as this …
Connect-VIServer –Server ip_of_vc -User “username” -Password “password”
Get-VMHost | Get-VMHostStorage –RescanAllHBA
Or, if you’d like the script to prompt you for the VirtualCenter IP, username and password, it would look like this …
$vc = Read-Host “Enter the IP address of your VirtualCenter server”
$un = Read-Host “Enter your username”
$pw = Read-Host “Enter your password”
Connect-VIServer –Server $vc -User $un -Password $pw
Get-VMHost | Get-VMHostStorage –RescanAllHBA
Pretty easy, huh? Here is a quick little script I wrote for a customer that’s standing up a large number of ESX servers and they didn’t want to manually add all of them to VirtualCenter. The DNS names of their ESX servers were computernamexxx.domain.com.
## Gather info and connect to VC
$vc = Read-Host “Enter the IP address of your VirtualCenter server”
$un = Read-Host “Enter your username”
$pw = Read-Host “Enter your password”
Connect-VIServer -Server $vc -User $u -Password $p
Write-Host “Connected to VirtualCenter ($vc)”
## List available datacenters and select which one to add the ESX servers
$adc = Get-Datacenter
Write-Host $adc
$dc = Read-Host “Which Datacenter?”
## Gather the number of ESX hosts, the root password and the first ESX to be added
$nh = Read-Host “How many hosts are we connecting?”
$rp = Read-Host “What is the root password of the hosts?”
[int]$start = Read-Host “What is the number of the first host (computernamexxx.domain.com)?”
$stop = $start + $nh
## Add them to VC
while ($start -le $stop) {
$x = “{0:0##}” -f $start ## <– leading zeros
$myhost = “computername” + “$x” + “domain.com”
Add-VMHost $myhost -Location (Get-Datacenter $dc) -User root -Password $rp
Write-Host “$myhost is now connected to VirtualCenter ($vc)”
$start++
}
Or, here’s a quick and dirty one liner I used to remove the VLAN ID’s on a port group. If you want to add or change the VLAN ID, rather than remove it, simply change the 0 at the end of the line to the actual VLAN ID.
$hosts = Get-VMHost; foreach ($x in $hosts) {get-virtualportgroup -host (get-vmhost $x) -name “portgroup” | set-virtualportgroup -vlanid 0}
As you can probably tell, I’ve decided that I like Powershell and the VI Toolkit for Windows. It’s VERY handy. In fact, I’m thinking of collecting my favority scripts, tools and links and compiling a separte page. So look for that coming soon. For now, here are some essentials:
- The official VMware VI Powershell blog
http://blogs.vmware.com/vipowershell/ - Managing VMware with Powershell FAQ
http://communities.vmware.com/docs/DOC-4210 - A very good GUI and script editor for Windows PowerShell (and what I personally use)
http://powergui.org/ - An awesome example by Mike DiPetrillo
http://mikedatl.typepad.com/mikedvirtualization/2008/10/quick-migration.html
Happy scripting!
With virtualization finding its way into every nook and cranny of the data center, it would seem that tier 1 applications are the only safe harbor for the few remaining “Server Huggers” out there. Their mantra usually sounds something like this …
“My application is too I/O intensive for virtualization,” or “MY xyz application vendor doesn’t support VMware” or possibly “My application is too important to be virtualized” (this is one of my favorites). Believe it or not, I even heard one guy say “you can virtualize my server when you pry it from my cold dead hands” … um, wow. He has issues. Last I heard, he was de-virtualizing a server farm at the NRA. Hehehe.
Anyway, for the rest of us with our heads NOT buried in the sand, I’m here to tell you that tier 1 applications can and should be virtualized. I’ll go so far to say that if you’re not virtualizing tier 1 applications, you are doing your company a major disservice.
Below is a brief overview of a presentation I gave in Cincinnati a few weeks ago to a group of about 75 professionals. The topic was “Virtualizing Microsoft Exchange.” And while the content that follows is geared towards the Microsoft Exchange application, it can really apply to any tier 1 application.
Performance
I’ll start with performance because this is typically the first objection to virtualizing a Tier 1 app. The perception is that virtualization creates too much overhead and therefore applications in a VM will certainly underperform applications running on a physical server. This current perception was born out of a previous reality. In the early days, virtualization really did introduce enough overhead to warrant physical servers for applications with high I/O. But a perfect storm is a-brewin’ and I summarize it with the following equation:
hypervisor improvements + server hardware improvements + application improvements =
better than native performance
That’s right. Mileage will vary, but given a properly architected solution, virtual can actually outperform physical. And even in scenarios where physical outperforms virtual, the delta is probably measurable, but not observable. So let’s take a closer look at the three areas I mentioned in the equation above.
Hypervisor Improvements
The hypervisor (AKA, the virtualization layer, AKA the Server Hugger’s worst nightmare) has come a long way in the past few years. And in VMware’s ESX product, the latest version has the following performance improvements over previous versions:
- Increased guest OS memory to 64GB
- Increased physical RAM on ESX to 256GB
- TCP segment offload to further lower CPU utilization
- NUMA optimizations improve multiple VM performance
- Support for 64-bit clustering with boot from SAN
These improvements alone can capture almost all tier 1 applications, but combined with the next two, almost no tier 1 app can hide from becoming a candidate for virtualization.
Server Hardware Improvements
We’re now seeing server hardware with 256GB+ of physical RAM. Multi-core CPU’s with 2 and 4 cores are running in production today and 6/8/12 cores are coming soon. And best of all, hardware-assisted virtualization technologies are emerging, pushing the virtualization overhead down to the hardware, getting the hypervisor ever closer to near native performance.
And because the vast majority applications simply can’t fully utilize hardware with this much horsepower, ironically, virtualization is the only way to truly capture the full ROI of these physical investments.
Application Improvements
As applications continue to evolve, bugs are fixed and bad code is optimized, performance improvements within the application are being realized, further reducing the need for a physical server. Speaking specifically about Microsoft Exchange, the following performance improvements exist in 2007 over 2003:
|
Exchange 2003 |
Exchange 2007 |
| 32-bit Windows | 64-bit Windows |
| 900MB database cache | Multi-GB database cache |
| 4Kb block size | 8Kb block size |
| High read/write ratio | 1:1 read/write ratio |
| Requires high-end storage | Affordable storage (iSCSI) |
| Storage is common pain point | Eliminates storage pain point |
| 50% reduction in disk I/O |
Of course the improvements for this piece of the equation will vary from one app to the next.
Bottom Line: Performance should not be a barrier to virtualizing an application.
A Virtual Server is Better than a Physical Server
Tier 1 applications are the most critical, important applications in your organization and therefore they need to run on the best infrastructure possible. So almost by definition, tier 1 applications need run in a VM. Here are a few of my favorite reasons why a VM is better than a physical server. Keep in mind, these aren’t the only reasons, just my favorites.
Reason #1: Better up time
The “eggs in one basket” argument no longer applies. And for those of you who don’t know what I’m talking about, the objection usually sounds something like this … “If I put 30 VMs on a single physical server, and that physical server crashes, then I’ve just lost 30 applications instead of one!” This was a very legitmate concern five years ago. But today you can get better uptime in a VM than you can with a physical machine. In the worst case scenario, if a physical server dies, those VMs are automatically powered up on a different physical server. In my experience, the VMs are usually back up and taking requests in under two minutes (and yes, I’ve timed it with a stop watch). And this is worst case scenario for a VM today! What’s best case scenario for restoring a physical server after a hardware crash? Weeks? Days? Hours (if you’re lucky and really prepared)?
So with today’s technology (and it’s only going to get better with what’s coming soon), worst case scenario for a VM is better than best case scenario for a physical server. And you might ask, what’s best case scenario? Even with hardware maintence, you can achieve 100% uptime with VMs. How? Check out a few of VMware’s features like VMotion, DRS and Update Manager.
Reason #2: Better hardware utilization
The average server utilization across the globe is less than 10% and in my experience, it’s often less than 5%. Why? A single application can rarely harness the power of the hardware it’s running on. And for a ton of different reasons (which I won’t go in to here), critical applications typically require a dedicated server. That is like buying a Ferrari and never driving it more than 5 mph … what an awful waste! Get the most for your money by putting each app in a VM, running multiple VMs per physical server. Open that baby up and let it do what it was built to do! I think the following two screen shots do a great job of showing you what I’m talking about.
CPU Utilization Before VMware
CPU Utilization After VMware
Reason #4: Avoid over provisioning
Why waste time and energy planning for future capacity (which is really nothing more than an educated guess based upon a ton of assumptions)? The tendency has been to over provision hardware to account for future growth, but this often leads to under utilized hardware. With Virtual Machines, additional CPU and RAM can be added at anytime with a few clicks of a mouse. And moving to more powerful systems in the future can be done in real time with VMotion and/or Storage VMotion. With virutalization, it only makes sense to simply build your application for the capacity you need and then throttle as necessary.
Reason #5: Better Security
Typically, protection engines come in two forms, host based and network based. The problem with network based security software is that it has no (or very limited) visibility in to the host. And the problem with host based security software is that it’s running in the same context as the malware that it’s trying to protect against. And the creators of malware are not stupid! They continually find new ways to hide their malware and/or attack the protection engine, creating a never ending viscious circle of cat-and-mouse.
But we now have new, trusted layer with the much smaller codebase of the hypervisor where we can provide protection from outside of the operating system. A protection engine from this layer provides a much stronger defense because it’s “underneath” the VM, completely isolated from the malware. And this is a great place for a protection engine to live because it can see all I/O of the VM and inspect each of the virtual components (CPU, Memory, Network and Storage). Better yet, we now have the ability to do things like:
- Intercept, view, modify and replicate I/O traffic from one, many or all VMs
- Provide inline protection or passive monitoring
- Mount and read virtual disks
Reason #6: DR made easy
In the physical world, DR is a pain in the butt and super expensive. The reason is DR solutions for physical servers often require similar hardware at the DR site to avoid issues with driver, hardware, and software compatibility. These dependencies are eliminated in a virtual world, which means any VM can run on any physical server with an ESX hypervisor. And because a VM is completely encapsulated, the entire VM exists in a small set of files. This simplifies replication and therefore simplifies the process of keeping your production and your DR environment in sync. And finally, servers at the DR site can be used for other purposes, like test and development, until they are required for DR purposes. Which means an investment in a DR infrastructure will not site idle.
Support
I love it when I hear someone say “my application vendor says they won’t support VMware.” Hmmmmm. Here’s a crazy question for ya, isn’t it VMware’s job to support VMware? Now, I’m sure what they really mean is that the vendor won’t support their application in a virtualized environment. But just to make things clear, if you have a problem with VMware … call VMware.
And support for applications in a virtualized environment is rapidly changing. Examples are numerous, but two big ones that come to mind are SAP and Microsoft. In the earlier part of the year, SAP announced full support for their software on VMware. And just recently, Microsoft announced the Server Virtualization Validation Program (SVVP) where they will support their OS’s and a good list of their applications in a virtualized environment. And VMware’s ESX is the industry’s first hypervisor to be validated by Microsoft.
What about those vendors who still don’t support their applications in a virtualized environment? Most of my customers do two things. First, they put pressure on the vendor to start providing support. For large companies, this can be very effective since the software providers want to keep their big customers happy. Second, many of them have a “swing server.” So when a vendor’s support team requires them to reproduce the problem on physical hardware, they simply V2P the VM on the swing server and continue on their merry way. (Yes, I know, this isn’t always as easy as I make it sound. Though it often can be just that easy)
Still not convinced?
The table above is the results of a survey of 500 VMware customers taken over a year ago, and the numbers are growing rapidly. Simply put, customers are virtualizing tier 1 applications today.
Powered by ScribeFire.
In my first post in this series I stated that you need to know about and care about virtualization in your company. And I sincerely mean this regardless of your role or position. I don’t care if you are in sales or marketing or finance or development. I don’t care if you are the CEO of a billion dollar company or if you are a software developer trying to debug some code or if you are a marketing manager relying on business systems (such as email) to do your job. Simply put, a properly virtual platform will have profound and compelling benefits that will affect your ability to do your job and grow your company.
But how?
I’m glad you asked. Quite frankly, the benefits are so numerous I find it hard to know where to begin. But at the end of the day, the bottom line is what counts here in corporate America, so let’s start there.
Virtual servers are not bound by the same physical limitations that constrict traditionally built servers. And what this means is that we can now safely put many virtual servers onto a single physical server. Consolidation ratios of 30:1 are typical and upwards of 75:1 are certainly not uncommon.
And let’s think about this for a second. Assuming a 50:1 ratio, if my company is spending millions of dollars a year on a data center that has 5000 physical servers, for example, how much money would my company save by creating 5000 identical virtual servers and running them on a solid virtual platform running on only 100 physical servers? That’s right, 5000 physical servers can be reduced to 100 or less. Think about what that means in terms of hardware, power and colling, provisioning and administration, etc. The dollar savings are absolutely profound. I’m not kidding. In a later post I’ll show you some real world TCO and ROI calculations and the numbers are seriously unbelievable. Even the most stern and stodgy of CFO’s will be as giddy as school girl. Very exciting.
And let’s think about this a little more. If a virtual platform can save your company millions of dollars, what could that money be used for? Could it be used to hire top notch talent to grow your business? Or could it be used to purchase more advertising, marketing, or sales talent? It’s a rhetorical question, you don’t have to answer.
The next benefit is undoubtedly the most powerful and exciting, but it’s one that most individuals don’t understand right away. Everyone gets the dollar cost savings instantly. It’s easy to see and understand. But really the most powerful benefit of virtualization is mobility.
Revisiting our VPN example from the last post, what does a VPN allow you to do? It allows you to be mobile and access the corporate office from anywhere. You are now free from the home office and you can quickly pick up and go get business done. Similarly, a virtual data center allows your infrastructure to quickly move, change and adapt to meet your business requirements. Once my servers and applications are virtual, they become highly mobile, and they can easily move from one physical server or infrastructure to another with very little effort.
This mobility has a tremendous impact on all sorts of things that you care about. All of the applications you use every day (email, web, file sharing, etc.) can now move around with zero downtime and in a manner that is absolutely transparent to you. This allows for things like zero downtime maintenance and significant performance improvements for your applications.
Since this post is getting a little long, I’ll continue to address the remaining benefits of creating a virtual infrastructure in part three of this series. Sorry for the delay between posts, it has been a very busy month. Part three should come along much quicker (I hope).