Tintri SyncVM

For most of you that know me, you are already aware I left VMware around 5 months ago to join Tintri. VMware is a great company and I’m very grateful for having the opportunity. While at VMware, I had several customers that deployed Tintri storage appliances and I never met a customer who simply didn’t rave about it. When the opportunity presented itself, I was extremely excited to take on a new journey.

Having been on board now for a little over 5 months, I simply can’t believe how simple, high performing, and feature rich our product line is. The following demo illustrates a recent feature release known as SyncVM. Not only can you synchronize an entire VM to multiple points in time, you can sync individual vDisks from other VMs.

This demonstration shows the simple process of synchronizing a production DB down to a test system. Then reverts back to the test systems previous state. Stay tuned as even more advanced SyncVM features will be announced soon!


VMworld Survival Guide For Introverts


With VMworld fast approaching, some are eager to party and catch up with Twitter friends. Wading through crowds vying for free USB keys and lighted rubber balls excites them. Others wonder how they will endure serial smalltalk. Or how they can make an appearance at an insanely loud, hot, alcohol-fueled soiree, and still execute a perfectly-timed exit.

This post will deal with the latter. You know who you are. You’re the guy or gal who loves technology, wants to be knee-deep in a tech conference. You would prefer about 12,000 less people engaging only in meaningful, passionate, technical conversations.

First things first. Realize your introversion doesn’t make you a freak who prefers being in your shell at all times. It simply means you derive your energy from within. You recharge your internal batteries alone, digesting your thoughts with few distractions. Extroverts recharge their batteries right there on the show floor. While they yell over loud music and megaphones to tell people what they had for dinner, they’re gaining energy. You are one to analyze a band’s musical prowess, while analyzing their tonal structures. Extroverts are the ones kicking you in the face crowd-surfing at the same concert.

Introverts can make gigantic tech conferences easier to digest, and condition their batteries at the same time.


1. Talk to people

The popular stereotype is that introverts don’t like people, and don’t like to carry on conversations. This couldn’t be further from the truth. Introverts love to carry on conversations with anyone who will engage on a topic we feel passionate about. Once engaged, it is hard to stop some introverts from talking. This is one reason I love attending Tech Field Day events. You’re in a pre-selected group of your peers, who are guaranteed to have passionate opinions, and want to engage on the topics you care about. Introverts generally think about a topic pretty deeply before discussing. According to Susan Cain, in the book QUIET

This “may also help explain why they’re so bored by small talk. “If you’re thinking in more complicated ways, – then talking about the weather or where you went for the holidays is not quite as interesting as talking about values or morality.””

Take advantage of vendors who are dying to tell you all about the intricacies of their products, and schedule one on one time with them. Find a few vendors you really want to learn more about. Most of the better ones will have times when you can sit for a on on one briefing, or quick “Genius Bar” type conversation with one of their engineers. I highly recommend doing this. It’ll get you engaged, and you’ll be talking to someone who is passionate, and deeply technical (most of the time).


2. Get a hotel that’s as close as possible to the conference center

Sounds like a no brainer, right? While all personality types want to minimize their walking distance, and maximize their conference time, this is especially important for the introvert. It allows you the flexibility to head back to the room, and catch your breath during the day, if you need to. When you feel your batteries running down, go ahead and skip that session you had scheduled. It’s going to be online later anyway. Head back, and wind down for an hour. Recharge, and veg out. This can make a dramatic difference in your day. If you’re stuck with all those people, and constantly shuffling from session, to hall, to crowded meals, you’ll be completely wiped before dinner.

3. Go out for meals

Yea. I know. Your company paid for a conference where meals were included. But unless you’re in Vegas, the catering is generally horrible anyway. Hit Tripadvisor, or Yelp to find close restaurants you’d like to try, and get away from the crowds for a bit. If you find other conference attendees at these places, guess what. . . They’re likely doing the same thing as you, and if you end up getting into a conversation with them, it will be engaging. They likely hate smalltalk as well, and want to share some of their complex thoughts with an equally complex thinker.

4. Don’t skip parties

Make sure you go to at least a couple parties. Most of the time, you can find fellow introverts hanging out, sipping slowly, drifting toward the door. If you do, execute a casual greeting, with all the tentativeness you’d want from them. If they do want to chat, it won’t be some asymmetrical, bombastic conversation, where you’re competing for volume. It’ll likely be on a technical topic you can appreciate, and will value. Exchange cards with that person. This is how we introverts can network without the high schmooze factor, and wasting valuable energy.

5. Don’t forget labs

If you need a break to recharge, you can always go do some labs. Nobody will bother you there, and it’s pretty quiet. Don’t stress out about missing sessions you wanted to see. Again, they’ll be available online just a few weeks after the conference.

6. General Sessions are great from hang spaces

Most of the hang spaces at the conference will be broadcasting the general sessions live. If you’re not feeling up to the crowds, you don’t need to stand in a sea of people, waiting to get a decent seat at these. Just head over to the beanbags, and watch from there.

Most of all, have fun. Don’t try to take it ALL in. Prioritize. There’s too much for even the most extroverted to experience all of VMworld.

Scale Applications Seamlessly and Remove the Infrastructure Roadblocks!

In today’s day and age, physical bare metal application servers are a thing of the past; niche!  Every business, regardless of size has a Virtual 1st Policy and most are approaching the 70% plus virtualized watermark.  As business leaders and technology professionals continue to virtualize more and more you’re likely placing the most mission critical applications within the virtual infrastructure.  So why not have an infrastructure that focuses the management and scale of these applications?  Tintri does just that.
My business, Clint’s Cool LLC, has a production site with an important group of servers.  Now, just like you every one of my applications require a Test/Dev, Staging, Production, Web, DB, Application and don’t forget DR server.  WHEW, that’s like 6 servers just for one application.  Well, OK. But now I can’t have everything running on the same Infrastructure.  Good thing I have that Secondary Site and Colocation I can replicate to.  Managing each individual VM would be very cumbersome on that old conventional CME or NappTap storage array!  It would have been impossible to even begin to group Servers and Applications together.  Management would have been extremely complex and I would be left with a complete management mess.  Good thing we run Tintri :)
We’ve got a couple Tintri VMstores deployed.  The first is in Pittsburgh at our Corporate HQ and the second is in Detroit.  We also replicate to a Colocation (It’s a super secret location, can’t tell you where).  Tintri Global Center is an intelligent control center that allows us to manage all of our Tintri’s together, allowing them to function as one.  This is wicked awesome for us and we’re even more pumped about the upcoming Tintri Global Center 2.0.  TGC allows us to build, respond and adapt to the changing market much faster than before!
We’ll now be able to track performance history and manage these “Service Groups” of VMs regardless of location. So where we’ve got applications scaled to Pittsburgh, Detroit and our “Super Secret Spot” everything is being managed, monitored and troubleshot all from my Network Operations Center.
Tintri Global Center 2.0 Main Dashboard
Tintri Global Center 2.0 Service Group

With our Service Group setup across my entire Tintri Cluster, Protecting the INDIVIDUAL Application is just as easy!  Nope, we’re not a “me too” storage product that’s protecting entire Volumes or LUNs like.  We’re granular baby – Applications and VM’s.

Tintri Global Center 2.0 Protection Setup

As my Application(s) migrate from location to location for recovery and testing purposes all of the policy and historic per VM and per Application metrics will follow.  API’s and PowerShell CMDLET’s within the Tintri PowerShell Toolkit 1.5 fully leverage all of the goodness in TGC 2.0 and Tintri OS 3.2.  A few of the other new features that further raise the bar are Single Sign On – this way I can seamlessly move from Global Center to VMstore UI and back again without having to re-enter authentication credentials.  We’ve also added support for Role Based Access Controls to the Global Center offering.  Be a Super Admin a Storage Admin or a Read Only User – all based on Active Directory or LDAP user groups.
In the end, Tintri Global Center 2.0 allows organizations of all shapes and sizes to seamlessly scale applications based on the demands of the business.
Follow me on Twitter @ClintWyckoff

Empower Your Operations with Tintri and vRealize!


Monitoring, analytics and predictive analysis, are all necessary elements that make up any successful IT Shop World Wide!  As environments and business’ move towards 100% uptime and continue to drive forward it’s absolutely critical to know the health of all elements that make up the IT portfolio.  There are many different operations framework solutions on the market today.  Many focus around the elements of ITIL and allow operations team to respond to issues in a timely fashion and drive down the mean-time to resolution.  ITIL focuses on Incident, Problem, Change and Release Management.  This blog post however, will focus on the analytics and heuristic elements that surround the Virtual Infrastructure.

Organizations are driving further and further towards virtualizing everything with a “Virtual 1st” strategy.  In fact, the vast majority of the shops I meet with on a daily basis are well down that path moving towards 90% + virtualized!  It’s clear that VMware is the market leader in the hypervisor space, so with that the choice of these shops is to utilize native VMware tools.  And no I didn’t mean the VMware tools you install in a guest VM ;)  Recently VMware renamed the vCenter Operations Manager (vCOps) to vRealize Operations Manager (vROps).

Tintri + vRealize Operations Manager

At Tintri we focus on removing the challenges that conventional storage brings to the virtual infrastructure.  We are the standard for running Virtual workloads efficiently and reliably while moving management from the traditional LUN or Volume level down to the individual VM.  This places us right in the wheelhouse for pulling all the rich Tintri visualizations and deep insights into vROps.  This also means everything Tintri knows about the VI, our customers’ main analysis engine will also know too!! You’ll correlate specific VMs and workloads to performance problems, identify sources of issues, and predict when and if an environment will potentially run out of resources.  For all existing Tintri customers the Management Pack for vROps is 100% free!  All that’s required is vROps 5.8 in Advanced Edition and well at least a single Tintri VMstore!  Since the Management Pack uses the Tintri REST API you’ll need to be at OS version 3.1.x.  Just like everything Tintri does, it’s THAT SIMPLE :)

Let’s take a deeper look into the technology and features.


Tintri’s Management Pack for VMware vROps provides a holistic view and deep insight into the health and overall efficiency of the Tintri Infrastructure.  At first glance you’ll quickly notice that all of the same rich Tintri information that’s typically found in the VMstore UI or through Tintri Global Center can now be retrieved from vROps.  One of the particular items that excites me is the ability to now have all of my Tintri performance and per-VM metrics available with the retention policy of vROps.  I’ve typically seen environments set the retention for around 365 days, so now I have the ability to go back to last year and see exactly how my VMs were performing.  Imagine trying to answer the “how did it look last year during peak holiday season?” question  Again, focusing on he analytics engine of vROps – I’m now empowered to make accurate and well guided decisions.

Let’s take a deeper look into the technology and features.

Another element that leveraging the vROps toolset allows us is the ability to utilize the unique badge identifiers.  If you’re not familiar, badges allow you to quickly see the health of an individual vSphere object.  Through the use of these badges and whether or not an object is green (good), yellow (warning) or red (bad!) the given object is assigned a health score.  Tintri MP brings forth Health, Workload and Capacity information.

  1. Health
    1. VMstore and per-VM Performance Isolation and Visibility
      1. Health Score is Calculated using:
        1. Latency
        2. Flash Miss
  2. Workload
    1. VMstore and per-VM Workload visibility with vROps Workload Badge
      1. Workload Score is Calculated using:
        1. Performance Reserve (VMstore)
        2. Throughput and IOPS (Per-VM)
  3. Capacity
    1. VMstore Capacity Visibility with vROps Capacity Badge
      1. Does not display Per-VM Capacity
      2. Capacity Score is Calculated using:
        1. Space Used (VMstore)


So how do i get this coolness?

  1. Login to Support.Tintri.com
  2. Click Downloads
  3. Choose vRealize Operations Management Pack
    1. Available in vApp…Linux…Windows
  4. Install Management Pack
  5. Discover the Adapter (Admin -> Support)
  6. Create Adapter Instance – using local VMstore login credentials.
  7. DONE!

Follow me on Twitter @ClintWyckoff

Power Up Your DR with Tintri and SRM!

Check out the video on VMware SRM Integration with Tintri ReplicateVM!


Disaster Recovery is something that’s very near and dear to my heart all the way back to my years on the end-user side of the fence. The annual or semi-annual Disaster Recovery event is typically a very painful and long process with lots of lost sleep! Dating myself a bit, but BC/DR was even more of a challenge when we had to recover applications running on physical machines.  System state restores to dislike hardware was never fun!

Virtualization’s changed the industry in many ways. One challenge IT Departments face is effectively protecting the business. What happens when we have that “smoking crater” or “How do we know we’re protected?” CIO’s around the world are asking these questions! So preparation is key.

Tintri ReplicateVM

For years Tintri customers have had the ability to efficiently replicate on premise VMs (yes, not LUNs or Volumes – individual VMs) off premise. VMs that have differing Recovery Point Objectives can be managed individually.


Tintri ReplicateVM with vCenter Site Recovery Manager (SRM)

Tintri OS 3.1 further integrates the ReplicateVM engine with vCenter Site Recovery Manager (SRM) to provide an automated orchestration and non-disruptive way to centralize recovery plans for every virtualized application! Let’s take a deeper look.

First off if you’re looking to implement Tintri VMstore with vCenter SRM you’ll want to check out the Best Practices Guide or watch the SRM Video above. These guides provide a step-by-step, soup to nuts explanation of exactly what’s required and how to get everything up and running without issue. Second, you’ll need to make sure you’ve got an appropriately setup infrastructure.

The requirements are rather basic. See illustration below.

  • vCenter and SRM Primary and Recovery Sites with independent compute backed by Tintri VMstore datastore.
  • Active Directory authentication at each location. Be sure to follow Microsoft Best Practice for replicating A/D. This is to utilize built-in replication and not host based or array based replication.
  • Rather than using the Embedded PostgreSQL database, SRM 5.8 supports MS SQL Server 2005 – 2014 and this is the recommended route.


Now we’re all setup and you need to grab the Tintri Site Recovery Adapter (SRA) off of the Tintri Support Portal. After you download, install the SRA on both of the vCenter servers. Pretty straightforward install on your vCenter Server on both Protected and Recovery sides, next, next….finish.


Next go through the normal SRM steps of Creating Mappings, and Setting up Tintri Replication

Mount the Tintri VMstore to your ESX hosts as normal – = However, you need to create a sub mount for each group of VMs you want to protect.

For instance my Gold RPO – – You can create this by browsing the datastore within the vSphere console and creating a folder /Gold_RPO and then mounting to your ESX hosts. Then sVmotion the Gold RPO VMs to the datastore.


On this datastore I now have all VMs that need recovered with the Gold Recovery Point Objective, now jump over to the VMstore UI and navigate to the Virtual Machines tab and click Service Groups. The easiest way to think of a Service Group is Service Group in Tintri = Protection Group in SRM.


For more granular RPO, 15 minutes – Click Custom, Hourly and then click in the Minutes field and choose the required RPO. Also, important to note is the ability to provide crash consistent OR VM Consistent snapshot.

VM Consistent will leverage the native VM Tools present inside of the Guest OS to quiesce applications like Sharepoint, SQL Server, Exchange, DB2, Active Directory…etc.


To wrap up the setup go through and create your Array Pair (Choosing Tintri SRA), Protection Group and Recovery Group. All of these steps are illustrated in great depth in the Video I created or in the Best Practices Guide.

One of the great parts of Tintri ReplicateVM + Tintri SRM is the ease of use and efficiency. ReplicateVM has always been extremely WAN friendly! Like many things on a Tintri VMstore, replication too is based on VM’s and snapshots. When replicating VM snapshots prior to even sending a block of data over the WAN we’ll send block fingerprints to check for which blocks are missing. Once identified we’ll send over those missing blocks in a compressed & deduplicated fashion to ensure that efficiency of the latency sensitive WAN is never over taxed with unneeded blocks of data!

Home Stretch!

With everything setup it’s pretty easy to go through and perform a test recovery of the protected VMs.  Within the SRM Plugin drill into the Recovery Plan that’s been created and mapped to the Protection Group.  It’s worth reiterating that the Protection Groups in SRM correlate directly to the Service Groups in Tintri.


Right click on the Recovery Group and choose Test. One of the options you’re asked with is “Do you want to replicate recent changes to the Recovery Site?” This will allow Tintri ReplicateVM to copy over the blocks of data that have changed since the last synch cycle.  After the test you’ll want to right click and run the Cleanup task.  During the test, since it’s not an actual failover – the Protected side still retains the authoritative copy of VMs, so Cleanup allows SRM to get everything back to the way it needs to be for normal replication to continue.  Never go through and perform a Recovery, unless it’s a true failure situation.  If you’re just looking to sanity check yourself, use test.  Recovery moves all authoritative rights over to the Recovery side and you’ll have to re-replicate everything back to the Primary.

Finish Line!

With that I’ll leave you with the 3 key pillars and differentiators.

  • Simplicity of Configuration
  • WAN Efficiency
  • Visibility at a Per-VM level

So what’s the takeaway? Again Tintri continues to deliver disruptive technology that focuses on the largest and fast growing area of the Modern Data Center – Virtualization!

Tintri – Get Thin for the Win!



At Tintri I talk with a lot of customers and prospects about their virtualization environments and how it relates to their storage configurations.  Virtual machine provisioning discussions come up quite a bit, so I thought I would write about some new features that Tintri just introduced.

The method in which we deploy virtual machines over the past many years has certainly changed on the storage side of the house.  Thin Provisioned, Eager Zero Thick, Lazy Zero Thick; there has always been a long menu of choices when deciding how to deploy your virtual machine’s that support your applications.  This has also created some confusion for people around “which choice is right for me when I deploy my virtual machine?”  I have also noticed recently that many customers thought they had deployed thin provisioned vmdk’s but in fact they were running thick due to default values being selected.

Thin Provisioning

First let me start off by saying Tintri is “pro virtual machine thin provisioning”.  You might be saying, wait a second, you’re NFS on vSphere, you are thin provisioned by default!  This is true, but with our VAAI implementation we can observe any of the other types of provisioning methods from VMware as well.  Let’s say you do a storage vMotion and move an inefficient thick provisioned virtual machine from an existing block storage environment over to a Tintri VMstore.  If VAAI is installed, we will observe the specifications of the existing format and retain this .vmdk format and punch zero’s.  (unless you decide to change the option when migrating).

Let me make note, there is no need to use older “Thick” provisioning methods when deploying workloads on Tintri.  Our VMstore operating system is designed to understand the workloads of every virtual machine down to an 8KB block.  Tintri has QoS built into our datastore to adapt as your VM’s change from a performance perspective.



It’s all about Efficiency

With our new T800 platform, we have upped the bar on giving you more value from your Tintri VMstore investment.  We have enabled compression at rest on all of the new models to help drive your storage costs down. This allows your organization to run as efficiently as possible from a capacity perspective.  With our current shipping version of Tintri OS ( we now add in some great capacity management features which I will highlight below.

Lab Environment

I deployed a few VM’s for illustration in the lab, they are empty, no operating system, you can see some are eager zero thick provisioned, one is lazy, and one is thin in the screenshot shown below:


Here is the overall capacity of the VMstore prior to making changes to the virtual machine formatting:


In the example above you can see our compression ratio numbers are a little low, so let’s examine why.  If a virtual machine is thick provisioned per VAAI, according to the specifications, you must “hard back” the zero’s, or reserve the space inside the virtual machine.  If you were to thin provision the .vmdk file, then compression would allow us to reclaim the white space.  This process typically involves doing storage VMotion so you can run the conversion process.  Not any more!

Convert to Thin!

Tintri has built in some great ways to help examine and fix how you can optimize your virtual infrastructure.  In the example below you can see the “Provisioned Type” field on the far right that I have exposed in our user interface to identify which VM’s are thick provisioned.


Let’s go ahead and right click and convert these VM’s within the Tintri user interface to thin disks!


Post conversion

This conversion process is instantaneous, and you can now see in the Tintri user interface we have converted our inefficient thick provisioned vm’s to thin without having to perform a storage VMotion.


You can see below the vSphere Web client now reflects an accurate savings on our capacity on each virtual machine:


Below you can now see the Tintri VMstore overall compression ratio is gone from 1.7x to 2.7x since we have migrated the virtual machines to thin provisioned vdisks!


Set it and forget it

Tintri has taken this one step further to help our customers (and thank you customers for your continuous feedback, this is a result!).  We now have a global option within the datastore settings to keep all virtual machines that get migrated to Tintri as a thin provisioned regardless!  No more going back to reclaim on accidental vm’s that were migrated over.


I hope you found this write up useful, let me know if you have any questions!


Peak Virtualization and Time Dilation

In the movie Interstellar, one of the central themes throughout is gravity, and its effect on the passage of time. This time dilation is something that has long fascinated me, and it was great to see it fleshed out on the big screen by a master like Christopher Nolan. While not giving any spoilers, the basic scientific principle is that relative velocity and gravity both cause time to elapse at a much slower relative pace. So an astronaut could travel to Mars, and come back having aged 18 months, while someone on Earth had aged 19 months. Of course this is a dramatic oversimplification, but my goal is not to explain relativity here.

What does any of this have to do with virtualization, technology, or anything else?

One of my favorite tech podcasts these days is In Tech We Trust. This group of guys has great chemistry, and a broad array of technical experience. If you haven’t checked it out, I would encourage you to give a listen.

The past couple episodes got me thinking about the relativity between those who are technology pioneers, versus the rest of the world. There were mentions of “peak virtualization”, and how technologies like Docker, and everyone rushing to the public cloud, we could be seeing this now. And this is where I believe we can see some time dilation between the relative velocity of a handful of “astronauts”, heavily into technology on the bleeding edge, and reality down here on Earth. People who attend the multitude of different tech conferences, stream Tech Field Day events, and keep abreast of exciting developments in tech, are not in the majority.

The majority is engaged every single day in making the technology they have execute their business goals. While we are musing about OpenStack, Docker, Swift, etc. these folks are grinding it out with programs that were written long ago, and will never be suitable for those types of cutting edge deployments. I know companies right now who are planning projects to migrate core apps off mainframes. And you know how they’re doing it? They’re basically porting applications over so that they will run on open systems. They’re taking apps written in ALGOL, or COBOL, and writing them the exact same way in a language they can sustain, and deploy on open systems.

They’re not re-architecting the entire application, or the way they do business. They’re interested in satisfying a regulator, or auditor, who has identified a risk. They need to do it as inexpensively as possible, and they need to do it without introducing the risk of switching to object based cloud storage, or a widely distributed, resilient cloud model. They’re not concerned with the benefits they can glean from containerization, OpenStack, or whatever. They need to address this finding, or get off this old behemoth mainframe before it dies, or they have to spend millions on a new support contract.

In the real world, which runs on Earth time, the companies I deal with are not willing to entertain dramatic re-architecture of the core parts of their business, just to take advantage of something they don’t see a need for, or business case around. And if you happen to get an astronaut in a company like this, and he or she mentions something about cloud, or a hot new technology, the response is usually befuddlement, or outright dismissal. How can you blame the C level people? They’re constantly seeing stories about gigantic cloud providers taking massive amounts of downtime, and silly outages that affect the majority of the globe. They don’t need that. They need their bonuses.

Remember, many of these people only implemented virtualization because they couldn’t justify NOT doing it.

While many in our circle of astronauts have the luxury of ruminating on the end of virtualization, and the next big thing, the people who are still in the atmosphere have concerns that are far different. Predicting the future is definitely a fool’s errand, but based on what I can see down here, I’d have to guess that we are not yet at peak virtualization.

You Down With NTP?

(Yea! You know me!)


NTP. How can I explain it? I’ll take you frame by frame it.

I’m sure that no readers of Virtual Insanity would ever neglect to setup NTP properly on every single ESXi host.  But occasionally, our NTP source hiccups, or something happens to skew the time.  Recently I found a host with the NTP service stopped.


Why?  No idea really.  Maybe it was stopped while someone was troubleshooting.  Maybe it just crashed.  But it will cause issues with backups, and with applications running during backups or vMotions.

When a snapshot is taken, or a VM is vMotioned, the time is sync’d inside the guest by default.  This can be a problem if your host NTP time is off.  All my guests use Active Directory for NTP, and the Linux guests use an AD domain controller for NTP, so I do not rely on guest time syncing up to my ESXi hosts.  Or so I thought. . .

Even if you have your guests configured NOT to do periodic time syncs with VMware Tools, it will still force NTP to sync to the host on snapshot operations, suspend/resume, or vMotion.  There is a way to prevent VMware Tools from syncing the time for these events, but it’s better just to make sure NTP is up and running, and getting the correct time.  There is a clear reason VMware insists on doing these sync’s during times when I/O is quiesced, or transferred to another host.  Timekeeping in a hypervisor environment when you’re sharing CPU cycles is no trivial task.

If you use a backup solution that snapshots the VM, VSS quiesces the I/O inside that guest.  When it does, there’s a VSS timeout for a snapshot to complete.  If the time is exceeded by the snapshot, VSS will timeout, and your job will fail with error code 4 quiesce aborted.


By default, this timeout is set to 10 mins on Windows guests.  Of course, my time was off on the ESXi host by 12 minutes, so when the backup job started, VSS kicked off, and then VMware Tools sync’d the time 12 minutes forward.  VSS times out instantly.  If you see this error code on your backups, an easy thing to check first is NTP.


I recommend setting the NTP service to start and stop automatically.


Previously, I had set this to start and stop with the host.  But if something happens, and it stops, or gets stopped for some reason, it will not restart until the host restarts.

So who’s down with NTP?

Hopefully all the homies. . .

Upgrading SRM from 5.0 to 5.5

If you’re one of those shops that skipped over 5.1, and are now catching up and going to 5.5, you will run into problems with your SRM upgrade.  Here’s how to fix them.

After you perform the upgrade, per the VMware Site Recovery Manager Installation Guide, you may run into permissions issues when launching the SRM plugin.

The error message is: Connection Error: Lost connection to SRM Server x.x.x.x:8095  –  Permission to perform this operation was denied.


To fix this, you’ll have to go into the database, and do some editing.

First, stop the VMware Site Recovery Manager service.

Connect to the SRM database server.  If you’re not using SQL, adjust these procedures accordingly.  This is SQL specific.

First, make sure you BACKUP your database.

Under your SRM Database, you’ll see a table called dbo.pd_acedata. Right click that table, and Select Top 1000 rows.


In your Results window, you’ll see that the only permissions that exist are old school pre-SSO “Administrators”.  We need to fix that.

To fix it, we’re going to delete that row. Right click the table again, and select Edit Top 200 Rows.


Now, select that row with the old Administrators permission, and right click to delete it.


Click yes.


Now we have to do the same thing to the dbo.pd_authorization table. Edit the first 200 rows.


Delete the first line that says DR_ACEData, and click yes again.

Now go start the SRM service.  This will automatically populate your database with the new permissions you’ll need to launch the SRM plugin, and connect. 

If you go back to the table, you can see it has the correct permissions.


For some reason, this is a known issue, but the KB is not public.  So here’s your KB.

Don’t forget to backup your database, so you can restore if you blow it up.  Happy upgrading.

New Tintri Collateral – Backup Best Practices and Veeam Integration!




Hopefully you no longer have a tape changer on staff as the above image eludes to but, just a short post to let the Tintri community know that we have released a few new technical documents to the community.  Personally I love seeing Tintri continue to produce both innovative products as well as technical collateral to support those solutions for our customer base.



The first document that we released is a document that covers backup and recovery best practices while utilizing the Tintri VMstore.  This document covers topics on data protection in general built into the Tintri VMstore, i.e. snapshots and replication.  It also includes how to achieve image level backupsand supported transports, VADP, HotAdd, NBD etc.  The document also covers how to recover data, an even more important component to backups!



The second document is one that has been sought after by many of my customers personally, so I am glad to see Tintri has brought this to fruition.  A great technical deep dive that documents leveraging Veeam software for data protection, and how that integrates with the Tintri VMstore.  (Nice work Dominic!).

I hope you enjoy these two new tech documents, just wanted to make a quick mention!  Look for more great things coming right around the corner, very excited to be delivering on some wonderful technical roadmap items this year!