Storage

I hate snapshots in VMware ESX Server

Snapshots are great for development and testing but managing them over time can become just an outright nightmare. Take this poll to let us know if you hate managing snapshots.

Snapshots are great for development and testing, but managing them over time can become just an outright nightmare. Take this poll to let us know if you hate managing snapshots.

If you have worked as an IT administrator with VMware's ESX Server, I wonder if you hate snapshots. Don't get me wrong, snapshots are a great feature and are very helpful in development and testing.

But let me ask you a question: What happens when multiple people take snapshots 12 levels deep and take up over 160 gb of disk space just in snapshots? I can tell you . It can crash the ESX Server because you have run out of disk space. Tools such as Snaphunter exist to help manage snapshots, but the overall management of them over time becomes a nuisance.

Let's start a discussion on how you manage snapshots in your organization. How do you keep snapshots from growing out of control? Do you hate managing snapshots as much as I do?

15 comments
CharlieSpencer
CharlieSpencer

And poor file and e-mail management too, I'm guessing.

meynon
meynon

Although I like snapshots we do not allow or grant permisions to our Windows or application engineers to use them in our poduction enviorment. If a snapshot is requested it is handled on an exception only basis and completed by one of our an ESX admin.

aegirth
aegirth

WE have a Hobbit monitor system that detects a snapshot after 30 days of inactivity and we use powershell script that goes then and erases it. Simple and practical.

mark
mark

As a newbie to ESX and fresh out of the training class I hope I'm not speaking out of naivety here, but the issue described seems to be one of planning. If you expect multiple people to do that many snaps, then the SAN volume should be created large enough to accommodate it. Otherwise, you might want to educate those doing the snapping. Maybe even restrict them. If this is a development team issue, then perhaps they should have their own ESX server and SAN volume to isolate them from production systems. Again, size their volume accordingly at the outset. Of course, you could offer them free beer for every day they don't snap....

cote6
cote6

We restrict the number of ESX admins to two. And both those people know how finicky and dangerous sanpshots can be (we both have crashed vm's due to running LUN's out of free space). Since we use vRanger for backups, it manages it's own snapshots, we just need to monitor it daily and make sure it has indeed removed the snapshots it took overnight.

dwdino
dwdino

What you are identifying has nothing to do with snapshots. What you have identified is a process failure. Snapshots have many benenfits when it comes to backups, patch management, and POC. Snapshots should only be performed by those with the proper understanding. We delegate access to our ESX environment to many developers and system admins, but none of them can create snapshots.

Steven S. Warren
Steven S. Warren

How you manage snapshots in your organization. How do you keep them from growing out of control? Do you hate managing them as much as I do?

jatwell
jatwell

I think I'm going to have to agree with some of the replies. I believe that this post unfairly villainizes snapshots. Yes, they can be a burden if you do not keep an eye on them but that is a process and education issue. To be fair, VMware has not provided adequate utilities for managing snapshots and preventing snapshots from consuming all available disk space. Unfortunately in this scenario you must plan for the lowest common denominator and would probably be well served to isolate the dev environment as much as possible. Using Lab manager is also an obvious choice to help manage their needs. In fact a previous poster mentioned vRanger which uses snapshots to backup. Should I hate vRanger if one of its backup scripts does not properly apply the snapshots back to file? It doesn't make me happy, but what bothers me more is that I get no notification that the snapshot is still running. I've written more detailed information about snapshots at http://day2dayadmin.blogspot.com/2009/01/death-by-snapshot.html

dwdino
dwdino

After further thought, ESX seems to be the wrong tool for the given scenario. Why not use VMware Server, Workstation or similiar desktop product that the developers can destroy without harming service delivery?

Steven S. Warren
Steven S. Warren

I agree with you but it is not implemented that easily. Too many complaints. Not as simple as it sounds.

Steven S. Warren
Steven S. Warren

Developers and testers have to create snapshots to test their products. You have to have the ability to revert back and forth for testing. What kind of dev shop do you work for?

porwig
porwig

That is what Lab Manager is for... Developers can check-out machines, do their testing and then either delete or check-in the changed system. Allowing indiscriminate use of checkpoints will inevitably create the nightmare situation you describe. We do not allow checkpoints to remain around longer than is required to perform the backup or recovery. We run scripts to alert us to the presence of snapshots. We then delete them unless they are needed longer than our two day limit.

Steven S. Warren
Steven S. Warren

Sometimes you have to work with the tool that powers that be give you. No money for new tools.

dwdino
dwdino

I do not work for a "dev shop", rather an international provider of IT services and products. We had a similar discussion on the handling of developer requirements and processes. It was found that the snapshot utilization tended to make documentation and tracking more difficult as developers would step in and out of code revisions without implementing the appropriate checks. The combination of quickly available templates and access to multiple virtual machines provided a clean and documented methodology. While this may not work for all development groups, the snapshot capability has been tightly reigned in.