Data Centers

Resolving TrueCrypt and Volume Shadow Copy conflicts

TrueCrypt is a great open source encryption solution to protect data, but it can lock horns with the Windows Volume Shadow Copy Service. Learn how to untangle the two products.

 

hero
Image: iStockphoto.com/enot-poloskun
 

A large part of working in IT involves figuring out how to prevent bad things from happening — or if they do occur, how to ensure they don't happen again. While some might term this "closing the barn door after the horse has escaped," I prefer to think of it as "building the habit of closing the barn door so he won't get out again."  Of course, that's often when you find out the barn door might not look too pretty when you're trying to keep it shut.

This was recently exemplified by an episode I experienced at a client site that involved a critical folder accidentally deleted from a Windows 2008 file server. It was a fairly typical scenario where the folder had somehow gotten lost through user mishap one afternoon. No problem. Just restore last night's backup, right? Well, no dice to that idea since the files had all been created that day, after the previous backup finished.

But the game wasn't quite over, because the client uses the Volume Shadow Copy Service on the server and it was set to take a snapshot of the data volume (H: drive) twice per day – at 10 am and 2 pm. We looked at the 2 pm snapshot using the Previous Versions function in Windows (whereby you right-click a network folder, choose Properties, click the Previous Versions tab, browse to the data you want, then copy it to the live location).  We managed to obtain three files from it, but the remaining dozen or so were still gone because they had been created after 2.

Figuring it couldn't hurt to gamble with free utilities, we tried the undelete programs Recuva and FreeUndelete but did not find any files to recover. I have only had middling at best luck with these types of programs, but they can still be worth a shot — though for some reason they always seem capable of recovering unimportant files rather than important ones.

Cutting losses and preparing for next time

That brought us to the end of the road. The user had to re-create the missing files, which wasn't the end of the world, but we figured rather than taking volume snapshots of the server H: drive twice per day, perhaps a better idea would be to do so hourly during business operations.

Configuring the Volume Shadow Copy Service snapshot schedule is easy. You just log onto the Windows server, right-click the volume in question, choose Properties, and then choose the Shadow Copies"tab. However, when we did this we got the error shown in Figure A.

Figure A

 

Figure A
 

Uh, what?

This error seemed to indicate a problem with the Volume Shadow Copy Service. The service seemed to be running okay, and as previously stated we were able to access the data it protected, but some vague errors appeared in the Application Logs:

"Volume Shadow Copy Service error: Error calling a routine on a Shadow Copy Provider {b5946137-7b9f-4925-af80-51abd60b20d5}. Routine details Cannot ask provider {b5946137-7b9f-4925-af80-51abd60b20d5} if volume is supported. [0x8000ffff] [hr = 0x8000ffff]."

Research indicated that the issue was caused by TrueCrypt running on the server. TrueCrypt is an open source encryption solution that allows you to encrypt entire disks, partitions, or special volumes (called containers) to securely store data. I use it for my personal documents and it provides great peace of mind.

In my client's case, they have an encrypted TrueCrypt 7.1a volume on this server mounted as its own drive (I:), which has folders that are shared and secured via the normal Windows server methods. This volume exists to safeguard extra-sensitive confidential data. When the server boots up and is logged in, a command runs automatically, which mounts the TrueCrypt volume after prompting for the password:

"c:\program files\truecrypt\truecrypt" /q /m /l i h:\Security.TC

This performs the following functions:

c:\program files\truecrypt\truecrypt calls the TrueCrypt executable

/q tells the TrueCrypt program to prompt for the volume password

/m tells the TrueCrypt program to mount a volume

/l i tells the TrueCrypt program to mount the encrypted volume as the I: drive

h:\Security.TC is the actual TrueCrypt encrypted container object

We decided to try dismounting the TrueCrypt volume to see if that Volume Shadow Copy Service error went away (Figure B).

Figure B

 

Figure B
 

This was as simple as launching TrueCrypt then selecting the I: drive and clicking Dismount. Once this was done, the Shadow Copies tab appeared as normal (Figure C).

Figure C

 

Figure C
 

We were then able to set hourly shadow copies of the H: drive, as shown. What would happen when we remounted the TrueCrypt volume, though?

As it turned out, the same error showed on the Shadow Copies tab, but it did not interfere with the actual Shadow Copy operation — nor were backups affected. As you can see in Figure D, the hourly snapshots were being faithfully generated (and we made sure to test this).

Figure D

  

Figure D
 

It seems this is a known issue, which TrueCrypt has acknowledged. There are lots of references to the issue on the TrueCrypt forums, and it's clear this situation has existed for some time. The site states that:

"The Windows Volume Shadow Copy Service is currently supported only for partitions within the key scope of active system encryption (e.g., a system partition encrypted by TrueCrypt, or a non-system partition located on a system drive encrypted by TrueCrypt, mounted when the encrypted operating system is running). Note: For other types of volumes, the Volume Shadow Copy Service is not supported because the documentation for the necessary API is not available."

It should be pointed out that this essentially says that we're using TrueCrypt in a non-supported fashion, but that's an acceptable scenario since it's performing per our needs and has been for some time.

I got curious to see if I could circumvent the error via other methods. I didn't want to make any changes to the system drive, but I did want to see whether encrypting an entire test volume with TrueCrypt (as opposed to the container method I described) might change the situation. Unfortunately, it did not. I also tried this on another server, making sure to keep the encrypted file container on a separate volume from the one for which I was trying to configure shadow copies, but the same error resulted.

Learning to live with things

In the end, some barn doors may get slammed shut and still look crooked, but at least they're shut. It's not a big deal for us to have to dismount the TrueCrypt volume to make changes to the Shadow Copy options on this server. In fact, we probably won't have to make any changes again anyway. So long as the Volume Shadow Copy Service is working as expected, we're satisfied with the results.

However, it's interesting to see how these kinds of problems might arise and how to handle them. If I were younger and more impetuous I probably would have slogged on, stubbornly searching for some kind of solution — perhaps backing up, reformatting, and then restoring the volume, for instance. Nowadays, though, the fear of losing valuable business hours to a cosmetic issue (as opposed to something that is actually broken) outweighs the lure of finding a solution that may not exist, like a treacherous will o' the wisp. You have to pick and choose your battles in the IT realm and decide where your priorities lie, just as you do everywhere else.

Share your feedback

Have you run into conflicts like the one described here? Share your experiences (and workarounds) with fellow TechRepublic members.

 

About

Scott Matteson is a senior systems administrator and freelance technical writer who also performs consulting work for small organizations. He resides in the Greater Boston area with his wife and three children.

18 comments
smmatteson
smmatteson

@TechrepLath  I'm consolidating all replies into one response.


OK, I see what you're saying with "If I copy a container at a time that I know the password. And you change the password on the original, I will still have access. This is because the master key does not change when you change the password. Only the header changes to allow the new password to combine with the new header to satisfy the same master key.  If one restores the old header to the new file the old password will unlock it." 

This is true, of course, since you've now got an old copy of the container with the old password.  I agree that a rogue administrator can take the steps above to retain the data even after they leave the organization.  The concern in this particular case isn't a rogue administrator but a user or intruder...  and an ex-administrator would not gain physical access to the data center later on to get at the new file.  


It's a moot issue anyhow since in the end a rogue administrator still has the capability to copy ANY files off to which they have access whether from the Truecrypt container or elsewhere.  This is understood and is part of the game of allowing administrators access to data in order to administer it, at least in this particular scenario.


The underlying goal of the Truecrypt container here is to ensure that any intruder entering the data center cannot steal this particular set of data by pulling the hard drives from the server.  

"That is not what I consider to be 'encrypting files and folders' You still have a container that is mounted as a virtual drive. Which yes, contains files and folder, but you are not encrypting files and folders in place on an existing drive. Also this method can also be used with Bitlocker to encrypt VHD files which act the same way as containers in truecrypt."


Not sure what the difference is between using the Truecrypt container and "encrypting files and folders in place on an existing drive."  The container does reside on an existing drive as a .TC file and the folders/files reside within it.  It's good that Bitlocker can perform similar functions but in this case Truecrypt is serving the requirements.


"@smmatteson 'Your argument seems to be that allowing anyone at all to know the Truecrypt container password breaks any  and all security benefits associated with this scenario.' No, my initial point was that leaving the volume unlocked and sharing it over the network makes the encryption unnecessary as the weakest link in the security is now the network access, not the access to the physical disk. Not that AD security is that bad, on the contrary, it should be sufficient. The only reason to have encryption outside of simple NTFS, AD is to protect data from those that manage the AD or have access to the physical disk. Neither is the case. Unless you are worried about someone stealing the server, which you indicated you weren't."


This latter sentence is not the case.  The entire purpose of the Truecrypt volume is to ensure that if someone steals the server or the drives they get none of the data.  Sorry if this was unclear in the communication.


"@smmatteson 'These people need access to these passwords, and so therefore the access has been provided to them in a secure fashion.' And that secure fashion is a security system based on personal accounts and privileges, not shared passwords. Non personal shared passwords can and should be avoided."


But in this case the Truecrypt volume cannot be mounted with the individual user's password, but must be shared.  Perhaps an encryption solution whereby admins use their OWN passwords to mount an encrypted container/folder would be a better option here, assuming this kind of thing exists.


"@smmatteson 'But in this case the administrators in question ARE the clients, since they access and write to the files being secured.' So why, pray tell, is a sufficiently secured NTFS folder not enough? It should be, cause as I said encrypting it and then sharing it doesn't increase the security. The AD/SMB/NTFS based access is still there, since the truecrypt container is unlocked and mounted."


We're back to the reply of "protecting the data if the server or hard drives are stolen by non-IT staff" to answer this one.


You have made some excellent points worthy of thought and discussion.  


Perhaps what would be beneficial is some sort of lockdown scenario where admins can mount the container/folder with their own ID/password (as I said previously) and users can't copy off or replicate files from the Truecrypt container but can only use them in the application of their job duties... almost like a DRM kind of situation.  It seems like this is what you're angling towards and I'd definitely be interested in hearing more on these kinds of possibilities.

undrepublic
undrepublic

i'm also a little intrigued by this setup (as well as a couple days late reading it), but...just some questions for clarification:


1. has truecrypt been updated since february 2012?  if not, does that give microsoft the opportunity to really mess up your process even more than what you are currently working around by completely killing interoperability with some as-yet-unreleased system patch?  personally, i think this question alone makes bitlocker a safer option.


2. does this environment support remote access?  if so, how do you address the possibility of a bona fide user's credentials being exploited and subsequently exposing the data to possible deletion/modification/corruption by an external attacker?

procedural question:

3. is this server being booted daily?  that's just a question.


4. when the server is up, is there ever a time when the truecrypt volume is dismounted and backed up in its entirety, or are the contained files only backed up individually during a user's session?

TechrepLath
TechrepLath

Mounting a truecrypt volume on a server and sharing it over SMB is quite possibly the most ridiculously over complicated and useless use of encryption technology I've seen to date. There is no value of the truecrypt encryption in this scenario.


The truecrypt volume is open, so anyone with physical access to the server has access to the files, and the security provided by truecrypt is not relevant to those accessing the files through the network.


Adding insult to injury putting the truecrypt password in a scheduled bootup command, is just, wow... It even hurts my brain writing this comment... Whoever supported this setup should be shot.

smmatteson
smmatteson

@undrepublic  Truecrypt has not been updated since February of 2012.  You make a good point about Bitlocker perhaps being a better alternative to which to migrate this scenario.


The environment does support  remote access and the users who connect to it do so from company-owned machines with current patches/AV software and whole disk encryption.  Passwords are rotated frequently, systems are scanned for vulnerabilities (which must then be remediated) and systems are used for company business only.  I realize nothing is a 100% ironclad guarantee but this helps cut down on the possibilities you describe regarding exploited credentials.


The server is booted once per month, on average, and the Truecrypt volume is not dismounted.  Backups are run by the backup software connecting to the server volume itself.  These are encrypted/secured.

smmatteson
smmatteson

@TechrepLath  The Truecrypt volume was requested by the Office of Security at the site in question, not by the sysadmin.  


While network access to the files on that encrypted volume do indeed depend on strong file system permissions (in this case AD), it's not the case that "anyone with physical access to the server has access to the files."  Without the ability to log into the server, how do they get to the files sitting on it in that mounted Truecrypt volume?  


They'd need to shut the server down, pull the hard drives, then access them from another system, such as via a USB enclosure or as secondary disks in a box they DO have access to.  And then they can get to the files on the Truecrypt volume?  Oh... wait.  That volume is encrypted.  Game over.


Also, to correct your claim, the password is NOT included in the scheduled bootup command.  I stated that the process PROMPTS for the password.  That /q switch "tells the TrueCrypt program to prompt for the volume password."  This password is then entered manually by the appropriate personnel.  Perhaps this assumption on your part explains why you feel anyone with physical access to the server can then compromises the Truecrypt volume, so hopefully this clears up that misconception.


With regards to your comment that "whoever supported this setup should be shot," I can tell this remark was intended as tongue-in-cheek, however it's all too easy for people to make disparaging remarks about others while tucked away behind an anonymous online alias.  Kindly have the backbone to include your real name when making these kinds of statements in the future, or else use a different phrase.

anuadewuyi
anuadewuyi

@TechrepLath I agree with your first two paragraphs. However, don't shoot anybody. Even if you being sarcastic

TechrepLath
TechrepLath

Still makes no sense.

Any attempt at unauthorized access to the data will never even encounter the encryption as an obstacle. If ad and smb is enough to secure it over the network, then truecrypt on the disk is like a lock on a screendoor. It's a placebo.

In fact, in this case, since the truecrypt volume is perpetually unlocked, it is in fact an unlocked lock on a screen door.

TechrepLath
TechrepLath

Disclaimer: I do not mean for anyone to actually get shot... Sheesh... Also, for future reference, if I ever use the word fired instead. I don't mean to say someone should get any matches.

TechrepLath
TechrepLath

@smmatteson "But in this case the administrators in question ARE the clients, since they access and write to the files being secured." So why, pray tell, is a sufficiently secured NTFS folder not enough? It should be, cause as I said encrypting it and then sharing it doesn't increase the security. The AD/SMB/NTFS based access is still there, since the truecrypt container is unlocked and mounted.

TechrepLath
TechrepLath

@smmatteson "These people need access to these passwords, and so therefore the access has been provided to them in a secure fashion." And that secure fashion is a security system based on personal accounts and privileges, not shared passwords. Non personal shared passwords can and should be avoided.

TechrepLath
TechrepLath

@smmatteson "Your argument seems to be that allowing anyone at all to know the Truecrypt container password breaks any  and all security benefits associated with this scenario." No, my initial point was that leaving the volume unlocked and sharing it over the network makes the encryption unnecessary as the weakest link in the security is now the network access, not the access to the physical disk. Not that AD security is that bad, on the contrary, it should be sufficient. The only reason to have encryption outside of simple NTFS, AD is to protect data from those that manage the AD or have access to the physical disk. Neither is the case. Unless you are worried about someone stealing the server, which you indicated you weren't.

TechrepLath
TechrepLath

@smmatteson "We are indeed using Truecrypt containers to encrypt secure folders and files." That is not what I consider to be "encrypting files and folders" You still have a container that is mounted as a virtual drive. Which yes, contains files and folder, but you are not encrypting files and folders in place on an existing drive. Also this method can also be used with Bitlocker to encrypt VHD files which act the same way as containers in truecrypt.

TechrepLath
TechrepLath

@smmatteson 


"Are you saying any administrator could copy off the Truecrypt container and then crack the encryption on it later?  If so, how? " Yes, that is exactly what I as saying. If I copy a container at a time that I know the password. And you change the password on the original, I will still have access. This is because the master key does not change when you change the password. Only the header changes to allow the new password to combine with the new header to satisfy the same master key.


If one restores the old header to the new file the old password will unlock it.

smmatteson
smmatteson

@TechrepLath @smmatteson  We are indeed using Truecrypt containers to encrypt secure folders and files.  Please Google "Truecrypt Container."  That's what we're doing and I assure you it works perfectly fine.  A 1 Gb main folder with subfolders and files sits inside the Truecrypt container.


Your argument seems to be that allowing anyone at all - even administrators who MUST have this access - to know the Truecrypt container password breaks any  and all security benefits associated with this scenario.  Your assessment of the technique of using a shared password database among authorized personnel notwithstanding, this is how the environment is secured and the administrators responsible for it are able to do their duties.  


These people need access to these passwords, and so therefore the access has been provided to them in a secure fashion.  No one has access to passwords they should not have or is provided greater access than they need.  I'm not sure what you would have them do other than memorize passwords in lieu of using a shared password database - but they'd still know them.


Please elaborate your claim that "changing the password on a truecrypt disk is basically useless when dealing with anyone who has any level of premeditation and basic knowledge of truecrypt's use of headers and master keys."  Are you saying any administrator could copy off the Truecrypt container and then crack the encryption on it later?  If so, how?  


"The only valid case for increased security using truecrypt to access server hosted data would be the sharing of a container file which is mounted by the client."


But in this case the administrators in question ARE the clients, since they access and write to the files being secured.

TechrepLath
TechrepLath

@smmatteson


"Truecrypt was chosen over Bitlocker for it's ability to encrypt specific files/folders (in "containers")". Excuse me? I think you must be mistaken. Truecrypt has no such functionality. The closest you could get is mounting virtual disk as a folder, but you could do that with Bitlocker too. As for encrypting specific files, there is simply no way to do that with Truecrypt.


"All passwords are stored in an encrypted database with access permitted only by authorized personnel. Passwords are rotated across the board when staff leave." That would be a serious red flag in my book. The access to that database becomes a weakspot for many other privileges. Besides, as I said before, changing the password on a truecrypt disk is basically useless when dealing with anyone who has any level of premeditation and basic knowledge of truecrypt's use of headers and master keys.


"The only people with access to the Truecrypt password are those responsible for administering the server." If the people with the truecrypt password are the same who can reboot the server, have physical access to the disks and work with backups... How can you still maintain that the truecrypt encryption has any value? Different layers of security only help when there is a separation of duty.


The only valid case for increased security using truecrypt to access server hosted data would be the sharing of a container file which is mounted by the client. This would allow securing of data beyond the control of a system admin. As risky as it is though, since incorrect use could cause data corruption.

smmatteson
smmatteson

@TechrepLath  


1)  Backups are encrypted, yes.  Local disk backups in the secured data center and offsite tapes handed off to/picked up from authorized offsite storage personnel.


2)  The only people with access to the Truecrypt password are those responsible for administering the server.  Are they disgruntled?  Not to my knowledge, but this solution poses no more of a risk from disgruntled employees than any solution which permits employees secure access to the data they need.


All passwords are stored in an encrypted database with access permitted only by authorized personnel.  Passwords are rotated across the board when staff leave.  


3)  Truecrypt was chosen over Bitlocker for it's ability to encrypt specific files/folders (in "containers") and entire volumes rather than just entire volumes.


4)  The data is protected from any unauthorized access by individuals not in the group that has access to the data.  This group includes the administrators of the server since the data pertains to their work efforts.



TechrepLath
TechrepLath

@smmatteson 


Before I formulate any additional critique, let me ask the following questions.


1) You mentioned backups. Are they encrypted?

2) Everyone who ever needed to reboot the server has the truecrypt password. Any disgruntled employees in that group of people? (note that changing the truecrypt password does very little to increase security)

3) Any reason why you discarded Bitlocker as an option?

4) Who are you protecting the data from? (In your previous post you downplay the importance of the data.)


5) Given your description of sufficient physical security (including burly IT guys) what do you think would be the most likely mode of attack for someone who would want to access the data? (don't answer that one, I just want you to think about it)

6) "the password is lengthy and convoluted" and possibly written down somewhere? (don't answer that one either)

smmatteson
smmatteson

@TechrepLath  "In fact, in this case, since the truecrypt volume is perpetually unlocked, it is in fact an unlocked lock on a screen door."  


I disagree and I will explain why.    


When the OS on the server is running and the Truecrypt volume is mounted, if you took a fire axe and smashed your way into this company's secured data center you'd have about two minutes (during business hours) to get at the physical server before you were overpowered by burly IT guys (yes, these exist :-).  Off hours, maybe ten minutes, but you'd have had to physically break into a lot more stuff setting off a lot more alarms and bringing more security guyards.  


So, in those precious few minutes you've got to figure out how you can access the mounted Truecrypt volume to steal files off it.  Can't log into the server, so all you can do is keep trying passwords - which are securely locked up in another encrypted database - and hope you gain access.  Your ONLY shot here is to yank those drives out of the server and run like hell.  That dismounts the Truecrypt volume once you do.


Back in your lair (assuming you get away) you mount the drive on another system and try to get at that Truecrypt volume.  You cannot.  It's not mounted.  You don't have the password to mount it.  You can try a brute force attack to get at it, but it would take years if not decades.  Oh, and the data on that volume isn't the kind that would cripple the company if you released it or blackmailed the organization with it.


The key here is that when the Truecrypt volume is mounted it's inaccessible by physical means since the passwords to the server are safeguarded.  Only the appropriate personnel can log into the server.  


When the volume is unmounted nobody except the appropriate personnel can actually mount it since the password is lengthy and convoluted.


I appreciate your questioning the methodology behind this - independent analysis of how these things are set up is always a good idea and an extra set of eyes is never harmful - but I fail to see how anyone can gain access to this Truecrypt volume based on your claims.

Editor's Picks