How do large companies handle Exchange backups?

Mike,

From my experience most large organisations set the deleted mailbox items and deleted mail items settings to a large enough value. This could be 1 month for both. By utilising this method you eliminate the need for brick level backups which are painfully slow. I personally would not use brick level backups.

The users have to understand how to recover their deleted items from within Outlook. If they shift-delete a mail then it does bypass the temp holding area, but then they must have REALLY wanted the mail gone!

Another aspect that has particular focus, is the VSS snapshot capabilities of Ex2k3 and SAN's. This has the ability to take quick backups and very fast restores.

Once you have a good backup then you can use the Recovery Storage Group on your Exchange 2003 Server to help get the deleted and purged mailbox back onto your production mailbox store.

Hope that helps,

Sunny ;-)

I no longer do the bricklevel backups, I use a tool called Powertools Single Mailbox Restore, it allows me to recover a single mailbox or even a single mail from a mailbox without having to recover the database to an alternate server.

The normal backups are done in the same fashion os you using Veritas.

I have also worked in a companies that used Networked Attached Storage (NetApp) devices which used a technology similar so shadow copy which backed up the database and kept this file on line, dependant on space it could hold 256 backups. its called Snapshot.

Really worth checking out.

Carl

I am responsible for a company w/ about 125 employees . . . here's my setup, and it works quite well:

* Exchange 2000
* ArcServe 2000 for Exchange 2000 (perform BOTH brick-level and Exchange database backups nightly)
* 12 / 24 GB DDS3 Tape drive is located on the same server as Exchange / ArcServe

* I limit my employee's mailboxes from 20 MB to 40 MB (depending on job title / function / that's everything from Outlook Today - Tasks) which forces them to use their Personal Folders which are stored on another server / captured on a different backup routine. Typical backup time is approximately 5 hours.

* I consistently educate my employees on proper email usage / storage / etc via "Tips" each month as well as a rigorous "New Hire Orientation" session on their first day of employment. As they say, an ounce of prevention . . .

Mike,

Prior to Exchange 2000, the answer to your question was to have multiple Exchange servers in your environment. Doing the same thing you are doing for many servers.

With Exchange 2000 came the ability to have multiple storage groups. Used wisely storage groups can split your user base into administrative groups of users based on geographical or organization considerations. Your backups sets should be based on these storage groups.

Finally using LTO drives or automated tape libraries and Exchange aware backup applications allow you to backup your users mailboxes, information store, and directory at any time you wish without disrupting your enviroment.

I myself use a two-stage server replication scenario. I replicate (using XOSoft WanSync) to a hot standby server in my location and to another hot-standby server in a remote location. I then take my backups off my replicated servers.

The type of backup you take also plays a huge part in the process. We take full info store backups with flushing our log files every Monday. Then full backups without flushing the logs every other day. On all days we take full backups of just our logs eveyr two hours and place them on another server. Each Tuesday you restore your full backup to a warm server without ever starting the information store. That way you can restore your hourly log files to the warm server, start the info store, replay the transactions and be up and running within a couple of hours. (our server replication has replaced this as our primary recovery method) we now use it for ad-hoc mailbox restores.

Many ways to tame a tiger, this is just one.

Heres what we do:
We still run our standard Exchange(Information Store - not Bricks Level) backup, but we also use Exmerge. This utility can be found at :/Program Files/Exchsrvr/bin. We configured Exmerge to run automatically every night and to dump the data to a network share.
When Exmerge runs it dumps a copy of the contents of each mailbox into a pst file. As mentioned earlier the *.pst file saved to a network share.
Running this process for the first time may take several hours as it has to backup all of the contents of each mailbox. Subsequent jobs usually only take a few minutes as exmerge basically does an incremental backup. Keep in mind that Exmerge adds new mail to the pst as it comes in nightly, but does not delete mail from the pst that the user has deleted from thier mailbox. To work around this we delete the saved pst's every Sunday and start the process over on Monday. Make sure you also back the pst's up to tape. If you follow this procedure you will always have instant access to any email that was accidentally deleted within the last seven days without having to go to tape, access to deleted mails older than a week by going to tape and no need to do a bricks-level backup. This process can also be used as an archiving process if done correctly. I could go on for quite a while, so....... jump in and give it a shot. You will better understand it by going thru the process itself.

I came across this article using just ntbackup to backup exchange 2003. Thought it might interest some of you. I haven't personally used it though.

http://www.microsoft.com/technet/itsolutions/msit/operations/exchbkup.mspx

Mike,

I am the Supervisor of Network Services for a community college in Arizona that has a very large on-line student population, in excess of 46,000. I just implemented Exchange 2003 on two Dell 6650s? in an Active/Passive backend server mode, with WIN 2003, and like you I have a Dell Powervault 136T with Veritas 9.1. I also implemented three Dell 1665 MC Blade servers as the Exchange Front end servers for the Outlook Web Access (OWA) component.

The first time I tried to backup the entire 46,000+ mailboxes Veritas only completed 1/3 of the mailboxes, and it took 36 hours to do that. The information store was 2.9 GB, and that was with all empty mailboxes.

I got on the line with Veritas and got connected to a Tech who worked with me to backup the information store and backup all the mailboxes, but in a unique way. Veritas suggested that I continue to use Veritas to backup the information store but not backup the mailboxes using Veritas as it would take too long.

Instead Veritas suggested that I use NTBackup to backup the mailboxes. When I did an NTBackup it backed up the entire information store in only 2 ? minutes. They also suggested I use the Microsoft Exchange Recovery Storage Group to be able to restore the information store to, and use the Microsoft ExMerge utility to recover a users? mailbox. This worked great as I can now schedule an NTBackup of the store every two hours. I also use Veritas to backup the NTBackup throughout the day giving me greater flexibility in data backup, data recovery, and at the same time provide excellent customer service. You can even restore a mailbox while the person is still logged in, and when they refresh their screen anything that was deleted is restored.

On another note Veritas also suggested that is you have a very large user base it is wise to split them up into different Storage groups, and you should have only one mailbox store per storage group.

Why you might ask?

It is very simple; it has to do with the logs that are created for each mailbox store. If you create multiple stores in one storage group then all the stores are sharing one log file. By creating different storage groups and putting only one store in each that you have only one log for that store. When you have to do a recovery your data and logs are together for each storage group and store which makes for a very fast recovery. All of this is located on the Microsoft website under Exchange 2003. It has plenty of great How To articles on creating the Recovery Storage Group, Recovery of Mailbox Stores and Mailboxes using the Recovery Storage Group, Merge the Recovered mailbox with the regular mailboxes, and the most important is article 322312. This article will explain what happened when the ?Mailbox Merge program tries to Open the Message Store, and the operation is Unsuccessful.? Read this article first; believe me it will save you time.

Good luck and if you need more information please do not hesitate to contact me.

Peter

Please always ensure that there is an up to date backup before running any tools on the Exchange Database.

Stop Exchange Information Store under Services
Goto c:\exchsrvr\bin

Private
Type in "isinteg -pri -fix -test alltests"

This will scan and fix all the private information store folders - or
mailboxes

Public
The following will scan and fix all public information folders.

Type in "isinteg -pub -fix -test alltests"

MTA Check
Stop the MTA service.

Run "mtacheck"

This is also a good thing to run at least once a month to check and test
the integrity of the database.

I have used both Veritas Backup Exec and ArcServe. ArcServe beats Veritas hands down in every regard. I will never touch Veritas products again if I have my way. Perhaps a newer faster tape unit? Of course, you can fix just about anything if you throw enough money at it.

I still suggest the Bricks Level and the information store for the fastest disaster recovery options no matter what happens. You need both, but I will be looking into the suggestions on using NTBackup for this. Interesting suggestions.

We tried Arcserve on an Active passive cluster for Exchange 2000. It always said that it backed it up, but the restore was a different issue entirely. They sent out a tech who could not resolve the issue. We went with Commvault which does very nicely. It facilitates many different types. Brick level Info Store level and you can even divide up info stores into logical pieces.

If you remove the dumpster.dll from the client, the users do not have access to the recover deleted items folders. That and a policy that states do not delete items for X amount of days can ensure that they don't get rid of email they may later want to get back. Commvault also backs up the deleted items folder nicely.

All in all I am very pleased with the performance and ways in which you can fine tune backups.

Forget about tapes -- go with continuous, off-site backup using LiveVault...
(http://amtechgroup.com/livevault/)

We have 2000 users and a 35 Gig rapidly growing IS on Exch 03. We moved up to Exch03 from 55 where we had manageable backups. Now, that is not the case.

We just purchased a NAS device and I'm interested to hear about NTBackup doing brick-level, which honestly I have not seen in there. We are doing file and Exchange IS backups now to disk on a separate server, but that server the backups are going to does not always cooperate and the job fails, can't find the media. This is bad.

When we got Exch 03 we had to stop writing directly to our tape library because after we bought new servers (one for mailboxes and one for OWA), new Exchange licenses, there was no $ left for new backup software. In the past I've used Veritas with Exchange agent, but I am definitely looking for an alternative.

One suggestion for some cases, might be to put in a second mailbox server in a separate administrative group and then move users to that server. This is the only way to establish separate backup policies. Just adding a second mailbox store will allow to set differential storage limits, but the IS still is one big gig.

We have never done brick level because it takes too long. I wonder if there is new software that would let me do brick levels to disk. I'm interested in Commvault or others that would easily do that and use the NAS device. It sounds as if this is something to consider and I appreciated joining this discussion and would like to hear more from anyone who has more disk-to-disk experience especially related to specific software that does that well for Exchange IS.

While i agree that RSG's eradicate the need for brick-level backups. To my understanding RSG's install onto the same box as the main information store. Could you setup an RSG on another box ? Because an RSG would be a futile point if the entire hard disk were to crash -- what to do then ?

Thanks --