Storage·2,078 discussions

A SAN of a WAN problem.

Locked

Recently, my company doubled in size by employees and datacenters through an acquisition of another company. My boss has presented the idea of using the newly acquired site for our disaster recovery site and replicate our SAN storage data to them. I currently run full backups every night that total in approximately 1.2 Terrabytes of data. He (my boss) decided that a DS3 line would suffice for the replication of the 1.2 Terrabytes, but I calculate this would take about three weeks. Obviously this would not work and installing OC-48 circuits is out of the question. Does anyone know of whitepapers or case studies of companies trying to replicate disk-to-disk backups of this size over a WAN connection? My goal is to present the infrastructure and technologies needed to accomplish this feat.

Topic

Several ideas

In reply to A SAN of a WAN problem.

You might want to look at “seeding” copy the data to additional hardware, and send the hardware over. Then when you’re making additions to the backups [not always fully resynching, I hope] that much traffic will fit over your pipes.

If you are doing full backups each night, you might consider changing strategies. Rather than spending the money on a fatter pipe, perhaps you want something like eVault to run your backups. We now backup each site locally, then backup the backups to the remote sites.

The % change for the backups is relatively small and easily fits over one to four T-1s. (several remote offices of various sizes)

I’m not associated with eVault, but we somewhat recently moved to their technologies (away from Veritas) and the whole team is rather enthusiastic about how much faster and easier the backups have become. When I find software like that, I share it.

From a support viewpoint, they have also been great. I highly recommend looking into them (or someone ith technology like theirs) before throwing money at a bigger pipe.

I see what you mean.

In reply to Several ideas

Sync’ng the hardware locally then shipping it to the remote location is actually the smarter idea. Changes (or differentials) can be done overnight across the WAN. If there were a mass disaster then we can always restore from tape, but anything else can be done from SAN to SAN. Full disk-to-disk backups would still be a viable solution, just resync only what has changed and should be okay.

Thanks so much for your input, Dpetrak!

sync or async replication

In reply to A SAN of a WAN problem.

YOu might want to investigate synchronous or asynchronous SAN replication. This approach provides a realtime (sync) or deferred (asnyc) replication capability. Also the use of a “snapshot” of the data and then performing backups from the “snapshot” might solve the issue.

One other thing

In reply to sync or async replication

Our most important data (SQL) is recreated every night! My company’s data is ITD (inception-to-date) which is in a nut shell every thing from day one until today. This means that I cannot perform differentials on most databases, with the exception of documents in the file server. Some of the database are not that large, but collectively we could be talking in the range of 100Gb to 200Gb of databases. We may end up needing a very high bandwidth circuit to accomodate our DR strategy.

TLog backup, transfer and restore

In reply to One other thing

Is it possible to make several transaction log backups for SQL, for example every 30 min., copy them across WAN and restore on DR site ?

Regards,

sql server

In reply to TLog backup, transfer and restore

yep, you cna do the tlog backup overnight, enterprise edition of sql 2000 is easier to do this on, put the database into full model and get the database to backup the tlog every hour and transfer this across and apply this log to a copy of the server you have created the other end. sql 2005 makes this a lot easier as it is designed with this functionality in mind and it isn’t too far away now.

Reply To: A SAN of a WAN problem.

In reply to sync or async replication

We do use Rsync todo incremental backups with the
security of SCP command. Also we do backup only
data and information and when things go wrong we
do not bother to restore the data we simply mount
the this directories over the wan as samba/nfs
directories. We do backup multiple servers this
way and it doesn’t let us down

More on async copy

In reply to sync or async replication

I do not know which is your storage hw, what kind of arrays you have (if any). If you have arrays enabled with iSCSI and you can use them to do replication via ip altogether with suitable software. I can tell at least of one: EMC’s Incremental Sancopy. Using that you dont have to do full copies all night, it will just copy the data that has changed the night before, so I think that could help with the bw problem.

Thanks

EVault

In reply to A SAN of a WAN problem.

Evault has a good product and it works very well, if you are interested let me know, and we can discuss or talk over the phone. Email me your personal or company contact infomration.

Bob

NetApp storage eliminates these concerns

In reply to A SAN of a WAN problem.

NetApp storage works in a SAN and NAS environment simultaneously and its disk to disk backup is at the block level. So, its use of WAN backup is optimized. The federal government loves using this technology for local central repositories that store repllicated backups from remote locations.

http://www.netapp.com/tech_library/3252.html

It’s a business problem first

In reply to A SAN of a WAN problem.

First and foremost your management team should find answers to the following questions prior to accepting the current mode of operating your data stores:

What data needs to be backed up and why?
When does the data need to be backed up and why?
When and/or how often does the backed up data need to be accessed and why?
What is the physical distance to manage an offsite storage solution that makes sense to your business?

A professionally developed business contingency plan will answer the questions above – plus dozens more.
Thank you,
James.

A good answer

In reply to A SAN of a WAN problem.

There are two software programs I will suggest. I have looked into them both, and am deciding to go with NSI Double Take. The other is Legato.

Both of these will backup your data to a remote site at the bit level. This is especially great for SQL, since you do not need to backup the whole DB across the WAN everynight. The system will send the updates instantly.

You can have your DR site running in less than 30 minutes.

Double-Take and WAN Accelerators

In reply to A SAN of a WAN problem.

Ditto on the recommendation for NSI Software’s Double-Take. You might want to do some I/O profiling to determine how much bandwidth real-time replication will consume. Double-Take gives you read access of the replication target, so you can backup the target with no disruption to the source server. You can also fail services over to the target.

HP recently announced WAN Accelerators that increases throughput up to 100x in environments where latency and application ‘chattiness’ is high, or up to 20x in environments where latency is low. Testing has yielded over 15Mb/s over a T1 line. Models are available for connection speeds ranging from 512Kb/s to 45Mb/s.

SAN Fabdric Replication

In reply to Double-Take and WAN Accelerators

There are some smart devices which will only capture the writes and send it across WAN. One of the vendor name comes to my mind Kashaya. Take a look at the technology if it meets your requirements.

Incremental

In reply to A SAN of a WAN problem.

Signiant has a white paper that takes a look at wide area backup best practices.

http://www.signiant.com/documents/WhitePaper_RemoteDataBestPractices2005.pdf

Their Mobilize product automatically initiates backup consolidation on a scheduled or event-driven basis. It identifies changes made to remote files since the last backup and moves only the bytes of those files that have changed to the central site. They have a calculator tool that will estimate the process time.

If you have to ask…..

In reply to A SAN of a WAN problem.

It is not the amount of your data (which is small) that is the problem. It is your mixed hardware assets. There are no case studies or white papers that can address your specific SAN to SAN backup issues because each situation is unique. They range from mildly problematic to impossible. It is the realm of high priced consultants who will gladly tell you that, ‘if you have to ask, you’re out of your league’. The fact that you are talking about ‘replicating…backups’ suggests that you may want to re-visit some fundamental concepts of disaster recovery. Not being offensive, just friendly advice.

Replicating 1.2 TB via a DS3 / T3

In reply to A SAN of a WAN problem.

You are correct that replicating your FULL 1.2 TB over a DS3/T3 (44.736 Mbps) connection is not workable, however it should only take about 3600 minutes (60 hours / 2.5 days) not three weeks. This assumes you have the full pipe available for your replication tasks – so you should probably double this replication time estimate to 120 hours / 5 days. However, I question why you want to replicate monolithic backup files versus doing block-level synchronization of your actual files and databases. Sure it might take 5 days to get the first copy across the DS3/T3 but the problem you should be looking at it is what % of your data CHANGES on a DAILY basis AT BLOCK LEVEL. In our work with hundreds of companies, we have seen a <5% DAILY RATE OF CHANGE AT BLOCK LEVEL (as opposed to file level changes). If the replication was done in a batch mode at night, this could be accomplished in about 180 minutes (3 hours) over a dedicated DS3/T3 circuit. Far better would be to synchronize changes several times a day - perhaps as frequently as every 10 minutes. Please contact me directly at plinden@datapreserve.com if you would like to discuss potential options.

HP OpenView Storage Manager

In reply to A SAN of a WAN problem.

Take a look at this product. After the first copy (full replication) it can replicate just the changes – very close to real time (you can choose how much bandwidth to use – which will impact on how fast the changes are replicated). We use it in just such a situation for our clients, it’s been around for 15 years so it’s very stable. Supports file and block data (such as Exchange and SQL).

ONLY 70+ hours for a FULL transfer

In reply to A SAN of a WAN problem.

The DS3 would give a throughput of 2.25 MB/sec of actual data transfer rate. Depending upon your current data size of 1200 GB and I ssume incremental growth of 12 GB daily, it would take 75 hours for a full transfer and 0.65 hours for incremental data to be transfered.

You need a REO4000 2TB box at both ends with the REO Multi-site Pac option providing you byte level incremental transfers with compression capabilities. The cost sums up to around 9000 $ per site.

Replication technology.

In reply to A SAN of a WAN problem.

Have you looked into what FalconStor has to offer?They offer replication as well as remote asynchronous replication for heterogeneous storage environments.
They should be able to provide you with multiple options for replicating.

[Author]

Replies