This article was originally published in the Internet Security Focus e-newsletter.
In a previous installment of the Internet Security Focus e-newsletter, I presented a scenario about an automated, distributed backup system that relied on someone to change the physical tapes. The fatal flaw in the system was that nontechnical staff at the branch offices was responsible for changing those tapes, thus forcing nontechnical personnel to perform technical operations.
Almost everyone who replied to the discussion thread immediately got what I was looking for—that nontechnical personnel would be responsible for technical issues. Bonus points go to those who offered suggestions such as moving the backup procedures to places where technical staff already exists.
Using nontechnical staff at branch offices to perform technical operations is a common concern among distributed-environment companies. For smaller shops with only one site, it isn't as much of an issue—unless the company has no tech staff at all. However, the majority of midsize to large enterprise shops have regional technology professionals located in only the largest offices for each region and no tech staff at the other sites.
How do you properly handle disaster recovery issues in offices where there isn't anyone skilled enough to handle such procedures? DR is by no means the only technical issue at stake, but it's a primary concern since the lack of technical knowledge will eventually cause a disaster at one of these sites.
One solution is to train and/or hire staff members who can properly configure, install, and manage the necessary technology at each location. Although this is an amazingly expensive option, it's the best possible way to handle the situation. With the proper staff at each site, they can easily handle both day-to-day tasks and emergencies.
A more cost-effective solution is to centralize as much of the technology as possible into locations where you already have trained staff. A common example of this is the centralization of backup systems to regional and/or central offices. You can use replication technology to drag data back to central offices on a regular schedule, and you can perform backups at the central sites. In addition, spare hardware is stored at the central site for whole-server restorations, and it's shipped to the regional sites when required. The restoration of data via WAN links is also performed if the remote server hardware is intact.
One of the drawbacks to centralization is that it takes longer to restore data over the WAN. Furthermore, the replication systems add products that must be learned, configured, and monitored. This increases the cost of the overall DR plan, but the cost is usually recouped because you eliminate a lot of backup hardware, software, agents, and offsite storage.
Backup is by no means the only DR system you can centralize. Numerous products on the market today allow high-availability systems to function over the WAN and VLAN, and other networking technologies provide access to systems from centralized locations. By using a combination of these technologies, you can avoid the expense of having trained IT personnel at each branch office.