The company's Financial Controller has dashed your hopes for a server upgrade, leaving you with a major storage problem. Your users are quickly chewing up hard drive space, and you can’t buy any more. What do you do? In a flash, your mood changes and all resentment towards the finance department evaporates as you realize that the tape drive on your network holds the key to your server-upgrade problems.
You can solve your storage problems using your network’s tape drive in conjunction with the Remote Storage System on Windows 2000. In this Daily Drill Down, I’ll introduce you to Remote Storage System and show you how to use it to increase storage space on your network.
What is remote storage?
Microsoft's Remote Storage System (Remote Storage) is a convenient and cheap method to free up disk space on hard drives by moving files from disk to a tape. Tapes are a lot cheaper per gigabyte than hard disks (high capacity SCSI ones, in particular). Therefore, if you can't get the funds to purchase extra hard drives, you can achieve virtually the same result by using remote storage. I say “virtually” because there is one important difference between the technologies: read and write speeds. Accessing files on a hard drive will always be faster than doing the same from a tape, but if Remote Storage is configured properly, the speed penalty shouldn't become a nuisance.
The efficacy of Remote Storage is largely dependent on how files and volumes are configured for Remote Storage. You’ll need to configure managed volumes and set attributes such as minimum disk space thresholds on volumes (beyond which Remote Storage kicks in), file size limits, and last-accessed dates, which determine when files are moved to and from Remote Storage. When files are moved, cached copies are left on the local disk with a placeholder pointing to the location of the managed file on tape.
Remote Storage is split into two levels, Local Storage and Secondary Storage. The former refers to the hard drive volumes on W2K Server, the latter to the tape library. When a user needs to access a file in Secondary Storage, the files appear as if they are still on a hard drive. The slight delay in file retrieval will usually produce a pop-up window explaining that the file is being recalled. If the file is used frequently after a long dormant period in Secondary Storage, then Remote Storage will determine whether it should be moved back into Local Storage.
Remote Storage only runs on Windows 2000 Server, not Windows 2000 Professional nor Windows 2000 Advanced Server. However, the Remote Storage Administration snap-in is in the W2K Professional Admin pack. Therefore, even if you can’t deploy Remote Storage directly on your client, you can still use Windows 2000 Professional to administer remote storage on your server.
Remote Storage only supports Windows 2000 Advanced Server as long as the server running Advanced Server isn’t part of a cluster. Remote Storage doesn’t include a failover system, so clustering isn’t compatible with remote storage.
Remote Storage only works with tape drives. You can’t use remote storage devices such as ZIP drives, Jaz drives, or CD-RWs. Remote storage only works with 4mm, 8mm, and DLT tape drives. You can double-check the Windows Hardware Compatibility List to see if your device is supported.
To run Remote Storage on your server, you must have formatted your server’s volumes for NTFS, which you should be doing anyway. Remote Storage works in conjunction with the Removable Storage Manager, which requires that you place sufficient media into the free media pool.
Don’t confuse remotely stored files on a tape with your backup tapes. Backup tapes contain mirror images of all of the data on your network. Remotely stored files exist only on tape, and not on the server. Also, backups are completely offline. You can only access the files contained on the tapes by using your backup program. To a user, files stored using remote storage look like any other files on your network. All that’s different is the location of the files, which are on tape rather than on disk.
Likewise, don’t neglect tape backups just because you’re using your tape drive for Remote Storage. Backups are equally important, if not more so when you’re using remote storage because the operating system must track file locations and conditions more closely than normal.
Installing and configuring Remote Storage
Remote Storage is not installed by default when you first install Windows 2000 on your server. You must manually add it as a service when you decide to deploy it. However, before you install the Remote Storage Service, you must configure the server.
First, install and configure the tape drive that you’ll be using for a remote storage device. Remote Storage does not detect devices that are added after you install Remote Storage. If you add devices after you install Remote Storage, you must remove and reinstall Remote Storage to configure the device.
The second thing you must do is to configure the volumes on your server. If you plan to use indexing and/or compression on the volumes that Remote Storage is going to manage, you need to set them up beforehand. As with hardware devices, if you decide later to add indexing or compression, you must remove and reinstall Remote Storage.
After you’ve prepared your server, you can add the Remote Storage services. Click Start | Settings | Control Panel | Add/Remove Programs. When the Add/Remove Programs window appears, click Add/Remove Windows Components. This will start the Windows Components Wizard.
Scroll through the Components list box and select Remote Storage. Click Next to Continue. The wizard will begin copying the Remote Storage files to your server. When the copy completes, restart your server.
When the server restarts, click Start | Programs Administrative Tools | Remote Storage. When the Remote Storage Setup Wizard starts, click Next. The Remote Storage Setup Wizard works basically like every other wizard you’ve ever used while administering Windows 2000. Some of the key points you need to be aware of while the wizard runs include:
- The wizard detects any compatible storage devices and checks the security settings. You need to have already installed a compatible storage device. If you haven’t, the wizard will display an error and exit.
- The wizard will prompt you to select the NTFS volumes with which you want to use Remote Storage.
- You can set the minimum amount of free space that you want to have available on the managed volumes in the Desired Free Space box.
- You can also set the size of the smallest files that you want Remote Storage to move to tape if you require more disk space in the Larger Than box.
- By setting the number of days in the Not Accessed box, you can control the amount of time that the file is not accessed before the file is moved to tape.
- The Change Schedule button allows you to change the date and the time that Remote Storage copies files to tape. Choose a time when most of your users are not on the system.
Managing Remote Storage
You’ll manage Remote Storage using the Remote Storage MMC. You can start the Remote Storage MMC by clicking Start | Programs | Administrative Tools | Remote Storage. You’ll see the following nodes in the left pane when the MMC starts:
- The Managed Volumes node, which shows all volumes that have been selected for management by Remote Storage. You can add or remove managed volumes from here.
- The Media node, which shows all statistics relating to media in the Remote Storage media pool. It is empty until a task is performed.
- The Event Viewer node, which generates service and driver logs specific to Remote Storage.
- The Removable Storage node, which displays all information regarding physical media, pools, offline status, and location.
Even though the Remote Storage MMC is the central management location for the service, managed volume statistics can be viewed in other areas as well, such as the Storage node in the Computer Management snap-in and/or Windows Explorer’s Volume Properties. These additional areas increase your options when you must quickly view properties. Settings, however, can only be changed in the Remote Storage MMC.
Once you familiarize yourself with the basic areas in the MMC, you can get down to the business of managing Remote Storage. In the sections that follow, I’ll show you some of the most common tasks you’ll face.
Working with Include and Exclude rules
One common task you’ll face involves creating, editing, or removing Include and Exclude rules. These rules allow you to control what file-types can or cannot be moved into Secondary Storage. To set Include and Exclude Rules, right-click the volume in question by selecting the Managed Volumes node in the left pane of the Remote Storage. Select the Include/Exclude Rules tab.
If you want to add a new rule, click Add, and then either type the folder path that you want to apply the rule to or type a backslash to apply the rule to the whole volume in the Path box. Select the type of file that you want to either include or exclude from Remote Storage in the File Type box.
You can choose specific files to include and exclude. Click Exclude Matching Files if you want to exclude files that match certain criteria from Remote Storage. Click Include Matching Files if you want to explicitly allow Remote Storage to manage files that match certain criteria.
If you want Remote Storage to apply the rule to all subfolders of the folder that is listed in the Path box, select the Apply Rule to Subfolders check box. Remember that Remote Storage works best when used with large files. If a managed volume contains numerous small files that are eligible to be copied to Remote Storage, the data transfer rate may not match the rating that is given for the tape library.
Finally, you need to think about how to handle conflicting rules. You can set the priority of rules using the up-down arrows on the Include/Exclude tab. The up arrow causes a rule to be given higher priority and vice versa.
When you’ve configured volume management and set any rules you need, Remote Storage will run a Copy On Demand task. This scans the volume for eligible files (according to the rules you set) and moves them onto tape. The Copy On Demand process can be kick-started manually by right-clicking the desired volume, selecting All Tasks and then selecting Copy Files To Remote Storage
Remote Storage leaves a cached copy of the files on the hard drive and only frees them when the space is actually needed. This gives you the best of both worlds because you know that you can free disk space safely if you need to, but until you do, your read-write speeds will remain unaffected because the file(s) will remain on hard drive.
When you need to free up that disk space, you can just right-click the volume in question, select All Tasks, and then select Create Free Space.
Both Copy On Demand and Free Disk Space On Demand run as tasks in Scheduled Tasks. You can set your own schedule if you did not already do so when you configured the managed volumes with the wizard. A default schedule will periodically run checks on all managed volumes to make sure the criteria you set are being adhered to if you did not specify a schedule in the wizard. Files that have been moved to Remote Storage are marked with an arrow, similar to the marked files in the Windows 2000 Offline Files system.
Dealing with runaways
The Runaway Recall Limit affects the number of files a user can recall over a 15-minute period. The purpose of such a limit is that recalled files are copied back to Local Storage, thus reducing the amount of free disk space. While this may not be a problem with only a few users, it could become one with several hundred users, where files are constantly being moved between Local and Secondary Storage. The workload on the server would increase accordingly, so some means of control is necessary.
The default value for the Runaway Recall Limit is 60. If a user makes more than 1 file request every 10 seconds, the count is incremented. The Runaway Recall Limit can be fine-tuned by preventing applications that are prone to cause numerous file recalls (e.g. searching, anti-virus, content indexing) from recalling files. This feature can be configured from the properties of the Remote Storage node in the Remote Storage MMC.
Working with media copy sets
Remote Storage allows administrators to make copies of data in Secondary Storage. It is important to protect the integrity of this data because it is not protected through conventional fault tolerance methods such as mirroring and/or striping. Copying media sets caters to the need for media rotation and makes offsite storage possible. These two items alone will make your life a lot easier in the event of media corruption or disaster recovery.
Remote Storage media sets can also be stored with backup sets, enhancing the value of a backup routine by putting all your data in one place where it is more quickly recoverable. To create media copy sets, select the Managed Volumes node in the left pane. Right-click Remote Storage, and then click Properties. When the Properties window appears, select the Media Copies tab, and then specify the number of media copy sets that you want to create for each media master set.
Synchronizing media copies on a regular basis ensures that data on a media copy is identical to the data on the media master. Synchronization updates media copies of all the media masters. You can only synchronize one media copy set at a time. Files cannot be managed or recalled while a media copy set is being synchronized, even if additional drives are available. You can synchronize media copies later if they are removed from a library.
To synchronize media copies, right-click Media in the left node and select Synchronize Media Copies. You’ll then see the Synchronization Wizard appear. Follow the prompts in the wizard, setting a synchronization schedule that best fits your organization’s needs.
Configuring Remote Storage validation
Remote Storage validation checks that all the data located on your managed volumes point to valid and correct data in Remote Storage. It also calculates volume statistics. Validation is automatically performed two hours after you use a backup application to restore a Remote Storage file. Validation also detects files that have been moved from one local volume to another or that have been modified.
Performing validation on a regular basis is a good idea, to both validate all files on a given volume and correct any inconsistencies. You should perform validation after you restore files on a local volume or after any disk errors on a volume.
The validation operation is integrated with Scheduled Tasks. You can use Scheduled Tasks to monitor validation. To perform validation, right-click the relevant volume and select All Tasks. Finally, select Validate Files.
Backing up Remote Storage data
Data managed by Remote Storage have a reparse point. This point provides the link from the current location of the data to its copy location on tape. This data must be protected by a backup routine. There are two methods that you can use to backup Remote Storage data: you can backup only reparse points or you can backup the complete file(s).
The first method provides a backup of the reparse point data on a volume, as well as all of the files that are not managed by Remote Storage. This is the default behavior of Windows 2000 Backup. Use the second method when you need to make a full copy of the files in Remote Storage. You should use this approach judiciously though, because it can take a long time to back up all of the files in Remote Storage.
To recover managed files from the backup set, perform the same restore operations as you would to restore files that are not managed by Remote Storage. As mentioned above, Remote Storage detects when restore operations have occurred and automatically schedules a validate job to run against the restored volume two hours after the restore.
Store more with Remote Storage
Remote Storage provides a powerful mechanism for managing hard drive resources. It offers numerous ways to configure the service so that most bases can be covered. Attributes such as file size, last-access dates, and inclusion/exclusion rules permit granular control of the Remote Storage service. Remote Storage can also be used in conjunction with removable storage and Windows 2000 Backup, thus providing a well-rounded system for data management, protection, integrity checking, copying, and restoration.