SolutionBase: Improve disaster recovery by learning to modify a disk in place

Discover how to use a disk editor to recover lost files.

As you know, Windows 9x isn't known for stability. One of the things that can sometimes go wrong with Windows 9x is directory corruption of the FAT32 file system. Being able to work on data in place will aid your ability to perform disaster recovery, since much enterprise data is still stored on the FAT32 file systems in old Windows 9x systems. You can work directly with the FAT32 file system using a disk editor in order to recover files directly from the disk. In some cases, you may be able to more quickly recover a lost or damaged file by working directly with the disk than by shipping it off to a data recovery service.

What you'll need
A drive editor with robust editing tools is important for getting this job done. My personal favorite is WinHex. Made by X-Ways Software Technology AG of Germany, WinHex is an advanced hex editor that includes powerful data recovery and analysis utilities. It also provides an excellent forensics tool, which you can use if you buy a Specialist license. Licenses depend on features requested, and cost $50 (Personal), $92 (Professional), and $145 (Specialist). For the features you get, it's a bargain.

Setting up a practice drive
Naturally, while you go through the steps in this Drilldown, you won't want to chance harming your data. So before practicing in-place file recovery, create a small FAT32 partition just for that purpose.

Configuring WinHex for file recovery
A few WinHex configuration changes will make it easier to recover data from FAT32 systems. If you are using another disk editor, make as many of these changes as it allows.

Make sure WinHex is not set up as an In-Place Editor. If it were, every data change would be saved immediately to the media. Under the Options menu, if In-Place Editor is checked, uncheck it. This will allow edits to be committed to disk only when you opt to save them. Note that when you're working with sensitive files, you can also check the Option marked Use As Viewer, which opens all data write-protected. Leave this option unchecked, however, because we'll be writing to the drive.

Next, make sure the Details Panel is open by selecting View | Show and checking Details Panel. This display reveals needed information about the disk. Each entry in the File Allocation Table consists of four bytes. To easily read the FAT32 structure as one cluster per line, set WinHex to display four bytes per row. Select View and click the arrow next to the option, One Column Less, until the display is four bytes across. Next to the four Hex entries will be four text entries.

Backing up the FAT
To help prevent unrecoverable disk disasters, it's an excellent practice to periodically back up your File Allocation Table. Why do this if the FAT file system already maintains a spare copy? The FAT 2 copy is stored immediately after FAT 1. Therefore, the same conditions that damage FAT 1 may damage the spare. For this reason, it's valuable to regularly backup the FAT and store it elsewhere.

The File Allocation Table for FAT32 begins at a location determined by that drive's structure. The length of the table depends on the disk size and formatting. The table needs to be large enough to contain entries for each available disk cluster (a cluster, you will recall, is the smallest unit used to store files). Fortunately, WinHex makes it unnecessary to perform many of these calculations.

Access FAT 1
In WinHex, open your FAT32-formatted test drive by selecting Tools | Disk Editor and selecting your partition or drive from the Logical Drives hierarchy. In Figure A, we're selecting the Test (E:) drive. Although it's possible to open entire hard drives through the Physical Media list, you need the Logical Drives list to access partitions. Physical Media are read directly through the BIOS and don't contain partition, directory, and other disk formatting information provided through the operating system under which WinHex is being run.

Figure A
Double-click your FAT32 partition or drive from the Logical Drives list.

After the drive loads, you'll see a drop-down box marked Access at the top of the window. Click the arrow to open and select FAT 1. The cursor will move to the beginning of the FAT 1 table. Note the starting address, at offset 4000h (h=Hex), in Figure B. Access also allows you to quickly move to the boot sector, root directory, and other frequently used disk areas. You can interpret the boot sector and root directory through templates.

Figure B
Use Access to navigate quickly to Fat 1.

The Details Panel contains much useful information, such as the drive letter, label, file system, and disk formatting structure. For example, the Panel in Figure B shows that Drive E, labeled Test, has a capacity of 36.9 MB and is formatted at 512 bytes per cluster. It also shows the location as FAT 1.

Now use the Access menu to move to the copy of FAT 2. Use the left arrow key to back up one hex value. This is the last position of FAT 1, in my case, 4CBFFh.

Block the FAT 1 and copy it to a file.
To define a block containing the FAT 1, choose Edit | Define Block. In the block definition dialog box, type 4000h as the beginning of the block and type your end address below.

Choose Edit | Copy Block | Into New File. Give the file a name such as efat1copy. Save efat1copy to a different partition—otherwise, it will be inaccessible should the test drive or partition fail. Worse, it may overwrite data you want to recover.

Congratulations. You've just backed up your File Allocation Table. Now you will be able to recover your file information in the event of a hard drive failure.

Backup your boot sector
You can use the same steps to access and save a backup copy of your boot sector to another partition or drive.

Disk destruction and reconstruction 101
Here's a quick how-to for working with a damaged FAT file table. Type a small text file containing the text "this is a test file." Save it to your new FAT partition as testfile.txt. Since your partition is empty, your system will save it to the first available cluster.

In WinHex, use Access to navigate to the FAT 1 copy. Use the down arrow to move through the File Allocation Table until you see your file, testfile.txt, listed in the Details Panel, as shown in Figure C. Because it is a small file, it will be contained in one cluster, and its FAT entry will simply be marked FF FF FF 0F, the end-of-file flag. The Panel indication Cluster 3: End indicates that this cluster is the end of a file.

Figure C
On this drive, testfile.txt is written to cluster 3 and is wholly contained within it.

Damaging the FAT
Click your cursor into the first byte of the hex (not text) display. Enter the following: 00 00 00 00. Four bytes of 00 in the FAT tells the system that the given cluster contains free space. Note that your edit is shown in blue. Blue means a change has been made, but has not yet been saved.

Select File | Save. WinHex will warn that the integrity of the drive could be severely damaged. Click OK. Again, click OK to dismiss the second warning. The entry is now displayed in black to show it has been committed to disk.

Use your text editor to try and open the testfile.txt file. You will be unable to open it, even though the file name is visible in the drive's root directory. This is because the directory entry points to unallocated space and Windows returns an error.

Repairing the damage
Using WinHex, restore the FAT entry of Cluster 3 to FF FF FF 0F and save your changes. Now you should be able to successfully open the testfile.txt file in your text editor.

If you can't open the file, make sure you entered the data in the hex—not the text—display area, and that you have entered it correctly. You have now successfully restored a file by directly editing your hard drive.

What this lesson shows
Some viruses erase both copies of the File Allocation Table, while leaving the data on the disk intact, including directory entries. This data is recoverable if it hasn't been overwritten, as in the situation we just emulated.

When FAT tables are erased, whether through disk failure, user error, a virus, or some other disaster, you have a few choices for recovering files:
  • Restoring from a backup. But often, a backup isn't available.
  • Running a file recovery program. However, most file recovery programs can only restore files that aren't fragmented. If they are, only partial files from the first clusters in the chain can be recovered.
  • Recovering files manually using a good disk editor, by copying blocks to a new file in a different partition. Although this choice can be labor-intensive for many files, if you're searching for a crucial file or two, it is often the simplest and quickest way to go.

More on navigating FAT entries
Navigate to the beginning of FAT 1. Now look at the Details Panel display. In the section next to the bottom of the Panel you'll see information about your location on the disk. Last Scanned tells you when WinHex last read the clusters of the drive you're examining. If there has been disk activity since you opened the disk, you'll want to rescan. Select Tools | Disk Tools | Rescan Cluster Chains.

Note that the first four-byte entry is marked "reserved." The beginning of the FAT contains two special entries, F8 FF FF FF, and FF FF FF FF, reserving clusters 0 and 1. The first cluster available for files is cluster 2.

From this point on, FAT32 four-byte entries tell the OS that:
  • The cluster is free space (00 00 00 00).
  • The cluster is bad (F7 FF FF 0F).
  • The cluster contains a file continued on another cluster. In that case, the Hex value points to the cluster where the file continues.
  • The cluster contains the end of a file (FF FF FF 0F).

The FAT32 table, then, is simply a database of pointers to clusters, where the address of the FAT is equivalent to a cluster. Here's how you can calculate it.

The cluster referenced by the FAT32 address equals the address, minus the offset of the start of the FAT table, divided by four (because there are four bytes per cluster). For example, a FAT32 address of 457Ch = cluster 351 (dec.). The FAT 1 begins at address 4000h. 457Ch-4000h = 57Ch. 57Ch = 1404 (dec) /4 = 351. If that entry contains 00 00 00 00h, then cluster 351 is free space.

Figure D shows a few entries from a typically dirty FAT (containing fragmented files). Line A points to a file entry at address 4034h, which is for cluster 13. The pointer in the FAT is 0B 00 00 00. Therefore, the data in cluster 13 is continued, not on 14, but on cluster 11 (OBh=11d). The entry in line C, on the other hand, flags address 403Ch (cluster 15) as free space.

Figure D
Three entries from a FAT in which files are fragmented show how the FAT database tracks data on the disk.

The cursor position at offset 4034 (shown in line B) is interesting. The Details Panel for offset 4034 displays a file named TechRepublic. This is actually a directory entry. In the FAT32 file system, directory entries are files, too. Just like other files, their minimum space allocation is one cluster, and they grow as needed, becoming fragmented if no adjacent free space is available.

Directory files contain information about the files and subdirectories within them, such as their DOS 8.3 names, long file names, size, and start cluster. These 32-byte directory entries, in conjunction with the FAT, allow you to manually recover files. File recovery programs also use this information to reconstruct as much of a file as they can.

The FAT reveals that the TechRepublic directory is not continuous. Its file chain begins at cluster 13 and skips way up to cluster 47,406. The Details Panel does the calculating for you, showing this skip as "13 > 47,406."

However, the two bytes of the entry, 2E B9, seems to point to cluster 11,961. Why this discrepancy? FAT32 flags are in reverse order, with the lowest order byte on the left. The correct cluster, therefore, is read as B9 2E: 47,406.

Quick conversion
To quickly make hex and dec conversions, open the Windows calculator by selecting Start | Run, typing calc, and pressing [Enter]. Switch to scientific mode by selecting View | Scientific. Click the Hex radio button. In the entry area, type your hex value, then click Dec to convert it. Or start with a decimal entry and click Hex. (You can also perform these conversions in Excel using the HEX2DEC and DEC2HEX functions.)

Recovering files manually
You now know all the information required to recover FAT32 files. If you have a backup copy of a FAT that contains the cluster chain for a lost fragmented file, and you act quickly enough so that all or most of the data hasn't been overwritten, you can retrieve the data, as in the following scenario.

Let's say you accidentally, permanently deleted the only copy of a file, HP Blade PC.DOC (44 KB). You need this file in order to make a presentation to your CIO. How could such an accident happen? Perhaps while you took a telecommuting coffee break your cat walked across your keyboard and pressed [Shift][Delete] with its paws while the file displayed in Windows Explorer. Hey, it happened to me. Suspend for a moment your knowledge that Word creates recovery copies of files all over the system. This example applies to files where auto recovery is not available.

Fortunately, even though you're behind on your data backups, you recently made a FAT backup that includes the cluster chain for HP Blade PC.DOC.

Figure E shows a partial listing of the original cluster chain for the file before it was deleted. In WinHex, you can obtain this list through WinHex's Directory Browser. Next to the Access drop-down menu is an a file folder icon and a check box. Check that box to open WinHex's Directory Browser. Navigate to the file. Double-click the name and a window opens, showing the cluster chain. Note the two skips in sequence, from 4873 to 695 and from 710 to 13472.

Figure E
HP Blade PC.DOC is a fragmented file.

WinHex assembles the chain just like the OS does, based on the directory entry and FAT data.

Delete the file and rescan the cluster chains. When you navigate back to the file, WinHex shows its name next to a faded-out folder icon to indicate it is deleted. Double-clicking the file name now brings up a block of continuous clusters, from 4810 to 4896, equal to the length of the file (in fact, the FAT entry for cluster 4896 actually belongs to another file). The jumps in the cluster chain of the fragmented file are no longer documented because the FAT entries have been replaced with free space flags. This is promising, indicating that the system may not yet have overwritten all the clusters with new data. In short, all or part of the data is recoverable.

This is where most file recovery programs fail. Other programs with data search features can recover data based on keywords you enter, but this can be laborious. But with a spare FAT copy, you can succeed where recovery programs fail. You'll need to overwrite the current FAT table with the backup.

First make a backup copy of the current FAT 1 (you'll need to restore it later). Call it currentfat. Then open the copy of the previously backed up FAT in WinHex. Select All [Ctrl]A. Copy the block to the clipboard [Ctrl]C.

Place the cursor in the affected drive's window at the start of FAT 1. Choose Edit | Clipboard Data | Write. WinHex alerts you that the copied data will be written to offset 4000. Click OK. The entire FAT 1 should now be displayed in blue. Select File | Save and follow the prompts.

One more step should resurrect the file. Using the Directory Browser, highlight the entry for the deleted file. Right-click it and select Go To Directory Entry. The WinHex display moves to the directory entry data for that file.

At the cursor position you'll see the Dos 8.3 version of the file name, with the first letter of the name replaced by the hex entry E5. (E5 flags the directory entry of a deleted file.) In the text entry area, replace the displayed character with the first letter of the filename (here, H). A few lines above, you'll see that the first position of the Long File Name entry has also been replaced with E5. In the Hex display area, replace this with 01, as shown in Figure F. (In the figure, 01 indicates a long filename entry that continues on another line.) Save your changes.

Figure F
Replace these two entries in the directory data to undelete the file in the OS's eyes.

Now you need to rescan the cluster chain. WinHex will give a warning. Click Cancel to dismiss it. The file appears in the Directory Browser as undeleted. Double-click and the original cluster chain will pop up. Now, right-click the file name and select Recover/Copy. All the original chains will be copied. Click OK to the Use original File Allocation Table If Possible prompt, then copy the data to another partition or drive. The file is now recovered.

Better partial recovery, too
There is another advantage to this recovery method. Even if some clusters of the file were overwritten, in one step you will pull out all the remaining data, including data that may be left in the slack space in a cluster where a newly written file ends. After retrieving the data, you may be able to clean up the file and restore it.

Now you need to restore your FAT table to its original state. Replace the directory values you changed with E5, and copy back the disk's proper FAT file. Save your changes to restore the disk as is was. If you receive an error message that the FAT tables are no longer identical, some disk activity must have occurred in the meantime. In that case, copy the FAT 2 over FAT 1.

Using this technique of modifying the disk, along with making periodic copies of a FAT (even at times between data backups), you'll be better able to recover files the next time you have a disk emergency.