Ever since I first saw an advertisement for the Apple II personal computer, I have had a strong fascination for the disk drive. I must have taken apart dozens of the little monsters in my quest to fully understand their functionality.
In this Daily Drill Down, I’ll disclose everything I learned along the way. In addition, I’ll unlock some of the secrets of the Master Boot Record (MBR), how BIOS boots from a disk, how disks are partitioned, and how the operating system formats the disk. I’ll then compare the disk-manipulation tools of the various operating systems: Linux, FreeBSD, and Windows.
Take a look at a floppy
Let’s begin our journey by thinking about a single-sided floppy drive. Such a drive has only a few basic parts. A servomotor spins the flexible disk media at a precisely controlled speed. The speed must be accurately controlled or the drive will never be able to find the data it previously stored. Magnetic read/write heads float above the magnetic media surface on air currents produced by the spinning disk. If the heads contact the media, the friction normally carves deep concentric rings in the media. This is called a head crash, and it’s usually catastrophic. The drive also has a positioning motor, which moves the heads back and forth above the disk.
At this point, we know just about everything about the physical hardware. Next, we need to understand how the disk drive finds the location of each data item. Those of you who can remember back to the 5-1/4-inch floppy might recall that those floppies had a tiny hole near the center spindle hole. If you manually spun the disk, you could see that there was a hole in the media as well. The floppy drive would shine a Light Emitting Diode (LED) through the hole and would sense the light pulse whenever the disk was in a specific position. By counting time from the detection of the disk hole, the disk drive could determine the position of any data bit on the floppy. This technique was known as hard sectoring since it depended on a physical feature of the disk.
Solving the hard-sectoring issue
The hard-sectoring technique was somewhat problematic because different disk drives would have to count time exactly the same way. The variations in timing crystals and other components made this a bad approach. Also, the longer the drive had to count time, the greater its chances were for being wrong! So different areas on the disk might be less reliable than other regions. I remember that disks made on one TRS-80 machine often wouldn't work on other identical TRS-80 machines in my high school computer lab.
A better technique, called soft sectoring, helps to eliminate this problem. Instead of relying on any hardware-based measurement, the disk drive itself writes a data pattern onto the disk. The pattern is generated based on crystal-controlled timing, so it should be nearly exact. Once this pattern is written over every track of the disk, the disk drive can watch the pattern go past the heads and easily determine where the heads are in relation to the stored data. Essentially, this data pattern acts like street signs or an on-the-fly road map. We often refer to this data pattern as low-level formatting. In the large disk drives of today, this pattern is generated at the factory and stored (permanently, we hope) on your drive.
A spot of trivia
Before we get into the more relevant discussion, I want to stop and relate some interesting engineering trivia I learned during one of my internships at IBM in Charlotte, NC. I was told that the very first disk drives IBM built had severe problems with humidity. It seems that the floppy disk media would expand and contract with the humidity. This would effectively move the data around to places the drive didn't expect it to be. Worse yet, it was a non-uniform deformation problem that would have been very complex to solve mathematically. Guess how they solved it? They mounted the read/write heads in the same media as the disks were made of! Through this solution, the heads would be migrated off their correct position by exactly the same deformation that the disk experienced. I don't know if that story is really true, but it was definitely clever and inspirational.
Tracks, sectors, cylinders, and heads... oh my!
Anyone who has ever installed a hard drive has no doubt seen the terms tracks, sectors, cylinders, and heads. Tracks are just the concentric rings on a disk surface. A track is divided into sectors, each of which is usually 512 bytes. A hard drive has a number of heads that all move in parallel. Thus, if the first head is on track 0, then every other head is also on track 0. For this reason, we normally discuss cylinders as being the collection of all tracks spanned by the heads at the same time. In other words, cylinder 0 would be track 0, head 0 plus track 0, head 1 plus track 0, head 2 plus.... You can imagine the disk drive as a bunch of disks stacked vertically with heads usually on both sides of each disk.
The Master Boot Record
Hard drives come in many geometric variations. The computer needs some way to know what kind of disk is being used. It gets the basic drive geometry from the BIOS, but that doesn't indicate what's on the disk. Thus, a standard for booting disks and disk drives is necessary. The most sensible location is, of course, cylinder 0, head 0, sector 1. No matter what disk geometry exists, the BIOS can always count on finding the first sector of the first cylinder with the lowest numbered head. This location has the distinguished name of Master Boot Record (MBR).
In the MBR, we find a tiny bootstrap program and the partition table. The entire MBR is first copied into memory, and then the processor begins executing the bootstrap program. The bootstrap program reads the partition table from memory and determines which partition is the active one. All the information needed to find code in the active partition exists within the partition table. The BIOS contains the code needed to fetch the data from the disk. So once the bootstrap code extracts the partition information from the partition table, the bootstrap program will load the operating system kernel from the disk and then transfer control to it. You can find a complete disassembly listing of the normal Microsoft MBR here.
BIOS, bootstrapping, and loaders
It’s important to understand that the bootstrap code relies heavily on the BIOS to handle the basic disk I/O functions. If you study the code described in the link above, you’ll see the INT instruction called in several places. This generates a software interrupt to a BIOS routine. This interrupt is necessary because the MBR is only 512 bytes long and couldn't possibly contain the required code to perform all the needed functions. Therefore, the boot code relies on the BIOS to enable it to read the disk. This is why BIOS limitations can impact your ability to boot partitions outside the geometries that the BIOS can understand.
Many people first experience this problem when they buy a big hard drive and then try to attach it to an older system. The operating system can handle the disk just fine, but the boot code can't get it started! This is why special software like EZ-Drive from Western Digital and MaxBlast from Maxtor was written. These programs circumvent the BIOS limitations for installing DOS and Windows, but they are a major source of confusion to people hoping to install Linux or FreeBSD into a dual- or multiple-boot system. If you’re considering doing this, I’d strongly recommend installing a second disk drive instead of trying to use this software along with Linux or FreeBSD on the same drive.
Systems that boot multiple operating systems have additional loaders or other programs that run before a kernel is loaded. For example, Linux boots a program called LILO (the Linux Loader), which allows the user to choose any bootable partition from the partition table.
FreeBSD has a multistage boot scheme as well. The MBR contains a program called boot0, which presents the user with a list of the operating system choices. It obtains that list from the partition table, as expected. If the FreeBSD option is chosen, the boot1 program is then loaded, which reads the FreeBSD disklabel since BSD operating systems have this extra layer of complexity. The disklabel gives FreeBSD the ability to create partitions within a physical partition. Thus a FreeBSD system can have a root partition, a swap partition, a /var partition, and a /usr partition, all contained within a single DOS slice or partition. This is a big bonus when you consider that you can have only four partitions declared in the MBR partition table!
Linux uses up one of these partitions for each swap partition declared. FreeBSD also has a very elegant loader program, which allows the operator to list all the bootable devices on the system, obtain listings of most any disk on the system, and then boot the desired kernel. This is especially helpful for kernel developers who might accidentally install a defective kernel. With this utility, they can just find the previously saved kernel and boot it instead of the new defective one.
Unfortunately, partition means different things in different contexts. Often, people use slice and partition interchangeably, so it’s often hard to know what someone is describing. In the DOS/Windows world, the only partitions we ever see are those physically defined by the partition table in the MBR. These are contiguous regions of a hard disk. They may span any number of heads, tracks, and cylinders, and they must always contain an integral number of sectors.
In the BSD world, those physical partitions that DOS knows about are instead called slices, which makes a lot of sense. You physically slice a disk into as many as four primary partitions using the DOS/Windows Fdisk.exe program. You can also create extended partitions, which allow you to slice the disk into more subsections. Under BSD, you can create partitions using the disklabel editor. Partitions exist within the various slices. A partition must be entirely contained within a slice, except possibly when configuring a RAID (Redundant Array of Inexpensive/Independent Disks). (I’m not sure what RAID stands for anymore, but I’m hoping the world will agree to use Independent instead of Inexpensive, since inexpensive is such a subjective term.) What I want to point out here is that with BSD you can arrange to have a number of partitions created within a single slice of a disk. Any operating system that can interpret the disklabel can then access the partition data and the filesystems stored within the various partitions.
Tools of the disk trade
Now that we understand the inner workings of the disk drive, let’s look at the software tools we use to manipulate the disks.
Let’s start with the DOS disk utilities. The DOS Fdisk.exe program pretty much lets the user examine the disk slices and modify them. Basically, this utility allows the operator to read, modify, and write the data in the MBR's partition table. Very few people realize that this utility can also reinstall the MBR by calling it with the command
FDISK, when called with this switch, shows no obvious signs that it has done anything. It immediately returns to the command prompt as if nothing has happened. You’d think that there would be a message stating whether the operation was successful or not. Regardless, FDISK is useful if the MBR has been corrupted. I found a Web site that explains FDISK and claims that there are a number of additional command-line switches.
The DOS/Windows program Format.exe is used to write filesystem information into the slices created by the FORMAT utility. FORMAT writes a boot sector, followed by the file allocation tables, a root directory, and a data area. If an operating system is installed into a partition (slice), its boot code is stored in the boot sector. This code needs to be smart enough to find, load, and execute the Command.com file in DOS/Windows. The file allocation tables keep track of files that are stored in the filesystem, which are all stored within the data area.
Linux also has several programs available to partition a disk. Linux originally came with an FDISK application that performed the basic partition-manipulation functions. This program supports the basic functions of displaying partition information, adding a partition, deleting a partition, and changing the partition type. It’s menu driven, so you can get hints from it rather than having to rely on a manual or a help file. Also, the really tricky commands are confined to an expert menu. From the main menu, the typical commands that I use are:
- · m—to display the list of commands.
- · p—to print the partition table to the screen.
- · d—to delete a partition.
- · n—to add a new partition.
- · l—to list the various partition type IDs.
- · t—to change the partition type.
- · w—to write the modifications to the disk.
- · q—to exit the program.
A complete briefing on the commands and available partition types can be found at http://wwwinfo.cern.ch/pdp/ose/linux/fdisk/.
Linux has two newer disk-partitioning utilities named cfdisk and sfdisk. I haven’t played with either, but cfdisk is supposed to be a very nice cursor-based (terminal screen-handling and optimization) alternative to the traditional FDISK. From what I’ve read, this is the ideal choice for people who want typical configurations. It does very good error checking and protects the operator from doing harmful things. The sfdisk program is recommended for hackers and people who want to do things directly from scripts. For more information, try these links to online man pages for cfdisk and sfdisk.
Uses for a small DOS/Windows partition
On every Linux/FreeBSD system I’ve set up, I’ve always insisted on having at least a small DOS/Windows partition on the primary IDE master drive. This is just in case the system will ever need to communicate with the DOS/Windows world on its terms. Although this is becoming less important, there are still some Web sites that don't interface well with Linux, FreeBSD, or UNIX in general. Therefore, it’s a good idea to maintain some degree of compatibility. If you want to use the machine for games, you simply must support Windows.
I always put Windows on the disk first. I use the traditional DOS/Windows FDISK to create a DOS/FAT32 partition and then use the traditional Format.exe to create the filesystem. Then I install Win95 or Win98. Next, I throw in the Linux install disk. Each variety of Linux has a slightly different installation process, but they all end up relying on FDISK at some point. I always use the p command to verify that the Windows partition is there and that FDISK knows about it. Then I use the n command to add three partitions: a low-hanging root partition, a swap partition, and a main filesystem partition.
The low-hanging root partition must be visible to the BIOS. The Linux Loader (LILO) and the FreeBSD boot loader both depend on BIOS disk I/O utilities to load the kernel into memory. Once the kernel is in memory and running, they no longer depend on the BIOS to understand the disk geometry; however, the boot loaders typically rely on the BIOS for disk I/O since these loaders must fit in a single sector. I usually stick with this strategy if I think there’s any chance of ever connecting the disk to a machine that might have an older BIOS. If you’re certain that the disk will be used only on a machine that has a modern BIOS, then you can create just a Linux filesystem partition and a Linux swap partition. The Linux operating system needs these other partitions.
Next, each filesystem must be formatted. Linux uses a utility called mkfs to build the filesystem information into a disk partition. This is not needed for swap partitions, only for partitions that will contain a filesystem. If you use either the Slackware or the Red Hat Linux install procedures, this step will be performed for you automatically. The man page for mkfs, as well as for these procedures, can be found at linux.com.
FreeBSD and all BSD systems do things slightly differently. The process still begins with FDISK, which displays and edits the disk slice information. BSD systems then require that a disklabel be written for each slice as well. This label identifies the partition information that the operating system will use to access data on the disk. The easiest way to manipulate the disklabel is from the FreeBSD sysinstall program. A good description of this editor is available here.
There is also a disklabel command, which uses your favorite editor (declared by the environment variable EDITOR) to edit the disklabel information in each slice. This command can also use predefined configurations that are stored in /etc/disktab. It has the ability to restore the boot code to the MBR and to individual BSD slices (or any slices for that matter, assuming you had the proper boot code for them). FreeBSD also has a command-line utility called newfs, which installs the filesystem within a BSD partition. Most people prefer to use the sysinstall program to perform both the disklabeling and the formatting in this easy-to-use graphical utility.
Disk drive disclosures
At the beginning of this Daily Drill Down, I promised to disclose everything I know about the disk drive. Here’s a fact that might interest you. I was told that the huge revolution in disk drive storage capacity largely resulted from a single very clever use of electromagnetic principles.
Originally, the read/write heads were basically U-shaped pieces of metal with an electromagnetic coil in the center. The magnetic field extending out one end of the head returns through the other end. Now, if one side was made larger and the other side made smaller, the same field strength would be concentrated into a much smaller area over the small end and diffused over a very large area on the wide end. This lets the head magnetize a very tiny area of the media over one end of the head, while the other end distributes the magnetic field over such a wide area that it doesn't affect the media at all. Thus, significant increases in media density were made possible by this improvement in head design! I doubt that this was the only factor that worked to improve drive density, but it always impressed me. If anybody has any other clever design stories, please send them my way.
In this Daily Drill Down, I’ve explained how the disk drive works and which tools are needed to configure drives to your needs. We’ve discussed the MBR, the boot code, and the slices (partitions). We’ve learned that each slice has its own additional boot block and that in BSD systems, each slice has a disklabel that can identify partitions within the slice.
We also examined how the BIOS limitations require kernel partitions to exist within the reach of the BIOS disk I/O utilities. And we’ve seen how the BIOS limitations forced hard disk vendors to produce software solutions to get around these BIOS issues.
Ed Gold grew up in Louisville, Kentucky, and he received his master’s degree in electrical engineering at the University of Louisville. Ed owns a small engineering consulting firm in Orlando, Florida, and he is working on the electro-optic subsystem for Lockheed Martin's Joint Strike Fighter proposal. Although his primary computing interests are in image processing and artificial intelligence, Ed is a dedicated FreeBSD/Linux enthusiast. He is currently working to improve the FreeBSD system install utility.The authors and editors have taken care in preparation of the content contained herein, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for any damages. Always have a verified backup before making any changes.