Disaster Recovery

Drive and partition backups with dd

Vincent Danen goes over the basics of using the standard dd tool to back up and copy partitions and drives.

There are a lot of backup utilities available, but the simplest backup tool is available already with every single Linux distribution out there. The 'dd' tool simply copies standard input to standard output, read in 512-byte blocks.

With this, dd can copy a file to another file, or a partition to another partition, or file to partition (and vice versa). This makes dd quite versatile, and perfect for cloning partitions and drives.

For instance, to make an exact clone of the /boot partition to a backup file, you could use:

# dd if=/dev/sda1 of=/srv/boot.img

This copies the contents of the partition /dev/sda1 (mounted, for example, as /boot) to the output file /srv/boot.img. Note that dd copies "empty" space too, so if the partition is 200MB in size, even if it only contains 100MB of data, the output file will be 200MB in size. As an example:

# df -hT | grep sda1
/dev/sda1     ext2    198M   86M  102M  46% /boot
# dd if=/dev/sda1 of=/srv/boot.img
417627+0 records in
417627+0 records out
213825024 bytes (214 MB) copied, 2.07951 s, 103 MB/s
# du -sh /srv/boot.img
204M   /srv/boot.img

The file can be compressed after the copy, or during by piping the input to gzip or bzip2:

# dd if=/dev/sda1 | bzip2 -9f >/srv/boot2.img.bz2
417627+0 records in
417627+0 records out
213825024 bytes (214 MB) copied, 31.5072 s, 6.8 MB/s
# du -sh boot2.img.bz2
111M   boot2.img.bz2

Likewise, partitions can be restored from these backup copies:

# dd if=/srv/boot.img of=/dev/sda1

or:

# bunzip2 -dc /srv/boot2.img.bz2 | dd of=/dev/sda1

If you wanted to duplicate an existing drive to another, you would obtain a drive of the same (or larger) size. Assuming the drive to copy is /dev/sda and the destination drive is /dev/sdb, first use fdisk to recreate the appropriately-sized partitions, then use dd to do the actual cloning:

# sfdisk -d /dev/sda | sfdisk /dev/sdb
# fdisk -l /dev/sda; fdisk -l /dev/sdb

Compare the output of the two fdisk commands and make sure the partitions on /dev/sdb match those on /dev/sda. Once this is done, you can copy each partition using:

# dd if=/dev/sda of=/dev/sdb bs=446 count=1
# dd if=/dev/sda1 of=/dev/sdb1
# dd if=/dev/sda2 of=/dev/sdb2
...

The first dd call copies the MBR from the first disk to the second. This will allow the second disk to be booted, when it replaces the first. The first 446 bytes are copied with this command; that is the boot code we need.

Depending on the size of the disk and partitions, this could take some time, but the end result will be a perfectly cloned system. For best results, if you do intend to replace the one drive with another, use a LiveCD or USB boot instead of booting off of /dev/sda; this will ensure that nothing changes on-disk during the copy.

Get the PDF version of this tip here.

About

Vincent Danen works on the Red Hat Security Response Team and lives in Canada. He has been writing about and developing on Linux for over 10 years and is a veteran Mac user.

20 comments
gearond
gearond

For those who use windows and certain programs, in particular, TrueCrypt, you need to copy much more than the first 2 sectors (approx). If I remember correctly, it's about 4k or 16 sectors. I used to use dd all the time using a Ubuntu Live CD and a Bash script. Saved me several times. It's easier with Linux. All my programs are free except one (which I am abandoning due to issues - 'Jungle Disk'). So I only backup my Postgres dev database, cron script, and my $HOME directory. MUCH smaller :-) However, when I get some more external USB drives, I will go back to using dd for the whole partition.

NickNielsen
NickNielsen

We use it at work to clone flash drives on thin clients. We also use it to build the thumbdrives we use to clone those flash drives. Great tool.

jpgravel
jpgravel

When compressing an image with this method, it might be useful to take some time to reduce the partition's empty spaces entropy by creating a large file filled with zeros. Once the file has been created, delete it then make the image using dd and {gzip|bzip2|...}. The compressed image file will be much smaller. To fill the partition's empty spaces with zeros, use the following command: cat /dev/zero > zero.dat; rm zero.dat

Neon Samurai
Neon Samurai

I've been using DD for years with failing flashdrives and such. As soon as it starts to behave badly or when it's being replaced; yank a disk image off it. After that, the SD or applicable removable storage can fail without concern. I'll still have the pre-fail ISO image along with the files mirrored through my normal rsync backup.

darkstate
darkstate

I've used dd and dcfldd many times to backup a full drive thats got either ,xp or win 7 thats been fully system encrypted with Truecrypt. Both programs do a great job, and are very reliable and accurate,for either the above example or for just copying/backing up files in general.

tbmay
tbmay

The imaging tools, including symantec ghost, partimage, clonezilla, etc....have really made dd an obsolete way of collecting them. My small openbsd firewalls are embedded though and I use dd for that. When I started setting up the obsd flash images the other solutions didn't work. They may now, I haven't checked. One thing about doing via dd is it works on everything.

NickNielsen
NickNielsen

To respond to the original article, you have to scroll all the way through any responses, to the bottom of the page, and post using the 'What do you think?' area. If you are in the flat view (view all) and click 'Add a Comment', it replies to the first post. Not sure why, but that's the way the forum software here works. New workings coming soon.

Neon Samurai
Neon Samurai

It's rare that I'm without a *nix I can plug the USB into but having more options for different platforms is never a bad idea.

Neon Samurai
Neon Samurai

Video DVD can be a little hit or miss. I prefer to watch my movies off the machine and DD'ing an ISO then watching it with VLC is a nice way to keep the DVD hardcopy fresh along with providing a backup/working-copy in a file. I work by cli when I can so dd without some front end is preferable for me. I wouldn't say it's been made obsolete by the availability of front ends since Clonezilla is using DD in the back end. It does depend on what your doing though. DD a drive image or ISO; sure.. no need to go find winimage or similar. For doing a full system image, I'd go something like Clonezilla also.

Neon Samurai
Neon Samurai

Along side the task of ripping a disk image, I'll be doing other stuff. Either I just leave it until finished or I check intermittently with a second terminal and "ls -h". I've not had reason to know the progress in more detail then that and if it was such a rush job, running dd through a GUI wrapper would actually add time on rather than just getting the task done. Copy a file, move a file, dd a disk image.. progress percentages are less relevant when I can just "&" it into the background and continue on when it's finished. Now, with network transfers, I'm much more interested in progress and ETA calculations.

darkstate
darkstate

Oh man you have a very lot of time on your hands, Whats your hobby? Watching paint dry and grass growing? Only joking, You want and need to try dcfldd instead as you can see the real time progress in megs copied so far, Its commandline and uses the same commands as dd.

Neon Samurai
Neon Samurai

I just watch the file DD is moving data into since I know the size of the disk it is reading from. That's not for everyone but the GUI is more often booted to support my displaying multiple terminals on the screen.

darkstate
darkstate

It does an amazing job , even for the novice as long as you get the commands the right way around like making sure you copy dd if=/dev/sda of=/dev/sdb ,if you get them the wrong way round there's going to be tears. There's an easier solution with the front end for DD called AIR http://www.howtoforge.com/creating_dd_images_with_air . Using DD is fine without the GUI but there's no on screen info about what its doing or how far its got in the copying process, But there is something that does exactly the same thing and uses the same commands called dcfldd. Its mainly advertised as a forensic program but it does the same as dd with the same command structure dcfldd if=/dev/sda of=/dev/sdb. I've used it in the past for cloning a hard drive that was encrypted with Truecrypt and it cloned it just fine, and the good thing was with dcfldd it tells you how many meg its copied in real time so you can keep an eye on things. Also as with dd you can do a raw command like above so if there is any errors it will stop and tell you but you can also give it the command conv=sync,noerror which will let it fill the space that's corrupted with something and continue, good for drives that have minutes till death.

Neon Samurai
Neon Samurai

Manually taking images of a full system.. I agree, it's better to just use an existing front end.

tbmay
tbmay

...from the standpoint of using it for workstation images is what I was referring too. I use if for iso's too.