Disaster Recovery

Five tips for performing better rsync backups

Keeping a directory in sync between two Linux machines is a snap, thanks to rsync. Here are a few tricks for getting the most out of this handy tool.

Backups are crucial to a sound disaster recovery policy. With a good backup system in place, you know that should disaster strike, you can restore your systems. Of course, there are many ways to back up a system. Some are costly and some are complicated. But what if you just want to keep a single directory in sync between two Linux machines? This is simple with the help of rsync, ssh, and cron. These tips will help you achieve this task.

1: Use ssh instead or rsh for secure transmission of data

When you use rsh, any data transmitted is done so insecurely. You can opt to use ssh (secure shell) by invoking rsync with the -e option instead, making the transmission of data far more secure. The new rsync command would look like this:

rsync -r -e ssh /home/USERNAME USERNAME@SERVER_IP_ADDRESS:/home/USERNAME

where USERNAME is the name of the user you want to back up and SERVER_IP_ADDRESS is the address of the backup server.

2: Set up password-less ssh authentication

This is necessary for unattended backups. Otherwise, you would have to be present to enter the ssh password when prompted. Understand this isn't the most secure method -- but it is effective, especially when you plan to run this script from a cron job. When this is the case, no password prompt will be presented. You could also set up certificate authentication if you are concerned about security.

To set up password-less authentication, follow these steps:

  1. On the client machine, issue the command ssh-keygen -t rsa. Accept the defaults. Do not enter a password; just hit [Enter] twice when prompted.
  2. Copy the keys to the server with the command scp ~/.ssh/id_rsa.pub SERVER_IP_ADDRESS:~/.ssh/authorized_keys.
  3. Log in to the server (using ssh) with the same user you plan to back up and make sure the ~/.ssh directory has the right permissions with the command chmod 700 .ssh.
  4. On the client machine, open up the file /etc/ssh/ssh_config and look for the line ForwardAgent yes. If this line is commented out, remove the "#" and save the file.
  5. On the server machine, open up the /etc/ssh/sshd_config file and make sure you have the following lines:

    RSAAuthentication yes

    PubkeyAuthentication yes

    AuthorizedKeysFile .ssh/authorized_keys
  6. Restart sshd.
  7. On the client issue the following commands:

    ssh-agent

    ssh-add

3: Create a shell script for your backup so it can be issued from a cron job

The script really needs to contain only the following:

#! /bin/bash
rsync -r -e ssh --delete /home/USERNAME/ USERNAME@SERVER_IP_ADDRESS:/home/USERNAME

where USERNAME is the name of the user you want to back up and SERVER_IP_ADDRESS is the address of the backup server.

Just remember to give that file executable permissions with the command chmod u+x backup.sh or the script will not be executable.

4: Don't run rsync jobs during production periods

Rsync can gobble up a lot of resources. As with any backup, you will want it to be automated and to run during off hours. This is especially true if you are backing up to a commonly used server. If you start an rsync job while others are using that server, trouble could ensue. If you MUST run a backup during work hours, don't run large backups at a time. Instead, run smaller backups (one at a time) to avoid all the I/O operations that can occur when trying to sync too many files at once. For example, instead of backing up

/home/jlwallen

I could back up (in succession):

/home/jlwallen/Documents

/home/jlwallen/Pictures

/home/jlwallen/Downloads

5: Create a cron job for a backup script to execute

Let's say you want this to run every night at 4:00 AM. For this, the cronjob will look like:

00 4 * * * * /path/to/backup.sh

where /path/to is the path to the backup script. That's it. Now your backup will run every morning at 4:00 AM.

More tips?

Rsync is an outstanding tool to set up quick and easy backups for Linux and UNIX-based machines. There are tons of ways to use rsync, which is a testament to its versatility. If you have any favorite rsync backup tips, share them with your fellow TechRepublic members.


Check out Five Tips... the newsletter

Get a concise roundup of solutions and techniques that will make your IT job go more smoothly. TechRepublic's Five Tips newsletter, delivered every Tuesday, gives you instant access to the information you need. Automatically sign up today.

About

Jack Wallen is an award-winning writer for TechRepublic and Linux.com. He’s an avid promoter of open source and the voice of The Android Expert. For more news about Jack Wallen, visit his website getjackd.net.

3 comments
techrepublic@
techrepublic@

Use nice and ionice to reduce rsync's priority. This way, if something more important needs to run while the backup is in progress it will not be hindered.

bruce.edge
bruce.edge

Use rsnapshot. It wraps rsync with a rotating snapshot backup mechanism that removes all the tedium from creating your own solution.

Styopa
Styopa

If you are now panicking about the security of every rsync transfer you have ever done, you probably don't need to. Mostly, rsync is configured to use ssh by default. (Phew!)