As
industry trends have shown, Linux
is taking server market share from both Microsoft Windows and especially
proprietary UNIX systems to become the number three server operating system.
Linux is rapidly becoming the major contender Microsoft must face in the
low-end server market. One of the reasons for this is security concerns, and
another is cost. Whether you are implementing automated backup and security
policies for anything from a single workstation to a network of servers and
workstations, the rsync utility can
address both contingencies, especially when properly used in conjunction with
other tools to enhance security. This article focuses on how admins can use rsync to secure
their servers.

The rsync utility is, by
design, a simple file copying tool. It has an advantage over utilities like cp and rcp, however, in that it uses a
checksum comparison to determine what actually needs to be copied and what is
already at the destination. For instance, if you use rsync to copy a directory
with thirty files in it from computer foo to computer bar,
cp will just start copying files, and will overwrite files with the same name
on bar as the files being copied from
foo. On the
other hand, after the first time you use rsync to copy files from foo to bar, rsync will do a quick comparison of
the same directory on foo
and bar to see what is different, and
will only copy the files that differ between the two. This can greatly reduce
the amount of time spent on file system replication, such as when backing up
data on one computer to a hard drive on another computer.

The
rsync utility is very straightforward and simple to use, but it must be used
wisely to ensure proper security. It can also be used as a security tool in and
of itself, without necessarily using it for anything else, to do file system
integrity auditing.

Secure file copying

Data
security requires backups of your critical data. While this can be accomplished
the way many mom-and-pop businesses still do, by going to each workstation in
the office and burning important files to CD-R, it is better in a networked
office to have some kind of automated backup system running. The most efficient
way to do this these days with the low cost and high speed of hard drives is
often with a backup server, where files are copied over the network to hard
drives attached to the backup server on a regular schedule. A computer old
enough to be considered obsolete for many office tasks can serve admirably in
such a simple role, especially if backups are performed in the middle of the
night or on weekends when nobody is using the network.

Copying
files between computers over the network with rsync is a simple procedure. At
its simplest, you can enter the following command from a Linux system that
contains the files you want to copy:

rsync localdir host:remotedir

This
invokes the rsync command with only default behaviors. It specifies the
directory on the local system (localdir) from which you’ll be copying, and specifies the
hostname of the remote system (host) and the directory on that machine (remotedir) to
which you’ll be copying files. For instance, if you copy the directory /home/baz from
host foo to
directory /home/backup/qux on host bar, your command on host foo looks like
this:

rsync /home/baz bar:/home/backup/qux

The
operation ends up being performed entirely “in the clear”, however,
without much in the way of security, if that is all you do. It also doesn’t
copy symbolic links, or preserve permissions, user and group ownership, or
creation times. For these, you’d add the -a option (for “archive”)
and the -e option to specify a remote shell
program to use for security. That remote shell should use an encrypted protocol
such as SSH for security purposes so that the remote shell option takes the
form -e ssh. Your command, tunneling rsync through
ssh, looks
like this:

rsync –a –e ssh localdir host:remotedir

If you
want to get a better look at what is going on when you copy files across, you
can use the -v and –progress options. The -v option specifies verbose output. The
default behavior of rsync is to run silently, simply performing its task
without bothering the user. Used once, the -v option tells you what files are being copied and gives you a summary at
the end. Used twice, by adding -vv to the command, causes rsync to
also tell you what files are being excluded and gives you more information in
the final summary.

The –progress option, meanwhile, will give you an ongoing indicator of
progress including the current file size, the percentage of the total transfer
that has been completed, the current speed of transfer, and the estimated time
remaining before the transfer is done. A command using double-verbosity with
archive mode and ssh tunneling, and requiring a
progress indicator, takes this form:

rsync –avv –e ssh –progress localdir host:remotedir

If a
transfer is interrupted in the middle of copying a file, rsync’s
normal behavior is to delete the partially copied file from the destination. You
can use the –partial option to prevent rsync from
deleting partially-copied files if you know in advance that you will be
attempting to copy the file again in the event of an interrupted transfer. This
speeds up the transfer on the second attempt, as rsync only copies data not
already copied. If you are using both –partial and –progress, rsync provides a shorthand option that can be used for
both; the -P option. Thus, to do a
double-verbose archive copy with deletion of partial files turned off,
requiring a progress indicator, and tunneling through ssh,
your command takes the following form:

rsync –avvP –e ssh localdir host:remotedir

To
automate the process according to a schedule, you can simply add the rsync
command to a cron job. You can accomplish this by
adding an entry to the /etc/crontab file, like so:

30 4 * * * user rsync –aP -e ssh localdir host:remotedir

The
first five fields in a cron job entry define when the
job is scheduled, and the sixth tells crontab what
user will run the job. Replace the “user” string in the example with
the name of the user account that will be running the job. The syntax of this cron job will cause it to be run once per day, at 4:30 in
the morning.

The
order of the first five fields is minutes, hours, day of the month, month of
the year, and day of the week. The time fields start at 0, so that minutes are
0-59, and hours are 0-23. The next two fields, for days of the month and months
of the year, are 1-31 and 1-12. The last field, day of the week, is 0-7, where
either 0 or 7 indicate Sunday. The configuration of cron
jobs can become much more complex than this, but this is the basics of creating
a cron job to schedule data backups with rsync. For
more information about how to create cron jobs, start
with the commands man cron, man crontab, and man 5 crontab.

If you
want to centrally manage your data backup cron jobs,
you can create crontab entries on the backup server
like this:

30 4 * * * user rsync –aP –e ssh host:remotedir localdir

Set up
that way, rsync will grab files from the remote directory (remotedir) on the specified
remote host and copy them to the local directory you indicate (localdir).

Integrity auditing

Normally,
filesystem
integrity auditing is an operation performed by way of dedicated security
tools like tripwire. Such tools tend to be very thorough and provide a
lot of power and flexibility in how the sysadmin goes
about performing filesystem integrity audits. There may be any number of
reasons for a sysadmin to prefer a different means of
filesystem integrity auditing, however.

Perhaps
surprisingly, the rsync utility can be used to do integrity audits. Its
checksum comparisons between primary and archived copies of files and
filesystems can be used to determine what changes have been made to a
filesystem, and those changes can then be examined to determine whether or not
they were authorized changes.

The
first thing you need to do in the process of using rsync for filesystem
integrity auditing is to create a snapshot of your filesystem. This must be
accomplished before there is need to check your filesystem’s
integrity, because you must have a “known good” copy against which to
check the filesystem.

To do
this, simply run an rsync copy command from the command line. To be smart about
it, though—and to ensure that you are using a version of rsync that cannot have
itself been compromised by an intruder, so that it won’t detect or report
changes to the filesystem—you should copy the rsync binary from the computer
(probably at /usr/bin/rsync) to a non-rewritable medium, such
as a CD-R. Then, when you specify the rsync utility to use, you should indicate
the rsync binary on the mounted CD-R instead of the one on your computer. For
instance, running this command copies files from one computer to another, if
your CD is mounted at /mnt/cdrom:

rsync –av –rsync-path=/mnt/cdrom/rsync –e ssh host:/ localdir

The
above copies the entire filesystem from a remote host (the computer whose
filesystem integrity you want to check) to a local directory on the machine you
are using to run rsync. Once you have that snapshot of your filesystem, you can
check filesystem integrity using the following command:

rsync –avn –rsync-path=/mnt/cdrom/rsync –e ssh host:/ localdir

The -n
option in the above command ensures that nothing is copied. Instead, rsync
merely reports the differences between the working copy and the archive copy. If
unexpected differences are shown, you should examine them to ensure that they
are not unauthorized changes. If there are changes, a list of changed files
will be displayed.

Changes
to utilities such as /bin/ls, /bin/bash, and /bin/sshd are highly suspect, and if no such
changes should have been made, you should consider your system compromised. If
you have an automated system update running, however, those files may have been
legitimately changed by that process. As such, it is a good idea to have
automated system updates log any updated packages so you can compare those
changes with the list of changes your rsync integrity audit reports.

Bringing
it together

Your
automated backups can be combined with your integrity auditing. For instance,
you may use automated backups to back up the entire system, and create a
similar cron job that occurs before the backup to run
an integrity audit, with a redirect to write its output to a logfile. Under such circumstances, it is usually a good
idea to have your audit only check parts of the filesystem that should not
change, while having your backups target the entire filesystem.

For
instance, your backups might target the root directory while your integrity
audits omit directories whose contents regularly change such as /home and /var. To be too zealous in deciding what
will be checked by an automated filesystem audit is likely to result in the
automation of backups becoming entirely ineffective.

If you
really want to get fancy, you can even write a shell script that is run by your
cron job, wherein it does integrity auditing first,
then runs the backup only if there are no problems with the integrity audit. Then,
when you check the log of the integrity audit for yourself later, if there’s
nothing wrong you can run the backup at your leisure to get up to date
(preferably after another integrity audit, to ensure nothing has changed since
the automated audit). Clearly, shell scripting is beyond the scope of this
article, but the ability to write shell or Perl scripts is a handy skill for a
Linux sysadmin to have.

Automated
backups are reasonably simple with rsync, which is to be expected since backups
are what rsync was designed to do. However, the way rsync can be used to
perform other tasks as well, such as filesystem integrity auditing, illustrates
the flexibility Linux provides by the development of many small utilities that
each do one thing well, as opposed to large programs that each try to do
everything. Such small utilities can then be combined to perform more complex
tasks, as demonstrated by the ability to tunnel rsync through SSH to encrypt
and compress data transfers over the network, to greatly enhance data and
system security simply and easily by use of simple command line utilities. In
the hands of a knowledgeable system administrator, rsync becomes a versatile
and effective security tool.