Storage

How do I configure RSync on FreeNAS to sync shares with a Windows NAS device?

Donovan Colbert illustrates how to create a suitable, inexpensive data replication solution for a small office or data center on a limited budget.

My home backup solution is a little elaborate. I've built a FreeNAS server with two 500GB drives in a JBOD configuration for 1TB of storage. That data is then replicated to a Windows Device-based NAS with four 250GB drives in a Raid 5 configuration. From there, the most essential folders and data are backed up to a Powervault 120t 4000 DLT IV tape backup library.

The aim of this document will be to illustrate how to recreate this configuration -- which is a suitable, inexpensive alternative to more expensive data replication solution for a small office or data center on a limited budget -- or overkill for the propeller head who can't be away from the hum of a data center without going into withdrawal.

This blog post is also available in PDF format in a TechRepublic download and as an illustrative Photo Gallery.

The setup

FreeNAS is a mature, robust open-source NAS application built on FreeBSD. Your configuration may vary, but I found a system that was small and elegant, had enough internal space for a couple of disk drives, and, most importantly, could boot from a USB device. You can also configure FreeNAS to boot from a portion of the internal hard drive or from a floppy or CD, but the first takes away a portion of storage space available to the NAS, and the second two examples are clunky, slow, and potentially unreliable.

By having the device boot off a USB drive, it maintains the impression that this is a dedicated NAS device and not a PC repurposed as storage. I run this box "headless," although it has a keyboard, mouse, and monitor ports. There is a Web-based interface that allows you to access and manage the box via a Web browser, like most commercial NAS devices. I'm using the FreeNAS software RAID, but if your box has a hardware RAID, you would most likely use that instead.

For the Windows NAS device, I purchased a used Dell Powervault NAS device from eBay. For many users, a Windows Home Server solution is effectively the same thing. A quick search on eBay shows that similar Dell Powervault NAS devices are going for $199, "Buy It Now." Likewise the Dell Powervault 120t is available on eBay and generally goes for around $20-$80.

A few words in particular about DLT tape devices -- these devices have become very affordable, but media can still be a little pricey. I would recommend buying a device bundled with media. Additionally, there are several different models of the Dell Powervault 120t. The native/compressed capacity and speed of the device varies by model. The most common models are the DLT 4000, DLT 7000, and DLT 1.

In the list below, the first number is the native capacity, and the second is the compressed capacity. Most people, in most situations, will not need much more than the native capacity for storage and are likely to experience less than the native capacity for speed.

  • DLT 4000 has a 20/40GB capacity per tape and is rated to transfer at between 1.5mb/second and 3.0mb/second.
  • DLT 7000 has a 35/70GB capacity per tape and is rated at between 5mb/sec and 10mb/sec.
  • DLT1 has a 40/80GB capacity per tape and is rated between 3mb/sec and 6mb/sec.

If you have more money to budget, there are higher capacity and speed tape drives. These drives offer a good balance between features, speed, and affordability for a very tight budget. Anything with noticeably more capacity still has a more significant value in your average production data center, and so you'll have to pay more.

There is also no reason for you to configure your solution exactly as I have here. You can take ideas from this setup where it makes sense and customize it to your particular needs. In particular, I put this solution together in stages. If I had done it all at once, I probably would have bought two like-configuration Powervault NAS devices -- or built two FreeNAS devices that were configured exactly the same. I am describing my solution exactly as it is configured only to give you a framework for how you might implement a similar solution.

My solution

For my home solution, FreeNAS has a couple of advantages, other than it being free. It also plays fairly well with SMB/CIFS "SAMBA" Windows Networks. It does not, unfortunately, support Windows security models for permissions very well. This was not a concern for my home network, but it may be a significant concern on a company network.

In general, if you can get onto my home network, you've got complete and unrestricted access to all files on my FreeNAS server. I like it this way, because my home network is a workgroup. All my machines map persistent shares to this FreeNAS server, which we'll refer to from here out as \\Freenas. The shared folder on \\Freenas is \homeshare. My Macintosh, my Ubuntu servers, even my Android phone can all easily map shares to \\Freenas\homeshare and all files and folders under that share. My wife and kid both have a directory on this device.

All data on \\Freenas\homeshare has been replicated to the second NAS, which we'll call \\Powervault. \\Powervault has a share named \homeshare as well. After the initial sync between \\Freenas\homeshare and \\Powervault\homeshare, all NEW files written to \\Freenas\homeshare are instantly replicated to \\Powervault\homeshare.

Changes to files are also replicated "instantly" (I've set Rsync to replicate every five minutes. You can decrease or increase this number according to your needs). Files that are removed on \\Freenas are not removed on \\Powervault. That is to make sure that an accidental erasure on \\Freenas is not instantly replicated to \\Powervault (thereby defeating the purpose of having a backup-to-disk solution).

I've then defined, in my backup solution, a weekly full backup of certain folders on \\Powervault to tape on my DLT 120t tape library. I rotate two sets of six tapes -- so I always have the last two weeks of critical data backed up to tape.

When I started this project, I had intended to write a very simple, high-level overview of how I have this set up. As I got into it, I realized it wouldn't be very clear unless I rebuilt the entire thing from scratch. As my environment is effectively "production" on my home LAN, I didn't want to disrupt what I already had in place. Instead, I decided to create a couple of VM machines and illustrate the configuration that way.

Somewhere along the way, I decided to include details on how to actually set up the test environment in a VM environment. In the end, I think this is the best way for most people to approach testing this solution to see if it will work in their own environment.

Using the How To document I've created, you'll be able to build yourself a Source and Target Virtual FreeNAS server and set up RSync to replicate data from the source share to the target share. With the experience you'll gain doing this, you'll be ready to move ahead and implement a solution like I describe above if you decide FreeNAS will work for you.

I used Sun VirtualBox as my Virtual Machine foundation. The following documents assume that you will be using Sun VirtualBox as well, but the process can easily be modified to use any other VM solution (VMWare, Citrix, or others) that you may prefer.

I've also decided to illustrate an Rsync from a virtualized FreeNAS machine to another virtualized FreeNAS machine. The lessons learned here will apply to using Rsync to perform this kind of data replication to any target server, including a Windows-based NAS. For anyone interested in Syncing from FreeNAS to a Windows-based server or appliance, "DeltaCopy" is the Rysnc Server/Client application that I use on my production replication environment.

Keep in mind, with this virtualized environment, I have made some assumptions in my documentation. I have illustrated the installation process of FreeNAS onto a VM for a single machine only. You'll need to do this twice, making the necessary changes (machine name and IP address) as appropriate on your network or test environment.

I've also set a very small, nonredundant drive on the FreeNAS machines. In production, obviously you would want to increase this size according to the needs of your organization. I suppose that there could be additional advantages of running both the source and target FreeNAS servers as actual virtual machines in a production environment. Those questions are outside the scope of this article, and so I have not addressed issues of what kind of hardware environment is ultimately suitable for a production example of the concept illustrated in this document.

Finally, I haven't described, in my test environment, how I back up from the B2D to tape. I'm using Backup Exec 10D running on my Windows NAS appliance to do the backup to tape. Depending on your needs, the important thing to know is that once you replicate from the Source NAS to a target NAS, you can then back up to tape from that target NAS without having an impact on your production network or NAS.

Sun VM setup

Figure A

Open Sun VirtualBox and click the New button to create a new machine (Figure A).

Figure B

The Welcome to the New Virtual Machine Wizard will run. Click Next to continue (Figure B).

Figure C

Name your virtual machine and select your operating system and version. In this case, I am creating a second FreeNAS VM and therefore selecting BSD for the Operating System and FreeBSD for the Version. Click Next when done (Figure C).

Figure D

Select the appropriate amount of memory. For my FreeNas VM, I've selected 512MB of memory. Click Next when done (Figure D).

Figure E

In Virtual Hard Disk dialog box, check Boot Hard Disk (Primary Master), select Create New Hard Disk, and click Next to continue (Figure E).

Figure F

The Welcome to the Create New Virtual Disk Wizard will run. Click Next to continue (Figure F).

Figure G

At the Hard Disk Storage Type dialog box, select Fixed-size Storage and click Next (Figure G).

Figure H

In the Virtual Disk Location and Size dialog box, set the size of the disk to 500MB either by using the slider or by typing the size in the field. Click Next to continue (Figure H).

Figure I

Verify that the information for the new virtual disk is correct and click Finish to create the disk (Figure I).

Figure J

At the Summary window, verify your settings and click Finish to create your FreeNAS Virtual Machine (Figure J).

Next Page >>

About

Donovan Colbert has over 16 years of experience in the IT Industry. He's worked in help-desk, enterprise software support, systems administration and engineering, IT management, and is a regular contributor for TechRepublic. Currently, his profession...

7 comments
spage
spage

What are you you using as the rsync server on the windows platform? DeltaCopy? I'm not sure if I saw it mentioned. Thanks and sorry if I'm being obtuse.

dcolbert
dcolbert

Late update: Recently I lost a drive on the Powervault that is the target server for this solution. The Powervault NAS is such a poor solution for a redundant storage solution, that I lost redundancy on the boot drive mirror *and* the Raid 5 stripe where the replication takes place. Once I replaced the drives and rebuilt the mirror and stripe, the source FreeNAS server found the share, and immediately began copying over all the data - which was lost, despite the RAID 5, during the rebuild. I felt vulnerable when the Powervault was down, but it was this EXTRA level of replicated redundancy that prevented me from losing any data - the solution worked exactly as planned in an "almost worst case" scenario. Good thing, too - as the backup from the Powervault to tape hasn't been done in quite awhile. :)

dcolbert
dcolbert

This was a pretty sprawling and complex howto. I've read through it, and some of the paragraphs and figures do not correspond correctly. Despite this, Mark has done an incredible job at editing the pile of documentation I sent him into a cohesive, easy to follow guide to setting up a test NAS environment. If you have any questions, problems, or concerns, drop me a line here and I'll do everything I can to get you back on track or help you through any rough spots. It would probably be best to post questions here in the forum, rather than sending me a private/direct message, so that others can also benefit from reading those comments and observations. Good luck, and I hope you find this document helpful.

Mark W. Kaelin
Mark W. Kaelin

What is your NAS setup? Do you have a better NAS solution that you'd like to share?

dcolbert
dcolbert

For expert-level discussion of the full features and possibilities of the RSync utility, you may also want to check out this article: http://blogs.techrepublic.com.com/five-tips/?p=143&tag=results;CR1 by Jack Wallen. Jack really gets into the nuts and bolts at an expert level about what RSync is capable of, and goes into far more detail than this article. Once this document introduces you to the broad concept of what RSync is capable of, if you want to find out how to really make it work well for you, I highly recommend checking out the link.

Neon Samurai
Neon Samurai

Emberassingly underpowered for what I really should be doing at home but it's better than nothing. Hardware RAID1 is a minimum after having lost single drive NAS before and only trusting software raid so far. RAID10 would be preferable though Drobo's raid expanding on the fly is also very nice. If I can get my head wrapped around Bacula, I'll build a dedicated box for that eventually. (I may have to give in and rtfm)