Using Rsync
Last Updated: Wed, Aug 5, 2009What is rsync?
Rysnc helps you transfer data from one location to another in an efficient manner. It is one of those tools that you learn to use and wonder how you lived without it. Rsync is the de facto standard in backup solutions because of its flexibility and power.
Rsync checks each file and transfers only what has changed. What does this mean exactly? Rsync will actually look and see what in the file has changed and upload only the part of the file that has changed. Unlike ftp and other transfer solutions rsync doesn’t simply re-upload the entire file.
The difference in the files are then compressed (an optionally encrypted through ssh) then sent so the transfer uses the minimal amount of bandwidth. Rsync is often used by Amazon S3 users as they must pay for bandwidth. When you are paying bandwidth bit by bit you can’t afford anything but rsync.
Rsync is Efficient but What Else Can it Do?
As you can probably tell by the name rsync is really good at syncing files across a network. If a file has changed it can detect the change and transfer only that change. This makes rsync a perfect candidate for doing incremental backups or mirroring a website.
For example to mirror one folder to another you could do:
rsync –av /path/to/source /home/nixtutor/rsync/daily
You can also use the same technique to sync from one computer to another:
rsync –av /path/to/source [email protected]:/home/nixtutor/rsync/daily
By default this will only upload new files and changes but not delete or remove files that no longer exist. To do this you can add the –delete flag. This is rsync’s way of protecting yourself from mirroring a blank directory.
rsync –av --delete /path/to/source [email protected]:/home/nixtutor/rsync/daily
Want to Sync Specific Files?
In this example we only sync .iso files.
rsync -zrv --include="*.iso" host:/home/nixtutor /home/
Things to Keep in Mind
Rsync is powerful but unforgiving
Rsync follows the unix methodology, do one thing and do it well. Thus it doesn’t provide encryption only efficient file transferring. Run rsync through SSH if you need encryption.
Rsync will not delete files that have been removed unless you supply the –delete flag.
Windows doesn’t keep file modification times to better than two seconds. Use the –modify-window=2 option to get around this when syncing to Windows file shares.
Most backup senarios can be done with rsync and cron. For an example on how to create a daily weekly backup scheme check out, Use Rsync for Daily, Weekly and Full Monthly Backups.