On rsync over NFS

Don’t rsync over NFS. The confusion should be clarified upon reading the now complete article.

I was doing a backup over an nfs mount. I copied all the files over, and then I thought it would be much smarter to use rsync. I put together a shell script to run as a cron job. When it ran, to my great surprise I discovered that rsync is actually transferring over all the files I already copied!

The problem was that, obviously, a simple copy of the file has a different modification time than the original file. The surprising thing to me was, that rsync was hauling the 170GB of data in some 30 files over a ~40Mbit link again. This is no surprise – the default modus operandi of rsync is to judge based on file’s size and modification time. Different modification time leads to an assumption on different contents, perhaps few bytes replaced here and there. But where’s the magic diff-only transfer? Quoting the man page:

It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination.

I gave rsync the -vvv flag, and it revealed that it thinks it’s running locally:

delta-transmission disabled for local transfer or --whole-file

However, using –no-whole-file or –append flags appeared not to change all that much. Perhaps it first needs to compute some checksums first. But I ran md5sum on all the files in both sets beforehand and there was no difference, so I just wanted rsync to update the dates, and behave like everything is fine. Using –itemize-changes and –dry-run I confirmed that the only changes that it sees fit are the times:

>f..T.... F20110617T215010Z.1.zip
>f..T.... F20110617T215010Z.10.zip
>f..T.... F20110617T215010Z.11.zip
>f..T.... F20110617T215010Z.12.zip
>f..T.... F20110617T215010Z.13.zip
>f..T.... F20110617T215010Z.14.zip
>f..T.... F20110617T215010Z.15.zip
>f..T.... F20110617T215010Z.16.zip
>f..T.... F20110617T215010Z.17.zip
>f..T.... F20110617T215010Z.18.zip
>f..T.... F20110617T215010Z.19.zip

(f stands for file, the capital T stands for mod time, all the dots mean no change)

Having no luck in forcing rsync to update the times only, I turned to a trusted friend, touch(1).

touch local-dir/file1 nfs-dir/file2

This was not accurate enough, since the files are residing on different servers. Then I found what I wanted. Touch (as found on linux boxes) invoked with -r:

touch -r local-dir/file1 nfs-dir/file2

copies the access and modification times from the file passed as file1 argument onto file2. Since I had around 30 such files, I typed, straight into the shell:

[root@mail local-dir]# for i in *.zip; do
> touch -r $i /nfs-dir/$i
> done

Now my backup script exits instantly when there’s no work to be done, horray!

Update:

That just solved to problem for then. On next backup, the rsync tried to update the same files over and over. Rsync is just not supposed to be run across nfs mounts. The preferred way is to run it as a server on the destination machine, and transfer the files using the remote host notation, like so:

rsync local/dir remote.nfs.server:/remote/dir

So that the delta transfer will work.

And yet another update:

My backup did not ran as supposed this time around either. One thing I failed to notice from the start, was that, on closer look at the man page, one thing becomes evident.

The ‘T‘ in the output stands for: the modification time will be set to the transfer time, which happens when a file/symlink/device is updated without –times and when a symlink is changed and the receiver can’t set its time.

The –times switch is used when -a (for archive copy, like preserving all the attributes) is choosen.

So rsync -a or rsync –times should be used, or else it’s very famous delta-transfer stands for nothing, neither over ssh, nor over nfs.

This confused me to a great extent. And the amount of people that accessed this document using google search and otherwise, just this month is 100, leading me to believe I’m not the only one misled.

Now I’m able to run it both across an NFS mount point and over ssh (I found NFS to be a bit faster), and the cron jobs takes care of making sure the remote copy stays in sync to the local one.

the modification time will be set to the transfer time, which happens when a file/symlink/device is updated without –times and when a symlink is changed and the receiver can’t set its time.