I did give Time Machine a try, but it just would not do it. No matter if I am using a network disk or a USB-attached dedicated hard drive, the I/O load is just not worth it. Even on my shiny new iMac. Sad.
Hey, if I wanted my CPU or I/O to be overloaded by some system task, I would be using Windows with some good heavy AV on it.
So back to the good old, reliable rsnapshot for daily, during-the-night backups.
There is catch that has been bothering me for quite some time, though, and I only found the solution very recently.
I like my Snow Leopard iMac to go to sleep after a reasonable period of time, because it is not meant to stay up 24×7 – unlike the Ubuntu backup/storage/kids game machine.
The iMac is fairly clever in that it can wake up from simple SSH requests – or from the preceding ARP requests.
Here is the issue: I want rsnapshot to wake up the iMac before the backup – and obviously let it go back to sleep afterwards.
Scripting the ability to wake up the iMac is not a big deal. This simple script would do:
#!/bin/bash
let "i=0"
sshup=1
while [ $i -lt 20 -a $sshup -ne 0 ]
do
let "i += 1"
sleep 2
ssh $IMAC echo
sshup=$?
done
exit $sshup
Their needs to be some ssh trust relationship in place between the backup server and the iMac; enough articles have been written on that topic, I will not contribute to the noise.
Inserting this script (let’s call it /usr/local/sbin/ssh_wake) in the rsnapshot workflow is not a big deal, thanks to the cmd_preexec configuration item in rsnapshot.conf:
cmd_preexec /usr/local/sbin/ssh_wake
(do mind that like usually there is a TAB between cmd_preexec and /usr/local/sbin/ssh_wake)
However, this design comes with an issue: in its default sequence,
- rsnapshot starts by launching cmd_preexec (hereby waking up the iMac),
- then rotates the backup directories (daily.n becomes daily.n+1),
- then rsyncs the backup.
On fairly large backups, there is going to be a significant delay between the « let’s wake up the iMac » and the « let’s perform the rsync data transfer ». To a point where the iMac would go back to sleep before the actual data transfer.
Quite a few weeks banging my head against that one.
Until I started reading the rsnapshot source code (it is only Perl after all ™), and noticing this sync_first configuration item that changes significantly the usual workflow.
sync_first was nowhere to be found in my rsnapshot.conf, because my configuration file dates back to a version of rsnapshot that did not support sync_first.
Makes you feel like a dinosaur. Just a bit.
A bit of RTFM later, here is how sync_first works:
- sync_first decouples data transfer and directory rotation, which by itself warrants my interest
- If there is no .sync folder in the backup directory (same level as the daily.X), then rsnapshot copies daily.0 to .sync
- If there is a .sync, then use it as a helper for the rsync data transfer
- Once the data transfer is completed, copy (hard link the usual rsnapshot way) .sync to daily.0
The key in all of this is that I can sneak in my cmd_preexec right before the data transfer, provided that I use this sync_first. Directory rotation and clean-up happens after sync, hence the sync_first.
In rsnapshot.conf:
sync_first 1
(TAB, again)
So the daily cron job just needs to be doing the following:
/usr/local/sbin/ssh_wake && \
/usr/bin/rsnapshot -c /etc/rsnapshot.conf sync && \
/usr/bin/rsnapshot -c /etc/rsnapshot.conf daily
(or whatever variation using ionice to lessen the I/O load on the backup server)
Final caveat: this sequence is likely to fail the first time it is executed, because the .sync directory does not exist yet.
But starting with the second day… things start working as expected. Voilà!