Automate server to server backups with rsync and ssh

Posted on April 04, 2013
rsync

Taken from Ubuntu Forums

Several times I’ve had to configure automated backups for my machines, and each time I do, a long enough period has elapsed since the previous time that I forget all the important details. So this little tutorial is as much to help me remember (and to keep a record and to trace the steps) as for other users.

1. What You Need

a) ssh (should be installed by default) and rsync (“sudo apt-get install rsync” if you don’t already have it).

b) Users on both machines (the source machine and the destination machine), ideally with the same username (so: if user “root” exists on source machine, user “root” should also exist on destination machine).

c) ssh and rsync installed on both machines.

2. Testing ssh (also see step 4)

Start on the source machine. As user (“root” in my case — not root) try to login to the destination machine using ssh, specifying either an ip address or a hostname (if the hostname appears in /etc/hosts). You’ll be asked for a password. This is the password for the user on the destination machine, not the local password (in my case, they’re the same). For me, the destination machine is called “Server”:

(I will show my full prompt several times in this tutorial, but obviously you just enter the last part. For above, it would be “ssh Server”, or whatever your hostname/ip is.)

Type yes to the authentication message.
Enter the password. (If you are logging on as another user, you can type the username before the host, i.e.: ssh root_other_user@Server).
All good so far. ssh works. If it doesn’t use google to search for the error message.
Type “exit” to return to the source machine.

3. Setup public/private key pair

Follow the prompts, just hitting enter for the passphrase. This will yield the id_dsa.pub and id_dsa files (the public and private key pair):

4. Copy the public key to the destination machine:

If you already tested ssh as in step 2 above, you’ll simply be asked for the passphrase. If not, you’ll get this:

You should be logged in to the remote machine, and you should (might) see a test message about confirming the setup.

5. Logout of the destination machine and try getting back in without the password:

This (with your correct username and hostname/ip address) should get you in to the destination machine. If it works, it will work every time for all ssh connections for that user. No password required (this is actually a small security risk, but it has a big payoff — see next step.

6. Make a backup script (mine’s in my home directory, but it could be anywhere):

Enter the rsync options and paths into the new file. The following line will login (with user “root”, for which we just setup password-less ssh logins) and rsync the home directory on my source machine with a backup directory on the destination machine (“Server”):

The closing slashes matter. Don’t leave them out. The varuzP options mean “verbose, archive, recursive, update, compress, partial”. Check “rsync -h” from a command prompt to see what this all means. Basically, it will create directories if they don’t exist, won’t overwrite newer files on the destination, and won’t delete any files on the destination that have been deleted from the source machine (for that, you use “–delete”).

(If you did not setup password-less logins, the script would halt and ask for a password. And this would prevent the procedure from being automated.)

Save and close backup.txt
From the directory in which backup.txt was saved, type:

If it balks (i.e. permissions), change the permissions. I would use “chmod 775 backup.txt”, though this is a fairly lax permission scheme. (I have a personal network, which only I and my Server use, and I’m behind a hardware firewall. Your setup may have different requirements. At least make the script exectuable (“chmod +x backup.txt”, I think, but permissions are not my specialty).

Run the backup and see how it goes. If it works and you like what it’s doing, go to step 7. Otherwise, adjust it.

7. Automate the process.

make a new file called crons.cron (I think you can call it anything, but I’not sure):

Put some cron variables in there. Mine says this:

The titles are just reminders for me. They are commented out, as you can see. This script runs my backup.txt script every day at 11pm.

Save and exit crons.cron
Add it to your crontab:

List your crontab, to check that it’s in there properly:

You should get a nice little message showing your cron settings.

Done.

A Final Word:

Don’t just follow this tutorial willy-nilly. Your system may have a configuration that makes parts of what I’m outlining dangerous or foolish. Try to understand (at least minimally) what you are doing. This setup does one thing only: it syncronizes files from a directory on a local machine (the source) to a directory on a destination machine. It does not make multiple copies of anything (as a true backup scheme would) and it will overwrite matching files on the destination unless they are newer. Be cautious. Rsync is an amazingly powerful tool, but with great power…

___________

Other users will likely have better or more efficient ways of doing this. Please do post them so folks can see what options there are. And, of course, I may have made a host of mistakes that I don’t see at the moment. Please help me correct them.

I hope this helps to keep someone’s data safe.

Related Posts

  • billtheamerican

    Very good article on rsync. Here is one from around 1999 but still the very best rsync tutorial I’ve ever read. Explains it very well IMO: http://tinyurl.com/l37guv8

Contact Us

Have a question? Send us a message. We'll get back to you soon.

captcha

0 Shares
Share
Share
Tweet
+1
Pin
Stumble