Contents:

Rsync-Incr

What is rsync-incr

rsync-incr is a linux wrapper shell (bash) script around rsync to perform automated, unattended, incremental, disk to disk backups, automatically removing old backups to make room for new ones. It produces standard mirror copies browsable and restorable without specific tools.I have been using it in production daily at work and at home since 2004.

This page is at https://colas.nahaboo.net/Code/RsyncIncr

Goals

I wanted to have a backup system with the following properties:

User Manual

rsync-incr [options] N sourcedir destdir

rsync-incr will create destdir as a perfect mirror, and save in destdir.past a directory per run with copies of changed files old versions. These N (at most) directories of old versions of changed files are named by their dates in the form destdir.past/YYYY-MM-DD.HHhMN-SIZEm

Date of last backup is in the contents of file destdir.past/LAST_DATE. No need to append a trailing / to source and destination.
Dest must be on the local machine (maybe NFS-mounted), sourcedir can be on a remote machine via the ssh syntax host:dir.
SIZE in name is the disk space taken that this backup, in megabytes, before an optional compression via --cbf, to help you find the good value of N m (e.g: the max of past SIZEs), as this size is hard to find in --snap mode SIZE is rounded to upper bound: 0m means 0 bytes, 2m less than 2m

This a simple script, making backups usables by standard rsync (no need for a dedicated restore script). It has 2 basic modes of operation: If N has "m" appended (2m, 34m, ..) old versions are removed before backup until we have at least N megabytes free on dest, and the max of space taken by previous backups (+ 10%, see --pbsm). Otherwise, just keep the last N backups.

Options are passed to rsync, but must be a single word parts (e.g: use --rsh=ssh, not -e ssh)

(this will create an perfect backup in /backups/me and a series of previous versions as dirs like /backups/me.past/2004-10-26.04:40:20-234 ...) rsync-incr uses rsync with options: -HSax --delete --force

Special non-rsync options: otherwise a global find will be used, which can be very slow. It will only remove backups on same filesystem as destdir, so you can list all LAST_DATE paths on all disks. For instance if all the backups are organized as /backups/host/partition, the following should be included at the start of backup scripts to decrease startup time:
RSYNCINCR_LASTDATES=`ls -1 /backups/*/*.past/LAST_DATE` To restore a backup, use standard rsync (trailing slashes are IMPORTANT):

Rsync errors are propagated (the script exits with rsync exit status), except for the error #24 which is trapped, as this error can happen on backups of live systems (being modified while backuped)

Download

From github: https://github.com/ColasNahaboo/rsync-incr

Installation

Just copy the rsync-incr shell script anywherein your PATH, e.g: /usr/local/bin.

Implementation

A 120 lines of shell script (excluding the embedded doc), based on the fantastic rsync, and Mike Rubel article

License

Pure Open source: GPL

More details

See also what inspired me:

New versions announcements

New releases and important issues will be annonced on the Rsyncincr blog, so I strongly suggest you monitor it, either

Examples of use

See a detailed example of what rsync-incr does

Also, here are (modified for privacy) real scripts I use daily:

Other backup systems

Rsync-incr is not the only open source smart and simple backup system. It is I think unique in its automated claculation of free space and smart removal of old backups to make room for new ones. But other ones, can be more relevant to your needs, for instance:

History

Comments