= rdiff-backup SOP

== Contact Information

Owner::
  Fedora Infrastructure Team
Contact::
  #fedora-admin
Location::
  Phoenix
Servers::
  backup03 and others
Purpose::
  backups of critical data

== Description

We are now running a rdiff-backup of all our critical data on a daily
basis. This allows us to keep incremental changes over time as well has
have a recent copy in case of disaster recovery.

The backups are run from backup03 every day at 22:10UTC as root. All
config is in ansible.

The cron job checks out the ansible repo from git, then runs
ansible-playbook with the rdiff-backup playbook. This playbook looks at
variables to decide which machines and partitions to backup.

* First, machines in the backup_clients group in inventory are operated
on. If a host is not in that group it is not backed up via rdiff-backup.
* Next, any machines in the backup_clients group will have their /etc
and /home directories backed up by the server running rdiff-backup and
using the rdiff-backup ssh key to access the client.
* Next, if any of the hosts in backup_clients have a variable set for
host_backup_targets, those directories will also be backed up in the
same manner as above with the rdiff-backup ssh key.

For each backup an email will be sent to sysadin-backup-members with a
summary.

Backups are stored on a netapp volume, so in addition to the
incrementals that rdiff-backup provides there are netapp snapshots. This
netapp volume is mounted on /fedora_backups and is running dedup on the
netapp side.

== Rebooting backup03

When backup03 is rebooted, you must restart the ssh-agent and reload the
rdiff-backup ssh key into that agent so backups can take place.

....
sudo -i
ssh-agent -s > sshagent
source sshgent
ssh-add .ssh/rdiff-backup-key
....

== Adding a new host to backups

[arabic]
. add the host to the backup_clients inventory group in ansible.
. {blank}
+
If you wish to backup more than /etc and /home, add a variable to:::
  inventory/host_vars/fqdn like: host_backup_targets: ['/srv']
. On the client to be backed up, install rdiff-backup.
. {blank}
+
On the client to be backed up, install the rdiff-backup ssh public key
to::
  `/root/.ssh/authorized_keys` It should be restricted from:
  +
....
from="10.5.126.161,192.168.1.64" 
....
  +
  and command can be restricted to:
  +
....
command="rdiff-backup --server --restrict-update-only"
....

== Restoring from backups

rdiff backup keeps a copy of the most recent version of files on disk,
so if you wish to restore the last backup copy, simply rsync from
backup03. If you wish an older incremental, see rdiff-backup man page
for how to specify the exact time.

== Retention

Backups are currently kept forever, but likely down the road we will
look at pruning them some to match available space.

== Public_key:

....
ssh-dss
AAAAB3NzaC1kc3MAAACBAJr3xqn/hHIXeth+NuXPu9P91FG9jozF3Q1JaGmg6szo770rrmhiSsxso/Ibm2mObqQLCyfm/qSOQRynv6tL3tQVHA6EEx0PNacnBcOV7UowR5kd4AYv82K1vQhof3YTxOMmNIOrdy6deDqIf4sLz1TDHvEDwjrxtFf8ugyZWNbTAAAAFQCS5puRZF4gpNbaWxe6gLzm3rBeewAAAIBcEd6pRatE2Qc/dW0YwwudTEaOCUnHmtYs2PHKbOPds0+Woe1aWH38NiE+CmklcUpyRsGEf3O0l5vm3VrVlnfuHpgt/a/pbzxm0U6DGm2AebtqEmaCX3CIuYzKhG5wmXqJ/z+Hc5MDj2mn2TchHqsk1O8VZM+1Ml6zX3Hl4vvBsQAAAIALDt5NFv6GLuid8eik/nn8NORd9FJPDBJxgVqHNIm08RMC6aI++fqwkBhVPFKBra5utrMKQmnKs/sOWycLYTqqcSMPdWSkdWYjBCSJ/QNpyN4laCmPWLgb3I+2zORgR0EjeV2e/46geS0MWLmeEsFwztpSj4Tv4e18L8Dsp2uB2Q==
root@backup03-rdiff-backup
....
