backup

Technical Notes

These notes are my personal online notebook of useful commands and "how-to's". You are welcome to make use of them if you find them helpful. They obviously don't come with any warranty! Click on one of the category tags above for the notes in any category.

Best Open-Source Dropbox Alternatives

We have used btsync very successfully for the last couple of years, but their pricing model would make the way we use it extremely expensive.

Linux Backup solutions

Some resources for setting up backups:

Comparisons

Duplicity

Rsync

  • Main resources

Setting up permissions for bittorrent sync

The goal is to sync a common directory structure over multiple sites.

Permissions on server

  • create sync user:
useradd -g office -d /home/common -c "btsync service" -m btsync
  • Don’t use a sticky bit, or sync will not be able to delete files
  • Make sure all contributors to /home/common have primary group office and umask=002
    • This will mean that they all have read/write access to the common files
  • Recommended to use the SGID bit to lock all new files with group office and prevent this changing
chgrp -R office /home/common
chmod g+s /home/common #just to top directory (assume no sub-directories)
chmod -R g+w /home/common
  • Note that if a user creates a new file, when btsync replicates it on other systems it will lose ownership information (and be owned by btsync). But because it will have group office it will be editable.

First time setup

Installing rdiff-backup on CentOS 6 or 7

wget http://pkgs.repoforge.org/rdiff-backup/rdiff-backup-1.2.8-4.el6.rf.x86_64.rpm
rpm -ivh rdiff-backup*.rpm
rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm -Uvh http://dl.fedoraproject.org/pub/epel/beta/7/x86_64/epel-release-7-0.2.noarch.rpm #for 7
yum-config-manager --disable epel     #disable since we on

dd and ddrescue to backup partitions

dd if=/dev/VolGroup00/LogVol00 of=/dev/vg1000/LogVol00
dd if=/dev/VolGroup00/LogVol00 | bzip2 -1 > /data/LogVol00.dd.bz
dd if=/dev/VolGroup00/amfhome | bzip2 -1 > /data/amfhome.dd.bz
dd if=/dev/sdc1 | bzip2 -1 > /data/sdc1_grub.dd.bz
#dd if=/dev/VolGroup00/vm_amfh | bzip2 -1 > /data/vm_amfh.dd.bz
dd if=/dev/VolGroup00/vm_amfh of=/data/vm_amfh.dd

dd if=/dev/VolGroup00/jessica | bzip2 -1 > /data/jessica.dd.bz #20Gb
dd if=/dev/VolGroup00/fred | bzip2 -1 > /data/fred.dd.bz #10Gb

Installing rdiff-backup version 1.2.2 on Ubuntu

Ubuntu Hardy and Intrepid come with older versions or rdiff-backup, but the current stable version (1.2.2) is in the Jaunty repositories. To install it:
Download and install the .deb file from the bottom of this page: http://packages.ubuntu.com/jaunty/utils/rdiff-backup
Install the recommended libraries:
aptitude install python-pylibacl python-pyxattr

Drupal backup script

First it is necessary to create a backup user on the database which has (only) read access to everything. This is because its password will be embedded in the backup script which reduces security. This can be done with:

BKUSR=backup
BKPW=backuppassword
BKDIR=/var/www/dbbackup
mkdir -p $BKDIR
echo "GRANT SELECT, LOCK TABLES ON *.* TO '$BKUSR'@'localhost' IDENTIFIED BY '$BKPW'; FLUSH PRIVILEGES;" | mysql -u root -p

In the backup script, first some variables are set up

BKUSR=backup
BKPW=backuppassword
BKDIR=/var/www/dbbackup
SITE=cfaw.info

Backup LVM-based VM to another computer

#first stop VM
#
lvname=vm_www
lvcreate -L5G -s -n $lvname-snap /dev/VolGroup00/$lvname
#the VM can be re-started now
#turn the snapshop into a file image (optionally gzip it)
#dd if=/dev/VolGroup00/$lvname-snap | bzip2 -9 >/backup/$lvname.dd.bz2
dd if=/dev/VolGroup00/$lvname-snap >/backups/$lvname.img
#remove the snapshot
lvremove -f /dev/VolGroup00/$lvname-snap
#copy to another machine
scp $lvname.img root@jordan:/amf300;
rm -f $lvname.img
#on the other machine, run bzip2 -9 vm_xxxx.img

rdiff-backup strategy

To backup windows using rdiff-backup to another computer, four sub-systems need to be configured:

  1. Install cygwin (including rdiff-backup) on the windows machine
  2. Configure and test rdiff-backup across ssh using manual passwords
  3. Configure public key authentication for passwords
  4. Configure scheduled tasks to run the process automatically

Syndicate content