backup

Technical Notes

These notes are my personal online notebook of useful commands and "how-to's". You are welcome to make use of them if you find them helpful. They obviously don't come with any warranty! Click on one of the category tags above for the notes in any category.

Recover disk drive using ddrescue

See user manual here: www.gnu.org/…escue_manual.html

Example script:

  • /dev/sdc is the bad drive which is being recovered to /dev/sdb
# first pass to copy the good sectors and make a log file of the bad
ddrescue -n /dev/sdc /dev/sdb logfile
# second pass keeps retrying the bad sectors
ddrescue -dr3 /dev/sdc /dev/sdb logfile

# have a look at what we have and do some validation
fdisk /dev/sdb
e2fsck -v -f /dev/sdb1
e2fsck -v -f /dev/sdb2

Best Open-Source Dropbox Alternatives

We have used btsync very successfully for the last couple of years, but their pricing model would make the way we use it extremely expensive.

Linux Backup solutions

Some resources for setting up backups:

Comparisons

Duplicity

Rsync

  • Main resources

Setting up permissions for bittorrent sync

The goal is to sync a common directory structure over multiple sites.

Permissions on server

  • create sync user:
useradd -g office -d /home/common -c "btsync service" -m btsync
  • Don’t use a sticky bit, or sync will not be able to delete files
  • Make sure all contributors to /home/common have primary group office and umask=002
    • This will mean that they all have read/write access to the common files
  • Recommended to use the SGID bit to lock all new files with group office and prevent this changing
chgrp -R office /home/common
chmod g+s /home/common #just to top directory (assume no sub-directories)
chmod -R g+w /home/common
  • Note that if a user creates a new file, when btsync replicates it on other systems it will lose ownership information (and be owned by btsync). But because it will have group office it will be editable.

First time setup

Installing rdiff-backup on CentOS 6 or 7

wget http://pkgs.repoforge.org/rdiff-backup/rdiff-backup-1.2.8-4.el6.rf.x86_64.rpm
rpm -ivh rdiff-backup*.rpm
rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm -Uvh http://dl.fedoraproject.org/pub/epel/beta/7/x86_64/epel-release-7-0.2.noarch.rpm #for 7
yum-config-manager --disable epel     #disable since we on

dd and ddrescue to backup partitions

dd if=/dev/VolGroup00/LogVol00 of=/dev/vg1000/LogVol00
dd if=/dev/VolGroup00/LogVol00 | bzip2 -1 > /data/LogVol00.dd.bz
dd if=/dev/VolGroup00/amfhome | bzip2 -1 > /data/amfhome.dd.bz
dd if=/dev/sdc1 | bzip2 -1 > /data/sdc1_grub.dd.bz
#dd if=/dev/VolGroup00/vm_amfh | bzip2 -1 > /data/vm_amfh.dd.bz
dd if=/dev/VolGroup00/vm_amfh of=/data/vm_amfh.dd

dd if=/dev/VolGroup00/jessica | bzip2 -1 > /data/jessica.dd.bz #20Gb
dd if=/dev/VolGroup00/fred | bzip2 -1 > /data/fred.dd.bz #10Gb

Installing rdiff-backup version 1.2.2 on Ubuntu

Ubuntu Hardy and Intrepid come with older versions or rdiff-backup, but the current stable version (1.2.2) is in the Jaunty repositories. To install it:
Download and install the .deb file from the bottom of this page: http://packages.ubuntu.com/jaunty/utils/rdiff-backup
Install the recommended libraries:
aptitude install python-pylibacl python-pyxattr

Drupal backup script

First it is necessary to create a backup user on the database which has (only) read access to everything. This is because its password will be embedded in the backup script which reduces security. This can be done with:

BKUSR=backup
BKPW=backuppassword
BKDIR=/var/www/dbbackup
mkdir -p $BKDIR
echo "GRANT SELECT, LOCK TABLES ON *.* TO '$BKUSR'@'localhost' IDENTIFIED BY '$BKPW'; FLUSH PRIVILEGES;" | mysql -u root -p

In the backup script, first some variables are set up

BKUSR=backup
BKPW=backuppassword
BKDIR=/var/www/dbbackup
SITE=cfaw.info

Backup LVM-based VM to another computer

#first stop VM
#
lvname=vm_www
lvcreate -L5G -s -n $lvname-snap /dev/VolGroup00/$lvname
#the VM can be re-started now
#turn the snapshop into a file image (optionally gzip it)
#dd if=/dev/VolGroup00/$lvname-snap | bzip2 -9 >/backup/$lvname.dd.bz2
dd if=/dev/VolGroup00/$lvname-snap >/backups/$lvname.img
#remove the snapshot
lvremove -f /dev/VolGroup00/$lvname-snap
#copy to another machine
scp $lvname.img root@jordan:/amf300;
rm -f $lvname.img
#on the other machine, run bzip2 -9 vm_xxxx.img

Syndicate content