Technical Notes

These notes are my personal online notebook of useful commands and "how-to's". You are welcome to make use of them if you find them helpful. They obviously don't come with any warranty! Click on one of the category tags above for the notes in any category.

Best Open-Source Dropbox Alternatives

We have used btsync very successfully for the last couple of years, but their pricing model would make the way we use it extremely expensive.

Linux Backup solutions

Some resources for setting up backups:




  • Main resources

Setting up permissions for bittorrent sync

The goal is to sync a common directory structure over multiple sites.

Permissions on server

  • create sync user:
useradd -g office -d /home/common -c "btsync service" -m btsync
  • Don’t use a sticky bit, or sync will not be able to delete files
  • Make sure all contributors to /home/common have primary group office and umask=002
    • This will mean that they all have read/write access to the common files
  • Recommended to use the SGID bit to lock all new files with group office and prevent this changing
chgrp -R office /home/common
chmod g+s /home/common #just to top directory (assume no sub-directories)
chmod -R g+w /home/common
  • Note that if a user creates a new file, when btsync replicates it on other systems it will lose ownership information (and be owned by btsync). But because it will have group office it will be editable.

First time setup

Installing rdiff-backup on CentOS 6 or 7

rpm -ivh rdiff-backup*.rpm
rpm -Uvh
rpm -Uvh #for 7
yum-config-manager --disable epel     #disable since we on

dd and ddrescue to backup partitions

dd if=/dev/VolGroup00/LogVol00 of=/dev/vg1000/LogVol00
dd if=/dev/VolGroup00/LogVol00 | bzip2 -1 > /data/
dd if=/dev/VolGroup00/amfhome | bzip2 -1 > /data/
dd if=/dev/sdc1 | bzip2 -1 > /data/
#dd if=/dev/VolGroup00/vm_amfh | bzip2 -1 > /data/
dd if=/dev/VolGroup00/vm_amfh of=/data/vm_amfh.dd

dd if=/dev/VolGroup00/jessica | bzip2 -1 > /data/ #20Gb
dd if=/dev/VolGroup00/fred | bzip2 -1 > /data/ #10Gb

Installing rdiff-backup version 1.2.2 on Ubuntu

Ubuntu Hardy and Intrepid come with older versions or rdiff-backup, but the current stable version (1.2.2) is in the Jaunty repositories. To install it:
Download and install the .deb file from the bottom of this page:
Install the recommended libraries:
aptitude install python-pylibacl python-pyxattr

Drupal backup script

First it is necessary to create a backup user on the database which has (only) read access to everything. This is because its password will be embedded in the backup script which reduces security. This can be done with:

mkdir -p $BKDIR
echo "GRANT SELECT, LOCK TABLES ON *.* TO '$BKUSR'@'localhost' IDENTIFIED BY '$BKPW'; FLUSH PRIVILEGES;" | mysql -u root -p

In the backup script, first some variables are set up


Backup LVM-based VM to another computer

#first stop VM
lvcreate -L5G -s -n $lvname-snap /dev/VolGroup00/$lvname
#the VM can be re-started now
#turn the snapshop into a file image (optionally gzip it)
#dd if=/dev/VolGroup00/$lvname-snap | bzip2 -9 >/backup/$lvname.dd.bz2
dd if=/dev/VolGroup00/$lvname-snap >/backups/$lvname.img
#remove the snapshot
lvremove -f /dev/VolGroup00/$lvname-snap
#copy to another machine
scp $lvname.img root@jordan:/amf300;
rm -f $lvname.img
#on the other machine, run bzip2 -9 vm_xxxx.img

rdiff-backup strategy

To backup windows using rdiff-backup to another computer, four sub-systems need to be configured:

  1. Install cygwin (including rdiff-backup) on the windows machine
  2. Configure and test rdiff-backup across ssh using manual passwords
  3. Configure public key authentication for passwords
  4. Configure scheduled tasks to run the process automatically

Syndicate content