Technical Notes

These notes are my personal online notebook of useful commands and "how-to's". You are welcome to make use of them if you find them helpful. They obviously don't come with any warranty! Click on one of the category tags above for the notes in any category.

Compress a folder into a single file for backup

tar -cvjSf filename.tar.bz2 foldername
-c create
-v verboase
-j bzip2
-S Sparse files handled efficiently
-f filename

Recover disk drive using ddrescue

See user manual here:…escue_manual.html

  • Install with apt install gddrescue

Example script:

  • /dev/sdc is the bad drive which is being recovered to /dev/sdb
# first pass to copy the good sectors and make a log file of the bad
ddrescue -n /dev/sdc /dev/sdb logfile
# second pass keeps retrying the bad sectors
ddrescue -dr3 /dev/sdc /dev/sdb logfile

# have a look at what we have and do some validation
fdisk /dev/sdb
e2fsck -v -f /dev/sdb1
e2fsck -v -f /dev/sdb2

# It might be necessary to manually copy partition

Best Open-Source Dropbox Alternatives

We have used btsync very successfully for the last couple of years, but their pricing model would make the way we use it extremely expensive.

Linux Backup solutions

Some resources for setting up backups:




  • Main resources

Setting up permissions for bittorrent sync

The goal is to sync a common directory structure over multiple sites.

Permissions on server

  • create sync user:
useradd -g office -d /home/common -c "btsync service" -m btsync
  • Don’t use a sticky bit, or sync will not be able to delete files
  • Make sure all contributors to /home/common have primary group office and umask=002
    • This will mean that they all have read/write access to the common files
  • Recommended to use the SGID bit to lock all new files with group office and prevent this changing
chgrp -R office /home/common
chmod g+s /home/common #just to top directory (assume no sub-directories)
chmod -R g+w /home/common
  • Note that if a user creates a new file, when btsync replicates it on other systems it will lose ownership information (and be owned by btsync). But because it will have group office it will be editable.

First time setup

Installing rdiff-backup on CentOS 6 or 7

rpm -ivh rdiff-backup*.rpm
rpm -Uvh
rpm -Uvh #for 7
yum-config-manager --disable epel     #disable since we on

dd and ddrescue to backup partitions

dd if=/dev/VolGroup00/LogVol00 of=/dev/vg1000/LogVol00
dd if=/dev/VolGroup00/LogVol00 | bzip2 -1 > /data/
dd if=/dev/VolGroup00/amfhome | bzip2 -1 > /data/
dd if=/dev/sdc1 | bzip2 -1 > /data/
#dd if=/dev/VolGroup00/vm_amfh | bzip2 -1 > /data/
dd if=/dev/VolGroup00/vm_amfh of=/data/vm_amfh.dd

dd if=/dev/VolGroup00/jessica | bzip2 -1 > /data/ #20Gb
dd if=/dev/VolGroup00/fred | bzip2 -1 > /data/ #10Gb

Installing rdiff-backup version 1.2.2 on Ubuntu

Ubuntu Hardy and Intrepid come with older versions or rdiff-backup, but the current stable version (1.2.2) is in the Jaunty repositories. To install it:
Download and install the .deb file from the bottom of this page:
Install the recommended libraries:
aptitude install python-pylibacl python-pyxattr

Drupal backup script

First it is necessary to create a backup user on the database which has (only) read access to everything. This is because its password will be embedded in the backup script which reduces security. This can be done with:

mkdir -p $BKDIR
echo "GRANT SELECT, LOCK TABLES ON *.* TO '$BKUSR'@'localhost' IDENTIFIED BY '$BKPW'; FLUSH PRIVILEGES;" | mysql -u root -p

In the backup script, first some variables are set up


Syndicate content