Use Rsync and SSH implementation Snapshot type incremental backup

Author:StephanJauBasedupontheworksof:FalkoTimme&MikeRubelIntroductionAsneitherhumannorcomputersareperfect(humanserr/computersmayfail)itisquiteobviousthatagoodbackupsystemwillpreventtoomuchdamageoncethecomputermaygodown.Thiscouldbeeitherbecausetheharddriveisfailing,becauseofhackers,becauseyouaccidentallydeletedsomethingimportant,...InthistutorialIwillshowyouhowtoautomatebackupsinanincrementalsnapshot-stylewaybyusingrSync.1.SettinguprSyncoverSS Firstofallyouneedarunningrsyncserverandclientthatconnecttoeachotherwithoutbeingrequiredtoenterapassword.MoresuitableeventohaveitrunthroughSSH(youmighttransfersensitivedata).Forthis,FalkoTimmehasalreadywrittenanexcellenhowto.YoucanfindithereMirrorYourWebSiteWithrsyncSincethathowtoisalreadyexcellentthere'snopointinwritinganotheroneaboutthissubject.FollowthishowtountilStep6(6TestrsyncOnmirror.example.com)andtestwhetheryoursetupworks.AsIwillusetwodifferentmethodsofmakingthisincrementalsnapshot-stylebackupsitisnecessaryforonethatthatthebackupservercanaccesstoproductionserverwithoutbeingpromptedforapasswordandfortheotheroneit'svice-versa.Note:InmycaseIdobackupmydataonafriendsserverandhebacksuphisdataonmine.SoinmycaseIneededtosetbothanyway.2.Non-RotatingBackupsInthissetupIwilltellyouhowyoujustkeepmakingbackupswithoutrotatingthemhenceneverdeleteanything.Forthissetupitismandatory,thattheproductionservercanaccessthebackupserverwithoutbeingpromptedforapassword.Onceyouhaveensured,thatyourproductionservercanconnecttoyourbackupserverwithoutbeingaskedforapasswordthenallyouneedisasmallshellscriptandacronjobtoactuallyaccomplishthebackup.backup.sh(backupshellscript)#!/bin/basunsetPAT#USERVARIABLESBACKUPDIR=/backup#FolderonthebackupserverKEY=/root/.ssh/id_rsaMYSQLUSER=rootMYSQLPWD=**********************MYSQLHOST=localostMYSQLBACKUPDIR=/mysql_backupBACKUP_USERroot@backup.server.comEXCLUDES=/backup/backup_exclude#Filecontainingexludes#PATHVARIABLESCP=/bin/cp;MK=/bin/mkdir;SSH=/usr/bin/ss;DATE=/bin/date;RM=/bin/rm;GREP=/bin/grep;MYSQL=/usr/bin/mysql;MYSQLDUMP=/usr/bin/mysqldump;RSYNC=/usr/bin/rsync;TOUCH=/bin/touc;######--DONOTEDITBELOWTHISHERE--#######CREATINGCURRENTDATE/TIMENOW=`$DATE'+%Y-%m'-%d_%:%M`MKDIR=$BACKUPDIR/$NOW/#CREATEMYSQLBACKUP#Removeexistingbackupdir$RM-Rf$MYSQLBACKUPDIR#Createnewbackupdir$MK$MYSQLBACKUPDIR#Dumpnewilesforiin$(echo'SHOWDATABASES;'|$MYSQL-u$MYSQLUSER-p$MYSQLPWD-h$MYSQLHOST|$GREP-v'^Database$');do$MYSQLDUMP\-u$MYSQLUSER-p$MYSQLPWD-h$MYSQLHOST\-Q-c-C--add-drop-table--add-locks--quick--lock-tables\$i>$MYSQLBACKUPDIR/$i.sql;done;#CREATENEWBACKUPDIR$SSH-i$KEY$BACKUP_USER"$MK$MKDIR"#RUNRSYNCINTOCURRENT$RSYNC\-avz--delete--delete-excluded\--exclude-from="$EXCLUDES"\-e"$SSH-i$KEY"\/$BACKUP_USER:/$BACKUPDIR/current;#UPDATETHEMTIMETOREFELCTTHESNAPSHOTTIME$SSH-I$KEY$BACKUP_USER"$TOUCH$$BACKUPDIR/current"#MAKEHARDLINKCOPY$SSH-i$KEY$BACKUP_USER"$CP-al$BACKUPDIR/current/*$MKDIR"Explanations:#!/bin/basunsetPAT#USERVARIABLESBACKUPDIR=/backup#FolderonthebackupserverKEY=/root/.ssh/id_rsaMYSQLUSER=rootMYSQLPWD=**********************MYSQLHOST=localostMYSQLBACKUPDIR=/mysql_backupBACKUP_USERroot@backup.server.comEXCLUDES=/backup/backup_exclude#Filecontainingexludes#PATHVARIABLESCP=/bin/cp;MK=/bin/mkdir;SSH=/usr/bin/ss;DATE=/bin/date;RM=/bin/rm;GREP=/bin/grep;MYSQL=/usr/bin/mysql;MYSQLDUMP=/usr/bin/mysqldump;RSYNC=/usr/bin/rsync;TOUCH=/bin/touc;Justsettheaccordingvariablesabove.NomuchexplanationneededIthink#CREATINGCURRENTDATE/TIMENOW=`$DATE'+%Y-%m'-%d_%:%M`MKDIR=$BACKUPDIR/$NOW/[...]#CREATENEWBACKUPDIR$SSH-i$KEY$BACKUP_USER"$MK$MKDIR"ThiswillcreateacurrentfolderforthebackupYYYY-MM-DD_HH:MM-ifyouwanttoyoucanaltertheformatofthis...Ijustthinkthisisteasytoread.#CREATEMYSQLBACKUP#Removeexistingbackupdir$RM-Rf$MYSQLBACKUPDIR#Createnewbackupdir$MK$MYSQLBACKUPDIR#Dumpnewilesforiin$(echo'SHOWDATABASES;'|$MYSQL-u$MYSQLUSER-p$MYSQLPWD-h$MYSQLHOST|$GREP-v'^Database$');do$MYSQLDUMP\-u$MYSQLUSER-p$MYSQLPWD-h$MYSQLHOST\-Q-c-C--add-drop-table--add-locks--quick--lock-tables\$i>$MYSQLBACKUPDIR/$i.sql;done;Thiswillfirstremoveallfilesinyourpreviousmysql-backup-dir.Thenitwillre-createit(Ichosetodoitthiswaybecauseonedoesnothavetoworryaboutanexistingfolderornot...).Thenitwillloop(asroot)throughallthedatabasesandcreateanown.sqlfileforeachdatabase.Youmaywanttoadjusttheparametersforthebackupofthedatabasesoryoumayjustwanttouseamysqldump--all-databaseswhichisprobablyquickerthanthelooping.HoweverIpreferhavingsingle.sqlfilesforallDBs#RUNRSYNCINTOCURRENT$RSYNC\-avz--delete--delete-excluded\--exclude-from="$EXCLUDES"\-e"$SSH-i$KEY"\/$BACKUP_USER:/$BACKUPDIR/current;#UPDATETHETIMETOREFLECTTHESNAPSHOTTIME$SSH-I$KEY$BACKUP_USER"$TOUCH$$BACKUPDIR/current"#MAKEHARDLINKCOPY$SSH-i$KEY$BACKUP_USER"$CP-al$BACKUPDIR/current/*$MKDIR"Thisnowmakesaanincrementalsyncofthefilesofyourproductionservertothebackupserver.Itwillallbestoredinthe"current"folder,afterwardsitwillcreateahardlinkcopytothepreviouslycreatednew"timestamp"folder.--exclude-from="$EXCLUDES"EXCLUDES=/backup/backup_excludeThiswillactasexclusionforthebackup.Iattachheremycurrentcontentofthisfile./backup//bin//boot//dev//lib//lost+found//mnt//opt//proc//sbin//sys//tmp//usr//var/log//var/spool//var/lib/pp4//var/lib/mysql/Thelastthingnowneededisacronthatwilldoallthebackups.Youcanusesomethingliketis:cron.txt(croncontrolfile)#MakeBackups00,6,12,18***sh/backup/backup.s Theabovewouldmakeabackupevery6hours.