Use Rsync and SSH implementation Snapshot type incremental backup

3.RotatingBackupsInthissetupIwilltellyouhowcanmakerotationbackups,sothatoldoneswillbedeletedeventually.Forthissetupitismandatory,thatthebackupservercanaccesstheproductionserverwithoutbeingpromptedforapasswordOnceyouhaveensured,thatyourbackupservercanconnecttoyourproductionserverwithoutbeingaskedforapasswordthenallyouneedisareacoupleofsmallshellscriptsandcronjobstoactuallyaccomplishthebackups.Indetailthishowtowillmake4backupsperday,then7backupsperweek(1perday)and4backupspermonth(1perweek)my_backup.sh(mysqlbackupshellscript)Thisfileneedstobeontheremote(production)serverthatyouwanttobackupfrom!Allotherfilesinthissectionneedtobeonthebackupserver!IcouldsomehowintegratethisshellscriptinthehourlybackupscriptthatwillfollowbelowbutIjustdidn'tworkoutyethowtodo.InsteadofincludingmysqlbackupscriptintothehourlybackupscriptIjustmakeanownshellscriptoutofitandIwilljustcallthatscriptfromthehourlybackupscript.Intheenditisthesamething.#!/bin/basunsetPAT#USERVARIABLESMYSQLUSER=rootMYSQLPWD=**********************MYSQLHOST=localostMYSQLBACKUPDIR=/mysql_backup#PATHVARIABLESMK=/bin/mkdir;RM=/bin/rm;GREP=/bin/grep;MYSQL=/usr/bin/mysql;MYSQLDUMP=/usr/bin/mysqldump;#CREATEMYSQLBACKUP#Removeexistingbackupdir$RM-Rf$MYSQLBACKUPDIR#Createnewbackupdir$MK$MYSQLBACKUPDIR#Dumpnewilesforiin$(echo'SHOWDATABASES;'|$MYSQL-u$MYSQLUSER-p$MYSQLPWD-h$MYSQLHOST|$GREP-v'^Database$');do$MYSQLDUMP\-u$MYSQLUSER-p$MYSQLPWD-h$MYSQLHOST\-Q-c-C--add-drop-table--add-locks--quick--lock-tables\$i>$MYSQLBACKUPDIR/$i.sql;done;Asyoucanseethisisthesameprocedureasbefore.backup_hourly.sh(backupshellscript)ThisscriptismostlybasedonMikeshandyrotating-filesystem-snapshotutilityscript#!/bin/basunsetPAT#USERVARIABLESBACKUPDIR=/backup#FolderwherethebackupsshallbesavedtoKEY=/root/.ssh/id_rsaMYSQL_BACKUPSCRIPT=/backup/my_backup.sPRODUCTION_USERroot@production.server.comEXCLUDES=/backup/backup_exclude#Filecontainingexludes#PATHVARIABLESCP=/bin/cp;MK=/bin/mkdir;SSH=/usr/bin/ss;DATE=/bin/date;RM=/bin/rm;RSYNC=/usr/bin/rsync;TOUCH=/bin/touc;SH=/bin/sh;#PathontheremoteserverMV=/bin/mv;######--DONOTEDITBELOWTHISHERE--#######CREATEMYSQLBACKUP#Runremotemysqlbackupscript$SSH-i$KEY$PRODUCTION_USER"$SH$MYSQL_BACKUPSCRIPT"#Rotatingthehourlysnapsots#step1:deletetheoldestsnapshot,ifitexists:if[-d$BACKUPDIR/hourly.3];then\$RM-Rf$BACKUPDIR//hourly.3;\i;#step2:shiftthemiddlesnapshots(s)backbyone,iftheyexistif[-d$BACKUPDIR/hourly.2];then\$MV$BACKUPDIR/hourly.2$BACKUPDIR/hourly.3;\i;#step3:shiftthemiddlesnapshots(s)backbyone,iftheyexistif[-d$BACKUPDIR/hourly.1];then\$MV$BACKUPDIR/hourly.1$BACKUPDIR/hourly.2;\i;#step4:makeahard-link-only(exceptfordirs)copyofthelatestsnapsot,#ifthatexistsif[-d$BACKUPDIR/hourly.0];then\$CP-al$BACKUPDIR/hourly.0$BACKUPDIR/hourly.1;\i;#step5:rsyncfromthesystem$RSYNC\-avz--delete--delete-excluded\--exclude-from="$EXCLUDES"\-e"$SSH-i$KEY"\$PRODUCTION_USER:/$BACKUPDIR/hourly.0;#step6:updatethemtimeofhourly.0toreflectthesnapshottime$TOUCH$BACKUPDIR/hourly.0;Well,prettymuchthescriptdoesthesameasthefirstone,justitwillrotatefolders....howeverthisisnowintendedfora6h-backupcycle(24dividedby4=6)...howeverwewanttohaveafewmorebackups.Sothenextthingisascriptthatcyclesbackupsonadailylevel.backup_daily.sh(dailyrotationshellscript)#!/bin/basunsetPAT#USERVARIABLESBACKUPDIR=/backup#Folderwherethebackupsshallbesavedto#PATHVARIABLESRM=/bin/rm;MV=/bin/mv;CP=/bin/cp;TOUCH=/bin/touc;#Rotatingthedailysnapsots#step1:deletetheoldestsnapshot,ifitexists:if[-d$BACKUPDIR/daily.6];then\$RM-Rf$BACKUPDIR/daily.6;\i;#step2:shiftthemiddlesnapshots(s)backbyone,iftheyexistif[-d$BACKUPDIR/daily.5];then\$MV$BACKUPDIR/daily.5$BACKUPDIR/daily.6;\i;#step3:shiftthemiddlesnapshots(s)backbyone,iftheyexistif[-d$BACKUPDIR/daily.4];then\$MV$BACKUPDIR/daily.4$BACKUPDIR/daily.5;\i;#step4:shiftthemiddlesnapshots(s)backbyone,iftheyexistif[-d$BACKUPDIR/daily.3];then\$MV$BACKUPDIR/daily.3$BACKUPDIR/daily.4;\i;#step5:shiftthemiddlesnapshots(s)backbyone,iftheyexistif[-d$BACKUPDIR/daily.2];then\$MV$BACKUPDIR/daily.2$BACKUPDIR/daily.3;\i;#step6:shiftthemiddlesnapshots(s)backbyone,iftheyexistif[-d$BACKUPDIR/daily.1];then\$MV$BACKUPDIR/daily.1$BACKUPDIR/daily.2;\i;#step7:shiftthemiddlesnapshots(s)backbyone,iftheyexistif[-d$BACKUPDIR/daily.0];then\$MV$BACKUPDIR/daily.0$BACKUPDIR/daily.1;\i;#step8:makeahard-link-only(exceptfordirs)copyofthelatesthourlysnapsot,#ifthatexistsif[-d$BACKUPDIR/hourly.3];then\$CP-al$BACKUPDIR/hourly.3$BACKUPDIR/daily.0;\i;#step9:updatethemtimeofdaily.0toreflectthesnapshottime$TOUCH$BACKUPDIR/daily.0;So,nowwehaveascriptthatdoes4backupsadayand7backupsaweek.Nowthelastonewillmake4backupsamonth.backup_weekly.sh(weeklyrotationshellscript)#!/bin/basunsetPAT#USERVARIABLESBACKUPDIR=/backup#Folderwherethebackupsshallbesavedto#PATHVARIABLESRM=/bin/rm;MV=/bin/mv;CP=/bin/cp;TOUCH=/bin/touc;#Rotatingtheweeklysnapsots#step1:deletetheoldestsnapshot,ifitexists:if[-d$BACKUPDIR/weekly.3];then\$RM-Rf$BACKUPDIR/weekly.3;\i;#step2:shiftthemiddlesnapshots(s)backbyone,iftheyexistif[-d$BACKUPDIR/weekly.2];then\$MV$BACKUPDIR/weekly.2$BACKUPDIR/weekly.3;\i;#step3:shiftthemiddlesnapshots(s)backbyone,iftheyexistif[-d$BACKUPDIR/weekly.1];then\$MV$BACKUPDIR/weekly.1$BACKUPDIR/weekly.2;\i;#step4:shiftthemiddlesnapshots(s)backbyone,iftheyexistif[-d$BACKUPDIR/weekly.0];then\$MV$BACKUPDIR/weekly.0$BACKUPDIR/weekly.1;\i;#step5:makeahard-link-only(exceptfordirs)copyofthelatestsnapsot,#ifthatexistsif[-d$BACKUPDIR/daily.6];then\$CP-al$BACKUPDIR/daily.6$BACKUPDIR/weekly.0;\i;#step4:updatethemtimeofweekly.0toreflectthesnapshottime$TOUCH$BACKUPDIR/weekly.0;Somuchtothescripts.cron.txt(crontabcontrolfile)Thelastthingthatismissingnowisacron.Iusethishere:#MakeBackups00**Sunsh/backup/backup_weekly.s |/usr/bin/mail-s"WeeklyCron" recipient@domain.com150***sh/backup/backup_daily.s |/usr/bin/mail-s"DailyCron" recipient@domain.com450,6,12,18***sh/backup/backup_hourly.s |/usr/bin/mail-s"HourlyCron" recipient@domain.com Well,Iruntheweeklycronatmidnightonsundaysandthedailycroneverydayaquarterpastmidnight.ThehourlycronsIrunaquartertomidnight,6am,noonand6pm.Ichosethistogiveenoughtimetocompleteallthetransfersiftherearenewadditions.Ofcourseyoucansetthisallindividually.Don'tforgettocreatetheexclusionfile(seeexplanationsinStep2)Youcanaddthiscronsimplybyissuingthefollowingcommand:crontabcron.txtJustmakesurethatyoucheckfirstthatyouhavenoothercronsrunning.Ifso,justaddthemtothecroncontrolfile.Listingthecronsforthecurrentuser:crontab-lWell,nowenjoythebackups.