Friday 6 March 2015

Cleaning old backups keeping other for specific time amount in bash

If you are writing scripts to manage backups on Linux by yourself I believe you were wondering how to manage them by some time period - not to delete too much and make your backup partition full too fast. Usually there is no need to use backups older than 1-2 weeks but sometimes you need to check some changes which were made 3-4 months ago. Especially when you are storing some databases and these were changing (or maybe they were some table addition/deletions).

In this scenario let say that we want to keep our backups for last 6 months. Because usually there is no need to keep daily backups older than 7-10 days so we decided to keep daily backups for last 10 days (made every day), then keep all backups from Sunday for next 3 months but also backups from Sunday from the beginning of month for next 9 months... Sounds crazy? Not at all! Many times in past I had to restore some pretty old backups to some important production servers. What is a simplest solution? I prefer not to use perl (don't know why, just don't like it) so I write that small solution with bash and awk:

# find /path/to/backup/ -daystart -mtime +10  -printf "%Te %Ta %h/%f\n" | awk '($2!="Sun"){print}' | xarg rm -rf {}
# find /path/to/backup/ -daystart -mtime +91  -printf "%Te %Ta %h/%f\n" | awk '($2=="Sun" && $1>7){print}'  | xarg rm -rf {}
# find /path/to/backup/ -daystart -mtime +273 -printf "%Te %Ta %h/%f\n" | awk '($2=="Sun" && $1<=7){print}' | xarg rm -rf {}

Indeed it is very simple! I think there is no need to explain these lines - every single one search on backup path and deletes files which matches some specific condition - files older than 10 days but not from Sunday, files older than 3 months, from Sunday, but not from first week of month and finally files older than 9 months, from Sunday which are from first week of month. In fact there is no need to use awk in last condition also to add $2=="Sun" in second line but I think that will a bit simpler explain how to manage some backups :).


No comments:

Post a Comment