Amazon S3 Backup script for codebase

Codebase Or Data Backup script

Importance of Backup

Backup of all relevant data and files on time is the best quality of good sysadmin.
In this article, I have provided you a S3 backup script which helps you to backup your valuable data time to time and stored it on AWS S3 Cloud.
If your server crashed, we have a copy of the backup file on the S3 cloud which we used as an option to restoring a website.
Also, I have explained you each line of code mentioned in this script which gives you a deeper understanding of working with the script.

Just follow below step by step process to configure backup script on your server.

step 1: Install s3cmd command

You need to install s3cmd command before setting up the backup script. This command helps in the configuration of S3 on the server.

  • apt-get install s3cmd
  • s3cmd --configure -c ~/.s3cfg

While configuring s3, you need to provide Access Key and Secret Key.

I assumed that you have already created an s3 bucket for your files and have access and secret key for same. If not or you are new to aws then follow below link to setting up S3 bucket on AWS Cloud

Step 2: Create Backup Directory

Always store at list two latest copies of backup files on the server which you use in an emergency.

mkdir /opt/scripts -- To store Backup script file

mkdir /CodeBackup -- To store Backup data

Step 3: Backup Script

vim /opt/scripts/CodeBackups.sh

#!/bin/bash

TODAY_DATE=$(date +%Y%m%d-%H%M) [Store Date in Variable]
TAR=$(which tar) [Store the path of tar command]
MAIL=$(which mail) [Store the path of mail command]
LOGFILE='/tmp/CodeBackups.log' [Store the path of logfile]

MAILTO='prashant.t@gmail.com' [Add Email ids on which you want notification on backup complete or failed]
BODY="nnFor more information please check $LOGFILE.nnnThanks,n$0"

DOCROOTS="$@" [It will store the path of codebase which we provide as argument]

# This if..else condition check for path is correct or not.
if [ -z "${DOCROOTS}" ]; then
echo -e "Please provide code base path.nUsage: $0 ......"
exit 1
fi


#Delete Last 7 days old Codebase Backup from server only.
find /CodeBackup/*.tar.gz -mtime +7 -exec rm {} ;

# Code base backups.
for DOC in ${DOCROOTS}; do
echo "`date '+%Y-%m-%d %H:%M:%S'` : ${DOC} backup started"
# Code base backup and S3 upload.
if [ -d ${DOC} ]; then
FILENAME=$(echo $DOC | awk -F '/' '{print $NF}')
TARFILE="$FILENAME-$TODAY_DATE.tar.gz"
echo "`date '+%Y-%m-%d %H:%M:%S'` : ${DOC} archiving
started."
$TAR -cvzf /CodeBackup/$TARFILE /data/docroot/$FILENAME
#Below if condition check for errors if condition is 0 means backup successful or else backup failed.
if [ $? -eq 0 ]; then
echo "`date '+%Y-%m-%d %H:%M:%S'` : ${DOC} archiving
done."
else
echo "Error while creating tar."
echo -e "Hi Sysadmin,nnError while creating tar file.$BODY" | $MAIL -s 'Error : Creating tar file' $MAILTO
exit 1
fi
fi
echo "`date '+%Y-%m-%d %H:%M:%S'` : ${DOC} backup completed"
Done

# Below s3 command will put your backup data from server to s3 bucket.
echo -e " Uploading Codedump on S3 bucket"
s3cmd --config=/root/.s3cfg put /CodeBackup/$TARFILE s3://S3-BucketName

# After successful completion of backup you will get the mail notification.
SITE_NAME=$(echo $DOCROOTS | sed 's: :n:g' | awk -F '/' '{print $NF}')
echo -e "Hi Sysadmin,nnBackups successfully done for below codebasenn`echo ${DOCROOTS} | sed 's: :n:g'`$BODY" | $MAIL -s "Completed : Backup Seccessfully Done. [`echo $SITE_NAME | sed 's: :, :g'`]" $MAILTO

Note : Remove comment from script which is red color before use.

Step 4: Edit S3 Policy on Aws

As we already created IAM user with the name of “backupuser” so go to “Services-->IAM-->Users” then click on “backupuser”.
In backupuser go to the second tab i.e. “Permissions”. Under the Permissions tab, you will see the attached policy “S3CodeMediaDBBackupsFullAccess

Open this policy and click on edit button then only add below two bold lines in that policy.

Policy Name : S3CodeMediaDBBackupsFullAccess
Description : Full Access of buckets.
Policy Document :

{
"Version": "2012-10-17",
      "Statement": [
       {
"Effect": "Allow",
"Action": "s3:*",
        "Resource": [
               "arn:aws:s3:::mysqldbbackups-AWS_ACCOUNT_NAME",
               "arn:aws:s3:::mysqldbbackups-AWS_ACCOUNT_NAME/*"
               "arn:aws:s3:::codebasebackups-AWS_ACCOUNT_NAME",
               "arn:aws:s3:::codebasebackups-AWS_ACCOUNT_NAME/*"
                     ]
                  }
          ]
 }

After adding bold lines in policy before saving policy first, click on Validate Policy then save it.

Step 5: Test S3 Backup Script

After configuring s3 and backup script, you should check it on the server as working correctly or not.

$ bash -x CodeBackups.sh /data/docroot/codebase
[This is the path of the codebase or data which you want to take backup]

Step 6: Cron Schedule

After a successful test of backup script. We will plan our backup script to execute at a particular time. Here you can provide time when less traffic comes on your website.
To schedule backup script. create file under the /etc/cron.d/ directory and set your cron.

vim /etc/cron.d/CodeBackups
#Add cron like below
30 20 * * * root /bin/bash /opt/scripts/CodeBackups.sh /data/docroot/codebase > /tmp/CodeMediaBackups.log 2>&1

Related Posts:

If you’re facing any difficulties while configuring backup script please, do comment your queries or problem via the comment section, till then stay tuned to techthings.org for more such valuable articles.

Leave a Reply

Your email address will not be published. Required fields are marked *