Useful rsync command examples for SysAdmins

Useful rsync commands examples for SysAdmins

Rsync Command

rsync (remote sync) is used to transfer data between two servers. it is basically a remote synchronization tool which helps to copy or synced data from one machine to another machine. it is a Unix-Linux based tool.

Benefits of Rsync command:

Rsync is a very useful tool for system admins because every sysadmin should use this command at list once in a day. It has many benefits we will see some of them important points.

1. Rsync efficiently sync files to and from systems.
2. It is faster sync tool than any other syncing tools like SCP.
3. It has the feature of incremental data transfer means it will not sync data which is already available on another system.
4. Rsync has provided progress of data transfer which is also plus. you will know how much data is copied and how much is remaining.
5. when sending and receiving data rsync will use compression and decompression method. It comes with benefits of using less bandwidth.
6. Rsync will copy everything as it is which means sync links, owner, permission, groups.

In this article, we are going to see some of the most used rsync command examples. Most of the Linux distros come with the by default rsync command. If you don't have rsync install on your system then run below command to install rsync.

$ apt-get update
$ apt-get install rsync

Once you install rsync command. You will also learn every parameter of rsync command using "rsync --help".

Here we are going to run some of them important most used parameters with rsync command.

Syntax of rsync command:

$ rsync [OPTION]... SRC [SRC]... DEST
$ rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST
$ rsync [OPTION]... SRC [SRC]... rsync://[USER@]HOST[:PORT]/DEST
$ rsync [OPTION]... [USER@]HOST:SRC [DEST]
$ rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]

Some important Options of rsync command:

-v, --verbose : Verbose or Providing detailed output for diagnostic purposes.
-a, --archive : archiving allow copying files with symlinks, permissions, owners, groups etc
-z, --compress : Compress data during file transfer
-h, --human-readable : output numbers in a human-readable format
-P, --progress : show progress during transfer

1. Sync files and directory locally

This command is used to copy or sync files from one location to other location on the same local machine.

prashant@local ~ $ rsync -avzP export_all_products.csv Documents/tmp/

sending incremental file list
created directory Documents/tmp
export_all_products.csv
9,095,542 100% 74.51MB/s 0:00:00 (xfr#1, to-chk=0/1)

sent 162,928 bytes received 71 bytes 325,998.00 bytes/sec
total size is 9,095,542 speedup is 55.80

You will see in the rsync output that if the directory is not present at destination then rsync will create it and transfer files.

2. Sync directory

This command is used to copy a directory recursively from one location to other location.

prashant@local~ $ rsync -avzP /home/prashant/testsync Documents/tmp/

sending incremental file list
testsync/
testsync/file1
0 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=4/6)
testsync/file2
0 100% 0.00kB/s 0:00:00 (xfr#2, to-chk=3/6)
testsync/file3
0 100% 0.00kB/s 0:00:00 (xfr#3, to-chk=2/6)
testsync/file4
0 100% 0.00kB/s 0:00:00 (xfr#4, to-chk=1/6)
testsync/file5
0 100% 0.00kB/s 0:00:00 (xfr#5, to-chk=0/6)

sent 336 bytes received 115 bytes 902.00 bytes/sec
total size is 0 speedup is 0.00

3. Copy/sync files and directory to remote server

This command used to copy or sync files and directory's recursively from the local machine to remote server.

In this example, we are transferring "/home/Prashant/testsync" directory from the local machine to remote server "root@192.168.3.79" under the "/data/backups/" path.

prashant@local~ $ rsync -avzP /home/prashant/testsync root@192.168.3.79:/data/backups/

sending incremental file list
testsync/
testsync/file1
0 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=4/6)
testsync/file2
0 100% 0.00kB/s 0:00:00 (xfr#2, to-chk=3/6)
testsync/file3
0 100% 0.00kB/s 0:00:00 (xfr#3, to-chk=2/6)
testsync/file4
0 100% 0.00kB/s 0:00:00 (xfr#4, to-chk=1/6)
testsync/file5
0 100% 0.00kB/s 0:00:00 (xfr#5, to-chk=0/6)

sent 336 bytes received 115 bytes 902.00 bytes/sec
total size is 0 speedup is 0.00

4. Copy/Sync Remote Directory to Local Machine

This command used to copy files or directories from the remote machine to the local system.

In this example, we are transferring "/data/backups/testdirectory" directory to local machine "Documents/tmp" directory.

prashant@local ~ $ rsync -avzP remote-machine@192.168.3.213:/home/mahesh/Public Documents/tmp
remote-machine@192.168.3.213's password:
receiving incremental file list
Public/
Public/file1
0 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=9/11)
Public/file10
0 100% 0.00kB/s 0:00:00 (xfr#2, to-chk=8/11)
Public/file2
0 100% 0.00kB/s 0:00:00 (xfr#3, to-chk=7/11)
Public/file3
0 100% 0.00kB/s 0:00:00 (xfr#4, to-chk=6/11)
Public/file4
0 100% 0.00kB/s 0:00:00 (xfr#5, to-chk=5/11)

sent 218 bytes received 576 bytes 16.04 bytes/sec
total size is 0 speedup is 0.00

5. Use --exclude and --include option in Rsync

--exclude and --include is a very useful option provided by rsync. Lots of time we need to copy data from the server or local machine by excluding some directory's that time you can use --exclude option. When you want to include specific directory or file then use --include.

In below example, we are excluding "test" directory and copy files from remote machine to local machine. check below output you will found "test" directory not copied on the local machine.

prashant@local ~ $ rsync -avzP --exclude 'test' remote-machine@192.168.3.213:/home/mahesh/Public Documents/tmp
remote-machine@192.168.3.213's password:
receiving incremental file list
Public/
Public/file1
0 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=3/5)
Public/file2
0 100% 0.00kB/s 0:00:00 (xfr#2, to-chk=2/5)
Public/file3
0 100% 0.00kB/s 0:00:00 (xfr#3, to-chk=1/5)
Public/file4
0 100% 0.00kB/s 0:00:00 (xfr#4, to-chk=0/5)

sent 114 bytes received 285 bytes 20.46 bytes/sec
total size is 0 speedup is 0.00

6. Use of --delete option in rsync

This --delete option used when you want to sync data from the local machine and some of the files which are not available on local but present on a remote machine or at the destination. This time --delete option will help you to remove unused files from the destination.

prashant@local ~ $ rsync -avzP --delete remote-machine@192.168.3.213:/home/mahesh/Public Documents/tmp
remote-machine@192.168.3.213's password:
receiving incremental file list
deleting Public/test/file9
deleting Public/test/file8
deleting Public/test/file7
deleting Public/test/file6
deleting Public/test/file5
deleting Public/test/file10
deleting Public/test/
Public/

sent 28 bytes received 137 bytes 36.67 bytes/sec
total size is 0 speedup is 0.00

7. Remote Source Files after sync

This option helps to delete or clear files which are transferred or synced to another path. --remove-source-files only removed files not directory.

prashant@local ~/Documents $ rsync -avzP --remove-source-files test.txt /home/prashant/Documents/tmp
sending incremental file list
test.txt
0 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=0/1)

sent 107 bytes received 43 bytes 300.00 bytes/sec
total size is 0 speedup is 0.00

prashant@local ~/Documents $ ll test.txt
ls: cannot access test.txt: No such file or directory

If you want to learn more options and more about the rsync command then check man page of rsync. 

I hope you like the article if you find any difficulties using rsync command then please do comment your queries or problem via the comment section, till then stay tuned to techthings.org for more such valuable articles.

Rsync command mostly use in backup scripts if you want to know how to use rsync for backup data then refer below links,

READ : Backup script data from Remote Server to Local System

READ : Amazon S3 Backup script for codebase

Leave a Reply

Your email address will not be published. Required fields are marked *