Syncing files between storage devices and nodes on a network is a tedious task. Things could get incredibly annoying when you need to move files and entire folders to different computers on your network.
Fortunately, Linux offers rsync, a simple yet powerful utility that has existed since 1996.
rsync, also known as Remote Sync, is a free command-line application that lets you move files and folders to and from local and remote destinations. It is an excellent utility for mirroring, taking backups, and data migration to other servers.
The utility is fast and efficient because it copies only the modifications from the original and allows customizing the transfer process.
This tutorial blog will introduce rsync, the utility’s installation and operation, and ten examples covering most use cases.
Let’s start with the prerequisites.
Table of content
The Prerequisites of Installing rsync
To install rsync on your system, you’ll need the following:
- A non-root user with root access.
- SSH access to the remote machine.
Install rsync on Linux Machines
Now that you’ve fulfilled the prerequisites. You can proceed to the simple installation process.
If you’re using a Debian-based distro (such as Ubuntu), use the following command:
#apt-get install rsync
If you’re on an RHEL-based distro (such as CentOS, Rocky Linux, AlmaLinux, or Fedora), use the following command to get the rsync package on your system:
#yum install rsync
Getting to Know the rsync Command
Now that you have installed rsync on your system, it’s time to understand the basics of the command and how you can modify its operations with multiple options.
Syntax of the rsync Command
The syntax of the rsync command varies depending on how you use the utility.
In the following examples, we will go over all critical circumstances.
The basic rsync syntax looks like this:
#rsync <options> SOURCE DESTINATION
Note: If you’re using rsync for remote transfer, you must provide a host address.
The rsync Options
The true power of rsync lies in the options it offers to modify and enhance the utility’s operation.
If you need a list of these options, simply enter rsync in the terminal to see all the details,
We’ll now take a brief look at the most important rsync options.
| -r | Allows recursive data sync but does not maintain ownership for users and groups, permissions, timestamps, or symbolic links. |
| -a | The archive mode works similarly to the recursive method but preserves all file permissions, symbolic links, file ownership, etc. |
| -z | Used to conserve space by compressing data during transmissions. |
| -b | Takes a backup during data synchronization. |
| -h | The numbers in the output are shown in a human-readable format. |
| -n | Performs a dry run with no actual data synchronization or transfer. This is often used to test the rsync operation. |
| -e | For remote transfers, instruct rsync to utilize the SSH protocol. |
| -progress | During synchronization, this function displays the transfer progress. |
| -v | The output is verbose. Displays the transfer details. |
| -q | Used to silence the rsync command and option output. |
The rsync Command in Action
rsync is a flexible command that allows fine control over file transfer and synchronization. This power is one of the reasons why the command has been around for more than 19 years.
Let’s see some of these use cases where you can see how rsync adds value to file transfer and sync processes.
1. Copy a Single File Locally
To transfer a file from one directory to another on a local machine, enter the full path of the source file, followed by the target destination.
#rsync -v /home/RS.txt /home/redswitches/
The command copies the file RS.txt to the redswitches directory.
Important: If the target directory does not exist, append a slash to the end, and rsync will create it.
Alternatively, if you wish to copy a file from the current working directory, use the file’s name rather than the complete path.
2. Copy Multiple Files Locally
To transfer multiple files in one go, add the full paths of the source files and then enter the destination’s path.
#rsync -v /home/RS1.txt /home/RS2.txt /home/redswitches/
Note: Don’t go overboard and add multiple source file links. Use the –exclude option if the list is too big.
3. Copy a Directory and All Subdirectories Locally (Copy Files and Directories Recursively)
Use the -a (archive mode) or -r option (transfers the entire directory’s hierarchy) to transfer a directory and its contents to another location on your system.
For this demonstration, we went with the archive option:
#rsync -av /home/Linux/ /home/redswitches/
The above example demonstrates how to transfer the Linux directory to the redswitches directory. As a result, the rsync utility first created the Linux directory and then transferred the content from the source location.
4. Copy a File or Directory from a Local to Remote Machine
We’ll now demonstrate how to copy the directory /home/redswitches/Linux to /home/ (located on a remote system).
Important: Remember to specify the target IP address. After the source directory, add the username, IP address, and destination. Remember to put a colon (:) after the remote host’s IP address, with no spaces before the destination.
The command is as follows:
#rsync -av /home/linux/ [email protected]:/home/
Enter the full path of the file and the destination to copy a single file to a remote system.
#rsync -av /home/RS1.txt [email protected]:/home/
5. Copy Multiple Files or Directories from the Local to Remote Machine
As you did when transferring data locally, provide multiple file or directory paths you want to copy to a remote system.
The same syntax is required to copy a single file or directory.
#rsync -av /home/ubuntu/ /home/redswitches.txt [email protected]:/home/
6. Specify a Protocol for Remote Transfers
The -e option directs the rsync utility to use a specific file transfer protocol.
To transfer files remotely through SSH, use the rsync utility with the -e ssh option.
#rsync -e “ssh -p 2222” /home/RS2.txt [email protected]:/home/
7. Copy a File or Directory from a Remote to a Local Machine
rsync can quickly transfer files from a distant server to your local workstation.
To retrieve a directory from a server, precede the source with its IP address.
#rsync -av 178.162.xxx.x:/home/Linux /home/
To transfer a specific file from a distant server, enter the full path of the source file and the destination on your local workstation.
#rsync -av 178.162.xxx.x:/home/sample.txt /home/
8. Copy Multiple Files or Directories from the Remote Machine to the Local
To get several files or folders from a remote server, specify the paths using curly brackets followed by the server’s IP address. Use a comma to separate the paths.
#rsync -av [email protected]:{/home/Linux/,/home/redswitches.txt} /home/
The same applies to the files—list as many files as you need within the curly brackets.
9. Show rsync Progress During Data Transfer
When doing a large data backup, you may want to keep track of the transfer’s progress. By default, rsync doesn’t offer much information about the transfer process.
To see the running volume of data transferred, transfer speed, and remaining time, use the –progress option with the rsync command.
#rsync -av --progress [email protected]:/home/redswitches.bin /home/
10. Delete Source Files After The Transfer
Sometimes, you may wish to remove files from the source location after the transfer finishes.
A good example is when you want to send the weekly backup to a new server. After the transfer, you no longer need the files on the source location. This scenario is an excellent example of highlighting the flexibility of the rsync utility.
In that scenario, use the –remove-source-files flag to remove the given source file.
#rsync -v --remove-source-files [email protected]:/home/redswitches.bin /home/
Also Read: Master the Linux Free Command: 6 Examples For Sysadmins and DevOps Engineers
Conclusion
This blog post covered the basics of rsync, including its syntax, options, and use cases.
Rsync is a powerful tool that you can use for various file transfer and synchronization tasks. It is fast, efficient, and flexible, making it a valuable addition to any sysadmin’s toolkit.
If you want a powerful and versatile file transfer tool, rsync is a great option. With some practice, you can use rsync to automate your file management tasks and keep your data safe and secure.
For those seeking reliable hosting solutions to complement their Linux environment, RedSwitches stands out as a reputable hosting provider. With their cutting-edge infrastructure and exceptional support, RedSwitches empowers users to leverage rsync and other Linux tools seamlessly, creating a robust and streamlined computing experience.
FAQs
Q. What are the limitations of rsync?
rsync is a robust file synchronization program, but it has some limitations, such as the following:
Unidirectional synchronization, lack of real-time synchronization, limited conflict handling, network connectivity required, and no encryption.
Despite these restrictions, rsync remains a versatile and extensively-used file synchronization tool, particularly in *nix systems. It offers compression, allows quick delta transfers, and can handle various file formats and circumstances.
Q. How much memory does rsync need?
The rsync utility requires around 100 bytes to keep all essential information for one file; thus, a run with 80,000 files would need approximately 80M of RAM.
Q. Why do we use the rsync command in Linux?
The rsync utility may traverse a directory structure recursively and update a second location with any new/changed/removed files. It checks to verify whether the files are already there at the destination before transmitting them, saving bandwidth and time for anything it skips.
Q. Does rsync need to be on both servers?
Before proceeding, the rsync tool must be installed on the client and server machines. Rsync is available in two modes: local and remote. The local mode is utilized if both machines are on the same network. This employs minimum transfer security and can quickly accomplish transfers and synchronization via LAN.
