Remote incremental synchronization artifact Rsync

brief introduction

I want to build an automatic backup code （ Or postpone the application , Various documents ） The system of , For this reason NAS, But the cost of buying off the shelf is a little high , It's too much trouble to toss about . Two days before , I accidentally found a game of the guest cloud brush machine Armbian（ One is based on Arm Of Linux） System tutorial , Overjoyed , I immediately painted my two-year game guest cloud as Armbian, For two days , It's finally done , adopt USB turn SATA, A piece of idle 200G The hard disk of , This is enough to store some code files . Here's the brush Armbian：

After changing the system Linux Small host ：

This game cloud is Cortex A5 Chip , Poor performance , But the power consumption is also low , It can be turned on all day , So you can continue to use it as an article , Find another auto sync 、 The solution of backing up files is OK .
It's true that , That's what we're going to introduce today rsync.

rsync Is a common one Linux Applications , For file synchronization . It can be between a local computer and a remote computer , Or synchronize files between two local directories （ But it doesn't support synchronization between two remote computers ）
With other file transfer tools （ Such as FTP or scp） Different ,rsync The biggest feature of is that it will check the existing files of the sender and the receiver , Only the parts with changes are transmitted （ The default rule is that the file size or modification time changes ）.

that Windows There's this on the rsync Do you , After all , I want to start from Windows Go to Linux Sync files . Yes ！Windows The platform has a cwRsync,

cwRsync is a packaging of Rsync for Windows

It is rsync stay Windows The realization of . There are already methods , I'll do it next .

stay Linux Install the configuration on the host rsync Server side

1. Check rsync Installation
   Input rsync --version, It has been found that , If not , adopt apt-get install rsync Just install it

2. To configure rsyncd.conf file
   although rsync Already installed , But you need it to run as a server , We need to use the daemon, that is daemon Run it in the same way .
   At this time, we need a etc In the catalog /etc/rsyncd.conf file , What needs to be noted here is , There are some Linux Distribution version , Like me Armbian Is based on Debian Of , It's not rsyncd.conf Of this file , You need to create , Then fill in the configuration contents . If you had this file , Just fill in the configuration content directly .
   Profile contents ：

# /etc/rsyncd: configuration file for rsync daemon mode

# See rsyncd.conf man page for more options.

# configuration example:
uid = root
gid = root
use chroot = no
max connections = 2
pid file = /var/run/rsyncd.pid
log file = /var/log/rsyncd.log
lock file = /var/run/rsyncd.lock

[feng]
path = /mnt/file
read only = no
auth users = rsync
secrets file = /etc/rsyncd.secrets
list = no

Here is a brief introduction to the above configuration items , It was also changed by referring to other tutorials , All the official configuration items should be more than these , But that's not the focus of my research , These are enough .

• uid、gid： When using non root Account number , Then make sure that the folder of the service has this non root The permission of account writing
• use chroot = no ： Can I switch to root Catalog , When chroot by yes When , When the client connects to the module , First chroot Go to the directory specified by the module parameters , You have to use root jurisdiction , The port number must be 1024 within , And it can't be backed up path Link files outside the path
• max connections = 2 ： Represents the maximum number of simultaneous connections , That is, only two clients can connect to themselves at the same time . And this option must be associated with lock file = /var/run/rsyncd.lock The common use , If not specified , The default is /var/run/rsyncd.lock
• [feng] The contents in the brackets are defined casually , It represents a module , You need to specify the name of this module when connecting to the client , For example, here I am feng, In this way, the client will transfer files according to the configuration rules under this module
• path = /mnt/file： It represents the path that will be saved when the client synchronizes the data , Just define it according to your own situation
• read only = no ： If it's read-only , Then there will be no write synchronization , So it has to be shut down
• auth users = rsync ： Users for synchronization , You don't need to have , Virtual is OK . I also define myself , Later, when the client initiates the synchronization command, it will use
• secrets file = /etc/rsyncd.secrets ： Represents the account password file , This file is free to specify , The document must be marked with username：password The format of . This file permission must be 600, Otherwise, an error will be reported .username That's what I customized above auth users, For example, here I am rsync, Then I set the password to 123, The document says rsync:123 That's it
• list = no ： When the server rejects the client's request , Whether it is direct display permission rejection , Or the display module does not exist , Generally speaking, it is related to hosts allow and hosts deny Use it together . When a request rejected by the server comes in , If set to list = no, So the direct return module doesn't exist . If not set , So what is wrong is returned .

3. To daemons （daemon） Way to run
After the above items are configured , adopt rsync --daemon To start up rsync That's all right. , such rsync It will run as a background process , And it will start automatically .

Here's another line , Starting is starting , So how to stop , What if I modify the configuration file and need to restart , The method I learned is to find out PID, And then through kill PID To end the process .rsync By default 873 Port number , So I'll go through lsof -i tcp:873 Order to find out PID, then kill PID The end of the process .

That's all Linux On rsync As the installation and deployment process of the server , Let's see Windows On rsync Installation and configuration process as a client .

Windows On rsync Installation and configuration process as a client

1. download cwAsync client
   stay https://www.itefix.net/cwrsync download cwRsync client . Pay attention to distinguish between server and client , I can't use the server here , So I didn't see .

I downloaded it x86 That version . Unzip it directly to the directory you want to put it in , Then put the table of contents bin Register the path to the environment variable path in , In this way, you can directly in cmd The command line is written in the window .

2. stay cmd File synchronization is initiated from the command line in
Let's first look at an example that can run successfully ：
rsync --port=873 -av --progress --password-file=/cygdrive/d/test.password /cygdrive/e/test_projects/nsetter [email protected]::feng

• 873 It's the server side rsync Corresponding port number
• --password-file=/cygdrive/d/test.password, Is the password file specified in the local directory , This password file needs to be created in advance , For example, I am here D Create a test.password file , Add... Before the path /cygdrive/ Is the required fixed writing , Is in accordance with the linux The style of , If it's written directly as “D:\test.password” You can't .
  The contents of the file correspond to the above server rsyncd.secrets The password written inside , For example, here I am 123, Then write a in this file 123 That's it .
• /cygdrive/e/test_projects/nsetter This is the directory I want to synchronize , According to the above rules, you can see , This is a E A directory under the disk , This method will synchronize the entire directory , And a directory with the same name will be created on the server .
• [email protected]::feng, among rsync Corresponding to the above server settings auth users, The latter is the server IP Address , Double colon followed by module name , Remember the module name I mentioned in the server configuration file , It's here
• as for -av、--progress Belong to rsync One or two of the many command parameters of the , You can see a tutorial written by Ruan Yifeng （ https://www.ruanyifeng.com/blog/2020/08/rsync.html）, It's clear . I won't repeat it here .

Running results

Here is me Windows A directory on is synchronized to Linux The next result , Maybe it's the LAN , It's still very fast .

If I want to exclude a file or directory under the source directory , adopt --exclude Parameters can achieve , It's simple , Like this one down here , I just put packages The directory excludes , It will not be synchronized

because rsync Incremental synchronization , Synchronize only changed files , So don't worry about repeated synchronization 、 The problem of increasing expenses , Like this one down here test.txt file , I synchronized twice in a row , Second time there was no synchronization , that sending list It's empty .

summary

The above is what I currently build and use rsync The whole process of , This has basically met my needs , The next step is probably to improve , Change the input command line into an automated script , in my opinion , This is a first-class synchronization 、 The solution for backing up files . After all , Not only did it not cost a penny , So powerful and easy to use .

---

Things are easy to break when they are too hard , I hope you can understand .《 Menghua record 》