Besides, rsync+inotify realizes real-time backup of data

rsync The advantages and disadvantages of

With the traditional cp、tar Compared with backup mode ,rsync It has high security 、 Fast backup 、 Support incremental backup and other advantages , adopt rsync It can solve the data backup requirements that do not require high real-time performance , For example, regularly backup files The server Data to remote end The server , Do data mirroring on local disk regularly .

With the continuous expansion of application system scale , Better requirements for data security and reliability ,rsync In the high-end business system has gradually exposed a lot of shortcomings , First ,rsync When synchronizing data , All documents need to be scanned and compared , Carry out differential transmission . If the number of files reaches millions or even tens of millions , Scanning all files will be very time consuming . And what's changing is often a small part of it , It's a very inefficient way . secondly ,rsync Can't monitor in real time 、 Synchronous data , Although it can pass through linux The way the daemons trigger synchronization , But there must be a time difference between the two triggers , This leads to inconsistency between the data of the server and the client , Unable to fully recover data in case of application failure . For the above reasons ,rsync+inotify The combination appears ！

inotify Introduce

Inotify It's a powerful 、 Fine grained 、 Asynchronous file system event monitoring mechanism ,linux Kernel from 2.6.13 rise , Joined the Inotify Support , adopt Inotify Can monitor the file system to add 、 Delete , modify 、 All sorts of subtle events like movement , Using this kernel interface , Third party software can monitor the changes of files in the file system , and inotify-tools It's such a third-party software .

We talked about it earlier ,rsync Can achieve triggered file synchronization , But by crontab Trigger in the way of daemons , There will be differences between the synchronized data and the actual data , and inotify You can monitor changes in the file system , When there is any change in the file , It triggers rsync Sync , This just solves the real-time problem of synchronous data .

install inotify and inotify-tools Tools

because inotify Feature needs Linux Kernel support , In the installation inotify-tools Make sure that Linux Whether the system kernel has reached 2.6.13 above , If Linux The kernel is lower than 2.6.13 edition , You need to recompile the kernel and add inotify Support for , You can also judge , Does the kernel support inotify.

Check whether the system supports inotify

#Linux shell

uname -a

Linux localhost 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3 x86_64 GNU/Linux # The kernel is larger than 2.6.13 Is to support the

ll /proc/sys/fs/inotify

Total usage 0

-rw-r--r-- 1 root root 0  5 month 19 20:02 max_queued_events

-rw-r--r-- 1 root root 0  5 month 19 20:02 max_user_instances

-rw-r--r-- 1 root root 0  5 month 19 20:02 max_user_watches

If you have the above three outputs , Indicates that the system has supported by default inotify, Then you can start installing inotify-tools 了 .

install inotify-tools

Debian System

apt-get install inotify-tools

CentOS System

yum install inotify-tools

Source mode

succeed in inviting sb. https://github.com/rvoicilas/inotify-tools/ Download source code, compile and install .

ll /usr/bin/inotifywa*

inotify-tools After installation , Will generate inotifywait and inotifywatch Two instructions , among ,inotifywait Used to wait for a specific event on a file or set of files , It can monitor any file and directory settings , And you can monitor the entire tree recursively .

inotifywatch Used to collect monitored file system statistics , Including each inotify How many times has the event happened .

inotify Parameter Introduction

inotify The following interface parameters are defined , Can be used to limit inotify Consume kernel memory Size . Because these parameters are memory parameters , therefore , According to the application requirements , Adjust its size in real time .

cat /proc/sys/fs/inotify/max_queued_evnets

Represents a call to inotify_init Time division rationing inotify instance In the queue event The maximum number of , Events that exceed this value are discarded , But it will trigger IN_Q_OVERFLOW event .

cat /proc/sys/fs/inotify/max_user_instances

For each one real user ID Can be created inotify instatnces The maximum number of .

cat /proc/sys/fs/inotify/max_user_watches

Represent each inotify instatnces The maximum number of directories that can be monitored . If the number of files monitored is huge , According to the situation , Increase the value appropriately , for example ：

echo 30000000 > /proc/sys/fs/inotify/max_user_watches

inotifywait Related parameters

Inotifywait It's a monitoring wait event , Can cooperate with shell Scripts use it , Here are some common parameters ：

•-m, namely –monitor, Indicates that the event listening state is always maintained .

•-r, namely –recursive, Represents a recursive query directory .

•-q, namely –quiet, Indicates that the monitoring event is printed out .

•-e, namely –event, This parameter allows you to specify the event to be monitored , Common events are modify、delete、create、attrib etc. .

Please refer to the manual for more detailed parameters .

man  inotifywait # Check out the manual