Research on redis High Availability Mechanism

1. Sentinel mechanism

1.1 The basic process Sentry is actually a process that runs in a special mode Redis process , It is mainly responsible for three tasks ： monitor 、 Elector （ Select the master library ） And notify the .

1) monitor . The sentinel process is running , Periodically send... To all master and slave libraries PING command , Check if they are still running online . If the slave does not respond to the sentry within the specified time PING command , The sentinel will mark it as “ Offline status ”; Again , If the main library doesn't respond to the sentry within the specified time PING command , The Sentry will judge that the main library is offline , Then start the process of master-slave switching .

2) Elector . After the main warehouse is hung , Sentinels need to get from multiple booths , Select a slave instance according to certain rules , Make it the new main library .

3) notice . After selecting the main library , The sentinel will send the connection information of the new master library to other slave Libraries . Execute... From the library replicaof command , Connect to the new master library , And copy the data . meanwhile , The sentinel will notify the client of the connection information of the new main database , Let them send the request operation to the new main library .

1.2 Subjective offline and objective offline

In a surveillance mission , Sentinels need to determine if the main library is offline .

1) Subjective offline : The sentinel process will use PING The command detects itself and the master 、 Network connection of slave library , Used to determine the state of an instance . If the sentry finds the master or slave library to PING The response to the command timed out , that , The sentinel will mark it as “ Subjective offline ”. If it's a slave Library , that , The sentry simply marked it as “ Subjective offline ”.

however , If it's a master library , It cannot simply be marked as “ Subjective offline ”. In order to prevent misjudgment , The cluster mode composed of multiple instances is usually adopted for deployment , This is also known as the sentry cluster . Introduce several sentinel examples to judge together , You can avoid a single sentry because your network is not good , And misjudge that the main database is offline .

2) Objective offline : When judging whether the main database is offline , It can't has the final say of a sentinel , Only most sentinel instances , It is judged that the master database has “ Subjective offline ” 了 , The master library will be marked as “ Objective offline ”, The principle of judgment is ： The minority is subordinate to the majority .

Simply speaking ,“ Objective offline ” The standard is , When there is N A sentinel instance , It's better to have N/2 + 1 The main database is “ Subjective offline ”, In order to determine the main database as “ Objective offline ”. thus , You can reduce the probability of miscalculation , It can also avoid unnecessary master-slave switch caused by misjudgment .

1.3 How to select a primary Library

First of all, let's look at the screening criteria . In general , We must make sure that the selected slave library is still running online .

therefore , In the election , In addition to checking the current online status of the slave Library , We also need to judge the network connection status before it . If the slave database is always disconnected from the master database , And the number of disconnection times exceeds a certain threshold , We have reason to believe that , The network condition of this slave database is not very good , You can sift this out of the library . How to judge specifically ？ You use the configuration item down-after-milliseconds * 10. among ,down-after-milliseconds It is the maximum connection timeout that we determine that the master-slave database is disconnected . If in down-after-milliseconds In milliseconds , The master and slave nodes are not connected through the network , We can think that the master-slave node is disconnected . If the disconnection occurs more than 10 Time , This shows that the network condition of the slave database is not good , Not suitable as a new master library .

We can grade three rounds according to the three rules :

The first round ： The one with the highest priority gets the highest score from the database . The user can go through slave-priority Configuration item , Set different priorities for different slaves . such as , You have two slaves , They have different memory sizes , You can manually set a high priority for instances with large memory . In the election , Sentinels will give high priority slaves high marks , If there is a slave library with the highest priority , So the main library is the new one . If the priority of the slave library is the same , So the sentry begins the second round of scoring .

The second round ： The slave database with the closest degree of synchronization with the old master database gets a high score . The rule is based on , If you choose the slave database that is closest to the old master database as the master database , that , There's the latest in the main database . How to judge the synchronization progress between the slave database and the old master database ？ I introduced you to , There is a command propagation process during master-slave synchronization . In the process , The main library will use master_repl_offset Record the current latest write operation in repl_backlog_buffer Position in , And from the library will use slave_repl_offset This value records the current replication progress . here , What we're looking for is from the library , its slave_repl_offset Need to be closest to master_repl_offset. If in all slave Libraries , There are... From the library slave_repl_offset Nearest master_repl_offset, Then its score is the highest , It can be used as a new main library . As shown in the figure below , From the old main library master_repl_offset yes 1000, Slave Library 1、2 and 3 Of slave_repl_offset Namely 950、990 and 900, that , Slave Library 2 Should be selected as the new master library .

Of course , If there are two from the library slave_repl_offset The value size is the same （ for example , Slave Library 1 And from the library 2 Of slave_repl_offset Values are 990）,

The third round ：ID The smaller one gets a higher score from the library . Each instance will have a ID, This ID It's similar to the slave library number here . at present ,Redis When selecting the master database , There is a default rule ： With the same priority and replication schedule ,ID The one with the smallest number gets the highest score from the library , Will be selected as the new master library . Come here , The new master library was selected ,“ Elector ” This process is done . Let's review the process again . First , The sentinel will be on line 、 Network state , Filter out a part of the library that does not meet the requirements , then , In order of priority 、 Replication progress 、ID Number size and then score the rest of the slave library , As long as there is the highest score from the library , Choose it as the new main library .