Redis master-slave replication principle

Master-slave replication

Redis Defined as KV, Non relational memory database . Why fast , Because all operations are carried out in memory , Single thread does not involve lock waiting in multithreaded concurrent transaction processing , So access and access speed are very fast .
Redis Master slave replication and other databases such as MySQL The principle of master-slave replication is similar , All for the following reasons ：

• data redundancy ： Online hot standby is realized from the Library , It is another way besides persistent backup
• A single point of failure ： After the main database is abnormal , You can switch to the slave library to continue providing services , Provide high availability services
• Load balancing ： It can be combined with read-write separation （ The main library is responsible for providing writing operation , The slave library is responsible for providing read operations ）, To share the server load
• High availability base ： In addition to the above reasons , Master-slave replication is also the implementation basis of sentinel and cluster , Both rely on master-slave replication to achieve high availability

Implementation and principle of master-slave replication

Create replication

Redis The implementation of master-slave replication is very simple , Just one command ：

 Suppose there are two Redis example , The addresses are 127.0.0.1:6379 and 127.0.0.1:6380, I want to make 6380 The instance of the port becomes 6379 Slave Library of instance , Then execute the following command ：127.0.0.1:6380> SLAVEOF 127.0.0.1 6379OK

Of course, there are two other ways to establish replication from nodes ：

1. The configuration file ： Add to profile from server      slaveof
2. Start command ： stay redis-server Add... After the start command --slaveof

Disconnect replication

adopt slaveof no one Break the replication relationship . It should be noted that , After disconnecting the replication from the node, the data still exists , Just no longer synchronize the data changes of the master node .

The principle of replication

1. Carry out orders ： After starting from the library , Through the command （slaveof master_ip master_port） Save master database information , At this time, the replication process has not yet started
2. Establishing a connection ：slave There's a scheduled task inside , Check every second for new master Node connection and replication , If you find any , Just follow master The node attempts to establish socket network connections （ If the slave cannot establish a connection , Scheduled tasks will be retried indefinitely until the connection is successful or executed slaveof no one Cancel copy ）
3. send out ping command ：slave The node sends ping Order to master node
4. password authentication ： If master Set up requirepass, that slave The node must send materauth Password for master authentication
5. Data synchronization ： First connection master Will start a full copy , Send the data to slave Synchronization
6. Follow up every time master Receiving new data will asynchronously transfer incremental data to the slave Library , The process of realizing master-slave replication synchronization

Copy in full / Sync

Full volume replication is Redis The master-slave must go through the same stage when establishing replication for the first time Redis The earliest supported replication method . The command to trigger full replication is sync perhaps psync.redis2.8 Before using sync Only full synchronization can be performed ,2.8 Then use psync It also supports full synchronization and partial synchronization （ The incremental synchronization ）. The specific steps of full replication are as follows ：

1. The slave will send... To the master 「PSYNC ？ -1」 command , Request the master server to execute 「 Copy in full 」 operation .
2. The master node returns... According to the command FULLRESYNC
3. The slave records the master ID and offset
4. Master node execution bgsave Produce in the background RDB file ( The long-running ) And save RDB To local , And use a buffer （ Called replication buffer ） Record all write commands executed from now on
5. The master node bgsave After execution , take RDB The file is sent to the slave node （ The long-running ）; From the node first clear their old data （ If you have a lot of data , It will become a time-consuming operation ）, Then load the received RDB file （ The long-running ）, Update the database status to the master node to execute bgsave The database state at
6. The master node sends all the write commands in the aforementioned copy buffer to the slave node , Execute these write commands from the node , Update the database state to the latest state of the master node
7. If the slave node is turned on AOF, It triggers bgrewriteaof Implementation , To ensure that AOF The file is updated to the latest state of the master node
   

explain ： It can be seen from the process of full replication , Full replication is a very heavy operation ：

• The master node passes through bgsave command fork The subprocess goes on RDB Persistence , The process is very consuming CPU、 Memory ( Page table copy )、 Hard disk IO Of ;
• The master node will RDB The file is sent to the slave node , The bandwidth of master and slave nodes will be greatly consumed
• Clear old data from nodes 、 Load new RDB The file process is blocked , Unable to respond to client's command ; If you execute from the node bgrewriteaof, It's also going to bring extra consumption

Partial reproduction / Sync

In normal operation , When the slave node is copying the master node data , In case of network flash off or other abnormalities , The slave node will let the master node reissue the lost command data , The master node only needs to copy the buffer (repl-backlog-buffer) The consistency of data can be guaranteed by sending the data to the slave node , Compared with full replication , It costs a lot less . Steps are as follows ：

1. When the slave node has network interruption , More than the repl-timeout Time （ The default value is 60 second ）, The primary node will break the replication connection
2. The master node writes the requested data to “ Copy backlog buffer ”,repl-backlog-size Default 1MB
3. When recovering from a node , Reconnect the master , The slave node will offset And master nodes id Send to master
4. After the primary node is verified , If the data after the offset is in the buffer , Is sent cuntinue Respond to —— Indicates partial replication is possible
5. The main node sends the data of the buffer to the slave node , Make sure the master-slave copy is in normal state

summary

• Redis Master slave replication is mainly through PSYNC Command implementation .
• Replication is divided into Partial reproduction as well as Complete copy .
• Partial replication through Copy offset 、 Copy backlog buffer 、 Master node ID To achieve .
• Full copy through RDB as well as Copy the backlog buffer to achieve .
• The main problem of master-slave replication is The data backup 、 The problem of separation of reading and writing .

Reference documents

https://redis.io/docs/manual/replication/