Redis master-slave replication - the underlying principle of partial resynchronization

Implementation of partial resynchronization

The partial resynchronization function consists of three parts ：

• The replication offset of the primary server and the replication offset of the secondary server
• Replication backlog buffer for primary server

• Running the server ID

Copy offset

Both the master and slave servers executing replication will save a replication offset ：

• Each time the master server propagates to the server N A byte of data , Just copy your own offset +N
• Each time the slave server receives a message from the master server N Bytes of data , Add the value of your copy offset to N

Start state

Suppose... Is transmitted 14 Status after bytes

By comparing the replication offsets of the master and slave servers, we can determine whether the master and slave servers are in a consistent state , Same in , The difference is not in .

Copy squash buffer

The replication squeeze buffer is a fixed length queue maintained by the server , The default size is 1MB（ The length of a fixed length queue is fixed , Only when the number of queued elements is greater than the queue length , The first element to join the queue will be ejected , New elements will join the team ）.

The replication buffer of the master server will hold some recently propagated write commands , And the copy squeeze buffer will record the corresponding offset for each byte in the queue

Offset	....	10087	10088	10089	10090	10091	10092	10093
Byte value	....	'*'	3	'\r'	'\n'	's'	4	'T'

When the secondary server reconnects to the primary server , The slave server will go through PSYNC Command to copy its own offset offset Send to the main server , The master server will determine what kind of synchronous replication to perform on the slave server based on this replication offset

• If offset Data after offset （ That is to say offset+1 The data of ） Still exists in the copy squeeze buffer , Then the master server will perform partial synchronization
• If there is no copy buffer inside , Then the master server enters the complete synchronization operation

Server running ID

Every Reids Every server has its own running ID, function ID Automatically generated when the server starts , from 40 Random hexadecimal characters .

When the primary server is replicated from the secondary server for the first time , The main server will run its own ID Send to slave , And the slave server will run this ID Save up , When the slave is disconnected and reconnected to a primary server , The slave server will send the previously saved run to the current master server ID：

• If Run saved from the server ID And the current connection to the main server running ID identical , So it means that the master server that is currently connected is the one that is copied before disconnection from the server , The primary server can try to perform partial resynchronization
• If the run saved from the server ID And the current connection to the main server running ID Is not the same , So the primary server copied before disconnection from the server is not the currently connected primary server , The master server will fully synchronize the slave server .