Project practice, redis cluster technology learning (11)

Redis.6 Fail over

Redis The cluster itself achieves high availability . High availability first needs to solve the scenario of partial cluster failure ： When a small number of nodes in the cluster fail, the cluster can normally provide services through automatic failover

Redis.6.1 Fault finding

When there is a problem with a node in the cluster , We need a robust way to ensure that we can identify whether the node

Something went wrong .Redis Nodes in the cluster pass through ping/pong Message implementation node communication , Messages can not only propagate node slot information , Other states such as ： Master slave state 、 Node failure, etc. . So fault discovery is also realized through message propagation mechanism , The main links include ： Subjective offline （pfail） And objective offline （fail）.

· Subjective offline ： Refers to a node that considers another node unavailable , That is, offline status , This state is not the final fault determination , It can only represent the opinions of one node , There may be misjudgment .

· Objective offline ： Indicators record the real offline of a node , Multiple nodes in the cluster think that this node cannot

use , The result of reaching a consensus . If the master node holding the slot fails , This node needs to be checked

Barrier transfer .

1. Subjective offline
2. Every node in the cluster will send... To other nodes on a regular basis ping news , Receiving node reply pong news
3. As a response . If in cluster-node-timeout Communication fails all the time , Then the sending node will
4. The receiving node is considered to be faulty , Mark the receiving node as the subjective offline （pfail） state . The flow is as shown in the figure .

Process description ：

1） node a send out ping Message to node b, If the communication is normal, it will be received pong news , node a Update last contact with node b Time of communication .

2） If node a And nodes b If there is a communication problem, disconnect , Next time there will be a reconnection . If communication fails all the time , The node a Recorded and nodes b Finally, the communication time will not be updated .

3） node a A timed task within the detected node b Finally, the communication time is very high cluster-nodetimeout when , Update local pair nodes b The state is subjective offline （pfail）.

Subjective offline is simply , When cluster-note-timeout One node can't finish with another node in time ping Message communication , The node is marked as a subjective offline state . Within each node cluster State Structures need to save other node information , It is used to judge the status of other nodes from their own perspective .

2. Objective offline

When a node judges the subjective offline of another node , The corresponding node status will follow the message in the cluster

spread .ping/pong The message of the message will carry the cluster 1/10 Other node status data , When the receiving node finds that the message body contains the node state of subjective offline , The fault node will be found locally

ClusterNode structure , Save to the offline report linked list .

Process description ：

1） When there are other nodes in the message body pfail State determines the state of the sending node , If the sending node is the master node, the pfail State processing , The slave node ignores .

2） find pfail Corresponding node structure , to update clusterNode Internal offline report list .

3） According to the updated offline report list, try to offline objectively .