Hands on redis master-slave replication, sentinel master-slave switching, cluster sharding

Background

Redis The following technologies are provided 「Redis Sentinel『 Master slave switch 』、Redis Cluster『 Fragmentation 』」, Effective implementation of Redis High availability 、 High performance 、 High Scalability , In this paper, the above technologies are practiced by hands .

1. Redis Sentinel「 Master slave switch 」

• Monitor the online status of the master and slave nodes , And complete the switch according to the configuration 「 be based on raft agreement 」.
• Mainly from the perspective of replication, from the perspective of capacity , It's still a single machine .

2. Redis Cluster「 Fragmentation 」

• Through consistency hash The way , Distribute data to multiple server nodes ： Designed 16384 Hash slot , And distributed to multiple redis-server.
• When need is in Redis Cluster Access one of key when ,Redis Client first key Use CRC16 The algorithm calculates a value , Then on 16384 modulus , So each of them key They all have a number in 0-16383 The Hashi trough between , Then operate on the node corresponding to this slot .

One 、 Master slave copy

Setting details

#  Known gateways  IP  by ：172.17.0.1

#  start-up  master  node 
docker run -it --name redis-6380  -p 6380:6379 redis
docker exec -it redis-6380 /bin/bash
redis-cli -h 172.17.0.1 -p 6380

#  start-up slave node  1
docker run -it --name redis-6381  -p 6381:6379 redis
docker exec -it redis-6381 /bin/bash
redis-cli -h 172.17.0.1 -p 6381
replicaof 172.17.0.1 6380

#  start-up slave node  2
docker run -it --name redis-6382  -p 6382:6379 redis
docker exec -it redis-6382 /bin/bash
redis-cli -h 172.17.0.1 -p 6382
replicaof 172.17.0.1 6380

Then you can check master Node information , stay master-redis Next , perform ：

> info Replication

# Replication
role:master
connected_slaves:2
slave0:ip=172.17.0.1,port=6379,state=online,offset=686,lag=0
slave1:ip=172.17.0.1,port=6379,state=online,offset=686,lag=1
master_replid:79187e2241015c2f8ed98ce68caafa765796dff2
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:686
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:686

Then operate master node ,slave Nodes will automatically synchronize .

slave-redis perform replicaof no one It can be changed to the primary node .

Key points

1. View network related information ：

docker network ls
docker network inspect bridge

2. Mutual access between containers , Internal port number can be used , You can also use an external mapping port number ;

3. perform docker network inspect bridge after , You can view the gateway IP And each container IP, You can use gateway IP : External mapping port , or Containers IP : 6379 visit Redis;

Reference material

1. command ：SLAVEOF

Two 、Sentinel High availability

current state ：

1. gateway IP：172.17.0.1
2. master port ：6390
3. slave port ：6391,6392

Operation steps

1. Recreate redis Of docker Containers :

redis.conf The configuration is as follows ：

#  Default port 6379
port 6390

#  binding ip, If it is an intranet, it can be directly bound  127.0.0.1,  Or ignore , 0.0.0.0  It's the Internet 
bind 0.0.0.0

#  Daemons start 
daemonize no

Change the listening port number , And recreate redis Containers ：

docker run -p 6390:6390 -v D:\develop\shell\docker\redis\conf6390:/usr/local/etc/redis --name redis-conf-6390 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6390 /bin/bash
redis-cli -h 172.17.0.1 -p 6390

docker run -p 6391:6391 -v D:\develop\shell\docker\redis\conf6391:/usr/local/etc/redis --name redis-conf-6391 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6391 /bin/bash
redis-cli -h 172.17.0.1 -p 6391
slaveof 172.17.0.1 6390

docker run -p 6392:6392 -v D:\develop\shell\docker\redis\conf6392:/usr/local/etc/redis --name redis-conf-6392 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6392 /bin/bash
redis-cli -h 172.17.0.1 -p 6392
slaveof 172.17.0.1 6390

Then you can check master Node information , You can see master Acquired slave The port number of has returned to normal . stay master-redis Next , perform ：

> info Replication

# Replication
role:master
connected_slaves:2
slave0:ip=172.17.0.1,port=6391,state=online,offset=84,lag=0
slave1:ip=172.17.0.1,port=6392,state=online,offset=84,lag=0
master_replid:ed2e513ceed2b48a272b97c674c99d82284342a1
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:84
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:84

2. create profile

establish sentinel.conf, Write the following in the file ：

sentinel monitor bitkylin-master 172.17.0.1 6390 2
sentinel down-after-milliseconds bitkylin-master 5000
sentinel failover-timeout bitkylin-master 10000
sentinel parallel-syncs bitkylin-master 1

Detailed command ： instructions Sentinel To watch a man named bitkylin-master The primary server of , Marking this master server as offline requires at least 2 individual Sentinel agree! ;
Response timeout 5 Seconds are marked as subjective offline , The migration process starts after the subjective offline , Overtime 10 Seconds is the migration timeout , I don't know the purpose .

3. Create two more redis-docker Containers

Copy the configuration file to docker In container , There are two containers that need to copy this file ：

docker run -it --name redis-6490 redis
docker run -it --name redis-6491 redis
docker cp ./sentinel.conf dcbd015dbc0e:/data/sentinel.conf
docker cp ./sentinel.conf 7c8307730bcc:/data/sentinel.conf

4. perform redis-sentinel command

 redis-sentinel sentinel.conf

5. Final effect

At this time, start and stop at will redis Containers , You can see sentinel Done automatically redis The master-slave switch of , Master slave configuration does not require manual operation .

Reference material

1. Redis Of Sentinel file
2. Docker Container file operation
3. > Overwrite write ; >> Append write

3、 ... and 、Cluster colony

Operation steps

1. to update redis The configuration file

Mainly add cluster configuration information , The sample configuration file is as follows ：

#  Default port 6379
port 6390

#  binding  ip, If it is an intranet, it can be directly bound  127.0.0.1,  Or ignore , 0.0.0.0  It's the Internet 
bind 0.0.0.0

#  Daemons start 
daemonize no

#  Cluster configuration 
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

2. establish 6 A container

Based on the second section , Based on the latest configuration file , establish 6 A container , Note the new cluster bus port mapping ：

docker run -p 6390:6390 -p 16390:16390 -v D:\develop\shell\docker\redis\conf6390:/usr/local/etc/redis --name redis-conf-6390 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6390 /bin/bash
redis-cli -h 172.17.0.1 -p 6390

docker run -p 6391:6391 -p 16391:16391 -v D:\develop\shell\docker\redis\conf6391:/usr/local/etc/redis --name redis-conf-6391 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6391 /bin/bash
redis-cli -h 172.17.0.1 -p 6391

docker run -p 6392:6392 -p 16392:16392 -v D:\develop\shell\docker\redis\conf6392:/usr/local/etc/redis --name redis-conf-6392 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6392 /bin/bash
redis-cli -h 172.17.0.1 -p 6392

docker run -p 6393:6393 -p 16393:16393 -v D:\develop\shell\docker\redis\conf6393:/usr/local/etc/redis --name redis-conf-6393 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6393 /bin/bash
redis-cli -h 172.17.0.1 -p 6393

docker run -p 6394:6394 -p 16394:16394 -v D:\develop\shell\docker\redis\conf6394:/usr/local/etc/redis --name redis-conf-6394 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6394 /bin/bash
redis-cli -h 172.17.0.1 -p 6394

docker run -p 6395:6395 -p 16395:16395 -v D:\develop\shell\docker\redis\conf6395:/usr/local/etc/redis --name redis-conf-6395 redis redis-server /usr/local/etc/redis/redis.conf
docker exec -it redis-conf-6395 /bin/bash
redis-cli -h 172.17.0.1 -p 6395

3. Create a cluster directly through the command

> redis-cli --cluster create 172.17.0.1:6390 172.17.0.1:6391 172.17.0.1:6392 172.17.0.1:6393 172.17.0.1:6394 172.17.0.1:6395 --cluster-replicas 1

#  The following is the result of the command execution ：

>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.17.0.1:6394 to 172.17.0.1:6390
Adding replica 172.17.0.1:6395 to 172.17.0.1:6391
Adding replica 172.17.0.1:6393 to 172.17.0.1:6392
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: a9678b062663957e59bc3b4beb7be4366fa24adc 172.17.0.1:6390
   slots:[0-5460] (5461 slots) master
M: 41a4976431713cce936220fba8a230627d28d40c 172.17.0.1:6391
   slots:[5461-10922] (5462 slots) master
M: 1bf83414a12bad8f2e25dcea19ccea1c881d28c5 172.17.0.1:6392
   slots:[10923-16383] (5461 slots) master
S: 3d65eadd3321ef34c9413ae8f75d610c4228eda7 172.17.0.1:6393
   replicates 41a4976431713cce936220fba8a230627d28d40c
S: b604356698a5f211823ada4b45a97939744b1d57 172.17.0.1:6394
   replicates 1bf83414a12bad8f2e25dcea19ccea1c881d28c5
S: 2c1cc93221dc3830aa1eb28601ac27e22a6801cc 172.17.0.1:6395
   replicates a9678b062663957e59bc3b4beb7be4366fa24adc
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 172.17.0.1:6390)
M: a9678b062663957e59bc3b4beb7be4366fa24adc 172.17.0.1:6390
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: b604356698a5f211823ada4b45a97939744b1d57 172.17.0.1:6394
   slots: (0 slots) slave
   replicates 1bf83414a12bad8f2e25dcea19ccea1c881d28c5
M: 41a4976431713cce936220fba8a230627d28d40c 172.17.0.1:6391
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 3d65eadd3321ef34c9413ae8f75d610c4228eda7 172.17.0.1:6393
   slots: (0 slots) slave
   replicates 41a4976431713cce936220fba8a230627d28d40c
M: 1bf83414a12bad8f2e25dcea19ccea1c881d28c5 172.17.0.1:6392
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 2c1cc93221dc3830aa1eb28601ac27e22a6801cc 172.17.0.1:6395
   slots: (0 slots) slave
   replicates a9678b062663957e59bc3b4beb7be4366fa24adc
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

Cluster created successfully

Be careful

1. You need to open the cluster bus port number , The default is Service port number + 10000
2. cluster reset The command can remove the current node from the cluster

Reference material

1. redis-cluster colony - Installation and status verification
2. Redis Cluster tutorial
3. Redis Command Reference - Cluster tutorial