Redis fragment cluster setup and use tutorial

Catalog

Preface

Set up the cluster architecture diagram

Lead to

Steps to build

Create clusters

Redis Hash slot description

Cluster scaling （ Add a node ）

Fail over

Use redistemplate Access the partitioned cluster

Preface

redis It can be said that it is used very frequently in actual project development , stay redis In common clusters , We talked redis Several common cluster schemes , Different clusters correspond to different scenarios , And the advantages and disadvantages of various clusters are explained in detail , This article will use redis Partition cluster is the starting point , from redis The construction of the partitioned cluster begins , Tell me more about redis Technical points related to partitioned clusters ;

A single point of failure ;

Write alone （ High concurrent write ） bottleneck ;

Single machine storage data capacity limit ;

Cluster auto scaling ;

Automatic cluster failover ;

Set up the cluster architecture diagram

Lead to

1、 Prepare one （ or 6 Servers , If conditions permit ） Virtual machine or server ;

This example prepares a memory for the demonstration effect 8G Server for , Will be done through the port master and slave Division

2、 Download ahead of time redis Installation package ;

Steps to build

1、 establish 6 A directory file , It is used to save each redis Sample data

mkdir 7001 7002 7003 8001 8002 8003

2、 Create a... In the current directory redis.conf file , The contents are as follows

port 6379# Turn on the cluster function cluster-enabled yes# The name of the configuration file for the cluster , We don't need to create , from redis Self maintenance cluster-config-file /usr/local/soft/redis/6379/nodes.conf# Timeout for node heartbeat failure cluster-node-timeout 5000# Persistent file storage directory dir /usr/local/soft/redis/6379# Binding address bind 0.0.0.0# Give Way redis Background operation daemonize yes# Registered instance ipreplica-announce-ip Local public network IP# Protected mode protected-mode no# Number of databases databases 1# journal logfile /usr/local/soft/redis/6379/run.log

3、 Copy the configuration file to various directories in batch

echo 7001 7002 7003 8001 8002 8003 | xargs -t -n 1 cp redis.conf

4、 Modify each directory redis.conf, Will be one of the 6379 Change to be consistent with the directory

printf '%s\n' 7001 7002 7003 8001 8002 8003 | xargs -I{} -t sed -i 's/6379/{}/g' {}/redis.conf

Open a random redis.conf See if the modification is successful

4、 Use the following command to start

/usr/local/soft/redis/redis60/src/redis-server /usr/local/soft/redis/7001/redis.conf/usr/local/soft/redis/redis60/src/redis-server /usr/local/soft/redis/7002/redis.conf/usr/local/soft/redis/redis60/src/redis-server /usr/local/soft/redis/7003/redis.conf/usr/local/soft/redis/redis60/src/redis-server /usr/local/soft/redis/8001/redis.conf/usr/local/soft/redis/redis60/src/redis-server /usr/local/soft/redis/8002/redis.conf/usr/local/soft/redis/redis60/src/redis-server /usr/local/soft/redis/8003/redis.conf

After execution , You can see 6 individual redis The instance has been opened

Create clusters

Through the above steps , It opens. 6 individual redis Example , But these instances do not form a cluster , Next, you need to execute relevant commands to form a cluster

1、 Use the following command to build the cluster （5.0 Later versions of the command ）

redis-cli --cluster create --cluster-replicas 1 IP Address :7001 IP Address :7002 IP Address :7003 IP Address :8001 IP Address :8002 IP Address :8003

Description of the above command

redis-cli --cluster perhaps ./redis-trib.rb： Represents the cluster operation command ;

create： The representative is to create a cluster ;

–replicas 1 perhaps –cluster-replicas 1 ： Specify each node in the cluster master The number of copies of is 1, Total number of nodes at this time ÷ (replicas + 1) What you get is master The number of . Therefore, the top in the node list n One is master, Other nodes are slave node , Randomly assigned to different master ;

It can also be done through redis-cli --cluster help Command to view cluster commands ;

After executing the command , The console prompts which example will be used as master, What do you do slave;

Continue to input yes after , A cluster will be created according to the above strategy

From this output log, you can get the following information ：

The port numbers are 7001,7002 and 7003 For the master node ;

The port numbers are 8001,8002 and 8003 For the slave node ;

The port numbers are 7001,7002 and 7003 Of master node , The slots in a certain range are allocated respectively , Namely ：[0-5460],[5461-10922],[10923-16383], I.e. total allocation 16384 Slots ;

2、 View the cluster status

redis-cli -p 7001 cluster nodes

By this order , You can clearly see the master-slave status of each instance node in the cluster , example ID（ Unique identification ）, Slot interval and other information

Redis Hash slot description

In fact the redis Students who know a little about the cluster should know ,redis Partitioned cluster logically forms a complete memory space for all nodes in the cluster , After the data is written in , To which node ？ So the cluster introduces a logical slot concept , Divide the cluster into 16384 Slots , Each node in the cluster occupies part of the slots （ You can see from this log ）;

So when a specific key When writing , The cluster will pass certain algorithms , Route the data to be written to the specified slot ;

Here's a point to note , data key Not bound to a node , Instead, it is bound to the slot .

redis The cluster will be based on key Calculates the slot value for the valid part of the , There are two situations ：

key Contained in the "{}", And “{}” It contains at least 1 Characters ,“{}” The part in is the effective part ;

key Contains no “{}”, Whole key They're all effective parts ;

for instance ：key yes num, So based on num Calculation , If it is { apply name }num, According to “ apply name ” Calculation . The calculation method is to use CRC16 The algorithm gets a hash value , Then on 16384 Remainder , The result is slot value

Let's take a look at the effect

From the above demonstration, we can find that , After cluster computing key Will be assigned to different slots , in other words ,key Is bound to the slot , Instead of binding to a node , Think about why ？

Consider the following requirements

How to keep the same kind of data in the same Redis example ？

Simply speaking , If key There are no rules for the distribution of , When the data of a business class is involved, it corresponds to key When randomly assigned to different nodes , When the value is taken, the redirection cross node problem like the above occurs , Improve performance to some extent ;

terms of settlement

This type of data uses the same valid part , for example key Are subject to { Business ID} The prefix

Cluster scaling （ Add a node ）

reids Of cluster In mode , A strong function is the scalability of the cluster , That is, based on the existing cluster , According to the actual business needs , Perform cluster scaling , Here are the steps to add a new node to the above cluster ;

1、 Copy a directory under the current directory

2、 Modify the port of the configuration file

sed -i s/6379/7004/g 7004/redis.conf

3、 Start this instance

/usr/local/soft/redis/redis60/src/redis-server /usr/local/soft/redis/7004/redis.conf

4、 Use the following command to 7004 The instance is a cluster

redis-cli --cluster add-node IP:7004 IP:7001

5、 Check the cluster status again

redis-cli -p 7001 cluster nodes

Through the above status information, it is found that ,7004 Although this node has joined the cluster , And became master, But the cluster does not assign any slots to it

6、 Assign slot

The basic idea of allocating slots is , You can migrate some slots from an existing node to a new node , Execute the following command to allocate slots

redis-cli --cluster reshard 101.34.23.80:7001

After executing the command, the following prompt will appear , Enter the number of slots you want to transfer here

take 7004 The corresponding example ID Just type in

Input yes Start to move the slot

After moving , Check the cluster status again , Now 7004 The corresponding node is assigned to the slave node 0 ~ 500 Number of slots

Fail over

redis Of cluster Cluster of patterns , It also has some failover capability , For example, in the above architecture mode , When someone in the cluster master After the node goes down , What will happen ？ Let's simulate this process , See how it works

1、 Use watch Command to monitor the status of the cluster

With this command, you can view the dynamic log changes of the cluster in real time

watch redis-cli -p 7001 cluster nodes

2、 Manual will 7002 Example of master Downtime

redis-cli -p 7002 shutdown

From the above monitoring log, it is not difficult to find , When 7002 After hanging up , After a while with the cluster and its slave nodes 8003 Lost contact , then 8003 This node is upgraded to master node ;

3、 Manual will 7002 The instance starts

/usr/local/soft/redis/redis60/src/redis-server /usr/local/soft/redis/7002/redis.conf

Analyze the log again , You can see this time 7002 Only with slave Joined the cluster as , In fact, in some cases , We still hope that the down node will remain the primary node after recovery , What should I do ？ This involves manual failover , The main operation steps are as follows ：

Use redis-cli Connect 7002 node ;

perform cluster failover command ;

After the above execution ,7002 It's a slave node

perform CLUSTER FAILOVER command , Observe the dynamic changes of the log , Change through log , You can see , After the command is executed ,7002 It soon became master, and 8003 a slave;

Through the above steps redis Of cluster The process of failover in mode

Use redistemplate Access the partitioned cluster

1、 Introduce dependencies

<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-redis</artifactId></dependency>

2、 Change the cluster address in the configuration file to the following

spring: redis: cluster: nodes: - colony IP:7001 - colony IP:7002 - colony IP:7003 - colony IP:7004 - colony IP:8001 - colony IP:8002 - colony IP:8003

As for the specific code part , You can refer to the following test case

@Autowired private RedisTemplate<String,String> redisTemplate; //localhost:8083/set?key=b&value=123 @GetMapping("/set") public void set(@RequestParam String key,@RequestParam String value){ redisTemplate.opsForValue().set(key,value); } //localhost:8083/get?key=b @GetMapping("/get") public String get(@RequestParam String key){ return redisTemplate.opsForValue().get(key); }

This is about redis  This is the end of the article on building and using partitioned clusters , More about redis  Please search the previous articles of SDN or continue to browse the related articles below. I hope you can support SDN more in the future ！