Redis principle and usage - installation and distributed configuration

Prior to blog in , Introduced redis Distributed lock implementation . In the work of the schedule ,redis It is also used very frequently , Including the cache , Distributed lock , Message queuing, etc . This series will systematically address redis Make a complete introduction and summary .

---

Redis Service installation and remote access

Redis There are many online tutorials , I won't introduce it here .

After installation, you can use redis-server start-up , Through the client redis-cli Visit , By default 6379 port （ The environment of this paper is ubuntu+redis3.0.6).

If you want to access remotely redis The server , You need to specify redis.conf The configuration file , And modify it redis Profile parameters . Comment out the following line ：
# bind 127.0.0.1
If you don't comment , An error will be reported when accessing remotely ：
Could not connect to Redis at ip:6379: Connection refused.
Thus, it can only be accessed locally .

Comment out binding ip after , adopt ps -ef | grep redis Find service process , adopt kill -9 [pid] Kill the service process and restart the service （ If the daemon is turned on , Then it will restart after killing , It can be done by /etc/init.d/redis-server stop To stop ）. At this time, it can be accessed remotely redis The server .
It can be done by info Order to see redis Server related information , As shown in the figure below ：

Input config get requirepass You can see that the result is :

here ,redis The server has no password . There are potential safety hazards in this situation , That is, unauthorized login . Access to internal data without authentication , Can lead to the disclosure of sensitive information , It can also be executed maliciously flushall To clear all data .
So in order to ensure safety , The login password is usually set ：
adopt config set requirepass [password] You can set a temporary password . After the setting is successful, any operation will prompt ：

(error) NOAUTH Authentication required.

It can be done by auth [password] authentication .
But this setting method is invalid after the server restarts , To keep password authentication valid , It also needs to be modified redis.conf The configuration file ：

And restart the service .

Redis colony

Redis There are three cluster models , They are master-slave mode , sentry (Sentinel) Patterns and clusters (Cluster) Pattern .

A master-slave mode （Master-Slave）

The master-slave mode is relatively simple , Follow mysql The master-slave mode of is similar , It consists of a master node and multiple slave nodes . The master node provides write capability , Provide read capability from the node . The problem with master-slave mode is ：master Nodes are unique in master-slave mode , if master Hang up , be redis Unable to provide external write Services .

Deploy

Master node ip：10.241.149.44, port 6379.
From the node ip： This machine . Ports are 6381 and 6382.

Existing local directory /usr/local/redis-node/6381 and /usr/local/redis-node/6382. take redis.conf Copy to these two directories respectively . modify 6381 Profile under ：

port 6381
...
masterauth xxx #  The master node has opened the password , You need to configure . See above 
...
slaveof 10.241.149.44 6379  #  Configure master-slave mode 

Modify similarly 6382 Profile under .

Last , Start two slave nodes ：

redis-server /usr/local/redis-node/6381/redis.conf
redis-server /usr/local/redis-node/6382/redis.conf

Connect to the master node ：
redis-cli -h 10.241.149.44 -p 6379
write in ：10.241.149.44:6379> set user howe success
Connect the slave node ：redis-cli -p 6381
Read user：

127.0.0.1:6381> get user
"howe"

It indicates that the data on the master node is synchronized to the slave node .
If you are writing on the slave node , Will prompt ：
(error) READONLY You can't write against a read only slave.

Sentinel mode (Sentinel)

The disadvantage of the master-slave model is that it does not have high availability , When master After hanging up ,Redis You will no longer be able to provide external write operations , therefore sentinel emerge as the times require .

Sentinel The Chinese meaning is sentry , seeing the name of a thing one thinks of its function , Its function is to monitor redis The health of the cluster . When the master node is found dead , It will elect a slave node to be upgraded to the master node , And let other slave nodes synchronize data from the new master node .

sentinel Because it is also a process , It's possible to hang up , therefore sentinel It will also start multiple to form one sentinel colony .

When using sentinel In mode , Do not connect the client directly Redis, It's about connecting sentinel, from sentinel To provide specific services Redis Realization , So when master After the node hangs up ,sentinel It's going to perceive and bring new master Nodes are provided to users .

Deploy

On the basis of the master-slave mode , You can further build sentinel mode .
redis The primary node is 10.241.149.44:6379, The two slave nodes are local ：10.234.209.24:6381, 10.234.209.24:6382.
Set the two slave nodes in the configuration file bind 127.0.0.1 Change to bind 0.0.0.0, So that remote clients can access .
meanwhile , In sentinel mode, the slave node may switch to the master node , Therefore, the slave node and the master node have the same status , The configuration needs to be consistent , Put two slave nodes requirepass open .

Suppose there is an existing local directory ：
/usr/local/redis-sentinel. Create... In this directory s1,s2,s3, And then sentinel.conf Copy to s1,s2,s3 Next .Sentinel.conf The initial content is as follows ：

port 26379
dir /tmp
# sentinel monitor <master-name> <ip> <redis-port> <quorum>
#  Primary node alias ,ip, port , confirm master Number of sentinels that failed 
sentinel monitor mymaster 127.0.0.1 6379 2
sentinel down-after-milliseconds mymaster 30000
sentinel parallel-syncs mymaster 1
sentinel failover-timeout mymaster 180000

modify s1 Of sentinel The configuration file ：
add to ：

daemonize yes
logfile “/usr/local/redis-sentinel/s1.log” 

Modify the listening master node to ：

sentinel monitor mymaster 10.241.149.44 6379 2

If the master node has the password enabled , You also need to add password authentication ：

sentinel auth-pass mymaster <password>

The remaining parameters remain unchanged .
Modify... In the same way s2,s3 Of sentinel The configuration file .

Start three sentinels by specifying the configuration file ：

redis-sentinel /usr/local/redis-sentinel/s1/sentinel.conf
redis-sentinel /usr/local/redis-sentinel/s2/sentinel.conf
redis-sentinel /usr/local/redis-sentinel/s3/sentinel.conf

Here, put three sentinels on the same machine , In fact, in order to avoid a single point of failure , Usually sentinels are deployed on different machines .

Now? kill Drop the primary node redis Service process . open s1.log, You can see the failover ：

The explanation is as follows ：

*****s1 The sentinel port of the node is 26379, Listen to the other two sentinel nodes 26380 and 26381, Listen to the master node 10.241.149.44 6379*****
10343:X 23 Apr 16:08:38.511 * +sentinel sentinel 10.234.209.24:26380 10.234.209.24 26380 @ mymaster 10.241.149.44 6379
10343:X 23 Apr 16:08:41.549 * +sentinel sentinel 10.234.209.24:26381 10.234.209.24 26381 @ mymaster 10.241.149.44 6379
***** Listen to the redis The master node is offline *****
10343:X 23 Apr 16:10:40.093 # +sdown master mymaster 10.241.149.44 6379
***** Elect a new redis Master node *****
10343:X 23 Apr 16:10:40.236 # +new-epoch 1
10343:X 23 Apr 16:10:40.242 # +vote-for-leader 9528d63b865c3fb8773c8ac6cfb52cb0a702f3e5 1
***** Three sentinels voted , Satisfy 2 Conditions for confirmation , Think redis The master node is indeed offline *****
10343:X 23 Apr 16:10:41.186 # +odown master mymaster 10.241.149.44 6379 #quorum 3/2
10343:X 23 Apr 16:10:41.186 # Next failover delay: I will not start a failover before Thu Apr 23 16:16:40 2020
***** Update sentinel Profile *****
10343:X 23 Apr 16:10:41.337 # +config-update-from sentinel 10.234.209.24:26380 10.234.209.24 26380 @ mymaster 10.241.149.44 6379
***** Switch redis The primary node is 6382 node ( One of the original two slave nodes )*****
10343:X 23 Apr 16:10:41.338 # +switch-master mymaster 10.241.149.44 6379 10.234.209.24 6382
10343:X 23 Apr 16:10:41.338 * +slave slave 10.234.209.24:6381 10.234.209.24 6381 @ mymaster 10.234.209.24 6382
10343:X 23 Apr 16:10:41.338 * +slave slave 10.241.149.44:6379 10.241.149.44 6379 @ mymaster 10.234.209.24 6382
***** here , primary redis The master node is switched to the slave node , And it is offline *****
10343:X 23 Apr 16:11:11.397 # +sdown slave 10.241.149.44:6379 10.241.149.44 6379 @ mymaster 10.234.209.24 6382

see s2.log and s3.log, The general process is similar .

Now open s1 Node sentinel.conf Configuration files can find , The listening master node is automatically updated to ：

sentinel monitor mymaster 10.234.209.24 6382 2

s2 and s3 The configuration file of is also automatically updated .

Start the original master node 10.241.149.44:6379, Find out s1.log One more line of log ：

10343:X 23 Apr 16:55:53.487 # -sdown slave 10.241.149.44:6379 10.241.149.44 6379 @ mymaster 10.234.209.24 6382

This indicates that this node is going online again as a slave node .

sentinel operation
Connect any sentinel ：
redis-cli -p 26379

View master status ：

View the slave node status ：sentinel slaves mymaster

Code connection

In sentinel mode , Master and slave nodes are dynamic , Its state has sentinel group management . So the client connection redis Server time , You need to connect through sentinels .

The configuration is as follows ：

[redis]
host = 10.234.209.24:26379,10.234.209.24:26380,10.234.209.24:26381
pass = root
timeout = 100

Code example ：

import settings
from redis.sentinel import Sentinel

sentinels = []
host_list = settings.get('redis', 'host').split(",")
for host in host_list:
    ip = host.split(":")[0]
    port = host.split(":")[1]
    sentinels.append((ip, port))
sentinel = Sentinel(sentinels=sentinels)


service_name = "mymaster"

redis_master = sentinel.master_for(service_name=service_name, password=settings.get('redis', 'pass'))
redis_slave = sentinel.slave_for(service_name=service_name, password=settings.get('redis', 'pass'))

redis_master.set("user", "test")  #  The master node writes 
print redis_slave.get("user")  #  Read from node 

Slice cluster mode (Cluster)

Redis Of sentinel Although sentinel mode can be solved Redis A single point of failure , However, the structure of one master and many slaves will inevitably cause the client to write data only in Master Operation on node , When the write volume is particularly large, the pressure on the master node will be great .

Redis from 3.0 After version, it provides cluster Cluster pattern , That solved the problem . But cluster distribution is not necessarily good , The client performance of distributed clusters is generally lower ,flush、mget、keys And other commands cannot be used across nodes , The maintenance of the client is also more complex , Therefore, the business can meet the needs under the sentinel, and try to use the sentinel mode .

I won't introduce the fragment cluster mode in detail here , You can search online by yourself .

Next Will introduce redis Supported data structures , Persistence method and expiration policy .

Reference material

[1].https://www.jianshu.com/p/897dcc654613
[2].https://blog.csdn.net/truelove12358/article/details/79612954