Redis getting started complete tutorial: client management

Redis Provides client related information API Monitor and manage its status , This section will cover
each API And the problems that may be encountered in the development, operation and maintenance .

4.4.1　 client API
1.client list
client list Commands can list and Redis All client connection information connected to the server , For example, next
The face code is in a Redis Execute on instance client list Result ：
127.0.0.1:6379> client list
id=254487 addr=10.2.xx.234:60240 fd=1311 name= age=8888581 idle=8888581 flags=N
db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
id=300210 addr=10.2.xx.215:61972 fd=3342 name= age=8054103 idle=8054103 flags=N
db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
id=5448879 addr=10.16.xx.105:51157 fd=233 name= age=411281 idle=331077 flags=N
db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ttl
id=2232080 addr=10.16.xx.55:32886 fd=946 name= age=603382 idle=331060 flags=N
db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
id=7125108 addr=10.10.xx.103:33403 fd=139 name= age=241 idle=1 flags=N db=0
sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=del
id=7125109 addr=10.10.xx.101:58658 fd=140 name= age=241 idle=1 flags=N db=0
sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=del
...
Each line of the output result represents the information of a client , You can see that each line contains more than a dozen
attribute , They are some execution states of each client , Understand these attributes for Redis Development of
And operation and maintenance personnel are very helpful . Here are some important properties to illustrate , The rest through the table
Display in the form of lattice .
（1） identification ：id、addr、fd、name
These four attributes belong to the identity of the client ：
·id： Unique identity of the client connection , This id With Redis The connection of is self increasing , restart
Redis It will be reset to 0.
·addr： Client connected ip And port .
·fd：socket File descriptor for , And lsof In the command result fd Is the same , If fd=-1
Represents that the current client is not an external client , It is Redis Internal camouflage client .
·name： The name of the client , hinder client setName and client getName Two commands
It will be explained .
（2） Input buffer ：qbuf、qbuf-free
Redis An input buffer is assigned to each client , Its function is to send the life sent by the client
Order temporary storage , meanwhile Redis Pull command from input buffer and execute , Input buffer for guest
The client sends a command to Redis Executing commands provides buffering , Pictured 4-5 Shown .
client list in qbuf and qbuf-free Represents the total capacity and remaining capacity of the buffer respectively ,
Redis There is no configuration to specify the size of each buffer , The input buffer will be based on the input
Dynamic adjustment of input content size , It only requires that the size of each client buffer should not exceed
1G, After that, the client will be shut down . Here is Redis Hard coding of the input buffer in the source code
code ：

chart 4-5 Input buffer basic model

 

/* Protocol and I/O related defines */
#define REDIS_MAX_QUERYBUF_LEN (1024*1024*1024) /* 1GB max query buffer. */
Improper use of input buffers can cause two problems ：
· Once a client's input buffer exceeds 1G, The client will be shut down .
· The input buffer is not affected by maxmemory control , Suppose a Redis The instance sets
maxmemory by 4G, It's already stored 2G data , But if the input buffer is used at this time
3G, Has more than maxmemory Limit , There may be data loss 、 Key value elimination 、OOM etc.
situation （ Pictured 4-6 Shown ）.

The implementation effect is as follows ：

127.0.0.1:6390> info memory
# Memory
used_memory_human:5.00G
...
maxmemory_human:4.00G
....
It has been seen above , The harm caused by improper use of input buffer is very great , Then cause input
What are the reasons why the buffer is too large ？ The input buffer is too large mainly because Redis The processing speed of
The input speed of the upper input buffer , And every command that goes into the input buffer contains a lot of
bigkey, This causes the input buffer to be too large . Another situation is Redis It happened.
Blocking , Can't handle orders in the short term , This causes a backlog of client input commands in the input buffer ,
The input buffer is too large .
So how to quickly discover and monitor ？ There are two ways to monitor input buffer exceptions ：
· Through regular implementation client list command , collect qbuf and qbuf-free Abnormal connection record found
And analyze , Finally, find the client that may have problems .
· adopt info Ordered info clients modular , Find the largest input buffer , For example, the following life
One of the orders client_biggest_input_buf Represents the largest input buffer , For example, you can set super
too 10M Call the police ：
127.0.0.1:6379> info clients
# Clients
connected_clients:1414
client_longest_output_list:0
client_biggest_input_buf:2097152
blocked_clients:0
These two methods have their own advantages and disadvantages , surface 4-3 The two methods are compared .

surface 4-3 contrast client list and info clients Monitor the strengths and weaknesses of the input buffer

 

 

Operation and maintenance tips
The probability of input buffer problems is relatively low , But we should also take precautions , Reduce... In development
bigkey、 Reduce Redis Blocking 、 Reasonable monitoring and alarm .
（3） Output buffer ：obl、oll、omem
Redis Each client is allocated an output buffer , Its function is to save the result of command execution
The result is returned to the client , by Redis Interact with the client and provide a buffer for the returned results , Pictured 4-7 Shown .
Unlike the input buffer , The capacity of the output buffer can be determined by the parameter client-output-
buffer-limit To set it up , And the output buffer is more detailed , According to the different clients
Divided into three ： Normal client 、 Publish and subscribe client 、slave client , Pictured 4-8 Shown .

chart 4-7 Client output buffer model

 

 

The corresponding configuration rule is ：
client-output-buffer-limit <class> <hard limit> <soft limit> <soft seconds>
·<class>： Client type , Divided into three .a）normal： Normal client ;b）
slave：slave client , Used to copy ;c）pubsub： Publish and subscribe client .
·<hard limit>： If the output buffer used by the client is greater than <hard limit>, client
Will be shut down immediately .
·<soft limit> and <soft seconds>： If the output buffer used by the client exceeds <soft
limit> And it went on <soft limit> second , The client will be shut down immediately .
Redis The default configuration for is ：
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
The same as the input buffer is , The output buffer will not be affected either maxmemory The limitation of , Such as
Improper use will also cause maxmemory Data loss caused by full use 、 Key value elimination 、OOM etc.
situation .
In fact, the output buffer consists of two parts ： Fixed buffer （16KB） And dynamic buffering
District , The fixed buffer returns a relatively small execution result , The dynamic buffer returns a larger knot
fruit , For example, large strings 、hgetall、smembers The result of the order, etc , adopt Redis Source code
redis.h Of redisClient Structure （Redis3.2 Version changed to Client） You can see two buffers
Implementation details ：
typedef struct redisClient {
//   Dynamic buffer list
list *reply;
//   The length of the dynamic buffer list ( Number of objects )
unsigned long reply_bytes;
//   Fixed number of bytes that the buffer has been used
int bufpos;
//   Byte array as fixed buffer
char buf[REDIS_REPLY_CHUNK_BYTES];
} redisClient;
Fixed buffers use byte arrays , Dynamic buffers use lists . When fixed buffer
When the area is full, it will Redis The new returned results are stored in the queue of the dynamic buffer , Every in the queue
An object is each returned result , Pictured 4-9 Shown .

 

client list Medium obl Represents the length of the fixed buffer ,oll Represents the length of the dynamic buffer list
degree ,omem Represents the number of bytes used . For example, the following represents the length of the fixed buffer of the current client
Degree is 0, Dynamic buffers have 4869 Objects , The two parts are used together 133081288 byte =126M
Memory ：
id=7 addr=127.0.0.1:56358 fd=6 name= age=91 idle=0 flags=O db=0 sub=0 psub=0 multi=-1
qbuf=0 qbuf-free=0 obl=0 oll=4869 omem=133081288 events=rw cmd=monitor
There are still two ways to monitor the output buffer ：
· Through regular implementation client list command , collect obl、oll、omem Abnormal connection record found
And analyze , Finally, find the client that may have problems .
· adopt info Ordered info clients modular , Find the maximum number of objects in the output buffer list , example
Such as ：
127.0.0.1:6379> info clients
# Clients
connected_clients:502
client_longest_output_list:4869
client_biggest_input_buf:0
blocked_clients:0
among ,client_longest_output_list Represents the maximum number of objects in the output buffer list , These two
The advantages and disadvantages of these statistical methods are the same as the input buffer , I won't go into that here . Compared with losing
Into the buffer , The probability of an exception in the output buffer is relatively high , So how to prevent ？ Fang
The method is as follows ：
· Do the above monitoring , Set the threshold , If it exceeds the threshold, it should be handled in time .
· Limit the size of the normal client output buffer , Nip mistakes in the bud , For example, you can
The following settings ：
client-output-buffer-limit normal 20mb 10mb 120
· Increase appropriately slave Of the output buffer , If master Node writes larger ,slave Customer
The output buffer at the end may be large , once slave The client connected because the output buffer overflowed
By kill, Will cause replication reconnection .
· Limit commands that easily increase the output buffer , for example , High and sent monitor Orders are
It's a dangerous order .
· Monitor memory in time , Once memory jitter is found frequently , Maybe the output buffer is too large .
（4） The survival state of the client
client list Medium age and idle Represents the time when the current client has been connected and the latest
In my spare time ：

id=2232080 addr=10.16.xx.55:32886 fd=946 name= age=603382 idle=331060 flags=N db=0
sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
For example, the above record represents the current client connection Redis At 603382 second , among
Free 331060 second ：
id=254487 addr=10.2.xx.234:60240 fd=1311 name= age=8888581 idle=8888581 flags=N db=0
sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
For example, the above record represents the current client connection Redis At 8888581 second , among
Free 8888581 second , In fact, this is not a normal situation , When age be equal to idle when ,
Indicates that the connection is always idle .
For a more intuitive description age and idle, Here's an example ：
String key = "hello";
// 1)   Generate jedis , And implement get operation
Jedis jedis = new Jedis("127.0.0.1", 6379);
System.out.println(jedis.get(key));
// 2)   rest 10 second
TimeUnit.SECONDS.sleep(10);
// 3)   Perform a new operation ping
System.out.println(jedis.ping());
// 4)   rest 5 second
TimeUnit.SECONDS.sleep(5);
// 5)   close jedis Connect
jedis.close();
Here's an analysis of each step in the code , use client list Command to observe age and idle Parameters
Corresponding changes in .

Be careful
In order to redis-cli Different clients , This test client IP Address ：10.7.40.98.
1） Before executing the code ,client list There's only one client , It's the current redis-
cli, In order to save space, ignore this client .
288
127.0.0.1:6379> client list
id=45 addr=127.0.0.1:55171 fd=6 name= age=2 idle=0 flags=N db=0 sub=0 psub=0
multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=client
2） Use Jedis A new connection has been generated , And implement get operation , You can see IP The address is
10.7.40.98 The client of , The last order to execute is get,age and idle Namely 1 The second and 0 second ：
127.0.0.1:6379> client list
id=46 addr=10.7.40.98:62908 fd=7 name= age=1 idle=0 flags=N db=0 sub=0 psub=0
multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
3） rest 10 second , here Jedis The client is not shut down , therefore age and idle All the time
increase ：
127.0.0.1:6379> client list
id=46 addr=10.7.40.98:62908 fd=7 name= age=9 idle=9 flags=N db=0 sub=0 psub=0
multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
4） Perform a new operation ping, After discovery and execution age It's still increasing , and idle from 0 Calculation , also
It's no longer idle ：
127.0.0.1:6379> client list
id=46 addr=10.7.40.98:62908 fd=7 name= age=11 idle=0 flags=N db=0 sub=0 psub=0
multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
5） rest 5 second , Observe age and idle increase ：
127.0.0.1:6379> client list
id=46 addr=10.7.40.98:62908 fd=7 name= age=15 idle=5 flags=N db=0 sub=0 psub=0
multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=ping
6） close Jedis,Jedis The connection has disappeared ：
redis-cli client list | grep "10.7.40.98 ” It's empty

（5） Client limitations maxclients and timeout
Redis Provides maxclients Parameter to limit the maximum number of client connections , Once the number of connections exceeds
maxclients, New connections will be rejected .maxclients The default value is 10000, Can pass info
clients To query the current Redis Connection number ：
127.0.0.1:6379> info clients
# Clients
connected_clients:1414
...
Can pass config set maxclients Set the maximum number of client connections dynamically ：
127.0.0.1:6379> config get maxclients
1) "maxclients"
2) "10000"
127.0.0.1:6379> config set maxclients 50
OK
127.0.0.1:6379> config get maxclients
1) "maxclients"
2) "50"
Generally speaking maxclients=10000 It's absolutely enough for most scenarios , But some feelings
Due to improper use by the business party （ For example, not actively closing the connection ） There could be a lot of idle Connect ,
Whether it's the cost of connecting from the Internet or more than maxclients It's not a good thing in terms of the consequences ,
therefore Redis Provides timeout（ The unit is in seconds ） Parameter to limit the maximum idle time of the connection , One
It's client connected idle Time has passed timeout, The connection will be closed , For example setting timeout
by 30 second ：
#Redis default timeout yes 0 , That is to say, the client's idleness will not be detected
127.0.0.1:6379> config set timeout 30
OK
Continue to use Jedis To simulate , The whole code is the same as above , Only the 2）
The steps are off 31 second ：

String key = "hello";
// 1)   Generate  jedis , And implement  get  operation 
Jedis jedis = new Jedis("127.0.0.1", 6379);
System.out.println(jedis.get(key));
// 2)   rest  31  second 
TimeUnit.SECONDS.sleep(31);
// 3)   perform  get  operation 
System.out.println(jedis.get(key));
// 4)   rest  5  second 
TimeUnit.SECONDS.sleep(5);
// 5)   close  jedis  Connect 
jedis.close();

By executing the above code, you can find that at the end of the execution, the 2） After step ,client list No more
Jedis The connection of , in other words timeout Has to take effect , Will be more than 30 Second idle connection closed ：
127.0.0.1:6379> client list
id=16 addr=10.7.40.98:63892 fd=6 name= age=19 idle=19 flags=N db=0 sub=0 psub=0
multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
#   exceed timeout after , Jedis The connection is closed
redis-cli client list | grep  “ 10.7.40.98 ” It's empty
And you can see that , stay Jedis In the code 3） Step 1 throws an exception , Because at this point the client
It's been shut down , So the exception thrown is JedisConnectionException, And prompt
Unexpected end of stream：
stream ：
world
Exception in thread "main" redis.clients.jedis.exceptions.JedisConnectionException:
Unexpected end of stream.
If you will Redis Of loglevel Set to debug Level , You can see the following log , It's the guest
The client is Redis Closed logs ：
12885:M 26 Aug 08:46:40.085 - Closing idle client
Redis Source code redis.c In file clientsCronHandleTimeout Function is for timeout
Parameters for testing , Just in the source code timeout It's assigned to server.maxidletime：

int clientsCronHandleTimeout(redisClient *c) {
//   current time 
time_t now = server.unixtime;
// server.maxidletime  Is the parameter  timeout
if (server.maxidletime &&
//   A lot of client side validation , There's no space here , The most important verification is that the following idle time exceeds  maxidletime  will 
//   The client was shut down 
(now - c->lastinteraction > server.maxidletime))
{
redisLog(REDIS_VERBOSE,"Closing idle client");
//   Close client 
freeClient(c);
}
}

Redis The default configuration of timeout=0, In this case, the client will not appear
The above exception , This is based on a protection of client development . For example, many developers are using
JedisPool Idle detection and validation will not be performed on connection pool objects , If set timeout>0, can
If you can, the above exception will appear , It has a certain impact on the application business , But if Redis The customer
Improper use of client or some problems of client itself , The client connection is not released in time , can
Can cause a lot of idle Connections occupy a lot of connection resources , Once you surpass maxclients; Consequences also
It's unimaginable . In the actual development and operation and maintenance , Need to put timeout Set to greater than 0, for example
It can be set to 300 second , At the same time, add idle detection and verification measures on the client , example
Such as JedisPool Use common-pool Three properties provided ：minEvictableIdleTimeMillis、
testWhileIdle、timeBetweenEvictionRunsMillis,4.2 Section has been explained , here
I won't go into that .
（6） Client type
client list Medium flag Is the type used to identify the current client , for example flag=S On behalf of current customers
The client is slave client 、flag=N Represents the current normal client ,flag=O Represents the current client
Being implemented monitor command , surface 4-4 Lists 11 Two client types .

surface 4-4 Client type

 （7） other
It's already up there client list The important properties in , surface 4-5 List the previously introduced
And some relatively simple or less important properties .

2.client setName and client getName
client setName xx
client getName
client setName Used to set the name of the client , In this way, it is easier to identify the client
Source , For example, name the current client test_client, You can do the following ：
127.0.0.1:6379> client setName test_client
OK
Then execute client list command , You can see the current client's name The attribute is
test_client：
127.0.0.1:6379> client list
id=55 addr=127.0.0.1:55604 fd=7 name=test_client age=23 idle=0 flags=N db=0 sub=0
psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=client
If you want to view the current client's name, have access to client getName command , for example
The next operation ：
127.0.0.1:6379> client getName
"test_client"
client getName and setName Commands can be used as a way to identify the source of the client , but
Generally speaking , stay Redis In the case of only one application ,IP And port as identification
More clarity . When multiple applications use one Redis, So at this time client setName Sure
As a basis for identifying clients .
3.client kill
client kill ip:port
This command is used to kill the specified IP Address and port of the client , For example, the current client list is ：
127.0.0.1:6379> client list
id=49 addr=127.0.0.1:55593 fd=6 name= age=9 idle=0 flags=N db=0 sub=0 psub=0
multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=client
id=50 addr=127.0.0.1:52343 fd=7 name= age=4 idle=4 flags=N db=0 sub=0 psub=0
multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=get
If you want to kill 127.0.0.1：52343 The client of , It can be executed ：
127.0.0.1:6379> client kill 127.0.0.1:52343
OK
After executing the command ,client list It turns out that all that's left is 127.0.0.1：55593 This client ：
127.0.0.1:6379> client list
id=49 addr=127.0.0.1:55593 fd=6 name= age=9 idle=0 flags=N db=0 sub=0 psub=0
multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=client
For some reason （ For example setting timeout=0 It's a long time idle The client of ）, Need to be
To manually kill the client connection , have access to client kill command .
4.client pause
client pause timeout( millisecond )
Pictured 4-10 Shown ,client pause The command is used to block the client timeout Number of milliseconds , In this period
The connection between clients will be blocked . 

For example, execute on a client ：
127.0.0.1:6379> client pause 10000
OK
Execute... On another client ping command , Discover the whole ping The order was executed 9.72 second （ Manual
perform redis-cli, Just to demonstrate , It doesn't mean real execution time ）：
127.0.0.1:6379> ping
PONG
(9.72s)
This command can work in the following scenarios ：

·client pause Only valid for normal and publish subscribe clients , For master-slave replication （ From the node
A client is camouflaged inside ） It's invalid , That is to say, master-slave replication is normal during this period ,
So this command can be used to make master-slave replication consistent .
·client pause You can connect clients from a Redis The node to switch
To another Redis node .
It's important to note that in a production environment , The cost of suspending clients is very high .
5.monitor
monitor The command is used to monitor Redis The order being executed , Pictured 4-11 Shown , We turn on
Two redis-cli, An executive set get ping command , Another implementation monitor command . You can see
To monitor Commands can listen to commands being executed by other clients , And recorded the detailed time
stamp . 

monitor It's obvious that , If developers and operators want to listen Redis The life being executed
Make , You can use it monitor command , But it's not so good , Each client has its own
Output buffer , since monitor Can listen to all the commands , once Redis Too much concurrency ,
monitor The client's output buffer will skyrocket , It can take up a lot of memory in an instant , chart 4-12 It shows
monitor Commands cause a lot of memory usage . 

4.4.2　 Client related configuration
4.4.1 Section has introduced some of the configuration of the client , This section will introduce the remaining configurations
Shao ：
·timeout： Timeout for detecting client idle connections , once idle Time has come to
timeout, The client will be shut down , If set to 0 No testing .
·maxclients： Maximum number of client connections ,4.4.1 The client survival status section in section has
Analyze , No more details here , But this parameter is limited by the operating system settings , The first 12
Chapter Linux This parameter will also be introduced in the relevant configuration section .
·tcp-keepalive： testing TCP The cycle of linking activity , The default value is 0, That is to say, not to carry out
testing , If you need to set , The suggestion is 60, that Redis Every time 60 Seconds created for it TCP even
Then carry out activity test , Prevent a large number of dead connections from occupying system resources .
·tcp-backlog：TCP After three handshakes , The accepted connections are queued ,tcp-
backlog It's the size of the queue , It's in Redis The default value in is 511. Generally speaking, this parameter is not
Need to adjust , But this parameter is affected by the operating system , For example, in Linux operating system
in , If /proc/sys/net/core/somaxconn Less than tcp-backlog, So in Redis It starts up with
See the following log , And suggest that /proc/sys/net/core/somaxconn Set it bigger .
# WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/
sys/net/core/somaxconn is set to the lower value of 128.
The modification method is also very simple , Just execute the following command ：
echo 511 > /proc/sys/net/core/somaxconn

4.4.3　 Client statistics fragment
For example, here's one info clients The results of the implementation of ：
127.0.0.1:6379> info clients
# Clients
connected_clients:1414
client_longest_output_list:0
client_biggest_input_buf:2097152
blocked_clients:0
The explanation is as follows ：
1）connected_clients： On behalf of the current Redis The number of client connections to the node , Key supervision is needed
control , Once you surpass maxclients, New client connections will be rejected .
2）client_longest_output_list： The current maximum number of queued objects in all output buffers
Great value .
3）client_biggest_input_buf： The maximum capacity currently occupied in all input buffers .
4）blocked_clients： Executing blocking command （ for example blpop、brpop、
brpoplpush） The number of clients .
besides info stats It also contains two statistical indicators related to the client , as follows ：
127.0.0.1:6379> info stats
# Stats
total_connections_received:80
...
rejected_connections:0
Parameter description ：
·total_connections_received：Redis The total number of client connections processed since startup
Count .
·rejected_connections：Redis Number of client connections rejected since startup , Need to focus
monitor .