Detailed explanation of redis

Statement ： This series is about redis My articles are all based on redis5.0.0. because redis In order to achieve better performance and better memory usage during the version iteration process, it will be continuously optimized , Even the data body of the underlying data structure has been greatly changed , and 5.0.0 The version changes are also relatively large , So the description of some technical points may be the same as what you are using now redis There are differences in the use of , I hope you can use it correctly ！

brief introduction

Redis It's a use ANSI C Compiling , Open source , Support network , Memory based , Optional persistent key value pair storage system , Known for outstanding performance , The official data can support 100000 The above QPS.

redis The advantages of

• redis It's a memory database
  redis Most of the command processing knowledge is pure memory operation , Therefore, the read-write speed is much faster than the disk type
• The working mode is single thread
  Actually a running redis Server There must be more than one thread , But there is only one thread to handle network requests , Avoid unnecessary context switching , At the same time, there is no lock / Release the lock and other synchronous operations . Other functions , Such as persistence 、 Delete asynchronously 、 Cluster data synchronization, etc , It is also executed by other threads
• redis The data structure in is specially designed
  redis The value in supports the storage of most data types , And it is binary safe . also redis The data structure of the has been specially designed , Make increase 、 Delete 、 Change 、 The operation of checking is relatively simple
• redis Using multiple channels I/O Reuse model , It can efficiently handle a large number of concurrent connections
• redis Support persistence
• Redis Support data persistence , May adopt RDB、AOF、RDB&AOF Three options . After the computer restarts, data can be recovered from the disk .
• redis Support master-slave structure
  redis Support master-slave structure , You can use to back up data from an instance .

The first four items are also redis The reasons for high performance .

Data structure and objects

as everyone knows ,redis Famous for its high performance , without doubt ,redis The high performance of the must have some special design 、 Easy to use and secure data structure as support , To meet the storage requirements in different scenarios . Now we are right redis Several commonly used data structures are explained in detail , You can see from it that redis How to use these data structures , The reasons for choosing these data instead of using similar data structures that already exist in the market and how these data structures guarantee redis High performance .

We know Redis yes key-value Form of database . The key in a key value pair can be a string 、 integer 、 Floating point, etc , And the key is unique . The type of value in a key value pair can be String、Hash、List、Set、SertedSet, These five objects are implemented with different underlying data structures .redis Provides six basic data structures ： Simple dynamic string 、 Set of integers 、 Compressed list 、 Skip list 、 Dictionaries 、quicklist.

Simple dynamic string （SDS）

Simple dynamic string （SDS） yes Redis One of the basic data types of , Used to store string and integer data . See ：
redis Dynamic string of （SDS）

Dictionaries

A dictionary is also called a hash table , Is used to store key values （key-value） A data structure for . however C There is no such data structure as dictionary in language ,Redis yes K-V Type database , The entire database uses dictionaries to store , So understand Redis How to implement Dictionary , What are the basic operations and applications of dictionaries , For us to fully understand Redis It's all important . For details, please see ：redis Chinese Dictionary

Skip list

Orderly collection is common in life , Such as ranking students according to their grades 、 Ranking players according to their scores, etc . For the underlying implementation of ordered sets , You can use arrays 、 Linked list 、 Balance tree, etc . But arrays are not convenient for inserting and deleting elements ; The query efficiency of the linked list is too low , You need to traverse all the elements ; Although the balanced tree or red black tree is efficient, its implementation is complex .Redis A new data structure is adopted ---- Skip list . The efficiency of jump table is comparable to that of red black tree , However, its realization is far simpler than that of red and black trees . See Redis Jump table

Set of integers

Tentatively , Subsequent complement

Compressed list

Tentatively , Subsequent complement

quicklist

Tentatively , Subsequent complement

Persistence

See redis It's persistent .

The principle of replication

See redis Master-slave replication .

event

Tentatively , Subsequent complement

Business

Tentatively , Subsequent complement

Memory obsolescence mechanism

See Redis Memory elimination mechanism

Expiration deletion policy

In understanding Redis Before the expiration of the delete policy , Let's get to know Redis How to save expiration time . stay Redis in ,redisDb Structural expires The dictionary holds the expiration time of all keys in the database , We call this dictionary Out of date Dictionary ：

• The key of an overdue dictionary is a pointer , This pointer points to a key object in key space （ That is, a database key ）.
• The value of the expired dictionary is a long,long Type integer , This integer holds the expiration time of the database key that the key points to ---- One millisecond precision UNIX Time stamp .

Three common deletion strategies ：

• Delete regularly
  While setting the key expiration time , Create a timer , Let the timer when the key expires , Delete the key immediately .
  advantage ： Memory is the most friendly , Because of the existence of timer , You can delete expired keys in a very timely manner , Free up memory .
  shortcoming ： Yes CPU Time is the most unfriendly , In the case of more expired keys, it will occupy a considerable part CPU Time , It may even hinder normal command execution , in addition , Creating a timer requires Redis Server time ,, The implementation of the current time event ---- Disordered list , Find the time complexity of an event O(N)---- It is not efficient to deal with a large number of time events . therefore , Must let Redis The server creates a lot of timers , So as to realize the scheduled deletion policy , It is not realistic at this stage .
• Lazy deletion
  Let the key expire , But every time you get a key from the key space , Check whether the obtained key is expired , If it's overdue , Delete the key ; If it doesn't expire , Just go back to the key .
  advantage ： Yes CPU Time is the friendliest : The program will only check the expiration of the key when the key is removed , This can ensure that the operation of deleting the expired key will only be carried out when it has to be done , And the target of deletion is limited to the currently processed key , This policy does not spend any time deleting other unrelated expired keys CPU Time .
  shortcoming ： Memory is the most unfriendly ： If there are too many expired keys in the database , And these expired keys are not accessed , Then they may never be deleted （ Unless the user manually performs FLUSHDB）.
• Delete periodically
  Every once in a while , Program on the database for a check , Delete the expiration key inside . As for how many expiration keys to delete , And how many databases to check , Then there is an algorithm to decide .
  This is a compromise between scheduled deletion and lazy deletion , Why do you say that ？ Therefore, regular checking of expired keys is a feature of scheduled deletion , But every time I don't check everything key, In other words, all expired items will not be deleted each time key, This is another characteristic of lazy deletion . The difficulty of periodic deletion strategy is to determine the duration and frequency of deletion operation .

Let's focus on Redis Expiration deletion policy in .Redis The server actually uses two strategies: lazy deletion and periodic deletion ： By using these two deletion strategies together , The server can be well used CPU Balance time and avoid wasting memory . The specific implementation is as follows ：

Redis Implementation of lazy deletion policy in

All read and write databases Redis Commands call... Before execution expireIfNeeded Function to check the input key ：

• If the input key has expired , that expireIfNeeded Function deletes the input key from the database .
• If the input key has not expired , that expireIfNeeded The function does not act .
  expireIfNeeded It's like a filter , It can be done before the command is actually executed , Filter out expired input keys , This prevents commands from touching expired keys .

Redis Implementation of periodic deletion policy in

whenever Redis The periodic operation of the server serverCron Function execution time ,activeExpireCvcle The function will be called , It's within the prescribed time , Traverse the databases in the server several times , From the database expires Randomly check the expiration time of some keys in the dictionary , And delete the expiration key .
activeExpireCvcle The working mode of the function can be summarized as follows ：

• Every time the function runs , Take a certain number of random keys from a certain number of databases for inspection , And delete the expiration key .
• Global variables current_db Will record the current activeExpireCvcle Function to check the progress , And next time activeExpireCvcle When a function is called , Then process the last progress .
• With activeExpireCvcle The continuous execution of functions , All databases in the server will be checked , Then the function will current_db The variable is reset to 0, Then start a new round of inspection again .

AOF、RDB And copy function for expired keys

Generate RDB file
In execution SAVE Order or BGSAVE Command to create a new RDB When you file , The program will check the key in the database , Keys that have expired will not be saved to newly created RDB file .
load RDB file
Start up Redis Server time , If the server is turned on RDB function , Then the server will respond to RDB File to load ：

• If the server is running in master mode , So before loading RDB When you file , The program will check the key saved in the file , Unexpired keys are loaded into the database , The expired key is ignored , So expired key pairs are loaded RDB The primary server of the file will not be affected .
• If the server is running in slave mode , So before loading RDB When you file , All keys saved in the file , Whether expired or not , Will be loaded into the database . however , If the master-slave synchronizes completely with the slave , The database from the server will be emptied , therefore , At this time, the expired key pair is loaded RDB File ears from the server will not cause any impact .

AOF File is written to
When the server is running in AOF Persistent mode runtime , If a key in the database has expired , But it hasn't been deleted lazily or regularly , that AOF The file will not be affected by this expiration key .
When the expired key is deleted lazily or periodically , The program will go to AOF Add a... To the document DEL command , To show that the key has been deleted .
AOF rewrite
And generation RDB Files are similar to , In execution AOF In the process of rewriting , The program will check the key in the database , Expired keys will not be saved to the rewritten AOF In file .
Copy
When the server is running in replication mode , Deletion of expiration keys from the server is controlled by the master server ：

• The master server deletes an expired key , It will send a message to all the slave servers DEL command , Tell to remove the expired key from the server .
• When the slave server executes the read command sent by the client , The expired key will not be deleted even if it is encountered , Instead, continue to process expired keys as if they were not expired .
• The slave server only receives a message from the master server DEL After the command , Will delete the expired key .

The master server controls the deletion of expired keys from the server , It can ensure the data consistency of the master and slave servers .

Sort

Tentatively , Subsequent complement

colony

Redis The cluster of
Redis Sentinel of

Redis common problem

Redis Frequently asked questions