Redis (7) -- database and expiration key

1, Database in server

Redis Corresponding concepts in ：
Simple dynamic string
Linked list
Dictionaries
Set of integers
Compressed list
object

Redis The server keeps all databases in the server state ：redisServer Structural db Array ,db Every element in an array is a redisDb structure , Each structure represents a database

struct redisServer{
    
    
    //...
    
    // An array , Save all databases in the service 
    redisDb *db;
    
    // Number of databases on the server 
    int dbnum;
    
    //...
    
};

dbnum The value of the property is configured by the server database The choice decides , The default is 16, So by default 16 A database

2, Switch database

Every Redis Clients all have their own target databases , Whenever you write a command on the client or execute a command in the database , The target database will become the operation object of these commands . By default , The target database of the client is 0 The database , Can pass SELECT Command switch database .

redis> SET msg "hello world"
OK

redis> GET msg
"hello world"

redis> SELECT 2
OK

redis[2]> GET msg
(nil) #  because 2 Database No  msg  This key ,msg It exists in 0 In database 

redis[2]> SET msg "another world"
OK

redis[2]> GET msg
"another msg"

Client status redisClient Structural db Property records the current target database of the client , This property is a point redisDb Pointer to structure .

typedef struct redisClent{
    
    
    //...
    // Record the database that the client is currently using 
    redisDb *db;
    
    //...
    
} redisClient;

redisClient.db Pointer to redisServer.db Elements in an array , This element represents the database currently being used by the client

For example, the current client is using 1 The database , that redisClient and redisServer The schematic diagram of the relationship is as follows ：

perform SELECT 2 after ：

By modifying the redisClient.db The pointer , Let it point to different elements in the array , So as to realize the function of switching target database , This is it. SELECT Command implementation principle .

3, Database key space

Redis Is a key value pair server , Each database consists of a redisDb Structural representation , among redisDb Structural dict All key value pairs in the dictionary , We call it bond space （key space）

typedef struct redisDb{
    
    
    //...
    
    // Database key space , Save all key value pairs in the database 
    dict *dict;
    
    //...
    
} redisDb;

The key space corresponds directly to the database seen by the user ：

• The key of the key space is also the key of the database , Each key is a string object
• The value of the key space is the value of the database , Each value can be a string object 、 List objects 、 Hash table object 、 Any of a collection object and an ordered collection object Redis object

If you execute the following command ：

redis> SET message "hello world"
OK

redis> RPUSH alphabet "a" "b" "c"
(integer) 3

redis> HSET book name "Redis in Action"
(integer) 1

redis> HSET book author "Josian L. Carlson"
(integer) 1

redis> HSET book publisher "Manning"
(integer) 1

After execution , The key space of the database will be like this ：【 It is worth noting that , there StringObject Represents a string object ,HashObject Represents a hash table object ,ListObject Represents a list object , These are to simplify the expression 】

• alphabet Is a list key , The name of the key is a string containing alphabet String object of , The value of the key is a list object containing three elements
• book Is a hash table key , The name of the key is a string containing book String object of , The value of the key is a hash table object containing three key value pairs
• message Is a string key , The name of the key is a string containing message String object of , The value of the key is a containing string hello world String object of

Because the key space of the database is a dictionary , So all operations against the database , In fact, it is realized by operating the key space dictionary .

3.1, Add new key

Add a new key , In fact, it is to add a new key value pair to the key space dictionary , Where the key is a string object , The value is of any type Redis object

For example, the key space diagram in the current situation is shown below ：

After adding a new key value pair to the key space ：

redis> SET date "2013.12.1"
OK

After adding the diagram ：

The key of this new key value pair is a string object , Value is a containing string “2013.12.1” String object of

3.2, Delete key

Delete a key in the database , In fact, it is to delete the key value pair object corresponding to the key in the key space .

Suppose the state of the key space is as shown in Figure 9-4 Shown , Execute the following command ：

redis> DEL book
(integer) 1

The key space state after executing the command is ：

3.3, Update key

Update a database key , In fact, it is to update the value object corresponding to the key in the key space , Depending on the type of value object , The specific method of updating will also be different

Suppose the state of the current key space is as shown in Figure 9-4 Shown , Execute the following command ：

redis> SET message "blah blah"
OK

After executing the command , The key space state should be ：

3.4, Value the key

Value a database key , In fact, it is to take out the value object corresponding to the key in the key space , Depending on the type of value object , The specific value taking methods will also be different

Suppose the state of the current key space is as shown in Figure 9-4 Shown , Execute the following command ：

redis> GET message
"hello world"

Then the value taking process will be like this ：

3.5, Maintenance operations when reading and writing key space

When performing corresponding read and write operations on the key space ,Redis Some additional maintenance operations will be performed , Include ：

• After reading a key （ Both read and write operations require read keys ）, The server will update the key space hit of the server according to whether the key exists （hit） Number of times or key space misses （miss） frequency
• After reading a key , The server will update the key LRU Time , This value can be used to calculate the idle time of the key
• If you find that this key has expired when reading , Then the server will delete the expired key first , Then do the rest
• If there is a client using WATCH Command to monitor a key , After the server modifies this key , Will mark this key as dirty （dirty）, This allows the transaction program to notice that the key has been modified
• After the server changes one key at a time , Will increase the value of the dirty key counter 1, This counter triggers persistence and replication operations on the server
• If the database turns on the database notification function , After modifying the key , The server will send the corresponding database notification according to the configuration

4, Set the key's lifetime or expiration time

4.1, Set expiration time

Redis There are four different commands that can be used to set the lifetime of the key （ How long can the key last ） Or expiration time （ When is the key deleted ）

• EXPIRE <key> <ttl>： The command is used to key key The lifetime of is set to ttl second
• PEXPIRE <key> <ttl>： The command is used to key key The lifetime of is set to ttl millisecond
• EXPIREAT <key> <timestamp>： The command is used to key key The expiration time for is set to timestamp The time stamp of the specified number of seconds
• PEXPIREAT <key> <timestamp>： The command is used to key key The expiration time for is set to timestamp The specified number of milliseconds timestamp

Although there are many different units and different forms of setting commands , But actually EXPIRE、PEXPIRE、EXPIREAT All three commands use PEXPIREAT Command implemented

# EXPIRE command 
def EXPIRE(key,ttl_in_sec):
	#  take TTL Convert from seconds to milliseconds 
	ttl_in_ms=sec_to_ms(ttl_in_sec)
        
    PEXPIRE(key,ttl_in_ms)

######################################
# PEXPIRE It can be converted into PEXPIREAT 
def PEXPIRE(key,ttl_in_ms):
	#  Gets the current value in milliseconds UNIX Time stamp 
	now_ms=get_current_unix_timestamp_in_ms()
        
    #  The current time plus TTL, Get the key expiration time in milliseconds 
    PEXPIREAT(key,now_ms+ttl_in_ms)
     
######################################
# EXPIREAT Commands can be converted to PEXPIREAT 
def EXPIREAT(key,expire_time_in_sec):
	#  Convert expiration time from seconds to milliseconds 
    expire_time_in_ms=sec_to_ms(expire_time_in_sec)
        
    PEXPIREAT(key,expire_time_in_ms)

4.2, Save expiration time

redisDb Structural expires The dictionary stores the expiration time of all keys in the database , We call this an expired dictionary

• The key of an overdue dictionary is a pointer , Point to a key object in the key space
• The value of the expired dictionary is a long long Type integer , This integer holds the expiration time of the database key that the key points to （ One millisecond precision UNIX Time stamp ）

typedef struct redisDb{
    
    
    //...
    
    // Out of date Dictionary , Save the expiration time of the key 
    dict *expires;
    
    //...
    
}redisDb;

It is worth mentioning that ： Keys in the out of date dictionary are key objects that point to the key space , chart 9-12 This expression is for the convenience of display

As mentioned earlier PEXPIREAT The pseudo code of the command is as follows ：

def PEXPIREAT(key,expire_time_in_ms):
	#  If the given key does not exist in the key space , Then you cannot set the expiration time 
	if key not in redisDb.dict:
		return 0
            
    #  Associate key and expiration time in expiration dictionary 
    redisDb.expires(key)=expire_time_in_ms
            
    #  Expiration time set successfully 
    return 1

4.3, Remove expiration time

PERSIST The command can remove the expiration time of a key

redis> PEXPIREAT message 1391234400000
(integer) 1

redis> TTL message
(integer) 13893281

redis> PERSIST message
(integer) 1

redis> TTL message
(integer) -1

PERSIST The order is PEXPIREAT Anti operation of ：PERSIST The command looks for the given key in the out of date dictionary , And disassociate the key and value in the expiration dictionary （ That is, the expiration time of the removal key ）

Assume that the current state of the database is as shown in Figure 9-12 Shown , After executing the following command ：

redis> PERSIST book
(integer) 1

PERSIST Pseudo definition of command ：

def PERSIST(key):
	# If the key doesn't exist , Or the key does not set the expiration time , Then go straight back 
	if key not in redisDb.expires:
		return 0
      
    # Removes the key value pair Association for the given key in the out of date dictionary 
    redisDb.expires.remove(key)
            
    # The expiration time of the key was removed successfully 
    return 1

4.4, Calculate and return the remaining survival time

• TTL： Return the remaining lifetime of the key in seconds
• PTTL： Returns the remaining lifetime of the key in milliseconds

PTTL Pseudo code

def PTTL(key):
	#  The key does not exist in the database 
	if key not in redisDb.dict:
		return -2
        
    #  Try to get the expiration time of the key 
    #  If the key doesn't set the expiration time , that expire_time_in_ms  Will be for none
    expire_time_in_ms=redisDb.expires.get(key)
            
    #  The key does not set an expiration time 
    if expire_time_in_ms is None:
		return -1
            
    #  Get the current time 
    now_ms=get_current_unix_timestamp_in_ms()
      
    #  The expiration time minus the current time , The difference is the remaining lifetime of the bond 
    return(expire_time_in_ms - now_ms)

TTL Pseudo code

def TTL(key):
	
	# Gets the remaining lifetime in milliseconds 
	ttl_in_ms=PTTL(key)
        
    if ttl_in_ms <0 :
		# The processing return value is -2 and -1 The situation of 
		return ttl_in_ms
    else:
		# Convert milliseconds to seconds 
		return ms_to_sec(ttl_in_ms)

4.5, Expired key determination

Through out of date dictionaries , The program uses the following steps to check whether a key is expired

• Check whether the given key exists in the expired Dictionary , If it is , Get its expiration time （ No expiration dictionary means no expiration time is set ）
• Check current UNIX Whether the time stamp is greater than the expiration time of the key ： If so , Then the key has expired ; conversely

4.6, Expiration key deletion policy

If a key is out of date , So when will it be deleted ？

There are three possible answers to this question , It represents three different deletion strategies ：

• Delete regularly ： While setting the key expiration time , Create a timer , Let the timer when the key expiration time comes , Delete the key immediately
• Lazy deletion ： Every time you get a key from the key space , Judge whether it is overdue . Delete the key if it is out of date ; conversely
• Delete periodically ： Every once in a while , Check the database once , Delete the expiration key inside , As for how many expiration keys to delete 、 How many databases to check is determined by the algorithm

Delete regularly

The scheduled deletion strategy is very memory friendly ： Because using a timer , Every time the key expires, it will be deleted immediately . But if there are a lot of expired keys , The program takes a part CPU Time to delete these expired keys （ It can be understood as ： Deleting the expired key will compete with the main task CPU Execution time of ）; And to create a timer, you need to use Redis Time events in the server , The implementation of the current time event ---- Disordered list , Because its search time complexity is O(N), Therefore, it is not efficient to deal with a large number of time events

Lazy deletion

Inert deletion policy pair CPU Time is very friendly ： Only when a key is used will it be judged whether the key is expired , It will be deleted after expiration ; conversely . And the target of deletion is limited to the currently processed key , It doesn't cost anything on other unrelated keys CPU Time .

But the disadvantage is ： If there are a large number of expired keys in the database , But these expired keys have not been accessed , Then they may never be deleted , This can be seen as a memory leak （ Garbage data takes up most of the memory , But it cannot be deleted ）

Delete periodically

• The disadvantage of scheduled deletion is ： Deleting may take up too much CPU Time , Snatch time with task execution , Affect the response time and throughput of the server
• The disadvantage of lazy deletion is ： Because you cannot handle keys that are not accessed , Therefore, it is easy to cause memory leakage

Periodic deletion strategy is an integration and compromise between scheduled deletion and lazy deletion

• Delete periodically. Delete expired keys every once in a while , And by limiting the execution time and frequency of the deletion operation to reduce the pair of deletion operations CPU The effect of time .
• And through this periodic deletion , To reduce memory waste caused by expired keys

The key to this strategy is ： Determination of the duration and frequency of the deletion operation . If the time is too long or the execution is too frequent , That will degenerate into a scheduled deletion strategy ; If the time is too short or the frequency is too low , It is easy to cause a large number of expired keys to accumulate , Degenerate into inert deletion .

Redis The server actually uses two strategies: lazy deletion and periodic deletion , Use two strategies together , Can reasonably be in CPU Balance time and memory