Redis technology leak detection and filling (II) - expired deletion strategy

Redis The memory recycling mechanism of is mainly reflected in two aspects ：

• By deleting the key object that reaches the expiration time , Free up memory
• Memory usage arrival maxmemory The upper limit triggers the memory overflow control strategy (Redis Technical leak detection and filling ( One )- Elimination strategy )

1 Delete expired key objects

Expiration deletion strategies can be roughly divided into three types ： Lazy deletion 、 Delete periodically 、 Delete regularly

1.1 Lazy deletion

Let the key expire , But every time you get a key from the key space , Check whether the obtained key is expired , If it's overdue , Delete the key ; If it doesn't expire , Just go back to the key .

The lazy deletion policy does not handle expired keys , At this time, because the memory does not reach Redis Of maxmemory The upper limit without triggering the elimination strategy ,Redis The expiration check will only be performed when accessing the key , When the key exceeds the expiration time, it will be deleted and return null . The disadvantage of lazy deletion is that it is not memory friendly , It will make the expired memory occupy memory space for a long time .

1.2 Delete periodically

stay 《Redis Development and operations 》 In, lazy deletion and scheduled task deletion are used to summarize the expiration deletion strategy , But through me to 《Redis Design and implementation 》、Redis Core technology and practical geek column And so on, the strategy of deleting within the default time is generally named periodic deletion strategy

Every once in a while , The program is checked once , Delete the expiration key inside . As for how many expiration keys to delete , And how many databases to check , It's up to the algorithm .

Redis The periodic deletion policy of maintains a periodic task internally ,Redis Every once in a while （ Default 100ms）, A certain number of keys will be selected randomly (ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP yes Redis A parameter of , The default is 20) The data of , Check if they are out of date , And delete the expired keys . Periodic deletion will be caused by the trigger mechanism Redis Performance degradation .

1.3 Delete regularly

stay 1.2 In this section, we have made my personal definition of regular deletion or regular deletion , We use 《Redis Design and implementation 》 Define and explain scheduled deletion , While setting the key expiration time , Each expiration key creates a timer , When the timer reaches the timing, delete the change key immediately . If there are too many expiration keys within a certain period of time, then too many timers will be right CPU Create a burden , Because of such problems, there is no efficient Redis Used by the .

2 Inert deletion in different cases

If Redis Enable persistence and master-slave synchronization , that Redis The expiration processing of is complex ⼀ some .

2.1 Persistence

It's generating RDB When you file , The program will check the keys in the database , Keys that have expired will not be saved to newly created RDB file .

When the server is running in AOF Persistent mode runtime , If Redis A key in has expired , But it hasn't been deleted lazily or regularly , that AOF The file will not be affected by this expiration key . After triggering lazy deletion or periodic deletion , The program will go to AOF Add one more DEL command , To explicitly record that the key has been deleted .

In execution AOF When rewriting, the content will be checked , Expired keys will not be saved to the rewritten AOF In file .

2.2  Master slave synchronization

Master slave synchronization , The slave server waits for the deletion command of the master server

If Redis Master slave synchronization is enabled , Then the processing of expired keys from the Library , Different versions have different strategies . For actual production use , I recommend Redis Upgrade to 3.2 Later versions , Because the secondary library is waiting for the deletion command of the primary Library , But for reading ：

If you're using Redis 3.2 Previous version , that , When reading a request from the library in the service , It does not determine whether the data is out of date , Instead, it will return expired data .

stay 3.2 After version ,Redis Improvements have been made. , If the read data has expired , Although it will not be deleted from the library , But it will return a null value , This prevents the client from reading expired data .

3 Problems caused by regular deletion

Regular deletion differs in the implementation details of each version , But to sum up, when the randomly selected key array exceeds 25% In case of expiration , The deletion operation will be executed circularly , Because of Redis It is a single thread that executes commands , Therefore, it will have a greater impact Redis Performance of .

The common reason for this is that there are a large number of keys with the same expiration time , As a result, a large number of Keys expire in a periodic deletion .

In view of the above situation , We know that the frequency of periodic deletion is in milliseconds , So when we need to expire a large number of keys at the same time , We can add a certain random number to these key expiration time milliseconds , We can guarantee that the data will expire in seconds , At the same time, it will not cause too many delete keys in a regular delete and cause impact Redis Performance of .

Books ：《Redis Development and operations 》

《Redis Design and implementation 》

Reference resources ：Redis Core technology and actual combat -32 | Redis Master slave synchronization and failover , What are the pit ？