Redis data deletion policy in 2022

Redis Data deletion strategy

Redis Data features in

Redis It's a memory level database , All data is stored in memory , The data in memory can be accessed by TTL Command gets its state

1. XX ： Time sensitive data
2. -1 ： Permanent data
3. -2： Expired data or deleted data or undeleted data

The storage structure of timeliness data

Setting time key, adopt hash To store key The address and the time of survival , When you need to delete data later, you can delete it according to hash You can delete the data when you find the address in the field of

1、 Delete regularly

Create a timer , When key Set an expiration time , And when the expiration time arrives , Delete the key immediately by the timer task

usage ：

advantage ： To save memory , Then delete , Free up unnecessary memory quickly

shortcoming ：CPU Under great pressure , No matter what CPU How high is the load at this time , All occupied CPU, It will affect redis Server response time and instruction throughput

summary ： Trade processor performance for storage （ Time for space ）

2、 Lazy deletion

Data arrival expiration time , Don't deal with it . The next time you access the data

• If not expired , Return the data
• Found expired , Delete , Return does not exist

advantage ： save CPU performance , Delete when you find that you have to delete

shortcoming ： There's a lot of memory pressure , Data that has occupied memory for a long time

summary ： Swapping memory for processor performance （ Time for space ）

There's one in here expireIfNeeded() It is used to detect whether it is expired

3、 Delete periodically

Periodic polling redis Time sensitive data in the library , Using random sampling strategy , Use the proportion of expired data to control the frequency of deletion

• Redis When starting server initialization , Reading configuration server.hz Value , The default is 10

• Every second server.hz Time serverCron()——》databasesCron()——》activeExpireCycle()

• **activeExpireCycle()** For each expires[*] Test them one by one , Every time you execute 250ms/server.hz

• To someone expires[] When testing , Random selection W individual key Detect a expires[] When testing , Random selection W individual key testing

  

1. If key Overtime , Delete key
2. If you delete in a round key The number of >W*25%, Cycle the process
3. If you delete in a round key The number of ≤W25%, Check the next expires[],0-15 loop
4. W Value =ACTIVE_EXPIRE_CYCLE_LOOKUPS_PER_LOOP Property value

• Parameters current_db Used to record activeExpireCycle() Which one to enter expires[*] perform

• If **activeExpireCycle()** Execution time expired , Next time from current_db Keep going down

characteristic 1：CPU Performance occupancy settings have peaks , The detection frequency can be customized

characteristic 2： Memory pressure is not great , Cold data that has occupied memory for a long time will be continuously cleaned up

summary ： Periodically spot check storage space expireIfNeeded()（ Random sampling , Focus on spot check ）

4、 The eviction algorithm

1. Redis Using memory to store data , Before executing every command , Would call **freeMemoryIfNeeded()** Check if there is enough memory . If the memory does not meet the minimum storage requirements for newly added data ,redis To temporarily delete some data and clean up the storage space for the current instruction . The strategy for cleaning up data is called the eviction algorithm

2. Be careful ： The process of evicting data is not 100% Can clean up enough available memory space , If not, repeat . After trying all the data , If the requirement of memory cleaning cannot be met , An error message will appear .

(error) OOM command not allowed when used memory >'maxmemory'

Related configurations that affect data eviction

Maximum available memory

maxmemory

The proportion of physical memory occupied , The default value is 0, Means unrestricted . In the production environment, set according to the demand , Usually set at 50% above .

Select the number of data to be deleted each time

maxmemory-samples

Data selection does not scan the entire database , Leading to serious performance consumption , Reduce read and write performance . Therefore, the method of randomly obtaining data is adopted as the data to be detected and deleted

Deletion policy

maxmemory-policy

After reaching the maximum memory , The strategy of deleting the selected data

1. Detect volatile data （ Data sets that may be out of date server.db[i].expires ）

   1. volatile-lru： Select the least recently used data
   2. volatile-lfu： Select the data that has been used least recently
   3. volatile-ttl： Select the data that will be out of date
   4. volatile-random： Choose any data to eliminate

2. Check the whole database data （ All data sets server.db[i].dict ）

   1. allkeys-lru： Select the least recently used data
   2. allkeys-lfu： Select the data that has been used least recently
   3. allkeys-random： Choose any data to eliminate

3. Give up data

   1. no-enviction（ deportation ）： Exclusion data （redis4.0 Default policy in ）, Error will be raised OOM（Out Of Memory）

   maxmemory-policy volatile-lru
   

Data eviction policy configuration basis

Use INFO Command output monitoring information , The query cache hit and miss The number of times , Tune according to business needs Redis To configure