Redis cache avalanche, cache penetration, cache breakdown

Redis Cache avalanche 、 Cache penetration 、 Cache breakdown

• Redis Caching process
• Cache avalanche
• • Solution
  • • Never expire
    • Reasonably set the expiration time
    • Use Redis Distributed locks for
• Cache penetration
• • Solution
  • • Filter illegal queries
    • Caching empty objects
    • The bloon filter
    • • New addition of Bloom filter
      • Query of Bloom filter
      • Deletion of Bloom filter
      • Bloom filter solves cache penetration
      • Features of bloon filter
• Cache breakdown
• • Solution
  • • Set hot spots Key Never expire
    • Use Redis Distributed locks for

Redis Caching process

Redis The database is a nosql database , The stored data format is key-value.Redis The database runs on Memory in , So it's faster than his query MySql Much faster . So we will put some data that users often query on Redis in , When Redis Sometimes it goes straight back , When Redis If not, go to the database to find . So as to increase the operation efficiency of the service .

Cache avalanche

Redis The cached data in has expiration time , When in When a large number of caches fail at the same time, it will cause cache avalanche .
for instance , stay 11 Everyone went hungry at o'clock and ordered some takeout , Are you hungry at this time Redis There are a large number of business information stored in , And the expiration time set by hungry programmers for this cache is 6 Hours . So in the afternoon 5 It's dinner time, and a large group of people are hungry. What's the takeout , This is the time Redis Your cache just expired , In a short time, a large number of query requests fall into the fragile MySql On , Lead to MySql Direct explosion ！

Solution

To solve Redis Cache avalanche requires avoid Redis A large number of caches expire in a short time

Never expire

Set up Redis Medium key Never expire , But this will take up a lot of server memory .

Reasonably set the expiration time

Set the expiration time reasonably according to business needs . But there are some unexpected situations .

Use Redis Distributed locks for

Since a large number of requests fall to MySql Can lead to MySql The explosion ！ Then add a limit , Let only Same request Fall on MySql On , Anyway, they all query the same information , After that, you can go to Redis I found .

Cache penetration

Redis Cache penetration refers to , stay Redis No relevant data can be found in the cache or database . In other words, this is an illegal query , The client sent out a large number of illegal queries such as id It's negative , So every time this query needs to go to Redis And query in the database . Lead to MySql Direct explosion ！

Solution

Filter illegal queries

Filter illegal queries in the background service , Just don't let him fall into Redis Service . such as id<=0 perhaps Paging content is too large etc.

Caching empty objects

If his query data is legal , But it is Redis and MySql None of them , So here we are Redis Store an empty object in , In this way, the next time the client continues to query, it can be in Redis In the back . however , If the client keeps sending this malicious query , It will lead to Redis There are many such empty objects in , Waste a lot of space

The bloon filter

The bloom filter consists of a binary array and k A hash array .

New addition of Bloom filter

When we want to add an element （ For example, new python）, The bloom filter will use hash Function calculates several index values , Then change the corresponding position in the binary array to 1.

Query of Bloom filter

When we want to query an element （ For example, query python）, The bloom filter will use hash Function calculates several index values , Then query whether the corresponding position in the binary array All for 1. If it's all for 1 It means that the modified element exists . But bloom filter has the possibility of misjudgment , Because different elements hash The value after may be the same , So let's say we query java,java after hash The calculated index value and python As like as two peas. , Then you think java Also in the bloom filter .

Deletion of Bloom filter

ditto , To delete the bloom filter is to delete hash The corresponding position of the rear array is changed to 0. But there is the possibility of accidental deletion . Delete... According to the above example python Will put java Delete it .

Bloom filter solves cache penetration

Let's first put MySql The data in is stored in the bloom filter （ Due to the use of binary arrays , That is, bitmap, so little space is used ）, Then if Redis No hits in cache , You need to check MySql Before the database, query whether it is in the bloom filter MySql There's data .

Features of bloon filter

1. There is a possibility of miscalculation
2. If the data exists , Then you must return to true
3. The time complexity of the query is O(k),k by hash Number of functions
4. k The bigger it is , The larger the array length , The lower the likelihood of miscalculation
5. Use bitmaps （ Binary array ） So the memory pressure is small

Cache breakdown

Cache breakdown is similar to cache avalanche , It is also because Redis in key Due to expiration . It's just that cache breakdown is a hot spot key Due to expiration . When a hotspot data suddenly expires , It will lead to a sudden large number of situations falling directly to MySql On , Lead to MySql Direct explosion ！

Solution

There are two main ideas ,

1. Let that hot key Don't expire suddenly
2. Don't let a lot of requests fall on MySql On .

Set hot spots Key Never expire

Simple and crude , I don't even expire , You can't bypass me to visit MySql. But it may be right Redis Memory causes great pressure , Therefore, it is usually set for a long time .

Use Redis Distributed locks for

Since a large number of requests fall to MySql Can lead to MySql The explosion ！ Then add a limit , Let only one same request fall at a time MySql On , Anyway, they all query the same information , After that, you can go to Redis I found .