Redis cache penetration

One 、 Problem description

key The corresponding data does not exist in the data source , Every time for this key The request for could not be fetched from the cache , Requests are pushed to the data source , That could crush the data source . For example, use a non-existent user id Get user information , No matter cache or database , If hackers exploit this vulnerability, they may crush the database

Two 、 Solution

There must be no data that cannot be cached or queried , Because the cache is written passively on Miss , And for the sake of fault tolerance , If no data can be found from the storage tier, it will not be written to the cache , This will cause the non-existent data to be queried by the storage layer every time it is requested , It loses the meaning of caching
Solution ：

1. Cache null values

If the data returned by a query is empty （ Whether the data doesn't exist or not ）, We still put this empty result （null） Cache , Setting the expiration time for empty results can be very short , Up to five minutes

2. Set up an accessible list （ White list ）

Use bitmaps Type defines a list of accessible , list id As bitmaps The offset , Every visit and bitmap Inside id Compare , If you visit id be not in bitmaps Inside , To intercept , Access not allowed

3. Use of Blum filter

• The bloon filter （Bloom Filter） yes 1970 Proposed by bron in . It's actually a very long binary vector （ Bitmap ） And a series of random mapping functions （ hash function ）
• The bloom filter can be used to retrieve whether an element is in a collection . Its advantage is that the space efficiency and query time are far more than the general algorithm , The disadvantage is that it has certain error recognition rate and deletion difficulty
• Hash all possible data to a large enough bitmaps in , A certain nonexistent data will be bitmaps Intercept , Thus, the query pressure on the underlying storage system is avoided
• Real-time monitoring ： If I found Redis Hit rate of begins to decrease rapidly , Need to check access objects and data , Cooperate with operation and maintenance personnel , You can set up a blacklist to restrict services