Redis cache penetration, breakdown, avalanche and consistency issues

1. Cache penetration

Cache penetration refers to querying a data that does not exist in the cache and database.In this case, the request will definitely hit the database, which may cause the database to go down.

There are two ways to solve this problem:

• Cache empty queries, but this will take up extra space
• Validation filters, such as bloom filters

Bloom Filter

Bloom filters are an efficient and space-saving way to perform relatively accurate existence verification. The underlying principle is to go through multiple layers of hash filtering.

Simply put, we maintain an array of hashes. For all existing key values, we will traverse them. For each key value, we use N hash functions to calculate the location to which it will be projected, so that we can onlyGot a filter of 01 array.

When a request comes, we will use N hash functions to calculate which bit of the array the requested key will be mapped to, and the request will be allowed to pass only if every bit is passed.

After the above steps, we can ensure that as long as the existing key value can pass, but it does not rule out that a small number of non-existing keys will pass the filter.

But we can change the expected value of the passing accuracy by adjusting the array length L and the number N of hash functions. If you are interested in the specific mathematical reasoning process, you can learn more.

2. Cache breakdown

Cache breakdown means that hot keys not in the cache but in the database are accessed a lot in a short period of time.

There are three ways to solve this problem:

• Set the hotspot key not to expire
• Add distributed lock
• Current limit

3. Cache Avalanche

Cache avalanche refers to cache failures in a certain period of time, causing all requests to go to the database, causing the database to crash.

To solve this problem, you can set a random expiration time.

4. Cache coherency issues

The mainstream approach to resolving database-cache inconsistencies is delayed double deletion.

The process of delayed double deletion is as follows:

1. The update operation is familiar with deleting the Redis cache
2. Update the data in the database later
3. The update operation sleeps for a period of time
4. The update operation deletes the Redis cache again

Let's analyze the meaning of each step step by step:

1. Delete the Redis cache for the first time: This is to prevent the dirty data in Redis from being read by read requests when the update operation is updating the MySQL database.
2. Delete the Redis cache again: This is to prevent the update operation from deleting the Redis cache for the first time, a read request arrives at MySQL prior to the update request, and the dirty data is read to the Redis cache again.
3. Sleep for a period of time: This is to prevent the update operation from deleting the Redis cache for the first time, a read request arrives at MySQL prior to the update request, but the Redis cache is deleted again later than the update operation, causing the Redis cache to be updated again to become dirtydata.