Redis cache avalanche, penetration, breakdown, bloom filter, detailed explanation of distributed lock

Cache avalanche
There are a lot of caches at a certain time key invalid
terms of settlement
1. Valid until ————> Add the expiration date of the water machine to each data
2.redis Hang up —————> Use redis colony , Share the key The storage

extraction redis Clustered hash Consistency algorithm
It's time to show me geecache There are explanations in it , You need a hash ring , Portal

3. Do not set the cache time , Not very recommended
4. Use a scheduled task to refresh the cache that is about to expire , To reset

---

Cache penetration
visit redis Cache that does not exist in , Cause direct query database . Put pressure on the database . such as id=-1, Generally, this kind of illegal access . Abnormal data request （ hackers ）

Cache penetration cannot be avoided , Because data caching is an unsolved problem （ Consistency can never be guaranteed ） Not afraid of low frequency penetration , Afraid of high frequency .

terms of settlement
1. Using the bloon filter
2. Illegal access , Business directly throws
3. This kind of abnormality key, Also cache storage . Prevent subsequent frequent access to the database
4.ip Pull black

The bloon filter ：

Put the filter in this position , The more reason is put in front, the better , But controllability is not guaranteed .

Is there any cache in the filter ？？？
No , Because the cache of the filter is also very tight

Bloom algorithm ： Through a certain error rate in exchange for Space
How to understand this sentence

100 Billion data 1.1g about

principle
use bit Array to identify data

Bloom algorithm is an error caused by hash conflict , It's better to kill threethousand by mistake than to let one go

So here we propose two methods to reduce the probability of hash collision
1. Increase the length of the array
2. Increase the number of hash functions

The second method explains ：

Suppose we have a collision probability of percent 1, We calculate that the values of three places are 1 The probability is very low , Then the probability is almost one percent to the power of three . The probability of collision is greatly reduced .

But the more, the better ！！！！ Because more and more, the final error rate must be 100% ！！！

So the number of functions needs to refer to the length of the array

Algorithm disadvantages ： What if we delete data ？

In one position 1 Hash values that may represent several numbers , Then our solution is to build a counter , At the second latitude

If the value of the counter is 0, Then we can rest assured bit The value above the array position becomes 0

---

Cache breakdown
When there is a large amount of business concurrency , visit A single cache key. Query the same data concurrently , Single cache key, Due to expiration . Cause a large number of requests to directly access the database .( Auction scene , Generally, companies do not need to solve , Because there is no such hot data to let him breakdown ！！！！！！ )

terms of settlement
1. Add distributed locks
2. According to business , Determine whether it needs to be set to never expire

Distributed lock details ：

background , Grab a toilet ：

There are three conditions for locking ：
1. Shared resources
2. Multitasking requests
3. Shared resources are mutually exclusive
Lock only when all three conditions are met

Deadlock ：
Suppose teacher Ma is in the toilet , But I was shocked after locking the door , No one else can get access to this toilet , This leads to a deadlock

How to solve deadlock ：
Make a monitor , If the lock exists longer than the set maximum , We'll kill him （ Teacher Zhou has been calling without answering 120）

But it also leads to new problems ：timeout（ monitor ） How long is the time set ？？
The settings are short , As a result, he was killed before he finished , Half of the toilet was pulled out .（ Real scene ： One ticket sells more ）. The setting is too long , Lead to waste of resources

So it is generally based on zookeeper To implement distributed locking !!!
Not for now , Because I don't quite understand

So what do we do based on distributed locks ？

Request first nginx,nginx Distribute according to the distribution policy , Then the query redis data , If it fails , Then let's get a distributed lock , Grab the first slow existence redis in , Check the others directly redis

Suppose again 1000 The request , Only one cache is enough , other 999 You can query