Redis (IX) - enterprise level solution (II)

Redis（ Nine ）—— Enterprise solutions （ Two ）

3、 Cache breakdown

Database server crash （2）

1. During the smooth operation of the system
2. Number of database connections An instant surge
3. Redis There's not a lot of servers key Be overdue
4. Redis Memory smooth , No fluctuations
5. Redis The server CPU normal
6. Database crash

Troubleshoot problems

1. Redis in Some key Be overdue , The key There's a lot of traffic
2. Multiple data requests are pressed directly from the server to Redis after , All miss
3. Redis A large number of accesses to the same data in the database are initiated in a short time

Problem analysis

• Single key High heat data
• key Be overdue

Solution （ Technique ）

1. Preset

   Take e-commerce for example , Each business according to the store level , Specify a number of key products , During the Shopping Festival , enlarge Such information key Of Expiration time

   Be careful ： Shopping day is not just about the day , And the next few days , The peak value of access is gradually decreasing

2. On site adjustment

   • Monitoring traffic , Extend the expiration time or set to permanent for data with natural traffic surge key

3. Background refresh data

   • Start timing task , Before the peak , Refresh data validity period , Make sure you don't lose

4. Second level cache

   • Set different failure times , It's not going to be eliminated at the same time

5. Lock Distributed lock , To prevent breakdown , But it's also a performance bottleneck , Be careful ！

summary

Cache breakdown is The moment when a single hot data expires , Large amount of data access , Not hit redis after , Initiated a large number of database queries on the same data , Causing pressure on the database server . Coping strategies should be in business data analysis and prevention , Cooperate with operation monitoring test and real-time adjustment strategy , After all, single key It's difficult to monitor the overdue data , Combined with avalanche treatment strategy

4、 Cache penetration

Malicious request

The primary keys in our database are from 0 At the beginning , Even if we put all the data in the database into the cache . When someone uses id=-1 To happen Malicious request when , because redis There's no data in , Will directly access the database , This is called cache penetration

terms of settlement

• Check the validity of data in the program , If it's illegal, go back
• Use The bloon filter

Brief introduction of bloon filter

Want to avoid cache penetration as much as possible , One way is to... The data Pre calibration , In the face of Redis Before operating with the database ,** First check whether the data exists , If it doesn't exist, go straight back .** If we want to query whether an element exists , To ensure query efficiency , You can choose HashSet, But if there is 10 Billion data , Use both HashSet For storage , Memory must not be able to hold . Then you need a bloom filter

The bloon filter （ English ：Bloom Filter） yes 1970 Proposed by bron in . It's actually a very long binary vector （bit Array ） And a series of random mapping functions （hash）. The bloom filter can be used to retrieve whether an element is in a collection

Because it's based on Digit group and hash function Of , So it's advantage yes Space efficiency and query The time is much longer than the general algorithm . but shortcoming Is also evident , That is, there is a certain false recognition rate and deletion difficulty . But by increasing the size of the digit group and increasing hash Number of functions to Reduce False recognition rate （ Can only lower , Can't be avoided ）

Put in the process

After the bloom filter is initialized , The values in the digit group are 0. When a variable is going to be put into the bloom filter , Will pass through multiple hash Function maps to each bit of the bit group , then Set the corresponding position to 1

The query process

The query is still through multiple hash Function maps to each bit of the bit group , If each bit is 1, Describe the element Possible , Note that there may be ！！. But if through mapping , The digit group corresponds to the upper digit Not for 1, Then the element must not exist

Put in the process diagram

For example, one of our Bloom filters 8 The number group of bits , And there are 3 individual hash function Calculate the elements , Map to each bit in the array

We will string ”Nyima” Put it in a bloom filter

Next, the string ”Cpower” Put it in a bloom filter

Query process diagram

For example, we need to query the string ”Cpower” Whether there is , adopt 3 individual hash The function maps to three positions of the digit group , All three positions are 1, Then the String may exist

For example, we need to query the string ”SWPU” Whether there is , adopt 3 individual hash The function maps to the three positions of the digit group , Found a location that is not 1, Then the The string must not exist

For example, we need to query the string ”Hulu” Whether there is , adopt 3 individual hash The function maps to the three positions of the digit group , Found that all locations are 1, But we didn't put the string ”Hulu” Put it in a bloom filter , So here There was a miscarriage of justice

Increase the size and size of the digit group hash The number of functions can reduce the misjudgment rate , But there is no way to avoid misjudgment