Cache breakdown! Don't even know how to write code???

Official account “Java Back end technology stack ” 

reply “000” Get the necessary e-books for programmers

stay Redis There are three major problems in ： Cache avalanche 、 Cache breakdown 、 Cache penetration , Let's talk today Cache breakdown .

10 swastika java Back end interview questions and answers https://download.csdn.net/download/o9109003234/14503134

About cache breakdown related theoretical articles , I believe you have seen a lot , But how is it implemented in the specific code , How to solve the problem , Maybe I'm confused .

today , Lao Tian will show you , Cache breakdown resolution and code implementation .

scene

Look at this code ：

/** 
 * @author  Tian Weichang 
 * @ official account  java Back end technology stack 
 * @date 2021/6/27 15:59
 */
@Service
public class UserInfoServiceImpl implements UserInfoService {

    @Resource
    private UserMapper userMapper;
    @Resource
    private RedisTemplate<Long, String> redisTemplate;

    @Override
    public UserInfo findById(Long id) {
        // The query cache 
        String userInfoStr = redisTemplate.opsForValue().get(id);
        // If it doesn't exist in the cache , Query the database 
        //1
        if (isEmpty(userInfoStr)) {
            UserInfo userInfo = userMapper.findById(id);
            // There is no... In the database 
            if(userInfo == null){
                  return null;
            }
            userInfoStr = JSON.toJSONString(userInfo);
            //2
            // Put into cache 
            redisTemplate.opsForValue().set(id, userInfoStr);
        }
        return JSON.parseObject(userInfoStr, UserInfo.class);
    }

    private boolean isEmpty(String string) {
        return !StringUtils.hasText(string);
    }
}

The whole process ：

If , stay //1 To //2 Time consuming between 1.5 second , That means here 1.5 In seconds, all queries will go to the query database . This is what we call cache “ Cache breakdown ”.

Actually , If the concurrency of your project is not very high , Also don't be afraid , And I've seen many projects written almost like this , Not so much , After all, cache breakdown may occur only for the first time .

but , Let's not write code with a fluke mentality , Since it is caused by multithreading , It is estimated that many people will think of locks , Let's use locks to solve .

Improved version

Since the lock is used , So what we should care about at the first time is the granularity of the lock .

If we put the method findById On , That is, all queries will have lock competition , Here I believe everyone knows why we don't focus on methods .

/** 
 * @author  Tian Weichang 
 * @ official account  java Back end technology stack 
 * @date 2021/6/27 15:59
 */
@Service
public class UserInfoServiceImpl implements UserInfoService {

    @Resource
    private UserMapper userMapper;
    @Resource
    private RedisTemplate<Long, String> redisTemplate;

    @Override
    public UserInfo findById(Long id) {
        // The query cache 
        String userInfoStr = redisTemplate.opsForValue().get(id);
        if (isEmpty(userInfoStr)) {
            // There is a lock only when it does not exist 
            synchronized (UserInfoServiceImpl.class){
                UserInfo userInfo = userMapper.findById(id);
                // There is no... In the database 
                if(userInfo == null){
                     return null;
                }
                userInfoStr = JSON.toJSONString(userInfo);
                // Put into cache 
                redisTemplate.opsForValue().set(id, userInfoStr);
            }
        }
        return JSON.parseObject(userInfoStr, UserInfo.class);
    }

    private boolean isEmpty(String string) {
        return !StringUtils.hasText(string);
    }
}

It seems to solve the problem , Actually , The problem still hasn't been solved , Still cache breakdown , Because after queuing to obtain the lock , The synchronization block code will still be executed , That is, it will also query the database , Cache breakdown is not resolved at all .

Double check lock

thus , We introduce Double check lock , We made a slight change in the previous version , In the synchronization module, check again whether there is in the cache .

/** 
 * @author  Tian Weichang 
 * @ official account  java Back end technology stack 
 * @date 2021/6/27 15:59
 */
@Service
public class UserInfoServiceImpl implements UserInfoService {

    @Resource
    private UserMapper userMapper;
    @Resource
    private RedisTemplate<Long, String> redisTemplate;

    @Override
    public UserInfo findById(Long id) {
        // Check cache 
        String userInfoStr = redisTemplate.opsForValue().get(id);
        // Verify whether the cache exists for the first time 
        if (isEmpty(userInfoStr)) {
            // locked 
            synchronized (UserInfoServiceImpl.class){ 
                // Query the cache again , The purpose is to determine whether the previous thread has set After that 
                userInfoStr = redisTemplate.opsForValue().get(id);
                // Check whether the cache exists for the second time 
                if (isEmpty(userInfoStr)) {
                    UserInfo userInfo = userMapper.findById(id);
                    // There is no... In the database 
                    if(userInfo == null){
                        return null;
                    }
                    userInfoStr = JSON.toJSONString(userInfo);
                    // Put into cache 
                    redisTemplate.opsForValue().set(id, userInfoStr);
                }
            }
        }
        return JSON.parseObject(userInfoStr, UserInfo.class);
    }

    private boolean isEmpty(String string) {
        return !StringUtils.hasText(string);
    }
}

such , It seems that we have solved the cache breakdown problem , Do you think it's solved ？

A malicious attack

Review the above case , Under normal circumstances, there is no problem , But once someone maliciously attacks ？

for instance ： Enter the reference id=10000000, There's no such thing in the database id, What shall I do? ？

First step 、 There is no... In the cache

The second step 、 Query the database

The third step 、 Because... Does not exist in the database , Straight back , No operation cache

Step four 、 Take the first step again ..... It's a dead cycle

programme 1： Set an empty object

That is, when neither the cache nor the database exists , With id by key, Empty object is value.

set(id, An empty object );

Go back to the four steps above , It becomes .

for instance ： Enter the reference id=10000000, There's no such thing in the database id, What shall I do? ？

First step 、 There is no... In the cache

The second step 、 Query the database

The third step 、 Because... Does not exist in the database , With id by key, Empty object is value Put in cache

Step four 、 Take the first step , here , The cache exists , It's just an empty object .

Code implementation part ：

/** 
 * @author  Tian Weichang 
 * @ official account  java Back end technology stack 
 * @date 2021/6/27 15:59
 */
@Service
public class UserInfoServiceImpl implements UserInfoService {

    @Resource
    private UserMapper userMapper;
    @Resource
    private RedisTemplate<Long, String> redisTemplate; 

    @Override
    public UserInfo findById(Long id) {
        String userInfoStr = redisTemplate.opsForValue().get(id);
        // Determine whether the cache exists , Whether it is an empty object 
        if (isEmpty(userInfoStr)) {
            synchronized (UserInfoServiceImpl.class){
                userInfoStr = redisTemplate.opsForValue().get(id);
                if (isEmpty(userInfoStr)) {
                    UserInfo userInfo = userMapper.findById(id);
                    if(userInfo == null){
                        // Build an empty object 
                        userInfo= new UserInfo();
                    }
                    userInfoStr = JSON.toJSONString(userInfo);
                    redisTemplate.opsForValue().set(id, userInfoStr);
                }
            }
        }
        UserInfo userInfo = JSON.parseObject(userInfoStr, UserInfo.class);
        // Empty object handling 
        if(userInfo.getId() == null){
            return null;
        }
        return JSON.parseObject(userInfoStr, UserInfo.class);
    }

    private boolean isEmpty(String string) {
        return !StringUtils.hasText(string);
    }
}

programme 2 The bloon filter

The bloon filter (Bloom Filter): It is a probabilistic algorithm and data structure with high spatial efficiency , Used to determine whether an element is in a collection （ similar Hashset）. Its core is a long binary vector and a series of hash function , Array length and hash function The number of is determined dynamically .

Hash function ：SHA1,SHA256,MD5..

The purpose of the bloom filter is , Can quickly determine whether an element is in a set . Therefore, it has the following three usage scenarios :

• Web crawler right URL De duplication of , Avoid climbing the same URL Address

• anti-spam , Judging whether a mailbox is spam or not from billions of spam lists （ Spam messages ）

• Cache breakdown , Put the existing cache into the bloom filter , When hackers access the nonexistent cache, they can quickly return to avoid cache and DB Hang up .

Its internal maintenance is all 0 Of bit Array , It should be noted that , Bron filter has a concept of miscalculation rate , The lower the miscalculation rate , The longer the array , The more space it takes . The higher the error rate, the smaller the array , The less space it takes . The relevant theories and algorithms of Bloom filter are not discussed here , Those who are interested can study by themselves .

Advantages and disadvantages

advantage

• Full storage but not the element itself , It has advantages in some occasions where confidentiality requirements are very strict ;

• Space efficiency

• Insert / The query time is constant O(k), Far more than the general algorithm

Inferiority

• There is a miscalculation rate (False Positive), Default 0.03, As the number of elements stored increases , The miscalculation rate increases with it ;

• In general, you can't remove elements from a bloom filter ;

• Array length and hash It is complex to determine the number of functions ;

Code implementation ：

/** 
 * @author  Tian Weichang 
 * @ official account  java Back end technology stack 
 * @date 2021/6/27 15:59
 */
@Service
public class UserInfoServiceImpl implements UserInfoService {

    @Resource
    private UserMapper userMapper;
    @Resource
    private RedisTemplate<Long, String> redisTemplate;
    private static Long size = 1000000000L;

    private static BloomFilter<Long> bloomFilter = BloomFilter.create(Funnels.longFunnel(), size);

    @Override
    public UserInfo findById(Long id) {
        String userInfoStr = redisTemplate.opsForValue().get(id);
        if (isEmpty(userInfoStr)) {
            // Check whether it is in bloom filter 
            if(bloomFilter.mightContain(id)){
                return null;
            }
            synchronized (UserInfoServiceImpl.class){
                userInfoStr = redisTemplate.opsForValue().get(id);
                if (isEmpty(userInfoStr) ) {
                    if(bloomFilter.mightContain(id)){
                        return null;
                    }
                    UserInfo userInfo = userMapper.findById(id);
                    if(userInfo == null){
                        // Put it in a bloom filter 
                        bloomFilter.put(id);
                        return null;
                    }
                    userInfoStr = JSON.toJSONString(userInfo);
                    redisTemplate.opsForValue().set(id, userInfoStr);
                }
            }
        }
        return JSON.parseObject(userInfoStr, UserInfo.class);
    } 
    private boolean isEmpty(String string) {
        return !StringUtils.hasText(string);
    }
}

programme 3 The mutex

Use Redis When implementing distributed , Useful to setnx, Here you can imagine , Can we use this distributed lock to solve the problem of cache breakdown ？

This scheme is left to everyone to realize , Just master it Redis Distributed locks for , Then the implementation is very simple .

summary

Get it done Cache breakdown 、 Use Double check lock To solve the problem , notice Double check lock , You must think of... At first impression The singleton pattern , Here is also a review of the use of a double check lock .

Cache breakdown due to malicious attacks , We have also implemented two solutions , At least in jobs and interviews , Must be able to cope with .

in addition , When using the lock, pay attention to Lock strength , It is suggested to change to Distributed lock (Redis perhaps Zookeeper Realization ), Because since we introduce caching , In most cases, multiple nodes will be deployed , meanwhile , Distributed locks are introduced , We can use the method Enter the reference id To use , Isn't that better ！

I hope you can understand some ideas in this article , It's not rote Technology .

If you like , Please order Fabulous 、 Looking at

Recommended reading

• You don't move , I won't move --- Observer mode

• After leaving office ,10 Day interview  4 A summary of the company

• A set of Spring Cloud Alibaba Video tutorial

• Back end developers , Way of learning

• 121 Distributed interview questions and answers ,25K Proper