Redis + guava local cache API combination, performance burst!

Preface

We often use... In our development Redis As caching , Put high-frequency data in Redis Can improve business performance , Reduce MySQL Equal relational database pressure , Even some systems use Redis Data persistence ,Redis The loose document structure is very suitable for business system development , In the precise query , Data statistics business has great advantages .

But in the high-frequency data stream processing system ,Redis It's going to be a lot of pressure , meanwhile I/O Cost is the main reason for time-consuming , At this time, in order to reduce Redis Read and write pressure we can use the local cache ,Guava It provides us with excellent local cache API, Including expiration policies and so on , Low coding difficulty , I recommend it very much .

Design example

Redis Lazy load cache

Data is being added to MySQL No caching , Cache in exact lookup , Do query is cache , No query, no cache .

The flow chart is as follows ：

Code example ：

//  Pseudo code example  Xx Represent your business partners   Such as User Goods wait 
public class XxLazyCache {

    @Autowired
    private RedisTemplate<String, Xx> redisTemplate;

    @Autowired
    private XxService xxService;//  Your business service

    /**
     *  Inquire about   By querying whether there is a driver cache load in the cache   It is recommended to guarantee the service in the front id The corresponding data absolutely exists in the database 
     */
    public Xx getXx(int id) {
        // 1. Query whether there is data in the cache 
        Xx xxCache = getXxFromCache(id);
        if(xxCache != null) {
            return xxCache;//  Guard statements make the code easier to read 
        }
        // 2. Query the database to get the data   Let's assume that when it comes to business , From here id There are corresponding data in the database 
        Xx xx = xxService.getXxById(id);
        // 3. Set the cache 、 This step is equivalent to Redis Cache lazy load , Check this again next time id, Will go 
        setXxFromCache(xx);
        return xx;
        }
    }

    /**
     *  Yes xx Modify or delete data   After successfully operating the database   Delete cache 
     *  Delete request  -  Delete database data   Delete cache 
     *  Modification request  -  Update database data   Delete cache   The next time you query, you will pull new data from the database to the cache 
     */
    public void deleteXxFromCache(long id) {
        String key = "Xx:" + xx.getId();
        redisTemplate.delete(key);
    }

    private void setXxFromCache(Xx xx) {
        String key = "Xx:" + xx.getId();
        redisTemplate.opsForValue().set(key, xx);
    }

    private Xx getXxFromCache(int id) {
        //  Assemble a unique primary key as a cache prefix Key  Such as Xxx Information   Namely Xxx:id
        String key = "Xx:" + id;
        return redisTemplate.opsForValue().get(key);
    }

}
//  Business class 
public class XxServie {
    @Autowired
    private XxLazyCache xxLazyCache;
    //  Query the database 
    public Xx getXxById(long id) {
        //  Omit the implementation 
        return xx;
    }

    public void updateXx(Xx xx) {
        //  to update MySQL data   Omit 
        //  Delete cache 
        xxLazyCache.deleteXxFromCache(xx.getId());
    }

    public void deleteXx(long id) {
        //  Delete MySQL data   Omit 
        //  Delete cache 
        xxLazyCache.deleteXxFromCache(xx.getId());
    }
}
//  Entity class 
@Data
public class Xx {
    //  Business primary key 
    private Long id;
    // ... Omit 
}

Advantages as follows ：

• Ensure that the minimum amount of cache meets the requirements of accurate query business , Avoid cold data occupying valuable memory space

• Small intrusion into the business of adding, deleting, modifying and checking 、 Delete to synchronize

• Pluggable , For old system upgrades , Historical data does not need to initialize the cache at startup

Drawbacks as follows ：

• The amount of data needs to be controllable , Not applicable in the unlimited growth business scenario

• In the microservice scenario, it is not conducive to global caching applications

summary ：

• Space minimization

• Meet the accurate query scenario

• The total amount of data is controllable. It is recommended to use

• The microservice scenario is not applicable

Redis Combined with local cache

In the micro service scenario , Multiple microservices use a large cache , Streaming data service , High frequency read cache pair Redis Under great pressure , We use local caching in conjunction with Redis Cache usage , Reduce Redis pressure , At the same time, the local cache has no connection overhead , Better performance .

The flow chart is as follows ：

Business scenario ： During stream processing , Microservices process data uploaded by multiple devices , Each device has one code, The frequency of streaming data is high , Partition sending is used during message queue sending , We need to provide equipment code Generate the corresponding auto increment number , Use the self increasing sign to kafka in topic The number of partitions is used for modulo .

So if there is 10000 device , The self increasing number is 0~9999, After the module is taken, it can be sent by partition, so that each partition can be evenly distributed .

This autoincrement we use redis Self increasing number generation of , After generation, put it in redis Of hash Structure for caching , One device at a time , Let's go to this hash Cache , If you don't get it, use the self incrementing number to generate one , Then put redis Of hash In cache .

At this time, the self increment of each device will not change once it is generated , We thought of using local cache for optimization , Avoid high frequency calls redis To get , Reduce redis pressure .

Code example ：

/**
 *  This cache demonstrates how to combine redis Since the number  hash  The local cache is used to generate the device self increment 、 cache 、 Local cache 
 *  Local cache usage Guava Cache
 */
public class DeviceIncCache {

    /**
     *  Local cache 
     */
    private Cache<String, Integer> localCache = CacheBuilder.newBuilder()
        .concurrencyLevel(16) //  Concurrency level 
        .initialCapacity(1000) //  Initial capacity 
        .maximumSize(10000) //  Maximum cache length 
        .expireAfterAccess(1, TimeUnit.HOURS) //  cache 1 Hours expire without being used 
        .build();

    @Autowired
    private RedisTemplate<String, Integer> redisTemplate;

    /**
     * redis Self incrementing cache key
     */
    private static final String DEVICE_INC_COUNT = "device_inc_count";

    /**
     * redis The equipment code corresponds to the number of self increment hash cache key
     */
    private static final String DEVICE_INC_VALUE = "device_inc_value";

    /**
     *  Get the self increment of the device 
     */
    public int getInc(String deviceCode){
        // 1. Get... From local cache 
        Integer inc = localCache.get(deviceCode);
        if(inc != null) {
            return inc;
        }
        // 2. Local cache misses , from redis Of hash Cache get 
        inc = (Integer)redisTemplate.opsForHash().get(DEVICE_INC_VALUE, deviceCode);
        // 3. redis Of hash Not in cache , Description is new equipment , First, generate a self increment number for the equipment 
        if(inc == null) {
            inc = redisTemplate.opsForValue().increment(DEVICE_INC_COUNT).intValue;
            //  Add to redis hash cache 
            redisTemplate.opsForHash().put(DEVICE_INC_VALUE, deviceCode, inc);
        }
        // 4. Add to local cache 
        localCache.put(deviceCode, inc);
        // 4. Returns the self increment 
        return inc;
    }

}

Advantages as follows ：

• redis Ensure data persistence , Local cache guarantees ultra-high read performance , Microservices share redis Large cache scenes can effectively reduce redis pressure

• guava As a local cache , Provides a wealth of api, Expiration strategy , The maximum capacity , Ensure that the service memory is controllable , Cold data will not occupy memory space for a long time

• The local cache emptying caused by service restart will not affect the business . in addition , official account java selected , reply java interview , Obtain interview question information .

• Microservices and distributed scenario usage , In the distributed situation, each service instance will only cache the self increment number of the part of the device it accesses , Local memory space is optimal

• In the example business , The self increasing number meets the demand of uniform distribution in the distribution area , It can also meet the business scenario of counting the number of device accesses , Kill two birds with one stone

Drawbacks as follows ：

• Increase coding complexity , Not directly

• It is only applicable to scenes where the cache content is only added but not changed

summary ：

• Local cache space is controllable , Expiration policy is preferred

• It is applicable to microservices and distributed scenarios

• Cache contents cannot be changed

• Excellent performance

Postscript

redis It provides rich data types and api, It is very suitable for business system development , Statistics count （increment,decrement）, Marker bit （bitmap）, Loose data （hash）, fifo 、 Queue read （list）.

guava Cache as local cache , Can read efficiently at the same time , A lot of api It is convenient for us to control the amount of locally cached data and cold data .

author ： Hot butter beer

https://juejin.cn/post/7000263632151904293

official account “Java selected ” The published content indicates the source of , All rights reserved （ Those whose copyright cannot be verified or whose source is not indicated all come from the Internet , Reprinted , The purpose of reprinting is to convey more information , The copyright belongs to the original author . If there is any infringement , Please contact the , The author will delete the first time ！

Many people have asked recently , Is there any readers Communication group ！ The way to join is simple , official account Java selected , reply “ Add group ”, You can join the group ！

Java Interview questions （ Wechat applet ）：3000+ The road test questions , contain Java Basics 、 Concurrent 、JVM、 Threads 、MQ series 、Redis、Spring series 、Elasticsearch、Docker、K8s、Flink、Spark、 Architecture design, etc , Brush questions online at any time ！

------ Special recommendation ------

Special recommendation ： Focus on the most cutting-edge information and technology sharing , Official account for preparing for overtaking on curves and various open source projects and efficient software ,「 Big coffee notes 」, Focus on finding good things , It's worth your attention . Click the official account card below to follow .

If the article helps , Click to see , Forward! ！