How to deal with cache hot key in redis

Redis How to deal with cache hot key problem

This article “Redis How to deal with cache hot key problem ” Most people don't quite understand the knowledge points of the article , So I made up the following summary for you , Detailed content , The steps are clear , It has certain reference value , I hope you can gain something after reading this article , Let's take a look at this article “Redis How to deal with cache hot key problem ” Article bar .

background

heat key What's the problem , How to cause ？

Generally speaking , The cache we use Redis Are multi node cluster versions , To someone key While reading and writing , Will be based on the key Of hash Calculate the corresponding slot, According to this slot You can find the corresponding fragment ( One master And multiple slave A group of redis colony ) To access the K-V. But in the process of practical application , For some specific business or some specific time period （ For example, the commodity spike of e-commerce business ）, There may be a large number of requests to access the same key. All requests （ And the proportion of such requests is very high ） Will fall into the same redis server On , The redis The load will be seriously increased , At this time, the whole system adds new redis Examples are useless , Because according to hash Algorithm , The same key Your request will still fall on the same new machine , This machine will still become the bottleneck of the system 2, Even cause the whole cluster to go down , If this hotspot key Of value It's bigger , It will also cause the network card to reach the bottleneck , This kind of problem is called “ heat key” problem .

Here's the picture 1、2 Shown , They are normal redis cluster Cluster and use a layer proxy Acting redis colony key visit .

As mentioned above , heat key It will bring ultra-high load pressure to a small number of nodes in the cluster , If not handled correctly , Then the downtime of these nodes is possible , This will affect the operation of the entire cache cluster , Therefore, we must find the heat in time key、 Solve the heat key problem .

1. heat key Probe

heat key Probe , See because redis The dispersion of clusters and hot spots key Some significant effects , We can do hot spots through the coarse and fine thinking process key Detection scheme .

1.1 Each of the clusters slot Of qps monitor

heat key The most obvious effect is the whole redis In the cluster qps Not so big premise , The traffic is distributed in the cluster slot The problem of inequality , Then the first thing we can think of is for each slot Monitor the traffic in , After reporting, do every slot Traffic comparison , Can be hot key Find the specific impact when it appears slot. Although this monitoring is the most convenient , But the granularity is too coarse , It is only applicable to the early cluster monitoring scheme , It is not suitable for accurate detection of heat key Scene .

1.2 proxy As the whole traffic entry statistics

If we use graph 2 Of redis colony proxy The proxy pattern , Because all requests will arrive first proxy Then to the specific slot node , So this hot spot key The detection statistics of can be placed in proxy In doing , stay proxy Based on Time slide window , For each key Count , Then calculate the number of key. To prevent excessive redundant statistics , You can also set some rules , Only the corresponding prefix and type key. This approach requires at least proxy Agent mechanism , about redis Architecture requirements .

1.3 redis be based on LFU The hot key Discovery mechanism

redis 4.0 The above versions support LFU The hot key Discovery mechanism , Use redis-cli –hotkeys that will do , perform redis-cli When combined with –hotkeys Options . You can regularly use this command in the node to find the corresponding hotspot key.

As shown below , You can see redis-cli –hotkeys The results of the implementation of , heat key Statistical information , This command takes a long time to execute , You can set regular execution to count .

1.4 be based on Redis The client does detection

because redis The command of is issued from the client every time , Based on this, we can redis client At some code of , Every client Do statistics based on time sliding window , Report to server, And then unified by server Distribute to each client, And configure the corresponding expiration time .

This way looks more Graceful , In fact, it is not so suitable in some application scenarios , Because in client The transformation of this side , It will bring more memory overhead to the running process , More directly , about Java and goLang This language of automatic memory management , Objects will be created more frequently , triggering gc Problems that cause interface response time to increase , This is not easy to predict .

Finally, through the infrastructure of each company , Make the right choice .

2. heat key solve

Through the above ways, we detected the corresponding heat key Or hot slot, Then we have to solve the corresponding heat key problem . Solve the heat key There are also several ideas for reference , Let's smooth it one by one .

2.1 To be specific key or slot Limit the flow

The simplest and most brutal way , For specific slot Or hot key Limit the flow , This plan is obviously detrimental to the business , Therefore, it is recommended to only use it in case of online problems , When a stop loss is needed, carry out a specific limit .

2.2 Use secondary （ Local ） cache

Local caching is also the most commonly used solution , Since our L1 cache can't withstand such a great pressure , Just add another L2 cache . Because every request is made by service Emitted , This L2 cache is added to service The end is perfect , Therefore, the corresponding heat can be obtained at the server every time key when , Use the local cache to store a , Wait for the local cache to expire before requesting again , Reduce redis Cluster pressure . With java For example ,guavaCache Is a ready-made tool . Following example ：

    // Local cache initialization and construction 
    private static LoadingCache<String, List<Object>> configCache
            = CacheBuilder.newBuilder()
            .concurrencyLevel(8)  // Level of concurrent read and write , It is recommended to set cpu Check the number 
            .expireAfterWrite(10, TimeUnit.SECONDS)  // How long does it expire after writing data 
            .initialCapacity(10) // initialization cache The size of the container 
            .maximumSize(10)//cache The container of is the largest 
            .recordStats()
            // build Method can specify CacheLoader, Pass... When the cache does not exist CacheLoader The implementation of automatic loading cache 
            .build(new CacheLoader<String, List<Object>>() {
                @Override
                public List<Object> load(String hotKey) throws Exception {
                    
                }
            });
    
    // Local cache fetch 
    Object result = configCache.get(key);

The biggest impact of local caching on us is the problem of data inconsistency , We set how long the cache expires , It will lead to the longest online data inconsistency problem , This cache time needs to measure its own cluster pressure and the maximum inconsistency time of business acceptance .

2.3 Demolition key

How to ensure that there is no heat key problem , And try to ensure data consistency ？ Demolition key It is also a good solution .

When we put it into the cache, we will cache the corresponding business key Split into several different key. As shown in the figure below , Let's first update the cache , take key Split into N Share , For example, a key The name is called "good_100", Then we can split it into four parts ,"good_100_copy1"、"good_100_copy2"、"good_100_copy3"、"good_100_copy4", You need to change this every time you update and add N individual key, This step is to dismantle key.

about service End to end , We need to find a way to make the traffic of our visits as uniform as possible , How to give yourself the heat of the upcoming visit key Add suffix . There are several ways , According to the local ip or mac Address do hash, The following values are related to disassembly key Take the remainder of the quantity , Finally decide what kind of splicing key suffix , So as to hit which machine ; A random number pair at service startup key Take the remainder of the quantity .

2.4 Another idea of local caching Configuration center

For partners who are familiar with the micro service configuration center , Our thinking can be changed to the consistency of the configuration center . take nacos For example , How does it achieve distributed configuration consistency , And the corresponding speed is very fast ？ Then we can compare cache to configuration , Do it like this .

Long polling + localization Configuration of . First, all configurations will be initialized when the service starts , Then start long polling regularly to check whether the configuration of current service monitoring has changed , If there are changes , The request for long polling will be returned immediately , Update local configuration ; If there is no change , For all business code, the local memory cache configuration is used . This ensures the timeliness and consistency of distributed cache configuration .

2.5 Other plans that can be made in advance

Each of the above solutions is relatively independent to solve the heat key problem , So if we are really facing business demands , In fact, there will be a long time to consider the overall scheme design . Some extreme seckill scenes bring heat key problem , If our budget is adequate , Business isolation of services can be done directly 、redis Cache cluster isolation , While avoiding affecting normal business , Better disaster recovery measures can also be taken temporarily 、 Current limiting measures .

Some integrated solutions

At present, there are a lot about hotKey Relatively complete application level solutions , Jd.com has open source in this regard hotkey Tools , The principle is client Make insight , Then report the corresponding hotkey,server After end detection , To the corresponding hotkey Send it to the corresponding server for local cache , And the local cache is in the remote corresponding key After the update , Updates will be synchronized , It is now relatively mature Automatically detect heat key、 Distributed consistent cache Solution , JD retail craze key.

That's about “Redis How to deal with cache hot key problem ” The content of this article , I believe we all have a certain understanding , I hope the content shared by Xiaobian will be helpful to you , If you want to know more about it , Please pay attention to the Yisu cloud industry information channel .