Write it at the front

Ha ha ha ha ha ha ha , This time it comes again redis,redis For all of you , Must have been familiar with it , The author recently encountered a problem when doing high concurrency , It's just one. http Interface , Carry one inside list ~ a, And then through a from redis Get data in , Considering the large amount of data , Later it added redis colony , But during the visit , It uses redisTemplate Of mget, No matter how you optimize it , The performance is on the second level , For cache reads , This magnitude is basically obsolete , So start somewhere else .

Performance test background

The data provided by the project is inserted into the interface redis-cluster Insert 1000 Bar record , And then through key Set of from redis-cluster Get data in .

Effect comparison

Before optimization

redisTemplate.mget

Like a query 1000 Data 2245ms There is no problem returning , A single average is 2245ms/1000=2.245ms, Pretty good , But for a highly concurrent scenario ,http Interface , It's just rubbish , Not at all . The average Internet company is interested in , If it is To C Interface of products , Required RT All are 200 Inside , Give up this kind of .

After optimization

Rewrote jedis Of get, And join in pipeline

Return the data

Cause analysis

about redisTeplate Implementation principle of , I won't repeat it here , Next, analyze the implementation principle after the transformation from the root , before this , Let's get to know redis An important attribute of clustering slot

stay redis In the cluster , The primary storage location for data , every last key Must correspond to one slot In the middle , The specific algorithm is CRC16(key) Algorithm , The flow is as follows: , See

redis.clients.util.JedisClusterCRC16#getSlot(java.lang.String)

CRC16 Algorithm

Implement source code analysis

Ideas ： First find out what key Which one is it slot in , And then through jedispool Get jedis, Every one here jedispool Corresponding to one redis example , from jedis Pass through pipeline Way to obtain , Get the results , Do it again merge

The following is a graphic representation of

1： take keys Mapping to jedispool,com.zcc.testrediscluster.jedis.SelfDefJedisSlotBasedConnectionHandler#getJedisPoolFromSlot This method works in native jedis There is no , We rewrote .

keys Mapping to jedispool

2： Rewrite native jedisHandler

Add one getJedisPoolFromSlot Method

3： Put this handler Trusteeship spring

bean trusteeship

4： Interface call

from redis Get the value

obtain jedis-pipeline

Data merging

Data collation

summary

The above results fully illustrate the importance of reading source code , When doing high concurrency , It highlights how important the source code is , If you don't go deep into the source code , When doing performance optimization , It's also a headache , The above implementation refers to Sohu open source cache-cloud design idea , While practicing , hold cache-cloud The source code of is also rolled out , Gain a lot , Will learn a lot from others , Suddenly I remembered Newton's words " If I look further , Because I stand on the shoulders of giants " Mutual encouragement ~