I heard that distributed IDS cannot be incremented globally?

Click on the blue words above “ Set to star ”

Hello everyone , I am a 【 Structure ferry 】, A ten-year procedural ape . This is the eleventh article in the practical experience series , This series will share a lot of useful experience in practical work , If there is harvest , Please share it with more friends .

 

We have time to add an article instead of the previous one ID Sort by page , The reason is that it is connected to the distributed network ID, But distributed ID There is no guarantee of order , Only global uniqueness can be guaranteed .

 

So today, let's discuss , Whether we can achieve orderly distributed ID Well ？

 

 

Distributed ID How to implement

Segment mode

The number segment mode is currently used to implement distributed ID The way , The number segment mode obtains a segment range in advance , Then all in memory ID Distribution of , Extremely high performance .

 

There are also many open source frameworks implemented in the number segment mode , For example, meituan's Leaf, Dripping Tinyid. If you don't know the implementation principle of the number segment mode, you can check the following address for in-depth study . It is difficult to realize increment in segment mode , At the end of the article, let's talk about whether there is any way to realize .

 

Leaf: https://github.com/Meituan-Dianping/Leaf

Tinyid: https://github.com/didi/tinyid

 

Snowflake

Snowflake yes Twitter Open source distributed ID generating algorithm , It is also used more in China . For example, baidu open source uid-generator Is based on Snowflake The algorithm is improved .

 

uid-generator：https://github.com/baidu/uid-generator

 

Snowflake Generate ID Performance is very good , But also meet the increasing requirements . But it depends on the time on the machine , If the time is inconsistent, there will be a problem of repetition .

 

Redis Incr

Redis You can use it directly Incr Command to increment numbers , So as to realize self increasing ID. If you use Redis Implement distributed ID Need to carry out ID The persistence of , Otherwise, it will disappear after restart, and there will be repeated problems .

 

Since you want to persist , Then you can't avoid using AOF perhaps RDB Achieve persistence . Use RDB Data may be lost , Lead to ID It happens repeatedly . Use AOF Be sure to configure the mode of brushing data as always, Each operation record is synchronized to the hard disk , In this way, we can ensure that the data is not lost , But the performance is poor . Of course Redis 4.0 And there's hybrid mode （AOF+RDB） Can be used is also a good choice .

 

Whether in a stand-alone or cluster environment , Some key It must be routed to a node . although Redis Stand alone can also support 10 Ten thousand basic QPS, Suppose your ID The demand exceeds this magnitude , This will become a bottleneck , Because you have to ensure self growth , There is no way to expand horizontally .

 

What do you want to do in an orderly way ?

Segment mode data will not be lost , High performance , It's an excellent scheme . However, the number segment mode cannot achieve global increment ID, If you want to achieve global increment , Then you can't split , There must be a fixed node , All requests are obtained from this node ID Talent . But the segment pattern is the characteristic of segmentation , Horizontal expansion can be achieved through segmentation , Lifting performance .

 

Segment mode can only realize local increment , For example, my order form is connected to distributed ID, If you use segment mode , Then there will be my first order ID yes 20001, Then place the second order ,ID It could be 10003. If you use ID null , There will be a problem of out of order , This is also why the previous article used high-precision time to sort .

 

In this case , Is it OK to ensure that the data under the same user is incremental , This is local increment . If you want to achieve local increment, you need to consider the following issues ：

 

• Fixed route

 

Increment of user dimension , You need to route the user's request to a fixed ID Service node , It's taken out like this ID Is local increment .

 

 

• ID A service node hangs up or restarts

 

Suppose the user A It's always routed to id-service-b above , here id-service-b Hang up , Then you need to find a new node to route , The problem is coming. , Route to a new node , The assumption is id-service-c.

 

And then id-service-c above ID Paragraph may be better than id-service-b Big , If it's big, it's ok , Or increasing . If it's small, there's a problem , So this problem needs to be solved .

 

 

How to solve the above problems ？

 

Personally, I don't think this problem is easy to solve , Because the users who route to a node are N, It is also impossible to the existing ID The query , Determine whether it is greater than the of the service to be routed ID paragraph . The whole logic is too complicated .

 

There is a simple way , When there is a fixed routing requirement ID When you want to switch routes , Lock the whole situation （ At this time, if there is a large amount of concurrency, it will affect the user's RT）, Put this ID All corresponding segments are cleared , In this way, a new segment is applied for when rerouting , The new segment must be larger than the previous segment , That is, increasing .

 

summary

In fact, if you want to realize a demand , There are many ways behind it . How to choose a suitable , The best solution is to test your usual accumulation and breadth of technology . You need to know the principle of each scheme , What's the difference , Which one costs less , Which one can fully meet the business needs .

 

original ： Structure ferry （ official account ID：jiagoubaiduren）, Welcome to share , Reprint please keep the source .