Repeated calls, messages, idempotent schemes, full collation

One 、 Preface

Recently, many businesses have been told in the early iteration process , Ignore many idempotent treatments , This is all technical debt , Take advantage of this opportunity , Sort out the idempotent schemes commonly used in the industry .

Two 、 scene

What is idempotency ？
What we are talking about here is ： There are many operations in the system , No matter how many times , Should have the same effect or return the same result .

for example ：

1. The front end repeatedly submits the selected data , Only one reaction result corresponding to this data should be generated in the background ;
   In this case, during the development process , I often meet , The user didn't respond when he clicked the button , Click again on the .
   Spike similar scenes , Users are crazy to click many times at one time .ps Of course, the front end can be locked .

2. We initiate a payment request , You should only deduct money from your user account once , When encountering network retransmission or system bug retransmission , It should only be deducted once ;

3. Sync message , The same message is processed only once .
   Don't underestimate this situation , In practice, we often encounter ：
   Its own message system ensures high reliability , For sending 、 If the reception times out, it will require retransmission .
   When the message system server fails , Mass message retransmission .
   Repair data for some business reasons , Need to go back to the previous message .

4. Create business order , Only one business request can be created at a time , Creating more than one will cause problems .

Many, many important situations need idempotent features to support .

3、 ... and 、 The concept of idempotence

idempotent （idempotent、idempotence） It is a concept of mathematics and computer science , Common in abstract algebra . In programming . The characteristic of an idempotent operation is that the effect of any multiple execution is the same as that of one execution . Idempotent function , Or idempotent method , It means that the same parameters can be used for repeated execution , And can get the same result function . These functions do not affect the state of the system , You don't have to worry about repeated execution that will change the system .

Four 、 Common solutions

unique index ：

That is, the unique index of the database or the unique composite index , To prevent duplicate data , This idempotent method is very strict , The general practice is ：
When operating data , First, query whether the record exists from the database , Insert if it doesn't exist , Insert the success , Continue with the next operation .
Of course, even though the concurrency was extremely high , Or database master-slave delay , As a result, some are not found , Continue to insert , The database layer directly throws dumpli key abnormal . Return directly if it exists .
such as ： Alipay's capital account , Alipay also has user accounts. , Each user can only have one fund account , How to prevent users from creating multiple fund accounts , Then to the users in the fund account table ID Add unique index , So a user adds a fund account record successfully .

token Mechanism ：

The cluster environment adopts token Add redis(redis Single threaded , Processing needs to be queued );
single JVM Environmental Science ： use token Add redis or token Add jvm Memory .
The general process ：1. Apply to the service before submitting the data token,token Put it in redis or jvm Memory ,token Valid time ;
2. Submit background verification token, At the same time to delete token, Generate a new token return .token characteristic ： To apply , Primary effectiveness , Can limit current .
In fact, it is similar to redis Distributed locks for , There are also many pits here , You can take a look at this ：redis Experience in deep water area redis The pits of distributed locks are all inside .

Pessimistic locking ：

Lock to get data .select * from table_xxx where id=‘xxx’ for update;
Be careful ：id Fields must be primary keys or unique indexes , Or lock the watch .
Pessimistic locks are usually used with transactions , Data locking time can be long , Select according to the actual situation ;
In fact, under high concurrency business , Rarely use , Unless the flow is relatively low

Optimism lock ：

Optimistic lock is just to lock the table at the moment of updating data , Do not lock the meter at other times , So compared with pessimistic lock , More efficient . There are many ways to implement optimistic locks version Or other state conditions ：

1. By version number ：

update bx_tablex  set name=#name#,version=version+1 where version=#version#

2. Pass condition limit :

  update bx_tabley  set bx_amount= bx_amount - #subAmount# where bx_amount-#subAmount# >= 0

requirement ：quality-#subQuality# >= , This scenario is suitable without version number , Update only is for data security verification , For example, it is used in inventory model , Deduct share and roll back share , Higher performance ;

Be careful ： Update operation of optimistic lock , It's best to update... With a primary key or a unique index , This is a line lock , Otherwise, the table will be locked during update , modify sql as follows

update bx_tablex  set name=#name#,version=version+1 where id=#id# and version=#version#;
update bx_tabley  set bx_amount= bx_amount - #subAmount# where  id=#id# and bx_amount-#subAmount# >= 0

Distributed lock ：

It is difficult to build a globally unique index in a distributed system , For example, a unique field cannot be determined .
Distributed locks are actually implemented through third-party systems (redis or zookeeper or etcd), Insert data or update data in the business system , Acquire distributed lock , And then do it , Then release the lock , In fact, this is the idea of locking multiple threads simultaneously , Introduce multiple systems , That is to say, the solution in the distributed system .

State machine idempotent ：

Business related to design documents , Or task related business , It's bound to involve state machines ( State change diagram ), There is a status on the business document , The state can change in different situations , In general, there are finite state machines , Now , If the state machine is already in the next state , At this time, there is a change of the previous state , In theory, it can't be changed , In this case , The idempotent of finite state machine is guaranteed . Be careful ： Order and other document business , There is a long flow of States , We must have a deep understanding of state machine , It is very helpful to improve the ability of business system design

5、 ... and 、 Idempotent duration

here , Let me specifically mention this matter , A lot of times , We are heavy-duty , In fact, it is the moment when the interface is called , Or the moment you receive the message , But retry after timeout , Users click more , This situation , We can store idempotent bonds key To redis 5 Minutes automatically expire , It can solve most scenarios .
If your business needs to be protected at all times , Then your idempotence needs to be permanently stored , Cost is very high . The company where the author works has encountered message system failure , Resend the news of the day ....
All in all , It depends on your business , Idempotent is also divided into many cases , Everyone chooses the best .

6、 ... and 、 follow-up

This is a common idempotent scheme in the industry , But not all , If you encounter more idempotency problems , Or a higher performance solution , Welcome to leave a message with me .