当前位置:网站首页>I think it is necessary to write a general idempotent component
I think it is necessary to write a general idempotent component
2020-11-06 01:18:00 【Yin Jihuan】
Contents of this article
background
Simple idempotent implementation
2.1 Database record judgment
2.2 Concurrency problem solving
- Universal idempotent implementation
3.1 design scheme
3.1.1 Universal storage
3.1.2 Easy to use
3.1.3 Support annotation
3.1.4 Multilevel storage
3.1.5 Concurrent reading and writing
3.1.6 Execute the process
3.2 Idempotent interface
3.3 Idempotent annotation
3.4 Automatically distinguish duplicate requests
3.5 Storage structure
3.6 Source code address
background
Answer questions from friends : Is there any general scheme and Practice for idempotent ?
About what is idempotent , This article will not elaborate . I'm sure you all know that , And have encountered similar problems and have their own set of solutions .
Basically, idempotency in all business systems is handled separately , It's not to say that it can't be dealt with in a unified way , There will be a lot to consider if we deal with it in a unified way .
I personally think that the core business is suitable for the business side to handle by itself , For example, order payment , There's going to be a payment record , An order can only be paid once , The idempotent effect can be achieved through the payment record form .
There are other businesses that are not core , But there is also a need for idempotency . For example, network problems , Multiple retries . The user clicks many times and so on . In this scenario, we still need a general idempotent framework to deal with , Will make business development easier .
Simple idempotent implementation
The realization of idempotent is not complicated , There are many kinds of solutions , First of all, it introduces the scheme based on database record to realize , The general scheme will be introduced later .
Database record judgment
Take the payment scenario at the beginning of the article as an example . The business scenario is that an order can only be paid once , So we will judge whether the order has been paid before payment , If not, pay , If you have paid , The payment was successful , idempotent .
This requires an extra table to store the actions done , In order to judge whether it has been done before .
It's like you're old , And then it's a single tech house . Your family is in a hurry at this time , Your mother introduces you to your little sister every day . You have to dress up every weekend , To meet the little sister your mother introduced you to .
You have to record it before you go ,8 I met in the first week of the month XXX, I saw you the next week YYY, If you're asked to meet again in the third week XXX, If you don't like XXX, You'll take a look at your little book , I've seen this before , There's no need to see you again , Otherwise, it would be embarrassing to meet you .
Concurrency problem solving
By querying payment records , There is no problem in business logic to determine whether payment can be made . But there are problems in concurrent scenarios .
1001 Two payment requests were initiated for the order of , The current two requests query the payment record at the same time , We didn't find out , And then they all started to go through the logic of payment , Finally, we found that the same order was paid twice , This is the idempotent problem caused by concurrency .
There are also many solutions to concurrency , Simple point of the direct use of the database's unique index to solve , A little troublesome will use distributed locks to lock the same resource .
For example, we have orders for 1001 To lock , If two payment requests are initiated at the same time , Then only one request can acquire the lock at a time , Another request that cannot obtain the lock can fail directly , You can also wait for the previous request to complete .
If you wait for the previous request to complete , And then deal with it , You can find out 1001 It has been paid for , Direct return payment succeeded .
Universal idempotent implementation
In order to enable us to focus more on the development of business functions , Idempotent operations in simple scenarios can be handled by unified encapsulation , Let's introduce the implementation of general idempotent .
design scheme
Universal storage
Generally, if we do idempotent inside the program, we query first , Then according to the query results do the corresponding operation . At the same time, the same resources will be locked to avoid concurrency problems .
Locking is universal , The unusual part is to judge whether the operation has been operated before , So we need to have a common store to record all operations .
Easy to use
Provides general idempotent components , The idempotent can be realized by injecting the corresponding class , Shielding and locking , Record, judge, etc .
Support annotation
Except for idempotent control through code , And to make it easier to use , You also need to provide annotations to support idempotency , Users only need to add corresponding annotations on the corresponding business methods , In this way, idempotent can be realized .
Multilevel storage
Need to support multi-level storage , For example, primary storage can use Redis To achieve , The advantage is high performance , Apply to 90% Scene . Because many scenarios are designed to prevent problems caused by repeated requests in a short period of time , By setting a certain failure time , Give Way Key Automatic failure .
Secondary storage can support Mysql, Mongo Such as the database , It is suitable for scenes with long time or permanent storage .
You can specify what primary storage uses by configuration , What does secondary storage use . This scenario is well suited to be implemented with a policy pattern .
Concurrent reading and writing
The introduction of multi-level storage is bound to involve concurrent read-write scenarios , There are two ways to support , Sequence and concurrency .
Order is to write first level storage , Write secondary storage , It's the same with reading . The problem is that the performance will be a little bit wasted .
Concurrency is multithreading writing at the same time , Read at the same time , Improve performance .
Idempotent execution process
Idempotent interface
Idempotent interface definition
public interface DistributedIdempotent {
/**
* Idempotent execution
* @param key idempotent Key
* @param lockExpireTime Expiration time of lock
* @param firstLevelExpireTime Primary storage expiration time
* @param secondLevelExpireTime Secondary storage expiration time
* @param timeUnit Storage time unit
* @param readWriteType Reading and writing types
* @param execute The logic to execute
* @param fail Key Already exist , Execution logic after idempotent interception
* @return
*/
<T> T execute(String key, int lockExpireTime, int firstLevelExpireTime, int secondLevelExpireTime, TimeUnit timeUnit, ReadWriteTypeEnum readWriteType, Supplier<T> execute, Supplier<T> fail);
}
Usage mode
/**
* Code mode idempotent - There is a return value
* @param key
* @return
*/
public String idempotentCode(String key) {
return distributedIdempotent.execute(key, 10, 10, 50, TimeUnit.SECONDS, ReadWriteTypeEnum.ORDER, () -> {
System.out.println(" Came in ....");
return "success";
}, () -> {
System.out.println(" repeated ....");
return "fail";
});
}
Idempotent annotation
Using annotations , It makes it easier to use , For example, our transaction processing , Caching and so on all use annotations to simplify logic .
Idempotent scenarios can also define general annotations to simplify the use of difficulty , Add comments on business methods that need to support idempotency , Configure basic information .
idempotentHandler It is the method to execute after triggering idempotent rules , That's when we use code to implement idempotent Supplier
In an idempotent situation , If it's repetitive , Usually return the same result as normal execution .
/**
* The way of annotation is idempotent - Specifies the method to be executed after the idempotent rule is triggered
* @param key
*/
@Idempotent(spelKey = "#key", idempotentHandler = "idempotentHandler", readWriteType = ReadWriteTypeEnum.PARALLEL, secondLevelExpireTime = 60)
public void idempotent(String key) {
System.out.println(" Came in ....");
}
public void idempotentHandler(String key, IdempotentException e) {
System.out.println(key + ":idempotentHandler It has been carried out ....");
}
Automatically distinguish duplicate requests
Code handling idempotent , Need to pass in idempotent Key, The annotation method deals with idempotency , The supporting configuration Key, Support SPEL expression . Both of them need to be used to determine the uniqueness of idempotency .
There is also a more common scenario of idempotency , It is to prevent repeated submission or network problems, timeout and try again . The same operation will be requested many times , In this scenario, you can apply for a unique ID, Bring it to the back end every time it's requested , This will identify the uniqueness of the entire request .
I currently have a function that automatically generates a unique identifier , In short, it's based on the requested information MD5, If MD5 No change in value is considered the same request .
Need to carry out MD5 There is a request for the content of URL Parameters , Request body , Request header information . The information in the request header is not related to the specified user Key The scene will be all stitched together , If the request header is configured userId For the user's logo , Then you can only use userId.
Idempotent at the entry of the request Key Automatic generation of , If you do not specify when using idempotent annotations spelKey, You'll use auto generated Key.
Storage structure
Redis: Use String Type storage ,Key It's idempotent Key, Value The default is 1.
Mysql: You need to create a record table .( Expired data needs to be cleaned up regularly , It can also be stored permanently )
CREATE TABLE `idempotent_record` (
`id` int(11) NOT NULL AUTO_INCREMENT COMMENT ' Primary key ',
`key` varchar(50) NULL DEFAULT '',
`value` varchar(50) NOT NULL DEFAULT '',
`expireTime` timestamp NOT NULL COMMENT ' Expiration time ',
`addTime` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ' Creation time ',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT=' Idempotent records ';
Mongo: Fields follow Mysql equally , convert to Json The format is just .Mongo The collection is automatically created .
About author : Yin Jihuan , Simple technology enthusiasts ,《Spring Cloud Microservices - Full stack technology and case analysis 》, 《Spring Cloud Microservices introduction Actual combat and advanced 》 author , official account Ape world Originator .
I have compiled a complete set of learning materials , Those who are interested can search through wechat 「 Ape world 」, Reply key 「 Learning materials 」 Get what I've sorted out Spring Cloud,Spring Cloud Alibaba,Sharding-JDBC Sub database and sub table , Task scheduling framework XXL-JOB,MongoDB, Reptiles and other related information .
WeChat search Ape world reply kitty Access to the source code
版权声明
本文为[Yin Jihuan]所创,转载请带上原文链接,感谢
边栏推荐
- Polkadot series (2) -- detailed explanation of mixed consensus
- PLC模拟量输入和数字量输入是什么
- How to get started with new HTML5 (2)
- 有关PDF417条码码制的结构介绍
- xmppmini 專案詳解:一步一步從原理跟我學實用 xmpp 技術開發 4.字串解碼祕笈與訊息包
- Flink on paasta: yelp's new stream processing platform running on kubernetes
- Swagger 3.0 天天刷屏,真的香嗎?
- Troubleshooting and summary of JVM Metaspace memory overflow
- 至联云解析:IPFS/Filecoin挖矿为什么这么难?
- Can't be asked again! Reentrantlock source code, drawing a look together!
猜你喜欢
C++和C++程序员快要被市场淘汰了
人工智能学什么课程?它将替代人类工作?
Didi elasticsearch cluster cross version upgrade and platform reconfiguration
Face to face Manual Chapter 16: explanation and implementation of fair lock of code peasant association lock and reentrantlock
This article will introduce you to jest unit test
熬夜总结了报表自动化、数据可视化和挖掘的要点,和你想的不一样
直播预告 | 微服务架构学习系列直播第三期
Character string and memory operation function in C language
Jmeter——ForEach Controller&Loop Controller
华为云“四个可靠”的方法论
随机推荐
03_ Detailed explanation and test of installation and configuration of Ubuntu Samba
Nodejs crawler captures ancient books and records, a total of 16000 pages, experience summary and project sharing
Real time data synchronization scheme based on Flink SQL CDC
The difference between Es5 class and ES6 class
Filecoin的经济模型与未来价值是如何支撑FIL币价格破千的
Working principle of gradient descent algorithm in machine learning
Serilog原始碼解析——使用方法
WeihanLi.Npoi 1.11.0/1.12.0 Release Notes
数据产品不就是报表吗?大错特错!这分类里有大学问
Filecoin最新动态 完成重大升级 已实现四大项目进展!
Installing the consult cluster
[performance optimization] Nani? Memory overflow again?! It's time to sum up the wave!!
Filecoin主网上线以来Filecoin矿机扇区密封到底是什么意思
深度揭祕垃圾回收底層,這次讓你徹底弄懂她
Character string and memory operation function in C language
Can't be asked again! Reentrantlock source code, drawing a look together!
ipfs正舵者Filecoin落地正当时 FIL币价格破千来了
比特币一度突破14000美元,即将面临美国大选考验
How to demote a domain controller in Windows Server 2012 and later
Details of dapr implementing distributed stateful service