当前位置:网站首页>I think it is necessary to write a general idempotent component
I think it is necessary to write a general idempotent component
2020-11-06 01:18:00 【Yin Jihuan】
Contents of this article
background
Simple idempotent implementation
2.1 Database record judgment
2.2 Concurrency problem solving
- Universal idempotent implementation
3.1 design scheme
3.1.1 Universal storage
3.1.2 Easy to use
3.1.3 Support annotation
3.1.4 Multilevel storage
3.1.5 Concurrent reading and writing
3.1.6 Execute the process
3.2 Idempotent interface
3.3 Idempotent annotation
3.4 Automatically distinguish duplicate requests
3.5 Storage structure
3.6 Source code address
background
Answer questions from friends : Is there any general scheme and Practice for idempotent ?
About what is idempotent , This article will not elaborate . I'm sure you all know that , And have encountered similar problems and have their own set of solutions .
Basically, idempotency in all business systems is handled separately , It's not to say that it can't be dealt with in a unified way , There will be a lot to consider if we deal with it in a unified way .
I personally think that the core business is suitable for the business side to handle by itself , For example, order payment , There's going to be a payment record , An order can only be paid once , The idempotent effect can be achieved through the payment record form .
There are other businesses that are not core , But there is also a need for idempotency . For example, network problems , Multiple retries . The user clicks many times and so on . In this scenario, we still need a general idempotent framework to deal with , Will make business development easier .
Simple idempotent implementation
The realization of idempotent is not complicated , There are many kinds of solutions , First of all, it introduces the scheme based on database record to realize , The general scheme will be introduced later .
Database record judgment
Take the payment scenario at the beginning of the article as an example . The business scenario is that an order can only be paid once , So we will judge whether the order has been paid before payment , If not, pay , If you have paid , The payment was successful , idempotent .
This requires an extra table to store the actions done , In order to judge whether it has been done before .
It's like you're old , And then it's a single tech house . Your family is in a hurry at this time , Your mother introduces you to your little sister every day . You have to dress up every weekend , To meet the little sister your mother introduced you to .
You have to record it before you go ,8 I met in the first week of the month XXX, I saw you the next week YYY, If you're asked to meet again in the third week XXX, If you don't like XXX, You'll take a look at your little book , I've seen this before , There's no need to see you again , Otherwise, it would be embarrassing to meet you .
Concurrency problem solving
By querying payment records , There is no problem in business logic to determine whether payment can be made . But there are problems in concurrent scenarios .
1001 Two payment requests were initiated for the order of , The current two requests query the payment record at the same time , We didn't find out , And then they all started to go through the logic of payment , Finally, we found that the same order was paid twice , This is the idempotent problem caused by concurrency .
There are also many solutions to concurrency , Simple point of the direct use of the database's unique index to solve , A little troublesome will use distributed locks to lock the same resource .
For example, we have orders for 1001 To lock , If two payment requests are initiated at the same time , Then only one request can acquire the lock at a time , Another request that cannot obtain the lock can fail directly , You can also wait for the previous request to complete .
If you wait for the previous request to complete , And then deal with it , You can find out 1001 It has been paid for , Direct return payment succeeded .
Universal idempotent implementation
In order to enable us to focus more on the development of business functions , Idempotent operations in simple scenarios can be handled by unified encapsulation , Let's introduce the implementation of general idempotent .
design scheme
Universal storage
Generally, if we do idempotent inside the program, we query first , Then according to the query results do the corresponding operation . At the same time, the same resources will be locked to avoid concurrency problems .
Locking is universal , The unusual part is to judge whether the operation has been operated before , So we need to have a common store to record all operations .
Easy to use
Provides general idempotent components , The idempotent can be realized by injecting the corresponding class , Shielding and locking , Record, judge, etc .
Support annotation
Except for idempotent control through code , And to make it easier to use , You also need to provide annotations to support idempotency , Users only need to add corresponding annotations on the corresponding business methods , In this way, idempotent can be realized .
Multilevel storage
Need to support multi-level storage , For example, primary storage can use Redis To achieve , The advantage is high performance , Apply to 90% Scene . Because many scenarios are designed to prevent problems caused by repeated requests in a short period of time , By setting a certain failure time , Give Way Key Automatic failure .
Secondary storage can support Mysql, Mongo Such as the database , It is suitable for scenes with long time or permanent storage .
You can specify what primary storage uses by configuration , What does secondary storage use . This scenario is well suited to be implemented with a policy pattern .
Concurrent reading and writing
The introduction of multi-level storage is bound to involve concurrent read-write scenarios , There are two ways to support , Sequence and concurrency .
Order is to write first level storage , Write secondary storage , It's the same with reading . The problem is that the performance will be a little bit wasted .
Concurrency is multithreading writing at the same time , Read at the same time , Improve performance .
Idempotent execution process
Idempotent interface
Idempotent interface definition
public interface DistributedIdempotent {
/**
* Idempotent execution
* @param key idempotent Key
* @param lockExpireTime Expiration time of lock
* @param firstLevelExpireTime Primary storage expiration time
* @param secondLevelExpireTime Secondary storage expiration time
* @param timeUnit Storage time unit
* @param readWriteType Reading and writing types
* @param execute The logic to execute
* @param fail Key Already exist , Execution logic after idempotent interception
* @return
*/
<T> T execute(String key, int lockExpireTime, int firstLevelExpireTime, int secondLevelExpireTime, TimeUnit timeUnit, ReadWriteTypeEnum readWriteType, Supplier<T> execute, Supplier<T> fail);
}
Usage mode
/**
* Code mode idempotent - There is a return value
* @param key
* @return
*/
public String idempotentCode(String key) {
return distributedIdempotent.execute(key, 10, 10, 50, TimeUnit.SECONDS, ReadWriteTypeEnum.ORDER, () -> {
System.out.println(" Came in ....");
return "success";
}, () -> {
System.out.println(" repeated ....");
return "fail";
});
}
Idempotent annotation
Using annotations , It makes it easier to use , For example, our transaction processing , Caching and so on all use annotations to simplify logic .
Idempotent scenarios can also define general annotations to simplify the use of difficulty , Add comments on business methods that need to support idempotency , Configure basic information .
idempotentHandler It is the method to execute after triggering idempotent rules , That's when we use code to implement idempotent Supplier
In an idempotent situation , If it's repetitive , Usually return the same result as normal execution .
/**
* The way of annotation is idempotent - Specifies the method to be executed after the idempotent rule is triggered
* @param key
*/
@Idempotent(spelKey = "#key", idempotentHandler = "idempotentHandler", readWriteType = ReadWriteTypeEnum.PARALLEL, secondLevelExpireTime = 60)
public void idempotent(String key) {
System.out.println(" Came in ....");
}
public void idempotentHandler(String key, IdempotentException e) {
System.out.println(key + ":idempotentHandler It has been carried out ....");
}
Automatically distinguish duplicate requests
Code handling idempotent , Need to pass in idempotent Key, The annotation method deals with idempotency , The supporting configuration Key, Support SPEL expression . Both of them need to be used to determine the uniqueness of idempotency .
There is also a more common scenario of idempotency , It is to prevent repeated submission or network problems, timeout and try again . The same operation will be requested many times , In this scenario, you can apply for a unique ID, Bring it to the back end every time it's requested , This will identify the uniqueness of the entire request .
I currently have a function that automatically generates a unique identifier , In short, it's based on the requested information MD5, If MD5 No change in value is considered the same request .
Need to carry out MD5 There is a request for the content of URL Parameters , Request body , Request header information . The information in the request header is not related to the specified user Key The scene will be all stitched together , If the request header is configured userId For the user's logo , Then you can only use userId.
Idempotent at the entry of the request Key Automatic generation of , If you do not specify when using idempotent annotations spelKey, You'll use auto generated Key.
Storage structure
Redis: Use String Type storage ,Key It's idempotent Key, Value The default is 1.
Mysql: You need to create a record table .( Expired data needs to be cleaned up regularly , It can also be stored permanently )
CREATE TABLE `idempotent_record` (
`id` int(11) NOT NULL AUTO_INCREMENT COMMENT ' Primary key ',
`key` varchar(50) NULL DEFAULT '',
`value` varchar(50) NOT NULL DEFAULT '',
`expireTime` timestamp NOT NULL COMMENT ' Expiration time ',
`addTime` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT ' Creation time ',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT=' Idempotent records ';
Mongo: Fields follow Mysql equally , convert to Json The format is just .Mongo The collection is automatically created .
About author : Yin Jihuan , Simple technology enthusiasts ,《Spring Cloud Microservices - Full stack technology and case analysis 》, 《Spring Cloud Microservices introduction Actual combat and advanced 》 author , official account Ape world Originator .
I have compiled a complete set of learning materials , Those who are interested can search through wechat 「 Ape world 」, Reply key 「 Learning materials 」 Get what I've sorted out Spring Cloud,Spring Cloud Alibaba,Sharding-JDBC Sub database and sub table , Task scheduling framework XXL-JOB,MongoDB, Reptiles and other related information .
WeChat search Ape world reply kitty Access to the source code
版权声明
本文为[Yin Jihuan]所创,转载请带上原文链接,感谢
边栏推荐
- Using Es5 to realize the class of ES6
- 钻石标准--Diamond Standard
- 直播预告 | 微服务架构学习系列直播第三期
- 熬夜总结了报表自动化、数据可视化和挖掘的要点,和你想的不一样
- Group count - word length
- Skywalking series blog 1 - install stand-alone skywalking
- 合约交易系统开发|智能合约交易平台搭建
- 【效能優化】納尼?記憶體又溢位了?!是時候總結一波了!!
- Python crawler actual combat details: crawling home of pictures
- 多机器人行情共享解决方案
猜你喜欢
從小公司進入大廠,我都做對了哪些事?
事半功倍:在没有机柜的情况下实现自动化
This article will introduce you to jest unit test
Calculation script for time series data
The difference between Es5 class and ES6 class
Troubleshooting and summary of JVM Metaspace memory overflow
网络安全工程师演示:原来***是这样获取你的计算机管理员权限的!【维持】
人工智能学什么课程?它将替代人类工作?
Can't be asked again! Reentrantlock source code, drawing a look together!
GUI 引擎评价指标
随机推荐
全球疫情加速互联网企业转型,区块链会是解药吗?
Top 10 best big data analysis tools in 2020
Analysis of ThreadLocal principle
快快使用ModelArts,零基礎小白也能玩轉AI!
Working principle of gradient descent algorithm in machine learning
Basic principle and application of iptables
带你学习ES5中新增的方法
Face to face Manual Chapter 16: explanation and implementation of fair lock of code peasant association lock and reentrantlock
Network security engineer Demo: the original * * is to get your computer administrator rights! 【***】
Leetcode's ransom letter
Vuejs development specification
2019年的一个小目标,成为csdn的博客专家,纪念一下
PN8162 20W PD快充芯片,PD快充充电器方案
Python + appium automatic operation wechat is enough
Introduction to Google software testing
Analysis of react high order components
[event center azure event hub] interpretation of error information found in event hub logs
GUI 引擎评价指标
Listening to silent words: hand in hand teaching you sign language recognition with modelarts
中国提出的AI方法影响越来越大,天大等从大量文献中挖掘AI发展规律