当前位置:网站首页>Essential for back-end programmers: distributed transaction Basics
Essential for back-end programmers: distributed transaction Basics
2020-11-08 12:11:00 【osc_lg0msmnd】
Preface
Recently I read several blogs about distributed transactions , Take notes . ha-ha ~
Database transactions
Database transactions ( abbreviation : Business ), It is a logical unit in the execution of database management system , It consists of a limited sequence of database operations , These operations are either all performed , Or none at all , It is an indivisible work unit .
Several typical features of database transactions : Atomicity (Atomicity )、 Uniformity ( Consistency )、 Isolation, ( Isolation) And persistence (Durabilily), The abbreviation is ACID.
- Atomicity : The transaction is executed as a whole , The operations on the database contained in it are either all performed , Either not .
- Uniformity : Before and after the transaction , The data will not be destroyed , If A Account to B Account transfer 10 Yuan , Whether it works or not ,A and B The total amount of is the same .
- Isolation, : When multiple transactions are accessed concurrently , Transactions are isolated from each other , That is, one transaction does not affect the running effect of other transactions . in short , It's just that the water doesn't flow into the river .
- persistence : Indicates that after the transaction is completed , Operational changes made by the transaction to the database , Will persist in the database .
How transactions are implemented
Local transactions
Traditional single server , Transactions in a single relational database , It's local business . Local transactions are managed by resource manager ,JDBC Transaction is a very typical local transaction .
Transaction log
innodb The transaction log includes redo log and undo log.
redo log( Redo log )
redo log Usually physical logs , What is recorded is the physical modification of the data page , Not how to modify a line or lines , It is used to recover the physical data page after submission .
undo log( Rollback log )
undo log It's a logical log , and redo log It's different to record physical logs . Think of it this way , When delete When a record ,undo log A corresponding insert Record , When update When a record , It records an opposite update Record .
Business ACID The realization idea of features
- Atomicity : It's using undo log To achieve , If there is an error during the transaction execution or the user executes rollback, The system goes through undo log The log returns the status of the start of the transaction .
- persistence : Use redo log To achieve , as long as redo log Log persistence , When the system crashes , You can pass redo log Recover data .
- Isolation, : Through locks and MVCC, To separate affairs from each other .
- Uniformity : By rolling back 、 recovery , And isolation in the case of concurrency , To achieve consistency .
Distributed transactions
Distributed transactions : It refers to the participants in the transaction 、 Servers that support transactions 、 Resource servers and transaction managers are located on different nodes of different distributed systems . Simply speaking , Distributed transaction refers to the transaction in distributed system , It exists to ensure the data consistency of different database nodes .
Why do I need distributed transactions ? Next, I will explain it in two aspects :
Distributed transaction under microservice architecture
With the rapid development of Internet , Lightweight and well-defined microservices , On the stage of history . such as , A user places an order , Buy live gift service , Be divided into three service, They are gold coin service (coinService), Order service (orderService)、 Gift service (giftService). These services are deployed on different machines ( node ), Corresponding database ( Gold coin database 、 Order database 、 Gift database ) Also on different nodes .
Users place an order to buy gifts , Gift database 、 Gold coin database 、 The order database is on different nodes , It's not possible to use local transactions , So how to guarantee different databases ( node ) Data consistency on ? This requires distributed transactions ~
Distributed transactions under sub databases and sub tables
As the business grows , The data in the database is getting bigger and bigger , More than ten million levels of data , We need to divide it into databases and tables ( The company used to use mycat Sub database and sub table , Later used sharding-jdbc). A bank , The data is distributed on different nodes , For example, some of them are in Shenzhen computer room , Some are in Beijing computer room ~ You want to use local affairs to guarantee , I've been indifferent ~ You still need distributed transactions .
such as A turn 10 Block to B,A The account data is in Beijing computer room ,B The account data of is in Shenzhen computer room . The process is as follows :
CAP theory &BASE theory
Learn about distributed transactions , Of course, we need to understand CAP Theory and BASE theory .
CAP theory
CAP Theory as the basic theory of distributed system , In a distributed system , Consistency( Uniformity )、 Availability( Usability )、Partition tolerance( Partition tolerance ), These three elements can only achieve two points at most .
Uniformity (C:Consistency):
Consistency is the ability of data to be consistent across multiple copies . For example, a data is updated after a partition node , The data read from other partition nodes is also the updated data .
Usability (A:Availability):
Availability means that the services provided by the system must always be available , For each operation request of the user, the result can always be returned in a limited time . Here's the point " In a limited time " and " Return results ".
Partition tolerance (P:Partition tolerance):
Distributed systems encounter any network partition failure , There is still a need to be able to ensure that external services meet the requirements of consistency and availability .
choice explain
CA Give up partition fault tolerance , Enhance consistency and availability , In fact, it is the choice of traditional stand-alone database
AP Give up consistency , Partition fault tolerance and availability , This is the choice of many distributed systems when they are designed
CP Discard availability , Pursue consistency and partition fault tolerance , Network problems will directly make the whole system unavailable
BASE theory
BASE theory , It's right CAP in AP An extension of , For our business system , We consider sacrificing consistency for system availability and partition fault tolerance .BASE yes Basically Available( Basic available ),Soft state( Soft state ), and Eventually consistent( Final consistency ) Abbreviations of three phrases .
Basically Available
Basic available : It is realized by supporting local fault rather than global fault of the system . If you partition users in 5 Database server , The failure of a user database only affects the specific host 20% Users of , Other users are not affected .
Soft State
Soft state , The state can be out of sync for a while
Eventually Consistent
Final agreement , Finally, the data is consistent , Instead of being consistent all the time .
Several solutions for distributed transactions
Distributed transaction solutions mainly include the following :
- 2PC( Two stage submission ) programme
- TCC(Try、Confirm、Cancel)
- Local message table
- Best effort notification
- Saga Business
Two stage proposal
Two phase commit is a common distributed transaction solution . The commit of a transaction is divided into two phases : Preparation phase and submission of implementation plan .
Successful submission of phase II
Preparation stage , The transaction manager sends a prepare message to each resource manager , If the resource manager's local transaction operation is successful , Then return to success .
Submission execution phase , If the transaction manager receives a success message from all resource managers , Then send a submit message... To each resource manager ,RM according to TM The order execution submission of . Pictured :
Failure of two-stage submission
Preparation stage , The transaction manager sends a prepare message to each resource manager , If the resource manager's local transaction operation is successful , Then return to success , If execution fails , Then it returns failure .
Submission execution phase , If the transaction manager receives a message that any resource manager fails , Then send a rollback message to each resource manager . The resource manager rolls back the local transaction operation according to the instructions of the transaction manager , Release all lock resources used in transaction processing .
Advantages and disadvantages of two stage submission
2PC It's easy to implement , Lower cost , But it has the following disadvantages :
- A single point of the problem : If the transaction manager fails , Explorer will always be locked .
- Performance issues : All resource managers are in a synchronous blocking state during the transaction commit phase , Occupy system resources , Until the submission is complete , To release resources , It is easy to cause performance bottleneck .
- Data consistency issues : If some resource managers receive a submitted message , Some didn't receive , It will lead to data inconsistency .
TCC( Compensation mechanism )
TCC Compensation mechanism has been adopted , The central idea is this : For each operation , To register a corresponding confirmation and compensation ( revoke ) operation .
TCC(Try-Confirm-Cancel) Model
TCC(Try-Confirm-Cancel) Distributed transaction is realized by decomposing business logic . For a specific business service ,TCC The distributed transaction model requires the business system to implement three sections of logic :
try Stage : Try to execute , Complete all business consistency checks , Reserve necessary business resources .
Confirm Stage : At this stage, confirm and submit the business , Don't do any tests , because try The stage has been checked , Default Confirm There is no mistake in the stage .
Cancel Stage : If business execution fails , Then enter the stage , It will release try All business resources occupied by the phase , And roll back Confirm All operations performed in phase .
TCC The distributed transaction model consists of three parts : Main business services 、 From business services 、 Business activity manager .
- Main business services : The main business service is responsible for initiating and completing the entire business activity .
- From business services : From business services is the whole business activity of the participants , Realization Try、Confirm、Cancel operation , For the main business service to call .
- Business activity manager : The business activity manager manages and controls the entire business activity , Including recording transaction status , Call... From business service Confirm operation , Call... From business service Cancel Operation etc. .
Let's take the user's purchase of gifts as an example to simulate TCC The process of implementing distributed transactions :
Suppose the user A The balance is 100 Gold coin , The present you have is 5 Duo .A It took 10 Gold coin , Place the order , Buy 10 Rose . balance 、 Order 、 Gifts are in different databases .
TCC Of Try Stage :
- Generate an order record , The order status is to be confirmed .
- Will the user A The balance in the gold coins of the account is updated to 90, Freeze the gold coin to 10( Reserve business resources )
- The number of users' gifts is 5, The quantity to be added in advance is 10.
- Try After success , Then enter Confirm Stage
- Try There's anything wrong with the process , All enter Cancel Stage
TCC Of Confirm Stage :
- Order status updated to paid
- Update user balance to 90, It can be frozen as 0
- The number of user gifts is updated to 15, Increase to 0
- Confirm There's anything wrong with the process , All enter Cancel Stage
- Confirm Process execution successful , Then the transaction ends
TCC Of Cancel Stage :
- Modify order status to cancelled
- Update user balance back 100
- Update the number of user gifts to 5
TCC Advantages and disadvantages
TCC The scheme allows the application to customize the granularity of database operation , Reduced lock conflict , Can improve performance , But it also has the following disadvantages :
- The application is invasive ,try、confirm、cancel All three phases require business logic implementation .
- Need to be based on the network 、 Different rollback strategies are implemented for different failure reasons such as system failure , It's hard to achieve , Generally with the help of TCC Open source framework ,ByteTCC,TCC-transaction,Himly.
Local message table
ebay At first, the scheme of local message table was proposed , To solve the problem of distributed transactions . At present, this kind of scheme is widely used in the industry , Its core idea is to split distributed transactions into local transactions for processing . Take a look at the basic implementation flow chart :
Basic realization ideas
The sender :
- There needs to be a message table , Record information about message status .
- Business data and message tables are in the same database , That is to make sure that they are in the same local business .
- After processing the business data and writing the message table in the local transaction , By writing a message to MQ Message queue .
-
The message will be sent to the message consumer , If sending fails , Try again .
Message consumer :
- Processing messages in the message queue , Complete your business logic .
- At this time, if the local transaction is successful , It shows that the processing has been successful .
- If the local transaction fails , Then the execution will be retried .
- If it's a business failure , Send a business compensation message to the message producer , Notify to roll back and other operations .
The producer and consumer scan the local message table regularly , Send the unfinished or failed message again . If there is reliable automatic reconciliation and account supplement logic , This scheme is very practical .
advantage & shortcoming :
The advantage of this scheme is that it solves the problem of distributed transaction well , Achieve final consistency . The disadvantage is that message tables are coupled to the business system .
Best effort notification
What is the biggest notification
Best effort notification is also a distributed transaction solution . Here is an example of enterprise online bank transfer
- The enterprise online banking system calls the front interface , Go to transfer page
- Enterprise online banking calls transfer system interface
- The transfer system completes the transfer processing , Send transfer result notice to enterprise online banking system , If the notification fails , Then the transfer system repeats the notification according to the strategy .
- The enterprise online banking system has not received the notice , Actively call the interface of the transfer system to query the transfer result .
- The transfer system will encounter situations such as refund of foreign exchange , I will come back to check the account regularly .
Try your best to inform the objectives of the program , That is to say, the party initiating the notification through a certain mechanism , Try your best to inform the receiver of the result of business processing . The best effort notification implementation mechanism is as follows :
Try your best to inform the solution
Try your best to inform , May adopt MQ Of ack Mechanism .
programme
- 1. The sponsor will send the notice to MQ.
- 2. The receiving party listens MQ news .
- 3. When the receiving party receives the message , Deal with the business , Respond ack.
- 4. If the receiving party does not respond ack, be MQ Meeting interval 1min、5min、10min Wait for a repeat notice .
-
5. The receiving party can use the message proofreading interface , Keep the message consistent .
Transfer business flow chart :
The interaction process is as follows : - 1、 The user requests the transfer system to transfer .
- 2、 The transfer system completes the transfer , Send the transfer result to MQ.
- 3、 Enterprise online banking system monitoring MQ, Receive transfer result notice , If the message is not received ,MQ Notifications will be sent repeatedly . Transfer result received , Update transfer status .
- 4、 The enterprise online banking system can also actively query the transfer result query interface of the transfer system , Update transfer status .
Saga Business
Saga Things are done by Hector Garcia-Molina and Kenneth Salem Put forward , Its core idea is to split long transactions into multiple local short transactions , from Saga Transaction coordinator coordination , If it's finished normally, then it's finished normally , If a step fails , The compensation operation is called once in reverse order .
saga brief introduction
- Saga = Long Live Transaction (LLT, Long life affairs )
- LLT = T1 + T2 + T3 + ... + Ti(Ti For local short transactions )
-
Every local transaction Ti There is a corresponding compensation Ci
Saga Execution order of
- Normal condition :T1 T2 T3 ... Tn
-
Abnormal situation :T1 T2 T3 C3 C2 C1
Saga Two recovery strategies
- Recover backward , If any local sub transaction fails , Compensation for completed transactions . Such as the execution sequence of abnormal conditions T1 T2 Ti Ci C2 C1.
- Recover forward , That is, retry the failed transaction , Let's say that at the end of the day, every sub transaction will succeed . Execution order :T1, T2, ..., Tj( Failure ), Tj( retry ),..., Tn.
for instance , Suppose the user places an order , flowers 10 $to buy 10 More roses , Then there are
T1= Place the order ,T2= Deduct users 10 Yuan ,T3= User plus 10 Rose , T4= Inventory decrease 10 Rose
C1= Cancellation of order ,C2= Add... To the user 10 Yuan ,C3 = Less users 10 Rose , C4= Inventory plus 10 Rose
Suppose the transaction is executed to T4 An unexpected rollback occurred , stay C4 When we need to add the roses back to the stock , I found that the user's rose has been used , This is a Saga A disadvantage of , Problems caused by no isolation between transactions .
You can solve this problem by :
- In response to ⽤ layer ⾯ Add ⼊ Logic of logic lock .
- Session layer ⾯ Isolation to ensure that the string ⾏ operation .
- The business layer ⾯ Mining ⽤ Pre freezing of funds ⾦ Of ⽅ To isolate this part of the resources ⾦.
- In the process of business operation, by reading the current status in time ⽅ Get updates .
Reference and thanks
- dried food | An article takes you to learn about distributed transactions
- Then someone asks you about distributed transactions , Throw this to him
- Talk about distributed transactions , And the solution
- Mysql Transaction implementation principle
- Detailed analysis MySQL Transaction log (redo log and undo log)
- 《Saga Distributed transaction resolution ⽅ Case and practice 》
- Best effort notification for distributed transaction solutions
Official account number
- If you think it's well written, please give me a compliment + Pay attention to , thank you ~
- At the same time, I am very much expecting my buddies to pay attention to my official account , Later, better dry goods will be introduced slowly ~ Hee hee
版权声明
本文为[osc_lg0msmnd]所创,转载请带上原文链接,感谢
边栏推荐
- Flink从入门到真香(6、Flink实现UDF函数-实现更细粒度的控制流)
- 漫画|讲解一下如何写简历&项目
- 优化if-else代码的八种方案
- 2018中国云厂商TOP5:阿里云、腾讯云、AWS、电信、联通 ...
- YGC问题排查,又让我涨姿势了!
- 阿里出品!视觉计算开发者系列手册(附网盘链接)
- Bohai bank million level fines continue: Li Volta said that the governance is perfect, the growth rate is declining
- Windows10关机问题----只有“睡眠”、“更新并重启”、“更新并关机”,但是又不想更新,解决办法
- 蘑菇街电商交易平台服务架构及改造优化历程(含PPT)
- 华为云重大变革:Cloud&AI 升至华为第四大 BG ,火力全开
猜你喜欢
BCCOIN告诉您:年底最靠谱的投资项目是什么!
啥是数据库范式
Can you do it with only six characters?
分布式文档存储数据库之MongoDB基础入门
分布式文档存储数据库之MongoDB基础入门
On the confirmation of original data assets
一文读懂机器学习“数据中毒”
If you don't understand the gap with others, you will never become an architect! What's the difference between a monthly salary of 15K and a monthly salary of 65K?
Improvement of rate limit for laravel8 update
还不快看!对于阿里云云原生数据湖体系全解读!(附网盘链接)
随机推荐
适合c/c++新手学习的一些项目,别给我错过了!
年轻一代 winner 的程序人生,改变世界的起点藏在身边
Close to the double 11, he made up for two months and successfully took the offer from a large factory and transferred to Alibaba
Analysis of ArrayList source code
Tencent, which is good at to C, how to take advantage of Tencent's cloud market share in these industries?
11 server monitoring tools commonly used by operation and maintenance personnel
Don't look! Full interpretation of Alibaba cloud's original data lake system! (Internet disk link attached)
Top 5 Chinese cloud manufacturers in 2018: Alibaba cloud, Tencent cloud, AWS, telecom, Unicom
Service architecture and transformation optimization process of e-commerce trading platform in mogujie (including ppt)
A scheme to improve the memory utilization of flutter
How to write a resume and project
【Python 1-6】Python教程之——数字
Flink从入门到真香(3、从集合和文件中读取数据)
2018中国云厂商TOP5:阿里云、腾讯云、AWS、电信、联通 ...
擅长To C的腾讯,如何借腾讯云在这几个行业云市场占有率第一?
Harbor项目高手问答及赠书活动
Ubuntu20.04下访问FTP服务器乱码问题+上传文件
C language I blog assignment 03
Top 5 Chinese cloud manufacturers in 2018: Alibaba cloud, Tencent cloud, AWS, telecom, Unicom
python基础教程python opencv pytesseract 验证码识别的实现