. Net distributed transaction and landing solution

This article mainly introduces the common distributed transactions and their applications in .Net Landing plan under the platform , A lot of references , Mainly from DTM Official documents of 、 You know

This article is posted on my personal blog site ：
The article links

Commonly used distributed transaction mode

XA（ Two stage submission ）

XA By X/Open The specification of distributed transaction proposed by the organization ,XA The specification mainly defines ( overall situation ) Transaction manager and ( Local ) Explorer (RM) Interface between . Local databases such as mysql stay XA It plays RM role

XA It is divided into two stages ：

The first stage （prepare）： That is, all participants RM Prepare to execute the transaction and lock the required resources . participants ready when , towards TM The report is ready .

The second stage (commit/rollback)： When the transaction manager confirms that all participants (RM) all ready after , Send... To all participants commit command .

At present, the mainstream databases basically support XA Business , Include mysql、oracle、sqlserver、postgre

The flow chart is as follows ：

TCC

TCC There are three stages ：

• Try
  Check the resources of each service , Lock or reserve resources in advance （ Generally speaking Try If you succeed, it's time Confirm The stage will succeed by default , If it is not successful, it may be due to abnormal network 、 Server downtime, etc ）
• Confirm
  Perform actual operations in each service
• Cancel
  If the business method execution of any service is wrong , Then compensation is needed here , That is to say, perform the rollback operation of the business logic that has been operated successfully

Take cross bank transfer as an example ：

• Try Stage
  First freeze the funds in the two bank accounts , No more operation

• Confirm Stage
  Perform the actual transfer operation ,A Capital deduction from bank account ,B The increase of funds in the bank account

• Cancel Stage
  If the operation of any bank fails , Then you need to roll back to compensate
  such as A If the bank account has been deducted , however B The increase in bank account funds failed , Then we have to put A Add back the bank account funds

Applicable scenario ：

High requirements for consistency , It is common in capital scenes , have access to TCC, Write a lot of business logic by yourself , Judge by yourself whether each link in a transaction is ok, No ok Just carry out the compensation / Rollback code

Local message table

Use database + Realize asynchronous distributed transactions in the way of message queue , The process is as follows ：

1. A While the system operates in a local transaction , Insert a piece of data into the message table

2. next A The system sends this message to MQ

3. B After the system receives the message , In a business , Insert a piece of data into your local message table , Perform other business operations at the same time , If the message has been processed , At this time, the transaction will roll back , This ensures that messages are not processed repeatedly

4. B After the system is executed successfully , Will update their local Message table And A Status of the system message table

5. If B System processing failed , Then the message table status will not be updated , So at this time A The system will scan its message table regularly , If there is an unprocessed message , It will be sent again to MQ In the middle , Give Way B To deal with

This scheme guarantees final consistency
Even if the B The transaction failed , however A I'll keep sending messages back , until B Until we succeed there

SAGA

Saga This is a database paper SAGAS A distributed transaction scheme mentioned above . Its core idea is to split long transactions into multiple local short transactions , from Saga Transaction coordinator coordination , If each local transaction completes successfully, it completes normally , If a step fails , The compensation operation is called once in reverse order

give an example

For example, we want to carry out a business similar to bank inter-bank transfer , take A Medium 30 Yuan to B, according to Saga How things work , We will deal with the overall situation , It is divided into the following services ：

• Transfer out （TransOut） service , Transfer out here will be operated A-30
• Transfer out compensation （TransOutCompensate） service , Rollback the above transfer out operation , namely A+30
• into （TransIn） service , The transfer will take place B+30
• Transfer in compensation （TransInCompensate） service , Rollback the above transfer in operation , namely B-30

About null compensation 、 Hang 、 Idempotent problems and solutions

The following is excerpted from DTM Documents

Anomaly classification

The biggest enemy of distributed systems may be NPC 了 , Here it is Network Delay, Process Pause, Clock Drift An acronym for . Let's take a look at the specific NPC What's the problem ：

• Network Delay, Network delay . Although the network works well in most cases , although TCP Ensure that the transmission sequence and will not be lost , But it can't eliminate the problem of network delay .
• Process Pause, Process halted . There are many reasons that can cause the process to pause ： For example, in programming languages GC（ Garbage collection mechanism ） Will pause all running threads ; Another example , We sometimes pause ECS , Thus, the ECS can be migrated from one host to another without restarting . We cannot predict with certainty how long the process will be suspended , You think it's been a long time for hundreds of milliseconds , But in fact, it is not uncommon for a process to pause for several minutes .
• Clock Drift, Clock drift . In real life, we usually think that time passes smoothly , Monotonously increasing , But not in computers . The computer uses clock hardware to time , It's usually a quartz clock , Timing accuracy is limited , At the same time, it is affected by the temperature of the machine . In order to synchronize the time between multiple machines on the network to a certain extent , Usually use NTP The protocol aligns the time of the local device with a dedicated time server , A direct result of this is that the local time of the device may suddenly jump forward or backward .

Since distributed transactions are distributed systems , Nature also have NPC problem . Because there is no timestamp involved , The main problems are NP.

Air compensation

Cancel Execution time ,Try unexecuted , Transaction branch Cancel The operation needs to judge Try unexecuted , At this time, we need to ignore Cancel Business data update in , Go straight back to

General solutions ： For this problem , In service design , Need to allow null compensation , That is, when the business primary key to be compensated is not found , Return compensation succeeded , And record the original business PK , Mark that the business flow has been compensated successfully

Hang

Try Execution time ,Cancel Execution completed , Transaction branch Try The operation needs to judge Cancel Has been carried out , At this time, we need to ignore Try Business data update in , Go straight back to

General solutions ： You need to check whether the current business primary key already exists in the business primary key recorded by empty compensation , If it exists, you should refuse to execute the service , To avoid data inconsistency

idempotent

Because any request may have network exceptions , Duplicate request , All distributed transaction branch operations , All need to ensure idempotency （ That is, the results of multiple requests and one request are consistent , For example, change the balance to 100, No matter how many times it is called 100, And reduce the balance 100, The result of multiple calls will be different , Therefore, this problem needs to be considered in the design ）

DTM Sub transaction barrier

dtm in , Pioneered the sub transaction barrier technology , Use the technology , It can be very convenient to solve abnormal problems , It greatly reduces the use threshold of distributed transactions ：

The principle of sub transaction barrier technology is , In the local database , Establish branch operation status table dtm_barrier, The only key is the global transaction id- Branch id- Branch operation （try|confirm|cancel）

1. Open local transaction
2. For the current operation op(try|confirm|cancel),insert ignore A piece of data gid-branchid-op, If the insertion fails , The commit transaction returns success （ Common idempotent control methods ）
3. If the current operation is cancel, So in insert ignore A piece of data gid-branchid-try, If the insertion is successful （ Attention is success ）, Then the commit transaction returns success
4. Call the business logic within the barrier , If the business returns successfully , Then the commit transaction returns success ; If the business returns failed , Rollback transaction returns failure

Under this mechanism , It solves the problems related to disorder

• Empty compensation control – If Try No implementation , Directly executed Cancel, that 3 in Cancel Insert gid-branchid-try You will be successful , No logic within the barrier , Air compensation control is ensured
• Idempotent control –2 You cannot insert a unique key repeatedly in any of the operations , It ensures that there will be no repetition
• Anti suspension control –Try stay Cancel After performing , that Cancel Will be in 3 Insert gid-branchid-try, Lead to Try stay 2 It's not a success , Do not execute the logic within the barrier , The anti suspension control is ensured

Here is my understanding , stay try Stage , If try Of op Insert successfully , stay cancel Stage insert try, Normal is failure , If the insertion is successful , explain try The insertion failed during the phase , That is, the business is not implemented , that cancel The corresponding compensation in does not need to be called , That is to say, through gid-branchid-op Field to avoid null compensation and anti hanging , meanwhile gid-branchid-op Is the only one. , Therefore, the problem of repeated requests is also avoided , The idempotency of the interface does not need to be considered by ourselves

Distributed transactions in .Net Landing scheme in

DTM

Official website ：https://dtm.pub/

Strongly recommend , Support multiple transaction modes , Include SAGA、TCC、 Phase II message

.Net Of Demo：https://github.com/dtm-labs/dtmcli-csharp-sample It contains SAGA、TCC And sub transaction barrier , Read it again after reading dtm Of .Net SDK And official documents , Basically, it's almost familiar

CAP

Official website ：https://github.com/dotnetcore/CAP

CAP It's based on .NET Standard Of C# library , It's a solution for distributed transactions , Also has EventBus The function of .

Its bottom layer is through the database + Message queue is used to ensure the reliability of distributed transactions , But because it's asynchronous , The implementation of compensation mechanism is more complex , It is suitable for scenarios that do not need compensation mechanism （ Try again and again until you succeed , Alarm after retry for a certain number of times ）

How to choose the appropriate transaction mode

Strong consistent transactions ：

• Two stage message mode : Suitable for scenarios that do not need rollback （dtm specific ）
• saga Pattern : Suitable for scenarios that require rollback
• tcc Transaction mode : Suitable for scenarios with high consistency requirements
• xa Transaction mode : Suitable for concurrency with low requirements , There is no scenario of database row lock contention （ Basically not ）

Asynchronous distributed transactions （ Final agreement ）：

• Local message table

If you use DTM It is better to use sub transaction barrier , You don't need to think about empty suspension 、 idempotent 、 Empty compensation and other problems , Writing business is very comfortable , Reduce your mental burden

A person who thinks it is better to answer

For the scenario of strict capital requirements, we must not be wrong , It can be used TCC programme

If it is a general distributed transaction scenario , After the order is inserted, the inventory service is called to update the inventory , Inventory data is not as sensitive as money , You can use the reliable message final consistency scheme

You actually use such a scheme of any distributed transaction , Will lead to the complexity of your code 10 times . In many cases , System A Call the system B、 System C、 System D, We may not do distributed transactions at all . If the call reports an error, the exception log will be printed .

Just a few a month bug, quite a lot bug It's functional , Experiential , It really involves some data level bug, Just a few a month , Two or three ？ If you want to ensure that the system automatically guarantees data 100% No mistake , Dozens of distributed transactions , The code is too complex ; The poor performance , System throughput 、 The performance is greatly reduced .

99% Distributed interface call , Don't do distributed transactions , Directly monitoring （ email 、 texting ）、 Log （ Once the error , Complete log ）、 Quick positioning afterwards 、 Troubleshoot and find solutions 、 Repair data .
Every month , Every few months , For a small amount of code bug, The data that caused the error , Manually repair the data , Write a program yourself , It may be necessary to supplement some data , You may want to delete some data , You may have to modify the values of some fields .

Than you do 50 A distributed transaction , The cost should be hundreds of times lower , Dozens of times lower

trade off, Balance , When using distributed transactions , There must be a cost , The code can be complex , Development for a long time , Performance and throughput decline , The system is more complex and fragile, but it is easier to get out bug; benefits , If it's done ,TCC、 Reliable message final consistency scheme , You can definitely make it 100% Make sure that your piece of data won't go wrong .

1%,0.1%,0.01% The business of , Money 、 transaction 、 Order , We will use a distributed transaction scheme to ensure , Membership score 、 Coupon 、 Commodity information , Actually, don't do that

Excerpt from above ： Xiaozhi answers in Zhihu , Refer to the link at the end of the article 3

Reference material

The above content and pictures or part of the content are extracted from the following blog or answer

1. What are the commonly used distributed transaction solutions ? - Netease Shufan's answer - You know https://www.zhihu.com/question/64921387/answer/225784480
2. The interview will be asked ： There are six solutions for distributed transactions - Ao Bing's article - You know https://zhuanlan.zhihu.com/p/183753774
3. What are the commonly used distributed transaction solutions ? - Xiao Zhi's answer - You know https://www.zhihu.com/question/64921387/answer/1976701060

About distributed transactions , You can focus on Dtm Official handbook , It's very detailed , Address ：https://dtm.pub/guide/start.html, It's not just about Dtm Implementation of various distributed transactions under the framework , There are also solutions under different business scenarios , It's very convenient to learn

in addition , Most problems can be found in dtm Of github repository Up lookup issue, Or in CAP Of github repository Up lookup issue, Most of the questions can be found here , Or into dtm Wechat group communication