Transaction and message semantics

One 、 What is business

1.1 Concept

A transaction is a program execution unit , All the operations in it are successful , Or all failed . That's what we often say （ACID）.

• Atomicity（ Atomicity ）： A business is an indivisible whole , All operations within the transaction are either successful , Or it's a total failure .
• Consistency（ Uniformity ）： Before and after the transaction , Data from one state to another must be consistent （A towards B Transfer accounts , Cannot appear A Buckle the money ,B Has received ）.
• Isolation（ Isolation, ）： Multiple concurrent transactions are isolated from each other , Don't interfere with each other .
• Durablity（ persistence ）： After the transaction completes , Changes to the data are permanently saved , Cannot be rolled back .

1.2 Distributed transactions

Distributed transaction refers to the participants of transaction 、 Servers that support transactions 、 Resource servers and transaction managers are located on different nodes of different distributed systems . Distributed transactions are usually used to ensure data consistency between different nodes in distributed systems .

There are generally several solutions for distributed transactions ：XA（2PC/3PC）、TCC

1.2.1 XA（2PC/3PC）

The most representative is by Oracle Tuxedo The system proposes XA Distributed transaction protocol .XA It's roughly divided into two parts ： Transaction manager and local resource manager . The local resource manager is usually implemented by database , such as Oracle、DB2 These commercial databases are all implemented XA Interface , The transaction manager is the global scheduler , Responsible for the submission and rollback of local resources .XA Agreements typically contain Two-phase commit （2PC） and Three stage commit （3PC） Two implementations . As the name suggests, two-stage submission is to carry out two-stage submission ： The first stage , Preparation stage ( Voting stage ); The second stage , Submission phase （ Execution phase ）. The implementation process is as follows ：

Two phase commit seems to provide atomic operations , But it has some defects , Three paragraph submission （3PC） It's a two paragraph submission （2PC） A kind of upgrade optimization , Interested can have an in-depth understanding , No more details here .

1.2.2 TCC

TCC（Try-Confirm-Cancel） yes Try、Commit、Cancel Abbreviations for the three instructions , Also known as Compensation Affairs , Its logical pattern is similar to XA Two-phase commit , The transaction flow is very similar , but 2PC Is applied to DB level ,TCC It can be understood as 2PC, We need to write business logic to implement .

TCC Its core idea is ：“ Register a confirmation for each operation （Try） And compensation （Cancel）”.

1.3 Message transaction

The so-called message transaction is a two-phase commit based on message queue , In essence, it is a special use of message queue , It puts local transaction and sending message in a distributed transaction , Ensure that either the local operation succeeds and the outgoing message succeeds , Or both fail .

Message queue based two-phase commit is often used in high concurrency scenarios , Split a distributed transaction into a message transaction （A Local operation of the system + Send a message ）+B Local operation of the system , among B The operation of the system is driven by messages , As long as the message transaction succeeds , that A The operation must be successful , The news must have come out , Now B Will receive a message to perform local operations , If the local operation fails , The news will be replayed , until B Successful operation , This is achieved in disguise A And B Of distributed transactions . The principle is as follows ：

Although the above plan can be completed A and B The operation of , however A and B Not strongly consistent , It is Final agreement （Eventually consistent） Of . And this is also satisfaction BASE Theoretical requirements . Here is an extension of ,BASE yes Basically Available（ Basic available ）、Soft state（ Soft state ） and Eventually consistent（ Final consistency ） Abbreviations of three phrases .BASE The theory is right CAP in AP（CAP It has been proved that a distributed system can only meet CAP Two of the three ） An extension of , Get availability by sacrificing strong consistency , In case of failure, it is allowed that some parts are not available but the core functions shall be available , Allow data to be inconsistent over time , But in the end it's consistent . Satisfy BASE Theoretical business , We call it “ Flexible business ”.

Two 、 Message semantics

In distributed systems , Any node may be abnormal or even down . The same is true in message queues , When Producer When producing messages , It could happen Broker Downtime unavailable , Or abnormal conditions such as sudden network interruption . According to, when an exception occurs Producer How messages are processed , The system can have the following three message semantics ：

2.1 At-least-once（ At least once ）

Producer By receiving Broker Of ACK（ Message confirmation ） Notification to ensure that the message is successfully written Topic. However , When Producer receive ACK Notification timeout , Or receive Broker Error message , Will try to resend the message . If Broker Just after successfully writing the message to Topic, But not yet Producer send out ACK Time goes down ,Producer The resend message will be written to Topic, Eventually, the message is repeatedly distributed to Consumer. namely ： Messages will not be lost , But it may be sent repeatedly .

2.2 At-most-once（ At most once ）

When Producer Receiving ACK Overtime , Or receive Broker Do not resend the message when an error message occurs , That may lead to the loss of this message , Not written to Topic in , And not by Consumer Consumption to . In some cases , In order to avoid repeated consumption , We can allow message loss to occur . namely ： The message may be lost , But it will never be sent repeatedly .

2.3 Exactly-once（ Exactly once ）

Exactly-once Semantics guarantees even if Producer Send the same message to the server multiple times , The server will only record once .Exactly-once Semantics is the most reliable , It is also the most difficult to understand .Exactly-once Semantics requires the message queue server , Only through the cooperation of message production end and consumer application end can . such as , When the consumer application successfully consumes and ACK After a message , Roll back the consumption site to the previous message ID, So from that news ID All messages in the future will be re consumed by the consumer application to . namely ： Messages will not be lost , It will not be sent repeatedly .