The new version of Alibaba Seata finally solves the idempotence, suspension and empty rollback problems of TCC mode

author ： Zhu Jinjun

Hello everyone , I'm brother Jun .

Today, let's talk about Alibaba Seata The new version （1.5.1） How to solve TCC Idempotence in mode 、 Suspension and empty rollback problems .

TCC review

TCC Pattern is the most classic distributed transaction solution , It divides distributed transactions into two phases to execute ,try Stage reserves resources for each branch transaction , If all branch transactions reserve resources successfully , entering commit Phase commit global transaction , If a node fails to reserve resources, enter cancel Phase rollback global transaction .

With traditional orders 、 stock 、 Take account services for example , stay try Phase tries to reserve resources , Insert order 、 Deducting the inventory 、 Deduction amount , These three services all commit local transactions , Resources can be transferred to the intermediate table . stay commit Stage , And then try The resources reserved in the phase are transferred to the final table . And in the cancel Stage , hold try Release the resources reserved in the stage , For example, return the account amount to the customer's account .

Be careful ：try The phase must be the one to commit the local transaction , For example, deduct the order amount , The money must be deducted from the customer's account , If not , stay commit There is not enough money in the customer's account at this stage , There will be problems .

try-commit

try In this phase, resources are reserved first , And then in commit Phase deduction resources . Here's the picture ：

try-cancel

try In this phase, resources are reserved first , Failed to deduct inventory when reserving resources, resulting in global transaction rollback , stay cancel Stage release resources . Here's the picture ：

TCC advantage

TCC The biggest advantage of the model is high efficiency .TCC Patterns in try Locking resources in a phase is not really locking , Instead, the local transaction is actually committed , Reserve resources in the intermediate state , There is no need to block waiting , Therefore, the efficiency is higher than other modes .

meanwhile TCC The pattern can also be optimized as follows ：

Asynchronous submission

try After the successful stage , Do not enter immediately confirm/cancel Stage , Instead, it thinks that the global transaction is over , Start a scheduled task to execute asynchronously confirm/cancel, Deduct or release resources , This will greatly improve the performance .

Same as library mode

TCC There are three roles in the pattern ：

• TM： Manage global affairs , Including starting global transactions , Submit / Roll back global transactions ;
• RM： Manage branch transactions ;
• TC: Manage the status of global and branch transactions .

The following figure comes from Seata Official website ：

TM When opening a global transaction ,RM You need to TC Send registration message ,TC Save the state of the branch transaction .TM When requesting commit or rollback ,TC You need to RM Send commit or rollback messages . So in a distributed transaction that contains two branch transactions ,TC and RM There are four times RPC.

The optimized process is shown in the following figure ：

TC Save the state of the global transaction .TM When opening a global transaction ,RM No need to talk to TC Send registration message , Instead, the branch transaction state is saved locally .TM towards TC After sending a commit or rollback message ,RM The asynchronous thread first finds out the uncommitted branch transactions saved locally , And then to TC Send a message to get （ The local branch office is located in ） Global transaction status , To decide whether to commit or rollback the local transaction .

After this optimization ,RPC Fewer times 50%, Performance improvement .

RM Code example

Take inventory service as an example ,RM The inventory service interface code is as follows ：

@LocalTCC
public interface StorageService {

    /**
     *  Deducting the inventory 
     * @param xid  overall situation xid
     * @param productId  product id
     * @param count  Number 
     * @return
     */
    @TwoPhaseBusinessAction(name = "storageApi", commitMethod = "commit", rollbackMethod = "rollback", useTCCFence = true)
    boolean decrease(String xid, Long productId, Integer count);

    /**
     *  Commit transaction 
     * @param actionContext
     * @return
     */
    boolean commit(BusinessActionContext actionContext);

    /**
     *  Roll back the transaction 
     * @param actionContext
     * @return
     */
    boolean rollback(BusinessActionContext actionContext);
}

adopt @LocalTCC This annotation ,RM During initialization, you will send a message to TC Register a branch transaction . stay try Stage approach （decrease Method ） There's one on @TwoPhaseBusinessAction annotation , Here we define the branch transaction resourceId,commit Methods and cancel Method ,useTCCFence This attribute will be discussed in the next section .

TCC Existing problems

TCC The three major problems in patterns are idempotence 、 Hang and empty rollback . stay Seata1.5.1 In the version , A transaction control table is added , Table name is tcc_fence_log To solve this problem . And in the last section @TwoPhaseBusinessAction The attributes mentioned in the comments useTCCFence To specify whether to enable this mechanism , The default value of this attribute is false.

tcc_fence_log Build the predicative sentence as follows （MySQL grammar ）：

CREATE TABLE IF NOT EXISTS `tcc_fence_log`
(
    `xid`           VARCHAR(128)  NOT NULL COMMENT 'global id',
    `branch_id`     BIGINT        NOT NULL COMMENT 'branch id',
    `action_name`   VARCHAR(64)   NOT NULL COMMENT 'action name',
    `status`        TINYINT       NOT NULL COMMENT 'status(tried:1;committed:2;rollbacked:3;suspended:4)',
    `gmt_create`    DATETIME(3)   NOT NULL COMMENT 'create time',
    `gmt_modified`  DATETIME(3)   NOT NULL COMMENT 'update time',
    PRIMARY KEY (`xid`, `branch_id`),
    KEY `idx_gmt_modified` (`gmt_modified`),
    KEY `idx_status` (`status`)
) ENGINE = InnoDB
DEFAULT CHARSET = utf8mb4;

idempotent

stay commit/cancel Stage , because TC No response received from branch transaction , Need to try again , This requires that branch transactions support idempotence .

Let's see how the new version solves the problem . The following code is in TCCResourceManager class ：

@Override
public BranchStatus branchCommit(BranchType branchType, String xid, long branchId, String resourceId,
         String applicationData) throws TransactionException {
 TCCResource tccResource = (TCCResource)tccResourceCache.get(resourceId);
 // Omit judgment 
 Object targetTCCBean = tccResource.getTargetBean();
 Method commitMethod = tccResource.getCommitMethod();
 // Omit judgment 
 try {
  //BusinessActionContext
  BusinessActionContext businessActionContext = getBusinessActionContext(xid, branchId, resourceId,
   applicationData);
  Object[] args = this.getTwoPhaseCommitArgs(tccResource, businessActionContext);
  Object ret;
  boolean result;
  // annotation  useTCCFence  Whether the property is set to  true
  if (Boolean.TRUE.equals(businessActionContext.getActionContext(Constants.USE_TCC_FENCE))) {
   try {
    result = TCCFenceHandler.commitFence(commitMethod, targetTCCBean, xid, branchId, args);
   } catch (SkipCallbackWrapperException | UndeclaredThrowableException e) {
    throw e.getCause();
   }
  } else {
   // Ellipsis logic 
  }
  LOGGER.info("TCC resource commit result : {}, xid: {}, branchId: {}, resourceId: {}", result, xid, branchId, resourceId);
  return result ? BranchStatus.PhaseTwo_Committed : BranchStatus.PhaseTwo_CommitFailed_Retryable;
 } catch (Throwable t) {
  // Omit 
  return BranchStatus.PhaseTwo_CommitFailed_Retryable;
 }
}

The above code can see , When executing the branch transaction commit method , First judgement useTCCFence Whether the property is true, If true, Then go TCCFenceHandler Class commitFence Logic , Otherwise, follow the normal submission logic .

TCCFenceHandler Class commitFence Method is called TCCFenceHandler Class commitFence Method , The code is as follows ：

public static boolean commitFence(Method commitMethod, Object targetTCCBean,
          String xid, Long branchId, Object[] args) {
 return transactionTemplate.execute(status -> {
  try {
   Connection conn = DataSourceUtils.getConnection(dataSource);
   TCCFenceDO tccFenceDO = TCC_FENCE_DAO.queryTCCFenceDO(conn, xid, branchId);
   if (tccFenceDO == null) {
    throw new TCCFenceException(String.format("TCC fence record not exists, commit fence method failed. xid= %s, branchId= %s", xid, branchId),
      FrameworkErrorCode.RecordAlreadyExists);
   }
   if (TCCFenceConstant.STATUS_COMMITTED == tccFenceDO.getStatus()) {
    LOGGER.info("Branch transaction has already committed before. idempotency rejected. xid: {}, branchId: {}, status: {}", xid, branchId, tccFenceDO.getStatus());
    return true;
   }
   if (TCCFenceConstant.STATUS_ROLLBACKED == tccFenceDO.getStatus() || TCCFenceConstant.STATUS_SUSPENDED == tccFenceDO.getStatus()) {
    if (LOGGER.isWarnEnabled()) {
     LOGGER.warn("Branch transaction status is unexpected. xid: {}, branchId: {}, status: {}", xid, branchId, tccFenceDO.getStatus());
    }
    return false;
   }
   return updateStatusAndInvokeTargetMethod(conn, commitMethod, targetTCCBean, xid, branchId, TCCFenceConstant.STATUS_COMMITTED, status, args);
  } catch (Throwable t) {
   status.setRollbackOnly();
   throw new SkipCallbackWrapperException(t);
  }
 });
}

You can see that in the code , When committing a transaction, you will first judge tcc_fence_log Whether there are records in the table , If there's a record , Judge the transaction execution status and return . In this way, if it is judged that the transaction status is already STATUS_COMMITTED, Will not submit again , It guarantees idempotency . If tcc_fence_log There are no records in the table , Then insert a record , For later retry .

Rollback The logic of commit similar , Logical in class TCCFenceHandler Of rollbackFence Method .

Empty rollback

Here's the picture , The account service is a cluster of two nodes , stay try Stage account services 1 This node has failed ,try Stage without considering retry , The global transaction must go to the end state , In this way, you need to execute on the account service once cancel operation , This idles a rollback operation .

Seata Our solution is to try Stage Go to tcc_fence_log Table inserts a record ,status The field value is STATUS_TRIED, stay Rollback Stage judge whether the record exists , If it doesn't exist , No rollback operation is performed . The code is as follows ：

//TCCFenceHandler  class 
public static Object prepareFence(String xid, Long branchId, String actionName, Callback<Object> targetCallback) {
 return transactionTemplate.execute(status -> {
  try {
   Connection conn = DataSourceUtils.getConnection(dataSource);
   boolean result = insertTCCFenceLog(conn, xid, branchId, actionName, TCCFenceConstant.STATUS_TRIED);
   LOGGER.info("TCC fence prepare result: {}. xid: {}, branchId: {}", result, xid, branchId);
   if (result) {
    return targetCallback.execute();
   } else {
    throw new TCCFenceException(String.format("Insert tcc fence record error, prepare fence failed. xid= %s, branchId= %s", xid, branchId),
      FrameworkErrorCode.InsertRecordError);
   }
  } catch (TCCFenceException e) {
   // Omit 
  } catch (Throwable t) {
   // Omit 
  }
 });
}

stay Rollback The processing logic of the phase is as follows :

//TCCFenceHandler  class 
public static boolean rollbackFence(Method rollbackMethod, Object targetTCCBean,
         String xid, Long branchId, Object[] args, String actionName) {
 return transactionTemplate.execute(status -> {
  try {
   Connection conn = DataSourceUtils.getConnection(dataSource);
   TCCFenceDO tccFenceDO = TCC_FENCE_DAO.queryTCCFenceDO(conn, xid, branchId);
   // non_rollback
   if (tccFenceDO == null) {
    // Do not execute rollback logic 
    return true;
   } else {
    if (TCCFenceConstant.STATUS_ROLLBACKED == tccFenceDO.getStatus() || TCCFenceConstant.STATUS_SUSPENDED == tccFenceDO.getStatus()) {
     LOGGER.info("Branch transaction had already rollbacked before, idempotency rejected. xid: {}, branchId: {}, status: {}", xid, branchId, tccFenceDO.getStatus());
     return true;
    }
    if (TCCFenceConstant.STATUS_COMMITTED == tccFenceDO.getStatus()) {
     if (LOGGER.isWarnEnabled()) {
      LOGGER.warn("Branch transaction status is unexpected. xid: {}, branchId: {}, status: {}", xid, branchId, tccFenceDO.getStatus());
     }
     return false;
    }
   }
   return updateStatusAndInvokeTargetMethod(conn, rollbackMethod, targetTCCBean, xid, branchId, TCCFenceConstant.STATUS_ROLLBACKED, status, args);
  } catch (Throwable t) {
   status.setRollbackOnly();
   throw new SkipCallbackWrapperException(t);
  }
 });
}

updateStatusAndInvokeTargetMethod Method execution sql as follows ：

update tcc_fence_log set status = ?, gmt_modified = ?
    where xid = ? and  branch_id = ? and status = ? ;

It can be seen that tcc_fence_log It's recorded in the table status The field value is from STATUS_TRIED Change it to STATUS_ROLLBACKED, If the update is successful , Execute rollback logic .

Hang

Hang up is because of network problems ,RM I didn't receive it at first try Instructions , But it was carried out Rollback after RM I got it again try Command and reserve resources successfully , At this time, the global transaction has ended , Finally, the reserved resources cannot be released . Here's the picture ：

Seata The solution to this problem is to execute Rollback Method first judge tcc_fence_log Whether there is a current xid The record of , If not, to tcc_fence_log Table inserts a record , Status is STATUS_SUSPENDED, And no more rollback operations . The code is as follows ：

public static boolean rollbackFence(Method rollbackMethod, Object targetTCCBean,
         String xid, Long branchId, Object[] args, String actionName) {
 return transactionTemplate.execute(status -> {
  try {
   Connection conn = DataSourceUtils.getConnection(dataSource);
   TCCFenceDO tccFenceDO = TCC_FENCE_DAO.queryTCCFenceDO(conn, xid, branchId);
   // non_rollback
   if (tccFenceDO == null) {
       // Insert anti hang record 
    boolean result = insertTCCFenceLog(conn, xid, branchId, actionName, TCCFenceConstant.STATUS_SUSPENDED);
    // Ellipsis logic 
    return true;
   } else {
    // Ellipsis logic 
   }
   return updateStatusAndInvokeTargetMethod(conn, rollbackMethod, targetTCCBean, xid, branchId, TCCFenceConstant.STATUS_ROLLBACKED, status, args);
  } catch (Throwable t) {
   // Ellipsis logic 
  }
 });
}

And then execute try In the phase method, the first step is to tcc_fence_log Table inserts a current xid The record of , This causes primary key conflicts . The code is as follows ：

//TCCFenceHandler  class 
public static Object prepareFence(String xid, Long branchId, String actionName, Callback<Object> targetCallback) {
 return transactionTemplate.execute(status -> {
  try {
   Connection conn = DataSourceUtils.getConnection(dataSource);
   boolean result = insertTCCFenceLog(conn, xid, branchId, actionName, TCCFenceConstant.STATUS_TRIED);
   // Ellipsis logic 
  } catch (TCCFenceException e) {
   if (e.getErrcode() == FrameworkErrorCode.DuplicateKeyException) {
    LOGGER.error("Branch transaction has already rollbacked before,prepare fence failed. xid= {},branchId = {}", xid, branchId);
    addToLogCleanQueue(xid, branchId);
   }
   status.setRollbackOnly();
   throw new SkipCallbackWrapperException(e);
  } catch (Throwable t) {
   // Omit 
  }
 });
}

Be careful ：queryTCCFenceDO Method sql Used in for update, So you don't have to worry Rollback Method cannot get tcc_fence_log Table records but cannot be judged try The execution result of the local transaction in the phase .

summary

TCC Patterns are the most used patterns for distributed transactions , But idempotent 、 Hang and empty rollback have always been TCC Patterns need to be considered ,Seata In the framework of 1.5.1 The version perfectly solves these problems .

Yes tcc_fence_log The operation of tables also needs to consider the control of transactions ,Seata Proxy data sources are used , send tcc_fence_log Table operation and RM Business operations are performed in the same local transaction , In this way, local operation and tcc_fence_log The operation of both succeeded or failed .