Summary of solutions to cross system data consistency problems

hello, Hello everyone , I'm Zhang Zhang ,「 The road to the advanced architecture 」 The public name author .

At present, with the popularization of micro service construction , There are more and more cross system data interactions , The problem of cross system data consistency has become increasingly prominent , How to effectively ensure the consistency of cross system data ？

The purpose of this paper is to summarize the experience of solving problems in precipitation work , Sort out the empirical methods to solve the problem of cross system data inconsistency .

1、 Why is there a problem of cross system data consistency ？

When it comes to data consistency , It is easy to think of transaction operations in the database .

The atomicity and persistence of transactions can ensure that , Operate on multiple pieces of data , Either they all succeed , Or they all failed . So inside a system , We can naturally use database transactions to ensure data consistency . But in today's microservices , When an operation involves multiple databases across multiple systems , There is no way to solve it with a single database transaction .

Another common situation is ： System services with dependencies , For example, the business side and the client side （ The business side is responsible for the production data , The client is responsible for displaying the data ）, Data synchronization is required to ensure data consistency across system services , What kind of data synchronization mode is used in many cases , It is very important to ensure the timeliness of data application . 

2、 Analysis of the difficulties of consistency problem

In order to better describe and understand the problem , We use a case to illustrate ：

Suppose there is an order system and an inventory system , In actual business, the creation of orders will be accompanied by the reduction of inventory . The two systems are deployed for microservice , Its application data is also stored in an independent database , The two systems communicate through the network .

2.1 CAP principle

CAP refer to Consistency（ Uniformity ）、Availability（ Usability ）、Partition tolerance（ Partition tolerance ）.

give up A （ Usability ） To protect CP

Specifically, after a communication failure occurs , The application will go into a blocking state , Try to resume communication with the inventory system until all data processing is completed . This scheme gives priority to data integrity , But the user experience of this scheme is very poor , Because the user will be waiting until all operations are completed .

CAP Itself is mutually exclusive , Only two of the three , about CA、AP、CP All have their own application scenarios , It should be selected according to the actual situation .

• CA Because partition tolerance is not considered , So all its operations need to be completed in the same process （ That is what we often call monomer application ）;
• AP Because of the abandonment of data consistency , It is suitable for projects with low data requirements but emphasizing user experience , Such as blog 、 News, etc ;
• CP Instead, you give up availability , It is suitable for trading systems with high data requirements , Such as bank transactions 、 E-commerce order transactions, etc , Even if the user waits for a long time , It is also necessary to ensure the integrity and reliability of data .

CAP Application of the principle in practical projects , For Internet applications , It is also undesirable to completely abandon data consistency for the sake of user experience , After all, data is the foundation of application .

So what's the solution ？

There are many measures to ensure final consistency , It mainly includes ： Distributed transactions and TCC Consistency program .

MySQL In fact, there is a two-phase commit distributed transaction scheme （MySQL XA）, However, this scheme has serious performance problems .

such as , Transactions between one database and multiple databases XA Transaction performance may differ 10 times . in addition , stay XA In the process of transaction processing, it will occupy lock resources for a long time , So we didn't consider this plan at first .

Here it is , Let's mainly discuss TCC Consistency program .

2.2 TCC Consistency program

TCC Is a data consistency scheme , We will divide the original interface into three interfaces ：

• Try Interface is used to check data 、 Reserve business resources .
• Confirm The interface is used to confirm the actual business operation 、 Update business resources .
• Cancel An interface is a release Try Resources reserved in the interface .

stay TCC in , It divides the distributed processing process into two stages ：

1、Try Is the first stage , Used to try and lock resources ;

2、 If the resource is locked successfully , The second stage begins Confirm Submit to complete data operation ;

3、 If the resource lock fails , The second stage will proceed Cancel Roll back data ;

TCC What are the precautions in the implementation process ？

1） stay Try Do as many things as you can at this stage

Put most of the business logic in Try Stages to complete , because TCC At the beginning of the design Confirm or Cancel It must be successful , So don't include any business code or remote communication in the second phase , Release frozen resources only through the simplest code .

2） guarantee Confirm or Cancel Successful implementation

If Confirm or Cancel An error occurred while executing , The specific application will also try to retry the execution to ensure successful execution , This process may be performed many times update sentence , So pay attention to the idempotency of the code .

3）Confirm or Cancel Implement the failed scheme

Under the minimum probability ,Confim or Cancel Failed after multiple retries , The data will eventually be inconsistent , This requires the development of additional data integrity verification procedures to remedy or manual supplementary recording .

TCC In the final analysis, it is a kind of theoretical design , It needs the support of the corresponding framework implemented by the manufacturer .

stay Java Famous in the open source field TCC Framework has ：ByteTCC、Hmily、Tcc-transaction And Seata.

3、 Practice of effective data synchronization scheme

Problem description ： Let's take the previous case scenario , Data needs to be synchronized from the order system to the inventory system .

Three types of data synchronization schemes commonly used to solve data consistency ： Real time synchronization 、 Timing synchronization 、 Manual sync .

3.1 Real time synchronization

Real time synchronization can be done from the database 、 Two levels of application processing are used to solve .

3.1.1 Database level

The data synchronization of database is generally adopted , Master slave solution , When master( Lord ) When the data in the library changes , Changes will be synchronized in real time to slave( from ) library .

advantage ：

• Expand the load capacity of the database horizontally .
• Fault tolerance , High availability ,Failover( Failed to switch )/High Availability
• The data backup .

How to achieve master-slave consistency

About MySQL The main synchronization of master-slave replication is binlog journal , Three threads are involved , One runs on the master node （log dump thread）, The other two (I/O thread, SQL thread) Running on the slave node , As shown in the figure below : 

（1） Master node binary log dump Threads

When the slave node connects to the master node , The master node creates a log dump Threads , Used for sending binlog The content of . In the reading binlog Operating time , This thread will change the binlog Lock , When the read is complete , Before sending to the slave , The lock will be released .

（2） From the node I/O Threads

When executed from a node start slave After the command , From the node, a I/O Threads are used to connect to the master node , Request the updated binlog.I/O Thread received master binlog dump After the update from the process , Keep it locally relay-log（ relay logs ） in .

（3） From the node SQL Threads

SQL The thread is responsible for reading relay log The content in , Parse into specific operations and execute , Finally, ensure the consistency of master-slave data .

3.1.2 API call

A business data operation , Multiple parties need to be called API Realize the synchronization of real-time data .

The disadvantages are obvious , Mainly in ：

1） It takes a long time to process , Multiple parties need to be called serially API And wait for a response , The user experience is poor ;

2） There is a certain chance of data inconsistency （ individual API Call error 、 No response, etc ）.

3.2 Asynchronous synchronization

3.2.1 Asynchronous message queuing

Message Queue（MQ）, Message Queuing Middleware

MQ By separating the sending and receiving of messages, the asynchronous and decoupling of applications can be realized , meanwhile MQ Shield the underlying complex communication protocols , Defines a set of application layer 、 Simpler communication protocol .

application MQ The advantages of ： decoupling , Peak shaving , Data dissemination .

In business system design , We often have a platform system A, It correlates and synchronizes the docking of many systems （ System B、C、D etc. ）.

utilize MQ It can solve the problems of system docking and data synchronization , At the same time, the stability of the docking system and other demands can be ignored . 

3.2.2 Timing synchronization

Timed tasks are not uncommon in systems , The main purpose is to process data regularly or perform an operation , Such as closing the order regularly , Or scheduled backup .

Common scheduled tasks are divided into 2 Kind of ：

1） The first one is ： Fixed time execution , Ensure data synchronization and calibration

Such as ： Execute every minute , Once a day .

2） The second kind ： How long to delay execution , After the action occurs , How long will it take to execute the task

Such as ：15 Close the order payment status in minutes ,24 Close the order and release the inventory after an hour .

4、 Application experience summary

Technology still needs to solve practical problems , Application scenarios are critical , Don't use technology for the sake of technology , In the final analysis, technology serves the application scenarios and industrial landing .

In the process of software design , There is no need to deliberately apply solutions that look tall , And when you need to introduce , Consider development at the same time 、 Maintenance cost and cost performance ratio of corresponding performance improvement , Otherwise, the gain is not worth the loss .

1、 Any architectural solution is evolving

Any data synchronization has no advantages or disadvantages , Have their own suitable application scenarios .

2、 The purpose of architecture is to solve business problems

Architecture solutions to current problems , At the same time, it is easy to expand and maintain , That is an excellent architecture .

With the development of Internet , The demand for cross system data consistency application will become more and more urgent , The real pain points of cross region and cross system scenarios will become more and more clear , I hope our research and Exploration on cross system data consistency can give you an idea and reference .

I hope today's explanation will be helpful to you , thank you ！

Thanks for reading

---

Adhere to the original and share valuable dry goods technical articles ！

Official account , Free learning materials

If you think it's good , Welcome to follow and forward ~     

This article is from WeChat official account. - The road to the advanced architecture （jiagou_jingjin）.
If there is any infringement , Please contact the [email protected] Delete .
Participation of this paper “OSC Source creation plan ”, You are welcome to join us , share .