Technology sharing | MySQL: how about copying half a transaction from the database?

author ： Hu Chengqing

Love can be born DBA Team members , Good at fault analysis 、 performance optimization , Personal blog ：https://www.jianshu.com/u/a95ec11f67a8, Welcome to discuss .

In this paper, the source ： Original contribution

* Produced by aikesheng open source community , Original content is not allowed to be used without authorization , For reprint, please contact the editor and indicate the source .

---

Copy exception

In the process of copying , The main database fails or the network is interrupted , Will cause slave io thread interrupt , It is possible that only half a transaction is copied from the Library . For example, the transactions executed by the main database are as follows ：

begin;
insert 1;
insert 2;
commit;

Received from library binlog It may only contain part of the transaction , such as ：

• situation 1： Contains only begin;
• situation 2： Contains only begin;insert 1;
• situation 3： Contains only begin;insert 1;insert 2;

From library slave sql thread Play back this part binlog after , Will wait for slave io thread Read the remaining... From the main library binlog, And before that sql The thread plays back the half transaction , It's the same as we manually perform half a transaction , Neither commit nor rollback .

How should we deal with this anomaly ？

• When slave io thread recovery , What to do ？
• When slave io thread Can't recover , What to do ？

Experimental process

The test method ：

##1.  Used in slave Library  tc  Simulate network latency , Intended to make reading  binlog  It's slowing down 
tc qdisc add dev eth0 root netem delay 3000ms 3000ms 100%

##2.  Execute a multi statement transaction in the main database 
begin;
update t2 set pad='4' where id < 40;
update t2 set pad='5' where id < 50;
update t2 set pad='6' where id < 60;
commit;

##3.  Execute... In the main library  commit  After success , Use immediately  iptables  Cut off the network between master and slave 
iptables -A OUTPUT -d 172.16.21.4 -j DROP
iptables -A INPUT -s 172.16.21.4 -j DROP

In this way, the phenomenon we can observe from the library is ：

• One of them worker The thread state is Waiting for an event from Coordinator, This status indicates work The thread has finished its work and is waiting Coordinator （ Coordination thread ） Assign a new relay log event , But it also shows that it is executing update t2 set pad='5' where id < 50, This is a contradiction 1：

• show slave status Output in progress ,Retrieved_Gtid_Set And Executed_Gtid_Set equal （ signify sql The thread has played back all relay log）, But above worker The thread is playing back again SQL , This is a contradiction 2：

Finally, we passed relay log Compaction hammer , You can see this transaction relay log Incomplete , To update t2 set pad='5' where id < 50; This Rows_query event It's over ：

When slave io thread Can't recover

If slave io thread Can't recover for a long time , that sql Threads can't wait for the rest binlog, Never commit or rollback , Will always hold the lock of this transaction ：

If it is caused by the failure of the main library slave io thread abnormal , It is likely to switch between master and slave , After upgrading from the library to the main ,SQL Transaction locks held by threads may block business requests .

It should be at this time stop slave stop it sql Threads , Let the transaction rollback release the lock . It should be noted that ： In this case stop slave Will wait for 60 second （ etc. slave io thread Receive the remaining binlog）,60 Seconds before it stops sql Threads ：

When slave io thread recovery

slave io thread After an abnormal interrupt ,sql Threads work normally ,sql The thread executes some transactions , And will wait io Thread sends new binlog.slave io thread After thread recovery , If it is based on GTID Copy , From the present GTID The transaction begins to retrieve the complete binlog, The current transaction will be rolled back first from the Library , Then replay the newly received binlog.