[teacher Zhao Yuqiang] MySQL flashback

MySQL DBA Or developers , Sometimes the data will be deleted or updated by mistake , If it is an online environment and has a great impact , You need to be able to roll back quickly . The traditional recovery method is to use the backup to duplicate the instance , Then apply to remove the error sql After binlog To recover data . This method is time-consuming and laborious , It even needs to be shut down for maintenance , Not suitable for fast rollback . There are also teams that use LVM Snapshot to shorten recovery time , But the disadvantage of snapshots is that they will affect mysql Performance of .MySQL Flashback (flashback) utilize binlog Roll back directly , Fast recovery without downtime .

Principle of flashback

MySQL binlog With event In the form of , Recorded MySQL server From enable binlog All change information since , Can help reproduce all the changes between .MySQL introduce binlog There are two main purposes ： One is for master-slave replication ; Second, some backup and restore operations need to be reapplied binlog. There are three options binlog Format , Each has its own advantages and disadvantages ：

• statement： be based on SQL Statement mode ,binlog Small amount of data , However, some statements and functions may cause data inconsistency or even errors during the copying process ;
• row： Row based mode , What is recorded is the complete change of the line . Very safe , however binlog It will be much larger than the other two modes ;
• mixed： Mixed mode , Choose yes according to the sentence statement still row Pattern ;

utilize binlog Flashback , Need to put binlog Format to row. Use the following statement to view the current binlog The pattern of .

show global variables like "%binlog_format%";

Flashback practice

Real flashback scene , The key is to quickly filter out the ones that really need rollback SQL. We use open source tools binlog2sql Come and practice .binlog2sql Comment by meituan DBA The team ( Shanghai ) Produce , Do fast rollback in online environment for many times .

① install binlog2sql Tools

 First installation Python Tool management table pip
yum -y install epel-release
yum -y install python-pip

 install binlog2sql
git clone https://github.com/danfengcao/binlog2sql.git && cd binlog2sql
pip install -r requirements.txt

② Flashback case practice

1、 We use the previous employee table data , Establish a separate database

create database testflashback;
use testflashback;
source /root/tools/scott.sql

2、 Misoperation , Do the following .

start transaction;
delete from emp where sal>3000;
update emp set sal=6000;
delete from emp where job='CLERK';
commit;

3、 Check out the current binlog file

show master logs;

4、 Abreast of the times binlog File is mysql-binlog.000001. Our goal is to filter out those that need rollback SQL, Because the misoperator only knows the approximate misoperation time , We first do a filter according to the time . Just parse testflashback library emp surface .( notes ： If there are more than one sql Misoperation , Then it generates binlog It may be distributed in multiple files , Multiple files need to be parsed )

python binlog2sql/binlog2sql.py -uroot -pWelcome_1 \
--start-file='mysql-binlog.000001' > /root/tools/raw.sql

The above statement will train emp All of the watch binlog journal . If the approximate time range can be determined , You can use parameters –start-datetime and –stop-datetime To filter . for example ：

--start-datetime='2016-12-26 11:44:00' --stop-datetime='2016-12-26 11:50:00'

Analytic processing of binlog as follows ：

5、 According to the location information , We have identified misoperation sql From the same transaction , The exact location is 14956-16791 Between (binlog2sql For the same transaction, the same start position). Then filter according to the location , Use -B Option generate rollback sql, Check the rollback sql Whether it is right .( notes ： In the real world , Generated rollback SQL Further screening is often required . combination grep、 Editor, etc )

python binlog2sql/binlog2sql.py -uroot -pWelcome_1 --start-file='mysql-binlog.000001' \
--start-position=14956 --stop-position=16791 -B > /root/tools/rollback.sql

The following is the generated flashback statement ：

6、 Confirm the rollback with the business party sql That's all right. , Execute rollback statement . Sign in mysql, Confirm that the rollback is successful .

mysql -uroot -pWelcome_1 < /root/tools/rollback.sql

7、 Check whether the data is recovered