JD home programmers delete databases and run away. Talk about binlog, the killer of MySQL data backup

Official account ： Micro Technology （ID：weiguanjishu） Originality is not easy. , Kneel down and thank you for your attention ~

Hello everyone , I am a Tom Brother ~

We all know , Data is very important

I often see some jokes on the Internet , Programmers of a company are dissatisfied with their work , Delete the library and run away , The boss suffered heavy losses , No tears to cry . This is not another recent case , Jingdong came home and the programmer deleted the library and ran away on the day he left ！

So is there any solution ？

Even if the database is really deleted , There are also backup data , Can recover quickly . It can even achieve real-time hot standby , Even if the internal blow up, external users will not perceive , It was calm .

MySQL As a popular database , stay The data backup 、 High availability Very competitive in , today , Let's focus on

What is? MySQL The main equipment

Situation 1 ：

• Business operation of the client , read 、 Write Access to the main library

• The main library through some mechanism , Synchronize the data to the standby database in real time

• The main library for some reasons , Unable to respond to the request of the client

Situation two ：

• Complete the active / standby switching

• Client read and write , You are accessing the standby database （ At this time, the standby database is upgraded to the new primary database ）

that , The core of this Data synchronization How is it realized ？

Master slave synchronization principle

1、 Execute in the standby database change master command , Bind the information of the main library

mysql> CHANGE MASTER TO MASTER_HOST = '192.168.1.1', MASTER_USER = 'repl', MASTER_PASSWORD = 'replpassword', MASTER_PORT = 3306, MASTER_AUTO_POSITION = 1, MASTER_RETRY_COUNT = 0, MASTER_HEARTBEAT_PERIOD = 10000; 

• MASTER_HOST ：master Host name （ or IP Address ）

• MASTER_PORT ：mysql Instance port number

• MASTER_USER： user name

• MASTER_PASSWORD： password

• MASTER_AUTO_POSITION： If carried out change master to When using MASTER_AUTO_POSITION = 1,slave Connect master Will use based on GTID Replication agreement for

• MASTER_RETRY_COUNT： Number of reconnections

• MASTER_HEARTBEAT_PERIOD： Copy heartbeat cycle

https://www.docs4dev.com/docs/zh/mysql/5.7/reference/change-master-to.html

2、 Standby database execution start slave command , The standby database starts two threads ：I/O thread and SQL thread

3、master Main library , There are data updates , Write the event type of this update to the main library binlog In file

4、 The main library will create log dump Threads , notice slave There are data updates

5、slave, towards master Node log dump The thread requests a specified binlog A copy of the file location , And will request back binlog Save to local Relay log In the relay log

6、slave Open one more SQL Threads Read Relay log journal , Parse out the commands in the log , And implement , So as to ensure the data synchronization of the primary and standby databases

binlog What are the formats

Now? , Let's take a closer look binlog journal .

binlog There are three formats ：row、statement、mixed

Next , Let's start an experiment ：

First create a table

CREATE TABLE `person` (
  `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT ' Since the primary key ',
  `income` bigint(20) NOT NULL COMMENT ' income ',
  `expend` bigint(20) NOT NULL COMMENT ' spending ',
  PRIMARY KEY (`id`),
  KEY `idx_income` (`income`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COMMENT=' Personal income statement ';

Insert 4 Bar record ：

insert into person values(50,500,500);
insert into person values(60,600,600);
insert into person values(70,700,700);
insert into person values(80,800,800);

see binlog Pattern ：

View what is currently being written binlog file ：

see binlog The content in , So let's see row Pattern

show binlog events in 'mysql-bin.000001';

explain ：

• SET @@SESSION.GTID_NEXT='ANONYMOUS’

• BEGIN Start a transaction

• Table_map Which record library has been updated 、 Which watch

• Write_rows Record what you did , See in detail binlog Need help mysqlbinlog Tools .

• COMMIT /* xid=157 */ End a transaction

lookup binlog The physical location of the file ：

[email protected]:/# find / -name mysql-bin.000001
/var/lib/mysql/mysql-bin.000001

With the help of mysqlbinlog command , Check the details ：

mysqlbinlog -vv mysql-bin.000001 --start-position=2986;

The content in the red box indicates that the insert command has been executed ,insert into person values(80,800,800);

among ,@1、@2、@3 Express surface person The first few fields of , No original name , To save space .

modify binlog Format , Set to STATEMENT , Look at the log format ：

set global binlog_format='STATEMENT';

After setting , Need to quit mysql Reconnect the , To see it take effect

show binlog events in 'mysql-bin.000001';

As we can see from the picture , When binlog_format=statement when ,binlog It records SQL The original text of the sentence .

among ,use tomge ： It means to switch to the corresponding database first

If you want to view from the specified location binlog, Can increase from Optional parameters , as follows ：

show binlog events in 'mysql-bin.000001'  from 5168;

statement And row contrast ：

statement Format binlog The record is sql sentence ;row Format binlog The record is event（Table_map,Write_rows,Delete_rows）

When binlog stay statement Under format , The record is sql sentence , The index may be used when the main library is executed A; But when synchronizing to the standby database , Maybe it used Indexes B.

Different indexes , Same article sql sentence , The running results may also be different .

For this scenario , We suggest that row Format binlog.

Even if we use a belt where Conditions （ Such as ：income>720） Of delete sentence , but binlog Records the primary key to be deleted id（id =80 ）, So there will be no mistakes .

mixed Format Of binlog What is a ？

because statement Format binlog It may cause the main library 、 The data synchronization between standby databases is inconsistent , So we will use row Format .

however ,row Format takes up a lot of space , Write binlog It also takes up a lot of IO resources .

therefore , The official proposed a mixed Mixed mode , Integrates the advantages of both .

The contents are as follows ：

• mysql Will judge automatically statement Format , Whether it will cause the inconsistency between active and standby

• If statement The format will cause the inconsistency between the active and standby , Automatic use row Format .

• If statement The format will not cause inconsistency between active and standby , Then use statement Format ,

Restore data

Of course , We also suggest that MySQL Of binlog Set to row Pattern , Because it can be used for data recovery . Let's see insert、update、delete Three DML How to recover data .

1、delete：

When we execute delete On command , If binlog_row_image Set up ‘FULL’, that Delete_rows Inside , Contains the values of all fields of the deleted row .

If deleted by mistake , because binlog The values of all fields are recorded , Reverse execution insert That's all right. .

When binlog_row_image Set to MINIMAL, Record only key information , such as id=80

2、insert：

row Under format ,binlog Meeting Record insert All field values for .

If misoperation , Just find the corresponding row based on these values , Re execution delete Just operate

3、update：

row Under format ,binlog Meeting Record update Before the change 、 The modified whole row of data .

If misoperation , Just overwrite with the data before modification .

Recover data through commands ：

If you want to perform data recovery , You can use the following command

mysqlbinlog mysql-bin.000001  --start-position=1  --stop-position=3000 | mysql -h192.168.0.1 -P3306 -u$user -p$pwd;

take mysql-bin.000001 File location from 1 To 3000 Of binlog stay 192.168.0.1 Playback on the machine's database , Restore .

---

The authors introduce ：Tom Brother , The school recruited Ali , I also took it during the period Baidu 、 Huawei 、 ZTE 、 tencent etc. 6 Jiada factory offer,P7 technician , I've published patents ,CSDN Blogger . Taobao double for many times 11 Programme of activities , Focus on microservices 、 High concurrency 、 Distributed architecture 、 High availability and other fields , Like to tap the bright spots of open source framework design .

1、Java High frequency examination site

2、 computer network

3、LeetCode Algorithm

As an interviewer of a large front-line factory ,Tom I collect many high-frequency interview questions from big factories , Scan the QR code below , Official account 【 Micro Technology 】, The background to reply “666”、“ Algorithm ”, Get relevant learning materials for free .

Has helped many small partners around to enter the byte 、 Ali and other big factories , You are also welcome to find Tom I'm chatting , Technical communication , Circle of friends , Life is no longer lonely