Read it once: talk about MySQL master-slave

Preface

Hello everyone , I am a A little boy picking up snails . During the interview , Interviewers often ask MySQL Master-slave . I'll talk to you today MySQL Principal and subordinate .

Database master-slave concept 、 advantage 、 purpose
Database master-slave replication principle
Lord 、 Master-slave 、 The difference between active and standby
MySQL How to ensure the consistency of master and slave
Causes and solutions of database master-slave delay
Talk about the high availability scheme of database
official account ： A little boy picking up snails

1. Database master-slave concept 、 advantage 、 purpose

Master slave database What does that mean , Master means master library , From means from the library . The main database provides external read-write operations , Provide read operations from the library .

Why does a database need a master-slave architecture ？

High availability , Real time disaster recovery , For failover . For example, the main library hangs up , Can be cut from library .
Read / write separation , Provide inquiry service , Reduce the pressure of main reservoir , Lifting performance
The backup data , Avoid affecting the business .

2. Database master-slave replication principle

Master slave replication principle , in short , In three steps ：

The main database has bin log Binary , All additions, deletions and modifications are recorded SQL sentence .（binlog Threads ）
From the database to the main database bin log Of documents SQL Copy the statement to your own relay log relay log（io Threads ）
From the database relay log Redo log file , Do these again sql sentence .（Sql Execute thread ）

The detailed master-slave replication process is shown in the figure ：

In the figure above, the master-slave replication process is divided into five steps ：

Update of main database SQL(update、insert、delete) Written to binlog
Initiate connection from library , Connect to the main library .
At this time, the main library creates a binlog dump thread, hold bin log Content sent from library .
After starting from the library , Create a I/O Threads , Read from the main library bin log Content and write to relay log
From the library will also create a SQL Threads , from relay log Read the contents inside , from ExecMasterLog_Pos The location starts executing the read update event , Write the update to slave Of db

3. Lord 、 Master-slave 、 The difference between active and standby

Database master ： Both are master databases , At the same time, it provides external read-write operation . The client accesses any one . There is bidirectional synchronization of data .

Database master slave ： One is the main database , At the same time, it provides external read-write operation . One is From database , Provide external read operation . Data synchronization from master database to slave database .

Database master / standby ： One is the main database , At the same time, it provides external read-write operation . One is For the library , For backup purposes only , No external reading and writing , When the host hangs up, it takes its place . Data is synchronized from the primary database to the standby database .

From warehouse and standby warehouse , Namely slave The library has different functions, so it is called differently . commonly slave Libraries will provide external reading functions , therefore , You hear more everyday, that is Master-slave .

4. MySQL How to ensure the consistency of master and slave

We learn about databases Master slave replication principle after , come to know Slave Library Get and execute the main library binlog journal , You can keep the data consistent with the main database . Why is that ？ What can lead to inconsistencies ？

4.1 Long link

What if the master database and slave database are interrupted in the process of synchronizing data , The data will be lost . Therefore, a connection is maintained between the master database and the slave database Long link , There is a thread inside the main library , Dedicated to this... From the library Long link Of .

4.2 binlog Format

binlog There are three formats for logs , Namely statement,row and mixed.

If it is statement Format ,binlog The record is SQL Original text , If the primary and secondary indexes are inconsistent , It may cause inconsistency in the main database . Let's analyze . Suppose the main database deletes this SQL（ among a and create_time They all have indexes ） as follows ：

delete from t where a > '666' and create_time<'2022-03-01' limit 1;
 Copy code

We know , The data selected a Index and select create_time Indexes , Last limit 1 The data is usually different . So there will be this situation ： stay binlog = statement When the format , The main library is executing this SQL when , Using the index a, And the slave library is executing this SQL when , Index is used create_time. Finally, the master-slave data is inconsistent .

How to solve this problem ？

You can put binlog Format changed to row.row Format binlog journal , It's not recorded SQL original text , It's two event:Table_map and Delete_rows.Table_map event Describe the table to be operated ,Delete_rows event Used to define the behavior to be deleted , Record the specific number of lines deleted .row Format binlog The primary key of the record to be deleted ID Information , Therefore, there will be no inconsistency between master and slave .

But if SQL Delete 10 Ten thousand rows of data , Use row The format will take up a lot of space ,10 Ten thousand pieces of data are binlog Inside , Write binlog It takes a lot of time IO. however statement Format binlog May cause data inconsistency , So design MySQL My uncle came up with a compromise ,mixed Format binlog. So-called mixed The format is actually row and statement Format mix , When MySQL When judging that the data may be inconsistent , Just use row Format , Otherwise, use statement Format .

5. Causes and solutions of database master-slave delay

Master-slave delay How do you define it ？ There are three time points related to master-slave data synchronization

The master database executes a transaction , write in binlog, We remember this moment as T1;
The master database synchronizes the data to the slave database , After receiving this from the library binlog The moment of , Record as T2;
The transaction is executed from the library , This moment is recorded as T3.

The so-called master-slave delay , In fact, it means the same thing , The difference between the time when the slave database is executed and the time when the master database is executed , namely T3-T1.

What can lead to master-slave delay ？

If the slave library is located on a machine larger than the master library The performance of the machine is poor , Will cause master-slave delay , This kind of situation is easier to solve , Just choose A machine of the same specification as the master-slave Library Just fine .
If High pressure from the reservoir , It will also lead to master-slave delay . For example, the main database directly affects the business , You may use Relatively restrained , So in general All queries are hit from the database 了 , As a result, queries from the library consume a lot of CPU, Affect synchronization speed , Finally, the master-slave delay . In this case , You can do it One master, many followers The architecture of , That is, connect several more from the library to share the pressure of reading . in addition , You can also put binlog Access to Hadoop This kind of system , Let them provide the ability to query .
Large transactions can also lead to master-slave delays . If a transaction is executed, it needs to 10 minute , After the main database is executed , To execute from the library , Finally, this transaction may result in a delay from the library 10 Five minutes . Daily development , Why do we emphasize , Don't do it all at once delete Too much SQL, It needs to be done in batches , In fact, it's also to avoid big things . in addition , Large watch DDL sentence , It can also lead to big things , Let's pay attention to daily development .
Network delay It will also lead to master-slave delay , In this case, you can only optimize your network , For example, bandwidth 20M Upgrade to 100M Similar meaning, etc .
If there are too many slave databases, it will also lead to master-slave delay , Therefore, avoid copying too many slave nodes . The data from the database is generally in 3-5 Individual is appropriate .
Low version of the MySQL Only single thread replication is supported , If the concurrency of the main database is high , It's too late to send it from the library , It will lead to a delay . You can switch to a higher version of Mysql, Can support multi-threaded replication .

6. Talk about the high availability scheme of data library

Dual active standby high availability
A master from
One master, many followers
MariaDB Synchronous multi host cluster
database middleware

6.1 Dual active standby high availability

Architecture description ： Two machines A and B,A Give priority to the library , Read and write ,B For backup , Backup data only . If A Library failure ,B The library becomes the main library and is responsible for reading and writing . After fixing the fault ,A Become a standby warehouse , Main library B Synchronize data to the standby database A
advantage ： When a machine fails, it can switch automatically , Simple operation .
shortcoming ： Only one library is working , High reading and writing pressure , Failed to separate reading and writing , Concurrency is also limited

6.2 A master from

Architecture description : Two machines A and B,A Give priority to the library , Read and write ,B For from the library , Read the data . If A Library failure ,B The library becomes the main library and is responsible for reading and writing . After fixing the fault ,A Become a slave , Main library B Synchronize data to slave Library A.
advantage ： Reading from the library is supported , Shared the pressure of the main reservoir , Increased concurrency . When a machine fails, it can switch automatically , Simple operation .
shortcoming ： A slave library , Concurrency support is still not enough , And there are two machines , There is still a chance of simultaneous failure , Not high enough availability .

6.3 One master, many followers

Architecture description : One master library and multiple slave libraries ,A Give priority to the library , Read and write ,B、C、D For from the library , Read the data . If A Library failure ,B The library becomes the main library and is responsible for reading and writing ,C、D Responsible for reading . After fixing the fault ,A Also become from the library , Main library B Synchronize data to slave Library A.
advantage ： Multiple slave libraries support reading , Shared the pressure of the main reservoir , Significantly improve the concurrency of reading .
shortcoming ： Only one host writes , Therefore, the concurrency of writing is not high

6.4 MariaDB Synchronous multi host cluster

Architecture description : There is an agent layer to realize load balancing , Multiple databases can be read and written at the same time ; Various databases can be connected through Galera Replication Method for data synchronization , Theoretically, the data of each library is completely consistent .
advantage ： The concurrency of reading and writing is significantly improved , You can read and write from any node , It can automatically eliminate the faulty nodes , With high reliability .
shortcoming ： The amount of data is not particularly large . Avoid getting stuck in big things , If a cluster node slows down , Other nodes will also slow down .

6.5 database middleware

Architecture description ：mycat Fragmentation storage , Each partition is configured with a master-slave cluster .
advantage ： High availability solution for high concurrency and high data volume
shortcoming ： The maintenance cost is relatively high .