Configuring MySQL master-slave and implementing semi synchronous replication in docker environment

One 、 The goal is

docker Environment mysql One master and two slaves , Data volume mounting is required to realize data persistence , Semi synchronous replication is adopted between master and slave replication . The reader of this article is required to be familiar with docker I have known , Content for the docker The command is no longer detailed .

Two 、 Environmental preparation

2.1 Pull the mirror image

docker pull mysql:5.7.32

2.2 Configuration file preparation

Here, start a container at random , Here is mainly to get mysql Configuration file for mysqld.cnf（/etc/mysql/mysql.conf.d/mysqld.cnf）, It is convenient to mount the configuration file from the outside . Someone may ask why you don't mount the directory directly , I have tried, which will cause the configuration file to be lost , The specific reason is not clear , If you know something, you can leave a message .

docker run --name mysql_3306 -d -e MYSQL_ROOT_PASSWORD=123456 -p 3306:3306 mysql:5.7.32

Go inside the container ：

docker exec -it eca958feb10d /bin/bash
mysql -uroot -p

see mysql Data file and log file path ：

show variables like 'general_log_file';           #  Log file path 
show variables like 'log_error';                  #  Error log file path 
show variables like 'slow_query_log_file';        #  Slow query log file path 
show variables like 'datadir';                    #  View the database data file path 

You can see the log 、 Slow query logs and data files are located in the container /var/lib/mysql, Error log is not configured , Therefore, you can configure the error log file to /var/lib/mysql in , Then the host only needs to mount this directory . In addition, you need to prepare a directory for storing configuration files . Here I mainly prepare the following contents ：

The configuration file ：/home/docker/mysql_3306/conf
Data files ：/home/docker/mysql_3306/lib/mysql

The container mysql_3306 Copy the configuration file of to the configuration file directory of the host ：

docker cp mysql_3306:/etc/mysql/mysql.conf.d/mysqld.cnf /home/docker/mysql_3306/conf

Modify the configuration file error log path to /var/lib/mysql

vim /home/docker/mysql_3306/conf/mysqld.cnf    #  Edit profile mysqld.cnf

#log-error = /var/log/mysql/error.log          #  Before the change 
log-error = /var/lib/mysql/error.log           #  After modification 

Come here , Get the required configuration file , You can delete the container .

3、 ... and 、 Master slave configuration

3.1 Node to start

Start three nodes respectively , One is used as the master node , Two are used as slave nodes .

//  Master node 
docker run --name mysql_3306 -d -e MYSQL_ROOT_PASSWORD=123456 -v /home/docker/mysql_3306/lib/mysql:/var/lib/mysql -v /home/docker/mysql_3306/conf:/etc/mysql/mysql.conf.d -p 3306:3306 mysql:5.7.32

// From the node 1
docker run --name mysql_3307 -d -e MYSQL_ROOT_PASSWORD=123456 -v /home/docker/mysql_3307/lib/mysql:/var/lib/mysql -v /home/docker/mysql_3307/conf:/etc/mysql/mysql.conf.d -p 3307:3306 mysql:5.7.32

// From the node 2
docker run --name mysql_3308 -d -e MYSQL_ROOT_PASSWORD=123456 -v /home/docker/mysql_3308/lib/mysql:/var/lib/mysql -v /home/docker/mysql_3308/conf:/etc/mysql/mysql.conf.d -p 3308:3306 mysql:5.7.32

After startup, you can see the corresponding data file in the corresponding directory, which indicates that the data mounting is normal .

In addition, don't forget to modify the permissions of the database , I open all connections by default , We can't do this in production , Do a good job of permission control .

grant all privileges on *.* to 'root'@'%' identified by '123456' with grant option;

3.2 Master slave replication principle

The master-slave configuration is based on binlog Log implementation . There are three main steps ：

1. The main database records the changes of the database to Binlog Log file
2. Read... From the main library Binlog Log file information is written to Relay Log In the relay log
3. Reading relay log information from the library is done from the library Replay, Update slave database information

In the above three processes , It involves Master Of BinlogDump Thread and Slave Of I/O Thread、SQL Thread, Their functions are as follows ：

1.master The server changes to the database are recorded in binlog in ,binlogDump Thread After receiving the write request , Read binlog Push information to slave Of I/O Thread.
2.slave Of I/O Thread Will read to binlog Information is written locally Relay Log in .
3.slave Of SQL Thread detected Relay Log Change request for , analysis Relay Log The content in is executing from the library .

All operations above are asynchronous , The principle is as follows ：

Of course , These are all mysql It's achieved internally , And for the upper echelons , We only need to modify the corresponding configuration file .

3.3 Configuration of master-slave replication

master：

# log bin
server-id=100
log_bin=mysql-bin
binlog_format=MIXED
sync_binlog=1
expire_logs_days=7
binlog-ignore-db=mysql
binlog_ignore_db=information_schema
binlog_ignore_db=performation_schema
binlog_ignore_db=sys

slave：

# log bin
server-id=200
master_info_repository=table
relay_log_info_repository=table

see master The state of the node

show master status;

File and Positon respectively binlog Location of the file and current offset , You need to configure the library later （ You need to specify which file to read from the main library and where to start reading ）.File The default is 1 Start , I have performed some operations later , So there are some changes in the values here .

Main library information read from library configuration ,MASTER_LOG_FILE and MASTER_LOG_POS Corresponding to the above main database File and Positon.

CHANGE MASTER TO 
MASTER_HOST='127.0.0.1',
MASTER_PORT=3306,
MASTER_USER='root',
MASTER_PASSWORD='123456',
MASTER_LOG_FILE='mysql-bin.000001',
MASTER_LOG_POS=154 
for channel '100';

##  The startup name is 100 Synchronization channel of 
start slave for channel '100';

Come here , The master-slave synchronization is completed , The test shows that the changes under the master database can be synchronized to the changes under the slave database .

Four 、 Semi-synchronous replication

The default master-slave replication mode is asynchronous replication , There may be data loss （ Master node hung , The slave node has not had time to read the master node binlog）. Semi synchronous replication is to solve this problem , To ensure data consistency between nodes .

mysql Give Way master Wait at a certain point in time slave Node ack news , Received ack Before committing the transaction , This is also the foundation of semi synchronous replication , Before we learn about semi synchronization, we can review the master-slave auxiliary master The process of node transactions . as everyone knows ,mysql The transaction is committed in two phases , The process is as follows ：

When master No need to pay attention slave Whether to accept binlog When an event is , That is, the traditional master-slave replication , When master Need to wait in the third step slave return ack when , That is, semi synchronous replication （after-commit）. There is another kind of enhanced semi synchronization , I won't discuss it here .

The above is the sequence diagram of semi synchronous replication given on the official website . The implementation of semi synchronous replication is also very simple , Need to rely on plug-in implementation , At the same time, modify the configuration file , Specify the imported plug-ins and synchronization methods .

# master Lower installation 
INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so';

# mysqld.cnf
# master Semi synchronous replication configuration 
plugin-load="rpl_semi_sync_master=semisync_master.so"
rpl_semi_sync_master_enabled=1
#  Semi synchronous timeout , Default 10s, Semi synchronous replication will turn to asynchronous replication when it times out 
rpl_semi_sync_master_timeout=1000

# slave Lower installation 
INSTALL PLUGIN rpl_semi_sync_slave SONAME 'semisync_slave.so';

# mysqld.cnf
# slave Semi synchronous replication configuration 
#  Semi synchronous replication configuration 
plugin-load="rpl_semi_sync_slave=semisync_slave.so"
rpl_semi_sync_slave_enabled=1

5、 ... and 、 summary

That's about docker The configuration mysql One master, two slaves and semi synchronous replication , All operations are verified , On this basis, you can also configure multiple masters and multiple slaves . If you have any questions about the article or think there is something wrong with the article , Welcome to leave a message / Private communication ~