MySQL master-slave multi-source replication (3 master and 1 slave) setup and synchronization test

1. One 、MySQL Introduction to master-slave replication
   1. 1.1、MySQL Introduction to master-slave replication
   2. 1.2、 The benefits of master-slave replication
   3. 1.3、MySQL Master slave replication process
   4. 1.4、 Master slave Topology
2. Two 、3 Lord 1 Build from multi-source replication
   1. 2.1、MySQL Environment initialization
   2. 2.2、 Main library configuration
   3. 2.3、 Configuration from library
   4. 2.4、 Query multi-source replication
   5. 2.5、 Test multi-source replication
   6. 2.6、 Points of attention

One 、MySQL Introduction to master-slave replication

1.1、MySQL Introduction to master-slave replication

MySQL Master slave copy （MySQL Replication） Means from a MySQL master server (master) Copy data to another or more MySQL From the server (slaves) The process of . The main database's DDL and DML Operation through binary log （binlog） To the slave server （slave） On , Then re execute these logs on the slave server , So that the master and slave servers keep the data synchronized .
MySQL from 3.23 Version begins to provide the function of replication .

MySQL Of Replication It's a lot MySQL Database as master-slave synchronization scheme , It is widely used in all kinds of MySQL Higher performance 、 Higher reliability requirements .

1.2、 The benefits of master-slave replication

Master-slave replication has the following benefits ：

1. The data backup (Data Backup)
   Simply back up the database , Reduce the risk of data loss ,
2. Offline statistics
   It is used in reports and other occasions that do not require high data timeliness .
3. Load balancing (Load Balance)、 Read / write separation
   Mainly used in MySQL colony , Solve single point of failure or failover ; To reduce the load and risk of a single server , For example, realize the separation of reading and writing , It can make the server access load more balanced .
4. Data dissemination (Data DistributIOn)、 Disaster preparedness
   It is mainly used for multi data center or remote backup , Realize data distribution and synchronization .
5. High availability and data fault tolerance (High Availability and Failover)
   MySQL Self contained health monitoring and detection , According to the configured time interval , You can check whether the main library is working properly , Once the main database is found to be down or not working properly , You will choose the best standby database .

1.3、MySQL Master slave replication process

1.4、 Master slave Topology

Two 、3 Lord 1 Build from multi-source replication

2.1、MySQL Environment initialization

--  build 3 Lord 1 from --  Configure the network environment docker pull mysql:5.7.30docker network create --subnet=172.72.0.0/24 mysql-network--  Create parameter directory mkdir -p /lhrmysqltest2/master1/conf.dmkdir -p /lhrmysqltest2/master2/conf.dmkdir -p /lhrmysqltest2/master3/conf.dmkdir -p /lhrmysqltest2/slave/conf.d--  Delete the previous docker rm -f mysql5730M33265 mysql5730M33266 mysql5730M33267 mysql5730S33268--  Apply for master database 1docker run -d --name mysql5730M33265 \   -h master1 -p 33265:3306 --net=mysql-network --ip 172.72.0.10 \   -v /lhrmysqltest2/master1/conf.d:/etc/mysql/conf.d \   -e MYSQL_ROOT_PASSWORD=lhr \   mysql:5.7.30--  Apply for master database 2docker run -d --name mysql5730M33266 \   -h master2 -p 33266:3306 --net=mysql-network --ip 172.72.0.11 \   -v /lhrmysqltest2/master2/conf.d:/etc/mysql/conf.d \   -e MYSQL_ROOT_PASSWORD=lhr \   mysql:5.7.30--  Apply for master database 3docker run -d --name mysql5730M33267 \   -h master3 -p 33267:3306 --net=mysql-network --ip 172.72.0.12 \   -v /lhrmysqltest2/master3/conf.d:/etc/mysql/conf.d \   -e MYSQL_ROOT_PASSWORD=lhr \   mysql:5.7.30--  Apply from the Library docker run -d --name mysql5730S33268 \   -h slave1 -p 33268:3306 --net=mysql-network --ip 172.72.0.13 \   -v /lhrmysqltest2/slave/conf.d:/etc/mysql/conf.d \   -e MYSQL_ROOT_PASSWORD=lhr \   mysql:5.7.30--  Configure the main library 1 Parameters of cat > /lhrmysqltest2/master1/conf.d/my.cnf << "EOF"[mysqld]port=3306character_set_server=utf8mb4secure_file_priv=server-id = 573033265log-bin =binlog_format=rowexpire_logs_days = 30max_binlog_size  = 100Mbinlog-ignore-db = mysqlbinlog-ignore-db = information_schemabinlog-ignore-db = performance_schemabinlog-ignore-db = sysreplicate_ignore_db=information_schemareplicate_ignore_db=performance_schemareplicate_ignore_db=mysqlreplicate_ignore_db=sysgtid-mode=ONenforce-gtid-consistency=onskip_name_resolvereport_host=172.72.0.10EOF--  Configure the main library 2 Parameters of cat > /lhrmysqltest2/master2/conf.d/my.cnf << "EOF"[mysqld]port=3306character_set_server=utf8mb4secure_file_priv=server-id = 573033266log-bin = binlog_format=rowexpire_logs_days = 30max_binlog_size  = 100Mbinlog-ignore-db = mysqlbinlog-ignore-db = information_schemabinlog-ignore-db = performance_schemabinlog-ignore-db = sysreplicate_ignore_db=information_schemareplicate_ignore_db=performance_schemareplicate_ignore_db=mysqlreplicate_ignore_db=sysgtid-mode=ONenforce-gtid-consistency=ONskip_name_resolvereport_host=172.72.0.11EOF--  Configure the main library 3 Parameters of cat > /lhrmysqltest2/master3/conf.d/my.cnf << "EOF"[mysqld]port=3306character_set_server=utf8mb4secure_file_priv=server-id = 573033267log-bin = binlog_format=rowexpire_logs_days = 30max_binlog_size  = 100Mbinlog-ignore-db = mysqlbinlog-ignore-db = information_schemabinlog-ignore-db = performance_schemabinlog-ignore-db = sysreplicate_ignore_db=information_schemareplicate_ignore_db=performance_schemareplicate_ignore_db=mysqlreplicate_ignore_db=sysgtid-mode=ONenforce-gtid-consistency=ONskip_name_resolvereport_host=172.72.0.12EOF--  Configure the parameters of the slave Library cat > /lhrmysqltest2/slave/conf.d/my.cnf << "EOF"[mysqld]port=3306character_set_server=utf8mb4secure_file_priv=server-id = 573033268log-bin = binlog_format=rowexpire_logs_days = 30max_binlog_size  = 100Mbinlog-ignore-db = mysqlbinlog-ignore-db = information_schemabinlog-ignore-db = performance_schemabinlog-ignore-db = sysreplicate_ignore_db=information_schemareplicate_ignore_db=performance_schemareplicate_ignore_db=mysqlreplicate_ignore_db=sysgtid-mode=ONenforce-gtid-consistency=ONskip_name_resolvereport_host=172.72.0.13master-info-repository = tablerelay-log-info-repository = tableEOF--  Restart the host docker restart mysql5730M33265docker restart mysql5730M33266docker restart mysql5730M33267docker restart mysql5730S33268docker psdocker exec -it mysql5730M33265 bashdocker exec -it mysql5730M33265 mysql -uroot -plhrmysql -uroot -plhr -h192.168.66.35 -P33265 -e "select @@hostname,@@server_id,@@server_uuid"mysql -uroot -plhr -h192.168.66.35 -P33266 -e "select @@hostname,@@server_id,@@server_uuid"mysql -uroot -plhr -h192.168.66.35 -P33267 -e "select @@hostname,@@server_id,@@server_uuid"mysql -uroot -plhr -h192.168.66.35 -P33268 -e "select @@hostname,@@server_id,@@server_uuid"

2.2、 Main library configuration

--  stay 3 The master database respectively creates replication users mysql -uroot -plhr -h192.168.66.35 -P33265mysql -uroot -plhr -h192.168.66.35 -P33266mysql -uroot -plhr -h192.168.66.35 -P33267grant replication slave on *.* to repl@'%' identified by 'lhr';select user,host,grant_priv,password_last_changed,authentication_string from mysql.user; show master status \G;show slave hosts;select @@hostname,@@server_id,@@server_uuid;

2.3、 Configuration from library

--  Make the following configuration from the Library mysql -uroot -plhr -h192.168.66.35 -P33268--  Configure the main library 1 Copy path to the slave Library change master to master_host='172.72.0.10',master_port=3306,master_user='repl',master_password='lhr',master_auto_position=1  FOR CHANNEL 'Master1';--  Configure the main library 2 Copy path to the slave Library change master to master_host='172.72.0.11',master_port=3306,master_user='repl',master_password='lhr',master_auto_position=1  FOR CHANNEL 'Master2';--  Configure the main library 3 Copy path to the slave Library change master to master_host='172.72.0.12',master_port=3306,master_user='repl',master_password='lhr',master_auto_position=1  FOR CHANNEL 'Master3';--  Start all SLAVEmysql> START SLAVE;--  You can also start the channel to be synchronized separately mysql> START SLAVE FOR CHANNEL 'master1';mysql> START SLAVE FOR CHANNEL 'master2';mysql> START SLAVE FOR CHANNEL 'master3';

2.4、 Query multi-source replication

--  Query from the Library show slave status \G;--  If you want to view the detailed status of replication of a single channel , You can use the following command ：mysql> SHOW SLAVE STATUS FOR CHANNEL 'master1'\G;--  Query through table select a.master_log_pos,a.host,a.user_name,a.user_password,a.port,a.uuid,a.channel_name from  mysql.slave_master_info a;select * from mysql.slave_relay_log_info;select * from mysql.slave_worker_info;select * from mysql.gtid_executed;--  stay  performance_schema  In the library , Provides some views related to replication , You can view replication related information .select * from performance_schema.replication_applier_configuration;select * from performance_schema.replication_applier_status;select * from performance_schema.replication_applier_status_by_coordinator;select * from performance_schema.replication_applier_status_by_worker;select * from performance_schema.replication_connection_configuration;select * from performance_schema.replication_connection_status;select * from performance_schema.replication_group_member_stats;select * from performance_schema.replication_group_members;--  Merge SQLselect rcc.CHANNEL_NAME,rcc.`HOST`,rcc.`PORT`,rcc.`USER`,rcc.CONNECTION_RETRY_COUNT,rcc.CONNECTION_RETRY_INTERVAL,rcs.SOURCE_UUID,rcs.THREAD_ID,rcs.SERVICE_STATE,rcs.COUNT_RECEIVED_HEARTBEATS,rcs.LAST_HEARTBEAT_TIMESTAMP,rcs.LAST_ERROR_NUMBER,rcs.LAST_ERROR_MESSAGE,rcs.LAST_ERROR_TIMESTAMPfrom performance_schema.replication_connection_configuration rcc,      performance_schema.replication_connection_status rcswhere rcc.CHANNEL_NAME=rcs.CHANNEL_NAME;--  Thread query SELECT *FROM performance_schema.threads aWHERE a.`NAME` IN ( 'thread/sql/slave_IO', 'thread/sql/slave_sql' ) or a.PROCESSLIST_COMMAND in ('Binlog Dump','Binlog Dump GTID') ;SELECT * FROM  information_schema.`PROCESSLIST` a where a.USER='system user' or a.command  in ('Binlog Dump','Binlog Dump GTID') ;MySQL [(none)]> SELECT *    -> FROM performance_schema.threads a    -> WHERE a.`NAME` IN ( 'thread/sql/slave_IO', 'thread/sql/slave_sql' ) or a.PROCESSLIST_COMMAND in ('Binlog Dump','Binlog Dump GTID') ;+-----------+----------------------+------------+----------------+------------------+------------------+----------------+---------------------+------------------+--------------------------------------------------------+------------------+------------------+------+--------------+---------+-----------------+--------------+| THREAD_ID | NAME                 | TYPE       | PROCESSLIST_ID | PROCESSLIST_USER | PROCESSLIST_HOST | PROCESSLIST_DB | PROCESSLIST_COMMAND | PROCESSLIST_TIME | PROCESSLIST_STATE                                      | PROCESSLIST_INFO | PARENT_THREAD_ID | ROLE | INSTRUMENTED | HISTORY | CONNECTION_TYPE | THREAD_OS_ID |+-----------+----------------------+------------+----------------+------------------+------------------+----------------+---------------------+------------------+--------------------------------------------------------+------------------+------------------+------+--------------+---------+-----------------+--------------+|        29 | thread/sql/slave_io  | FOREGROUND |              4 | root             | 172.72.0.1       | NULL           | Connect             |              252 | Waiting for master to send event                       | NULL             |               28 | NULL | YES          | YES     | NULL            |           75 ||        30 | thread/sql/slave_sql | FOREGROUND |              5 | root             | 172.72.0.1       | NULL           | Connect             |              325 | Slave has read all relay log; waiting for more updates | NULL             |               28 | NULL | YES          | YES     | NULL            |           76 ||        31 | thread/sql/slave_io  | FOREGROUND |              6 | root             | 172.72.0.1       | NULL           | Connect             |              252 | Waiting for master to send event                       | NULL             |               28 | NULL | YES          | YES     | NULL            |           77 ||        32 | thread/sql/slave_sql | FOREGROUND |              7 | root             | 172.72.0.1       | NULL           | Connect             |              322 | Slave has read all relay log; waiting for more updates | NULL             |               28 | NULL | YES          | YES     | NULL            |           78 ||        33 | thread/sql/slave_io  | FOREGROUND |              8 | root             | 172.72.0.1       | NULL           | Connect             |              252 | Waiting for master to send event                       | NULL             |               28 | NULL | YES          | YES     | NULL            |           79 ||        34 | thread/sql/slave_sql | FOREGROUND |              9 | root             | 172.72.0.1       | NULL           | Connect             |              320 | Slave has read all relay log; waiting for more updates | NULL             |               28 | NULL | YES          | YES     | NULL            |           80 |+-----------+----------------------+------------+----------------+------------------+------------------+----------------+---------------------+------------------+--------------------------------------------------------+------------------+------------------+------+--------------+---------+-----------------+--------------+6 rows in set (0.09 sec)

2.5、 Test multi-source replication

--  Test multiple sources -- mysql -uroot -plhr -h192.168.66.35 -P33265create database master1;use master1;CREATE TABLE `test1` (`id` int(11) DEFAULT NULL,`count` int(11) DEFAULT NULL);insert into test1 values(1,1);-- mysql -uroot -plhr -h192.168.66.35 -P33266create database master2;use master2;CREATE TABLE `test2` (`id` int(11) DEFAULT NULL,`count` int(11) DEFAULT NULL);insert into test2 values(2,2);-- mysql -uroot -plhr -h192.168.66.35 -P33267create database master3;use master3;CREATE TABLE `test3` (`id` int(11) DEFAULT NULL,`count` int(11) DEFAULT NULL);insert into test3 values(3,3);--  Query from the library -- mysql -uroot -plhr -h192.168.66.35 -P33268show databases;SELECT * FROM master1.test1;SELECT * FROM master2.test2;SELECT * FROM master3.test3;

Query from the library ：

MySQL [(none)]> show databases;+--------------------+| Database           |+--------------------+| information_schema || master1            || master2            || master3            || mysql              || performance_schema || sys                |+--------------------+7 rows in set (0.05 sec)MySQL [(none)]> SELECT * FROM master1.test1;+------+-------+| id   | count |+------+-------+|    1 |     1 |+------+-------+1 row in set (0.06 sec)MySQL [(none)]> SELECT * FROM master2.test2;+------+-------+| id   | count |+------+-------+|    2 |     2 |+------+-------+1 row in set (0.05 sec)MySQL [(none)]> SELECT * FROM master3.test3;+------+-------+| id   | count |+------+-------+|    3 |     3 |+------+-------+1 row in set (0.05 sec)MySQL [(none)]>

2.6、 Points of attention

1、 The initial configuration takes a long time , You need to master The data of dump Come down , Again source To slave On .

2、 Each... Needs to be considered master Data growth frequency ,slave The data growth frequency of is the sum of these data . If it's too high , Will result in a large number of disks IO, Cause data update delay , The most serious problem is that it will affect the normal query .

3、 If a library with the same name exists in multiple primary database instances , Then the tables of the library with the same name will be put into one library ;

4、 If the table names in the library with the same name are the same and the structure is the same , Then the data will be merged ; If the structure is different , The first one is effective .