MySQL master-slave replication, read-write separation

1. MySQL Master slave copy

MySQL The database supports master-slave replication by default , There is no need to rely on other technologies , Simply configure it in the database .

1.1 Introduce

MySQL Master-slave replication is an asynchronous replication process , The bottom layer is based on Mysql The database comes with Binary log function . Just one or more MySQL database （slave, namely Slave Library ） From the other MySQL database （master, namely Main library ） Copy logs , Then parse the log and apply it to itself , Final realization Slave Library The data and Main library The data are consistent .MySQL Master slave replication is MySQL The database has its own functions , No third party tools required .

Binary log ：

​ Binary log （BINLOG） It records all the DDL（ Data definition language ） Statement and DML（ Data manipulation language ） sentence , But it does not include data query statements . This log plays an extremely important role in data recovery in case of disaster ,MySQL Master-slave replication of , Through this binlog Realized . Default MySQL The log is not opened .

MySQL The principle of master-slave replication is as follows ：

MySQL The replication process is divided into three steps ：

1). MySQL master Write data changes to binary log ( binary log)

2). slave take master Of binary log Copy to its trunk log （relay log）

3). slave Redo events in relay log , Change the data to reflect its own data

1.2 build

1.2.1 preparation

Prepare two servers in advance , And install... In the server MySQL, The server information is as follows ：

database	IP	Database version
Master	192.168.200.200	5.7.25
Slave	192.168.200.201	5.7.25

And do the following preparations on the two servers :

1). Firewall opening 3306 Port number

firewall-cmd --zone=public --add-port=3306/tcp --permanent

firewall-cmd --zone=public --list-ports

2). And start up the two database servers ：

systemctl start mysqld

Sign in MySQL, Verify whether it starts normally

1.2.2 Main library configuration

The server ： 192.168.200.200

1). modify Mysql Profile of the database vim /etc/my.cnf

Add configuration at the bottom :

log-bin=mysql-bin   #[ must ] Enable binary logging 
server-id=200       #[ must ] The server is unique ID( The only can )

2). restart Mysql service

Execution instruction ：

 systemctl restart mysqld

3). Create a user for data synchronization and authorize

Sign in mysql, And execute the following command , Create users and authorize ：

GRANT REPLICATION SLAVE ON *.* to 'xiaoming'@'%' identified by '[email protected]';

notes ： above SQL The function of is to create users
user xiaoming ,
The password for [email protected] ,
And give xiaoming User grants REPLICATION SLAVE jurisdiction . It is often used to establish the user permissions required for replication , That is to say slave Must be master Authorize the user with this permission , Can be copied by this user .

MySQL Password complexity description :

​ at present mysql5.7 The default password verification policy level is MEDIUM , This level requires the password to be composed of : Numbers 、 Lowercase letters 、 Capital 、 Special characters 、 Length at least 8 position

4). Sign in Mysql database , see master sync

Do the following SQL, Record the results File and Position Value

show master status;

notes ： above SQL The function of is to check Master The state of , After that SQL Do not perform any more operations after

1.2.3 Configuration from library

The server ： 192.168.200.201

1). modify Mysql Profile of the database vim /etc/my.cnf

server-id=201 	#[ must ] The server is unique ID

2). restart Mysql service

systemctl restart mysqld

3). Sign in Mysql database , Set the main database address and synchronization location

mysql>change master to master_host='192.168.200.200',master_user='xiaoming',master_password='[email protected]',master_log_file='mysql-bin.000001',master_log_pos=154;

mysql>start slave;

Parameter description ：

​ A. master_host : The main library IP Address

​ B. master_user : User name to access the master database for master-slave replication ( Created in the main library above )

​ C. master_password : The password corresponding to the user name that accesses the master database for master-slave replication

​ D. master_log_file : Which log file to start synchronization from ( The above query master The status shows )

​ E. master_log_pos : Specify the location of the log file from which to start synchronization ( The above query master The status shows )

4). View the status of the slave database

show slave status\G;

Then through the... In the status information Slave_IO_running and Slave_SQL_running It can be seen whether the master-slave synchronization is ready , If both parameters are Yes, Indicates that master-slave synchronization has been configured .

MySQL Command line techniques ：

​ \G : stay MySQL Of sql Add after statement \G, Indicates that the query results are printed by column , You can make each field print to a separate line . The structure to be found rotates 90 The degree becomes vertical ;

1.3 test

Master slave replication environment , It's set up , Next , We can go through Navicat Connect two MySQL The server , To test . When testing , We just need to be in the main library Master Perform the operation , View from library Slave Whether to synchronize the data in the past .

1). stay master Create database , Refresh slave See if you can sync the past

2). stay master Created under the created data user surface , Refresh slave See if you can sync the past

3). stay master Of user Insert a piece of data into the table , Refresh slave See if you can sync the past

2. Read write separation cases

2.1 Background introduction

In the face of increasing system access , Database throughput is facing a huge bottleneck . For an application system with a large number of concurrent read operations and fewer write operations at the same time , Split the database into Main library and Slave Library , The main database is responsible for the transaction of addition, deletion and modification , The slave library is responsible for handling query operations , It can effectively avoid row lock caused by data update , The query performance of the whole system has been greatly improved .

• Separate by reading and writing , You can reduce the access pressure of a single database , Improve access efficiency , It can also avoid single machine failure .

• The structure of master-slave replication , It's done , Need to introduce the use of a new technology ShardingJDBC, adopt java Code to complete read-write separation , In execution select When querying from the library , And in the execution insert、update、delete When , Operate the main library

2.2 ShardingJDBC

Sharding-JDBC Position as lightweight Java frame , stay Java Of JDBC Additional services provided by layer . It uses the client direct connection database , With jar Service in package form , No additional deployment and dependencies , Can be understood as an enhanced version of JDBC drive , Fully compatible with JDBC And all kinds of ORM frame .

Use Sharding-JDBC You can easily realize the separation of database reading and writing in the program .

Sharding-JDBC It has the following characteristics ：

• Apply to any based on JDBC Of ORM frame , Such as ：JPA, Hibernate, Mybatis, Spring JDBC Template Or use it directly JDBC.

• Support database connection pool of any third party , Such as ：DBCP, C3P0, BoneCP, Druid, HikariCP etc. .

• Support arbitrary implementation JDBC Standardized database . At present, we support MySQL,Oracle,SQLServer,PostgreSQL And any compliance with SQL92 Standard database .

rely on

<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
    <version>4.0.0-RC1</version>
</dependency>

2.3 Database environment

Create a database in the main library rw, And create a table , After the database and table structure are created, it will be automatically synchronized to the slave database ,SQL The statement is as follows ：

create database rw default charset utf8mb4;

use rw;

CREATE TABLE `user` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `name` varchar(255) DEFAULT NULL,
  `age` int(11) DEFAULT NULL,
  `address` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

2.5 Read write separation configuration

increase shardingJdbc Of maven coordinate

<dependency>
    <groupId>org.apache.shardingsphere</groupId>
    <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
    <version>4.0.0-RC1</version>
</dependency>

application.yml Add the configuration of data source in

spring:
  shardingsphere:
    datasource:
      names:
        master,slave
      #  The main data source 
      master:
        type: com.alibaba.druid.pool.DruidDataSource
        driver-class-name: com.mysql.cj.jdbc.Driver
        url: jdbc:mysql://192.168.200.200:3306/rw?characterEncoding=utf-8
        username: root
        password: root
      #  From a data source 
      slave:
        type: com.alibaba.druid.pool.DruidDataSource
        driver-class-name: com.mysql.cj.jdbc.Driver
        url: jdbc:mysql://192.168.200.201:3306/rw?characterEncoding=utf-8
        username: root
        password: root
    masterslave:
      #  Load balancing 、 Read write separation configuration 
      load-balance-algorithm-type: round_robin # polling 
      #  The final data source name 
      name: dataSource
      #  Main database data source name 
      master-data-source-name: master
      #  From the list of library data source names , Multiple commas separate 
      slave-data-source-names: slave
    props:
      sql:
        show: true # Turn on SQL Show , Default false
  # allow Bean Define overlay ,sharding frame JDBC and Druid frame JDBC, Because both data source objects need to be created Bean conflict 
  main:
    allow-bean-definition-overriding: true

Configuration analysis :

3). stay application.yml Add configuration

spring:  
  main:
    allow-bean-definition-overriding: true

The purpose of this configuration item , If there is a with the same name in the current project bean, Later defined bean Will override the previously defined .

If this item is not configured , An error will be reported after the project is started ：

• The error message indicates , In a statement org.apache.shardingsphere.shardingjdbc.spring.boot Under bag SpringBootConfiguration Medium dataSource This bean An error occurred when , The reason is that there is a with the same name dataSource Of bean stay com.alibaba.druid.spring.boot.autoconfigure Under bag DruidDataSourceAutoConfigure When the class is loaded, it has been declared .

And what we need is shardingjdbc Under bag dataSource, So we need to configure the above properties , Let the later loaded overwrite the first loaded .

2.6 test

We use shardingjdbc To achieve read-write separation , Just use the above simple configuration . Once configured , We can restart the service , adopt postman To visit controller Methods , To complete the addition, deletion, modification and query of user information , We can go through debug And log to view each addition, deletion, modification and query operation , Which data source is used , Which database is connected .

Using annotations @Slf4j

1). Save the data

Console output log , You can see the operation master Main library ：

2). Modifying data

Console output log , You can see the operation master Main library ：

3). Query data

Console output log , You can see the operation slave Main library ：

4). Delete data

Console output log , You can see the operation master Main library ：