MYCAT read / write separation and MySQL master-slave synchronization

Many development frameworks introduce a read-write separation mechanism at the bottom of the framework , In the face of some frameworks without underlying support for read-write separation , How to realize read-write separation ？

Mycat

An open source high-performance database middleware product , Support for read/write separation , Support MySQL Master-slave , Data segmentation and galera cluster colony .
It is an open source distributed database system , It's an implementation MySQL Agreed
Server, Front end users can think of it as a database agent , use MySQL Client tools and command line access , And its back end can use
MySQL Native （Native） Agreement with multiple MySQL Server communication , It can also be used. JDBC The protocol communicates with most mainstream database servers ,
Its core function is to divide tables and databases , Divide a large table horizontally into N Small tables , Store on the back end MySQL In servers or other databases .

Installation guide

1. install JDK ( A little )
2. install MySQL ( A little )
3. install MyCAT
   http://dl.mycat.io/1.6-RELEASE/ The latest stable version is 1.6
   Download the compressed package of the specified system , After decompression, you can use . Installation free .

Use guide

To configure

1. server.xml System profile

<user name="root">
		<property name="password">123456</property>
		<property name="schemas">dsy</property>
		
		<!--  Table level  DML  permissions  -->
		<!-- <privileges check="false"> <schema name="TESTDB" dml="0110" > <table name="tb01" dml="0000"></table> <table name="tb02" dml="1111"></table> </schema> </privileges> -->
	</user>

user Tags are used to add users , Manage user rights .

1. schema.xml Manage configuration files for logical libraries and logical tables

<schema name="dsy" checkSQLschema="false" sqlMaxLimit="100">
		<!-- auto sharding by id (long) -->
		<table name="xn_user_info" dataNode="dn1"/>
	</schema>

explain ：schema The tag defines the name of the logical library ,table Define logical tables .dataNode Attributes represent data nodes, that is, data Shards . If the data is not fragmented , Usually only one is enough .

<dataNode name="dn1" dataHost="localhost1" database="dsy" />

explain ： Create a name called dn1 Data nodes of , The name of this node is localhost1 On the database instance dsy This library consists of data slices .

<dataHost name="localhost1" maxCon="1000" minCon="10" balance="1" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
		<heartbeat>select user()</heartbeat>
		<!-- can have multi write hosts -->
		<writeHost host="hostM1" url="localhost:3306" user="root" password="root">
			<!-- can have multi read hosts -->
			<!-- <readHost host="hostS2" url="localhost:3306" user="root" password="root" /> -->
            <!-- <readHost host="hostS2" url="192.168.0.164:3306" user="root" password="root" /> -->
		</writeHost>
		<writeHost host="hostS1" url="192.168.0.164:3306" user="root" password="root" />
		<!-- <writeHost host="hostM2" url="localhost:3316" user="root" password="123456"/> -->
	</dataHost>

explain ： Create a file called localhost1 Database instance of . The maximum number of links in the connection pool is 1000. The number of initialized connection pools is 10.dbType Identification use mysql Binary protocol . dbDriver Use native Indicates that it can be used mysql and mariadb, There are also optional values JDBC drive .switchType=1 Indicates automatic switching . If switchType Set to 2 also slaveThreshold="100, The statement to maintain the heartbeat is changed to show slave status, It means to turn on MySQL The read-write separation and switch mechanism of master-slave replication state binding .
heartbeat Is to use select user This statement is used to maintain the heartbeat .
writehost Is used to define write instances . In a dataHost You can define more than one writeHost and readHost. however , If writeHost The specified backend database is down , So this writeHost All of the binding readHost Will not be available . On the other hand , Because of this writeHost Downtime system will automatically detect , And switch to the standby writeHost Up .

balance	explain
0	Do not turn on the read-write separation mechanism , All read operations are sent to the currently available writeHost On .
1	All of the readHost And stand by writeHost Participate in select Statement load balancing , To put it simply , When dual master dual slave mode (M1->S1,M2->S2, also M1 And M2 Prepare for each other ), Under normal circumstances ,M2,S1,S2 All involved select Statement load balancing .
2	All the reading operations are random in writeHost、readhost To distribute .
3	All read requests are randomly distributed to wiriterHost Corresponding readhost perform ,writerHost No pressure to read , Be careful balance=3 Only in 1.4 And later versions have ,1.3 No, .

writeType	Load balancing type explanation
0	All writes are sent to the first of the configuration writeHost, The first one is cut to the second one that still exists writeHost, After restart, the one that has been switched shall prevail , The switch is recorded in the configuration file :dnindex.properties .
1	All writes are sent randomly to the configured writeHost,1.5 It is not recommended to discard it later .

switchType	Switch explanation
-1	Does not automatically switch
1	Indicates automatic switching
2	be based on MySQL The state of master-slave synchronization determines whether to switch , The heartbeat statement is show slave status
3	be based on MySQL galary cluster Switching mechanism of （ Suitable for clustering ）（1.4.1）, The heartbeat statement is show status like ‘wsrep%’.

Read write separation configuration

Above dataHost In the configuration , Two level writeHost node . It's a scheme .
Or in the first writeHost There is a readHost Node for read / write separation . For the second scheme .
The second is when the writing hangs up , Reading is not available to colleagues . But the first can continue to be used . The first configuration is recommended . That's what the code above says .
All operations inside the transaction will go to the write node , So don't add transactions to the read operation .

Start the service

Get into MYCAT/bin Catalog perform ./mycat start

Connect mycat

and mysql Command consistent mysql -uroot -p123456 -h127.0.0.1 -P8066 -Ddsy
If you see the following words, it means normal operation .
Your MySQL connection id is 1
Server version: 5.6.29-mycat-1.6-RELEASE-20161028204710 MyCat Server (OpenCloundDB)

To configure MySQL Master slave synchronization

Existing host A：192.168.0.107
host B：192.168.0.164
Master from sync .A by master node B by slave node .

1. A,B On the host binlog function .
   Configuration file in /etc/my.cnf. Cancel log-bin=mysql-bin Previous comments .
2. stay A Create a... On the host slave user , Only to him REPLICATION SLAVE Permissions can be .
3. stay A Perform the following... On the host SQL sentence

FLUSH LOGS
SHOW MASTER STATUS

Make sure you are performing FLUSH LOGS moment ,AB The data of the two libraries are identical .
After execution SHOW MASTER STATUS Record later binlog Document and pos Location .

1. stay B On the main engine master The node points to A host

CHANGE MASTER TO MASTER_HOST = '192.168.0.107', MASTER_USER = 'slave', MASTER_PASSWORD = 'slave', MASTER_PORT = 3306, MASTER_LOG_FILE = 'mysql-bin.000002'  MASTER_LOG_POS = 105  ;

1. Start master-slave synchronization

START SLAVE

1. Check the master-slave synchronization status

SHOW SLAVE STATUS

If you see Slave_IO_Running and Slave_SQL_Running All are Yes
This explanation mysql Master slave synchronization configuration is completed

Step on the hole guide

Try to make sure that AB Host computer mysql Versions, .
If there is no guarantee , Try to make sure that mysqlbinlog Versions, .
If even mysqlbinlog No version can guarantee consistency , At least the primary node mysqlbinlog Cannot be higher than that of the slave host mysqlbinlog.
When I was testing, the primary node mysqlbinlog yes V4.4 Of , From node's mysqlbinlog yes V3.3 Of . It's a nightmare .
The old version of mysqlbinlog Unable to read the new version binlog Log files .

1. Nameless exception of master-slave , I don't know why , Write the solution first , Perform the following sql

stop slave;
set global sql_slave_skip_counter =1;
start slave;
show slave status;

The above scheme can solve the master-slave exception , But we need to pay attention ：
When it comes to this problem , Think about how to satisfy replication , Instead of skipping the transaction ; Skipping errors... Is not recommended , Errors should be corrected , Then connect the master database replication , Otherwise, the data from the library will become more and more inconsistent ！

Manual repair is a little slow , Can target 1062 and 1032 error , Write an automated monitoring correction script .

Be careful ： If the data are often inconsistent , Select the business low peak period , Check the data once （pt-table-checksum）, Check whether the data is consistent , If too many data inconsistencies are detected , The library can no longer be used , Create a slave library ！

Detected commands ( Run this command in the main library )
Note that the master and slave libraries need to have an accessible user , That is, create a main database on the main database ip by host Users of , To give permission , This will automatically synchronize to the slave library , The slave database can also normally receive requests from the master database
pt-table-checksum --host=' The main library host' --user='user' --password='password' --port='3306' --databases=zs_www --recursion-method=processlist --no-check-binlog-format