MYCAT sub database and sub table

MyCat Installation and deployment

Official website download address http://dl.mycat.org.cn/

Three configuration files ：schema.xml Define logical library 、 surface 、 Fragment node and so on 、rule.xml Define fragmentation rules 、server.xml Define users and system related variables such as ports

MyCat docking MySQL

Modify the configuration file server.xml User information in , Configure the access user and library name of the logical library

<user name="mycat">
	<property name="password">mycat</property>
	<property name="schemas">TESTDB</property>
	<property name="readOnly">true</property>
</user>

Modify the configuration file schema.xml

<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100" dataNode="dn1">
</schema>
<dataNode name="dn1" dataHost="bigdata" database="hudi" />
<dataHost name="bigdata" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
	<heartbeat>select user()</heartbeat>
	<!-- can have multi write hosts -->
	<writeHost host="hostM1" url="bigdata:3306" user="root" password="root">
		<!-- can have multi read hosts -->
		<readHost host="hostS2" url="bigdata:3306" user="root" password="root" />
	</writeHost>
</dataHost>

Verify database access

# If remote access reports an error , Please create corresponding user  
grant all privileges on *.* to root@' lack of host'  identified by ' password '; 

Start the program

#  Console launch 
$MYCAT_HOME/bin/mycat console
#  Background start 
$MYCAT_HOME/bin/mycat start

Log in to the background management window ： For management and maintenance MyCat

mysql -umycat -pmycat -P9066 -hbigdata

#  Common commands 
show database
show @@help

Log in to the data window ： Used by MyCat Query data

mysql -umycat -pmycat -P8066 -hbigdata

Use MyCat Set up read-write separation -MySQL A master from

One host is used to handle all write requests , A slave is responsible for all read requests , The architecture is as follows

modify MySQL The configuration file /etc/my.cnf

Host configuration (ip:host79)

#  The primary server is unique ID 
server-id=1 
#  Enable binary logging  
log-bin=mysql-bin 
#  Set up the database not to be copied ( Multiple can be set ) 
binlog-ignore-db=mysql 
binlog-ignore-db=information_schema 
#  Set up the database to be copied  
binlog-do-db= The name of the master database to be copied  
#  Set up logbin Format  
binlog_format=STATEMENT 

Slave configuration (ip:host80)

#  From the server only ID 
server-id=2 
#  Enable relay logging  
relay-log=mysql-relay 

MySQL Implement master-slave replication

And both the master and the slave need to be restarted MYSQL service 、 And turn off the firewall

systemctl restart mysqld
systemctl status mysqld

Establish an account on the host and authorize slave

# Host computer MySQL To carry out the authorization order in the library  
GRANT REPLICATION SLAVE ON *.* TO 'slave'@'%' IDENTIFIED BY '123123'; 
#  see master state 
show master status
#  Record File Of Position Value , Do not operate the master after this step MySQL, Prevent the state value of the primary server from changing 

Configure the master to be replicated on the slave

#  Command to copy host 
CHANGE MASTER TO MASTER_HOST=' The host IP Address ',
MASTER_USER='slave',
MASTER_PASSWORD='123123',
MASTER_LOG_FILE='mysql-bin. Specific figures ',MASTER_LOG_POS= Specific value ;

#  Start copy from server 
start slave;
#  View the status of the slave server 
show slave status\G;

Host new library 、 new table 、insert Record 、 Copy from the machine

Stop the master-slave copy function

stop slave;

Reconfigure master-slave

stop slave;
reset master;

adopt MyCat Read and write separation

modify MyCat Configuration file for schema.xml Medium dataHost Labeled balance attribute , Configure the read-write separation type through this property

balance="0",  Do not turn on the read-write separation mechanism , All read operations are sent to the currently available  writeHost  On 

balance="1", All of the  readHost  And  stand by writeHost  Participate in  select  Statement load balancing , To put it simply , When dual master dual slave mode (M1->S1,M2->S2, also  M1  And  M2  Prepare for each other ), Under normal circumstances ,M2,S1,S2  All involved  select  Statement load balancing 

balance="2", All the reading operations are random in  writeHost、readhost  To distribute 

balance="3", All read requests are randomly distributed to  readhost  perform ,writerHost  No pressure to read 

<dataHost name="host1" maxCon="1000" minCon="10" balance="2"
		  writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">

Verification mode

Verification mode ： Because of the configuration binlog_format=STATEMENT , So when using system variables or time functions, there will be inconsistencies , Insert the system in this way IP, Check whether it is the same service reading and writing to verify .

#  start-up MyCat

#  Insert... In the write host 
insert into mytbl values (1,@@hostname);
#  stay MyCat Look in 
select * from mytbl;

#  stay Mycat Look in mytbl surface , You can see that the query statement switches between the master and slave hosts 

Use MyCat Set up read-write separation -MySQL Two masters and two slaves

A mainframe M1 Used to process all write requests , Its slaves S1 And another host M2 And its slaves S2 Responsible for all read requests . When M1 After the host goes down ,M2 The host is responsible for writing requests ,M1、M2 Stand by each other . The architecture is as follows

Number	role	IP Address	machine name
1	Master1	192.168.140.128	host79
2	Slave1	192.168.140.127	host80
3	Master2	192.168.140.126	host81
4	Slave2	192.168.140.125	host82

modify MySQL The configuration file /etc/my.cnf

Master1 To configure

#  The primary server is unique ID 
server-id=1 
#  Enable binary logging  
log-bin=mysql-bin 
#  Set up the database not to be copied ( Multiple can be set ) 
binlog-ignore-db=mysql 	 
binlog-ignore-db=information_schema 
#  Set up the database to be copied  
binlog-do-db= The name of the master database to be copied  
#  Set up logbin Format 
binlog_format=STATEMENT 
#  As a slave database , Update binary log files when there are write operations 
log-slave-updates  
#  Represents the amount of each increment of the self growing field , It refers to the starting value of the auto increment field , The default value is 1, The value range is 1 .. 65535 
auto-increment-increment=1
#  Indicates the number from which the self growing field starts , Refers to how many fields are incremented at a time , His range is 1 .. 65535 
auto-increment-offset=1 

Master2 To configure

#  The primary server is unique ID
server-id=3
#  Enable binary logging 
log-bin=mysql-bin
#  Set up the database not to be copied ( Multiple can be set )
binlog-ignore-db=mysql
binlog-ignore-db=information_schema
#  Set up the database to be copied 
binlog-do-db= The name of the master database to be copied 
#  Set up logbin Format 
binlog_format=STATEMENT
#  As a slave database , Update binary log files when there are write operations 
log-slave-updates 
#  Represents the amount of each increment of the self growing field , It refers to the starting value of the auto increment field , The default value is 1, The value range is 1 .. 65535
auto-increment-increment=1
#  Indicates the number from which the self growing field starts , Refers to how many fields are incremented at a time , His range is 1 .. 65535
auto-increment-offset=1

Slave1 To configure

#  From the server only ID
server-id=2
#  Enable relay logging 
relay-log=mysql-relay

Slave2 To configure

#  From the server only ID
server-id=4
#  Enable relay logging 
relay-log=mysql-relay

And both the master and the slave need to be restarted MYSQL service 、 And turn off the firewall

systemctl restart mysqld
systemctl status mysqld

Set up accounts on both hosts and authorize slave

# Host computer MySQL To carry out the authorization order in the library 
GRANT REPLICATION SLAVE ON *.* TO 'slave'@'%' IDENTIFIED BY '123123';
# Inquire about Master1 The state of 
show master status;
#  Record File Of Position Value , Do not operate the master after this step MySQL, Prevent the state value of the primary server from changing 

Configure the master to be replicated on the slave

Slave1 Copy Master1、Slave2 Copy Master2

# Command to copy host 
CHANGE MASTER TO MASTER_HOST=' The host IP Address ',
MASTER_USER='slave',
MASTER_PASSWORD='123123',
MASTER_LOG_FILE='mysql-bin. Specific figures ',MASTER_LOG_POS= Specific value ;

# Start the replication function of two slave servers 
start slave;
# View the status of the slave server 
show slave status\G;

Two hosts copy each other

Master2 Copy Master1,Master1 Copy Master2

# Start the replication function of two master servers 
start slave;
# View the status of the slave server 
show slave status\G;

Verification test

Master1 Host new library 、 new table 、insert Record ,Master2 And slave copy

adopt MyCat Read and write separation

<dataHost name="host1" maxCon="1000" minCon="10" balance="1" writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100" >
	<heartbeat>select user()</heartbeat>
	<!-- can have multi write hosts -->
	<writeHost host="hostM1" url="192.168.140.128:3306" user="root"password="123123">
		<!-- can have multi read hosts -->
		<readHost host="hostS1" url="192.168.140.127:3306" user="root" password="123123" />
	</writeHost>
	
	<writeHost host="hostM2" url="192.168.140.126:3306" user="root"password="123123">
		<!-- can have multi read hosts -->
		<readHost host="hostS2" url="192.168.140.125:3306" user="root" password="123123" />
	</writeHost>
</dataHost>

#  start-up MyCat

#  Insert... In the write host 
insert into mytbl values (3,@@hostname);
#  stay MyCat Look in 
select * from mytbl;

# Master1 Downtime  
systemctl stop mysqld
#  stay Mycat Inserting data into the is still successful ,Master2 Automatically switch to write host 
# Master1、Master2  Mutual standby , The host responsible for writing is down , The standby switch is responsible for writing , Ensure the separation of database reading and writing and high availability .

MyCat Split Vertically - sub-treasury

Sub database division

A database consists of many tables , Each table corresponds to a different business , Vertical segmentation is to classify tables according to business , To different databases , In this way, the data or pressure can be distributed to different libraries . The principle of dividing tables ： Closely related tables should be in a library , Tables that are not related to each other can be divided into different libraries .

Realize the sub library

modify schema.xml The configuration file

<schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100" dataNode="dn1">
	<table name="customer" dataNode="dn2" ></table>
</schema>

<dataNode name="dn1" dataHost="host1" database="orders" />
<dataNode name="dn2" dataHost="host2" database="orders" />

<dataHost name="host1" maxCon="1000" minCon="10" balance="0"writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
	<heartbeat>select user()</heartbeat>
	<!-- can have multi write hosts -->
	<writeHost host="hostM1" url="192.168.140.128:3306" user="root"password="123123">
	</writeHost>
</dataHost>

<dataHost name="host2" maxCon="1000" minCon="10" balance="0"writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
	<heartbeat>select user()</heartbeat>
	<!-- can have multi write hosts -->
	<writeHost host="hostM2" url="192.168.140.127:3306" user="root"password="123123">
	</writeHost>
</dataHost>

#  visit  Mycat
mysql -umycat -p123456 -h 192.168.140.128 -P 8066
#  Switch to  TESTDB
#  establish  4  A watch 
#  View table information , You can see the successful sub database 

MyCat Horizontal split - table

Table division

Split relative to vertical , Horizontal splitting is not about classifying tables , But according to a certain rule of a certain field, it is scattered into multiple libraries , Each table contains some data . That is, according to the segmentation of data rows , It is to segment some rows in the table into a database , And some of the other rows are split into other databases .

Realize the sub table

First step ： Select the table to split ,MySQL A single table achieves 1000w Pieces of data have reached the bottleneck , It will affect the query efficiency .

The second step ： Sub table field , It's usually a primary key ID、 Foreign keys ID Or a timestamp .

The third step ： Modify the configuration file schema.xml

#  by orders Table set the data node to dn1、dn2, And specify the partition rule as mod_rule（ Custom name ）
<table name="orders" dataNode="dn1,dn2" rule="mod_rule" ></table>

Step four ： Modify the configuration file rule.xml

#  stay  rule  Add fragmentation rules to the configuration file  mod_rule, And specify the rule for the field  customer_id,
#  There are also selective slicing algorithms  mod-long（ Moduling fields ）,customer_id  Moduling two nodes , Slice according to the results 
#  Configuration algorithm  mod-long  Parameters  count  by  2, Two nodes 
<tableRule name="mod_rule">
	<rule>
		<columns>customer_id</columns>
		<algorithm>mod-long</algorithm>
	</rule>
</tableRule>

<function name="mod-long" class="io.mycat.route.function.PartitionByMod">
	<!-- how many data nodes -->
	<property name="count">2</property>
</function>

Step five ： restart MyCat Make the configuration work , Access test shards

# stay  mycat  Inward  orders  Table insert data ,INSERT  Fields cannot be omitted 
INSERT INTO orders(id,order_type,customer_id,amount) VALUES (1,101,100,100100);
INSERT INTO orders(id,order_type,customer_id,amount) VALUES(2,101,100,100300);
INSERT INTO orders(id,order_type,customer_id,amount) VALUES(3,101,101,120000);
INSERT INTO orders(id,order_type,customer_id,amount) VALUES(4,101,101,103000);
INSERT INTO orders(id,order_type,customer_id,amount) VALUES(5,102,101,100400);
INSERT INTO orders(id,order_type,customer_id,amount) VALUES(6,102,100,100020);
# stay mycat、dn1、dn2 View in orders Table data , Sub table success 

MyCat Security Settings

User Tag permission control

Tag attributes	explain
name	Logical connection of user name application middleware
password	The password of the user
TESTDB	Apply the corresponding logical table in the logical library of the current connection .schemas You can configure one or more
readOnly	The permissions of application connection middleware logical library .true As read-only ,false For both reading and writing , The default is false

Privileges Tag permission control

stay user Label under privileges Tags can be used for logical Libraries （schema）、 surface （table） To refine DML Access control .

privileges Label under check attribute , If true Turn on permission check , by false Don't open , The default is false.

because Mycat One user's schemas Property to configure multiple logical Libraries （schema） , therefore privileges The child nodes of schema Nodes can also be configured with multiple , Fine grained... For multiple databases and tables DML Access control .

DML jurisdiction	increase （insert）	to update （update）	Inquire about （select）	Delete （select）
0000	prohibit	prohibit	prohibit	prohibit
0010	prohibit	prohibit	Sure	prohibit
1110	Sure	prohibit	prohibit	prohibit
1111	Sure	Sure	Sure	Sure

SQL Intercept

White list ： You can set the white list , Realize that a host and a user can access Mycat, Other host users are forbidden to access .

The blacklist ： You can set up a blacklist , Realization Mycat To be specific SQL Operational interception , Such as adding, deleting, modifying, checking, etc