当前位置：网站首页>Shardingsphere proxy 4.1 sub database and sub table

Shardingsphere proxy 4.1 sub database and sub table

2022-06-25 10:05:00 【@@Mr.Fu】

One 、ShardingSphere-Proxy Core concept of

ShardingSphere-Proxy Concept
Official address ：https://shardingsphere.apache.org/index_zh.html
ShardingSphere-Proxy Is the agent of the database , Pictured ：
ShardingSphere-Proxy Which databases are the main agents
Default agent ：Mysql、PostSql
Achieve the purpose of the agent
1. It is mainly to complete the division of warehouse and table
2. Read and write separation
  These two are also ShardingSphere-Proxy The two core functions of .
Sub database and sub table
- Concept and purpose of sub database
  - Concept
    The tables in the database are stored in different databases ; Pictured ：
  - Purpose
    Prevent resource competition among multiple tables in a library 【CPU、 Memory 】, Performance degradation .
- Concept and purpose of sub table
  - Concept
    Divide a table in the database into multiple tables , Pictured ：
  - Purpose
    Split table is to solve the problem that the amount of data in the table is too large , Improve the performance of user query and data addition .
    such as ： With mysql Database, for example , When users add data, they will pass mysql Of InnoDB The engine stores data ,InnoDB The engine should ensure that the performance of data is within a certain range , The peak value of the largest amount of data in the table is 2000w, If exceeded 2000W Then the performance of adding data will decline , So we're going to surpass 2000W The table of data volume is split into multiple tables , Only in this way can we ensure the user experience .
  - defects
    1. Too much concurrency , There will be resource competition in the table [CPU、 Memory ] The problem of , This leads to performance degradation , The user experience becomes worse .
      Solution ： sub-treasury
Sub database and sub table
- Purpose
  Solve the problems of table resource competition and large amount of data .

Two 、ShardingSphere-Proxy Application scenarios of

scene
Both individual projects and micro service projects can use sub database and sub table .

3、 ... and 、ShardingSphere-Proxy Distribution table landing

Tools
ShardingSphere-Proxy
programme
- In process
  Pictured ：
  - defects
    1. Resource competition .
    2. Abnormal influence problem .
- Out of process 【 recommend 】
  Pictured ：
  - defects
    1. The problem of large amount of maintenance .
    2. The performance is weaker than that in the process .
      - It can be placed in the intranet for communication 【docker】

Realization

Conditions
- Mysql database edition ：5.7
- ShardingSphere-Proxy
  - Download address of online disk
```
 link ：https://pan.baidu.com/s/15yUIDQOdDDwUtVLNxNa9Cg 
 Extraction code ：3hp3
```
- Java Of JDK
  - Download address of online disk
```
 link ：https://pan.baidu.com/s/1A-ksNN0YicT3hXjFscGGwA 
 Extraction code ：r9e0
```
- download Mysql The connection driver of Put the file in the root directory lab Under the folder
  Download address of online disk ：
```
 link ：https://pan.baidu.com/s/1924iUe7wxGpStAzxxv2K3g 
 Extraction code ：jy7z
```

To configure

table

To configure

config-sharding.yaml Fragmented configuration file

# 3、 Create client connection Library  hmms： Virtual database name 【 It's better to have the same name as the real database 】  stay server.yaml name 
schemaName: hmms

# 1、 Connect mysql
dataSources:
  hmmsdatasources-0:  # The name of the node   Customize 
    url: jdbc:mysql://127.0.0.1:3306/ Real database name ?serverTimezone=UTC&useSSL=false
    username:  Database user name 
    password:  Database password  
    connectionTimeoutMilliseconds: 30000
    idleTimeoutMilliseconds: 60000
    maxLifetimeMilliseconds: 1800000
    maxPoolSize: 50
    
# 2、 Fragmentation rule 
shardingRule:
  tables: # surface 
    user: # Logical table name   Which table to divide 
      actualDataNodes: hmmsdatasources-0.user-${
              0..1} # Divide it into several tables   This is two tables  #hmmsdatasources-0： The name of the node  
      tableStrategy: # The data is divided into tables 
        inline:
          shardingColumn: useid # Sub table field 
          algorithmExpression: user-${
              useid % 2} # Yes useid Die taking table  
     # Create multiple tables 
    # Table name : # Logical table name   Which table to divide 
      #actualDataNodes: hmmsdatasources-0. Table name -${0..1} # Divide it into several tables   This is two tables  #hmmsdatasources-0： The name of the node

server.yaml

authentication:
  users:
    root: # Database user name 
      password:  Data password 
    sharding:
      password: sharding 
      authorizedSchemas: hmms

ShardingSpere-Proxy
Run the command
```
# root directory bin To execute a command under a document 
 start.bat
```
The operation result is as shown in the figure ：

MySql database

New real database name

SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;

-- ----------------------------
-- Table structure for user
-- ----------------------------
DROP TABLE IF EXISTS `user`;
CREATE TABLE `user`  (
  `useid` int(11) NOT NULL,
  `usenam` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT ' Login name ',
  `usepwd` varchar(100) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT ' The login password ',
  `usestate` int(11) NULL DEFAULT 2 COMMENT '-1： Delete 1： Cancellation  2： normal  3： Report the loss of ',
  `usekey` varchar(100) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT ' User secret key ',
  `usetel` varchar(11) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT ' User's mobile phone ',
  `createbyid` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT ' Add by ',
  `createbytime` datetime(0) NULL DEFAULT NULL COMMENT ' Add the time ',
  `modifybyid` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT ' Modifier ',
  `modifybytime` datetime(0) NULL DEFAULT NULL COMMENT ' Modification time ',
  PRIMARY KEY (`useid`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;

SET FOREIGN_KEY_CHECKS = 1;

Pictured ：

Insert picture description here

newly build 3307 Virtual database connection
Pictured ：
Delete table
Delete 3306 in hmms Library user surface , And execute the script of creating a new table in the virtual database , The operation results are as follows ：
Virtual database ：
Real database
Add data
Add two pieces of data to the virtual database , The operation results are as follows ：
Real database Table 1 , The operation results are as follows ：
Real database Table two , The operation results are as follows ：

sub-treasury

To configure

# 3、 Create client connection Library 
schemaName: hmms

# 1、 Connect mysql
dataSources:
  hmmsdatasources-0: # Real database 0
    url: jdbc:mysql://127.0.0.1:3306/hmms-0?serverTimezone=UTC&useSSL=false
    username:  user name 
    password:  password  
    connectionTimeoutMilliseconds: 30000
    idleTimeoutMilliseconds: 60000
    maxLifetimeMilliseconds: 1800000
    maxPoolSize: 50
  hmmsdatasources-1: # Real database 1
    url: jdbc:mysql://127.0.0.1:3306/hmms-1?serverTimezone=UTC&useSSL=false
    username:  user name 
    password:  password  
    connectionTimeoutMilliseconds: 30000
    idleTimeoutMilliseconds: 60000
    maxLifetimeMilliseconds: 1800000
    maxPoolSize: 50 
    
# 2、 Fragmentation rule 
shardingRule:
  tables: # surface 
    user: # Logical table name 
      actualDataNodes: hmmsdatasources-${
            0..1}.user # table 
      tableStrategy: # The data is divided into tables 
        inline:
          shardingColumn: useid # Sub table field 
          algorithmExpression: user-${
            useid % 2} # Yes useid Die taking table  
  defaultDatabaseStrategy: #  Data sub database strategy 
    inline:
     shardingColumn: useid # Sub database field 
     algorithmExpression: hmmsdatasources-${
            useid % 2}  # Yes Id Take the mold branch library productdatasources-0

Sub database and sub table

To configure

# 3、 Create client connection Library 
schemaName: hmms

# 1、 Connect mysql
dataSources:
  hmmsdatasources-0: # Real database 0
    url: jdbc:mysql://127.0.0.1:3306/hmms-0?serverTimezone=UTC&useSSL=false
    username:  user name 
    password:  password  
    connectionTimeoutMilliseconds: 30000
    idleTimeoutMilliseconds: 60000
    maxLifetimeMilliseconds: 1800000
    maxPoolSize: 50
  hmmsdatasources-1: # Real database 1
    url: jdbc:mysql://127.0.0.1:3306/hmms-1?serverTimezone=UTC&useSSL=false
    username:  user name 
    password:  password  
    connectionTimeoutMilliseconds: 30000
    idleTimeoutMilliseconds: 60000
    maxLifetimeMilliseconds: 1800000
    maxPoolSize: 50    
    
# 2、 Fragmentation rule 
shardingRule:
  tables: # surface 
    user: # Logical table name 
      actualDataNodes: hmmsdatasources-${0..1}.user-${0..1} # table 
      tableStrategy: # The data is divided into tables 
        inline:
          shardingColumn: useid # Sub table field 
          algorithmExpression: user-${useid % 2} # Yes useid Die taking table 
  defaultDatabaseStrategy: #  Data sub database strategy 
    inline:
     shardingColumn: useid # Sub database field 
     algorithmExpression: hmmsdatasources-${useid % 2}  # Yes Id Take the mold branch library productdatasources-0

Four 、ShardingSphere-Proxy Operation principle

The overall architecture
in total 6 Stages ：
1、Database Adaptors： The choice of database
2、SQL Parser： analysis sql
3、SQL Router：sql route Which real database to execute
4、SQL Rewriter：sql Optimized rewriting The core Guaranteed performance
5、SQL Executor Engine： perform sql sentence Get results from real database
6、Result Merger： The results merge Get results from multiple tables

5、 ... and 、ShardingSphere_Proxy Fragmentation principle

The concept of fragmentation
Is to slice data into different tables .
Patch key
The fragment key is the field in the table . It is divided according to what field .
Sharding algorithm
According to rules 【 Sharding algorithm 】 Press the fragment key to divide the data into different tables .
- Modular algorithm
  - defects
    Only numeric types can be used
- hash+ modulus
  - If the fragment key is of character type , Just use hash+ Take the mold and divide it .
```
Math.abs( Patch key .hashCode()%2)
```