Shardingsphere sharding proxy actual combat scenario

shardingsphere-proxy Installation and startup

download shardingsphere-4.0.0-RC1 edition （ Just choose a version you like ）

http://mirrors.tuna.tsinghua.edu.cn/apache/incubator/shardingsphere/4.0.0-RC2/apache-shardingsphere-incubating-4.0.0-RC2-sharding-proxy-bin.tar.gz

decompression

tar ‐zxvf apache‐shardingsphere‐incubating‐4.0.0‐RC2‐sharding‐proxy‐ bin.tar.gz 

Unzipped directory ：

Introduce dependencies  

If the back-end connection MySQL database , Need to download manually mysql drive jar package , And will mysqlconnector-java-${version}.jar copy to ${sharding-proxy}\lib Catalog

A launch configuration

Rule configuration ‐ You can choose according to your needs

edit %SHARDING_PROXY_HOME%\conf\config‐xxx.yaml

(config‐sharding.yaml‐ Data slicing settings , Reference resources sharding‐jdbc)

(config‐master_slave.yaml‐ Master slave read / write separation settings , Reference resources sharding‐jdbc)

(config‐encrypt.yaml‐ Data desensitization settings , Reference resources sharding‐jdbc)

Global configuration ： Registry Center 、 Authentication information and common attributes edit %SHARDING_PROXY_HOME%\conf\server.yaml

Data fragmentation config-sharding.yam

schemaName: sharding_db # Logical data source name 

dataSources:
  ds0: 
    url: jdbc:mysql://127.0.0.1:3306/ds0?serverTimezone=UTC&useSSL=false
    username: root
    password: 
    connectionTimeoutMilliseconds: 30000 // Connection timeout milliseconds 
    idleTimeoutMilliseconds: 60000 // Idle connection recycle timeout milliseconds 
    maxLifetimeMilliseconds: 1800000 // Maximum connection lifetime in milliseconds 
    maxPoolSize: 65 // maximum connection 
  ds1:
    url: jdbc:mysql://127.0.0.1:3306/ds1?serverTimezone=UTC&useSSL=false
    username: root
    password: 
    connectionTimeoutMilliseconds: 30000
    idleTimeoutMilliseconds: 60000
    maxLifetimeMilliseconds: 1800000
    maxPoolSize: 65
# Same as sharding-jdbc Set up 
shardingRule:
  tables:
    t_order:
      actualDataNodes: ds${0..1}.t_order${0..1}
      databaseStrategy:
        inline:
          shardingColumn: user_id
          algorithmExpression: ds${user_id % 2}
      tableStrategy: 
        inline:
          shardingColumn: order_id
          algorithmExpression: t_order${order_id % 2}
      keyGenerator:
        type: SNOWFLAKE
        column: order_id
    t_order_item:
      actualDataNodes: ds${0..1}.t_order_item${0..1}
      databaseStrategy:
        inline:
          shardingColumn: user_id
          algorithmExpression: ds${user_id % 2}
      tableStrategy:
        inline:
          shardingColumn: order_id
          algorithmExpression: t_order_item${order_id % 2}
      keyGenerator:
        type: SNOWFLAKE
        column: order_item_id
  bindingTables:
    - t_order,t_order_item
  defaultTableStrategy:
    none:

Read / write separation config-master_slave.yaml

schemaName: master_slave_db

dataSources:
  ds_master:
    url: jdbc:mysql://127.0.0.1:3306/ds_master?serverTimezone=UTC&useSSL=false
    username: root
    password: 
    connectionTimeoutMilliseconds: 30000
    idleTimeoutMilliseconds: 60000
    maxLifetimeMilliseconds: 1800000
    maxPoolSize: 65
  ds_slave0:
    url: jdbc:mysql://127.0.0.1:3306/ds_slave0?serverTimezone=UTC&useSSL=false
    username: root
    password: 
    connectionTimeoutMilliseconds: 30000
    idleTimeoutMilliseconds: 60000
    maxLifetimeMilliseconds: 1800000
    maxPoolSize: 65
  ds_slave1:
    url: jdbc:mysql://127.0.0.1:3306/ds_slave1?serverTimezone=UTC&useSSL=false
    username: root
    password: 
    connectionTimeoutMilliseconds: 30000
    idleTimeoutMilliseconds: 60000
    maxLifetimeMilliseconds: 1800000
    maxPoolSize: 65
# Master-slave routing rules 
masterSlaveRule:
  name: ds_ms
  masterDataSourceName: ds_master
  slaveDataSourceNames: 
    - ds_slave0
    - ds_slave1
  loadBalanceAlgorithmType: ROUND_ROBIN

Data desensitization

dataSource: # Omit data source configuration 

encryptRule:
  encryptors:
    <encryptor-name>:
      type: # Encryption and decryption type , You can customize or select the built-in type ：MD5/AES 
      props: # Attribute configuration ,  Be careful ： Use AES Encrypting device , Need configuration AES Encryptor's KEY attribute ：aes.key.value
        aes.key.value: 
  tables:
    <table-name>:
      columns:
        <logic-column-name>:
          plainColumn: # Fields that store plaintext 
          cipherColumn: # The field where the ciphertext is stored 
          assistedQueryColumn: # Auxiliary query fields , in the light of ShardingQueryAssistedEncryptor Type of encryptor for auxiliary query 
          encryptor: # Encryptor name 
props:
  query.with.cipher.column: true # Whether to use ciphertext column to query 

Global configuration server.yaml

Sharding-Proxy Use conf/server.yaml Configure the registry 、 Authentication information and common attributes .

 Data governance 

orchestration:
  name: # Governance instance name 
  overwrite: # Does the local configuration override the registry configuration . If it can cover , Every time you start, the local configuration will prevail 
  registry: # Registry configuration 
    type: # Configuration center type . Such as ：zookeeper
    serverLists: # Connect to the list of registry servers . Include IP Address and port number . Multiple addresses are separated by commas . Such as : host1:2181,host2:2181
    namespace: # The namespace of the registry 
    digest: # Access token to connect to the registry . The default is no permission verification 
    operationTimeoutMilliseconds: # The number of milliseconds the operation timed out , Default 500 millisecond 
    maxRetries: # Maximum number of retries after connection failure , Default 3 Time 
    retryIntervalMilliseconds: # The number of milliseconds between retries , Default 500 millisecond 
    timeToLiveSeconds: # The number of seconds the temporary node survives , Default 60 second 
 example ：   
orchestration:
  name: orchestration_ds
  overwrite: true
  registry:
    type: zookeeper
    namespace: orchestration
    serverLists: localhost:2181

 Authentication information ( Connect to sharding-proxy Identity authentication is required )
authentication:
  users:
    root: #  Custom user name 
      password: root #  Custom user name 
    sharding: #  Custom user name 
      password: sharding #  Custom user name 
      authorizedSchemas: sharding_db, masterslave_db( The logical data source configured in the data slice )
      #  The database that the user is authorized to access , Multiple separated by commas . Default will have root jurisdiction , Access to all databases .
      
# Omit and Sharding-JDBC Consistent configuration properties 
props:
  acceptor.size: # Used to set the number of worker threads that receive client requests , The default is CPU Check the number *2
  proxy.transaction.type: # The default is LOCAL Business , allow LOCAL,XA,BASE Three values ,
  XA use Atomikos As a transaction manager ,BASE The type needs to copy the implementation ShardingTransactionManager The interface of jar Package to lib Directory 
  proxy.opentracing.enabled: # Whether the link tracking function is enabled , The default is not on .
  check.table.metadata.enabled: # Whether to check the metadata consistency of the sub table at startup , The default value is : false
  proxy.frontend.flush.threshold: #  For a single large query , How many network packets are returned once 
  executor.size: 16
  sql.show: false (or true) Whether or not to print SQL sentence 

Special attention shall be paid to data governance service startup ：

 Following example ：
authentication:
  users:
    root:
      password: root
    tuling:
      password: tuling
      authorizedSchemas: shop_ds_proxy
#
orchestration:
  name: tuling_orchestration_proxy
  overwrite: false
  registry:
    type: zookeeper
    serverLists: 192.168.241.198:2181
    namespace: sharding_zookeeper_proxy
    
 start-up sharing-proxy You need to be in zookeeper Create the corresponding Node The nodes are as follows ：
${namespace}/${orchestration.name}/config/authentication
 Corresponding to the above configuration nodes Node Should be ：/sharding_zookeeper_proxy/tuling_orchestration_proxy/config/authentication
 Then set the data node data to ：
users:
  root:
    password: root
  tuling:
    password: tuling
    authorizedSchemas: shop_ds_proxy
 If not set , Then throw node value null Null pointer exception ,proxy Service failed to start normally    

Yaml Syntax description

!!  Represents instantiation of this class

-  The representation can contain one or more

[]  Represents an array , Can be used interchangeably with the minus sign

start-up proxy service

Start with the default configuration item

${sharding-proxy}\bin\start.sh

Configure port startup （ Default port 3307）

${sharding-proxy}\bin\start.sh ${port}

Out of Service

${sharding-proxy}\bin\stop.sh

sharding-proxy The basic characteristics of

Custom segmentation algorithm ：

When the user needs to use a custom sharding algorithm class , Can no longer pass the simple inline The expression in yaml File . You can configure and use the custom sharding algorithm in the following ways .

1. Realization ShardingAlgorithm Interface defines the algorithm implementation class .
2. Put the above java The files are packed into jar package .
3. Put the above jar Package copy to ShardingProxy After decompression conf/lib Under the table of contents .
4. Implement the above custom algorithm into the class java File reference is configured in yaml In the document tableRule Of algorithmClassName Attribute

Distributed transactions

Sharding-Proxy Distributed transactions accessed API Same as Sharding-JDBC bring into correspondence with , Support LOCAL,XA,BASE Type of transaction .

XA Business

Sharding-Proxy Native support XA Business , The default transaction manager is Atomikos. It can be done by Sharding-Proxy Of conf Add to catalog jta.properties To customize Atomikos Configuration item .

BASE Business

BASE Currently, it is not packaged to Sharding-Proxy in , When using, you need to implement ShardingTransactionManagerSPI Of jar Copy to conf/lib Catalog , Then switch the transaction type to BASE.

SCTL(Sharding-Proxy control language)

SCTL by Sharding-Proxy Unique control statements , It can be modified and queried at run time Sharding-Proxy The state of , The currently supported syntax is ：

sentence	explain
sctl:set transaction_type=XX	Modify the current TCP Connected transaction type , Support LOCAL,XA,BASE. example ：sctl:set transaction_type=XA
sctl:show transaction_type	Query the current TCP Connected transaction type
sctl:show cached_connections	Query the current TCP The number of cached physical database connections in the connection
sctl:explain SQL sentence	View logic SQL Implementation plan of , example ：sctl:explain select * from t_order;
sctl:hint set MASTER_ONLY=true	Aiming at the present TCP Connect , Do you want to forcibly route database operations to the primary database
sctl:hint set DatabaseShardingValue=yy	Aiming at the present TCP Connect , Set up hint Only valid for database sharding , And add the slice value ,yy： Database partition value
sctl:hint addDatabaseShardingValue xx=yy	Aiming at the present TCP Connect , Is a table xx Add slice value yy,xx： Logical table name ,yy： Database partition value
sctl:hint addTableShardingValue xx=yy	Aiming at the present TCP Connect , Is a table xx Add slice value yy,xx： Logical table name ,yy： Table slice value
sctl:hint clear	Aiming at the present TCP Connect , eliminate hint All settings
sctl:hint show status	Aiming at the present TCP Connect , Inquire about hint state ,master_only:true/false,sharding_type:databases_only/databases_tables
sctl:hint show table status	Aiming at the present TCP Connect , Query logical table hint Slice value

Sharding-Proxy Not supported by default hint, If you need support , Please be there. conf/server.yaml in , take props Properties of proxy.hint.enabled Set to true. stay Sharding-Proxy in ,HintShardingAlgorithm The generics of can only be String type .

Instructions ：

Like use SQL Same as normal use ; Can pass mysql-client Tools are connected and used

mysql -h 192.168.241.198 -P3308 -u tuling(authentication Under the configuration of user) -ptuling((authentication Under the configuration of pwd) Log in and execute sctl sentence , As shown in the figure below

 

matters needing attention

1. Sharding-Proxy By default 3307 port , You can append parameters as the startup port number through the startup script . Such as : bin/start.sh 3308
2. Sharding-Proxy Use conf/server.yaml Configure the registry 、 Authentication information and common attributes .
3. Sharding-Proxy Support multiple logical data sources , Each with config- Prefix named yaml The configuration file , It is a logical data source .