当前位置：网站首页>ShardingSphere read-write separation (8)

ShardingSphere read-write separation (8)

2022-07-31 00:48:00 【Rongji】

概述

面对日益增加的系统访问量,数据库的吞吐量面临着巨大瓶颈. 对于同一时间有大量并发读操作和较少写操作类型的应用系统来说,将单一的数据库拆分为主库和从库,主库负责处理事务性的增删改操作,从库负责处理查询操作,能够有效的避免由数据更新导致的行锁,使得整个系统的查询性能得到极大的改善.透明化读写分离所带来的影响,让使用方尽量像使用一个数据库一样使用主从数据库,It is the main function of the read-write separation middleware.
Sharding-JDBC Support for read-write separation has been provided,Take a look at the following two documents：
ShardingSphere > 概念 & 功能 > 读写分离
ShardingSphere > 用户手册 > Sharding-JDBC > 使用手册 > 读写分离

This article does not involve the configuration of database master-slave synchronization,Just manually inserting data in the read library to simulate.

一、 引入依赖

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.3.0.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>dynamic-datasource-sharding-jdbc-02</artifactId>

    <dependencies>
        <!-- 实现对数据库连接池的自动化配置 -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>
        <dependency> <!-- 本示例,我们使用 MySQL -->
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.48</version>
        </dependency>

        <!-- 实现对 MyBatis 的自动化配置 -->
        <dependency>
            <groupId>org.mybatis.spring.boot</groupId>
            <artifactId>mybatis-spring-boot-starter</artifactId>
            <version>2.1.1</version>
        </dependency>

        <!-- 实现对 Sharding-JDBC 的自动化配置 -->
        <dependency>
            <groupId>org.apache.shardingsphere</groupId>
            <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
            <version>4.0.0-RC2</version>
        </dependency>

        <!-- 保证 Spring AOP 相关的依赖包 -->
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-aspects</artifactId>
        </dependency>

        <!-- Easy to write unit tests -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

二、Application

package cn.iocoder.springboot.lab17.dynamicdatasource;

import org.mybatis.spring.annotation.MapperScan;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.EnableAspectJAutoProxy;

/** * @description 启动类 * @ClassName: Application * @author: Guo Xiuzhi [email protected] * @date: 2020/6/30 17:17 * @Copyright: */
@SpringBootApplication
@MapperScan(basePackages = "cn.iocoder.springboot.lab17.dynamicdatasource.mapper")
@EnableAspectJAutoProxy(exposeProxy = true) // http://www.voidcn.com/article/p-zddcuyii-bpt.html
public class Application {
    
    public static void main(String[] args) {
    
        SpringApplication.run(Application.class, args);
    }
}

三、配置文件

在 [resources] 目录下,创建 [application.yaml]配置文件.配置如下：

spring:
  # ShardingSphere 配置项
  shardingsphere:
    # 数据源配置
    datasource:
      # 所有数据源的名字
      names: ds-master, ds-slave-1, ds-slave-2
      # 订单 orders 主库的数据源配置
      ds-master:
        type: com.zaxxer.hikari.HikariDataSource # 使用 Hikari 数据库连接池
        driver-class-name: com.mysql.jdbc.Driver
        jdbc-url: jdbc:mysql://101.133.227.13:3306/test_orders?useSSL=false&useUnicode=true&characterEncoding=UTF-8
        username: root
        password: IA8oD
      # 订单 orders 从库数据源配置
      ds-slave-1:
        type: com.zaxxer.hikari.HikariDataSource # 使用 Hikari 数据库连接池
        driver-class-name: com.mysql.jdbc.Driver
        jdbc-url: jdbc:mysql://101.133.227.13:3306/test_orders_01?useSSL=false&useUnicode=true&characterEncoding=UTF-8
        username: root
        password: IA8o
      # 订单 orders 从库数据源配置
      ds-slave-2:
        type: com.zaxxer.hikari.HikariDataSource # 使用 Hikari 数据库连接池
        driver-class-name: com.mysql.jdbc.Driver
        jdbc-url: jdbc:mysql://101.133.227.13:3306/test_orders_02?useSSL=false&useUnicode=true&characterEncoding=UTF-8
        username: root
        password: IA8
    # 读写分离配置,对应 YamlMasterSlaveRuleConfiguration 配置类
    masterslave:
      name: ms # 名字,任意,需要保证唯一
      master-data-source-name: ds-master # 主库数据源
      slave-data-source-names: ds-slave-1, ds-slave-2 # 从库数据源

# mybatis 配置内容
mybatis:
  config-location: classpath:mybatis-config.xml # 配置 MyBatis 配置文件路径
  mapper-locations: classpath:mapper/*.xml # 配置 Mapper XML 地址
  type-aliases-package: cn.iocoder.springboot.lab17.dynamicdatasource.dataobject # 配置数据库实体包路径
  • spring.shardingsphere.datasource 配置项下,我们配置了 一个主数据源 ds-master 、两个从数据源 ds-slave-1、ds-slave-2 .
  • spring.shardingsphere.masterslave 配置项下,配置了读写分离.对于从库来说,Sharding-JDBC 提供了多种负载均衡策略,默认为轮询.
  • mybatis 配置项,设置 mybatis-spring-boot-starter MyBatis 的配置内容.

四、创建表

Because it is not built locally MySQL 一主多从的环境,所以是通过创建了 test_orders_01、test_orders_02 库,手动模拟作为 test_orders 的从库.
对应的创建表的 SQL 如下：

CREATE DATABASE `test_orders` CHARACTER SET 'utf8' COLLATE 'utf8_general_ci';

-- 在 `test_orders` 库中.
CREATE TABLE `orders` (
  `id` int(11) DEFAULT NULL COMMENT '订单编号',
  `user_id` int(16) DEFAULT NULL COMMENT '用户编号'
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin COMMENT='订单表';

五、MyBatis 配置文件

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration PUBLIC "-//mybatis.org//DTD Config 3.0//EN" "http://mybatis.org/dtd/mybatis-3-config.dtd">
<configuration>

    <settings>
        <!-- 使用驼峰命名法转换字段. -->
        <setting name="mapUnderscoreToCamelCase" value="true"/>
    </settings>

    <typeAliases>
        <typeAlias alias="Integer" type="java.lang.Integer"/>
        <typeAlias alias="Long" type="java.lang.Long"/>
        <typeAlias alias="HashMap" type="java.util.HashMap"/>
        <typeAlias alias="LinkedHashMap" type="java.util.LinkedHashMap"/>
        <typeAlias alias="ArrayList" type="java.util.ArrayList"/>
        <typeAlias alias="LinkedList" type="java.util.LinkedList"/>
    </typeAliases>
</configuration>

六、OrderDO

package cn.iocoder.springboot.lab17.dynamicdatasource.dataobject;

/** * 订单 DO */
public class OrderDO {
    

    /** * 订单编号 */
    private Integer id;
    /** * 用户编号 */
    private Integer userId;

    public Integer getId() {
    
        return id;
    }

    public OrderDO setId(Integer id) {
    
        this.id = id;
        return this;
    }

    public Integer getUserId() {
    
        return userId;
    }

    public OrderDO setUserId(Integer userId) {
    
        this.userId = userId;
        return this;
    }

    @Override
    public String toString() {
    
        return "OrderDO{" +
                "id=" + id +
                ", userId=" + userId +
                '}';
    }
}

七、OrderMapper

package cn.iocoder.springboot.lab17.dynamicdatasource.mapper;

import cn.iocoder.springboot.lab17.dynamicdatasource.dataobject.OrderDO;
import org.apache.ibatis.annotations.Param;
import org.springframework.stereotype.Repository;
/** * @description * @ClassName: OrderMapper * @author: Guo Xiuzhi [email protected] * @date: 2020/6/30 17:42 * @Copyright: */
@Repository
public interface OrderMapper {
    
    OrderDO selectById(@Param("id") Integer id);
    int insert(OrderDO entity);
}

八、OrderMapper.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="cn.iocoder.springboot.lab17.dynamicdatasource.mapper.OrderMapper">
    <sql id="FIELDS">
        id, user_id
    </sql>

    <select id="selectById" parameterType="Integer" resultType="OrderDO">
        SELECT
        <include refid="FIELDS"/>
        FROM orders
        WHERE id = #{id}
    </select>

    <insert id="insert" parameterType="OrderDO" useGeneratedKeys="true" keyProperty="id">
        INSERT INTO orders (
          user_id
        ) VALUES (
          #{userId}
        )
    </insert>
</mapper>

九、简单测试

创建 OrderMapperTest 测试类,我们来测试一下简单的 OrderMapper 的读写操作.代码如下：

// OrderMapper.java

@RunWith(SpringRunner.class)
@SpringBootTest(classes = Application.class)
public class OrderMapperTest {
    

 @Autowired
 private OrderMapper orderMapper;

 @Test
 public void testSelectById() {
     // 测试从库的负载均衡
 for (int i = 0; i < 10; i++) {
    
 OrderDO order = orderMapper.selectById(1);
 System.out.println(order);
 }
 }

 @Test
 public void testSelectById02() {
     // 测试强制访问主库
 try (HintManager hintManager = HintManager.getInstance()) {
    
 // 设置强制访问主库
 hintManager.setMasterRouteOnly();
 // 执行查询
 OrderDO order = orderMapper.selectById(1);
 System.out.println(order);
 }
 }

 @Test
 public void testInsert() {
     // 插入
 OrderDO order = new OrderDO();
 order.setUserId(10);
 orderMapper.insert(order);
 }
}
  • testSelectById() 方法,Test the load balancing query from the library.
  • testSelectById02() 方法,测试强制访问主库.在一些业务场景下,对数据延迟敏感,所以只能强制读取主库.此时,可以使用 HintManager 强制访问主库.
    • 不过要注意,在使用完后,需要去清理下 HintManager （HintManager 是基于线程变量,透传给 Sharding-JDBC 的内部实现）,避免污染下次请求,一直强制访问主库.
    • Sharding-JDBC 比较贴心,HintManager 实现了 AutoCloseable 接口,可以通过 Try-with-resources 机制,自动关闭.代码为：try (HintManager hintManager = HintManager.getInstance())
  • testInsert() 方法,Test the insertion of the main library.
    跑下测试用例.If it runs through,Indicates that the configuration is successful.
    另外,在 testSelectById()测试方法中,测试 slave Grouping is not really load balancing.所以在数据库中,Insert the data as follows.
主库：[id = 1, user_id = 1]
从库 01：[id = 1, user_id = 2]
从库 02：[id = 1, user_id = 3]

这样,Same by setting it manually id = 1 的记录,对应不同的 user_id ,Then we can observe testSelectById()The output of the test method.如果是,user_id = 2 和 user_i = 3 交替循环输出,Description is normal.
在这里插入图片描述

十、 详细测试

在 cn.iocoder.springboot.lab17.dynamicdatasource.service 包路径下,创建 OrderService.java 类.代码如下：

// OrderService.java

@Service
public class OrderService {
    

 @Autowired
 private OrderMapper orderMapper;

 @Transactional
 public void add(OrderDO order) {
    
 // <1.1> 这里先假模假样的读取一下.读取从库
 OrderDO exists = orderMapper.selectById(1);
 System.out.println(exists);

 // <1.2> 插入订单
 orderMapper.insert(order);

 // <1.3> 这里先假模假样的读取一下.读取主库
 exists = orderMapper.selectById(1);
 System.out.println(exists);
 }

 public OrderDO findById(Integer id) {
    
 return orderMapper.selectById(id);
 }
}
  • 在 #add(OrderDO order) 方法中,开启事务,插入一条订单记录.
    • <1.1> 处,Initiate an order query to the slave library.在 Sharding-JDBC 的读写分离策略里,默认读取从库.
    • <1.2> 处,Initiate an order write to the main library.写入,肯定是操作主库的.
    • <1.3> 处,Initiate an order query to the main database.在 Sharding-JDBC 中,读写分离约定：同一线程且同一数据库连接内,如有写入操作,以后的读操作均从主库读取,用于保证数据一致性.
  • 在 #findById(Integer id) 方法,Initiate an order query to the slave library.
    我们创建了 [OrderServiceTest]测试类,可以测试上面编写的两个方法.
package cn.iocoder.springboot.lab17.dynamicdatasource.service;

import cn.iocoder.springboot.lab17.dynamicdatasource.Application;
import cn.iocoder.springboot.lab17.dynamicdatasource.dataobject.OrderDO;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

/** * @description 测试orderservice * @ClassName: OrderServiceTest * @author: Guo Xiuzhi [email protected] * @date: 2020/7/1 10:52 * @Copyright: */
@RunWith(SpringRunner.class)
@SpringBootTest(classes = Application.class)
public class OrderServiceTest {
    

    @Autowired
    private OrderService orderService;

    @Test
    public void testAdd() {
    
        OrderDO order = new OrderDO();
        order.setUserId(20);
        orderService.add(order);
    }

    @Test
    public void testFindById() {
    
        OrderDO order = orderService.findById(1);
        System.out.println(order);
    }
}

运行测试：
在这里插入图片描述

参考

Sharding-JDBC 读写分离

原网站

版权声明
本文为[Rongji]所创，转载请带上原文链接，感谢
https://yzsam.com/2022/212/202207310039543633.html

边栏推荐

猜你喜欢

随机推荐