Dynamically switch data sources

Ideas

1. Customize an annotation @DataSource, This annotation can be added in the future service Layer method or class , Indicates that all methods in a method or class use a certain data source .
2. For the first step , If a method has @DataSource annotation , Then save the data source name required by this method to ThreadLocal.
3. Custom facets , Parse in slice @DataSource annotation , When a method or class has @DataSource When annotating , take @DataSource The data source marked by the annotation is stored in ThreadLocal in .
4. Last , When Mapper When it comes to execution , need DataSource, He will go automatically AbstractRoutingDataSource Class to find the required data source , All we need to do is AbstractRoutingDataSource Back in ThreadLocal The value in the .

Project code link :https://github.com/1040580896/dynamic_datasourece

1. Preliminary knowledge

Want to customize dynamic data source switching , You have to understand a class first AbstractRoutingDataSource：

AbstractRoutingDataSource Is in Spring2.0.1 Introduced in （ Note that Spring2.0.1 No Spring Boot2.0.1, So this is actually Spring A very old feature ）, This class acts as DataSource The routing intermediary , It can at run time , According to some key Value to dynamically switch to the real DataSource On .

The general usage is that you prepare various data sources in advance , Deposit into a Map in ,Map Of key Is the name of this data source ,Map Of value This is the specific data source , And then put this Map Configuration to AbstractRoutingDataSource in , Last , Every time you execute a database query , Take one key come out ,AbstractRoutingDataSource We will find the specific data source to perform this database operation .

2、 Introduce dependencies

First let's create one Spring Boot project , introduce Web、MyBatis as well as MySQL rely on , After the project is created successfully , Then manually add Druid and AOP rely on , as follows ：

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aop</artifactId>
</dependency>
<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>druid-spring-boot-starter</artifactId>
    <version>1.2.9</version>
</dependency>

3、 The configuration file

#  Data source configuration 
spring:
  datasource:
    type: com.alibaba.druid.pool.DruidDataSource
    driverClassName: com.mysql.cj.jdbc.Driver
    ds:
      #  Main database data source 
      master:
        url: jdbc:mysql://localhost:3306/test1?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=false&serverTimezone=GMT%2B8
        username: root
        password: th123456
        #  From library data source 
      slave:
        url: jdbc:mysql://localhost:3306/test2?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=false&serverTimezone=GMT%2B8
        username: root
        password: th123456
        #  The number of initial connections 
    initialSize: 5
    #  Minimum number of connection pools 
    minIdle: 10
    #  Maximum number of connection pools 
    maxActive: 20
    #  Configure the timeout time for getting connection waiting 
    maxWait: 60000
    #  Configure how often to test , Detects idle connections that need to be closed , In milliseconds 
    timeBetweenEvictionRunsMillis: 60000
    #  Configure the minimum lifetime of a connection in the pool , In milliseconds 
    minEvictableIdleTimeMillis: 300000
    #  Configure the maximum lifetime of a connection in the pool , In milliseconds 
    maxEvictableIdleTimeMillis: 900000
    #  Configure to detect whether the connection is valid 
    validationQuery: SELECT 1 FROM DUAL
    testWhileIdle: true
    testOnBorrow: false
    testOnReturn: false
    webStatFilter:
      enabled: true
    statViewServlet:
      enabled: true
      #  Set the whitelist , If it is not filled in, all accesses are allowed 
      allow:
      url-pattern: /druid/*
      #  Console management user name and password 
      login-username: tienchin
      login-password: 123456
    filter:
      stat:
        enabled: true
        #  slow SQL Record 
        log-slow-sql: true
        slow-sql-millis: 1000
        merge-sql: true
      wall:
        config:
          multi-statement-allow: tru

All are Druid The general configuration of , There's nothing to say , The only thing to note is the format of our entire configuration file .ds Configure all our data sources inside , Each data source has a name ,master Is the name of the default data source , Do not modify the , Other data sources can have custom names . At the end, we also configured Druid The monitoring function of , If the kids don't understand Druid The monitoring function of , You can see Spring Boot How to monitor SQL Operation of the ？.

4、 Customize annotations and facets

annotation

This annotation will be added to Service Layer method , When using this annotation , You need to specify a data source name , If not specified , Use by default master As a data source .

/** *  In the future, it can be added to a service  On a class or method , adopt value Property to specify which data source the class or method should use  */
@Retention(RetentionPolicy.RUNTIME)
@Target({
    ElementType.TYPE,ElementType.METHOD})
public @interface DataSource {
    

    /** *  If a method adds  @DataSource  But the data source name is not specified , So by default  master  data source  * @return */
    String value() default DataSourceType.DEFAULT_DS_NAME;
}

section

too AOP To parse the current custom annotation

It's used here ThreadLocal To save which data source is needed

MethodSignature signature = (MethodSignature) pjb.getSignature();

AnnotationUtils.findAnnotation Tool class

ThreadLocal Characteristics , Simply put, it is the data stored in which thread , In which thread can I get it out , You can't get it out with another thread , This ensures data security in a multithreaded environment .

@Component
@Aspect
public class DataSourceAspect {
    

    /** *  Tangent point  * * @annotation(com.th.annotation.DataSource  The expression method has  @DataSource  The annotation intercepts the method  * @within(com.th.annotation.DataSource)  Indicates if there is  @DataSource  annotation , Just intercept the methods in the class  */
    @Pointcut("@annotation(com.th.annotation.DataSource) || @within(com.th.annotation.DataSource)")
    public void pc() {
    

    }


    @Around("pc()")
    public Object around(ProceedingJoinPoint pjb) {
    
        // Get the annotation above the method 
        DataSource dataSource = getDataSourece(pjb);
        if (dataSource != null) {
    
            // The name of the data source 
            String value = dataSource.value();
            DynmaicDataSourceContextHolder.setDataSourceType(value);
        }
        try {
    
            return pjb.proceed();
        } catch (Throwable e) {
    
            e.printStackTrace();
        } finally {
    
            DynmaicDataSourceContextHolder.clearDataSourceType();
        }
        return null;
    }

    private DataSource getDataSourece(ProceedingJoinPoint pjb) {
    

        MethodSignature signature = (MethodSignature) pjb.getSignature();

        // Find the comment above the method 
        DataSource annotation = AnnotationUtils.findAnnotation(signature.getMethod(), DataSource.class);
        if (annotation != null) {
    
            // The description method has  DataSource  annotation 
            return annotation;
        }
        // Class 
        return AnnotationUtils.findAnnotation(signature.getDeclaringType(), DataSource.class);
    }
}

summary

1. First , We are dsPc() The tangent point is defined on the method , Let's intercept everything with @DataSource Method of annotation , At the same time, because the annotation can also be added to the class , If the annotation is added to the class , It means that all methods in the class use the data source .
2. Next, we define a surround notification , First, according to the current tangent point , call getDataSource Method to get @DataSource annotation , This annotation may come from methods or classes , The priority on method is higher than that on class . If the annotation you get is not empty , Then we are DynamicDataSourceContextHolder Set the current data source name in , Call the method after setting ; If the annotation is empty , Then call the method directly , No more data sources （ The default data source will be used automatically in the future ）. Finally, remember that after the method call is completed , from ThreadLocal Remove data source from .

4、 Get configuration dynamically

@ConfigurationProperties(prefix = "spring.datasource"), Pay attention to the structure , Choose the right type of storage

package com.th.datasource;

import com.alibaba.druid.pool.DruidDataSource;
import org.springframework.boot.context.properties.ConfigurationProperties;

import javax.sql.DataSource;
import java.util.Map;

/** * @program: dynamic_datasourece * @description: * @author: xiaokaixin * @create: 2022-05-22 15:45 **/
@ConfigurationProperties(prefix = "spring.datasource")
public class DruidProperties {
    

    private String type;
    private String driverClassName;
    private Map<String,Map<String,String>> ds;
    private Integer initialSize;
    private Integer minIdle;
    private Integer maxActive;
    private Integer maxWait;


    /** *  Build one on the outside  DruidDataSource  object , But this object contains only three core attributes  url,username,passowd *  In this method , Set public properties for this object  * @param druidDataSource * @return */
    public DataSource dataSource(DruidDataSource druidDataSource){
    
        druidDataSource.setInitialSize(initialSize);
        druidDataSource.setMaxActive(maxActive);
        druidDataSource.setMinIdle(minIdle);
        druidDataSource.setMaxWait(maxWait);
      	//.....
        return druidDataSource;
    }

    public String getType() {
    
        return type;
    }

    public void setType(String type) {
    
        this.type = type;
    }

    public String getDriverClassName() {
    
        return driverClassName;
    }

    public void setDriverClassName(String driverClassName) {
    
        this.driverClassName = driverClassName;
    }

    public Map<String, Map<String, String>> getDs() {
    
        return ds;
    }

    public void setDs(Map<String, Map<String, String>> ds) {
    
        this.ds = ds;
    }

    public Integer getInitialSize() {
    
        return initialSize;
    }

    public void setInitialSize(Integer initialSize) {
    
        this.initialSize = initialSize;
    }

    public Integer getMinIdle() {
    
        return minIdle;
    }

    public void setMinIdle(Integer minIdle) {
    
        this.minIdle = minIdle;
    }

    public Integer getMaxActive() {
    
        return maxActive;
    }

    public void setMaxActive(Integer maxActive) {
    
        this.maxActive = maxActive;
    }

    public Integer getMaxWait() {
    
        return maxWait;
    }

    public void setMaxWait(Integer maxWait) {
    
        this.maxWait = maxWait;
    }
}

5、 Storage data source name

Which data source to use for the current database operation ？ We have many different settings , Of course, the most convenient way is to store the currently used data source information in ThreadLocal in ,ThreadLocal Characteristics , Simply put, it is the data stored in which thread , In which thread can I get it out , You can't get it out with another thread , This ensures data security in a multithreaded environment .

ThreadLocal(DynmaicDataSourceContextHolder)

/** *  This class is used to store the name of the data source used by the current thread  */
public class DynmaicDataSourceContextHolder {
    

    private static ThreadLocal<String> CONTEXT_HOLDER = new ThreadLocal<>();

    public static void setDataSourceType(String dsType){
    
        CONTEXT_HOLDER.set(dsType);
    }

    public static String getDataSourceType(){
    
       return CONTEXT_HOLDER.get();
    }

    public static void clearDataSourceType(){
    
        CONTEXT_HOLDER.remove();
    }
}

6、 Load data source

@EnableConfigurationProperties(DruidProperties.class)

Let's talk about the function first ：

@EnableConfigurationProperties The function of annotation is ： Make use of @ConfigurationProperties Annotated class takes effect .

explain ：

If a configuration class only configures @ConfigurationProperties annotation , Instead of using @Component, So in IOC It's not available in the container properties Configuration file conversion bean. To put it bluntly @EnableConfigurationProperties Equivalent to using @ConfigurationProperties Class is injected once .
Tests found @ConfigurationProperties And @EnableConfigurationProperties It matters a lot

@Component
@EnableConfigurationProperties(DruidProperties.class)
public class LoadDataSource {
    

    @Autowired
    DruidProperties druidProperties;

    public Map<String, DataSource> loadAllDataSource(){
    

        Map<String,DataSource> map = new HashMap<>();
        Map<String, Map<String, String>> ds = druidProperties.getDs();
        try {
    
            Set<String> keySet = ds.keySet();
            for (String key : keySet) {
    
                //druidProperties.dataSource((DruidDataSource) DruidDataSourceFactory.createDataSource(ds.get(key)))  Public properties are also set 
                map.put(key, druidProperties.dataSource((DruidDataSource) DruidDataSourceFactory.createDataSource(ds.get(key))));
            }
        } catch (Exception e) {
    
            e.printStackTrace();
        }

        return map;
    }
}

7、 Data source switching

This is what we said at the beginning of the article AbstractRoutingDataSource 了 , This class has a method named determineCurrentLookupKey, When you need to use a data source , The system will automatically call this method , Get the tag of the current data source , Such as master perhaps slave Or other , After you get the mark , Then you can get a data source .

When we configure DynamicDataSource When , Two key parameters need to be configured , One is setTargetDataSources, This is all the current data sources , Tell all the current data sources to AbstractRoutingDataSource, These data sources are key-value In the form of （ Future basis determineCurrentLookupKey Method key You can get the specific data source ）; Another way is setDefaultTargetDataSource, This is the default data source , When we perform a database operation , If no data source is specified （ for example Service The layer method does not add @DataSource annotation ）, This data source is used by default .

Last , Let's put this bean Sign up to Spring In the container , as follows ：

@Component
public class DynmaicDataSource extends AbstractRoutingDataSource {
    

    public DynmaicDataSource(LoadDataSource loadDataSource) {
    

        //1、 Set all data sources 
        Map<String, DataSource> allDs = loadDataSource.loadAllDataSource();
        super.setTargetDataSources(new HashMap<>(allDs));
        //2、 Set the default data source 
        // future , Not all methods have  @DataSourece annotation   For which there is no  @DataSouce  Method of annotation , Which data source should be used ？
        super.setDefaultTargetDataSource(allDs.get(DataSourceType.DEFAULT_DS_NAME));
        //3
        super.afterPropertiesSet();
    }

    /** *  Used to return the data source name , When the system needs data sources , This method will be called automatically to get the name of the data source  * @return */
    @Override
    protected Object determineCurrentLookupKey() {
    
        return DynmaicDataSourceContextHolder.getDataSourceType();
    }
}

7、 test

All right. , Be accomplished , Let's test it again , Write a UserMapper：

@Mapper
public interface UserMapper {
    

    @Select("select * from user")
    List<User> getAllUsers();
}

adopt @DataSource Annotation to specify the data source of the specific operation , If this annotation is not used to specify , Use by default master data source .

Finally, go to the unit test to test , as follows ：

One more service：

@Service
@DataSource("slave")
public class UserService {
    

    @Autowired
    UserMapper userMapper;


    //@DataSource("master")
    public List<User> getAllUsers(){
    
        return userMapper.getAllUsers();
    }

}

Manually switch data sources based on the page

aop

@Order(10) The smaller it is , After execution aop Will overwrite the previous data source

@Aspect
@Component
@Order(10)
public class GloalbDataSourceAspetc {
    

    @Autowired
    HttpSession session;

    @Pointcut("execution(* com.th.service.*.*(..))")
    public void pc(){
    
    }

    @Around("pc()")
    public Object around(ProceedingJoinPoint pjb){
    
        DynmaicDataSourceContextHolder.setDataSourceType((String) session.getAttribute(DataSourceType.DS_SESSION_KEY));
        try {
    
            return pjb.proceed();
        } catch (Throwable e) {
    
            e.printStackTrace();
        }finally {
    
            DynmaicDataSourceContextHolder.clearDataSourceType();
        }
        return null;
    }
}

HttpSession session, It can also be stored in other ways

@RestController
public class DataSourceController {
    

    private static final Logger log = LoggerFactory.getLogger(DataSourceController.class);

    @Autowired
    UserService userService;

    /** *  Modify the interface of the data source  */
    @PostMapping("/dstype")
    public void setDsType(String dsType, HttpSession session){
    

        // Store the information of the data source in session
        session.setAttribute(DataSourceType.DS_SESSION_KEY,dsType);
        log.info(" Switch the data source to :{}",dsType);

    }
    @GetMapping("/users")
    public List<User> getAllUsers(){
    
        return userService.getAllUsers();
    }
}

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Title</title>
    <script src="jquery.js"></script>
</head>
<body>
<div>
     Please select a data source 
    <select name="" id="" onchange="dfChange(this.options[this.options.selectedIndex].value)">
        <option value=" Please select "> Please select </option>
        <option value="master">master</option>
        <option value="slave">slave</option>
    </select>
</div>

<div id="result">

</div>

<button onclick="loadData()"> Load data </button>

<script> function loadData(){
       $.get("/users",function (data){
       $("#result").html(JSON.stringify(data)) }) } function dfChange(value){
       $.post("/dstype",{
      dsType:value}) } </script>
</body>
</html>