精华内容
下载资源
问答
  • 2021-11-24 18:35:26

    我的项目中需要手动切换数据源,切换后显示不同数据源的内容;而dynamic-datasource-spring-boot-starter的文档要收费,就自己琢磨了一下,这里记录一下。

    需要解决的核心类:

    DynamicRoutingDataSource
    /*
     * Copyright © 2018 organization baomidou
     *
     * Licensed under the Apache License, Version 2.0 (the "License");
     * you may not use this file except in compliance with the License.
     * You may obtain a copy of the License at
     *
     *     http://www.apache.org/licenses/LICENSE-2.0
     *
     * Unless required by applicable law or agreed to in writing, software
     * distributed under the License is distributed on an "AS IS" BASIS,
     * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
     * See the License for the specific language governing permissions and
     * limitations under the License.
     */
    package com.baomidou.dynamic.datasource;
    
    import com.baomidou.dynamic.datasource.ds.AbstractRoutingDataSource;
    import com.baomidou.dynamic.datasource.ds.GroupDataSource;
    import com.baomidou.dynamic.datasource.exception.CannotFindDataSourceException;
    import com.baomidou.dynamic.datasource.provider.DynamicDataSourceProvider;
    import com.baomidou.dynamic.datasource.strategy.DynamicDataSourceStrategy;
    import com.baomidou.dynamic.datasource.strategy.LoadBalanceDynamicDataSourceStrategy;
    import com.baomidou.dynamic.datasource.toolkit.DatabasebUtils;
    import com.baomidou.dynamic.datasource.toolkit.DynamicDataSourceContextHolder;
    import lombok.Setter;
    import lombok.extern.slf4j.Slf4j;
    import org.springframework.beans.factory.DisposableBean;
    import org.springframework.beans.factory.InitializingBean;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.util.StringUtils;
    
    import javax.sql.DataSource;
    import java.util.HashMap;
    import java.util.List;
    import java.util.Map;
    import java.util.concurrent.ConcurrentHashMap;
    
    /**
     * 核心动态数据源组件
     *
     * @author TaoYu Kanyuxia
     * @since 1.0.0
     */
    @Slf4j
    public class DynamicRoutingDataSource extends AbstractRoutingDataSource implements InitializingBean, DisposableBean {
    
        private static final String UNDERLINE = "_";
        /**
         * 所有数据库
         */
        private final Map<String, DataSource> dataSourceMap = new ConcurrentHashMap<>();
        /**
         * 分组数据库
         */
        private final Map<String, GroupDataSource> groupDataSources = new ConcurrentHashMap<>();
        @Autowired
        private List<DynamicDataSourceProvider> providers;
        @Setter
        private Class<? extends DynamicDataSourceStrategy> strategy = LoadBalanceDynamicDataSourceStrategy.class;
        @Setter
        private String primary = "master";
        @Setter
        private Boolean strict = false;
        @Setter
        private Boolean p6spy = false;
        @Setter
        private Boolean seata = false;
    
        @Override
        public DataSource determineDataSource() {
            String dsKey = DynamicDataSourceContextHolder.peek();
            return getDataSource(dsKey);
        }
    
        private DataSource determinePrimaryDataSource() {
            log.debug("dynamic-datasource switch to the primary datasource");
            DataSource dataSource = dataSourceMap.get(primary);
            if (dataSource != null) {
                return dataSource;
            }
            GroupDataSource groupDataSource = groupDataSources.get(primary);
            if (groupDataSource != null) {
                return groupDataSource.determineDataSource();
            }
            throw new CannotFindDataSourceException("dynamic-datasource can not find primary datasource");
        }
    
        /**
         * 获取当前所有的数据源
         *
         * @return 当前所有数据源
         */
        public Map<String, DataSource> getCurrentDataSources() {
            return dataSourceMap;
        }
    
        /**
         * 获取的当前所有的分组数据源
         *
         * @return 当前所有的分组数据源
         */
        public Map<String, GroupDataSource> getCurrentGroupDataSources() {
            return groupDataSources;
        }
    
        /**
         * 获取数据源
         *
         * @param ds 数据源名称
         * @return 数据源
         */
        public DataSource getDataSource(String ds) {
            if (StringUtils.isEmpty(ds)) {
                return determinePrimaryDataSource();
            } else if (!groupDataSources.isEmpty() && groupDataSources.containsKey(ds)) {
                log.debug("dynamic-datasource switch to the datasource named [{}]", ds);
                return groupDataSources.get(ds).determineDataSource();
            } else if (dataSourceMap.containsKey(ds)) {
                log.debug("dynamic-datasource switch to the datasource named [{}]", ds);
                return dataSourceMap.get(ds);
            }
            if (strict) {
                throw new CannotFindDataSourceException("dynamic-datasource could not find a datasource named" + ds);
            }
            return determinePrimaryDataSource();
        }
    
        /**
         * 添加数据源
         *
         * @param ds         数据源名称
         * @param dataSource 数据源
         */
        public synchronized void addDataSource(String ds, DataSource dataSource) {
            DataSource oldDataSource = dataSourceMap.put(ds, dataSource);
            // 新数据源添加到分组
            this.addGroupDataSource(ds, dataSource);
            // 关闭老的数据源
            if (oldDataSource != null) {
                try {
                    closeDataSource(oldDataSource);
                } catch (Exception e) {
                    log.error("dynamic-datasource - remove the database named [{}]  failed", ds, e);
                }
            }
            log.info("dynamic-datasource - add a datasource named [{}] success", ds);
        }
    
        /**
         * 新数据源添加到分组
         *
         * @param ds         新数据源的名字
         * @param dataSource 新数据源
         */
        private void addGroupDataSource(String ds, DataSource dataSource) {
            if (ds.contains(UNDERLINE)) {
                String group = ds.split(UNDERLINE)[0];
                GroupDataSource groupDataSource = groupDataSources.get(group);
                if (groupDataSource == null) {
                    try {
                        groupDataSource = new GroupDataSource(group, strategy.getDeclaredConstructor().newInstance());
                        groupDataSources.put(group, groupDataSource);
                    } catch (Exception e) {
                        throw new RuntimeException("dynamic-datasource - add the datasource named " + ds + " error", e);
                    }
                }
                groupDataSource.addDatasource(ds, dataSource);
            }
        }
    
        /**
         * 删除数据源
         *
         * @param ds 数据源名称
         */
        public synchronized void removeDataSource(String ds) {
            if (!StringUtils.hasText(ds)) {
                throw new RuntimeException("remove parameter could not be empty");
            }
            if (primary.equals(ds)) {
                throw new RuntimeException("could not remove primary datasource");
            }
            if (dataSourceMap.containsKey(ds)) {
                DataSource dataSource = dataSourceMap.remove(ds);
                try {
                    closeDataSource(dataSource);
                } catch (Exception e) {
                    log.error("dynamic-datasource - remove the database named [{}]  failed", ds, e);
                }
    
                if (ds.contains(UNDERLINE)) {
                    String group = ds.split(UNDERLINE)[0];
                    if (groupDataSources.containsKey(group)) {
                        DataSource oldDataSource = groupDataSources.get(group).removeDatasource(ds);
                        if (oldDataSource == null) {
                            if (log.isWarnEnabled()) {
                                log.warn("fail for remove datasource from group. dataSource: {} ,group: {}", ds, group);
                            }
                        }
                    }
                }
                log.info("dynamic-datasource - remove the database named [{}] success", ds);
            } else {
                log.warn("dynamic-datasource - could not find a database named [{}]", ds);
            }
        }
    
        private void closeDataSource(DataSource dataSource) throws Exception {
            DatabasebUtils.closeDataSource(dataSource);
        }
    
        @Override
        public void destroy() throws Exception {
            log.info("dynamic-datasource start closing ....");
            for (Map.Entry<String, DataSource> item : dataSourceMap.entrySet()) {
                closeDataSource(item.getValue());
            }
            log.info("dynamic-datasource all closed success,bye");
        }
    
        @Override
        public void afterPropertiesSet() throws Exception {
            // 检查开启了配置但没有相关依赖
            checkEnv();
            // 添加并分组数据源
            Map<String, DataSource> dataSources = new HashMap<>();
            for (DynamicDataSourceProvider provider : providers) {
                dataSources.putAll(provider.loadDataSources());
            }
            for (Map.Entry<String, DataSource> dsItem : dataSources.entrySet()) {
                addDataSource(dsItem.getKey(), dsItem.getValue());
            }
            // 检测默认数据源是否设置
            if (groupDataSources.containsKey(primary)) {
                log.info("dynamic-datasource initial loaded [{}] datasource,primary group datasource named [{}]", dataSources.size(), primary);
            } else if (dataSourceMap.containsKey(primary)) {
                log.info("dynamic-datasource initial loaded [{}] datasource,primary datasource named [{}]", dataSources.size(), primary);
            } else {
                log.warn("dynamic-datasource initial loaded [{}] datasource,Please add your primary datasource or check your configuration", dataSources.size());
            }
        }
    
        private void checkEnv() {
            if (p6spy) {
                try {
                    Class.forName("com.p6spy.engine.spy.P6DataSource");
                    log.info("dynamic-datasource detect P6SPY plugin and enabled it");
                } catch (Exception e) {
                    throw new RuntimeException("dynamic-datasource enabled P6SPY ,however without p6spy dependency", e);
                }
            }
            if (seata) {
                try {
                    Class.forName("io.seata.rm.datasource.DataSourceProxy");
                    log.info("dynamic-datasource detect ALIBABA SEATA and enabled it");
                } catch (Exception e) {
                    throw new RuntimeException("dynamic-datasource enabled ALIBABA SEATA,however without seata dependency", e);
                }
            }
        }
    }
    @Setter
    private String primary = "master";

    修改默认的数据源别名字段:primary。

    
        @Autowired
        private DynamicDataSourceAutoConfiguration dynamicDSConfig;
    
     
        DataSource dataSource = dynamicDSConfig.dataSource();
        if (dataSource instanceof DynamicRoutingDataSource){
              DynamicRoutingDataSource currentDS = (DynamicRoutingDataSource)dataSource;
               //dbStr是自己要改动的数据源
              currentDS.setPrimary(dbStr);
         }
         //DynamicDataSourceContextHolder.push(dbStr);
    
    

    更多相关内容
  • 赠送jar包:dynamic-datasource-spring-boot-starter-3.4.1.jar; 赠送原API文档:dynamic-datasource-spring-boot-starter-3.4.1-javadoc.jar; 赠送源代码:dynamic-datasource-spring-boot-starter-3.4.1-sources...
  • 赠送jar包:dynamic-datasource-spring-boot-starter-3.4.1.jar; 赠送原API文档:dynamic-datasource-spring-boot-starter-3.4.1-javadoc.jar; 赠送源代码:dynamic-datasource-spring-boot-starter-3.4.1-sources...
  • Dynamic-Datasource (opens new window)- 基于 SpringBoot 的多数据源组件,功能强悍,支持 Seata 分布式事务。 支持 数据源分组 ,适用于多种场景 纯粹多库 读写分离 一主多从 混合模式。 支持数据库敏感配置信息 ...
  • Dynamic-Datasource (opens new window)- 基于 SpringBoot 的多数据源组件,功能强悍,支持 Seata 分布式事务。 支持 数据源分组 ,适用于多种场景 纯粹多库 读写分离 一主多从 混合模式。 支持数据库敏感配置信息 ...
  • mybatis-plus 与多数据源初体验
  • dynamic-datasource.zip

    2021-08-19 15:18:30
    dynamic-datasource.zip
  • dynamic-datasource-spring-boot-starter-3.2.0.jar
  • spring-boot-dynamic-datasource-started-master.rar
  • 从github上拉下来的dynamic-datasource-spring-boot-starter 3.1.1版本,用于学习交流
  • dynamic-datasource-aop-copy

    2020-07-09 23:15:21
    springboot结合自定义注解整合动态数据源实现通过注解切换,事务回滚springboot结合自定义注解整合动态数据源实现通过注解切换,事务回滚springboot结合自定义注解整合动态数据源实现通过注解切换,事务回滚
  • 基于springboot+mybatis轻量多数据源框架,同时支持mysql及oracle数据源链接。内部未附sql建表语句,可自行根据entity类进行表创建。
  • 基于 SpringBoot 多数据源 动态数据源 主从分离 快速启动器
  • dynamic-datasource-aop-loop.,springboot结合切面,加上自定义注解实现动态数据源切换,事务处理,循环遍历数据库文件
  • dynamic-datasource-spring-boot-starter 是一个基于springboot的快速集成多数据源的启动器。 github: https://github.com/baomidou/dynamic-datasource-spring-boot-starter 文档: ...

    目录

    一、简介

    二、源码分析

    2.1 整体结构

    2.2 自动配置怎么实现的

    2.3 如何集成众多连接池

    2.4 DS注解如何被拦截处理的

    2.5 多数据源动态切换及如何管理多数据源

    2.6 组数据源的负载均衡怎么实现的

    2.7 如何动态增减数据源

    三、总结


    一、简介

    dynamic-datasource-spring-boot-starter 是一个基于springboot的快速集成多数据源的启动器。
    github: https://github.com/baomidou/dynamic-datasource-spring-boot-starter
    文档: https://github.com/baomidou/dynamic-datasource-spring-boot-starter/wiki

    它与mybatis-plus属于同一个生态圈,很容易集成mybatis-plus。

    特性:

    1. 数据源分组,适用于多种场景:纯粹多库,读写分离,一主多从,混合模式。
    2. 内置敏感参数加密和启动初始化表结构schema数据库database。
    3. 提供对Druid、Mybatis-Plus、P6sy和Jndi的快速集成。
    4. 简化Druid和HikariCp配置,提供全局参数配置。
    5. 提供自定义数据源来源接口(默认使用yml或properties配置)。
    6. 提供项目启动后增减数据源方案。
    7. 提供Mybatis环境下的纯读写分离方案。
    8. 使用spel动态参数解析数据源,如从session、header或参数中获取数据源。(多租户架构神器)
    9. 提供多层数据源嵌套切换。(ServiceA >>> ServiceB >>> ServiceC,每个Service都是不同的数据源)
    10. 提供“不使用注解而使用正则或 spel”来切换数据源方案(实验性功能)。
    11. 基于seata的分布式事务支持。
    无论是动态增减数据源、数据源分组,还是纯粹多库、读写分离、一主多从、从其他数据库或者配置中心读取数据源,比起Mapper分包方式或自定义AOP注解切片方式实现多数据源方案,使用dynamic-datasource-spring-boot-starter要便捷许多,极大简化了工作量。

    具体使用,请参考示例:

    springboot整合mybatis-plus、druid连接池和多数据源配置_WorldMvp的专栏-CSDN博客_druid连接池多数据源

    二、源码分析

    本文源码解析基于3.3.1版本。由于篇幅限制,只截了重点代码,如果需要看完整代码,可以去github拉取。

    2.1 整体结构

    整体结构如下图所示:

     拿到代码后,要找到入手点,带着问题阅读代码。

    2.2 自动配置怎么实现的

    一般情况下,一个starter的最好入手点就是自动配置类,在 META-INF/spring.factories文件中指定自动配置类入口

    org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
    com.baomidou.dynamic.datasource.spring.boot.autoconfigure.DynamicDataSourceAutoConfiguration

    在spring.factories中,可以看到这个工程的自动配置类路径。从核心自动配置类DynamicDataSourceAutoConfiguration入手,可以认为这就是程序的Main入口。

    /**
     * 动态数据源核心自动配置类
     */
    @Slf4j
    @Configuration
    @AllArgsConstructor
    // 读取以spring.datasource.dynamic为前缀的配置
    @EnableConfigurationProperties(DynamicDataSourceProperties.class)
    // 需要在spring boot的DataSource bean自动配置之前注入我们的DataSource bean
    @AutoConfigureBefore(DataSourceAutoConfiguration.class)
    // 引入了Druid的autoConfig和各种数据源连接池的Creator
    @Import(value = {DruidDynamicDataSourceConfiguration.class, DynamicDataSourceCreatorAutoConfiguration.class})
    // 当含有spring.datasource.dynamic配置的时候,启用这个autoConfig
    @ConditionalOnProperty(prefix = DynamicDataSourceProperties.PREFIX, name = "enabled", havingValue = "true", matchIfMissing = true)
    public class DynamicDataSourceAutoConfiguration {
    
        private final DynamicDataSourceProperties properties;
    
         /**
         * 多数据源加载接口,默认从yml中读取多数据源配置
         * @return DynamicDataSourceProvider
         */
        @Bean
        @ConditionalOnMissingBean
        public DynamicDataSourceProvider dynamicDataSourceProvider() {
            Map<String, DataSourceProperty> datasourceMap = properties.getDatasource();
            return new YmlDynamicDataSourceProvider(datasourceMap);
        }
    
        /**
         * 注册自己的动态多数据源DataSource
         * @param dynamicDataSourceProvider 各种数据源连接池创建者
         * @return DataSource
         */
        @Bean
        @ConditionalOnMissingBean
        public DataSource dataSource(DynamicDataSourceProvider dynamicDataSourceProvider) {
            DynamicRoutingDataSource dataSource = new DynamicRoutingDataSource();
            dataSource.setPrimary(properties.getPrimary());
            dataSource.setStrict(properties.getStrict());
            dataSource.setStrategy(properties.getStrategy());
            dataSource.setProvider(dynamicDataSourceProvider);
            dataSource.setP6spy(properties.getP6spy());
            dataSource.setSeata(properties.getSeata());
            return dataSource;
        }
    
        /**
         * AOP切面,对DS注解过的方法进行增强,达到切换数据源的目的。
         * @param dsProcessor 动态参数解析数据源。如果数据源名称以#开头,就会进入这个解析器链。
         * @return advisor
         */
        @Role(value = BeanDefinition.ROLE_INFRASTRUCTURE)
        @Bean
        @ConditionalOnMissingBean
        public DynamicDataSourceAnnotationAdvisor dynamicDatasourceAnnotationAdvisor(DsProcessor dsProcessor) {
            // aop方法拦截器在方法调用前后做操作,设置动态参数解析器
            DynamicDataSourceAnnotationInterceptor interceptor = new DynamicDataSourceAnnotationInterceptor(properties.isAllowedPublicOnly(), dsProcessor);
            // 使用AbstractPointcutAdvisor将pointcut和advice连接构成切面
            DynamicDataSourceAnnotationAdvisor advisor = new DynamicDataSourceAnnotationAdvisor(interceptor);
            advisor.setOrder(properties.getOrder());
            return advisor;
        }
    
        /**
         * seata分布式事务支持
         *
         */
        @Role(value = BeanDefinition.ROLE_INFRASTRUCTURE)
        @ConditionalOnProperty(prefix = DynamicDataSourceProperties.PREFIX, name = "seata", havingValue = "false", matchIfMissing = true)
        @Bean
        public Advisor dynamicTransactionAdvisor() {
            AspectJExpressionPointcut pointcut = new AspectJExpressionPointcut();
            pointcut.setExpression("@annotation(com.baomidou.dynamic.datasource.annotation.DSTransactional)");
            return new DefaultPointcutAdvisor(pointcut, new DynamicTransactionAdvisor());
        }
    
        /**
         * 动态参数解析器链
         * @return DsProcessor
         */
        @Bean
        @ConditionalOnMissingBean
        public DsProcessor dsProcessor() {
            DsHeaderProcessor headerProcessor = new DsHeaderProcessor();
            DsSessionProcessor sessionProcessor = new DsSessionProcessor();
            DsSpelExpressionProcessor spelExpressionProcessor = new DsSpelExpressionProcessor();
            headerProcessor.setNextProcessor(sessionProcessor);
            sessionProcessor.setNextProcessor(spelExpressionProcessor);
            return headerProcessor;
        }
    
    }
    

    这里自动配置的五个Bean都是非常重要的。

    自动配置类的几个注解都写了注释,其中重要的是这个注解:

    // 读取以spring.datasource.dynamic为前缀的配置
    @EnableConfigurationProperties(DynamicDataSourceProperties.class)

    @EnableConfigurationProperties:使 @ConfigurationProperties 注解的类生效,主要是用来把properties或者yml配置文件转化为bean来使用,这个在实际使用中非常实用。

    @Slf4j
    @Getter
    @Setter
    @ConfigurationProperties(prefix = DynamicDataSourceProperties.PREFIX)
    public class DynamicDataSourceProperties {
    
        public static final String PREFIX = "spring.datasource.dynamic";
        public static final String HEALTH = PREFIX + ".health";
    
        /**
         * 必须设置默认的库,默认master
         */
        private String primary = "master";
        /**
         * 是否启用严格模式,默认不启动. 严格模式下未匹配到数据源直接报错, 非严格模式下则使用默认数据源primary所设置的数据源
         */
        private Boolean strict = false;
        /**
         * 是否使用p6spy输出,默认不输出
         */
        private Boolean p6spy = false;
        /**
         * 是否使用开启seata,默认不开启
         */
        private Boolean seata = false;
        /**
         * seata使用模式,默认AT
         */
        private SeataMode seataMode = SeataMode.AT;
        /**
         * 是否使用 spring actuator 监控检查,默认不检查
         */
        private boolean health = false;
        /**
         * 每一个数据源
         */
        private Map<String, DataSourceProperty> datasource = new LinkedHashMap<>();
        /**
         * 多数据源选择算法clazz,默认负载均衡算法
         */
        private Class<? extends DynamicDataSourceStrategy> strategy = LoadBalanceDynamicDataSourceStrategy.class;
        /**
         * aop切面顺序,默认优先级最高
         */
        private Integer order = Ordered.HIGHEST_PRECEDENCE;
        /**
         * Druid全局参数配置
         */
        @NestedConfigurationProperty
        private DruidConfig druid = new DruidConfig();
        /**
         * HikariCp全局参数配置
         */
        @NestedConfigurationProperty
        private HikariCpConfig hikari = new HikariCpConfig();
    
        /**
         * 全局默认publicKey
         */
        private String publicKey = CryptoUtils.DEFAULT_PUBLIC_KEY_STRING;
        /**
         * aop 切面是否只允许切 public 方法
         */
        private boolean allowedPublicOnly = true;
    }
    

    可以发现,我们在spring.datasource.dynamic配置的属性都会注入到这个配置Bean中。需要注意的是,使用了@NestedConfigurationProperty嵌套了其他的配置类。如果不清楚配置项是什么,看看DynamicDataSourceProperties这个类就清楚了。比如DruidConfig,这个DruidConfig是自定义的一个配置类,不是Druid里面的,它下面有个toProperties方法,为了实现yml配置中每个dataSource下面的durid可以独立配置(若不独立配置,则使用全局配置),根据全局配置和独立配置结合转换为Properties,然后在DruidDataSourceCreator类中根据这个配置创建druid连接池。

    2.3 如何集成众多连接池

    集成连接池配置项是通过DynamicDataSourceProperties配置类实现的,但是如何通过这些配置项生成真正的数据源连接池?让我们来看creator包。

    见名知意,可知支持哪些类型的数据源。

    在自动装配中配置DataSource的时候,new了一个DynamicRoutingDataSource,该类实现了InitializingBean接口,在bean初始化时做一些操作。

    @Slf4j
    public class DynamicRoutingDataSource extends AbstractRoutingDataSource implements InitializingBean, DisposableBean {
    
        private static final String UNDERLINE = "_";
        /**
         * 所有数据库
         */
        private final Map<String, DataSource> dataSourceMap = new ConcurrentHashMap<>();
        /**
         * 分组数据库
         */
        private final Map<String, GroupDataSource> groupDataSources = new ConcurrentHashMap<>();
        @Setter
        private DynamicDataSourceProvider provider;
        @Setter
        private Class<? extends DynamicDataSourceStrategy> strategy = LoadBalanceDynamicDataSourceStrategy.class;
        @Setter
        private String primary = "master";
        @Setter
        private Boolean strict = false;
        @Setter
        private Boolean p6spy = false;
        @Setter
        private Boolean seata = false;
    
        @Override
        public DataSource determineDataSource() {
            return getDataSource(DynamicDataSourceContextHolder.peek());
        }
    
        private DataSource determinePrimaryDataSource() {
            log.debug("dynamic-datasource switch to the primary datasource");
            return groupDataSources.containsKey(primary) ? groupDataSources.get(primary).determineDataSource() : dataSourceMap.get(primary);
        }
    
        /**
         * 获取当前所有的数据源
         *
         * @return 当前所有数据源
         */
        public Map<String, DataSource> getCurrentDataSources() {
            return dataSourceMap;
        }
    
        /**
         * 获取的当前所有的分组数据源
         *
         * @return 当前所有的分组数据源
         */
        public Map<String, GroupDataSource> getCurrentGroupDataSources() {
            return groupDataSources;
        }
    
        /**
         * 获取数据源
         *
         * @param ds 数据源名称
         * @return 数据源
         */
        public DataSource getDataSource(String ds) {
            if (StringUtils.isEmpty(ds)) {
                return determinePrimaryDataSource();
            } else if (!groupDataSources.isEmpty() && groupDataSources.containsKey(ds)) {
                log.debug("dynamic-datasource switch to the datasource named [{}]", ds);
                return groupDataSources.get(ds).determineDataSource();
            } else if (dataSourceMap.containsKey(ds)) {
                log.debug("dynamic-datasource switch to the datasource named [{}]", ds);
                return dataSourceMap.get(ds);
            }
            if (strict) {
                throw new RuntimeException("dynamic-datasource could not find a datasource named" + ds);
            }
            return determinePrimaryDataSource();
        }
    
        /**
         * 添加数据源
         *
         * @param ds         数据源名称
         * @param dataSource 数据源
         */
        public synchronized void addDataSource(String ds, DataSource dataSource) {
            DataSource oldDataSource = dataSourceMap.put(ds, dataSource);
            // 新数据源添加到分组
            this.addGroupDataSource(ds, dataSource);
            // 关闭老的数据源
            if (oldDataSource != null) {
                try {
                    closeDataSource(oldDataSource);
                } catch (Exception e) {
                    log.error("dynamic-datasource - remove the database named [{}]  failed", ds, e);
                }
            }
    
            log.info("dynamic-datasource - load a datasource named [{}] success", ds);
        }
    
        /**
         * 新数据源添加到分组
         *
         * @param ds         新数据源的名字
         * @param dataSource 新数据源
         */
        private void addGroupDataSource(String ds, DataSource dataSource) {
            if (ds.contains(UNDERLINE)) {
                String group = ds.split(UNDERLINE)[0];
                GroupDataSource groupDataSource = groupDataSources.get(group);
                if (groupDataSource == null) {
                    try {
                        groupDataSource = new GroupDataSource(group, strategy.getDeclaredConstructor().newInstance());
                        groupDataSources.put(group, groupDataSource);
                    } catch (Exception e) {
                        throw new RuntimeException("dynamic-datasource - add the datasource named " + ds + " error", e);
                    }
                }
                groupDataSource.addDatasource(ds, dataSource);
            }
        }
    
        /**
         * 删除数据源
         *
         * @param ds 数据源名称
         */
        public synchronized void removeDataSource(String ds) {
            if (!StringUtils.hasText(ds)) {
                throw new RuntimeException("remove parameter could not be empty");
            }
            if (primary.equals(ds)) {
                throw new RuntimeException("could not remove primary datasource");
            }
            if (dataSourceMap.containsKey(ds)) {
                DataSource dataSource = dataSourceMap.remove(ds);
                try {
                    closeDataSource(dataSource);
                } catch (Exception e) {
                    log.error("dynamic-datasource - remove the database named [{}]  failed", ds, e);
                }
    
                if (ds.contains(UNDERLINE)) {
                    String group = ds.split(UNDERLINE)[0];
                    if (groupDataSources.containsKey(group)) {
                        DataSource oldDataSource = groupDataSources.get(group).removeDatasource(ds);
                        if (oldDataSource == null) {
                            if (log.isWarnEnabled()) {
                                log.warn("fail for remove datasource from group. dataSource: {} ,group: {}", ds, group);
                            }
                        }
                    }
                }
                log.info("dynamic-datasource - remove the database named [{}] success", ds);
            } else {
                log.warn("dynamic-datasource - could not find a database named [{}]", ds);
            }
        }
    
        /**
         * 关闭数据源。
         * <pre>
         *    从3.2.0开启,如果是原生或使用 DataSourceCreator 创建的数据源会包装成ItemDataSource。
         *    ItemDataSource保留了最原始的数据源,其可直接关闭。
         *    如果不是DataSourceCreator创建的数据源则只有尝试解包装再关闭。
         * </pre>
         */
        private void closeDataSource(DataSource dataSource) throws Exception {
            if (dataSource instanceof ItemDataSource) {
                ((ItemDataSource) dataSource).close();
            } else {
                if (seata && dataSource instanceof DataSourceProxy) {
                    DataSourceProxy dataSourceProxy = (DataSourceProxy) dataSource;
                    dataSource = dataSourceProxy.getTargetDataSource();
                }
                if (p6spy && dataSource instanceof P6DataSource) {
                    Field realDataSourceField = P6DataSource.class.getDeclaredField("realDataSource");
                    realDataSourceField.setAccessible(true);
                    dataSource = (DataSource) realDataSourceField.get(dataSource);
                }
                Class<? extends DataSource> clazz = dataSource.getClass();
                Method closeMethod = clazz.getDeclaredMethod("close");
                closeMethod.invoke(dataSource);
            }
        }
    
        @Override
        public void destroy() throws Exception {
            log.info("dynamic-datasource start closing ....");
            for (Map.Entry<String, DataSource> item : dataSourceMap.entrySet()) {
                closeDataSource(item.getValue());
            }
            log.info("dynamic-datasource all closed success,bye");
        }
    
        @Override
        public void afterPropertiesSet() throws Exception {
            // 检查开启了配置但没有相关依赖
            checkEnv();
            // 添加并分组数据源
            Map<String, DataSource> dataSources = provider.loadDataSources();
            for (Map.Entry<String, DataSource> dsItem : dataSources.entrySet()) {
                addDataSource(dsItem.getKey(), dsItem.getValue());
            }
            // 检测默认数据源是否设置
            if (groupDataSources.containsKey(primary)) {
                log.info("dynamic-datasource initial loaded [{}] datasource,primary group datasource named [{}]", dataSources.size(), primary);
            } else if (dataSourceMap.containsKey(primary)) {
                log.info("dynamic-datasource initial loaded [{}] datasource,primary datasource named [{}]", dataSources.size(), primary);
            } else {
                throw new RuntimeException("dynamic-datasource Please check the setting of primary");
            }
        }
    
        private void checkEnv() {
            if (p6spy) {
                try {
                    Class.forName("com.p6spy.engine.spy.P6DataSource");
                    log.info("dynamic-datasource detect P6SPY plugin and enabled it");
                } catch (Exception e) {
                    throw new RuntimeException("dynamic-datasource enabled P6SPY ,however without p6spy dependency", e);
                }
            }
            if (seata) {
                try {
                    Class.forName("io.seata.rm.datasource.DataSourceProxy");
                    log.info("dynamic-datasource detect ALIBABA SEATA and enabled it");
                } catch (Exception e) {
                    throw new RuntimeException("dynamic-datasource enabled ALIBABA SEATA,however without seata dependency", e);
                }
            }
        }
    }

    这个类就是核心动态数据源组件。它将DataSource维护在map里,这里重点看如何创建数据源连接池。它所做的操作就是:初始化时从provider获取创建好的数据源map,然后解析这个map对其分组。下面来看看这个provider里面是如何创建这个数据源map的。

        @Bean
        @ConditionalOnMissingBean
        public DynamicDataSourceProvider dynamicDataSourceProvider() {
            Map<String, DataSourceProperty> datasourceMap = properties.getDatasource();
            return new YmlDynamicDataSourceProvider(datasourceMap);
        }

    在自动装配中注入的这个bean,是通过yml读取配置文件(还有通过jdbc读取配置文件)生成的。
    通过跟踪provider.loadDataSources()方法,发现在createDataSourceMap()方法中调用的是dataSourceCreator.createDataSource(dataSourceProperty, publicKey)。进一步追踪可以发现,具体使用哪种类型的连接池,是在DynamicDataSourceCreatorAutoConfiguration自动配置类中完成的。

    @Slf4j
    @Configuration
    @AllArgsConstructor
    @EnableConfigurationProperties(DynamicDataSourceProperties.class)
    public class DynamicDataSourceCreatorAutoConfiguration {
    
        private static final int JNDI_ORDER = 1000;
        private static final int DRUID_ORDER = 2000;
        private static final int HIKARI_ORDER = 3000;
        private static final int DEFAULT_ORDER = 5000;
        private final DynamicDataSourceProperties properties;
    
        @Primary
        @Bean
        @ConditionalOnMissingBean
        public DefaultDataSourceCreator dataSourceCreator(List<DataSourceCreator> dataSourceCreators) {
            DefaultDataSourceCreator defaultDataSourceCreator = new DefaultDataSourceCreator();
            defaultDataSourceCreator.setProperties(properties);
            defaultDataSourceCreator.setDataSourceCreators(dataSourceCreators);
            return defaultDataSourceCreator;
        }
    
        @Bean
        @Order(DEFAULT_ORDER)
        @ConditionalOnMissingBean
        public BasicDataSourceCreator basicDataSourceCreator() {
            return new BasicDataSourceCreator();
        }
    
        @Bean
        @Order(JNDI_ORDER)
        @ConditionalOnMissingBean
        public JndiDataSourceCreator jndiDataSourceCreator() {
            return new JndiDataSourceCreator();
        }
    
        /**
         * 存在Druid数据源时, 加入创建器
         */
        @ConditionalOnClass(DruidDataSource.class)
        @Configuration
        public class DruidDataSourceCreatorConfiguration {
            @Bean
            @Order(DRUID_ORDER)
            @ConditionalOnMissingBean
            public DruidDataSourceCreator druidDataSourceCreator() {
                return new DruidDataSourceCreator(properties.getDruid());
            }
    
        }
    
        /**
         * 存在Hikari数据源时, 加入创建器
         */
        @ConditionalOnClass(HikariDataSource.class)
        @Configuration
        public class HikariDataSourceCreatorConfiguration {
            @Bean
            @Order(HIKARI_ORDER)
            @ConditionalOnMissingBean
            public HikariDataSourceCreator hikariDataSourceCreator() {
                return new HikariDataSourceCreator(properties.getHikari());
            }
        }
    
    }
    

    2.4 DS注解如何被拦截处理的

    注解拦截处理离不开AOP,这里介绍代码中如何使用AOP。

    我们还是从DynamicDataSourceAutoConfiguration入口配置类中dynamicDatasourceAnnotationAdvisor(DsProcessor dsProcessor)方法入手,该方法注入了一个DynamicDataSourceAnnotationAdvisor类型的bean对象。

    在讲解这个advisor之前,这里多提一点AOP相关的。

    在 Spring AOP 中,有 3 个常用的概念:Advices 、 Pointcut 、 Advisor ,解释如下:
    Advice :一个 method 执行前或执行后的动作。
    Pointcut :根据 method 的名字或者正则表达式等方式,去拦截一个 method 。
    Advisor : Advice 和 Pointcut 组成的独立的单元,并且能够传给 proxy factory 对象。

    @Component
    //声明这是一个切面Bean
    @Aspect
    public class ServiceAspect {
        //配置切入点,该方法无方法体,主要为方便同类中其他方法使用此处配置的切入点
        @Pointcut("execution(* com.xxx.aop.service..*(..))")
        public void aspect() {
        }
    
        /*
         * 配置前置通知,使用在方法aspect()上注册的切入点
         * 同时接受JoinPoint切入点对象,可以没有该参数
         */
        @Before("aspect()")
        public void before(JoinPoint joinPoint) {
        }
    
        //配置后置通知,使用在方法aspect()上注册的切入点
        @After("aspect()")
        public void after(JoinPoint joinPoint) {
        }
    
        //配置环绕通知,使用在方法aspect()上注册的切入点
        @Around("aspect()")
        public void around(JoinPoint joinPoint) {
        }
    
        //配置后置返回通知,使用在方法aspect()上注册的切入点
        @AfterReturning("aspect()")
        public void afterReturn(JoinPoint joinPoint) {
        }
    
        //配置抛出异常后通知,使用在方法aspect()上注册的切入点
        @AfterThrowing(pointcut = "aspect()", throwing = "ex")
        public void afterThrow(JoinPoint joinPoint, Exception ex) {
        }
    }
    

    平时,我们可能使用这种AspectJ注解多一些,通过@Aspect注解的方式来声明切面,spring会通过我们的AspectJ注解(比如@Pointcut、@Before) 动态生成各个Advisor。

    Spring还提供了另一种切面----顾问(Advisor),其可以完成更为复杂的切面织入功能。我们可以通过直接继承AbstractPointcutAdvisor来提供切面逻辑,生成对应的Advisor实例,如下图:

    其中,最重要的就是getAdvicegetPointcut方法,可以简单认为advisor=advice+pointcut

    再回到DynamicDataSourceAutoConfiguration入口配置类中dynamicDatasourceAnnotationAdvisor(DsProcessor dsProcessor)方法,跟进观察DynamicDataSourceAnnotationAdvisor类:

    public class DynamicDataSourceAnnotationAdvisor extends AbstractPointcutAdvisor implements BeanFactoryAware {
    
        // 通知
        private final Advice advice;
        // 切入点
        private final Pointcut pointcut;
    
        public DynamicDataSourceAnnotationAdvisor(@NonNull DynamicDataSourceAnnotationInterceptor dynamicDataSourceAnnotationInterceptor) {
            this.advice = dynamicDataSourceAnnotationInterceptor;
            this.pointcut = buildPointcut();
        }
    
        @Override
        public Pointcut getPointcut() {
            return this.pointcut;
        }
    
        @Override
        public Advice getAdvice() {
            return this.advice;
        }
    
        @Override
        public void setBeanFactory(BeanFactory beanFactory) throws BeansException {
            if (this.advice instanceof BeanFactoryAware) {
                ((BeanFactoryAware) this.advice).setBeanFactory(beanFactory);
            }
        }
    
        private Pointcut buildPointcut() {
            // 类级别
            Pointcut cpc = new AnnotationMatchingPointcut(DS.class, true);
            // 方法级别
            Pointcut mpc = new AnnotationMethodPoint(DS.class);
            // 合并类和方法上添加的注解,类上的注解会绑定到每个方法上。
            return new ComposablePointcut(cpc).union(mpc);
        }
    
        /**
         * In order to be compatible with the spring lower than 5.0
         */
        private static class AnnotationMethodPoint implements Pointcut {
    
            private final Class<? extends Annotation> annotationType;
    
            public AnnotationMethodPoint(Class<? extends Annotation> annotationType) {
                Assert.notNull(annotationType, "Annotation type must not be null");
                this.annotationType = annotationType;
            }
    
            @Override
            public ClassFilter getClassFilter() {
                return ClassFilter.TRUE;
            }
    
            @Override
            public MethodMatcher getMethodMatcher() {
                return new AnnotationMethodMatcher(annotationType);
            }
    
            private static class AnnotationMethodMatcher extends StaticMethodMatcher {
                private final Class<? extends Annotation> annotationType;
    
                public AnnotationMethodMatcher(Class<? extends Annotation> annotationType) {
                    this.annotationType = annotationType;
                }
    
                @Override
                public boolean matches(Method method, Class<?> targetClass) {
                    if (matchesMethod(method)) {
                        return true;
                    }
                    // Proxy classes never have annotations on their redeclared methods.
                    if (Proxy.isProxyClass(targetClass)) {
                        return false;
                    }
                    // The method may be on an interface, so let's check on the target class as well.
                    Method specificMethod = AopUtils.getMostSpecificMethod(method, targetClass);
                    return (specificMethod != method && matchesMethod(specificMethod));
                }
    
                private boolean matchesMethod(Method method) {
                    return AnnotatedElementUtils.hasAnnotation(method, this.annotationType);
                }
            }
        }
    }
    

    现在看下@DS注解的advisor实现,在buildPointcut方法里拦截了被@DS注解的方法或类,并且使用ComposablePointcut组合切入点,可以实现方法优先级大于类优先级的特性。DynamicDataSourceAnnotationAdvisor通过构造方法传过来的参数类型是DynamicDataSourceAnnotationInterceptor类,跟进观察该类:

    public class DynamicDataSourceAnnotationInterceptor implements MethodInterceptor {
    
        /**
         * The identification of SPEL.
         */
        private static final String DYNAMIC_PREFIX = "#";
    
        private final DataSourceClassResolver dataSourceClassResolver;
        private final DsProcessor dsProcessor;
    
        public DynamicDataSourceAnnotationInterceptor(Boolean allowedPublicOnly, DsProcessor dsProcessor) {
            dataSourceClassResolver = new DataSourceClassResolver(allowedPublicOnly);
            this.dsProcessor = dsProcessor;
        }
    
        @Override
        public Object invoke(MethodInvocation invocation) throws Throwable {
            String dsKey = determineDatasourceKey(invocation);
            // 把获取到的数据源标识(如master)存入本地线程
            DynamicDataSourceContextHolder.push(dsKey);
            try {
                return invocation.proceed();
            } finally {
                DynamicDataSourceContextHolder.poll();
            }
        }
    
        private String determineDatasourceKey(MethodInvocation invocation) {
            String key = dataSourceClassResolver.findDSKey(invocation.getMethod(), invocation.getThis());
            // 如果DS注解内容是以#开头,则解析动态最终值;否则,直接返回。
            return (!key.isEmpty() && key.startsWith(DYNAMIC_PREFIX)) ? dsProcessor.determineDatasource(invocation, key) : key;
        }
    }
    

    这是它的advice通知(也可以说是方法拦截器)执行的动作:在要切换数据源的方法执行前,将“切换的数据源”放入了holder里,等方法执行完后在finally中释放掉,完成当前数据源的切换。该类的determineDatasource()方法决定具体使用哪个数据源。

    2.5 多数据源动态切换及如何管理多数据源

    在DynamicDataSourceAnnotationInterceptor类中切换数据源的方法中,前后调用了DynamicDataSourceContextHolder.push()和poll()。跟进观察下DynamicDataSourceContextHolder方法:

    public final class DynamicDataSourceContextHolder {
    
        /**
         * 为什么要用链表存储(准确的是栈)
         * <pre>
         * 为了支持嵌套切换,如ABC三个service都是不同的数据源
         * 其中A的某个业务要调B的方法,B的方法需要调用C的方法。一级一级调用切换,形成了链。
         * 传统的只设置当前线程的方式不能满足此业务需求,必须使用栈,后进先出。
         * </pre>
         */
        private static final ThreadLocal<Deque<String>> LOOKUP_KEY_HOLDER = new NamedThreadLocal<Deque<String>>("dynamic-datasource") {
            @Override
            protected Deque<String> initialValue() {
                return new ArrayDeque<>();
            }
        };
    
        private DynamicDataSourceContextHolder() {
        }
    
        /**
         * 获得当前线程数据源
         *
         * @return 数据源名称
         */
        public static String peek() {
            return LOOKUP_KEY_HOLDER.get().peek();
        }
    
        /**
         * 设置当前线程数据源
         * <p>
         * 如非必要不要手动调用,调用后确保最终清除
         * </p>
         *
         * @param ds 数据源名称
         */
        public static void push(String ds) {
            LOOKUP_KEY_HOLDER.get().push(StringUtils.isEmpty(ds) ? "" : ds);
        }
    
        /**
         * 清空当前线程数据源
         * <p>
         * 如果当前线程是连续切换数据源 只会移除掉当前线程的数据源名称
         * </p>
         */
        public static void poll() {
            Deque<String> deque = LOOKUP_KEY_HOLDER.get();
            deque.poll();
            if (deque.isEmpty()) {
                LOOKUP_KEY_HOLDER.remove();
            }
        }
    
        /**
         * 强制清空本地线程
         * <p>
         * 防止内存泄漏,如手动调用了push可调用此方法确保清除
         * </p>
         */
        public static void clear() {
            LOOKUP_KEY_HOLDER.remove();
        }
    }
    

    它使用栈处理当前数据源。使用了ArrayDeque这个线程不安全的双端队列来实现栈功能,比原生Stack性能好。使用栈数据结构,嵌套过程中进来push、出去就pop,实现了这个嵌套调用service的业务需求。

    下面来看切换数据源的核心类AbstractRoutingDataSource:

     该项目没有使用Spring的AbstractRoutingDataSource做多数据源动态切换,而是自定义实现了一个AbstractRoutingDataSource类,如下所示:

    public abstract class AbstractRoutingDataSource extends AbstractDataSource {
    
        /**
         * 由子类实现,决定最终数据源。
         *
         * @return 数据源
         */
        protected abstract DataSource determineDataSource();
    
        @Override
        public Connection getConnection() throws SQLException {
            String xid = TransactionContext.getXID();
            if (StringUtils.isEmpty(xid)) {
                return determineDataSource().getConnection();
            } else {
                String ds = DynamicDataSourceContextHolder.peek();
                ConnectionProxy connection = ConnectionFactory.getConnection(ds);
                return connection == null ? getConnectionProxy(ds, determineDataSource().getConnection()) : connection;
            }
        }
    
        @Override
        public Connection getConnection(String username, String password) throws SQLException {
            String xid = TransactionContext.getXID();
            if (StringUtils.isEmpty(xid)) {
                return determineDataSource().getConnection(username, password);
            } else {
                String ds = DynamicDataSourceContextHolder.peek();
                ConnectionProxy connection = ConnectionFactory.getConnection(ds);
                return connection == null ? getConnectionProxy(ds, determineDataSource().getConnection(username, password))
                        : connection;
            }
        }
    
        private Connection getConnectionProxy(String ds, Connection connection) {
            ConnectionProxy connectionProxy = new ConnectionProxy(connection, ds);
            ConnectionFactory.putConnection(ds, connectionProxy);
            return connectionProxy;
        }
    
        @Override
        @SuppressWarnings("unchecked")
        public <T> T unwrap(Class<T> iface) throws SQLException {
            if (iface.isInstance(this)) {
                return (T) this;
            }
            return determineDataSource().unwrap(iface);
        }
    
        @Override
        public boolean isWrapperFor(Class<?> iface) throws SQLException {
            return (iface.isInstance(this) || determineDataSource().isWrapperFor(iface));
        }
    }
    

    该抽象类也是实现了DataSource接口的getConnection方法,现在来看下子类如何实现determineDataSource方法:

    @Slf4j
    public class DynamicRoutingDataSource extends AbstractRoutingDataSource implements InitializingBean, DisposableBean {
    
        private static final String UNDERLINE = "_";
        /**
         * 所有数据库
         */
        private final Map<String, DataSource> dataSourceMap = new ConcurrentHashMap<>();
        /**
         * 分组数据库
         */
        private final Map<String, GroupDataSource> groupDataSources = new ConcurrentHashMap<>();
        @Setter
        private DynamicDataSourceProvider provider;
        @Setter
        private Class<? extends DynamicDataSourceStrategy> strategy = LoadBalanceDynamicDataSourceStrategy.class;
        @Setter
        private String primary = "master";
        @Setter
        private Boolean strict = false;
        @Setter
        private Boolean p6spy = false;
        @Setter
        private Boolean seata = false;
    
        @Override
        public DataSource determineDataSource() {
            return getDataSource(DynamicDataSourceContextHolder.peek());
        }
    
        private DataSource determinePrimaryDataSource() {
            log.debug("dynamic-datasource switch to the primary datasource");
            return groupDataSources.containsKey(primary) ? groupDataSources.get(primary).determineDataSource() : dataSourceMap.get(primary);
        }
    
        /**
         * 获取当前所有的数据源
         *
         * @return 当前所有数据源
         */
        public Map<String, DataSource> getCurrentDataSources() {
            return dataSourceMap;
        }
    
        /**
         * 获取的当前所有的分组数据源
         *
         * @return 当前所有的分组数据源
         */
        public Map<String, GroupDataSource> getCurrentGroupDataSources() {
            return groupDataSources;
        }
    
        /**
         * 获取数据源
         *
         * @param ds 数据源名称
         * @return 数据源
         */
        public DataSource getDataSource(String ds) {
            if (StringUtils.isEmpty(ds)) {
                return determinePrimaryDataSource();
            } else if (!groupDataSources.isEmpty() && groupDataSources.containsKey(ds)) {
                log.debug("dynamic-datasource switch to the datasource named [{}]", ds);
                return groupDataSources.get(ds).determineDataSource();
            } else if (dataSourceMap.containsKey(ds)) {
                log.debug("dynamic-datasource switch to the datasource named [{}]", ds);
                return dataSourceMap.get(ds);
            }
            if (strict) {
                throw new RuntimeException("dynamic-datasource could not find a datasource named" + ds);
            }
            return determinePrimaryDataSource();
        }
    
        /**
         * 添加数据源
         *
         * @param ds         数据源名称
         * @param dataSource 数据源
         */
        public synchronized void addDataSource(String ds, DataSource dataSource) {
            DataSource oldDataSource = dataSourceMap.put(ds, dataSource);
            // 新数据源添加到分组
            this.addGroupDataSource(ds, dataSource);
            // 关闭老的数据源
            if (oldDataSource != null) {
                try {
                    closeDataSource(oldDataSource);
                } catch (Exception e) {
                    log.error("dynamic-datasource - remove the database named [{}]  failed", ds, e);
                }
            }
    
            log.info("dynamic-datasource - load a datasource named [{}] success", ds);
        }
    
        /**
         * 新数据源添加到分组
         *
         * @param ds         新数据源的名字
         * @param dataSource 新数据源
         */
        private void addGroupDataSource(String ds, DataSource dataSource) {
            if (ds.contains(UNDERLINE)) {
                String group = ds.split(UNDERLINE)[0];
                GroupDataSource groupDataSource = groupDataSources.get(group);
                if (groupDataSource == null) {
                    try {
                        groupDataSource = new GroupDataSource(group, strategy.getDeclaredConstructor().newInstance());
                        groupDataSources.put(group, groupDataSource);
                    } catch (Exception e) {
                        throw new RuntimeException("dynamic-datasource - add the datasource named " + ds + " error", e);
                    }
                }
                groupDataSource.addDatasource(ds, dataSource);
            }
        }
    
        /**
         * 删除数据源
         *
         * @param ds 数据源名称
         */
        public synchronized void removeDataSource(String ds) {
            if (!StringUtils.hasText(ds)) {
                throw new RuntimeException("remove parameter could not be empty");
            }
            if (primary.equals(ds)) {
                throw new RuntimeException("could not remove primary datasource");
            }
            if (dataSourceMap.containsKey(ds)) {
                DataSource dataSource = dataSourceMap.remove(ds);
                try {
                    closeDataSource(dataSource);
                } catch (Exception e) {
                    log.error("dynamic-datasource - remove the database named [{}]  failed", ds, e);
                }
    
                if (ds.contains(UNDERLINE)) {
                    String group = ds.split(UNDERLINE)[0];
                    if (groupDataSources.containsKey(group)) {
                        DataSource oldDataSource = groupDataSources.get(group).removeDatasource(ds);
                        if (oldDataSource == null) {
                            if (log.isWarnEnabled()) {
                                log.warn("fail for remove datasource from group. dataSource: {} ,group: {}", ds, group);
                            }
                        }
                    }
                }
                log.info("dynamic-datasource - remove the database named [{}] success", ds);
            } else {
                log.warn("dynamic-datasource - could not find a database named [{}]", ds);
            }
        }
    
        /**
         * 关闭数据源。
         * <pre>
         *    从3.2.0开启,如果是原生或使用 DataSourceCreator 创建的数据源会包装成ItemDataSource。
         *    ItemDataSource保留了最原始的数据源,其可直接关闭。
         *    如果不是DataSourceCreator创建的数据源则只有尝试解包装再关闭。
         * </pre>
         */
        private void closeDataSource(DataSource dataSource) throws Exception {
            if (dataSource instanceof ItemDataSource) {
                ((ItemDataSource) dataSource).close();
            } else {
                if (seata && dataSource instanceof DataSourceProxy) {
                    DataSourceProxy dataSourceProxy = (DataSourceProxy) dataSource;
                    dataSource = dataSourceProxy.getTargetDataSource();
                }
                if (p6spy && dataSource instanceof P6DataSource) {
                    Field realDataSourceField = P6DataSource.class.getDeclaredField("realDataSource");
                    realDataSourceField.setAccessible(true);
                    dataSource = (DataSource) realDataSourceField.get(dataSource);
                }
                Class<? extends DataSource> clazz = dataSource.getClass();
                Method closeMethod = clazz.getDeclaredMethod("close");
                closeMethod.invoke(dataSource);
            }
        }
    
        @Override
        public void destroy() throws Exception {
            log.info("dynamic-datasource start closing ....");
            for (Map.Entry<String, DataSource> item : dataSourceMap.entrySet()) {
                closeDataSource(item.getValue());
            }
            log.info("dynamic-datasource all closed success,bye");
        }
    
        @Override
        public void afterPropertiesSet() throws Exception {
            // 检查开启了配置但没有相关依赖
            checkEnv();
            // 添加并分组数据源
            Map<String, DataSource> dataSources = provider.loadDataSources();
            for (Map.Entry<String, DataSource> dsItem : dataSources.entrySet()) {
                addDataSource(dsItem.getKey(), dsItem.getValue());
            }
            // 检测默认数据源是否设置
            if (groupDataSources.containsKey(primary)) {
                log.info("dynamic-datasource initial loaded [{}] datasource,primary group datasource named [{}]", dataSources.size(), primary);
            } else if (dataSourceMap.containsKey(primary)) {
                log.info("dynamic-datasource initial loaded [{}] datasource,primary datasource named [{}]", dataSources.size(), primary);
            } else {
                throw new RuntimeException("dynamic-datasource Please check the setting of primary");
            }
        }
    
        private void checkEnv() {
            if (p6spy) {
                try {
                    Class.forName("com.p6spy.engine.spy.P6DataSource");
                    log.info("dynamic-datasource detect P6SPY plugin and enabled it");
                } catch (Exception e) {
                    throw new RuntimeException("dynamic-datasource enabled P6SPY ,however without p6spy dependency", e);
                }
            }
            if (seata) {
                try {
                    Class.forName("io.seata.rm.datasource.DataSourceProxy");
                    log.info("dynamic-datasource detect ALIBABA SEATA and enabled it");
                } catch (Exception e) {
                    throw new RuntimeException("dynamic-datasource enabled ALIBABA SEATA,however without seata dependency", e);
                }
            }
        }
    }

    之前已经将creator生成的数据源连接池放入map中,现在从map中获取数据源即可,可以发现数据源组优先于单数据源

    2.6 组数据源的负载均衡怎么实现的

    跟进观察DynamicRoutingDataSource的getDataSource方法:

     else if (!groupDataSources.isEmpty() && groupDataSources.containsKey(ds)) {
                log.debug("dynamic-datasource switch to the datasource named [{}]", ds);
                return groupDataSources.get(ds).determineDataSource();
            }

    如果数据源组不为空,并且DS注解写的数据源组名存在,那么就会通过GroupDataSource类的

    determineDataSource方法在数据源组中选取一个数据源。

    @Data
    public class GroupDataSource {
    
        private String groupName;
        // 数据源切换策略
        private DynamicDataSourceStrategy dynamicDataSourceStrategy;
    
        private Map<String, DataSource> dataSourceMap = new ConcurrentHashMap<>();
    
        public GroupDataSource(String groupName, DynamicDataSourceStrategy dynamicDataSourceStrategy) {
            this.groupName = groupName;
            this.dynamicDataSourceStrategy = dynamicDataSourceStrategy;
        }
    
        /**
         * add a new datasource to this group
         *
         * @param ds         the name of the datasource
         * @param dataSource datasource
         */
        public DataSource addDatasource(String ds, DataSource dataSource) {
            return dataSourceMap.put(ds, dataSource);
        }
    
        /**
         * @param ds the name of the datasource
         */
        public DataSource removeDatasource(String ds) {
            return dataSourceMap.remove(ds);
        }
        // 根据切换策略,获取一个数据源。
        public DataSource determineDataSource() {
            return dynamicDataSourceStrategy.determineDataSource(new ArrayList<>(dataSourceMap.values()));
        }
    
        public int size() {
            return dataSourceMap.size();
        }
    }

    GroupDataSource使用策略模式来决定一个数据源,目前实现的策略有两种:随机和轮询,默认是轮询。在DynamicDataSourceProperties属性中写了默认值,也可以通过配置文件配置。

    public class LoadBalanceDynamicDataSourceStrategy implements DynamicDataSourceStrategy {
    
        /**
         * 负载均衡计数器
         */
        private final AtomicInteger index = new AtomicInteger(0);
    
        @Override
        public DataSource determineDataSource(List<DataSource> dataSources) {
            return dataSources.get(Math.abs(index.getAndAdd(1) % dataSources.size()));
        }
    }

    如果想通过jdbc获取数据源,该项目有个自定义的抽象类AbstractJdbcDataSourceProvider,需要实现其executeStmt方法,即:从其他数据库查询出url、username、password等信息(就是在yml配置的属性),然后拼接成一个配置对象DataSourceProperty去调用createDataSourceMap方法。

    2.7 如何动态增减数据源

    这个也是很实用的功能,其实现还是通过DynamicRoutingDataSource这个核心动态数据源组件完成。

    @Slf4j
    public class DynamicRoutingDataSource extends AbstractRoutingDataSource implements InitializingBean, DisposableBean {
     省略
    /**
         * 删除数据源
         *
         * @param ds 数据源名称
         */
        public synchronized void removeDataSource(String ds) {
            if (!StringUtils.hasText(ds)) {
                throw new RuntimeException("remove parameter could not be empty");
            }
            if (primary.equals(ds)) {
                throw new RuntimeException("could not remove primary datasource");
            }
            if (dataSourceMap.containsKey(ds)) {
                DataSource dataSource = dataSourceMap.remove(ds);
                try {
                    closeDataSource(dataSource);
                } catch (Exception e) {
                    log.error("dynamic-datasource - remove the database named [{}]  failed", ds, e);
                }
    
                if (ds.contains(UNDERLINE)) {
                    String group = ds.split(UNDERLINE)[0];
                    if (groupDataSources.containsKey(group)) {
                        DataSource oldDataSource = groupDataSources.get(group).removeDatasource(ds);
                        if (oldDataSource == null) {
                            if (log.isWarnEnabled()) {
                                log.warn("fail for remove datasource from group. dataSource: {} ,group: {}", ds, group);
                            }
                        }
                    }
                }
                log.info("dynamic-datasource - remove the database named [{}] success", ds);
            } else {
                log.warn("dynamic-datasource - could not find a database named [{}]", ds);
            }
        }
    
     省略
    }

    可以发现,该项目预留了相关接口给开发者,以方便添加、删除数据库。

    添加数据源的步骤:
    1、注入DynamicRoutingDataSource和DataSourceCreator。
    2、通过数据源配置(url、username、password等)构建一个DataSourceProperty对象。
    3、通过dataSourceCreator,根据配置属性构建一个真实的DataSource。
    4、调用DynamicRoutingDataSource的addDataSource方法添加这个DataSource。
    同理,删除数据源的步骤:
    1、注入DynamicRoutingDataSource。
    2、调用DynamicRoutingDataSource的removeDataSource方法。

        @PostMapping("/add")
        @ApiOperation("通用添加数据源(推荐)")
        public Set<String> add(@Validated @RequestBody DataSourceDTO dto) {
            DataSourceProperty dataSourceProperty = new DataSourceProperty();
            BeanUtils.copyProperties(dto, dataSourceProperty);
            DynamicRoutingDataSource ds = (DynamicRoutingDataSource) dataSource;
            DataSource dataSource = dataSourceCreator.createDataSource(dataSourceProperty);
            ds.addDataSource(dto.getPollName(), dataSource);
            return ds.getCurrentDataSources().keySet();
        }
    
        @DeleteMapping
        @ApiOperation("删除数据源")
        public String remove(String name) {
            DynamicRoutingDataSource ds = (DynamicRoutingDataSource) dataSource;
            ds.removeDataSource(name);
            return "删除成功";
        }

    三、总结

    通过阅读该项目源码,熟悉了spring aop、spring事务管理、spring boot自动配置等spring知识点,可以根据业务需求去进一步扩展这个starter。

    展开全文
  • mybatis报错Error querying database. Cause: com.baomidou.dynamic.datasource.exception.CannotFindDataSourceException: dynamic-datasource can not find primary datasource解决方案

    mybatis报错Error querying database. Cause: com.baomidou.dynamic.datasource.exception.CannotFindDataSourceException: dynamic-datasource can not find primary datasource解决方案

    问题背景

    因为之前mybatis使用了多源数据,引入了dynamic-datasource-spring-boot-starter依赖,导致报错

    Creating a new SqlSession
    SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@72eb6200] was not registered for synchronization because synchronization is not active
    Closing non transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@72eb6200]
    
    org.mybatis.spring.MyBatisSystemException: nested exception is org.apache.ibatis.exceptions.PersistenceException: 
    ### Error querying database.  Cause: com.baomidou.dynamic.datasource.exception.CannotFindDataSourceException: dynamic-datasource can not find primary datasource
    ### The error may exist in file [D:\code\csdn\mybatis-plus\target\classes\mapper\UserMapper.xml]
    ### The error may involve com.yg.mybatisplus.mapper.UserMapper.selectUserById
    ### The error occurred while executing a query
    ### Cause: com.baomidou.dynamic.datasource.exception.CannotFindDataSourceException: dynamic-datasource can not find primary datasource
    
    	at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:96)
    	at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:441)
    	at com.sun.proxy.$Proxy65.selectOne(Unknown Source)
    	at org.mybatis.spring.SqlSessionTemplate.selectOne(SqlSessionTemplate.java:160)
    	at com.baomidou.mybatisplus.core.override.MybatisMapperMethod.execute(MybatisMapperMethod.java:89)
    	at com.baomidou.mybatisplus.core.override.MybatisMapperProxy$PlainMethodInvoker.invoke(MybatisMapperProxy.java:148)
    	at com.baomidou.mybatisplus.core.override.MybatisMapperProxy.invoke(MybatisMapperProxy.java:89)
    	at com.sun.proxy.$Proxy66.selectUserById(Unknown Source)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    	at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:137)
    	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
    	at com.sun.proxy.$Proxy67.selectUserById(Unknown Source)
    	at com.yg.mybatisplus.MybatisPlusTest.testSelectUserById(MybatisPlusTest.java:139)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
    	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
    	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
    	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
    	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
    	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
    	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
    	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
    	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
    	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
    	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
    	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
    	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
    	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
    	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
    	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
    	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
    	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
    	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    	at java.util.ArrayList.forEach(ArrayList.java:1257)
    	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
    	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    	at java.util.ArrayList.forEach(ArrayList.java:1257)
    	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
    	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
    	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
    	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
    	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
    	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
    	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
    	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
    	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
    	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
    	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
    	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
    	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
    	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
    	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
    	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:221)
    	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
    Caused by: org.apache.ibatis.exceptions.PersistenceException: 
    ### Error querying database.  Cause: com.baomidou.dynamic.datasource.exception.CannotFindDataSourceException: dynamic-datasource can not find primary datasource
    ### The error may exist in file [D:\code\csdn\mybatis-plus\target\classes\mapper\UserMapper.xml]
    ### The error may involve com.yg.mybatisplus.mapper.UserMapper.selectUserById
    ### The error occurred while executing a query
    ### Cause: com.baomidou.dynamic.datasource.exception.CannotFindDataSourceException: dynamic-datasource can not find primary datasource
    	at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30)
    	at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:153)
    	at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:145)
    	at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:140)
    	at org.apache.ibatis.session.defaults.DefaultSqlSession.selectOne(DefaultSqlSession.java:76)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:427)
    	... 85 more
    Caused by: com.baomidou.dynamic.datasource.exception.CannotFindDataSourceException: dynamic-datasource can not find primary datasource
    	at com.baomidou.dynamic.datasource.DynamicRoutingDataSource.determinePrimaryDataSource(DynamicRoutingDataSource.java:91)
    	at com.baomidou.dynamic.datasource.DynamicRoutingDataSource.getDataSource(DynamicRoutingDataSource.java:120)
    	at com.baomidou.dynamic.datasource.DynamicRoutingDataSource.determineDataSource(DynamicRoutingDataSource.java:78)
    	at com.baomidou.dynamic.datasource.ds.AbstractRoutingDataSource.getConnection(AbstractRoutingDataSource.java:48)
    	at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:159)
    	at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:117)
    	at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)
    	at org.mybatis.spring.transaction.SpringManagedTransaction.openConnection(SpringManagedTransaction.java:80)
    	at org.mybatis.spring.transaction.SpringManagedTransaction.getConnection(SpringManagedTransaction.java:67)
    	at org.apache.ibatis.executor.BaseExecutor.getConnection(BaseExecutor.java:337)
    	at org.apache.ibatis.executor.SimpleExecutor.prepareStatement(SimpleExecutor.java:86)
    	at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:62)
    	at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:325)
    	at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156)
    	at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109)
    	at com.baomidou.mybatisplus.extension.plugins.MybatisPlusInterceptor.intercept(MybatisPlusInterceptor.java:81)
    	at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:62)
    	at com.sun.proxy.$Proxy100.query(Unknown Source)
    	at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:151)
    	... 93 more
    

    解决方案

    删除掉dynamic-datasource-spring-boot-starter依赖,或者在application中设置为多源数据

    <!--        <dependency>-->
    <!--            <groupId>com.baomidou</groupId>-->
    <!--            <artifactId>dynamic-datasource-spring-boot-starter</artifactId>-->
    <!--            <version>3.5.0</version>-->
    <!--        </dependency>-->
    




    作为程序员第 160 篇文章,每次写一句歌词记录一下,看看人生有几首歌的时间,wahahaha …

    Lyric: 脑袋瓜有一点秀逗

    展开全文
  • 3、springboot2.6+Mybatis注解多数据源使用dynamic-datasource-spring-boot-starter为依赖 说明 前两篇博客介绍了用基本的方式做多数据源,可以应对一般的情况,但是遇到一些复杂的情况就需要扩展下功能了,比如

    多数据源系列

    1、springboot2.6+Mybatis静态多数据源(集成JTA(Atomikos案例)实现分布式事务控制)
    2、springboot2.6+Mybatis动态多数据源AOP切换(AbstractRoutingDataSource)
    3、springboot2.6+Mybatis注解多数据源使用dynamic-datasource-spring-boot-starter为依赖

    说明

    前两篇博客介绍了用基本的方式做多数据源,可以应对一般的情况,但是遇到一些复杂的情况就需要扩展下功能了,比如:动态增减数据源、数据源分组,纯粹多库 读写分离 一主多从、从其他数据库或者配置中心读取数据源等等。其实就算没有这些需求,使用这个实现多数据源也比之前使用AbstractRoutingDataSource要便捷的多。

    dynamic-datasource-spring-boot-starter 是一个基于springboot的快速集成多数据源的启动器。
    github: https://github.com/baomidou/dynamic-datasource-spring-boot-starter
    文档: https://github.com/baomidou/dynamic-datasource-spring-boot-starter/wiki

    动态数据源方案

    文件结构

    在这里插入图片描述

    maven引入:

            <dependency>
                <groupId>com.baomidou</groupId>
                <artifactId>dynamic-datasource-spring-boot-starter</artifactId>
                <version>3.1.0</version>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-aop</artifactId>
                <version>2.3.3.RELEASE</version>
            </dependency>
    

    通过yml配置好数据源

    spring:
      datasource:
        dynamic:
          primary: master1
          strict: false
          datasource:
            master1:
              url: jdbc:mysql://127.0.0.1:3306/master1?useUnicode=true&characterEncoding=UTF-8&useSSL=false&autoReconnect=true&failOverReadOnly=false&serverTimezone=GMT%2B8
              username: root
              password: andrew
              driver-class-name: com.mysql.cj.jdbc.Driver
            master2:
              url: jdbc:mysql://127.0.0.1:3306/master2?useUnicode=true&characterEncoding=UTF-8&useSSL=false&autoReconnect=true&failOverReadOnly=false&serverTimezone=GMT%2B8
              username: root
              password: andrew
              driver-class-name: com.mysql.cj.jdbc.Driver
            master3:
              url: jdbc:oracle:thin:@10.132.212.63:1688:TESTDB
              username: flx
              password: flx202108
              driver-class-name: oracle.jdbc.OracleDriver
    
    logging:
      level:
        com.xkcoding: debug
        com.xkcoding.orm.mybatis.mapper: trace
    
    server:
      port: 8080
    #  servlet:
    #    context-path: /demo
    
    mybatis:
      type-aliases-package: com.orm.mybatis.dsannotation.entity
      mapper-locations: classpath:mapper/*/*.xml
      configuration:
        map-underscore-to-camel-case: true
    
    

    service层

    里面在想要切换数据源的方法上加上@DS注解就行了,也可以加在整个service层上,方法上的注解优先于类上注解

    @Service
    public class UserServiceImpl {
    
        @Resource
        private UserMapper1 userMapper1;
    
        @Resource
        private UserMapper2 userMapper2;
    
        @Resource
        private AsusPoInfoMapper3 asusPoInfoMapper3;
    
        @DS("master1")
        public List<User> findAllUser(){
            List<User> list = userMapper1.selectAllUser();
            return list;
        }
    
        @DS("master2")
        public List<User> findAllUser1(){
            List<User> list = userMapper2.selectAllUser();
            return list;
        }
    
        public User findUserById(Long id){
            return userMapper1.selectUserById(id);
        }
    
        @Transactional  //与dynamic不同的是,这两个注解可以一起使用会先切换数据源再事务
        @DS("master1")  //与dynamic不同的是,这两个注解可以一起使用会先切换数据源再事务
        public void insertUser1(){
            SimpleDateFormat simpleDateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
            String date =  simpleDateFormat.format(new Date());
            String UUID = java.util.UUID.randomUUID().toString().substring(0,5);
            User user = User.builder().email("andrew@qq.com"+UUID).name("andrew"+UUID).password("123456"+UUID).phoneNumber("123"+UUID)
                    .lastUpdateTime(date).createTime(date).status(0).salt("password"+UUID).build();
            userMapper1.saveUser(user);
        }
    
        @Transactional
        @DS("master2")
        public void insertUser2(){
            SimpleDateFormat simpleDateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
            String date =  simpleDateFormat.format(new Date());
            String UUID = java.util.UUID.randomUUID().toString().substring(0,5);
            User user = User.builder().email("andrew@qq.com"+UUID).name("andrew"+UUID).password("123456"+UUID).phoneNumber("123"+UUID)
                    .lastUpdateTime(date).createTime(date).status(0).salt("password"+UUID).build();
            userMapper2.saveUser(user);
        }
    
        public void testTransitional() {
            ((UserServiceImpl)AopContext.currentProxy()).insertUser1();
            ((UserServiceImpl)AopContext.currentProxy()).insertUser2();
            ((UserServiceImpl)AopContext.currentProxy()).insertOracle();
        }
    
        @DS("master3")
        public List<AsusPoInfo> selectOracle(){
           return asusPoInfoMapper3.selectAllAsusPoInfo();
        }
    
        @DS("master3")//与dynamic不同的是,这两个注解可以一起使用会先切换数据源再事务
        @Transactional
        public void insertOracle(){
            AsusPoInfo asusPoInfo = AsusPoInfo.builder().id(java.util.UUID.randomUUID().toString().substring(0,20))
                    .woNo("andrew").po("123456").poLine("poline").cPo("cpo123456").shipType("Direct").build();
            asusPoInfoMapper3.insertAsusPoInfo(asusPoInfo);
        }
    
    }
    
    

    测试多数据源回滚

    package com.orm.mybatis.dsannotation;
    
    import com.orm.mybatis.dsannotation.entity.AsusPoInfo;
    import com.orm.mybatis.dsannotation.entity.User;
    import org.junit.jupiter.api.Test;
    import org.springframework.boot.test.context.SpringBootTest;
    import com.orm.mybatis.dsannotation.serviceImpl.UserServiceImpl;
    
    import javax.annotation.Resource;
    import java.util.List;
    
    @SpringBootTest
    class SpringbootMybatisDsannotationDatasourceApplicationTests {
    
        //事务测试
        @Resource
        private UserServiceImpl userService;
    
        @Test
        void contextLoads1() {
         List<AsusPoInfo> list = userService.selectOracle();
         System.out.println(list);
        }
    
        @Test
        void contextLoads2() {
            List<User> list = userService.findAllUser();
            System.out.println(list);
        }
    
        @Test
        void contextLoads3() {
            List<User> list = userService.findAllUser();
            List<User> list1 = userService.findAllUser1();
            List<AsusPoInfo> list2 = userService.selectOracle();
            list.addAll(list1);
            System.out.println(list);
            System.out.println(list2);
        }
    
        @Test
        void contextLoads4() {
            userService.testTransitional();
        }
    
    }
    
    
    

    切换数据源成功,而且事务能回滚,但如果是多数据源事务,只能回滚报错的数据源的事务。

    方案的权衡

    1. 静态多数据源方案优势在于配置简单并且对业务代码的入侵性极小,缺点也显而易见:我们需要在系统中占用一些资源,而这些资源并不是一直需要,一定程度上会造成资源的浪费。如果你需要在一段业务代码中同时使用多个数据源的数据又要去考虑操作的原子性(事务)可以用spring的jta实现事务,那么这种方案无疑会适合你。
    2. (aop和dynamic)动态数据源(AbstractRoutingDataSource)方案配置上看起来配置会稍微复杂一些,但是很好的符合了“即拿即用,即用即还”的设计原则,我们把多个数据源看成了一个池子,然后进行消费。它的缺点正如上文所暴露的那样:我们往往需要在事务的需求下做出妥协。而且由于需要切换环境上下文,在高并发量的系统上进行资源竞争时容易发生死锁等活跃性问题。我们常用它来进行数据库的“读写分离”,不需要在一段业务中同时操作多个数据源。这种动态形式并不能用spring的jta实现,而且其他实现方式(seata等)虽然可以实现,但配置复杂且实用度不高。
    3. 如果需要使用事务,一定记得使用分布式事务进行Spring自带事务管理的替换,否则将无法进行一致性控制。
    4. 写到这里本文也就结束,好久没有撰写文章很多东西考虑不是很详尽,谢谢批评指正!

    项目地址

    springboot2.6+mybatis
    https://gitee.com/liuweiqiang12/springboot-mybatis-dynamic-datasource

    springboot2.6+mybatis-plus
    https://gitee.com/liuweiqiang12/springboot-mybatis-plus-dynamic-datasource

    展开全文
  • SpringBoot使用dynamic-datasource-spring-boot-starter搭建多数据源
  • 本文主要阐述用dynamic-datasource-spring-boot-starter配置多数据源时,按需动态初始化数据库or按需加载数据源的问题处理。 背景:有个做数据处理的组件,数据来源是某几个数据库,这时候可用dynamic-datasource-...
  • dynamic-datasource + sharding 动态刷新表
  • 3、使用dynamic-datasource-spring-boot-starter做多数据源及源码分析 文章目录多数据源系列简介实操基本使用集成druid连接池源码分析整体结构总结 简介 前两篇博客介绍了用基本的方式做多数据源,可以应对一般的...
  • 前面做了一个基于dynamic-datasource实现读写分离的实现,基于dynamic-datasource实现多数据源的读写分离_邋遢道的博客-CSDN博客 最近公司遇到一个多租户的的需求,刚好就继续使用dynamic-datasource来做多数据源的...
  • 代码分析 研究一个starter的源码,最好的入手点就是MATE-INF/spring.factories。从这里可以看到自动配置类,在通过自动配置类,就...com.baomidou.dynamic.datasource.spring.boot.autoconfigure.DynamicDataSourceAut
  • dynamic-datasource就是解决这个问题的,当然解决多数据源的问题绰绰有余了。 Dynamic-datasource复杂的条件配置 详细参考:https://www.kancloud.cn/tracy5546/dynamic-datasource/2264611 环境搭建 pom.xml依赖 &...
  • dynamic-datasource多数据源快速体验

    千次阅读 2022-03-12 11:21:21
    多数据源依赖dynamic-datasource框架 https://www.kancloud.cn/tracy5546/dynamic-datasource/2264611 环境搭建 1 mvn依赖 <?xml version="1.0" encoding="UTF-8"?> <project xmlns=...
  • dynamic-datasource 源码分析 源码地址 分析: 自动配置 首先,这是一个SpringBoot启动器,所以我们先从spring.factories入手。 发现帮我们自动配置了DynamicDataSourceAutoConfiguration 查看...
  • 所以现在决定引入dynamic-datasource来实现多数据源的切换。 思路:配置一个数据源,为主数据源,使用dynamic-datasource来管理数据源的切换,并且把Shardingsphere管理的数据源加入进去。也就是对分表的SQL使用...
  • 环境 <dependency>...dynamic-datasource-spring-boot-starter</artifactId> <version>3.4.1</version> </dependency> <dependency> <groupId>co
  • dynamic-datasource-spring-boot-starter 是一个基于springboot的快速集成多数据源的启动器。 其支持 Jdk 1.7+, SpringBoot 1.4.x 1.5.x 2.x.x。 Gitee地址: baomidou / dynamic-datasource-spring-boot-starter ...
  • dynamic-datasource-spring-boot-starter 是一个基于springboot的快速集成多数据源的启动器 文档地址: [https://www.kancloud.cn/tracy5546/dynamic-datasource/2264611]: 示例 说明:从两个库读取相同的表,显示两...

空空如也

空空如也

1 2 3 4 5 ... 20
收藏数 16,390
精华内容 6,556
关键字:

dynamic-datasource