当前位置:网站首页>The dynamic thread pool in Kitty supports Nacos and Apollo multi configuration centers
The dynamic thread pool in Kitty supports Nacos and Apollo multi configuration centers
2020-11-06 01:32:00 【Yin Jihuan】
Catalog
- Looking back on yesterday
- nacos Integrate
- Spring Cloud Alibaba The way
- Nacos Spring Boot The way
- Apollo Integrate
- Self research configuration center docking
- No configuration center docking
- Implement source code analysis
- compatible Apollo and Nacos NoClassDefFoundError
- Apollo Auto refresh problem
Looking back on yesterday
Last article 《 I'm itching for a while , A dynamic thread pool , Source code put Github 了 》(https://mp.weixin.qq.com/s/JM9idgFPZGkRAdCpw0NaKw) After it was sent out, many readers asked me privately if this could be used in my work , You can use it , In itself, it's an extension of the thread pool , And then the configuration center and monitoring are connected .
At present, there are mainly the following problems :
- It hasn't been released to Maven The central warehouse ( I'll do it later ), You can compile, package and publish it to a private repository ( A temporary plan )
- Coupled Nacos, If you don't use it in your project Nacos Or how to use other configuration centers ?( The content of this article is )
- Only business thread pools can be replaced , For example, the thread pool in some frameworks cannot be replaced ( In the process of conception )
The focus of this article is to introduce how to connect Nacos and Apollo, Because it supported from the beginning Nacos, But the way to support it is to rely on Spring Cloud Alibaba , If it doesn't work Spring Cloud Alibaba How to support , It also needs to be expanded .
Nacos Integrate
Nacos There are two ways to integrate , One is that your project uses Spring Cloud Alibaba , The other is to use only Spring Boot The integration of methods .
Spring Cloud Alibaba The way
Join the rely on :
<dependency>
<groupId>com.cxytiandi</groupId>
<artifactId>kitty-spring-cloud-starter-dynamic-thread-pool</artifactId>
</dependency>
And then in Nacos Add thread pool configuration in , such as :
kitty.threadpools.executors[0].threadPoolName=TestThreadPoolExecutor
kitty.threadpools.executors[0].corePoolSize=4
kitty.threadpools.executors[0].maximumPoolSize=4
kitty.threadpools.executors[0].queueCapacity=5
kitty.threadpools.executors[0].queueCapacityThreshold=22
And then in the project bootstrap.properties Configuration to use in Nacos data-id.
spring.cloud.nacos.config.ext-config[0].data-id=kitty-cloud-thread-pool.properties
spring.cloud.nacos.config.ext-config[0].group=BIZ_GROUP
spring.cloud.nacos.config.ext-config[0].refresh=true
Nacos Spring Boot The way
If your project only uses Nacos Of Spring Boot Starter, For example, below :
<dependency>
<groupId>com.alibaba.boot</groupId>
<artifactId>nacos-config-spring-boot-starter</artifactId>
</dependency>
So the steps of integration follow Spring Cloud Alibaba The same way , The only difference is how the configuration is loaded . Use @NacosPropertySource Loading .
@NacosPropertySource(dataId = NacosConstant.HREAD_POOL, groupId = NacosConstant.BIZ_GROUP, autoRefreshed = true, type = ConfigType.PROPERTIES)
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
Then it needs to be in bootstrap.properties Closed in Spring Cloud Alibaba Nacos Config Automatic configuration of .
spring.cloud.nacos.config.enabled=false
Apollo Integrate
Apollo We all use it client, Depends on the following :
<dependency>
<groupId>com.ctrip.framework.apollo</groupId>
<artifactId>apollo-client</artifactId>
<version>1.4.0</version>
</dependency>
Integrate Thread-Pool It's the same old step , To add Maven rely on :
<dependency>
<groupId>com.cxytiandi</groupId>
<artifactId>kitty-spring-cloud-starter-dynamic-thread-pool</artifactId>
</dependency>
Then configure the thread pool configuration namespace:
apollo.bootstrap.namespaces=thread-pool-config
Properties No suffix , If it is yaml The file needs to be suffixed :
apollo.bootstrap.namespaces=thread-pool-config.yaml
If you use more than one in your project namespace Words , Need thread pool namespace It is specified in , It is mainly used to modify the monitoring configuration .
kitty.threadpools.apolloNamespace=thread-pool-config.yaml
Self research configuration center docking
If your project uses a self-developed configuration center, how to use dynamic thread pool ?
The best way is to follow Nacos equally , Will configure with Spring To integrate , Encapsulated into PropertySource.
Apollo In the integration Spring Code reference :https://github.com/ctripcorp/apollo/blob/master/apollo-client/src/main/java/com/ctrip/framework/apollo/spring/config/PropertySourcesProcessor.java
Because the configuration class uses @ConfigurationProperties, This is equivalent to seamless integration .
If not with Spring To integrate , There is a way , You can get your configuration after the project starts , And then modify
DynamicThreadPoolProperties Configuration class , Reinitialize the thread pool , The specific steps are consistent with the following docking without configuration center .DynamicThreadPoolManager Provides createThreadPoolExecutor() To create a thread pool .
No configuration center docking
What if you don't use configuration center in your project ? You can still use dynamic thread pools as well .
Put the configuration information of thread pool directly in the project's application Configuration file , But the disadvantage is that the configuration information can't be modified dynamically .
If you want to have the ability to dynamically modify the configuration , You can expand it a little bit , Here's my idea .
Write a Rest API, Parameters are the contents of the entire thread pool configuration , It can be Properties Documents can also be Yaml File format .
This API The logic of "is what we put into it DynamicThreadPoolProperties, call refresh() Refresh Properties file , call refreshYaml() Refresh Yaml file .
Then inject DynamicThreadPoolManager, call refreshThreadPoolExecutor() Refresh thread pool parameters .
Implement source code analysis
First , What we need to achieve is to adapt at the same time Nacos and Apollo Two mainstream configuration centers , There are generally two ways .
The first one is : Will follow Nacos and Apollo The relevant code is separated into a module , Users introduce on demand .
The second kind : It's still a project , Internal compatibility .
I take the second one , Because there's not a lot of code , There's no need to split it into two .
Need to be in pom Add the dependency of two configuration centers at the same time , It needs to be set to optional (optional=true).
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-alibaba-nacos-config</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>com.ctrip.framework.apollo</groupId>
<artifactId>apollo-client</artifactId>
<version>1.4.0</version>
<optional>true</optional>
</dependency>
Then the logic of dynamically adjusting thread pool parameters by monitoring configuration is separated internally ,ApolloConfigUpdateListener and NacosConfigUpdateListener.
In automatic assembly Bean When you need to assemble the corresponding Listener.
@ImportAutoConfiguration(DynamicThreadPoolProperties.class)
@Configuration
public class DynamicThreadPoolAutoConfiguration {
@Bean
@ConditionalOnClass(value = com.alibaba.nacos.api.config.ConfigService.class)
public NacosConfigUpdateListener nacosConfigUpdateListener() {
return new NacosConfigUpdateListener();
}
@Bean
@ConditionalOnClass(value = com.ctrip.framework.apollo.ConfigService.class)
public ApolloConfigUpdateListener apolloConfigUpdateListener() {
return new ApolloConfigUpdateListener();
}
}
compatible Apollo and Nacos NoClassDefFoundError
adopt @ConditionalOnClass To determine which configuration center is used in the current project , And then assemble the corresponding Listener. The code above looks ok , In the actual use of the process to report the following error :
Caused by: java.lang.NoClassDefFoundError: Lcom/alibaba/nacos/api/config/ConfigService;
at java.lang.Class.getDeclaredFields0(Native Method) ~[na:1.8.0_40]
at java.lang.Class.privateGetDeclaredFields(Class.java:2583) ~[na:1.8.0_40]
at java.lang.Class.getDeclaredFields(Class.java:1916) ~[na:1.8.0_40]
at org.springframework.util.ReflectionUtils.getDeclaredFields(ReflectionUtils.java:755) ~[spring-core-5.1.8.RELEASE.jar:5.1.8.RELEASE]
... 22 common frames omitted
Caused by: java.lang.ClassNotFoundException: com.alibaba.nacos.api.config.ConfigService
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_40]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_40]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[na:1.8.0_40]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_40]
... 26 common frames omitted
For example, my project uses Apollo, Then I integrated the dynamic thread pool , The error was reported at boot time , The reason for the mistake is that you can't find Nacos Related to the class .
But I've already used it @ConditionalOnClass To judge , This is because of you DynamicThreadPoolAutoConfiguration Class is in effect ,Spring It's going to load DynamicThreadPoolAutoConfiguration class ,DynamicThreadPoolAutoConfiguration There is NacosConfigUpdateListener Instantiation of , And there is no dependency in the project Nacos, So I made a mistake .
In this case, we need to split the logic of the assembly in more detail , Use a separate class to configure , take @ConditionalOnClass Put it on the class .
Here I take the static inner class approach , If there are no dependencies in the project Nacos, that NacosConfiguration It won't work , It doesn't initialize NacosConfigUpdateListener.
@Configuration
@ConditionalOnClass(value = com.alibaba.nacos.api.config.ConfigService.class)
protected static class NacosConfiguration {
@Bean
public NacosConfigUpdateListener nacosConfigUpdateListener() {
return new NacosConfigUpdateListener();
}
}
@Configuration
@ConditionalOnClass(value = com.ctrip.framework.apollo.ConfigService.class)
protected static class ApolloConfiguration {
@Bean
public ApolloConfigUpdateListener apolloConfigUpdateListener() {
return new ApolloConfigUpdateListener();
}
}
I'd like to mention a point by the way , That's why we usually have to look at the source code of the open source framework . Because it's common for logic to fit multiple frameworks like this , So there must be similar logic in some open source frameworks . If you've seen how other frameworks are implemented before , So here you're going to take that approach directly .
such as Spring Cloud OpenFeign Chinese vs Http The client has made the adaptation of multiple frameworks , You can use it. HttpClient It can also be used. Okhttp, This is the same logic as ours .
Let's take a look at the source code , Here's the picture :
Apollo Auto refresh problem
In the process of implementation, I also encountered a problem and shared it with you , Namely Apollo in @ConfigurationProperties Configuration class , It will not refresh automatically after the configuration information is changed , Need to cooperate with RefreshScope perhaps EnvironmentChangeEvent To achieve .
The picture below is Apollo The original words of the document :
Nacos It's OK to refresh , It's just that when you receive a configuration change message , The configuration information has not been refreshed to Bean Go inside , So when refreshing, a single thread is started to do it , And then I sleep in this thread 1 Second ( It can be adjusted by configuration ).
If according to Apollo The way given in the document , It must be possible . But not so good , Because we need to rely on Spring Cloud Context. The main consideration is that users may not necessarily use Spring Cloud, Our foundation is Spring Boot.
In case the user is in Spring Boot Used in the project Apollo, And then I used my dynamic thread pool , What's going on ?
Finally, I used a manual refresh , When the configuration changes , I will pass Apollo The client of , Pull the contents of the entire configuration file again , Then manually refresh the configuration class .
config.addChangeListener(changeEvent -> {
ConfigFileFormat configFileFormat = ConfigFileFormat.Properties;
String getConfigNamespace = finalApolloNamespace;
if (finalApolloNamespace.contains(ConfigFileFormat.YAML.getValue())) {
configFileFormat = ConfigFileFormat.YAML;
// Remove .yaml suffix ,getConfigFile It will be automatically appended according to the type
getConfigNamespace = getConfigNamespace.replaceAll("." + ConfigFileFormat.YAML.getValue(), "");
}
ConfigFile configFile = ConfigService.getConfigFile(getConfigNamespace, configFileFormat);
String content = configFile.getContent();
if (finalApolloNamespace.contains(ConfigFileFormat.YAML.getValue())) {
poolProperties.refreshYaml(content);
} else {
poolProperties.refresh(content);
}
dynamicThreadPoolManager.refreshThreadPoolExecutor(false);
log.info(" Thread pool configuration changed , Refresh complete ");
});
Refresh logic :
public void refresh(String content) {
Properties properties = new Properties();
try {
properties.load(new ByteArrayInputStream(content.getBytes()));
} catch (IOException e) {
log.error(" transformation Properties abnormal ", e);
}
doRefresh(properties);
}
public void refreshYaml(String content) {
YamlPropertiesFactoryBean bean = new YamlPropertiesFactoryBean();
bean.setResources(new ByteArrayResource(content.getBytes()));
Properties properties = bean.getObject();
doRefresh(properties);
}
private void doRefresh(Properties properties) {
Map<String, String> dataMap = new HashMap<String, String>((Map) properties);
ConfigurationPropertySource sources = new MapConfigurationPropertySource(dataMap);
Binder binder = new Binder(sources);
binder.bind("kitty.threadpools", Bindable.ofInstance(this)).get();
}
Currently only supported Properties and Yaml File configuration format .
Interested in Star Let's go :https://github.com/yinjihuan/kitty
About author *: Yin Jihuan , Simple technology enthusiasts ,《Spring Cloud Microservices - Full stack technology and case analysis 》, 《Spring Cloud Microservices introduction Actual combat and advanced 》 author , official account Ape world Originator . Personal wechat jihuan900 , Welcome to hook up with .
I have compiled a complete set of learning materials , Those who are interested can search through wechat 「 Ape world 」, Reply key 「 Learning materials 」 Get what I've sorted out Spring Cloud,Spring Cloud Alibaba,Sharding-JDBC Sub database and sub table , Task scheduling framework XXL-JOB,MongoDB, Reptiles and other related information .
版权声明
本文为[Yin Jihuan]所创,转载请带上原文链接,感谢
边栏推荐
- Windows 10 tensorflow (2) regression analysis of principles, deep learning framework (gradient descent method to solve regression parameters)
- 这个项目可以让你在几分钟快速了解某个编程语言
- Flink的DataSource三部曲之一:直接API
- Building and visualizing decision tree with Python
- How to use parameters in ES6
- 一篇文章带你了解CSS3圆角知识
- Arrangement of basic knowledge points
- Named entity recognition in natural language processing: tanford core LP ner (1)
- MeterSphere开发者手册
- 至联云解析:IPFS/Filecoin挖矿为什么这么难?
猜你喜欢
零基础打造一款属于自己的网页搜索引擎
Face to face Manual Chapter 16: explanation and implementation of fair lock of code peasant association lock and reentrantlock
01. SSH Remote terminal and websocket of go language
Free patent download tutorial (HowNet, Espacenet)
Custom function form of pychar shortcut key
How to customize sorting for pandas dataframe
The difference between gbdt and XGB, and the mathematical derivation of gradient descent method and Newton method
TensorFlow中的Tensor是什么?
git rebase的時候捅婁子了,怎麼辦?線上等……
Linked blocking Queue Analysis of blocking queue
随机推荐
I've been rejected by the product manager. Why don't you know
一篇文章带你了解CSS3图片边框
Lane change detection
Installing ns-3 on ubuntu18.04
ES6学习笔记(二):教你玩转类的继承和类的对象
零基础打造一款属于自己的网页搜索引擎
零基础打造一款属于自己的网页搜索引擎
Skywalking series blog 1 - install stand-alone skywalking
Summary of common string algorithms
Music generation through deep neural network
[JMeter] two ways to realize interface Association: regular representation extractor and JSON extractor
一篇文章带你了解CSS 渐变知识
Advanced Vue component pattern (3)
Analysis of etcd core mechanism
JVM memory area and garbage collection
6.1.1 handlermapping mapping processor (1) (in-depth analysis of SSM and project practice)
零基础打造一款属于自己的网页搜索引擎
ES6学习笔记(五):轻松了解ES6的内置扩展对象
Named entity recognition in natural language processing: tanford core LP ner (1)
H5 makes its own video player (JS Part 2)