[Skill] Using Sentinel to achieve priority processing of requests

“Because business requests take up all of themServletThe container worker thread cannot process the health check interface in time”的问题.

1. 背景

The one I recently took over"微服务架构"系统中,In its design idea, the proxy for third-party services is made into a micro-service registration mode——Register each third-party service accordinglyNacos/ConsulSuch a service discovery component,After that, access to these third-party services is unified through the gateway for transition.

Guess at the present moment,The purpose of this design was probably to reuse the existing microservice processing flow,Avoid adding new processes to increase maintenance costs.

But in routine maintenance,The author found that this process has the following problems：

Some third-party proxy services,Or some of the services in our own microservice components,Its normal processing time is relatively long.If there are too many such requests in a short period of time,directly exceeds the presetServletThe number of worker threads for the container（例如undertow默认的64,tomcat默认的200）,It will cause the application service to respond slowly（This includes the externally provided health check interface）,进而导致ConsulKick the app service out of the health service because it no longer exists.

This article attempts to useSentinelalleviate such problems,Offers two smooth solutions with minimal modifications.

2. 思路

1. 思路一： A separate port is created to be responsible for the response of the health check interface.
   a. 针对这个思路,SentinelIn fact, the implementation basis has been provided.SentinelThe default listening port is created after startup8719的ServerSocket,Dispatch the request to the corresponding oneCommandHandler<R>实现类.See source code entrySimpleHttpCommandCenter.start() .
   b. 在SimpleHttpCommandCenter.start()中的ServerSocket实现,Take a dedicated self-built thread pool（对应字段executor和bizExecutor）,因此不会和ServletContainer worker threads conflict,自然也不会发生“Because business requests take up all of themServletThe container worker thread cannot process the health check interface in time”的问题发生.
   c. rather than confronting the problem,This idea is more of a way to bypass the problem.

2. 思路二：Take all business requests as a whole"并发线程数"Class throttling,制造出“A thread is reserved for processing the health check interface”的效果.
   a. Compared to the idea above,This way of thinking confronts the problem,直接借助Sentinelcurrent limiting characteristics,Reserve worker threads for interfaces such as health checks,Ensure timely response.
   b. By limiting the number of threads processing business requests to ServletBelow the number of container worker threads（例如 The maximum number of worker threads is reduced by one.By default this is forundertow为63,对于tomcat为199）,Make sure there is always a thread in the ready state,to respond to the request of the special interface in time.

3. 实现

For the above two ideas,The corresponding implementation codes are provided below.

3.1 思路一：实现CommandHandler<R>

Just like the idea above,SentinelBy default it will listen for extras8719端口,Respond to a specific command.

To reuse this feature,我们需要实现自己的CommandHandler<R> ,并按照Sentinel提供的SPIThe extension method is registered in the processing flow.

1. 实现自己的CommandHandler<R> .

@CommandMapping(name = "health", desc = "health check")
public class HealthCheckCommandHandler implements CommandHandler<Object> {
    

	@Override
	public CommandResponse<Object> handle(CommandRequest request) {
    
		final Map<String, String> statusInfo = Collections.singletonMap("status", "UP");
		return CommandResponse.ofSuccess(JSONUtil.toJsonPrettyStr(statusInfo));
	}
}

2. SPI注册.
   新建文件META-INF/services/com.alibaba.csp.sentinel.command.CommandHandler,Fill in the aboveHealthCheckCommandHandler类完整名称.

3. 启动应用.访问 localhost:8719/health .

注意：

1. sentinel-dashboard After the standalone app starts,It is also occupied by default8719端口,而在SimpleHttpCommandCenter.getServerSocketFromBasePort(int basePort) 实现中,sentinel会自8719The port is the initial value,递增循环,Find the first unused port that is not used as a port for external services. 例如上面提到的8719端口被占用的话,will be used progressively8720端口.
2. 也可以通过配置 spring.cloud.sentinel.transport.port to force the port to be specified.
3. 可以通过访问 localhost:8719/api 来获取sentinel对外提供的CommandHandler<R>实现.
4. SentinelThe above functions are not directly integrated insentinel-core中,but as separate components.相关GAV如下：

	<dependency>
       	<groupId>com.alibaba.csp</groupId>
       	<artifactId>sentinel-transport-common</artifactId>	
       	<version>1.8.0</version>
       	<exclusions>
       		<exclusion>
       			<groupId>com.alibaba</groupId>
       			<artifactId>fastjson</artifactId>
       		</exclusion>
       	</exclusions>
	</dependency>

3.2 思路二：Restricted business requests"并发线程数"

默认情况下,Sentinel适配spring-mvc是使用SentinelWebInterceptorto intervene in the request processing response.

SentinelWebInterceptor实现了SpringMVCThe classic interception interface in HandlerInterceptor,通过实现其preHandleinterface to perform current limiting logic judgment.

Sentinel默认提供了两种HandlerInterceptor实现类：

名称	特点	举例
SentinelWebInterceptor	将请求的urlAddresses are used as statistical units respectively,On this basis, the judgment of current limit is made	/hello 和 /hello2 will be treated as two different resources,The current limiting configuration is performed separately./hello触发限流,will not affect for/hello2的访问.
SentinelWebTotalInterceptor	All business requests to the systemurladdress as the same,On this basis, the judgment of current limit is made.	/hello 和 /hello2 It will be treated as the same resource for current limit judgment.例如设置QPS为20,Then continuous access within one second/hello 20次后,再访问/hello2,The current limit will still be triggered.

按照我们的需求,可以直接复用Sentinel默认提供的SentinelWebTotalInterceptor来实现.

1. 禁用默认的SentinelWebInterceptor .
   Sentinel是在SentinelWebAutoConfiguration类中将SentinelWebInterceptor注册到Spring容器中.而且提供了spring.cloud.sentinel.filter.enabledImplement disabling it.

2. 启用SentinelWebTotalInterceptor.

   /** * <p> Refer To {@code SentinelWebAutoConfiguration} * <p> 配置 spring.cloud.sentinel.filter.enabled 为 FALSE * @author fulizhe * */
   @Configuration(proxyBeanMethods = false)
   public class SentinelWebInterceptorConfiguration implements WebMvcConfigurer {
         
   
   	@Autowired
   	private SentinelProperties properties;
   
   	@Autowired
   	private Optional<UrlCleaner> urlCleanerOptional;
   
   	@Autowired
   	private Optional<BlockExceptionHandler> blockExceptionHandlerOptional;
   
   	@Autowired
   	private Optional<RequestOriginParser> requestOriginParserOptional;
   
   	@Autowired
   	private Optional<SentinelWebMvcConfig> sentinelWebMvcConfig;
   
   	@Override
   	public void addInterceptors(InterceptorRegistry registry) {
         
   		// ！！！注意： 注册的这个Interceptor, 不参与 /actuator/xx Access blocking(That is not to us/actuator/health The health check interface is used for current limiting)
   
   		SentinelWebMvcTotalConfig sentinelWebMvcTotalConfig = new SentinelWebMvcTotalConfig();
   		BeanUtil.copyProperties(sentinelWebMvcConfig.get(), sentinelWebMvcTotalConfig);
   		// 使用SentinelWebTotalInterceptor 代替默认的SentinelWebInterceptor
   		AbstractSentinelInterceptor sentinelWebTotalInterceptor = new SentinelWebTotalInterceptor(
   				sentinelWebMvcTotalConfig);
   		SentinelProperties.Filter filterConfig = properties.getFilter();
   		registry.addInterceptor(sentinelWebTotalInterceptor)//
   				.order(filterConfig.getOrder())
   				.addPathPatterns(filterConfig.getUrlPatterns());
   		log.info("[Sentinel Starter] register SentinelWebInterceptorEx with urlPatterns: {}.",
   				filterConfig.getUrlPatterns());
   	}
   
   	/** * COPY FROM {@code SentinelWebAutoConfiguration} * @return */
   	@Bean
   	public SentinelWebMvcConfig sentinelWebMvcConfigX() {
         
   		SentinelWebMvcConfig sentinelWebMvcConfig = new SentinelWebMvcConfig();
   		sentinelWebMvcConfig.setHttpMethodSpecify(properties.getHttpMethodSpecify());
   		sentinelWebMvcConfig.setWebContextUnify(properties.getWebContextUnify());
   
   		if (blockExceptionHandlerOptional.isPresent()) {
         
   			blockExceptionHandlerOptional.ifPresent(sentinelWebMvcConfig::setBlockExceptionHandler);
   		} else {
         
   			if (StringUtils.hasText(properties.getBlockPage())) {
         
   				sentinelWebMvcConfig.setBlockExceptionHandler(
   						((request, response, e) -> response.sendRedirect(properties.getBlockPage())));
   			} else {
         
   				sentinelWebMvcConfig.setBlockExceptionHandler(new DefaultBlockExceptionHandler());
   			}
   		}
   
   		urlCleanerOptional.ifPresent(sentinelWebMvcConfig::setUrlCleaner);
   		requestOriginParserOptional.ifPresent(sentinelWebMvcConfig::setOriginParser);
   		return sentinelWebMvcConfig;
   	}
   }
   

3. Set the number of concurrent threads to limit the current flow.
   Because our current limiting configuration is simple,So it is not intended to be introduced heresentinel-dashboard,Instead, use manual registration directly.

   a. Read the local throttling configuration file,注册到Sentinel中.

   // 实现Sentinel提供的扩展接口InitFunc ; 
   // 参见官方文档: https://github.com/alibaba/Sentinel/tree/master/sentinel-demo/sentinel-demo-dynamic-file-rule
   public class RegisterFlowRuleInit implements InitFunc {
         
   
   	@Override
   	public void init() throws Exception {
         
   		registerFlowRule();
   	}
   
   	static void registerFlowRule() throws Exception {
         
   		Converter<String, List<FlowRule>> flowRuleListParser = source -> JSON.parseObject(source,
   				new TypeReference<List<FlowRule>>() {
         
   				});
   		// Read the local throttling configuration file,注册到Sentinel中.
   		ClassLoader classLoader = SpringSentinelApplication.class.getClassLoader();
   		String flowRulePath = URLDecoder.decode(classLoader.getResource("FlowRule.json").getFile(), "UTF-8");
   
   		// Data source for FlowRule
   		FileRefreshableDataSource<List<FlowRule>> flowRuleDataSource = new FileRefreshableDataSource<>(flowRulePath,
   				flowRuleListParser);
   		FlowRuleManager.register2Property(flowRuleDataSource.getProperty());
   
   	}
   }
   

   b. 对上面的RegisterFlowRuleInit进行SPI注册.
   c. "FlowRule.json"文件内容样例：（因为笔者使用tomcat作为servlet容器,So the thread count limit here is set to199.That is, a thread is reserved to deal with the health check）

   [
   	{
         
   	    "resource": "spring-mvc-total-url-request",
   	    "limitApp": "default",
   	    "grade": 0,
   	    "count": 199,
   	    "strategy": 0,
   	    "refResource": null,
   	    "controlBehavior": 0,
   	    "warmUpPeriodSec": 10,
   	    "maxQueueingTimeMs": 500,
   	    "clusterMode": false,
   	    "clusterConfig": {
         
   	        "flowId": null,
   	        "thresholdType": 0,
   	        "fallbackToLocalWhenFail": true,
   	        "strategy": 0,
   	        "sampleCount": 10,
   	        "windowIntervalMs": 1000
   	    }
   	}  
   ]
   

4. 注意.
   This idea is based on the following two foundations：
   a. sentinel中的SentinelWebTotalInterceptor实现,是将所有的**业务请求(Including proxy forwarding)**current limiting as a whole.
   b. SpringBoot提供的/actuator/xx类接口,does not belong to the above article"业务请求",So it is not affected by the above current limit settings.Therefore, as long as the idle ones are reserved in the above current limiting configurationServletContainer worker thread,那么/actuator/healthSuch a health check interface can get a timely response.（The specific principle is simple,负责/actuator/xxThe class interface deals with thatWebMvcEndpointHandlerMapping类型,Responsible for the business request interface[@RequestMapping定义]处理的是RequestMappingHandlerMapping类型,The current limit we added in the second ideaInterceptoronly affects the latter）

4. 补充

1. The above two methods are actually usedSentinel提供的扩展InitFunc,而默认情况下SentinelIt requires an access to the server to activate the pairInitFunc的回调;即对于InitFunc,SentinelThe lazy loading strategy is adopted.We need to use configurationspring.cloud.sentinel.eager=trueto modify this logic.

2. The advantages and disadvantages of the above two methods

   category of ideas	优点	缺点
   实现CommandHandler<R>	简单粗暴	Because of the change of health check address,Therefore, it is necessary to modify the service registration implementation logic;For already registered toConsulThe services should be adjusted accordingly
   Restricted business requests"并发线程数"	The health check address remains unchanged,Consul端无感知	It is difficult to understand

5. 参考

1. 官方文档 - 网关限流
2. 官方文档 - 在生产环境中使用-Sentinel
3. 官方文档 - 介绍