The biggest pain point of traffic management - the resource utilization rate cannot go up

Python Wechat ordering applet course video

https://edu.csdn.net/course/detail/36074

Python Actual quantitative transaction financial management system

https://edu.csdn.net/course/detail/35475
Hello everyone , I'm a ferry , This is the third in the traffic management series 9 Original articles , If there is harvest , Please share it with more friends .

I was once asked , Is there any pain in current limiting ？ My answer at the time was ： The current limiting threshold is not easy to evaluate and reduces the user experience , This is what I think is the pain point .

How to evaluate the current limiting threshold is still subject to the action of pressure measurement , Especially the current e-commerce platform , The full link voltage test will be carried out before the promotion , Expose the problem , See how much traffic you can handle . Then according to the business expectation , I know whether to expand the capacity , And the indicators of flow control , It's not hard to say , It needs a variety of means to prove .

Reducing the user experience is a must for current limiting , The user is placing an order , Then you will be prompted that the server is currently too crowded , Network anomalies , Please try again and wait , To put it simply, it reduces the user experience , Because the user needs to operate repeatedly . The serious point is that it affects the conversion rate of orders , Therefore, this is also the reason why pressure measurement is required before the promotion of e-commerce platform . It doesn't mean limiting the flow directly to a very low water level , User experience and system complexity must be weighed .

In fact, there is another more painful point is ： Resource utilization is not going up .

When we go through the pressure test , When evaluating the maximum performance of a single node , The flow control threshold is 70% It's a safer approach . For example, you press a single machine to 1000 QPS, then CPU And memory 50% about , Then direct flow control 1000 that will do .

this 1000 It may also subdivide interfaces , The current limiting values of different interfaces are also different . For example, you have an interface configured 500 Current limiting , As long as the number of requests for this interface exceeds 500 Then the current must be limited , However, it is possible that only this interface has a large number of visits at this time , The others have little traffic , Mechanical CPU, Memory or something is still very low , The response, including the database, is also fast . At this time, it seems that it doesn't matter to limit the current , However, the current limit is configured with a fixed value , Can only limit .

In the face of this situation , A new current limiting method was born , Namely Adaptive current limiting . Adaptive current limiting means that there is no fixed current limiting threshold , The current limiting threshold will change , The factor of change is whether the dependent resources are sufficient , such as CPU, Memory and other resources .

Adaptive current limiting is still complex , The complexity lies in the need to collect all kinds of data in real time , Then through a lot of calculations , Then extract the decision , This decision is whether to limit the current , And the calculation speed is fast .

The general structure is as follows ：

Of course, this calculation is a separate service , Embedded or embedded can be used in the current application , Depends on performance impact and whether cluster flow control .

With adaptive current limiting , The utilization of resources will be greatly improved . Because there is no need to set a more stable fixed value , Instead, it determines whether to limit the flow according to the use of resources , Maximum program utilization resources .

In the open source framework Sentinel It also has such an adaptive current limiting function , Those who are interested can learn ：https://github.com/alibaba/Sentinel/wiki/%E7%B3%BB%E7%BB%9F%E8%87%AA%E9%80%82%E5%BA%94%E9%99%90%E6%B5%81

One of the advantages of adaptive current limiting is that it can improve the utilization of resources , There is also the ability to adapt to the impact of uncertain factors . Compared with the static current limiting configuration , We must configure it based on the results of pressure measurement , Even after pressure measurement , It's time for a really big traffic attack , No guarantee as like as two peas. . In case of an emergency , One slow Sql Put the database CPU It's full , Your current limit threshold is still so much QPS, slow SQL Increase hundreds of times , This current limiting is now inoperative .

Through adaptive current limiting , Decide whether to use flow control according to the results of real-time calculation , Ensure system stability . At the same time, it also avoids the cumbersome manual configuration process , The system automatically adjusts .

Another example ： The current limit of single interface is 1000, Order details are 500. If the current is limited according to a fixed value , More than 1000 Will be limited , If the requested quantity of the order is greater than the quantity of the order details , Can adaptive current limiting first meet the traffic with higher business value , Let this part of the flow pass through , If there is other flow at this time, can it be automatically limited first . I don't need to say more about the value of this part , You know .

So adaptive current limiting can do a lot of things , Mainly intelligent , Intelligent algorithm level , It means a little artificial intelligence .