Current limiting and fusing of gateway gateway in Spirng cloud

shunt : Originally, only one server was placed in the database , No matter how many can access this server , Queue up if you can't access （ Delay ）（ If the concurrency is high at the same time, limit the current ）

Current limiting : Limit the number of visitors at the same time

Current limiting algorithm

     Leaky bucket algorithm ： Put the request in a container , Control the speed of processing

     Token algorithm ： Assign a token to each request , No access without a token ,1/QPS （ The number of accesses to the interface at the same time is generally one tomcat yes 200~250）

0. Premise

   1. redis You can use

   2. JMeter

1. spring cloud gateway Gateway current limiting

    1. Introduction to current limiting

         Limiting flow is limiting flow , Because the server can handle a limited number of requests , If the number of requests is very large , We need to limit the flow （ Or let the request wait , Or throw the request away ）,

         Current limiting can protect our API Service availability to all users , It can also prevent network attacks . In high concurrency applications , Current limiting is a hot topic .

         notes 1： Why limit the flow ？ see ：images/00

     

    2. Common current limiting means

         In general, the common current limitation in developing high concurrency systems is ： Limit the total number of concurrent （ For example, database connection pool 、 Thread pool ）、 Limit the number of instantaneous concurrencies （ Such as nginx Of limit_conn modular , Used to limit the number of instantaneous concurrent connections ）、

         Limit the average rate in the time window （ Such as Guava Of RateLimiter、nginx Of limit_req modular , Limit the average rate per second ）; Others include limiting the call rate of remote interfaces 、 Limit MQ ( Message queue ) The rate of consumption .

         In addition, it can also be based on the number of network connections 、 The network traffic 、CPU Or memory load to limit current .

         This paper discusses in gateway Implementation of integration

    3. Current limiting algorithm

        1. Leaky bucket algorithm (Leaky Bucket)

            A little ...

        2. Token bucket algorithm (Token Bucket)

            Over time , The system will press constant 1/QPS The time interval （ If QPS=100, The interval is 10ms） Add... To the barrel Token（ Imagination is the opposite of leakage , There's a tap that keeps adding water ）,

            If the bucket is full, it will not be added . When a new request comes , Take one of them Token, without Token Can take it to block or refuse service .

            Another advantage of token bucket is that it can change speed conveniently ： Once you need to speed up , Increase the rate of tokens placed in the bucket as needed

            notes 1： See... For the flow chart of token bucket algorithm ：images/01

          1 Second generation 20 token

           Token bucket capacity 100

    4. gateway Fast implementation of gateway current limiting

        Spring Cloud Gateway The official provided RequestRateLimiterGatewayFilterFactory class , Use redis and lua Script to implement token bucket .

         We can also be based on Google Guava Medium RateLimiter、Bucket4j、RateLimitJ To achieve . however , This article will be implemented in an official way .

        Gateway Through the built-in RequestRateLimiter The filter realizes current limiting , Using the token bucket algorithm , With the help of Redis Save intermediate data . Users can customize KeyResolver Set the current limiting dimension , for example ：

        1. For the purpose of the request URL Carry out current limiting

        2. To the source IP Carry out current limiting

        3. Current limiting for specific users

         This case realizes to IP Carry out current limiting

        1. Import redis rely on

            <!--redis rely on -->

            <dependency>

                <groupId>org.springframework.boot</groupId>

                <artifactId>spring-boot-starter-data-redis-reactive</artifactId>

            </dependency>

        2. modify application.yml File to add redis Related configuration

            # This is a stand-alone configuration , In development , It should be a redis colony .redis For cluster configuration, see ： Information /07 springboot2.x Integrate redis colony

            spring:

               redis:

                   host: 192.168.217.132

                   port: 6379

                   password: 123456

                   database: 0

             notes 1： If redis The connection fails , The current limiting function will not be enabled . Because no redis As a container to hold tokens , The current limiting function naturally fails . Bear in mind ~~~ Bear in mind ~~~ Bear in mind ~~~

             notes 2： Can be redis Save your configuration information to nacos in , By adding nacos Configure the central client to read

        3. Create current limit Bean   

             For details, see ：src/RequestRateLimiterConfig.java

             notes 1： Current limiting bean Name , Must and step 4 Same name as when quoted

        4. Add the current limiting configuration in the routing configuration

            #filter The name must be RequestRateLimiter

            - name: RequestRateLimiter

              args:

                # Of the parser for current limiting keys Bean The name of the object , Use SpEL The expression is based on #{@beanName} obtain Bean object

                key-resolver: '#{@ipAddrKeyResolver}'

                # Token bucket fill rate , How many requests per second are allowed to be processed by the user

                redis-rate-limiter.replenishRate: 10

                # Total token bucket capacity , The maximum number of requests allowed to complete in one second

                redis-rate-limiter.burstCapacity: 20

      

        5. Use JMeter Conduct current limiting test

             test result , A request without a token is returned 429, The current limit here is equivalent to the average request:10/s

  

        6. On the front page , such as ：vue Handle 429 error , Show “ The service is busy. Please try again later ”

            Response code:429

            Response message:Too Many Requests

2. Fuse

     The gateway is the gateway to all requests , If some back-end services are severely delayed , It may cause a large number of requests to accumulate on the gateway , Bring down the gateway and then the whole system .

     This requires time-out and fast failure processing for services with slow response , The fuse

    

     There are two types of components ：Hystrix And Sentinel, This chapter introduces Spring Cloud Gateway be based on Hystrix Realized fusing

    1. modify gateway-server Sub module of pom, increase pom Related dependencies

        <!--Gateway be based on Hystrix Realized fusing -->

        <dependency>

            <groupId>org.springframework.cloud</groupId>

            <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>

        </dependency>

    2. modify application.yml To configure , Add fusing configuration in the route

       

        Hystrix Two parameters are supported ：

        name： namely HystrixCommand Name

        fallbackUri：fallback Corresponding uri, there uri Support only forward:schemed

          filters:

            - name: Hystrix

              args:

                name: fallback

                fallbackUri: forward:/fallback

     3. establish FallBackControlle

         @RestController

         public class FallBackController {

             @GetMapping("/fallback")

             public String fallback() {

                 return "Error:fallback";

             }

         }

     4. test

          Start each micro service , And access , Shut down the producer after success

         

          thus ： The simple configuration of the fuse is completed , To customize the fusing strategy, you need to learn HystrixGatewayFilter Related content of

3. Spring Cloud Optimization of service first request timeout

    Spring Cloud After the project starts , For the first time to use FeignClient Requests are often time consuming , And there is a certain probability that the request will time out (java.net.SocketTimeoutException: Read timed out), As a result, it is possible to trigger a fuse

     This is because before invoking other microservice interfaces , Will request the relevant information of the micro service ( Address 、 Port, etc ), And do some initialization operations , Due to the default lazy loading feature , This results in... On the first call , There is a time-out situation ,

           

     There are two main solutions ：

    1. The first way is to set the timeout , How much to set , Project specific , The configuration is as follows

        #hystrix Timeout time of calling method , The default is 1000 millisecond

        hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds=5000

         In theory, this is a temporary solution , This process can solve the problem of timeout , But it can't solve the problem that takes a long time for the first time . At the same time, it is necessary to set the timeout time of the fuse longer ,

         Equivalent to limiting the scope of application of the fuse to a certain extent . So you can use this method , But this method is not recommended

    2. Ways to recommend ： To configure ribbon Immediately load ,

         Link analysis , The order of calls between services is ：gateway-> consumer -> producer ,   

         This problem is solved in two parts , One is the call between services Ribbon The hungry load of , Corresponding to the above test, call the producer for the consumer ; The second is the hungry loading of the gateway .

         

         First step ：

         Find the consumer's application.yml file , Add the following configuration ：

        # Note here , Immediate loading of optical configuration does not take effect , Also configure the client list

        ribbon:

          eager-load:

            enabled: true                                                                  # Start hunger mode

            clients: user-service, material-product, outer-data        # Producer's service name , Comma separated between multiple

         After restarting the consumer , You will find that although no call request is sent , But the log already shows Feign The client of was created successfully         

    

         The second step ： The hungry mode of gateway

        ribbon:

          eager-load:

            enabled: true

            clients: user-service-api

4. Personal experience

     Current limiting for external requests 、 Fuse for internal request

Appendix a ：JDK8 New features ——Lambda expression

stay JDK8 Before ,Java Functional programming is not supported , So called functional programming , It is understood that a function （ Also known as “ Behavior ”） Pass as a parameter .

Usually we talk more about object-oriented programming , Object oriented programming is the abstraction of data （ All kinds of POJO class ）, Functional programming is an abstraction of behavior （ Passing behavior as a parameter ）.

stay JavaScript This is a very common grammatical feature in , But in Java Passing a function as an argument doesn't work , Fortunately JDK8 The emergence of the new era has broken Java This limitation of . A simple example is as follows ：

     new Thread(new Runnable() {

         @Override

         public void run() {

             System.out.println("Hello World!");

         }

     });

      Use Lambda Expressions can replace the above way of using anonymous classes with just one sentence .

     new Thread(() -> System.out.println("Hello World!"));

Appendix 2 ： How to use JMeter Run concurrent tests

1. install

     Unzip the downloaded compressed package , Here I unzip the file to my computer D:\tools\apache-jmeter-5.2.1

2. function

     Click on bin In the catalog jmeter.bat You can start Jmeter.

3. A simple example of pressure measurement

     There is one http Request interface localhost:5000/usr3/hello, To use Jmeter Pressure test it , The test steps are as follows :

    1. Create a new thread group

    2. Set thread group parameters

       1.Number of Threads 10 Number of threads 10 Threads

       2.Ramp-Up Period      0    How many seconds does all threads start , Set to 0 Indicates simultaneous start

       3.Loop Count              1    Thread repeat data

        The configuration here is ：10 Threads , At the same time to start , Cycle time

    3. newly added http Request defaults

        On the thread group created in the previous step , newly added http Request defaults , All requests will use the default value set , This sets the protocol to http,IP by localhost, Port is 8080

    4. Add the http request

         agreement 、IP、 The port does not need to be set , Can use steps c The default value set in , Just set the request path Path that will do , Fill in here /usr3/hello

    5. New monitor , Used to check the pressure test results . Add three kinds of ： Aggregation report 、 Graphic results 、 Look at the results in a table , The difference is that the results are presented in different forms

    6. Click the run button to start the pressure test , And look at the results

    

Appendix III ：@Primary annotation

This annotation is used on classes . It means that where a single valued dependency needs to be automatically injected , There are multiple candidate dependencies , Then this annotation will specify a class as preference（ Preference ） choice .

It can be simply understood as , We put @Primary Annotations are marked on any class , In the use of @Autowired At the time of Injection , If not specified （ For special instructions, please see @Qualifier Explanation ）,

Then the default injection is @Primary Marked class . But only one class can be specified as a preference class , Otherwise, there will still be conflicts .

Appendix 4 ：SpringCloud Solution of service consumer's first call timeout problem

The first time you visit a service consumer （ The consumer invokes the service provider service ） The following exceptions will occur ：

com.netflix.hystrix.exception.HystrixRuntimeException: TestService#hello(String) timed-out and no fallback available

The solution is application.properties Add the following configuration information ：

#hystrix Timeout time of calling method , The default is 1000 millisecond

hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds=5000

A more absolute solution is to disable hystrix:

feign.hystrix.enabled=false