Resilience4j retry source code analysis and retry index collection

Preface

demand

   To prevent network jitter , Need to retry processing , Retry the alarm notification after reaching the threshold , Timely response to problems

Technology selection

type	Sync 、 asynchronous	Whether declarative calls are supported （ annotation ）	Whether monitoring is supported
resilience4j-retry	Sync	yes	yes
Guava Retry	Sync	no	no , Monitoring and statistics can be realized by the listener
Spring Retry	Sync	yes	no , Monitoring and statistics can be realized by the listener

Based on the comparison of the above schemes , Choose to use resilience4j-retry, Mainly based on the following two points ：

1. It provides monitoring data , Perfect access premethus
2. resilience4j In addition to providing retry capability , There is also Hystrix The same ability , Including circuit breakers 、 partition 、 Current limiting 、 cache . Provide with Spring Boot Integration dependencies , Greatly simplifies integration costs .（ In the later stage, we can consider from Hystrix Migrate to resilience4j）

Raise questions

1. resilience4j-retrry How to integrate into the project and how to use ？
2. How to customize the time interval ？
3. resilience4j-retry Realization principle ？
4. How to count the monitoring data and premethus How to collect ？

Problem analysis

resilience4j-retrry How to use

1. maven introduce resilience4j-spring-boot2 package

   <dependency>
   		<groupId>io.github.resilience4j</groupId>
   		<artifactId>resilience4j-spring-boot2</artifactId>
   		<version>1.7.1</version>
   </dependency>
   

2. Configure retry service

   //  Corresponding @Retry Annotated name attribute 
   resilience4j.retry.instances.sendConfirmEmail.max-attempts=3
   

3. Add... To the method that needs to be retried @Retry annotation

   @Retry(name= "sendConfirmEmail",fallbackMethod = "sendConfirmEmailFallback")
   public void sendConfirmEmail(SsoSendConfirmEmailDTO ssoSendConfirmEmail) {
      // Omit method content 
      throw new ServiceException("send confirm email error"); 
   }
   

4. Definition fallbackMethod
   4.1 It's important to remember ,fallbackMethod Should be placed in the same class , And must have the same method signature , Only one additional target exception parameter is required
   4.2 If there are more than one fallbackMethod Method , The closest matching method will be called

   public void sendConfirmEmailFallback(SsoSendConfirmEmailDTO ssoSendConfirmEmail,ServiceException e){
      // Send email notification 
   }
   

Custom time interval

1. By default, retry at a fixed time interval , But if you want to do it now 1s->2s-3s Time interval increment , You need to customize the time interval
2. Realization IntervalBiFunction Interface , Custom time interval class

   public class SendEmailIntervalBiFunction implements IntervalBiFunction<Integer> {
   
       private final Duration waitDuration = Duration.ofSeconds(1);
   
       @Override
       public Long apply(Integer numOfAttempts, Either<Throwable, Integer> either) {
           return numOfAttempts * waitDuration.toMillis();
   	}
   }
   

3. Configure the specified custom time interval class
   3.1 adopt Class.forName To load the custom time interval class

   resilience4j.retry.instances.sendConfirmEmail.interval-bi-function=com.xxx.xxx.retry.SendEmailIntervalBiFunction
   

resilience4j-retry Source code analysis

1. Create a test method to debug

   @RunWith(SpringRunner.class)
   @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
   public class RetryTest {
   
       @Resource
       private UserApiService userApiService;
   
       @Test
       public void testRetryThreeTimes() throws InterruptedException {
           SsoSendConfirmEmailDTO ssoSendConfirmEmailDTO = null;
           userApiService.sendConfirmEmail(ssoSendConfirmEmailDTO);
       }
   }
   

2. Definition Retry section ：RetryAspect, Yes @Retry The class or method identified by the annotation is intercepted
   2.1 according to @Retry Annotated name establish Retry Implementation class ：RetryImpl
   2.2 according to @Retry Annotated fallbackMethod establish FallbackMethod（ According to the method 、 Parameters 、 Abnormal reflection gets the corresponding method ）
   2.3 Try processing again （ Finally, there is the retry implementation class completion function ：RetryImpl#executeCheckedSupplier）

   @Around(value = "matchAnnotatedClassOrMethod(retryAnnotation)", argNames = "proceedingJoinPoint, retryAnnotation")
   public Object retryAroundAdvice(ProceedingJoinPoint proceedingJoinPoint,
       @Nullable Retry retryAnnotation) throws Throwable {
       // according to name establish Retry Implementation class ：RetryImpl   ---> Retry retry = retryRegistry.retry(backend)
       io.github.resilience4j.retry.Retry retry = getOrCreateRetry(methodName, backend);
       
       //  according to @Retry Annotated fallbackMethod establish FallbackMethod -->FallbackMethod#create
   	FallbackMethod fallbackMethod = FallbackMethod
           .create(fallbackMethodValue, method, proceedingJoinPoint.getArgs(),
               proceedingJoinPoint.getTarget());
   	
   	// Try processing again ：RetryAspect#proceed  --> Final trigger RetryImpl#executeCheckedSupplier
   	return fallbackDecorators.decorate(fallbackMethod,
           () -> proceed(proceedingJoinPoint, methodName, retry, returnType)).apply();
   }
       
   

Try processing again

1. The core approach ：Retry#decorateCheckedSupplier（do…while(true)）
   1.1 Get retry context ：RetryImpl$ContextImpl
   1.2 Called by @Retry Modified business methods
   1.3 Process the results （ And if an exception occurs , Handle exceptions ）

   static <T> CheckedFunction0<T> decorateCheckedSupplier(Retry retry,
                                                          CheckedFunction0<T> supplier) {
       return () -> {
       	// Get retry context ：RetryImpl$ContextImpl
           Retry.Context<T> context = retry.context();
           do {
               try {
               	//  Tone quilt @Retry Modified business methods 
                   T result = supplier.apply();
                   final boolean validationOfResult = context.onResult(result);
                   if (!validationOfResult) {
                       context.onComplete();
                       return result;
                   }
               } catch (Exception exception) {
                   context.onError(exception);
               }
           } while (true);
       };
   }
   

2. Retry processing after exception ：RetryImpl$ ContextImpl#onError
   2.1 If the exception is a retrieable exception , Then retry ：RetryImpl$ContextImpl#throwOrSleepAfterException

   private void throwOrSleepAfterException() throws Exception {
       int currentNumOfAttempts = numOfAttempts.incrementAndGet();
       Exception throwable = lastException.get();
       //  If the number of retries exceeds the threshold , Throw an exception 
       if (currentNumOfAttempts >= maxAttempts) {
           failedAfterRetryCounter.increment();
           publishRetryEvent(
               () -> new RetryOnErrorEvent(getName(), currentNumOfAttempts, throwable));
           throw throwable;
       } else {
       	//  Within retry range , be sleep Time interval between 
           waitIntervalAfterFailure(currentNumOfAttempts, Either.left(throwable));
       }
   }
   

Retry data collection

1. The role of data ： Retry through analysis service succeeded 、 Retry fail 、 Failed to retry 、 No retry success data , Judge the stability of the service
2. When retrying processing , Store statistics in RetryImpl Attribute
   2.1 stay RetryImp$ ContextImpll#onComplete Statistics succeededAfterRetryCounter、failedAfterRetryCounter、succeededWithoutRetryCounter
   2.2 stay RetryImp$ ContextImpll#onError Statistics failedWithoutRetryCounter

   // Number of successful attempts after retry 
   private final LongAdder succeededAfterRetryCounter;
   //  Number of failures after retry （ Failure after exceeding the threshold ）
   private final LongAdder failedAfterRetryCounter;
   //  The number of successful attempts without retrying 
   private final LongAdder succeededWithoutRetryCounter;
   //  The number of failures without retry （ Not a retryable exception ）
   private final LongAdder failedWithoutRetryCounter;
   

3. premethus Collect retry data
   3.1 introduce premethus Collect related packages , Expose the acquisition interface

   <dependency>
   	<groupId>io.micrometer</groupId>
   	<artifactId>micrometer-registry-prometheus</artifactId>
   	<version>1.7.1</version>
   </dependency>
   <dependency>
   	<groupId>io.micrometer</groupId>
   	<artifactId>micrometer-core</artifactId>
   	<version>1.7.1</version>
   </dependency>
   

   3.2 To configure actutor to open up premethus Acquisition interface
      3.2.1 premethus Acquisition interface ：PrometheusScrapeEndpoint#scrape
      3.2.2 send out /actutor/prometheus Trigger collection ：AbstractRetryMetrics#registerMetrics

   management.server.port=9099
   management.endpoint.health.show-details=always
   management.endpoints.web.exposure.include=health,prometheus
   

Bundle language

1. I understand that retrying can only solve network exceptions , Business exception retry cannot be solved
2. If it is a page interaction trigger , In this way, the interaction time will be prolonged （ Can't accept ）
   2.1 Add @Aync The annotation asynchronizes the retry method , Avoid page waiting （ If the application is down at this time, how to deal with it ？）
3. Welcome to the discussion , Give a good solution