Starting from 1.5, build a micro Service Framework -- log tracking traceid

Hello , I'm Wukong .

Preface

Recently, I am building a basic version of the project framework , be based on SpringCloud Microservice framework .

If you put SpringCloud This framework is used as 1, Now there are some basic components, such as swagger/logback Wait, that's it 0.5 , Then I'm here 1.5 Assemble based on , Complete a microservice project framework .

Why build the second generation wheels ？ Isn't the ready-made project framework on the market fragrant ？

Because the project team is not allowed to use external ready-made frameworks , such as Ruoyi. In addition, our project needs have their own characteristics , Technology selection will also choose the framework we are familiar with , So it's also a good choice to build the second generation wheels by yourself .

Core functions

The following core functions need to be included ：

• Split multiple micro service modules , Draw out one demo Microservice module for expansion , Completed

• Extract core framework modules , Completed

• Registry Center Eureka, Completed

• The remote invocation OpenFeign, Completed

• journal logback, contain traceId track , Completed

• Swagger API file , Completed

• Configure file sharing , Completed

• Log retrieval ,ELK Stack, Completed

• Customize Starter, undetermined

• Consolidated cache Redis,Redis Sentinel high availability , Completed

• Consolidate databases MySQL,MySQL High availability , Completed

• Integrate MyBatis-Plus, Completed

• Link tracking component , undetermined

• monitor , undetermined

• Tool class , To be developed

• gateway , Technology selection to be determined

• Audit log entry ES, undetermined

• distributed file system , undetermined

• Timing task , undetermined

• wait

This article is about log link tracking .

One 、 Pain points

It hurts a little ： Multiple logs in the process cannot be traced

A request calls , Suppose you will call more than a dozen methods on the back end , Print the log more than ten times , These logs cannot be concatenated .

As shown in the figure below ： The client calls the order service , Methods in order service A Calling method B, Method B Calling method C.

Method A Print the first log and the fifth log , Method B Print the second log , Method C Print the third log and the fourth log , But this 5 This log has no connection , The only connection is that time is printed in chronological order , But if there are other concurrent request calls , It will interfere with the continuity of the log .

Pain point two ： How to correlate logs across Services

Every micro service will record its own log of this process , How to correlate logs across processes ？

As shown in the figure below ： Order service and coupon service belong to two micro Services , Deployed on two machines , Order service A Method to call the coupon service remotely D Method .

Method A Print the log to the log file 1 in , Recorded 5 Logs , Method D Print the log to the log file 2 in , Recorded 5 Logs . But this 10 Logs cannot be associated .

Pain point three ： How to correlate logs across threads

How the logs of the main thread and the sub thread are related ？

As shown in the figure below ： Method of main thread A Started a sub thread , The sub thread executes the method E.

Method A Printed the first log , Sub thread E The second log and the third log were printed .

It hurts four ： The third party calls our service , How to track ？

The core problem to be solved in this article is the first and second problems , Multithreading has not been introduced yet , At present, there is no third party to call , We will optimize the third and fourth questions later .

Two 、 programme

1.1 Solution

① Use Skywalking traceId Link tracking

② Use Elastic APM Of traceId Link tracking

③ MDC programme ： Make your own traceId and put To MDC Inside .

At the beginning of the project , Don't introduce too many middleware , Try a simple and feasible solution first , So here is the third solution MDC.

1.2 MDC programme

MDC（Mapped Diagnostic Context） Used to store context data for a specific thread running context . therefore , If you use log4j Logging , Each thread can have its own MDC, The MDC Global to the entire thread . Any code belonging to the thread can easily access the thread's MDC Exists in .

3、 ... and 、 Principle and practice

2.1 Track multiple logs of a request

Let's first look at the first pain point , How to in a request , Connect multiple logs .

The principle of this scheme is shown in the figure below ：

（1） stay logback Add... To the log format in the log configuration file %X{traceId} To configure .

<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %X{traceId} %-5level %logger - %msg%n</pattern>

（2） Customize an interceptor , From request header In order to get traceId , If it exists, put it in MDC in , Otherwise, use it directly UUID treat as traceId, Then put MDC in .

（3） Configure interceptors .

When we print the log , It will print automatically traceId, As shown below , Of multiple logs traceId identical .

Sample code

Interceptor code ：

/**

 * @author www.passjava.cn, official account ： Wukong chat structure 

 * @date 2022-07-05

 */

@Service

public class LogInterceptor extends HandlerInterceptorAdapter {

    private static final String TRACE_ID = "traceId";

    @Override

    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {

        String traceId = request.getHeader(TRACE_ID);

        if (StringUtils.isEmpty(traceId)) {

            MDC.put("traceId", UUID.randomUUID().toString());

        } else {

            MDC.put(TRACE_ID, traceId);

        }

        return true;

    }

    @Override

    public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView) throws Exception {

        // Prevent memory leaks 

        MDC.remove("traceId");

    }

}

Configure interceptors ：

/**

 * @author www.passjava.cn, official account ： Wukong chat structure 

 * @date 2022-07-05

 */

@Configuration

public class InterceptorConfig implements WebMvcConfigurer {

    @Resource

    private LogInterceptor logInterceptor;

    @Override

    public void addInterceptors(InterceptorRegistry registry) {

        registry.addInterceptor(logInterceptor).addPathPatterns("/**");

    }

}

2.2 Track multiple logs across Services

The schematic diagram of the solution is shown below ：

The order service calls the coupon service remotely , You need to add OpenFeign Interceptor , What the interceptor does is go Requested header Add traceId, When calling the coupon service in this way , From the header Get this request traceId.

The code is as follows ：

/**

 * @author www.passjava.cn, official account ： Wukong chat structure 

 * @date 2022-07-05

 */

@Configuration

public class FeignInterceptor implements RequestInterceptor {

    private static final String TRACE_ID = "traceId";

    @Override

    public void apply(RequestTemplate requestTemplate) {

        requestTemplate.header(TRACE_ID, (String) MDC.get(TRACE_ID));

    }

}

In the logs printed by two microservices , Two logs traceId Agreement .

Of course, these logs will be imported into Elasticsearch Medium , And then through kibana Visual interface search traceId, You can string the whole call link ！

Four 、 summary

This article passes the interceptor 、MDC function , The full link is added traceId, And then traceId Output to log , You can trace the call link through the log . Whether it's in-process method level calls , Or cross process service invocation , Can be tracked .

In addition, the log also needs to pass ELK Stack Technology imports logs into Elasticsearch in , Then you can search traceId, The whole call link is retrieved .

- END -

from 1.5 Start to build a micro service framework —— Log tracking traceId More articles about

1. Java Advanced topics ( Twenty-two ) Build a microservice architecture system from scratch ( On )

   Preface " Microservices " The word comes from Martin Fowler The name is Microservices Of , post , You can find it on his official blog http:/ /martinfowler . com/art ...

2. spring cloud introduction , Take a look at a microservice framework 「 the viscera 」

   Spring Cloud It's based on Spring Boot Implemented microservice framework , It contains the various components needed to implement the microservice architecture . notes :Spring Boot To understand simply is to simplify Spring Project setup . To configure . Group ...

3. from Spring Cloud Take a look at a microservice framework 「 the viscera 」

   original text :https://webfe.kujiale.com/spring-could-heart/ Spring Cloud It's based on Spring Boot Implemented microservice framework , It includes the implementation of microservice Architecture ...

4. from Spring Cloud Take a look at a microservice framework 「 the viscera 」（ turn ）

   Spring Cloud It's based on Spring Boot Implemented microservice framework , It contains the various components needed to implement the microservice architecture . This article will start from Spring Cloud set out , There are two sections about the microservice framework 「 the viscera 」: ...

5. build SpringCloud Microservice framework ： One 、 Structure and components

   Build a micro service framework ( Structure and components ) brief introduction SQuid Is based on Spring,SpringBoot, Used SpringCloud Build components under , The purpose is to build a set of systems that can be rapidly developed and deployed , And a set of microservice framework that is very useful ...

6. 【 Microservices 】 Use spring cloud Build a micro service framework , Organize learning materials

   Write it at the front Use spring cloud Build a micro service framework , It's one of my most important jobs recently , At first I used bubbo Add zookeeper We've made one based on dubbo The microservice framework of , Then it was rejected by the architect , The architect said : This thing is out of date . along with ...

7. 【 Distributed 】-- be based on Nacos、OpenFeign Build a small background case of micro service lottery system

   1. Project introduction Before entering the project recently, I need to be familiar with SpringCloud Nacos Microservices are based on Feign Interface call and integrate Swagger2 Display the interface document to the front end , So I wrote and integrated a set of software based on SpringC ...

8. Spring coud Key instructions for the specific implementation of the microservice framework

   Build a micro service , The problems considered involve operation and maintenance , Data management , performance , Concurrency and other aspects . Project use Spring coud Build microservices , Technology selection , To realize technology, we should consider the problem of service in all aspects . The following is a brief summary of the technical instructions used in the construction process ...

9. Dubbo Ecosystem - From micro service framework to micro service ecosystem

   From micro service framework to micro service ecosystem , This is the inevitable trend of the development of microservices , It's also Dubbo The community meets the mission and responsibility of developers to build a more efficient micro service system . In the near future ,Apache Dubbo PPMC Wang Tao ( Community nickname :ralf0131) do ...

10. .Net Microservice practice （ One ）： Micro service framework selection

    Microservice framework Microservices (Microservices) It's an architectural style , A large complex software application consists of one or more microservices . Each microservice in the system can be deployed independently , The microservices are loosely coupled . Each microservice focuses on completing only one task and is easy to implement ...

Random recommendation

1. springmvc Of foward and redirect Jump simple analysis

   Spring MVC in , When we go back to the logical view , The framework will go through viewResolver To get the concrete View, Then render to the browser . Suppose the logical view is named hello, By configuring , We   Configure a ViewRe ...

2. wpf Next datagrid Several points needing attention during use （ One ）

   MainWindow.xaml The code in is as follows : <DataGrid CanUserAddRows="False" ItemsSource="{Binding}&quo ...

3. Manually shut down searchDisplayControlelr

   Two lines of code [searchBar endEditing:YES]; [searchDisplayControllersetActive:NO];

4. Matrix fast power UVA 10870 Recurrences

   Subject portal The question :f(n) = a1f(n − 1) + a2f(n − 2) + a3f(n − 3) + . . . + adf(n − d), for n > d, seek f (n) % m. training ...

5. Spring in Aware Related interface principles

   Spring There are some Aware Related interfaces , Like BeanFactoryAware. ApplicationContextAware.ResourceLoaderAware.ServletContextA ...

6. ubuntu System installation flashplayer

   Open the browser , Input adobe flashplayer Access to official website , download Linux 32-bit, Simplified Chinese , Firefox, download .tar.gz package . Then click download now . After downloading, find and unzip the folder , find ...

7. 201521123048 《Java Programming 》 The first 7 Weekly learning summary

   1. This week's learning summary 2. Written work 1.ArrayList The code analysis 1.1 explain ArrayList Of contains Source code public boolean contains(Object o) { re ...

8. Make U disc Win10 PE

   1. install Windows ADK Download address  http://go.microsoft.com/fwlink/p/?LinkID=232339 2. Started as administrator “ Deployment and imaging tools environment ” 3. establish WinP ...

9. Xiao Wei, let's go vijos-1572 jdoj-1572

   Xiao Wei, let's go The main idea of the topic : A length of n String , from ()[]{} form . Find the longest harmonious string : Our definition of the longest harmonious string is that each bracket has a bracket with the same nature to match it , The middle of these two brackets is either empty , Or it's harmonious . Several consecutive ...

10. Swagger How to access the Ocelot With permission verification API

    Source code first :https://github.com/axzxs2001/Asp.NetCoreExperiment/tree/master/Asp.NetCoreExperiment/SwaggerDe ...