An online accident, I suddenly realized the essence of asynchrony

---

Source of the article ：【 official account ： Brother Yong java Actual combat sharing 】

Catalog

• Preface

• Business scenario

• Thread pool mode

• Local memory + Timing task

• MQ Pattern

• Agent service +MQ Pattern

• summary

Preface

In a high concurrency scenario , Asynchrony is an extremely important optimization direction . Some time ago , An accident occurred in the production environment , The author believes that the scene of the accident is very typical .

Write this article , The author wants to discuss the architecture optimization scheme of this scenario with you . I hope that after reading , You can have a deeper understanding of asynchrony .

Business scenario

Teachers log in to the teaching and research platform , You will see the list of courses , After clicking on the course , The course will be shown in the form of video .

Visit the course details page , Contains two core actions ：

• Read the course video information ： From the cache server Redis Get the video information of the course , Back to the front end , The front end is rendered by video components .

• Write the record of course watching behavior ： When the teacher watches the video , Browser every 3 Request in seconds , The teaching and research service inserts the viewing behavior record into the database table . And as the number of users online increases , The frequency of write operations will also increase exponentially .

Early online , This design works fairly well , But with the increase of online users , The system response is getting slower , A large number of threads are blocking writing to the video viewing schedule Dao On the way .

First of all, we will think of a very intuitive scheme , Improve the ability to write to the database ：

• Optimize SQL sentence

• promote MySQL Database hardware configuration  

• Sub database and sub table

In fact, this scheme can also meet our needs , But expanding the hardware is not cheap , In addition, the write operation can allow appropriate delay and loss of a small amount of data , Then this kind of scheme is more cost-effective .

Then the direction of architecture optimization should be ：“ Reduce the time of writing , Improve the concurrency of write actions ”, Only in this way can the system run more smoothly .

therefore , We came up with the second plan ： Write request asynchronization .

• Thread pool mode

• Local memory + Timing task

• MQ Pattern

• Agent service + MQ Pattern

Thread pool mode

2014 year , The author is in charge of the red envelope system related work in eLong Travel Network . The operation system will call the red packet system to send red packets to specific users , When these users log in app after ,app The end will call the activation red packet interface of the red packet system .

Activating the red packet interface is a write operation , It's also faster （20 Millisecond or so ）, The daily request volume of the interface is 2000 All around .

Application access peak , The red envelope system will become unstable , The activation interface often times out , In order to solve the problem quickly , Adopted a very rough plan ：" After the controller receives the request , Put the write operation into a separate thread pool , Return to the front end immediately , The thread pool will execute the activation red packet method asynchronously ".

At that time, after optimization according to this rough method , The red envelope system is very stable , There is no longer the problem of interface response timeout .

Back to the scene of teaching and research , See the picture below , We can also design a scheme similar to the thread pool model ：

Use thread pool mode , We need to pay attention to the following points ：

• The number of threads should not be too high , Avoid occupying too many database connections

• You need to consider the size of the queue of the evaluation route pool , In order to avoid the problem of memory overflow

Local memory + Timing task

Open source China's scheme for counting the number of views is very classic . The user visited the article once 、 Journalism 、 Code details page , Number of visits field plus 1 , stay oschina The last operation is asynchronous , When accessing, you just save the data in memory , Write these data to the database every fixed time .

The sample code is as follows ：

We can learn from the scheme of open source China ：

• After the controller receives the request , Watch progress information stored in local memory LinkedBlockingQueue In the object

• Asynchronous threads every 1 Minutes to get data from the queue , Assemble into List object , Last call Jdbc batchUpdate Method batch write to the database

• Batch writing is mainly to improve the overall throughput of the system , Each batch write List The size should not be too large

The advantage of this scheme is ： Do not change the original business structure , Simple and easy to use , High performance . This scheme also needs to consider the risk of memory overflow .

MQ Pattern

Many students will think MQ Pattern , The core functions of message queue are asynchrony and decoupling ,MQ The pattern structure is clear , extensible .

The core process is as follows ：

• The controller receives a write request , Convert video viewing behavior records into messages  

• The teaching and research service sends a message to MQ  , Return the success information of the write operation to the front end  

• Consumer service starts from MQ Get message from , Batch operation database

The advantage of this scheme is ：

• MQ It supports high availability and asynchrony , It's efficient to send messages , It also supports mass consumption

• Message in MQ The server will persist , Reliability is higher than saving in local memory

however MQ Patterns need to introduce new components , Add extra complexity .

Agent service + MQ Pattern

Internet companies also have a common asynchronous solution ：Agent service + MQ Pattern .

Deploy on the teaching and research server Agent service （ Independent process ） , After the teaching and research service receives the write request , Follow the request in a fixed format （ such as JSON ） Write to disk , Then return the success message to the front end .

Agent The service will listen for file changes , Send the contents of the file to the message queue , Consumer services get viewing behavior records , Store it in MySQL In the database .

The biggest advantage of this scheme is ： The architecture is layered clearly , Business services do not need to be introduced MQ Components . The performance monitoring platform that the author used to contact , Or log analysis platforms use this mode .

summary

Learning requires progressive thinking layer by layer .

first floor ： Under what scenario does asynchrony need

as follows ：

• A large number of write operations occupy too many resources , Affect the normal operation of the system

• After the write operation is asynchronous , It doesn't affect the main process , Allow appropriate delays

The second floor ： Asynchronous external mental skill

This article mentioned four asynchronous methods ：

• Thread pool mode

• Local memory + Timing task

• MQ Pattern

• Agent service + MQ Pattern

Their common feature is ： Store write commands behind a pool , Immediately respond to the front end , Reduce the time of writing . The task service asynchronously obtains tasks from the pool and executes .

The third level ： The essence of asynchrony

In my opinion , Asynchrony is a more granular way to use system resources . In the scene of teaching and research course details , The resources of the database are fixed , But write operations occupy a lot of database resources , Cause blocking of the whole system , But writing is not the core business process , It should not occupy so much system resources .

Cannot be asynchronous for the sake of asynchrony , Whether using thread pools , Or local memory + Timing task , Or is it MQ , The use of database resources should be within a reasonable range , Otherwise, asynchrony will not achieve the desired effect .

-------------  END  -------------
 Scan the code for free 600+ Page Shishan teacher's original boutique article summary PDF



 Summary of original technical articles 


 You'd better watch it