The core idea of performance optimization, dry goods sharing

The core idea of performance optimization ( Live recording screen )

One 、 background

During the technical communication at ordinary times , Many students will ask some questions about performance optimization .

such as ：

There is an order form to keep order records , Cannot delete when updating data , You need to plan to add the version number , When querying, take the latest version number to the front end , Another scenario is to query the order history to the front end . The order quantity is generally not too small , Each time, group the query first and get the largest one , Then the outer layer fetches the corresponding data and paging again. Obviously, the performance is not good . What do I do ？

There may also be some suggestions , so what ？ What to do next time I meet ？

Few articles on the Internet systematically explain the methodology of performance optimization , So take this opportunity to sum up , Share with you .

As the saying goes, it is better to teach a man to fish than to teach him to fish , This paper summarizes its performance optimization methodology here , Here are the concepts and core ideas of performance optimization , Help you systematically deal with performance optimization problems , Be able to understand the principles of performance optimization more clearly .

Two 、 dried food

2.1 The essence

The contradiction between good user experience and limited resources

2.2 The thinking source of performance optimization

2.2.1 Learn the optimization methods of operating system and various middleware

There are many classic cases of operating system performance optimization

Various middleware will also be optimized to improve performance

This requires everyone to take the initiative to learn , To accumulate .

2.2.2 The core idea

【1】 Heap hardware + Optimize software （ Algorithm 、 step ）

【2】 Open source （ Pile up the machine 、 Improve configuration ） + throttle （ Improve resource utilization , Less resources ）

【3】 Input 、 Calculation 、 Output

【4】 Balance

It's hard to get the best of it , Often only two out of three ！

2.2.3 Induction and deduction

In reading 、 Work to put together a class of optimization cases , Summarize the core law , Then when you encounter new problems , Look at these core rules , To design a corresponding solution to a practical problem .

2.2.4 The core idea

break up the whole into parts

Distributed Read / write separation

Gather parts into a whole

Batch interface Buffer

Space for time （ cache 、 Prewriting 、 Reserve space 、 Heap hardware ）

cache （ Browser cache 、CDN Speed up 、 Memory cache 、CPU cache …） Prewriting （ Prepare some operations in advance ） Reserve space （ArrayList Capacity expansion 1.5 times ; HashMap Capacity expansion mechanism of ;Redis String Space will also be reserved , Avoid frequent expansion ）

Heap hardware （ By adding more machines , Increase the number of cluster machines , Resist more traffic ） Improve configuration （ Give more memory 、 Broadband, etc ） Read / write separation （ Write main and read from ;COW When writing copy ）

Serial to parallel

Use parallel API Send a message to MQ in , Load balancing to multiple consumers , Increase spending power

Synchronous to asynchronous

Reduce the use of resources （ Compress 、 Merge 、 Reuse 、 Reduce output 、 Reduce context switching 、 Reduce orders ）

Pooling （ Thread pool 、 Connection pool 、 Object pool ） Compress 、 Merge 、 Reuse 、 Reduce output 、 Reduce context switching 、 Reduce orders

Spatiotemporal locality

Give full play to the advantages of all aspects （ Choose a reasonable data structure 、 Reasonable middleware, etc ）

For example, you need to use Set duplicate removal , instead of List such as HashMap In case of conflict , Linked list greater than 8 individual Turn into a red and black tree Use Redis, If applicable Set The structure should not be spliced every time String Storage , Take it out and cut it again every time . such as MySQL Of InnoDB choose B+ Trees , Mainly due to the low number of layers , such IO Fewer times , Each leaf node forms a linked list , So it's easier to find the range .

Optimize according to technical characteristics （ Deep page turning problem 、 Attribute conversion problem ）

Deep page turning problem , By moving where Conditions to achieve, always check only the first page , Improve performance Write manually get/set Conversion methods or use bytecode enhanced attribute conversion tools , Better performance than reflection based attribute conversion tools

Reduce conflict （ Time 、 Space ） The scope of the

Lock in sections COW Optimism lock

Push back products to optimize performance （ With restrictions ）

For example, message push optimization , Thousands of messages have to be sent 3 minute , The product was supposed to be in advance 5 Minutes to the user . Is it possible to discuss with the product in advance 10 Minutes ？ such as ES Deep page turning has performance problems , Is it right to discuss with the product to limit the search to 2000 strip ？

2. 4 matters needing attention

2.4.1 Avoid premature optimization

Premature optimization is the source of all evils .

For example, in the early stage of business development , The most important thing is to cater to the market , Grab market share , So if the performance of a function is average , But it doesn't particularly affect the user experience , You can also optimize later .

2.4.2 Consider input-output ratio

If a function is invested heavily , The optimization effect is not good , So should we think twice before we act ？

2.4.3 Consider correctness

While optimizing performance, we must consider correctness , If you can't guarantee the correctness after optimization, what's the significance of optimization ？

Performance optimization should be accompanied by more tests to ensure correctness .

2.4.4 Consider stability

2.4.5 Consider expansibility 、 Readability 、 Consider security

Performance is very good , But the readability is poor , Expansibility 、 The security is very poor , Then it's easy to bury a lot of hidden dangers .

For example, why foreigners rarely use fastjson, One of the important reasons is that there are a lot of hard coded things , And there are always security holes .

etc.

2.5 Book recommendation

Top of performance （ Strongly recommend ）

Graphic performance optimization

web Performance authority Guide

java Performance authority Guide

Deep understanding of computer systems

3、 Case study

3.1 The version number causes performance problems

Back to the case at the beginning

There is an order form to keep order records , Cannot delete when updating data , You need to plan to add the version number , When querying, take the latest version number to the front end , Another scenario is to query the order history to the front end . The order quantity is generally not too small , Each time, group the query first and get the largest one , Then the outer layer fetches the corresponding data and paging again. Obviously, the performance is not good . What do I do ？

1 Space for time

See also an order history sheet , Specifically for querying history .

Create... For the first time , Insert the record into the history table when modifying the order , Then directly update the order table records .

Add an index like this , Query the order list quickly , Querying order history is also fast .

2 Choose the right technology （ storage ）

The order form may be large , Larger order history , Although there is an index , But if the data volume exceeds ten million, it will also be very slow .

It may be necessary to consider sub database and sub table .

Then you can use the order form MySQL, The order history table uses HBase. In this way, the amount of data in the order history table is large and not slow .

3 Optimize according to technical characteristics

Such as rowkey Design shopid_order_id If a store has a large amount of data , Easy to become a hot spot ,

The data will tilt Then it can be designed as reverse(shopid_order_id) As rowkey

There may be a better solution , Here's an example .

3.2 Live broadcast optimization

For example, live broadcasting service , It is expected that millions of users will pour into... In a few minutes , A few days before the live broadcast, some users will subscribe to the live broadcast , Most users will before and after the live broadcast 5 Minutes into . How to optimize ？

1 Pressure test , Server expansion （ Hardware aspect ）

2 Warm up before the activity （ Software aspect ）

According to the live broadcast id And activity time , Set before live broadcast N Warm up in minutes , Cache the results of the purchase ;

If you unsubscribe, you can also delete the cache , Unsubscribe is a minority after all Below redis by redis colony By preheating , When users rush into the live broadcast room for authentication in a short time , Most traffic goes to the cache cluster ,DB The pressure will be small .

3 be based on Hytrix Downgrade （ To the retry page ） Or current limiting

3.3 Report export

Like a scene , Suppose a node user must download a file , The generation of this file is time-consuming .

Two common options

• asynchronous
• Generated in advance

Then you can generate a file to be downloaded in advance at an event before the user downloads a file , Users can download quickly when they need to download , Great experience .

This also reflects the concept of exchanging space for time .

More examples , Please directly see the supporting live video at the end of the text .

4、 Summary

5、 summary

This article mainly explains my performance optimization methodology , The most fundamental thinking direction goes to the core optimization principles and then to specific optimization cases , I hope it can help you understand the essence of performance optimization , Master the core idea of performance optimization , Help everyone to avoid detours in practical work .

When considering performance optimization , Multiple core principles can be combined , In addition, consider the limitations or shortcomings of your design , Also conduct sufficient tests to ensure correctness .

Original article , Reprint please indicate the source ：https://blog.csdn.net/w605283073/article/details/107589987