Amazing SQL query experience, group by slow query optimization!

Click on the top “ Terminal R & D department ”

 Set to “ Star standard ”, Master more database knowledge with you 

author ：dijia478

source ：https://www.cnblogs.com/dijia478/p/11550902.html

One 、 The problem background

Now the network appears slow query , stay 500 In the case of tens of thousands , The query speed of single table is 30 More than a second , Need to be right sql To optimize ,sql as follows ：

I built... In the test environment 500 Ten thousand data , Simulated this slow query .

Simply speaking , Is to query under certain conditions , Who are the users . Very simple. sql, You can see , The query time is 37 second .

The way app_account The distribution of fields , Randomly generated 5000 A different random number , And then it's distributed here 500 In 10000 pieces of data , On average , Every app_account There will be 1000 One is the value of repetition , There is a total of 5000 individual .

Two 、 Look at the execution plan

You can see ,group by I have index on the field , Also used. .

3、 ... and 、 Optimize

Tell the truth , I don't know how to optimize , How can this thing be optimized ！ First say , The following ideas are useless .

Train of thought ：

You should add order by null; Avoid useless sorting , But it doesn't really affect the results , It's still slow. .

Train of thought two ：

where The conditions are too complicated , No index , Resulting in slow query , But I gave it to you where All fields of the condition are combined with the index , It still doesn't work

Train of thought three ：

since group by slow , in distinct try ？？（ This is the magic part of this blog ）

Oh my god ？？？！！！ What's going on here , The moment is so fast ？？！！！

Though I know group by and distinct There is a small performance gap , But I didn't expect , The gap is so big ！！！ Great discovery ！！

Four 、 Do you think it's over

I really hope it's over , Then this problem is solved very simply , By the way, I think I found a new knowledge .

however ！

This bug After the test , Take a test , It's still 30 More than a second ！？ What's going on here ！！？？？

Of course I don't believe , Go to test the computer to execute sql, That's true 30 More than a second ...

I went back to my computer , Connect to the same database , One execution sql,0.8 second ！？

What circumstance , The same library , The same sql, How can there be such a big gap between the two computers ！

Later, it was executed directly on the server ：

drunk , It's still 30 More than a second ....

That seems to be my computer problem .

Later, I experimented with computers of several colleagues , The final conclusion is ：

It's because I used SQLyog！

Ah , Now we find , Only with sqlyog Execute this “ After optimization ” Of sql Would be 0.8 second , stay navcat And directly on the server , All are 30 More than a second .

That's it sqlyog The problem. , It's not clear sqlyog Is there any optimization , This slow query problem is still being solved （ I think the problem may be mysql On its own parameters ）.

This is just a record of the pit ,sqlyog perform sql Speed , And the server sql Speed , There are sql There is a huge difference between , It's not reliable .

5、 ... and 、 follow-up （ It's not settled yet ）

Thank you for your advice , I'll respond to the question ：

1. So-called sqlyog Quick query , Slow command line query , The reason has been found . Because sqlyog Will add... By default after the query statement limit 1000, So it leads to very quickly . This problem is no longer tangled .

2. The way I've tried （ It's no use ）：

① to app_account Field plus index .

② to sql Add... After the statement order by null.

③ adjustment where The query order of the fields in the condition , Put the indexed ones in front of you .

④ To all where Conditional fields plus Composite Index .

⑤ In the form of subqueries , First check where What's in the condition , Go back to the heavy .

The test environment and the current network environment data are a little different , I'll post a current website to execute sql Graph （1 minute ...）：

6、 ... and 、 Final solution

Thanks for the comments 42 Building @ Yanfeng ！

After your reminding , I did find out ,explain In the execution plan , The index doesn't seem to use what I created idx_end_time.

Then decisively tried in the present net , Force the use of idx_end_time Indexes , It turns out that as long as 0.19 second ！

So far, the problem has been solved , In fact, my colleagues were suspicious yesterday , Is there too many indexes in this table , It leads to the wrong use of , The original use was idx_org_id and idx_mvno_id.

Now it's mandatory to specify idx_end_time Just ok 了 ！

Finally, compare the implementation plan before and after the reform

Before the reform （ Query to 1 About minutes ）：

After the reform （ The query takes only a few hundred milliseconds ）：

——End——

 reply  【idea Activate 】 You can get idea How to activate 
 reply  【Java】 obtain java Relevant video tutorials and materials 
 reply  【SpringCloud】 obtain SpringCloud Many relevant learning materials 
 reply  【python】 Get the full set 0 Basics Python Knowledge Manual 
 reply  【2020】 obtain 2020java Related interview questions tutorial 
 reply  【 Add group 】 You can join the technical exchange group related to the terminal R & D department 
 Read more 
 use  Spring  Of  BeanUtils  front , I suggest you understand these pits first ！

lazy-mock , A lazy tool for generating backend simulation data 

 In Huawei Hongmeng  OS  Try some fresh food , My first one “hello world”, take off ！

 The byte is bouncing ：i++  Is it thread safe ？

 One  SQL  Accidents caused by , Colleagues are fired directly ！！

 Too much ！ Check Alibaba cloud  ECS  Of  CPU  Incredibly reach 100%

 a vue Write powerful swagger-ui, A little show （ Open source address attached ）


 Believe in yourself , Nothing is impossible , Only unexpected, not only technology is obtained here ！



 If you like, just give me “ Looking at ”