When MySQL uses left join to query tables, the query is slow because the connection conditions are not properly guided

background

A recent list of background functions , Business staff feedback that the speed of query and export is very slow .

Through locating, we find that both list query and data export use the same linked table query SQL.

This function has just been launched , At first, the query and export speed are quite fast , Put this SQL It's also very fast to put into the test environment .

Compare the test environment with the production environment. The related table structure is the same , Then we focused on the problem of quantity , However, the data volume of several related tables is also small , Less than 1w Why is the amount of data so slow .

screening

adopt Explain Find out , In linked table query table c No index is used and full table scanning . In addition to Extra It is specifically stated in Using join buffer (Block Nested Loop).

among table c Medium filtered=100% Indicates that the right table does not apply index pushdown （ICP）, because where Condition has no index .

in addition Using join buffer (Block Nested Loop) Because the right watch is not join The construction of cables on the column leads to nested loops .

solve

Through to table c Connection fields in content_id and user_no The indexes are added respectively ,

The execution plan after adding the index is as follows  

summary

We need to pay attention to ： Participate in join Table of , You need to index the connection conditions .

The extension of knowledge

MySQL Use nested loop algorithm or its variants to join tables .

stay 5.5 Before the release ,MySQL Only one inter table association method is supported , That is, nested loops (Nested Loop). If the associated table has a large amount of data , that join The association will take a long time . stay 5.5 After the version ,MySQL Introduced BNL Algorithm to optimize nested loops .

1. Nested loop join algorithm (Nested-Loop Join Algorithm)

A simple nested loop connection （NLJ） The algorithm reads a row by row from the first table in the loop , Pass each row to the nested loop that handles the next table in the join . This process will be repeated many times , Because there are still remaining tables to be connected .

Suppose you want to execute three tables using the following connection types t1,t2 and t3 Connection between ：

Log in and copy

      
      

       
       Table   Join Type
       
       
t1      range
       
       
t2      ref
       
       
t3      ALL
      
      
      
      

       
       
1.
       
       
2.
       
       
3.
       
       
4.
      
      

If you use a simple NLJ Algorithm , Connections are handled like this ：

Log in and copy

      
      

       
       for(row_1 in table_1){
       
       
    for(row_2 in table_2){
       
       
        if(row_1,row_2 Satisfy join Conditions ){
       
       
            ...
       
       

       
       
            for(row_n in table_n){
       
       
                if(row_1,row_2...row_n All satisfied with join Conditions ){
       
       
                     hold row_1,row_2...row_n Of join The result is added to the result set 
       
       

       
       
                }
       
       

       
       
            }    
       
       

       
       
      }
      
      
      
      

       
       
1.
       
       
2.
       
       
3.
       
       
4.
       
       
5.
       
       
6.
       
       
7.
       
       
8.
       
       
9.
       
       
10.
       
       
11.
       
       
12.
       
       
13.
       
       
14.
      
      

As shown in the figure

 

The defect of this algorithm is also obvious , With join An increase in the number of tables , The amount of calculation increases exponentially . If there is a table with a large amount of data , It also has a great impact on the efficiency of the whole process .

therefore ,mysql5.5 The algorithm is optimized , Added Index Nested-loop Join,Block Nested-loop Join.

2. Index nested loop join algorithm (Index Nested-loop Join Algorithm)

Index Nested-loop Join It is for the situation with index , and Block Nested-loop Join It is for the case that the index is not hit .

 

Because the efficiency of index is higher than that of item by item cycle , So when using the index join table , It can greatly speed up the query , But indexing is not everything , If you need to get fields other than the index , Then you still need to go back to the table to find the corresponding data .

3. Block nested loop connection algorithm (Block Nested-Loop Join Algorithm)

Block Nested-loop Join Block nesting loop （BNL） The join algorithm uses the buffer of rows read in the external loop to reduce the number of times that tables in the internal loop must be read . 

A simple example ： The outer loop result set has 1000 Row data , Use NLJ The algorithm needs to scan the inner table 1000 Time , But if BNL Algorithm , Take out the result set of the outer table first 100 Line deposit join buffer,  Then use each row of data in the inner table to compare with this 100 Compare row result sets , Can be used with 100 Compare row data , In this way, the inner table only needs to cycle 1000/100=10 Time , Less 9/10.

 

 

Reference article ：

​ ​http://blog.sina.com.cn/s/blog_a1e9c7910102x1bz.html​​

​

 

If this article helps you , Please 「 Lingyejun 」 Point a praise , Thank you for your support .

author ： Lingyejun

If you like or want to see more of my articles , Pay attention to my WeChat official account 《 Lingyejun 》.

Please keep the source and signature when reprinting the article , Otherwise, the right to pursue legal responsibility is reserved .