Understand what MySQL index push down (ICP) is in one article

One 、 brief introduction

ICP（Index Condition Pushdown） Is in MySQL 5.6 Query optimization strategy launched on version , Put the original Server The index condition check performed by the storage engine layer is pushed down to the storage engine layer , To reduce the number of times to return to the table and access the storage engine , Improve query efficiency .

Two 、 principle

In order to understand ICP How it works , Let's first understand that it is not used ICP Under the circumstances ,MySQL How to query ：

• The storage engine reads index records ;
• According to the primary key value in the index , Locate and read the complete row record ;
• The storage engine gives the records to Server Layer to detect whether the record meets WHERE Conditions .

Use ICP Under the circumstances , The query process is as follows ：

• Read index records （ Not a complete line record ）;
• Judge WHERE Whether the condition part can be checked with the columns in the index , Condition not satisfied , Then process the next row of index records ;
• Conditions met , Use the primary key in the index to locate and read the complete row record （ It's the so-called back table ）;
• The storage engine gives the records to Server layer ,Server The layer detects whether the record meets WHERE The rest of the condition .

3、 ... and 、 practice

First create a table , And insert the record

CREATE TABLE user (
id int(11) NOT NULL AUTO_INCREMENT COMMENT " Primary key ",
name varchar(32)  COMMENT " full name ",
city varchar(32)  COMMENT " City ",
age int(11)  COMMENT " Age ",
primary key(id),
key idx_name_city(name, city)
)engine=InnoDB default charset=utf8;

insert into user(name, city, age) values("ZhaoDa", "BeiJing", 20),("QianEr", "ShangHai", 21),("SunSan", "GuanZhou", 22), ("LiSi", "ShenZhen", 24), ("ZhouWu", "NingBo", 25),  ("WuLiu", "HangZhou", 26), ("ZhengQi", "NanNing", 27), ("WangBa", "YinChuan", 28), ("LiSi", "TianJin", 29), ("ZhangSan", "NanJing", 30), ("CuiShi", "ZhengZhou", 65),  ("LiSi", "KunMing", 29), ("LiSi", "ZhengZhou", 30);

Check the table records

mysql> select * from user;
+----+----------+-----------+------+
| id | name     | city      | age  |
+----+----------+-----------+------+
|  1 | ZhaoDa   | BeiJing   |   20 |
|  2 | QianEr   | ShangHai  |   21 |
|  3 | SunSan   | GuanZhou  |   22 |
|  4 | LiSi     | ShenZhen  |   24 |
|  5 | ZhouWu   | NingBo    |   25 |
|  6 | WuLiu    | HangZhou  |   26 |
|  7 | ZhengQi  | NanNing   |   27 |
|  8 | WangBa   | YinChuan  |   28 |
|  9 | LiSi     | TianJin   |   29 |
| 10 | ZhangSan | NanJing   |   30 |
| 11 | CuiShi   | ZhengZhou |   65 |
| 12 | LiSi     | KunMing   |   29 |
| 13 | LiSi     | ZhengZhou |   30 |
+----+----------+-----------+------+
13 rows in set (0.00 sec)

Be careful , A union index is created in this table (name, city), Suppose we want to query the following statement ：

select * from user where name="LiSi" and city like "%Z%" and age > 25;

3.1 Push down without index

Without index push down , According to the joint index “ The leftmost match ” principle , Only name Columns can use indexes ,city Columns are then fuzzy matched , You can't use indexes , The execution process is as follows ：

• The storage engine is based on (name, city) Joint index , find name The value is LiSi The record of , common 4 Bar record ;
• Then according to this 4 Of records id value , Scan the table one by one , Remove the complete row record from the clustered index , And return these records to Server layer ;
• Server The layer receives these records , And according to the conditions name="LiSi" and city like "%Z%" and age > 25 To filter , Finally stay ("LiSi", "ZhengZhou", 30) This record .

Draw a picture to see ：

 

3.2 Use index to push down

When using index push down , The execution process is like this ：

• The storage engine is based on (name, city) Joint index , find name='LiSi' The record of , common 4 strip ;
• Because the union index contains city Column , The storage engine presses... Directly in the federated index city like "%Z%" To filter , After filtering, there is 2 Bar record ;
• According to the filtered records id value , Scan the table one by one , Remove the complete row record from the clustered index , And return these records to Server layer ;
• Server According to WHERE Other conditions of the statement age > 25, Filter the row records again , In the end, only ("LiSi", "ZhengZhou", 30) This record .

Draw a picture to see ：

 

  in addition , You can also see from the execution plan that index push down is used （Extra Inside display Using index condition）

mysql> explain select * from user where name="LiSi" and city like "%Z%" and age > 25;
+----+-------------+-------+------------+------+---------------+---------------+---------+-------+------+----------+------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key           | key_len | ref   | rows | filtered | Extra                              |
+----+-------------+-------+------------+------+---------------+---------------+---------+-------+------+----------+------------------------------------+
|  1 | SIMPLE      | user  | NULL       | ref  | idx_name_city | idx_name_city | 99      | const |    4 |     7.69 | Using index condition; Using where |
+----+-------------+-------+------------+------+---------------+---------------+---------+-------+------+----------+------------------------------------+
1 row in set, 1 warning (0.00 sec)

 

Four 、 Conditions of use

• It can only be used for range、 ref、 eq_ref、ref_or_null Access method ;
• It can only be used for InnoDB and MyISAM Storage engine and its partition table ;
• Yes InnoDB For storage engines , Index push down is only applicable to secondary indexes （ Also called secondary index ）;

The purpose of index push down is to reduce the number of table returns , That is to reduce IO operation . about InnoDB Of Cluster index Come on , The complete row record has been loaded into the cache , There is no point in pushing down the index .

• Conditions that reference subqueries cannot be pushed down ;
• A condition that references a storage function cannot be pushed down , Because the storage engine cannot call the storage function .

5、 ... and 、 Related system parameters  

Push is enabled by default under index conditions , You can use system parameters optimizer_switch To check whether the controller is on .

View default status ：

mysql> select @@optimizer_switch\G;
*************************** 1. row ***************************
@@optimizer_switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,engine_condition_pushdown=on,index_condition_pushdown=on,mrr=on,mrr_cost_based=on,block_nested_loop=on,batched_key_access=off,materialization=on,semijoin=on,loosescan=on,firstmatch=on,duplicateweedout=on,subquery_materialization_cost_based=on,use_index_extensions=on,condition_fanout_filter=on,derived_merge=on
1 row in set (0.00 sec)

  Switch state ：

set optimizer_switch="index_condition_pushdown=off";
set optimizer_switch="index_condition_pushdown=on";