Detailed index of MySQL

MySQL Indexes

1. What is the index ?

• Index is help MySQL Efficient access to data data structure .
• Indexes are often stored in files on disk
• The index includes : Clustered index , Overlay index , Composite index , Prefix index , Unique index, etc , The default is to use B+ The tree structure organizes the index

2. Advantages and disadvantages of index ?

• advantage ：
  • retrieval : Can improve the efficiency of data retrieval , Reduce the IO cost
  • Sort : Sort data through index columns , To reduce the CPU Consumption of
• Inferiority :
  • Take up disk space
  • Reduce the efficiency of updating tables

3. Classification of indexes

3.1 Single index

• General index ： No restrictions .add index
• unique index : Values in index columns must be unique , Allow null value .add unique index
• primary key : Special unique index , Null value... Is not allowed .PK

3.2 Composite index

• An index created on a pair of field combinations in a table add index(col1, col2……)
• Follow the leftmost prefix principle ( Left most matching principle )

3.3 Full-text index （MyISAM,InnoDB5.6 in the future )

• Only in CHAR,VARCHAR,TEXT Full-text indexes are used on type fields .fulltext
• The highest priority , Execute first
• Storage index , Decide to execute an index

3.4 Spatial index

4. Use of index

4.1 Index related statements

The primary key index does not need to be created , The system will automatically generate .

4.1.1 General index of single column index

create index index_name on table(column(length));
alter table table_name add index index_name(column(length));

4.1.2 Unique index of single column index

create unique index index_name on table (column(lnegth));
alter table table_name add unique index index_name(column);

4.1.3 Full text index of single column index

create fulltext index index_name on table(column(length));
alter table table_name add fulltext index_name(column);

4.1.4 Composite index

alter table table_name add index index_name_(title(50),time(10),......);

4.1.5 Delete index

drop index index_name on table;

4.1.6 Look at the index

show index from table_name

5. Index principle analysis

5.1 Storage structure of index

• Indexing is implemented in the storage engine ( Different engines will only use different indexes )
• MyISAM and InnoDB Storage engine : Only support B+tree Indexes
• MEMORY/HEAP Storage engine : Support HSAH and BTREE Indexes

MyISAM Non clustered index is adopted ,InnoDB Clustering index is adopted

5.1.1 B Trees

• B The height of the tree is generally 2-4, The height of the tree directly affects IO The number of times you read and write
• Three layer tree structure ---- The supporting data can reach 20G, If it is a four layer tree structure ---- The supporting data can reach dozens T

5.1.2 B Trees and B+ The difference between trees

• B Trees and B+ The biggest difference between trees is Whether a non leaf node stores data The problem of .

because B Other child nodes of the tree also store data data, So it takes up a considerable part of memory in each page , and B+ The tree has only primary key indexes , No data data Domain , each page ((4kB,8KB,16KB) There are relatively many stored primary key indexes .

5.2 Clustered index (InnoDB)

• primary key ( Clustered index ) The leaf node will store data rows , in other words Data and index together
• The secondary index only stores the primary key value

5.3 Nonclustered indexes (MyISAM)

• B+ Tree leaf nodes only store data rows （ Data files ） The pointer to , In a nutshell Data and index are not together
• Nonclustered indexes contain primary key and Secondary index To store the value of the pointer

5.4 primary key Primary key

• InnoDB Table must have primary key (MyISAM There can be no ), without ,MySQL The system will automatically select a column that uniquely identifies the data record as the primary key

• MyISAM The index file of (mdi） Only the address where the data record is kept

• MyISAM Data files for (ibd) The record corresponding to the record in

5.5 Secondary index Secondary key( Secondary index )

• There is no difference between structure and primary key search index
• The same with B+Tree,data The domain stores the value of the corresponding record primary key instead of the address

Clustered index is very efficient through primary key search , But secondary index search needs to retrieve the indexes on both sides ： First retrieve the secondary index to get the primary key , Then use the primary key to retrieve the record in the primary index .

6. example

Understand the basic concept of index , Here is an example to illustrate :

6.1 Framework experimental environment , Create table

--  Delete t1 surface 
DROP TABLE t1;

--  establish t1 surface 
CREATE TABLE t1(
	a INT PRIMARY KEY,
	b INT,
	c INT,
	d INT,
	e VARCHAR(20)
)ENGINE = INNODB;

By creating a table , We need to understand ,MySQL When creating tables, the database will automatically add ( Primary key ) Indexes , If there is no primary key , System comes with rows Build index .

Pay attention to Oracle It's in the database rownum keyword , Query the serial number automatically generated by the system

select rownum from table_name;

6.2 Add test data :

--  towards t1 Add data to table 
insert into t1 values(4,3,1,1,'d');
INSERT INTO t1 VALUES(1,1,1,1,'a');
INSERT INTO t1 VALUES(8,8,8,8,'h');
INSERT INTO t1 VALUES(2,2,2,2,'b');

INSERT INTO t1 VALUES(5,2,3,5,'e');
INSERT INTO t1 VALUES(3,3,2,2,'c');
INSERT INTO t1 VALUES(7,4,5,5,'g');
INSERT INTO t1 VALUES(6,6,4,4,'f');

6.3 Data stored procedures

Note that when adding data, the system will automatically sort when storing

When a page cannot be stored , Stored on the next page

You can insert data by constantly select Query observation details

--  Delete all data in the table 
delete  from t1;

--  Delete the specified id The data of 
delete from t1 where a in(1,4,8);

Stored procedures are slowly generated B+ Trees ( The data is in the leaf node )

adopt explain Check out some sql Simply query the performance of the statement

--  adopt primary Index query 
EXPLAIN SELECT * FROM t1 WHERE a > 1;

--  A full table query all
EXPLAIN SELECT * FROM t1 WHERE  b = 1 AND c = 1 AND d = 1 ;

--  Direct primary key index query , High performance 
EXPLAIN SELECT * FROM t1 WHERE a = 8;

--  By looking at the index , We found that only primary key queries 
SHOW INDEX FROM t1;

So we create other indexes , Meet the requirements of fast query .

--  establish bcd Composite index 
CREATE INDEX idex_t1_bcd ON t1(b,c,d);

After creating the composite index , Execute the following statement ： understand key_len

--  By combining indexes index_t1_bcd Inquire about , But because of * So you need to go back to the table to query using where
-- key_len Is the actual index size used , Here is 15
EXPLAIN SELECT * FROM t1 WHERE  b = 1 AND c = 1 AND d = 1 ;

-- key_len Here is 5
EXPLAIN SELECT * FROM t1 WHERE  b = 1;

-- key_len Here is 10,e be not in index_t1_bcd Columns in the composite index , Don't count 
EXPLAIN SELECT * FROM t1 WHERE  b = 1 AND c = 1 AND e = 1 ;

After practicing the above cases , See the picture below , You will understand that there are so many principles and keywords in the index

Leftmost prefix principle

Use it Composite index when , We must follow the leftmost principle ( That is, the leftmost column must appear in where The first after )

--  A full table query all
EXPLAIN SELECT * FROM t1 WHERE   c = 1 AND d = 1 ;

--  Although it USES the index_t1_bcd Index but because c > 1 Is the scope of , therefore key_len Here is 10
EXPLAIN SELECT * FROM t1 WHERE  b = 1 AND c > 1 AND d = 1 ;

Understand the effect of the above statement after execution , There are also special cases of using composite indexes ：

-- possible_keys Show possible use here index_t1_bcd Indexes 
--  But because of b > 1 Is the scope of ,type yes all, The system automatically selects full table query 
EXPLAIN SELECT * FROM t1 WHERE   b > 1;

--  By combining indexes index_t1_bcd Inquire about ,key_len Here is 5
--  Someone said b > 6 Isn't it the scope ？ How can I use combination ？ Don't you follow the leftmost principle ？
--  This is because b > 6 There is only one record , So the system will choose the best query path 
EXPLAIN SELECT * FROM t1 WHERE   b > 6;

Index overlay

The use of index coverage can reduce the number of tree searches , avoid Back to the table

--  By combining indexes index_t1_bcd Inquire about ,key_len Here is 5
--  Someone has another question ,b > 1 Isn't it the scope ？ How can there be a combined index ？
--  This is because only b Therefore, it is faster to query through combination .
EXPLAIN SELECT b FROM t1 WHERE   b > 1;

--  In the composite index id The storage , So there is no need to query back to the table , This is index coverage 
EXPLAIN SELECT a,b FROM t1 WHERE   b > 1;

Index push down

MySQL5.6 appear , It is mainly used for Non primary key index Especially the joint index ( Composite index ), Reduce Number of queries back to the table

--  Before pushing down without index , There are several entries in the composite index b = 1 The record of ( Ignore d), It will return to the table for query (d) How many times 
--  For this sql sentence b = 3 There are two ((4,3,1,1,'d'),(3,3,2,2,'c')) The query process needs to return to the table twice 
--  And after the index is pushed down and optimized , Will not be ignored in the composite index d This field , Then judge the qualified , So you only need to query back to the table once .
--  The index is pushed down in Extra Will show Using index condition
--  Although you need to query back to the table , But after the index is pushed down , Satisfy b = 3 and d = 1 The only record is 1 strip 
--  therefore row In this case, it should show 1
EXPLAIN SELECT * FROM t1 WHERE  b = 3 AND d = 1 ;

--  No, d Field , So the need will meet b = 3 All records of are returned to the table for query ,row Show 2
EXPLAIN SELECT * FROM t1 WHERE  b = 3 AND e = 1 ;

All of these are int Index of type , Here you go varchar Type create index

CREATE INDEX idx_t1_e ON t1(e);

The implementation is as follows sql sentence : Data type conversion

--  The return result is 1
SELECT 1 = 1;

--  The return result is 0
SELECT 1 = 0;

--  The return result is 1
SELECT 1 = '1';

EXPLAIN SELECT * FROM t1 WHERE a = 1;

EXPLAIN SELECT * FROM t1 WHERE a = '1';

EXPLAIN SELECT * FROM t1 WHERE e = 1;

EXPLAIN SELECT * FROM t1 WHERE e = '1';

7. Database file storage

--  View storage engine 
SHOW ENGINES;

7.1 Physical file ( The order IO)

• Log files

MySQL Log database operation information and error information .

--  View the log usage information in the current database 
SHOW VARIABLES LIKE 'log_%';

• Error log (errorlog)

Default on . All serious errors encountered in the operation and MySQL Details of each startup and shutdown

Error log name ;hostname.err

View the file path of the log ：log_error

log_warings： Whether the warning message is turned on

• Binary log (binlog)
• General query log (general querylog)
• Slow query log (slow query)
• Redo log (redo log)
• Rollback log (undo log)
• relay logs (relay log)

7.2 Data files ( Random IO)

• see MySQL Data files :

show variables like '%datadir%';

ibd by InnoDB Storage ,.MYD,.MYI by MyISAM Storage

InnoDB Data files

• .frm file : It mainly stores data information related to tables , It mainly includes Table structure The definition information of
• .ibd: Use exclusive Table spaces store table data and index information , One table corresponds to one ibd file
• ibdata file : Use share Table spaces store table data and index information , All tables use one or more ibdata file

MyISAM Data files ;

• .frm file : It mainly stores data information related to tables , It mainly includes Table structure The definition information of
• .myd file : It is mainly used to store tables data Information
• .myi file : It is mainly used to store any... In the table data file Indexes The data tree of

--  Check the total memory size 
SHOW GLOBAL STATUS LIKE 'innodb_buffer_pool_pages_%';

Binary tree , Red and black trees ,B-tree,B+tree

• hash surface ( Hash function )
• When the primary key is self incremented , Binary tree has no effect , Only save to the right , The depth of the tree is too deep , Not suitable for index structure
• Red and black trees , Solved the problem of unilateral growth , But when there's too much data , The depth of the tree will also be very deep
• B Tree is a balanced lookup tree designed for external storage devices such as disks
  • B-tree
  • B+tree Non leaf node does not store data , Store index only

According to information , It mainly includes Table structure The definition information of

• .myd file : It is mainly used to store tables data Information
• .myi file : It is mainly used to store any... In the table data file Indexes The data tree of

--  Check the total memory size 
SHOW GLOBAL STATUS LIKE 'innodb_buffer_pool_pages_%';

keyword :

• OLTP ( On line transaction processing )
• OLAP ( On line analytical processing )

All the above are sorted out in the learning materials during the learning process , If there is any infringement , Contact deletion .