MySQL Execution process and sequence

When we initiate a sql To show detailed query data , What kind of process did it go through ？MySQL How does the server handle requests , How to execute sql Of the statement ？ This blog will explore this issue in the future ：

MySQL Architecture

MySQL Architecture

adjoining course

• At the top are some clients and link services , Mainly complete some similar to Connection processing 、 Authorized certification 、 And related safety programme . The server will also verify the operation permissions it has for each client of secure access .

Service layer

• The second tier architecture mainly completes most Core service function , Such as SQL Interface , And complete the cache query ,SQL Analysis and optimization of , Execution of some built-in functions . All the cross storage engine functions are also implemented in this layer , Like the process 、 Functions, etc .

Engine layer

• The storage engine is really responsible for MySQL The storage and extraction of data in , Server pass API Communicating with the storage engine . Different storage engines have different functions , So that we can according to our own needs , To choose the right storage engine .

Storage layer

• Mainly is to Data is stored on the file system , And complete the interaction with the storage engine .

1. MySQL Execution process

MySQL The overall implementation process is shown in the figure below ：

1.1. The connector

The main responsibilities of the connector :

1. Responsible for communication with clients , It's half duplex mode , This means that the client can only request from the server or the server can send data to the client at a fixed time , Not at the same time , among MySQL In connection with the client TC/IP Of
2. Verify that the requested user's account and password are correct , If the account and password are wrong , Will report a mistake : Access denied for user 'root'@'localhost' (using password: YES)
3. If the user's account and password are verified , Will be in MySQL Query the permissions of the current user in the built-in permission table :

MySQL in 4 Tables with control rights , Respectively user surface ,db surface ,tables_priv surface ,columns_priv surface MySQL The validation process of permission table is ：

1. User surface ： Deposit User account information and global level （ All databases ） jurisdiction , Determines which users from which hosts can access the database instance
   Db surface ： Deposit Database level Authority , Determines which users from which hosts can access this database
   Tables_priv surface ： Store table level permissions , This table determines which users from which hosts can access the database
   Columns_priv surface ： Store column level permissions , This field determines which users from which hosts can access the database table
   Procs_priv surface ： Store stored procedures and functions Level permissions
2. First from user In the table Host,User,Password this 3 Determine connected ip、 user name 、 Whether the password exists , Verify if it exists .
3. After identity authentication , Assign permissions , according to user,db,tables_priv,columns_priv Verify in order of . Check the global permission table first user, If user The corresponding permissions in are Y, Then this user's permission to all databases is Y, Will no longer check db, tables_priv,columns_priv; If N, Then to db Check the specific database corresponding to this user in table , And get the db In Chinese, it means Y Authority ; If db In Chinese, it means N, Then check tables_priv The specific table corresponding to this database in , Get permissions in table Y, And so on
4. If the authority verification fails in any process , All will report wrong.

1.2. cache

MySQL The main function of cache is to improve Query efficiency , Cache with key and value In the form of a hash table ,key It is concrete. sql sentence ,value It's a collection of results .

If you can't hit the cache , Go on to the analyzer , If the hit cache is returned directly to the client . But here's the thing stay MySQL Of 8.0 After the version , The cache has been officially deleted . Why delete , It's because query cache failures are very frequent , If in an environment of more writing and less reading , Cache will be added and invalidated frequently . For some databases that are under pressure to update , The hit rate for the query cache will be very low ,MySQL In order to maintain the cache, there may be some scalability problems , Currently in 5.6 Has been turned off by default in the version of , One of the recommended practices is to put the cache on the client side , Performance will probably improve 5 About times

1.3. analyzer

The main function of the analyzer is to send the client sql Statement analysis , This will include Preprocessing and parsing process , At this stage it will parse sql The semantics of the statement , And extract keywords and non keywords 、 analysis , And make up a parse tree .

Specific keywords include not limited to ：select/update/delete/or/in/where/group by/having/count/limit etc. . If we analyze grammatical errors , It will throw an exception directly to the client :ERROR:You have an error in your SQL syntax.

such as ：select * from user where userId =1234;

In the parser, we use the semantic rules to select from where These keywords are extracted and matched ,MySQL Will automatically determine keywords and non keywords , Identify the user's matching fields and custom statements . There will also be some checks at this stage : For example, check whether the current database exists user surface , At the same time, if User There is no userId This field will also report an error ：unknown column in field list.

1.4. Optimizer

Be able to enter the optimizer stage to express sql Is in accordance with MySQL Of the standard semantic rules and can be implemented , This stage is mainly for sql Statement optimization , Make the best choice according to the execution plan , Match the appropriate index , Choose the best execution plan . For example, a typical example is ：

surface T, Yes A、B、C Columns are federated , During the query , When sql The result of the query is :select xx where B=x and A=x and C=x. A lot of people think it's useless to use indexes , But it actually uses , though However, the index must conform to the leftmost principle before it can be used , But in essence , The optimizer will automatically send this sql Optimize to :where A=x and B=x and C=X, This optimization will be for the underlying to match the index , At the same time, in this stage, preprocessing is performed automatically according to the execution plan ,MySQL Accounting calculates the best time for each method of execution , To finalize an executive sql To the last actuator

1.5. actuator

In the stage of the actuator , At this point, the storage engine's API,API Will call the storage engine , There is mainly a storage engine , But the usual one is myisam and innodb:

The engine used to be called :** Watch processor ( In fact, I think this name can better express the meaning of its existence )** Responsible for the operation of specific data files , Yes sql For example select perhaps update Analyze , Perform specific operations . After execution, the specific operation will be recorded to binlog in , One thing to note is that :select It will not be recorded binlog in , Only update/delete/insert It's recorded that binlog in . and update It will be submitted in two stages , All records redolog in

InnoDB

---

* Introduce
InnoDB It is a general storage engine with high reliability and high performance , stay MySQL 5.5 after ,InnoDB By default MySQL Storage engine .
* characteristic
DML Operation follows ACID Model , Support ;
, Improve concurrent access performance ;
Support FOREIGN KEY constraint , Ensure the integrity and correctness of data ;
* file
xxx.ibd：xxx Represents the table name ,innoDB Each table of the engine will correspond to such a tablespace file , The table structure that stores the table （frm、sdi）、 Data and index .
Parameters ：innodb_file_per_table

MyISAM

* Introduce
MyISAM yes MySQL Early default storage engine .
* characteristic
Unsupported transaction , Foreign key not supported
Support table lock , Row locks are not supported
Fast access
* file
xxx.sdi： Store table structure information
xxx.MYD: Store the data
xxx.MYI: Storage index

Memory

* Introduce
Memory The table data of the engine is stored in memory , Due to hardware problems 、 Or the impact of power failure , These tables can only be used as temporary tables or caches .
* characteristic
Memory storage
hash Indexes （ Default ）
* file
xxx.sdi： Store table structure information

2. Status of execution

By command :show full processlist, Show all the processes , It mainly includes the following states , Indicates the state of the server processing the client , State includes the process from the client to the background server , Including the process of locking 、 Statistics storage engine information , Sorting data 、 Search the middle table 、 Send data, etc .

It's all about MySQL All states , The specific meaning is shown in the figure below ：

3. sql Execution order of

in fact ,sql It's not in the order we write from front to back 、 From left to right , It's parsed in a fixed order , The main function is to provide the results from the execution of the previous stage to the next stage ,sql There will be different temporary intermediate tables during execution , Generally in the following order ：

Example : select distinct s.id from T t join S s on t.id=s.id where t.name="Yrion" group by t.mobile having count(*)>2 order by s.create_time limit 5;

3.1. from

The first step is to choose from The table following the key words , This is also sql The first step of execution : Indicates which table to execute from the database .

Example is given to illustrate : In this case, first find the table from the database T

3.2. join on

join Is the table to be associated ,on It's the connection condition . adopt from and join on Select the database table to execute T and S, Produce Cartesian product , Generate T and S The merged interim table Temp1.on: Determine the binding relationship of the table , adopt on Create a temporary middle table Temp2.

Example is given to illustrate ： Find the watch S, Generate temporary middle table Temp1, then Find the watch T Of id and S Of id The same parts make up the table Temp2,Temp2 It contains T and Sid All equal data

3.3. where

where Means screening , according to where The following conditions are filtered , According to the value of the specified field ( If there is and The connector will be federated ) From temporary middle table Temp2 To filter the required data , Note if data is not found at this stage , Will return to the client directly , Not going down . This process will generate a temporary middle table Temp3. Pay attention to where You cannot use aggregate functions in , The aggregate function is mainly (min\max\count\sum Such as function )

Example is given to illustrate : stay temp2 Found... In temporary table set T Tabular name="Yrion" The data of , After finding the data, it will be a temporary middle table Temp3,temp3 It contains name As a "Yrion" All table data for

3.4. group by

group by It's grouping , Yes where Condition filtered temporary table Temp3 Group by fixed fields , Create a temporary middle table Temp4, This process is just a change in the order of the data , And the amount of data doesn't change , The data in the table exists as a group

Example is given to illustrate : stay temp3 In the table data mobile Grouping , Find out mobile The same data , And put it together , produce temp4 A temporary table .

3.5. Having

For the temporary middle watch Temp4 Aggregate , This can be count Wait for the count , And then the middle table Temp5, You can use... At this stage select Alias in

Example is given to illustrate : stay temp4 The number of items found in the temporary table is greater than 2 The data of , If it is less than 2 To be abandoned directly , And then you're born Form a temporary intermediate table temp5

3.6. select

Select the data to be queried from the grouped and aggregated tables , If * It will be resolved to all data , The middle table will be generated Temp6

Example is given to illustrate ： This stage is Yes temp5 In the temporary aggregation table S In the table id To produce Temp6, here temp6 It only includes s Tabular id Column data , also name=“Yrion”, adopt mobile The number of groups is greater than 2 The data of

3.7. Distinct

distinct De duplicate all data , If there is min、max The function performs field function calculations , And then a temporary watch Temp7

Example is given to illustrate : This stage is right for temp5 The data in is de duplicated , engine API Will call the de duplication function to filter the data , In the end, just keep id The first piece of data , then Create a temporary middle table temp7

3.8. order by

Will be based on Temp7 To arrange in order or reverse order , then Insert temporary intermediate table Temp8, This process is more resource consuming

Example is given to illustrate ： This paragraph will bring all temp7 The data in the temporary table is based on the creation time (create_time) Sort , There will be no column or row loss in this process

3.9. limit

limit On the middle watch Temp8 paging , Create a temporary middle table Temp9, Return to the client .

Example is given to illustrate : stay temp7 Ordered data in , Then insert the first five into Temp9 In this temporary table , Finally back to the client

ps: In fact, this process is not absolutely so , middle MySQL There will be some optimizations to achieve the best results , For example select Filter out the data set found

4. summary

This blog summarizes MySQL Implementation process of , as well as sql Execution order of , It helps us to understand these sql Statement optimization , And understand MySQL Medium sql The trace of the statement from writing to final execution , It helps us to sql Have a more in-depth and detailed understanding of , Improve our database understanding . meanwhile , For complexity sql Implementation process of 、 Writing will have a certain degree of significance .