MySQL actual combat 45 lecture learning (I)

Query statement execution process （select * from T where ID=10;）

mysql The logical architecture diagram

In general ,MySQL Can be divided into Server layer and Storage engine layer Two parts , Different storage engines share one Server layer .

Server layer Including connectors 、 The query cache 、 analyzer 、 Optimizer 、 Actuators etc. , cover MySQL Of Most core service functions , And all Built in functions （ Such as date 、 Time 、 Mathematics and cryptographic functions ）, all Cross storage engine capabilities All implemented on this layer , Like stored procedures 、 trigger 、 View etc. .

and Storage engine layer be responsible for Data storage and extraction . Its architecture model is Plug in Of , Support InnoDB、MyISAM、Memory Wait for multiple storage engines . Now the most commonly used storage engine is InnoDB, It is from MySQL 5.5.5 Version began to be the default storage engine . But it can be in create table Use in statement engine=xxx To specify the engine

What follows is select * from T where ID=10; This article SQL Running process

1. The connector

The connector is responsible for establishing a connection with the client 、 Access permissions 、 Maintaining and managing connections .
Connection command ：

mysql -h$ip -P$port -u$user -p
 Such as 
mysql -hlocalhost -uroot -proot

You can just write -p And then enter the password on a new line , This is safer . If you're connected to a production server , It is strongly recommended that you do this .

In the command mysql It's a client tool , Used to establish a connection with the server .
After entering the user name and password , If there is any mistake , False report Access denied for user Otherwise, it is in the permission table Find out your authority , Your next operation is limited to this permission
however , If a user Already logged in 了 , Even if the administrator modifies his permissions , His current authority is also It won't be affected , Unless he logs in again , Establish a new connection

After the connection is completed, if you do not operate on this connection , So the connection is idle , Can be in show processlist See it in the command .

More clients than 8h No response , Will disconnect this connection , This time is determined by the parameter wait_timeout The control of the , The default is 8h
If you send the request again , You'll get an error alert ： Lost connection to MySQL server during query.

In use, we must try to use long connections , however Long connections also have drawbacks ：MySQL In the process of execution Temporarily used memory Is management in Connection object Inside . These resources will be in Connection is broken It's only released when . The accumulation of these memories may cause the process to be killed by the system , That is, abnormal restart
There are two solutions to this ：

1. Specify a maximum connection time , Connection time Beyond that time Is disconnected ; If you have just executed one Operations that take up a lot of memory , Just disconnect .
2. If you're using a MySQL 5.7 Or later , You can do this after each large operation , Through execution mysql_reset_connection Come on Reinitialize connection resources . This process does not require reconnection and re-authorization , But it will connect Restore to the state when it was just created .

2. The query cache

In most cases, the author does not recommend using query caching , Because the disadvantages outweigh the advantages
reason ： As long as you check a table to update , This watch is The query cache will be emptied , So unless it's something that won't change for a long time 、 Approximate static or static table , This query cache is recommended .
It should be noted that ,MySQL 8.0 Version of the query cache directly removed the entire block , in other words 8.0 At first, there was no such function 了 .

3. analyzer

That is, analysis SQL sentence

1. Lexical analysis ： Extract "select" This query operation keyword ,"T" This table name
2. Syntax analysis ： analysis SQL Whether it conforms to the grammar , If it doesn't meet the requirements, it will You have an error in your SQL syntax, What we should pay attention to in error checking is the following use near What follows .

4. Optimizer

The optimizer has... In the table Multiple indexes When , Decide which one to use Indexes ; Or there are multiple table associations in a statement （join） When , Determine the... Of each table Connection sequence . Sometimes several execution methods The logical result is the same Of , But the Executive Efficiency will be different , And the optimizer's role is Decide which one to use programme .

5. actuator

First judge whether the user has checked the table T For query operation jurisdiction
If you have authority , Open the table and continue , The actuator will follow the table engine Definition , Use what this engine provides Interface , Get the result set and return it to the client .

You will see a... In the slow query log of the database rows_examined Field of , Indicates that the statement is scanned during execution How many rows? . This value is every time the executor calls the engine to get the data row Add up Of , But the number of lines scanned by the engine is the same as rows_examined It's not exactly the same Of .

After class questions ：
If the table T There are no fields in k, And you execute this statement select * from T where k=1, That must be the report Unknown column ‘k’ in ‘where clause, At which stage did we report this error ？
answer ： analyzer . The analyzer will extract the input SQL Key words , And judge whether the statement is correct , Does the table exist , Whether the column exists, etc .

Update the log module during the operation （update T set c=c+1 where ID=2;）

Unlike the query process , The update process also involves two important logging modules ,redo log（ Redo log ） and binlog（ Archive log ）.
redo log yes InnoDB Engine specific logs , binlog yes server Layer log

redo log

If every update needs to be written to disk , Then the disk also needs to find the corresponding record , Then update , The whole process IO cost 、 Search costs are high .

So there is Mysql Of WAL(Write-Ahead Logging) technology , namely Write the log , Write the disk again . When a record needs When it's updated ,InnoDB The engine will First record writes redo log Inside , and Update memory , Update at this time It's done . meanwhile ,InnoDB The engine will... At the right time , Record this operation Update to disk Inside , And this update is often in the system Do it in your spare time ,

Of course ,redo log There are also size limits , When the size is insufficient, it will first stop to update some records to disk , Make room for the following operation

With redo log,InnoDB You can guarantee that even if the database happens Abnormal restart , Previously submitted records They won't lose , This ability is called crash-safe.

Comparison of two log modules

There are two reasons for the log module ：Mysql Self contained MyISAM The engine cannot crash-safe,binlog Logs can only be used for archiving , So that wrote innoDB Our company is innoDB It is written with crash-safe The ability of redo log

The difference between the two logs ：

1. redo log yes InnoDB engine Peculiar ;binlog yes MySQL Of Server layer Realized , All engines can use .

2. redo log yes Physical log , The record is “ In a certain Data pages What changes have been made to ”;`binlog1 yes Logic log , The record is this SQL sentence The original logic of , such as “ to ID=2 In this line c Field plus 1 ”.

3. redo log yes Write in cycles Of , The space will be used up ;binlog Yes. Additional writing In .“ Additional writing ” Refer to binlog file After writing to a certain size, it will switch to the next , Does not overwrite previous logs .

4. redo log（ Redo log ） Give Way InnoDB The storage engine has crash recovery capability .
   binlog（ Archive log ） To ensure the MySQL Data consistency of cluster architecture .

The actuator is in use innoDB The engine does this update The process of operation

1. The actuator first finds the data from the engine , If it is in memory, it returns directly , If it does not return after the memory query
2. After the actuator gets the data , The data will be modified first , Then the engine interface is rewritten to the data.
3. The engine updates the data to memory , Write at the same time redo log, At this time prepare state
4. The actuator generates the binlog
5. The executor calls the transaction commit interface of the engine , take redo log Change the state to commit state , Update complete

The author's diagram

Two-phase commit

Pay attention to the bottom three
Found a record to write redo log The operation of , Leng is to set the first as prepare state ,、 Set it to commit state , Can't you set it directly ？

Let's assume that we don't do two-stage submission , And cite two cases

1. First write binlog, To write redo log
   After the current transaction is committed , write in binlog success , Then the master node crashes . After the primary node is restarted , Because... Is not written redo log, Therefore, this data will not be recovered .
   The slave node is based on binlog After local playback , There will be one more piece of data relative to the primary node , Resulting in master-slave inconsistency .

2. First write redo log, To write binlog
   After the current transaction is committed , write in redo log success , Then the master node crashes . After the primary node is restarted , The master node uses redo log Resume , will Compared with one more piece of data from the node , Cause inconsistency between master and slave data .

therefore , Only once redo log And binlog, The consistency of these two logs after transaction commit cannot be guaranteed . That is, the consistency between the crash recovery of the master node and the local playback data of the slave node cannot be guaranteed .

It's not just This process is needed to recover data after misoperation . When you need Capacity expansion When , That is to say, it is necessary to build more backup databases to increase the reading ability of the system , Now common practice is to use Full backup plus Application binlog To achieve , This “ atypism ” It will lead to inconsistency between master and slave databases on your line .

redo log and binlog Can be used to represent the commit state of a transaction , And two-phase commit is to keep these two states logically consistent .
namely ：

1. redo Crash before writing , At this time binlog Not written yet , Recovery data is not affected
2. redo Well done. ,binlog Crash before writing , This is a redo be in prepare state , Not yet submitted , The transaction is rolled back upon recovery ,binlog There's no record yet , So it won't affect
3. redo Well done. ,binlog Well done. , But not yet commit He collapsed when he died , At this time, the corresponding transaction will be judged binlog Whether it exists and is complete ：
   （1） If it exists and is complete, commit the transaction , At this time, it will return to the state after the transaction is committed , because binlog There's a record in , So the recovery was successful
   （2） If binlog Nonexistent or incomplete , At this time, it will return to the state before the transaction is committed , because binlog Records that are not recorded or incomplete will not take effect , So the recovery was successful .
4. redo Already there. commit identification , Commit the transaction directly , Also because binlog Records , Then the recovered data will not be affected

redo log Used to guarantee crash-safe Ability .innodb_flush_log_at_trx_commit This parameter is set to 1 When , For each transaction redo log All persist directly to disk . This parameter is recommended to be set to 1, This ensures MySQL Data will not be lost after abnormal restart .

sync_binlog This parameter is set to 1 When , For each transaction binlog All persistent to disk . This parameter is also recommended to be set to 1, This ensures MySQL After abnormal restart binlog No loss .