MySQL combat optimization expert 02 in order to execute SQL statements, do you know what kind of architectural design MySQL uses?

1、 hold MySQL As a black box SQL sentence

We have already said , Our system uses database connection pool to access the database concurrently , Then the database itself will also maintain a connection pool , Its All connections between various systems and this database server are managed in

Let's look at the picture below to review

             

When our system can only get a database connection from the database connection pool , We can add, delete, modify and check SQL Statement

From the picture above, we can see , We can connect to the database to execute SQL The statement is sent to MySQL database .

so what ？ Most of the students stop when they know this , Then you think the main concern is the table structure in the database , What indexes have been built , however Then follow SQL Write, add, delete, and check the grammar SQL sentence , hold MySQL When a black box goes to execute SQL The sentence is enough .

We only know the execution insert After statement , There will be an extra piece of data in the table ; Yes update After statement , Changes will be made to the data in the table ; Yes delete After statement , Will delete a piece of data in the table ; Yes select After statement , Will look up some data from the table .

If the statement performance is a little bit poor ？ No problem , Just build some indexes in the table ！ Maybe this is a recognition of database by many engineers in the industry , Be completely He is a black box , To build tables and execute SQL sentence .

But since we started to learn with me , From now on, we need to break the cognitive level of treating database as a black box , To get to the bottom , To explore the work of databases Principle and optimization means of production problems ！

2、 An unchanging principle ： Network connections must be handled by threads

Now suppose that a connection in the connection pool of our database server receives a network request , Suppose it's just one SQL sentence , So let's think about a question , Who is responsible for monitoring network requests from this connection ？ Who is responsible for reading the request data from the network connection ？

I'm afraid many people haven't thought about it , But if you have a simple understanding of basic computer knowledge , You should know more or less , That's it The network connection must be allocated to a thread for processing , A thread listens for requests and reads request data , For example, read and parse a from the network connection Our system sent the past SQL sentence , As shown in the figure below ：

             

3、SQL Interface ： Be responsible for handling the received SQL sentence

Then let's think about , When MySQL The internal worker thread reads a... From a network connection SQL After statement , How to execute this SQL language Sentence ？

Actually SQL It's a great invention , He invented a simple and easy-to-use grammar and model for data reading and writing , Even a product manager , Or the operations specialist , Even sales Commissioner, , Even if he doesn't know how to , He can also easily learn to use SQL sentence .

But if you're going to do this SQL sentence , To complete the addition, deletion, modification and query of the underlying data , So this is an extremely complex task ！

therefore MySQL The interior first provides a component , Namely SQL Interface （SQL Interface）, He is a set of executors SQL Statement interface , Dedicated to the execution of our Send to MySQL Of those who added, deleted, and checked SQL sentence

therefore MySQL The worker thread of received SQL After statement , It will be transferred to SQL Interface to execute , Here's the picture .

             

4、 Query parser ： Give Way MySQL Can understand SQL sentence

Then the next question comes ,SQL How to implement the interface SQL Sentence? ？ You put SQL Give the sentence to MySQL, He can read and understand these SQL Statement ？

Let's take an example , Now we have this one SQL sentence ：

select id,name,age from users where id=1

This SQL sentence , We can deal with it directly with human brain , Just know SQL Grammatical person , Everyone knows what he means right away , however MySQL Own this The body is also a system , It's a database management system , He can't understand that directly SQL sentence ！

So there's a key component coming out at this point , That's it Query parser

This query parser （Parser） Is responsible for SQL Statement for parsing , Like the one above SQL The statement is disassembled , Break it down into the following parts ：

We are going to start from “users” Look up the data in the table Inquire about “id” The value of the field is equal to 1 That line of data We need to extract the data in that line “id,name,age” Three fields .

So-called SQL analysis , According to the established SQL grammar , Follow... To us SQL Grammar rules are written SQL Statement parsing , Then understand this SQL The sentence should be done What thing , As shown in the figure below ：

             

5、 Query optimizer ： Choose the best query path

When we understand through the parser SQL After what the statement is about , Then we'll find the query optimizer （Optimizer） To choose an optimal query path .

Maybe some students don't understand the best query path , This one really looks abstract , Of course , How the query optimizer works , The follow-up will be Is the focus of our analysis , You don't have to worry about his principles now .

But we can use a very simple example , Let's understand what the so-called optimal query path is .

Just use the example we just talked about , We now understand a SQL Want to do such a thing ： We are going to start from “users” Look up the data in the table , check Inquiry “id” The value of the field is equal to 1 That line of data , We need to extract the data in that line “id,name,age” Three fields .

It's clear , But how to achieve it in the end ？

You see , To do this, we have the following query paths （ It's just an example for everyone to understand , It doesn't mean true MySQL principle , But through this example Son , We can definitely understand the meaning of the so-called optimal query path ）：

Go straight to “users” In the table “id” Field equals 1 A row of data , And find out the line of data “id,name,age” The value of three fields is enough The first “users” Of each row of data in the table “id,name,age” The values of all three fields are found , And then filter it out of the data “id” Field equals 1 Of That line of data “id,name,age” Three fields

This is the simplest one SQL Two Implementation Paths of the statement , In fact, we will find that , To do this SQL The goal of the statement , Both paths can do , but Which one is better ？ Obviously it feels better to be the first query path .

So the query optimizer probably does this , He will write dozens of lines for you 、 The complexity of hundreds or even thousands of lines SQL Statement to generate a query tree , And then from inside Face select an optimal query path .

It's like he'll tell you , What kind of steps and order should you follow , What to do , Then step by step SQL The sentence is finished .

Let's take a look at the picture below ：           

6、 Call the storage engine interface , True execution SQL sentence

 

The last step , It is the optimal query path selected by the query optimizer , That is, what kind of order and steps should you follow to carry out this SQL Statement plan draw , Give this plan to the underlying storage engine to actually execute . The storage engine is MySQL There is a special part in the architecture design of .

I don't know if you've thought about , Really executing SQL At the time of statement , Or update the data , Otherwise, it's data query , So where do you think the data is stored ？

To put it bluntly , Database is not something mysterious , You can think of him as a library management system similar to what you usually write 、 Telecommunication billing department system 、 E-commerce order system and so on .

Database itself is a system written by programming language , And then it's a process after it's started , Execute all kinds of code in him , That's what we said above All those things that you're looking for . So for databases , Our data would otherwise be in memory , Or put it in the disk file , Nothing special ！

So let's think about , Suppose that some of our data is stored in memory , Some are stored in disk files , As shown in the figure below .

             

So here's the problem , We already know one SQL How to execute the statement , But now how do we know what data is in memory ？ What data is on disk in ？ When we execute, we update the data in memory ？ Or update the disk data ？ If we update the disk data , Which disk file to query first , Where to update Disk files ？

Do you feel confused

So we need a storage engine at this time , The storage engine is actually execution SQL Of the statement , He will follow certain steps to query the memory cache data , Number of disks updated According to the , Query disk data , wait , Perform a series of operations like this , As shown in the figure below .

             

MySQL In the architectural design of ,SQL Interface 、SQL Parser 、 Query optimizers are all universal , He's just a set of components .

But for storage engines , He supports all kinds of storage engines , Like the ones that we see all the time InnoDB、MyISAM、Memory wait , We can choose Which storage engine is used to be responsible for specific SQL Statement executed .

Now, of course, MySQL It's generally used InnoDB Storage engine , As for the principle of the storage engine , We will analyze it step by step later , You don't have to worry .

7、 actuator ： Call the interface of the storage engine according to the execution plan

So after looking at the storage engine , Let's turn around and think about a question , The storage engine can help us access the data in memory and on disk , Then who calls The interface of the storage engine ？

In fact, we still miss the concept of an actuator , This executor will execute according to the scheme chosen by the optimizer , Call the interface of the storage engine in a certain order step , Just put SQL The logic of the statement is executed .

for instance , For example, the executor may call an interface of the storage engine first , To get “users” The first row of data in the table , Then judge the “id” Is the value of the field equal to a value we expect , If not , Then continue to call the interface of the storage engine , To get “users” The next row of the table According to the .

Based on the above ideas , The executor will generate a set of execution plans based on our optimizer , And then keep calling various interfaces of the storage engine to complete SQL Statement execution plan , It's about constantly updating or extracting some data

Let's see the diagram below

                                                                                                                   

8、 Small thinking questions ： Open the brain hole , What do you think different storage engines are used for ？

Today, I'd like to leave you with a small question , Just leave it alone MySQL What storage engines are there , You start from the business scenario , Some scenarios may be highly concurrent Update , Some scenarios may be large-scale data query , Some scenarios may allow data loss

So you think if you design the storage engine , What storage engines do you think there should be , What scenarios do they apply to ？

I hope you can leave a message in the comment area , Think and reply enthusiastically .