MySQL advanced - Architecture

MySQL Architecture diagram

Connect 、 service 、 engine 、 Storage

Introduction to hierarchy

1. adjoining course ： yes MySQL Components that provide access to external clients , The server application uses its own API Connect to the database .
2. Service layer ： After the client receives the request , It must have something to do , The service layer provides service support for clients .
3. Storage engine layer ： Responsible for working with MySQL Data storage and extraction in , Interact with the underlying system files .
4. System file layer ： It is mainly responsible for the storage of database data and logs on system files , And complete the interaction with the storage engine , It's the physical storage layer of the file .

Service layer module division

It contains the right SQL The whole process of statement processing .

Manage services and tools

• In charge of data backup and recovery 、 Copy .

Connection pool

• Manage connections and Permission authentication Of .
• management Buffer user connections , Thread processing requires caching Of .
• be responsible for Listening about MySQL Server All kinds of requests .
• The maximum number of connections available show variables like max_connections Check it out. , The general maximum default connection amount is 151.

sql Interface

• Responsible for receiving user's SQL command , And data management language and data definition language , stored procedure , View , Trigger, etc SQL, And return the query result that the user needs .
• such as select from It's called SQL interface.

Parser

• Generate a complete and legal SQL.
• analysis SQL command ,SQL When the command is sent to the parser, it will be verified by the parser .
• Verification is to One SQL sentence Progressive language Analysis and decomposition of semantic grammar Become the so-called data structure , according to Different operation types are classified , To make a Targeted forwarding .
• If Analysis error , That explains. At present SQL Illegal statement .

Optimize the controller

• SQL Statement before the query , Will use the query optimizer to optimize the query .

Cache and buffer pool

• Submit the client to MySQL Of select request Of Result set cache Into memory .
• When inquiring If cache hits , Just get the query results directly from the cache .
• The hit rate is too low , requirement SQL It has to be exactly the same ,MySQL5.7 It's been shut down ,8.0 Then it was completely taken off the shelf .

MySQL stay Linux Basic command and file path under

Contains the startup of the service , stop it , Status view , Basic commands such as viewing database contents .

File path

Because it is installed with pagoda panel , So there may be something different .

route	describe
/www/server/data	mysql The storage path of the database
/etc/my.cnf	Configuration file directory location
/www/server/mysql/bin	Related command directory
/etc/init.d	Service startup related

Linux Some related instructions

#  see Linux About mysql Group 
[root@localhost lib]# cat /etc/group|grep mysql
mysql:x:1002:

#  see Linux About mysql Users of the group 
[root@localhost lib]# cat /etc/passwd|grep mysql
mysql:x:1002:1002::/home/mysql:/sbin/nologin

Check to see if there is an installation MySQL service

#  Check to see if there is an installation MySQL service 
[root@localhost lib]# rpm -qa|grep -i mysql

#  see mysql The installation directory 
[root@localhost lib]# ps -ef|grep mysql

Stop and start of service

#  Service startup 
service mysqld start

#  Service stopped 
service mysqld stop

#  View service status 
service mysqld status

#  Set up power on auto start service 
chkconfig mysqlid on
#  View the setting status ,2、3、4 All for on Then start the machine to start automatically 
chkconfig --list | grep mysqlid
# centos7 Use it next 
systemctl enable mysqld.service

lookup MySQL The configuration file

#  Installed with pagoda panel mysql Configuration file in  /etc/my.cnf
[root@localhost /]# find / -name my.cnf
/etc/my.cnf
/www/server/mysql/mysql-test/suite/federated/my.cnf
/www/server/mysql/mysql-test/suite/group_replication/my.cnf
/www/server/mysql/mysql-test/suite/ndb/my.cnf
/www/server/mysql/mysql-test/suite/ndb_big/my.cnf
/www/server/mysql/mysql-test/suite/ndb_binlog/my.cnf
/www/server/mysql/mysql-test/suite/ndb_ddl/my.cnf
/www/server/mysql/mysql-test/suite/ndb_memcache/my.cnf
/www/server/mysql/mysql-test/suite/ndb_rpl/my.cnf
/www/server/mysql/mysql-test/suite/ndb_team/my.cnf
/www/server/mysql/mysql-test/suite/ndbcluster/my.cnf
/www/server/mysql/mysql-test/suite/rpl/extension/bhs/my.cnf
/www/server/mysql/mysql-test/suite/rpl/my.cnf
/www/server/mysql/mysql-test/suite/rpl_ndb/my.cnf

Modify character set

It is recommended that the installation is completed mysql Then set

#  View character set 
show variables like 'character%';
show variables like '%char%';

#  Locate the specified configuration file and open 
[root@localhost ~]# cd /etc
[root@localhost etc]# ls -i my.cnf
19620831 my.cnf
[root@localhost etc]# vim my.cnf

  1 [client]
  2 #password = your_password
  3 port            = 3306
  4 socket          = /tmp/mysql.sock
  5 default-character-set=utf8 #  Newly added character set settings 

  7 [mysqld]        
  8 port            = 3306
  9 character_set_server=utf8  #  Newly added character set settings 
 10 character_set_client=utf8  #  Newly added character set settings 
 11 collation-server=utf8_general_ci  #  Newly added character set settings 
 12 socket          = /tmp/mysql.sock
 13 datadir = /www/server/data
 14 default_storage_engine = InnoDB
 15 performance_schema_max_table_instances = 400
 16 table_definition_cache = 400
 17 skip-external-locking
 18 key_buffer_size = 32M
 19 max_allowed_packet = 100G
 20 table_open_cache = 128
 21 sort_buffer_size = 768K #  Sort buffer pool size , Statistical sorting of long difficult sentences , It should be properly enlarged here 
 22 net_buffer_length = 4K 
 23 read_buffer_size = 768K 
 24 read_rnd_buffer_size = 256K
 25 myisam_sort_buffer_size = 8M
 26 thread_cache_size = 16 #  Thread buffer pool size 
 27 query_cache_size = 16M #  Query buffer pool size 
 28 tmp_table_size = 32M
 29 sql-mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
 ...
 64 [mysql]
 65 no-auto-rehash #  Whether to enable automatic completion 
 66 default-character-set=utf8 #  Newly added character set settings 
 ...

View the local files of the current database

[root@localhost data]# cd db01/
[root@localhost db01]# ll
total 112
-rw-r-----. 1 mysql mysql 67 Nov 25 16:27 db.opt
-rw-r-----. 1 mysql mysql 8586 Nov 25 16:28 user01.frm #  Table structure file , Used for database migration 
-rw-r-----. 1 mysql mysql 98304 Nov 25 16:29 user01.ibd # 5.7 Then default InnoDB

A query procedure for a statement

For reference sql sentence :

SELECT customer_id,first_name,last_name FROM customer WHERE customer_id = 14;

Flow chart

1. The client looks like MySQL The server sends a query request .
2. The server first checks the query cache , If Hit the cache and immediately return the results in the cache , Otherwise move on to the next stage .
3. Server run SQL analysis , Preprocessing , Then the optimizer generates the corresponding execution plan .
4. MySQL According to the execution plan , To invoke the storage engine API To query .
5. Returns the result to the client , Also cache query results .

Establishing a connection

Carry the correct parameters through the client （ host , user name , Password and other information ） Establish a connection to the database server , Connection layer of database Always will In a listening state , If after the connection is established , First Determine whether the query statement satisfies the data in the cache , If yes, directly return the cached data to the client , Otherwise, take the next step .

Query cache stage

For our MySQL The cache will be stored in a reference table , You can think of it as one HashMap Structure , By calculating the version number ,sql Statement content and so on , That is to say, one more space is not enough , Therefore, the cache hit probability is extremely low .

Parser phase

If you do not hit the cache, you will enter the parser stage , The main stage is analysis SQL command .SQL It is passed in the form of string , We have to parse it into a database 、 legal SQL sentence .

The parsing process mainly includes the following five stages ：

• Lexical analysis ：

  • according to , Split words , Take one SQL Split into parts , Take the one above SQL The statement is ：

  • SELECT,customer_id,first_name,last_name,FROM,customer,WHERE,customer_id,=,14
    

• Syntax analysis ：

  • Conduct SQL Identification of , Complete SQL The parse tree ：
  • according to Lexical analysis Got SQL The classification of each part , Divide words with the same characteristics into the same category .

• The parse tree ：

  

• The preprocessor ：

  • To further check whether the parse tree is legal , For example, it is Go to check whether the table exists , Does the column exist .

• New parse tree ：

  • adopt The new parse tree generated after the preprocessor checks , The new parse tree may have the same structure as the old parse tree .

• Query optimizer ：

  • Responsible for obtaining What is the generation strategy of the optimal execution plan .
  • SQL Statement before execution , That is, after generating a new parse tree , according to Tree hierarchy analytically .
    • Usually Redefine table associations , Similar to instruction rearrangement .
    • SQL Queries usually select an index to execute .
    • Terminate query ahead of time .

** After performing all the above steps, we will get a Relatively complete and efficient execution plan .** Then we give it to our storage engine to interact with the underlying files .

Storage engine

Mainly responsible for and The underlying file layer interacts thus == Complete the real storage and reading of data at the physical level ==.

The storage engine used by a table determines the storage structure of the data in the table , It is For table and Not for the library Of , Different tables in a library can use different storage engines .

Plug in storage engine architecture Separate query processing from other system tasks and data storage and extraction .

#  Query the storage engine of the current database 
show variables like '%storage_engine%';

classification

Storage engines can be roughly divided into the following categories ：

InnoDB（ a key ）

• 5.5 After the version MySQL database The default storage engine .
• Support transactions , Row level locking , But the processing speed is relatively low .

characteristic

• Support transactions .
• Table space ： Big .
• MVCC Mechanism ： Read without lock , Reading and writing can be concurrent , Writes do not block reads .
  • Through this mechanism, we can solve It can't be read repeatedly The problem of , Lock solution The problem of unreal reading .
• concurrency ： Row-level locks , Table lock , Smaller particle size , Suitable for high concurrency
• Unique index structure .
• cache ： Both index and real data should be cached , Yes High memory requirements , and == Memory size has a decisive impact on performance ==.

Applicable scenario

• need Transaction support also High concurrency 、 Data updates are frequent Application scenarios of .

MyISAM

• 5.5 Before the release MySQL database The default storage engine .
• High insertion and deletion speed , However, row locks are not supported , Business , Foreign keys .
• Table data consists of MYD and MYI Two documents .
• For concurrency, it is a table lock .
• Unsupported transaction .
• Engine mainly used for temporary tables .

Applicable scenario

1. Non transactional , Read more, write less, or read-only applications , Because transactions are not supported .

CSV

• Data files exchanged across platforms are generally stored in it .

characteristic

• Use ordinary csv File storage data .
• Index not supported .
• Column cannot be null！

Applicable scenario

Suitable as an intermediate table , Data exchange .

• Excel Files can be directly converted to csv file , Can be directly in MySQL Server side Use in .
• Data from other storage engines , Sure Transfer to csv In the table of the storage engine , The user can then use the file directly .

Memory

• Memory storage engine , It has a very high speed of adding, deleting, modifying and checking .
• But the data it stores is volatile .

characteristic

• The data is stored in In the memory , No persistence , But the query speed is very fast .
• Support index type ：HASH and BTREE Indexes .

Applicable scenario

• Not recommended , Data is volatile , stay MySQL The table will be cleared after restart .
• Data stored in memory , Limited by memory size .

Code cloud warehouse synchronization notes , You can take it yourself. Welcome star correct :https://gitee.com/noblegasesgoo/notes

 If something goes wrong, I hope the leaders in the comment area can discuss and correct each other , Maintain the health of the community. Let's work together , There is no tolerance for wrong knowledge .
										——————————————————————  Love you  noblegasesgoo