MySQL tuning - overview of MySQL parameters and status

mysql Parameters and states

Execution status

List MySQL The server runs various status values

show global status;

Common variable names are as follows ：

• Aborted_clients The connection has died because the customer did not close it properly , Number of connections that have been dropped .
• Aborted_connects The attempt has failed MySQL The number of connections to the server .
• Connections Trying to connect MySQL The number of servers .
• Created_tmp_tables When executing a statement , The number of implicit temporary tables that have been created .
• Delayed_insert_threads The number of latency insertion processor threads in use .
• Delayed_writes use INSERT DELAYED Number of rows written .
• Delayed_errors use INSERT DELAYED There are some errors in writing ( Possible duplicate key values ) The number of rows .
• Flush_commands perform FLUSH The number of orders .
• Handler_delete Number of requests to delete rows from a table .
• Handler_read_first The number of requests to read the first row in the table .
• Handler_read_key The request number is based on the key read line .
• Handler_read_next The number of requests to read a line based on a key .
• Handler_read_rnd The number of requests to read a line based on a fixed location .
• Handler_update Number of requests to update a row in the table .
• Handler_write Number of requests to insert a row into the table .
• Key_blocks_used The number of blocks used for the keyword cache .
• Key_read_requests Number of requests to read a key value from the cache .
• Key_reads The number of times a key value is read from disk .
• Key_write_requests The number of times a keyword block was requested to be written to the cache .
• Key_writes The number of times a key block is physically written to disk .
• Max_used_connections The maximum number of simultaneous connections .
• Not_flushed_key_blocks Key blocks that have changed in the key cache but have not been cleared to disk .
• Not_flushed_delayed_rows stay INSERT DELAY The number of rows in the queue waiting to be written .Open_tables Number of open tables .
• Open_files The number of open files .
• Open_streams The number of open streams ( It is mainly used for logging ）
• Opened_tables The number of tables that have been opened .
• Questions The number of queries sent to the server .
• Slow_queries It costs more than long_query_time Number of time queries .
• Threads_connected The number of connections currently open .
• Threads_running The number of threads that are not sleeping .
• Uptime How long has the server been working , Unit second .

Configuration parameters

Inquire about MySQL Server configuration information statement

show variables;

The number of connections

maximum connection

Maximum number of connections allowed for clients

MySQL [(none)]> show variables like 'max_connections';
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id:    99652
Current database: *** NONE ***

+-----------------+-------+
| Variable_name   | Value |
+-----------------+-------+
| max_connections | 10000 |
+-----------------+-------+
1 row in set (0.00 sec)

The machine MySQL The maximum number of server connections is 10000, Then query the maximum number of connections that the server responds to ：

MySQL [(none)]> show global status like 'Max_used_connections';
+----------------------+-------+
| Variable_name        | Value |
+----------------------+-------+
| Max_used_connections | 220   |
+----------------------+-------+
1 row in set (0.00 sec)

MySQL The maximum number of connections to the server in the past was 220, The maximum number of server connections has not been reached 10000, There should be no 1040 error (Too many connections The hint means that there are too many connections ), The ideal setting is ：
　　Max_used_connections / max_connections * 100% ≈ 85%
　　 The maximum number of connections accounts for... Of the maximum number of connections 85% Left and right optimal , If it turns out that the ratio is 10% following ,MySQL The maximum number of server connections is set too high .

View connections

View all current connections

show processlist

show processlist Of Info If it is too long, it will intercept （100 character ）, It also does not support filtering by attributes , Not conducive to analysis .

show full processlist All information will be displayed , But the presentation takes up too much space 、 It is not conducive to batch analysis .

processlist Table structure

MySQL [(none)]> desc information_schema.processlist;
+---------+---------------------+------+-----+---------+-------+
| Field   | Type                | Null | Key | Default | Extra |
+---------+---------------------+------+-----+---------+-------+
| ID      | bigint(21) unsigned | NO   |     | 0       |       |
| USER    | varchar(32)         | NO   |     |         |       |
| HOST    | varchar(64)         | NO   |     |         |       |
| DB      | varchar(64)         | YES  |     | NULL    |       |
| COMMAND | varchar(16)         | NO   |     |         |       |
| TIME    | int(7)              | NO   |     | 0       |       |
| STATE   | varchar(64)         | YES  |     | NULL    |       |
| INFO    | longtext            | YES  |     | NULL    |       |
+---------+---------------------+------+-----+---------+-------+
8 rows in set (0.00 sec)

Attribute specification

Term contract ：MySQL General use Thread( Threads ) Represents an execution plan / Mission , In order to understand from the perspective of application , Use connection instead of thread as expression . Official basis ：Each connection to mysqld runs in a separate thread. MySQL edition ：5.6/5.7/8.x

• ID ID It's not difficult to understand. , Namely mysql Unique identification of the connection . There are two purposes for this logo ：

Manipulate or filter specific connections , For example, use kill On command Locate the problem connection . Such as viewing transactions 、 Lock isochronous , There will be one thread_id, This is the corresponding processlist.ID; Through this relationship , When analyzing complex problems, you can locate specific connections In the next blog post 《 Business / Lock related statistics 》 Mentioned in

• USER Literally , The user who created the database connection . Can be used for user behavior tracking , It can also be used for user behavior statistics .

HOST Create a connected server , Generally, the server IP+ Port composition ; In practice , Often only IP Partially useful , For example, statistics by request source , When in use, you can intercept ：substring(host, 1, instr(host, “:”)-1).

• DB The connection performs SQL The database of . Some connections may be created without a database specified , Therefore, this item may be empty .
• COMMAND I thought it was “ command ”, Only when you use it can you find the connection status .

The common values of this item are Sleep( Sleep )、Query( Inquire about )、Connect( Connect ), Other values are generally not common / use , If you are interested, please refer to the official documentation 2.

• TIME The connected is in the current state (STATE) Duration of , The unit is in seconds . Note that “ Duration of the current state ”.

Interpretation of official documents 1：

The time in seconds that the thread has been in its current state. For a slave SQL thread, the value is the number of seconds between the timestamp of the last replicated event and the real time of the slave machine.

The general state changes very quickly 、 The duration in each state is very short , If it lasts for more than a second , That means there is a problem . So this setting , Let time be judgment SQL Key elements of normality . If TIME Is the time that the connection existed , Then the meaning will be lost .

There are friends in master-slave mode , I have encountered the case that the time is negative , Please refer to this article ：https://www.jianshu.com/p/9f180c37d983

• STATE SQL Status of execution .

An action, event, or state that indicates what the thread is doing. 1

This item is very important , They tend to point out the problem . STATE To combine TIME To use , That is, the duration is relatively long , The greater the probability of problems .

Most states correspond to very quick operations. If a thread stays in a given state for many seconds, there might be a problem that needs to be investigated.1

STATE More values , It is suggested that after reading this article , read 《MySQL Performance analysis - ( 3、 ... and ) processlist Of state Properties, 》 To learn more

• INFO The complete SQL sentence .

In practical analysis , This is also very important information .

Locate the SQL, Specific positioning business code 、 It is not far to solve the problem thoroughly You can extract multiple SQL features , Perform consolidated statistics

More details , Please refer to ：https://www.yisu.com/zixun/448114.html

Timeout parameters

show variables  like '%timeout%'
+-----------------------------+----------+
| connect_timeout             | 10       |　　　　## Connection timeout ,10 second 
| delayed_insert_timeout      | 300      |　　　　## Delay insertion timeout ,300 second 
| have_statement_timeout      | YES      |　　　　##
| innodb_flush_log_at_timeout | 1        |　　　　## Refresh redo log buffer Timeout time ,1 second 
| innodb_lock_wait_timeout    | 120      |　　　　## The maximum time that a transaction waits to get a resource , If the resource is not allocated after this time, the application will fail ,120 second 
| innodb_rollback_on_timeout  | ON       |
| interactive_timeout         | 28800    |　　　　##mysql Client interactive connection timeout , Default 8 Hours , Used to control the sleep Overtime 
| lock_wait_timeout           | 31536000 |　　　　## Mainly aimed at DDL Produced metadata locks Timeout time 
| net_read_timeout            | 60       |　　　　## Network read data timeout ,60 second 
| net_write_timeout           | 60       |　　　　## Write data for network timeout 60 second 
| rpl_stop_slave_timeout      | 31536000 |　　　　## Stop slave library service timeout 
| slave_net_timeout           | 60       |　　　　##slave Network timeout 
| thread_pool_idle_timeout    | 60       |
| wait_timeout                | 28800    |　　　　##jdbc/odbc Connection timeout , Default 8 Hours , Used to control the sleep Overtime 
+-----------------------------+----------+

Threads

MySQL In terms of architecture, it is divided into Server Layer and storage engine layer , Their threading model is slightly different . stay Server Each connection in the layer corresponds to a thread , There is little thread concurrency control , As long as you can also create connections , The corresponding thread will be created , These threads execute concurrently . And in the InnoDB Storage engine layer , To prevent too many concurrent threads , Thread switching overhead is too high , You can limit the number of concurrent threads , To improve performance

[ Failed to transfer the external chain picture , The origin station may have anti-theft chain mechanism , It is suggested to save the pictures and upload them directly (img-VUWgQELE-1643101235968)(http://media-hk01.oss-cn-hongkong.aliyuncs.com/images%2Fliaomin%40jieztech.com%2Fmysql%E4%BC%98%E5%8C%96.md%2F5ac8ad10-e5c4-40f9-935e-c4a09a2f2b12_image.png?Expires=1800154570&OSSAccessKeyId=LTAIXyAB35iFpHM0&Signature=%2BCjvMB2RMoX%2FlFeC6HDY8WiK8ro%3D&x-oss-process=image%2Fformat%2Cpng)]

Server Layer thread model

MySQL The working mode of is to assign a thread to each connection , When the client and MySQL Server setup TCP After the connection ,MySQL The server will assign a thread to the connection , When this connection receives SQL when , The corresponding thread executes this SQL, And when SQL After execution , This thread goes Sleep, Waiting for a new request from the client . This thread will live forever , Until the client logs out , And close the connection , This thread will exit （ Or enter MySQL Of ThreadCache）

InnoDB Threading model

From the foregoing, we can know ,MySQL stay Server The layer does not control the number of concurrent threads at all , When MySQL When the number of concurrent connections increases , This will cause the number of corresponding threads to increase . This may put a lot of machinery CPU Resources are spent on thread switching , Resulting in a sharp drop in performance .InnoDB In order to alleviate this situation , By setting the system variable set global innodb_thread_concurrency ＝ x You can control the number of internal concurrent threads , That is, at most innodb_thread_concurrency Threads at the same time InnoDB Internal operation . That is to say Server Layer can create many threads , But here we are InnoDB Inside , Only a few threads will execute concurrently , Other threads are all in sleep state . The schematic diagram is as follows ：

Thread Pool

Thread Pool stay MySQL The official is based on Enterprise edition plugin In the form of , And in the MariaDB and Percona It's all intrusion MySQL Source code , They are similar in implementation , There is no essential difference . Here is the main analysis Percona Of Thread Pool Realization . As mentioned earlier ,MySQL Server Every time a client connects , A thread will be created to serve the connection . and Thread Pool The idea is to unbind the connection from the thread , Connections and threads are no longer one-to-one , It is m Yes n The relationship between . The specific way is through epoll Listen to network ports , Once there is client data to be processed , Put this connection in the queue , Let the background thread pool fetch the connection information from the queue , And serve this connection . in other words MySQL The number of connections can be very high , But the number of threads can be only a few dozen . adopt Thread Pool Completely solve the problem of too many threads , Problems that cause performance degradation , The schematic diagram is as follows ：

Specific source code analysis reference （ Content reference ）：http://www.wangyulong.cn/blog/1534473143/

Thread related statements

Parameters and states

View Thread Parameters （ Thread pool size ）：

MySQL [(none)]> show variables like '%thread_cach%'
    -> ;
+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| thread_cache_size | 200   |
+-------------------+-------+
1 row in set (0.00 sec)

The other parameters

• thread_handling Application Thread_Cache How to handle connections
  no-threads: The server uses a thread ,
  one-thread-per-connection: The server uses one thread for each client request .
• thread_stack When every connection is created ,mysql The memory allocated to it . This value is generally considered to be applicable to most scenarios by default , Don't touch it unless necessary .

MySQL [(none)]> show variables like 'thread_%';
+-------------------+---------------------------+
| Variable_name     | Value                     |
+-------------------+---------------------------+
| thread_cache_size | 200                       |
| thread_handling   | one-thread-per-connection |
| thread_stack      | 196608                    |
+-------------------+---------------------------+
3 rows in set (0.00 sec)

View the current thread status .

MySQL [(none)]> show global status like 'Thread%'
    -> ;
+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| Threads_cached    | 92    |
| Threads_connected | 157   |
| Threads_created   | 249   |
| Threads_running   | 7     |
+-------------------+-------+
4 rows in set (0.00 sec)

• Thread_cached Indicates the number of free threads in the current thread pool .
• Threads_connected Indicates the thread that has been bound to the connection .
• Threads_created Indicates the number of threads that have been created .
• Threads_running Indicates running （ perform sql etc. ） Number of threads for

Be careful ：thread_cache_size Is the size of the connection pool , The actual number of connections that can be created is determined by max_connections Decisive .

show global status like ‘Max_used_connections’; The status should be the same as Thread_created The values are consistent .

show processlist Number and Threads_connected Value consistent ,show processlist Medium id The thread id.

<font color=red> The number of connections determines the number of threads , Thread switching of channels leads to poor performance , Use as much as possible Thread pool Pattern , The number of connections can be as many as possible , Number of threads =cpu*2 Number .</font>

kill Threads

mysql>show proccesslist ;
mysql>kill progress_id; #proccess_id  namely show proccesslist  Found out id

But for lock threads , Its status is sleep, So find it id use proccesslist Cannot see ,
But it works sys In the system library performance_schema （MySQL5.6 Start by default , Compared to setting to off There will be 10% Loss of performance around ）

adopt sys.schema_table_lock_waits Look up table lock , Find the cause of the blockage process id , Use this connection kill Just disconnect the command :

mysql>select blocking_pid from sys.schema_table_lock_waits ;
mysql>kill blocking_pid;

adopt sys.innodb_lock_waits Check line lock
Can pass sys.innodb_lock_waits The watch found # MySQL 5.7

mysql>select * from sys.innodb_lock_waits where locked_table=`'test'.'t'`\G
mysql>kill blocking_pid;

tablecache

table file

The storage engine is InnoDB, stay data You'll see it in the directory 2 Class file ：.frm、.ibd

1. *.frm– Table structured file .
2. *.ibd– Table data and index files . The index of the table (B+ Trees ) Each of the non leaf nodes stores the index , The leaf node stores the index and the data corresponding to the index .
   *.MYD–"D" Data information file , Is the data file of the table .
   *.MYI–"I" Index information file , Is the data tree of any index in the table data file

When a connection accesses a table ,MySQL Will check the currently cached table （ Note that the table structure is not data ） The number of . If the table is already open in the cache , The table in the cache will be accessed directly to speed up the query ; If the table is not cached , The current table will be added to the cache and queried .

Before the cache operation is performed ,table_open_cache Used to limit the maximum number of cache tables ： If the current cached table does not reach table_open_cache, The new table is added ; If this value has been reached ,MySQL According to the last query time of the cache table 、 Query rate and other rules before releasing the cache .

data structure

• table: 　MySQL For each query sql Create a table in TABLE object （ structure , This object is connected to one each ）
• table_share: MySQL Create a for each table table_share object , And frm File corresponding to the （ Multiple connections share one ）
• handler:　　 Corresponds to each TABLE object ,innodb The engine creates a handler
• dict_table_t: innodb For every one ibd surface load A data dictionary object

[ Failed to transfer the external chain picture , The origin station may have anti-theft chain mechanism , It is suggested to save the pictures and upload them directly (img-byyDPk2P-1643101235970)(http://media-hk01.oss-cn-hongkong.aliyuncs.com/images%2Fliaomin%40jieztech.com%2Fmysql%E4%BC%98%E5%8C%96.md%2F8d085efb-d7bf-4229-9330-8cfc08f86944_image.png?Expires=1800176555&OSSAccessKeyId=LTAIXyAB35iFpHM0&Signature=elmOax5Owkirba2MHMR6B9gj7zA%3D&x-oss-process=image%2Fformat%2Cpng)]

Parameters and states

View two of the parameters about the table cache value

mysql> show variables like 'table%cache%';
+----------------------------+-------+
| Variable_name              | Value |
+----------------------------+-------+
| table_definition_cache     | 1400  |   Is the above table_share Of cache Count 
| table_open_cache           | 2000  |  Is the above table Of cache Count 
| table_open_cache_instances | 16    |
+----------------------------+-------+
3 rows in set (0.00 sec)

these two items. cache The recommended value is ：
table_open_cache= maximum connection * Number of tables （ Because each connection thread will create a ）
table_definition_cache= Number of tables （ Because all threads share ）

View the currently open tables and the number of historical open tables

mysql>  show global status like 'open%tables%'; 
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Open_tables   | 102   |
| Opened_tables | 109   |
+---------------+-------+
2 rows in set (0.00 sec)

• Open_tables： Represents the number of open tables .
• Opened_tables： Represents the number of tables that have been opened , If Opened_tables Too many , Description configuration table_cache(5.1.3 And then this value is called table_open_cache) The value may be too small

The more suitable value is ：
　　Open_tables / Opened_tables * 100% >= 85%
　　Open_tables / table_open_cache* 100% <= 95%

View all open tables

MySQL [(none)]> show open tables where in_use >0;
+----------------+--------------------+--------+-------------+
| Database       | Table              | In_use | Name_locked |
+----------------+--------------------+--------+-------------+
| apos_order     | order              |      2 |           0 |
| apos_commodity | skusale            |      2 |           0 |
| apos_commodity | mall_agent_product |      2 |           0 |
| apos_stock     | batchstockbusiitem |      1 |           0 |
| apos_order     | orderconsignee     |      2 |           0 |
| apos_order     | orderdrawee        |      2 |           0 |
| apos_mall      | mall_mobile_page   |      1 |           0 |
| apos_commodity | product            |      2 |           0 |
| apos_commodity | mallproductsale    |      2 |           0 |
| apos_commodity | skusalestock       |      2 |           0 |
| apos_order     | orderexpress       |      1 |           0 |
+----------------+--------------------+--------+-------------+
11 rows in set (0.00 sec)

The other parameters ：

• table_open_cache

Server Layer parameters .

This parameter indicates that for all threads Of table cache The sum of the ,5.6.7 The default is 400,5.6.8 And then 2000.

This is a server Parameters of the layer ,mysql Parallel queries are not supported ,mysql There is no conversation PGA The concept of , One thread quote myisam Table needs to be in server Create a... On the layer table object （ The index also needs to be created but shared ,self join Will create 2 individual , Partition table each partition is treated as a single table ）, If multiple sessions reference a table at the same time, multiple table objects will also be created , Although it will increase memory usage , But it greatly reduces the contention of internal table locks .

The recommended number of this value is max_connections* The number of your watches , Of course, you may also need to reserve some temporary tables and other objects , But the number is big enough .

that mysql When to release these table objects ？

1. When the buffer is full , When the connection wants to open a table that is not in the buffer .
2. When the number of buffers has exceeded table_open_cache Set the value of the ,mysql Start using LRU The algorithm releases the table object .
3. When you use flush tables; When the sentence is .

• open_files_limit

Engine layer parameters .

This parameter represents mysqld Maximum number of file descriptors available , If you meet “Too many open files” Error of , Consideration should be given to increasing it . The default value for this parameter is 0 Means unlimited （ Greater than 5.6.7 The default value is no longer 0, Refer to the official website ）, But in fact, its value is related to the operating system , stay Unix The number of this value in the system cannot be greater than ulimit -n.

This parameter should be greater than or equal to table_open_cache.

It's not clear when innodb_open_files This parameter refers to the limit on the number of handles for all storage engines , Or not innodb Limit on the number of handles to , You need to study the source code before you can sort it out , Let's consider the former for the time being .

• innodb_open_files

Engine layer parameters .

This parameter is only valid for InnoDB Memory engine active , It designates mysql The maximum that can be opened at the same time .ibd The number of files . This parameter does not affect table_open_cache I don't want to open_files_limit influence , Is independent only to InnoDB Effective . So the default is InnoDB You can store the engine without considering open_files_limit Just set innodb_open_files.

this 3 The relationship between the parameters can be summarized as follows , To guarantee performance , You should set the following values ：

max_connections* The number of your watches = table_open_cache <=open_files_limit< ulimit -n

innodb_open_files<ulimit -n

Hit test

test tablecache

1. Create a table test, Then insert a database .
2. Restart the database so that cache Release .
3. Use strace monitor mysqld The process of

[[email protected] ~]# strace -f -ttt -e file -y -p 17050
strace: Process 17050 attached with 27 threads
strace: Process 17149 attached
[pid 17149] 1642495912.743274 open("/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 43</sys/devices/system/cpu/online>
[pid 17149] 1642495927.389855 open("./test/test.frm", O_RDONLY) = 43</var/lib/mysql/test/test.frm>

4. Write a query statement , Then check to see if the cache is hit , Find out hit yes 0, Simultaneous discovery strace It appears that open(test.frm) The action is reading the table structure file .

show session status like 'Table_open_cache%';

6. Write a query statement , Check whether the cache is hit , Find out hit yes 1, Simultaneous discovery strace No, open, Description use cache

   show session status like 'Table_open_cache%';
   

7. Opening a conversation , Execute the query statement to find strace No, open But it also misses the cache hit=0, The instructions use table_share cache .

The slow query

MySQL The slow query , The full name is Slow query log , yes MySQL A type of logging provided , To record in MySQL in Response time exceeds threshold The sentence of .

Parameters and states

MySQL Slow query related parameter interpretation ：

• slow_query_log： Open slow query log or not ,1 Open for indication ,0 Means closing .
• slow-query-log-file： new edition （5.6 And above ）MySQL Database slow query log storage path . This parameter can not be set , The system will default to a default file host_name-slow.log
• long_query_time： Slow query threshold , When the query time is longer than the set threshold , Log , The default is ：10s.
• log_queries_not_using_indexes： Queries that are not indexed are also recorded in the slow query log （ optional ）, The default is FALSE.
• log_output： How to store logs .log_output=‘FILE’ Means to save the log to a file , The default value is ’FILE’.log_output='TABLE’ Indicates that the log is stored in the database .

MySQL [(none)]> show variables like 'slow%';
+---------------------+-------------------------------+
| Variable_name       | Value                         |
+---------------------+-------------------------------+
| slow_launch_time    | 2                             |
| slow_query_log      | ON                            |
| slow_query_log_file | /data/mysql/3306/log/slow.log |
+---------------------+-----

If you want to query how many slow queries there are , have access to Slow_queries System variables .

MySQL [(none)]> show global status like '%Slow_queries%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Slow_queries  | 550   |
+---------------+-------+
1 row in set (0.00 sec)

Analysis tools

In the production environment , If you want to analyze the log manually , lookup 、 analysis SQL, It's obviously individual work .

MySQL Provides log analysis tools mysqldumpslow

see mysqldumpslow Help for ：

[root@DB-Server ~]# mysqldumpslow --help
 Usage: mysqldumpslow [ OPTS... ] [ LOGS... ]
 
Parse and summarize the MySQL slow query log. Options are
 
  --verbose    verbose
  --debug      debug
  --help       write this text to standard output
 
  -v           verbose
  -d           debug
  -s ORDER     what to sort by (al, at, ar, c, l, r, t), 'at' is default（ sort order ）
                 al: average lock time（ Average lock time ）
                 ar: average rows sent（ Average number of returned records ）
                 at: average query time（ Average query time ）
                  c: count（ Access count ）
                  l: lock time（ Lock time ）
                  r: rows sent（ Back to the record ）
                  t: query time（ Query time ）
   -r           reverse the sort order (largest last instead of first)
   -t NUM       just show the top n queries（ Back to front n Data ）
   -a           don't abstract all numbers to N and strings to 'S'
   -n NUM       abstract numbers with at least n digits within names
   -g PATTERN   grep: only consider stmts that include this string（ Regular match pattern , Case insensitive ）
   -h HOSTNAME  hostname of db server for *-slow.log filename (can be wildcard),
                default is '*', i.e. match all
   -i NAME      name of server instance (if using mysql.server startup script)

First, get the location of the slow log

show variables like '%slow_query_log_file%';

such as , Get the most returned recordset 10 individual SQL.

mysqldumpslow -s r -t 10 /data/mysql/3306/log/slow.log

The most visited 10 individual SQL

mysqldumpslow -s c -t 10 /data/mysql/3306/log/slow.log

Get the top... In chronological order 10 There are left connected query statements in the bar .

mysqldumpslow -s t -t 10 -g "left join" /data/mysql/3306/log/slow.log

It is also recommended to use these commands in conjunction with | and more Use , Otherwise, screen swiping may occur .

mysqldumpslow -s r -t 20 /data/mysql/3306/log/slow.log | more

A temporary table

If SQL The data read in the execution process cannot directly get the results , Then you need extra memory to store the intermediate results , To arrive at the final result , This extra memory is the internal temporary table , What operations generate temporary tables ,

• 1、UNION Inquire about ;
• 2、 be used TEMPTABLE Algorithm or UNION View in query ;
• 3、ORDER BY and GROUP BY When the clauses of are different ;
• 4、 Table connecting ,ORDER BY Is not in the drive table ;
• 5、DISTINCT Search and add ORDER BY when ;
• 6、SQL It is used in SQL_SMALL_RESULT Option ;
• 7、FROM Subqueries in ;
• 8、 Subquery or semi-join Table created when

Parameters

Two parameters related to the temporary table

1.tmp_table_size
tmp_table_size Specifies the maximum value of the internal memory temporary table , Unit is byte , Each thread is assigned . If the memory temporary table exceeds the limit ,MySQL Will automatically convert it to disk based MyISAM surface , Store in designated tmpdir Under the table of contents , Default :

mysql> show variables where Variable_name in ('tmp_table_size', 'max_heap_table_size'); 
mysql> show variables like "tmpdir"; 
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| tmpdir        | /tmp/ |
+---------------+-------+

When optimizing query statements , Avoid using temporary tables , If it can't be avoided , To ensure that these temporary tables are stored in memory . If necessary, and you have a lot of group by sentence , And you have a lot of memory , increase tmp_table_size( and max_heap_table_size) Value . This variable does not apply to user created memory tables (memory table).
You can compare the total number of internal disk based temporary tables with the total number of temporary tables created in memory （Created_tmp_disk_tables and Created_tmp_tables）, The general proportional relationship is :Created_tmp_disk_tables/Created_tmp_tables<5%
2.max_heap_table_size
This variable defines the memory tables that users can create (memory table) Size . This value is used to calculate the maximum row value of a memory table . This variable supports dynamic changes ,
namely set @max_heap_table_size=N. But there is no use for existing memory tables , Unless the table is recreated (create table) Or modify (alter table) perhaps truncate table. Service restart will also set the existing memory table to global max_heap_table_size Value . This variable and tmp_table_size Together, it limits the size of the internal memory table .

state

mysql> show global status like 'created_tmp%';
　　+-------------------------+---------+** **
　　| Variable_name | Value |** **
　　+-------------------------+---------+** **
　　| Created_tmp_disk_tables | 21197 |** **
　　| Created_tmp_files | 58 |** **
　　| Created_tmp_tables | 1771587 |** **
　　+-------------------------+---------+

Every time you create a temporary table ,Created_tmp_tables increase , If you are creating a temporary table on disk ,Created_tmp_disk_tables Also increase ,Created_tmp_files Express MySQL Number of temporary files created by the service , The ideal configuration is ：

Created_tmp_disk_tables / Created_tmp_tables * 100% <= 25%
　　 For example, the server above Created_tmp_disk_tables / Created_tmp_tables * 100% = 1.20%, It should be pretty good . Let's see MySQL Server configuration of temporary tables ：** **

The query cache

Query cache failures are very frequent , As long as there is an update to a table , All query caches on this table will be cleared .
So it's likely that you're struggling to save the results , It's not in use yet , It's all cleared by an update . For databases that are under pressure to update , The hit rate for the query cache will be very low . Unless your business has a static table , Update for a long time , For example, the system configuration table , Then the query of this table is suitable for query caching .

Most applications implement caching in the application logic layer , Simple as a map Of mybatis, Complex can be used redis perhaps memcache, Direct access to memory is far faster than network access , therefore mysql Directly discards the query cache

Even though MySQL Query Cache Designed to improve performance , But it has serious scalability problems , And it's easy to be a serious bottleneck .
since MySQL 5.6（2013） since , Query caching is disabled by default , Because as we all know , It cannot scale with high throughput workloads on multi-core computers .

Watch lock condition

mysql> show global status like 'table_locks%';
　　+-----------------------+-----------+** **
　　| Variable_name | Value |** **
　　+-----------------------+-----------+** **
　　| Table_locks_immediate | 490206328 |** **
　　| Table_locks_waited | 2084912 |** **
　　+-----------------------+-----------+

• Table_locks_immediate： Indicates the number of table locks released immediately .
• Table_locks_waited： Indicates the number of table locks to wait for , If Table_locks_immediate / Table_locks_waited > 5000, It's better to use InnoDB engine , because InnoDB It's the line lock and MyISAM It's a watch lock , For high concurrency write applications InnoDB It will be better . The server in the example Table_locks_immediate / Table_locks_waited = 235,MyISAM That's enough .

View the transaction being locked

select * from information_schema.innodb_locks;

View transactions waiting for locks

select * from information_schema.innodb_locks_waits;

Using system tables for lock queries ：

select r.trx_isolation_level, r.trx_id waiting_trx_id,r.trx_mysql_thread_id waiting_trx_thread, r.trx_state waiting_trx_state,lr.lock_mode waiting_trx_lock_mode,lr.lock_type waiting_trx_lock_type, lr.lock_table waiting_trx_lock_table,lr.lock_index waiting_trx_lock_index,r.trx_query waiting_trx_query, b.trx_id blocking_trx_id,b.trx_mysql_thread_id blocking_trx_thread,b.trx_state blocking_trx_state, lb.lock_mode blocking_trx_lock_mode,lb.lock_type blocking_trx_lock_type,lb.lock_table blocking_trx_lock_table, lb.lock_index blocking_trx_lock_index,b.trx_query blocking_query from information_schema.innodb_lock_waits w inner join information_schema.innodb_trx b on b.trx_id=w.blocking_trx_id inner join information_schema.innodb_trx r on r.trx_id=w.requesting_trx_id inner join information_schema.innodb_locks lb on lb.lock_trx_id=w.blocking_trx_id inner join information_schema.innodb_locks lr on lr.lock_trx_id=w.requesting_trx_id G

Involved 3 A table shows ：

information_shcema The next three tables （ And this can be monitored through the current three lock table ）

• innodb_trx （ Print innodb Currently active in the kernel （ACTIVE） Business ）
• innodb_locks （ Print the current status generated by innodb lock Print only when there is a lock waiting ）
• innodb_lock_waits （ Print the current status generated by innodb Lock wait Print only when there is a lock waiting ）

Common diagnostic problems

The longest execution time sql

adopt pidstat Find the occupation in the current process cpu The largest thread ·

1. pidstat -t -p <mysqld_pid> 1: find CPU high THREAD_OS_ID ;

2. according to THREAD_OS_ID Find the SQL Text ;
   select * from threads where thread_os_id=1487;

Check the process io Occupy

^Croot@iZwz9e8jq6kptfkfpyo2pxZ:~# pidstat -d -p 1316
Linux 4.19.0-16-amd64 (iZwz9e8jq6kptfkfpyo2pxZ)         01/24/2022      _x86_64_        (8 CPU)

04:27:53 PM   UID       PID   kB_rd/s   kB_wr/s kB_ccwr/s iodelay  Command
04:27:53 PM  1000      1316      8.82    386.00      0.00       1  mysqld

IO Statistics (-d)
Use -d Options , We can see the progress IO Statistical information ：
kB_rd/s: The amount of data the process reads from disk per second ( With kB In units of )
kB_wr/s: The amount of data the process writes to disk per second ( With kB In units of )
Command: Pull up the command corresponding to the process