MySQL query optimization - detailed explanation

SQL General optimization plan :
\1. Using parameterized queries ： prevent SQL Inject , precompile SQL Command to improve efficiency
\2. Remove unnecessary query and search fields ： In fact, in the practical application of the project , Many query conditions are optional , The redundant functions that can be avoided from the source should be cut down as much as possible , This is the simplest and crudest solution .
\3. Select the most efficient table name order : The parser of the database processes from right to left FROM Table name in clause ,FROM The last table in the clause will be processed first , stay FROM Clause contains more than one table , You must choose the table with the least number of records to put at the end , If there is 3 More than table join queries , Then you need to select the table referenced by other tables and put it at the end .
\4. Do not use select ： Do not use select , To improve query efficiency , Reduce the amount of data output , Increase transmission speed （ Database or in the parsing process "" Convert to all column names in turn , This means consuming more time ）
\5. Try to avoid returning large amounts of data to the client , If the amount of data is too large , We should consider whether the corresponding demand is reasonable .
\6. Reduce access to databases ： Through stored procedures, etc , Put multiple statements into a stored procedure to execute , Reduce database access times
\7. Integrate simple 、 No associated database access ： If you have a few simple database queries , You can integrate them into one query ( Even if there's no relationship between them )）
\8. Delete duplicate records ： Delete duplicate records , Reduce database file size
\9. Use the alias of the table (Alias)： When in SQL When multiple tables are joined in a statement , Please use the alias of the table and prefix the alias with each Column On . thus , It can reduce the parsing time and reduce the parsing time Column Grammatical errors caused by ambiguity .
\10. Use the alias of the column ： When the column name is very long , Using short column aliases can make query results clearer , More concise .
\11. use EXISTS replace IN、 use NOT EXISTS replace NOT IN： In either case ,NOT IN Are the most inefficient ( Because it performs a full table traversal of the tables in the subquery ). To avoid using NOT IN , We can make it an outer connection (Outer Joins) or NOT EXISTS.
\12. Statistics related queries , The impact on the result set is often huge , Statistics related queries should be avoided during business peak periods , Or just perform statistical queries from the Library . At the same time, it is recommended to save the data in memory first 、 In cache ( Such as redis), Then write to the database according to a certain strategy .
\13. select count() from table; This is without any conditions count Will cause a full table scan , And there's no business sense , It must be eliminated , Other methods can be used instead .
\14. Use truncate Instead of delete Empty the entire data table ： When deleting a record in a table , In general , Rollback segment (rollback segments ) Used to store information that can be recovered . ( It can be restored to the state before executing the delete command ) And when you use TRUNCATE when , The rollback segment no longer stores any recoverable information . When the command runs , Data cannot be recovered . So very few resources are called , The execution time will also be very short .

Field type optimization ：
\1. Try to use numeric fields , If only contains the numerical value information the field as far as possible does not design for the character type , This reduces the performance of queries and connections , And it increases storage overhead . This is because the engine is processing queries and joins Then it compares each character in the string one by one , For digital models, only one comparison is enough .
\2. It's better not to leave it in the database NULL, Use as much as possible NOT NULL Fill the database .（ remarks 、 describe 、 Comments and the like can be set to NULL）

where Conditional statement optimization ：
\1. WHERE Join order in clause ： The database is parsed from right to left WHERE Clause , According to this principle , Connections between tables must be written in other WHERE Before condition ( Left ), The conditions that can filter out the maximum number of records must be written in WHERE End of clause ( Right ).
\2. To avoid the where The type conversion of the field in the statement （ Implicit type conversion occurs when the type of the field is inconsistent with the type of the passed in parameter ）
\3. Not in where In a conditional statement "=" On the left is the function 、 Arithmetic operations or other expression operations , You can replace function operations by using redundant fields , Otherwise, the system cannot use the index correctly
\4. where Field in the clause null Value judgement 、 contain not、!=、<> Wait for the operator , or like Before the keyword of %(like ‘% key word ’), Can't use indexes , Thus causing a full table scan .
\5. Use like Pay attention to , Unless necessary , Otherwise, don't add keywords %, Otherwise, it will inevitably lead to full table query

Index optimization ：
\1. The more indexes, the better , The index can certainly improve the corresponding select The efficiency of , But it also reduces insert And update The efficiency of , because insert or update It's possible to rebuild the index , So how to build an index needs careful consideration , As the case may be . The index number of a table should not exceed 6 individual , If there are too many, consider whether it is necessary to build indexes on columns that are not often used .
\2. Using indexes to improve efficiency ： Reasonable use of index and composite index can also improve efficiency . But using indexes comes at a price , Indexes need space to store , It also needs regular maintenance , Whenever a record is added or deleted in a table or an index column is modified , The index itself will also be modified . This means that every recorded INSERT , DELETE , UPDATE Will pay more for this 4 , 5 Secondary disk I/O . Because indexes need extra storage and processing , Those unnecessary indexes will slow down the response time of queries .. It is necessary to refactor the index regularly .
\3. When using index fields as conditions , If the index is a federated index , Then the first field in the index must be used as a condition to ensure that the system uses the index , Otherwise the index will not be used .
\4. Pay attention to index maintenance , Periodically rebuild the index , Recompile stored procedures .

Paging statement optimization ：
\1. Optimization of paging query . When there are many pages , Such as limit 10000,10 The result set of influence is 10010 That's ok , The query speed will be slow . The recommended solution is ： First, only query the primary key select id from table where … order by … limit 10000,10( Please index the search criteria and sorting ), Then get the data through the primary key .
\2. For multiple large data volumes （ Hundreds of them are big ） Table of JOIN, Page first and then JOIN, Otherwise logical reading will be very high , Poor performance .

Optimization of transactions :
\1. Use as much as possible COMMIT： Use in the program as much as possible commit, In this way, the performance of the program is improved , Demand can also be caused by COMMIT Less resources are released ,commit Every database engine is different ,sqlserver Default one sql Statement is a transaction .
\2. Try to avoid large transaction operations , Improve system concurrency .
\3. Use only when necessary begin tran：begin tran Ensure the consistency of data , You can ensure that either several tables are modified successfully , Or it didn't work , but Begin tran The price is before the submission , all SQL Statement locked resources cannot be released , until commit fall .Begin tran The principle used is , On the premise of data consistency ,begin tran Trapped SQL Fewer sentences is better ！ In some cases, trigger can be used to synchronize data , Not necessarily begin tran.

Temporary table optimization ：
\1. Avoid frequent creation and deletion of temporary tables , To reduce the consumption of system table resources .
\2. Use “ A temporary table ” Staging intermediate results ： Temporarily store temporary results in temporary table , The next query is tempdb It's in , This avoids multiple scans of the main table in the program , It also greatly reduces program execution “ Shared lock ” Blocking “ Update lock ”, Reduced congestion , Improved concurrent performance . It can also simplify sql Sentence complexity .
\3. When creating a temporary table , If a large amount of data is inserted at one time , Then you can use select into Instead of create table, Avoid creating a lot of log , In order to speed up ; If the amount of data is small , In order to ease the resources of system tables , Should first create table, then insert.
\4. If a temporary watch is used , Be sure to explicitly delete all temporary tables at the end of the stored procedure , First truncate table , then drop table , This avoids the long-term locking of the system tables .
\5. If the amount of data in the temporary table is large , Index required , Then you should put the process of creating temporary tables and indexing in a separate sub stored procedure , In this way, the system can make good use of the index of the temporary table .
\6. Avoid using distinct、order by、group by、having、join、cumpute, Because these statements will aggravate tempdb The burden of .
\7. Carefully use the connection between large temporary tables and other large tables to query and modify , Reduce the burden on the system table , Because this operation will be used multiple times in a statement tempdb System table of .

Cursor optimization ：
\1. Try to avoid using cursors , Because the efficiency of cursors is poor , If the data of cursor operation exceeds 1 Line ten thousand , Then we should consider rewriting .
\2. Before using cursor based methods or temporary table methods , You should first look for a set based solution to solve the problem , The set based approach is usually more efficient .
\3. Just like a temporary watch , Cursors are not unusable . Use... For small datasets FAST_FORWARD Cursors are usually better than other line by line processing methods , Especially when you have to reference several tables to get the data you need .

More optimization ： Read / write separation 、 Master slave backup 、 Log etc.

Interview simple answer ：
From three aspects ：
1. Table field design and index establishment
2. Sub database, sub table and partition
3. Master slave architecture