On database optimization (taking MySQL as an example)

Database optimization follows a funnel rule , Here's the picture ：

The funnel rule of database optimization can be basically divided into 5 A hierarchical ：

1. Reduce data access （ Reduce disk access ）
   • Indexes , Reduce full table scanning
2. Return less data （ Reduce network transfers or disk access ）
   • Return the required data according to the actual business requirements
3. Reduce the number of interactions （ Reduce network transmission ）
   • cache
   • stored procedure
   • Batch query
4. Reduce servers CPU expenses （ Reduce CPU Several memory overhead ）
   • Batch processing
   • Handle a large number of operations on the client side
5. Use more resources （ Increase resources ）

   • Hardware resources

For the above figure, we have studied an optimization direction ：

 PS: Let's talk about the actual development , Basically is sql Optimize , At most, it is database table structure optimization ( Sub database and sub table )

Talk about sql The idea of optimization ：

First of all, when we sql The statement execution time has reached 3s When above , It's unacceptable , It is usually a millisecond response , We need to turn on mysql Slow query log of database ( Need to be in mysql Configuration in the configuration file ), Locate the slow execution sql After the statement , We use... For this statement explain Keyword to view its execution plan , Analyze whether the index failure caused the full scan .

Here are some common cases of index failure ：

 

Let's talk about the optimization of database table structure ( That is, sub database and sub table )：

Common splits include horizontal split and vertical split , However, horizontal splitting is used at most , I have not been exposed to vertical splitting at present , Let's talk about horizontal splitting , Like an order form , A lot of data is inserted every day , Suppose that the data in a table has reached 1000 ten thousand ( Theoretically mysql Single table greater than 500 Ten thousand data , We need to consider the horizontal sub table , Because the larger the amount of data in a single table, the slower the query , The query efficiency drops exponentially ), Let's split this table horizontally , Then there will be a problem that is how to know which database you want to check data in .

We use hash The idea of dividing tables ( The precondition is to ensure that the primary key is a number ), Suppose the two tables are divided into 100 individual hash Slot , The table above shows 0~49, The table below for 50~99, Suppose the incoming id by 50 when , use 50/100 Take the remainder as 50, Then look it up in the table below .

We can also use third-party plug-ins , such as mycat, It just helps us integrate hash The idea of dividing tables .