Learning notes of MySQL series by database and table

1、 Architecture evolution and database and table division

1.1、 Single application single database

This is before the rise of micro services , The architecture of many projects , With the accumulation of business , The project is getting bigger and bigger , The amount of data is also growing , If concurrency comes up , It is easy to have some performance problems . And the project is too big , It is not easy to maintain .

1.2、 Multi application single database

The multi application single database system is to split the previous single application system , Split into multiple services , That is, a distributed system , But the database layer still uses the same database , Just split the business , If there is a large amount of data , The pressure on the database is still so great .

1.3、 Multi application, multi database

Optimize database performance , Cluster improves performance stability after various tuning , If there is still a performance problem , Only when the amount of data is very large can we consider the sub database , This is a multi application and multi database system .

2、 Problems encountered in single application

• The number of user requests is too large ： Single server tps、 Memory 、io There is an upper limit , So you can spread the requests across multiple servers
• The amount of data in a single database is too large ： Single database disk space ,io Upper limit , There is also an upper limit to processing capacity
• The amount of data in a single table is too large ： Inquire about 、 newly added 、 Updates and other operations will slow down , Add field 、 Suo Yin 、 Machine migration will affect efficiency

3、 The type of sub database and sub table

Database and table splitting can be divided into vertical splitting and horizontal splitting

• Split Vertically ： Split based on table or field , Table structure changes , We have separate tables in a single library , There are also sub databases with multiple databases
• Horizontal split ： Based on data splitting , The table structure will not change , The data are different . There are single database and multi database horizontal splitting

3.1、 Split Vertically

• Vertical sub table When the data table contains too many fields and contains large fields , The of the database during query io、 Memory will be affected , Updating data is also slow ,binlog The files will be big , At this time, you can consider splitting the table , Especially for tables with large fields , It can be split into other tables . Pictured , Split the information table of the merchant , It is divided into the common basic information table and the less common detailed information table , Sometimes you can only query basic information according to the business , The detailed information is required for association query .

• Vertical sub database We split some tables with many fields , But they are all in the same library , For databases , The query pressure is still so great , At this time, the business can be split according to the business and the current micro service architecture , It also splits the database , Pictured ：

3.2、 Horizontal split

• Horizontal sub table The previous introduction is to split some tables , There are also those that do not change the table structure , Just split the data , The rules for splitting can be time, division, etc , This method of not changing the table structure is called horizontal split table , Pictured , There is too much data in the order table , According to time , Divided into multiple tables

Rules for horizontal tables

1. RANGE:
   • Time ： By year 、 month 、 The day goes by . for example order_2016、order_201701
   • regional ： Split by province or city . for example order_beijing、order_shanghai
   • size ： from 0 To 1000000 A table ,1000001~2000000 Put a watch
2. HASH： For example, according to users ID modulus

Different business scenario level table rules are different , The rules mentioned above , Examples are as follows ：

1. mail ： In station communication is a function of many systems , Users can only see the information sent to them , Other users can't see , This can be done according to the user ID hash Sub database and sub table
2. A running meter ： In terms of time dimension , It can be according to the year 、 Month and even date sub database
3. The user login ： User enters mobile number to log in , To find the corresponding user ID, But the user information is distributed in multiple databases , In this case, you can design an association table , The association table saves users ID Association with mobile phones , Find the corresponding user directly through the association table ID, Then go to find the corresponding user information

• Horizontal sub database The previous split is based on some ranges or rules , But the amount of data is still growing crazily , At this time, the sub warehouse can be divided , Multiple sub tables divided by time , such as order_1,order_2,order_3,...,order_n Distributed to multiple libraries , Pictured ：

4、 When do you need to divide the warehouse and table ？

With the rapid growth of business volume , Some commonly used business tables have tens of millions of data , Or hundreds of millions of data , At this point if SQL Didn't pay attention to , May cause some slowness SQL Appearance .

So when is it necessary to divide the tables ?500 ten thousand ,1000 ten thousand ？ In fact, there are no specific regulations , But you can refer to 《 Alibaba Java Development Manual 》 The advice inside ： recommend 500 The capacity of ten thousand rows or single table exceeds 2GB It's time to divide the tables

Of course, the data reaches 500 Ten thousand needs to be divided into tables ？ This is not certain , There are only suggestions in the manual , If the data sheet is properly designed , Indexing is also reasonable , The system can also support 500 Thousands or tens of millions of data , Database and table splitting can reduce the pressure on the database , But the separation of databases and tables also means that business processing is more troublesome . For example, to query data , Sometimes you need to span several libraries or multiple tables . How to ensure efficient query is another problem . Therefore, the division of databases and tables should be based on the business or actual situation , Reasonable design .

Problems caused by sub database and sub table ：

• Cross database association query ： If you want to query the user's order information , Through the order form , Then conduct Association query , But the bank has been divided again , Certainly not directly through SQL Find out , At this time, it can be found out separately , And then encapsulate it with code . Of course, sometimes you can use redundant fields , Add another table to some fields , Although it does not follow the database paradigm .
• Distributed transactions ： Dividing databases and tables will also bring about the problem of distributed transactions , Because two libraries are involved , Some business operations need to ensure transaction consistency , You need some distributed transaction processing solutions , Such as compensation mechanism, etc , You can use some middleware , Like Ali's seata Do a good job in distributed transaction processing
• Complex business problems ： The business was relatively simple and good , If the business is complicated , Sort 、 Page turning 、 Function calculation and so on must be used , But sometimes the watch is divided into several pieces 、 Cross database , It must be very troublesome to deal with
• Global annotation problem ： After sub warehouse and sub table , To ensure the uniqueness of annotations , More solutions are needed , such as uuid、 Snowflake algorithm and so on