How MySQL - depots table?A look at will understand

I. Why do we need to sub-library and sub-table

If the business of a website develops rapidly, the traffic of this website will also increase, and the pressure of data will also follow. For example, for the e-commerce system, the Double Eleven promotion puts a lot of pressure on order data, and the Tps is more than 100,000.Concurrency, if the traditional architecture (one master and multiple slaves), the main database capacity will definitely not be able to meet such a high Tps, the business is getting bigger and bigger, the single table data exceeds the capacity supported by the database, persistent disk IO, traditional database performanceBottleneck, product manager business · must do, change the program, database knife segmentation optimization.If the number of database connections is not enough, the database needs to be divided, the data volume of the table is large, and the query performance after optimization is still very low, so it needs to be divided.

Second, what is a sub-library and sub-table

The sub-database sub-table scheme is a supplement to the relational database data storage and access mechanism.
Sub-library: split the data of one library into multiple identical libraries, and access one library when accessing
Sub-table: put the data of one table into multiple tables, and operate the corresponding oneJust the table

Three, several ways of sub-library sub-table

1. Vertical split

(1) Database vertical split
According toBusiness split, as shown in the figure, e-commerce system, split into order library, member library, commodity library

(2) Table vertical split

Split the table according to the business, as shown in the figure, split the user table into user_base table and user_info table, use_base is responsible for storing login, and user_info is responsible for storing basic user information

Vertical Split Features
1. The structure of each library (table) is different
2. At least one column of data in each library (table) is the same
3.The union of each library (table) is the full amount of data

Vertical split pros and cons

Advantages:
1. After splitting, the business is clear (dedicated database is split by business)
2. Data maintenance is simple, according to different business, business is placed on different machines

Disadvantages:
1. If there is a large amount of data in a single table, the pressure of writing and reading is high
2. It is determined by a certain business or limited, that is to say, a business will oftenAffect the bottleneck of the database (performance problems, such as double 11 panic buying)
3. Some businesses cannot be associated with join, and can only be called through the java program interface, which increases the complexity of development

2. Split horizontally

(1) Database split horizontally

As shown in the figure, according to the member database, it is divided into member 1 database, member 2 database, split by userId, userId tail number 0-5 is database 1
6-9 is database 2, and there are other ways, take the modulo, put even numbers in bank 1, and put odd numbers in bank 2

(2) Table split horizontally

As shown in the figure, split the users table into users1 table and users2 table, split by userId, take the modulo, put the even numbers in the users1 table, and put the odd numbers in the users2 table

Other ways to split horizontally

1. Divided by range, each database has a piece of continuous data, which is generally based on time range, but this is generally less used, because it is easy to generate hot issues, and a large amount of traffic is hit on the latest data.Now, advantages: when expanding, it is very easy, because you only need to prepare, prepare a library for each month, and when a new month comes, naturally, a new library will be written. Disadvantages: largePart of the request is to access the latest data.The actual production range depends on the scenario. Your users do not only access the latest data, but evenly access the current data and historical data
2. Hash distribution, advantages: can evenly distribute the data volume and request pressure of each database Disadvantages: it is more troublesome to expand, there will be such a process of data migration

Horizontal splitting features
1. The structure of each library (table) is the same
2. The data of each library (table) is different
3.The union of each library (table) is the full amount of data

Horizontal split pros and cons

Advantages:
1. The data of a single database/single table is kept to a certain amount (reduced), which helps to improve performance
2. Improve the stability and load of the systemAbility
3. The structure of the split table is the same, and there are less program modifications.

Disadvantages:
1. It is difficult to expand the data and maintain a large amount of maintenance
2. It is difficult to abstract the splitting rules
3. Consistency of fragmented transactionsSome businesses cannot be associated with join, and can only be called through the java program interface

4. Problems caused by sub-library and sub-table

1. Distributed Transaction
2. Cross-database join query
3. Distributed globally unique id
4. Development costs are high on programmers

5. How to select the technology of sub-library and sub-table

Open source framework for sub-database and sub-table
jdbc direct connection layer: shardingsphere, tddl
proxy proxy layer: mycat, mysql-proxy (360)

jdbc direct layer

jdbcThe direct connection layer is also called the jdbc application layer, because of all sharding rules, all sharding logic, including processing distributed transactions
All these problems are in the application layer, all projects are composed of war packages, allThe sharding is written as a jar package and placed in the war package. Java needs a virtual machine to run. When the virtual machine runs, the byte file in the war package will be loaded into the jvm memory by classLoder. All sharding logicIt is based on memory.

proxy layer

as shown,proxy proxy layer, all sharding rules, all sharding logic, including processing distributed transactions are written in mycat, all sharding logic is based on mycat side operations

Advantages and disadvantages of jdbc direct connection layer and proxy layer

• jdbc direct connection layer has high performance, only supports java language, and supports cross-database
• The proxy layer has low development cost, supports cross-language, and does not support cross-database