1: Why should databases be divided into databases and tables

1： Why do we need to divide the database and the table

Business is growing fast , When performance bottlenecks occur in a single database , To segment data . Put the data originally on a database , Distributed to multiple databases , Reduce the load of single database

2： Data segmentation type

1： vertical partitioning

Vertical segmentation is Split multiple business modules into multiple databases , That is to put the tables of the original single database into multiple databases according to the business module .

such as , The order table and the item table are in the same database , Now we want to segment it , Make the order table and commodity table fall into different databases in different physical machines , Make it completely isolated , So as to achieve the effect of reducing the database load . As shown in the figure ：

advantage ：

• After the split, the business is clear , The split rules are clear
• Easy to expand and integrate between systems
• Data maintenance is simple

shortcoming ：

• Some business tables cannot join, Multi table associated query . Can only be called through an interface , It increases the complexity of the system
• Cross database transactions are difficult to handle
• After vertical segmentation , Some data are too large , There are still monomer performance bottlenecks

2： horizontal partitioning

Horizontal segmentation yes Divide the original table into multiple tables according to the law

Horizontal split compared to vertical split , More complex . It needs to put the data in a table , Split into different databases according to certain rules , for example ： Orders with odd ending numbers are placed in the order database 1 in , Orders with even tail numbers are placed in the order database 2 in . In this way, the data originally existed in a database , It is horizontally divided into two databases . When querying order data , We'll have to follow the tail number of the order , Judge whether the order is in the database 1 in , Still in the database 2 in , Then put this SQL Statement sent to the correct evidence base minutes , Find out the order . The architecture of horizontal segmentation is as follows ：

  Split data horizontally , First, the rules of order splitting , Find out which dimension you want to split , Or the example of the previous order , We split according to the odd and even number of the end of the order , So what will be the impact of such a split ？ If I were a user , I placed two orders , An order has an odd ending number , An order has an even ending number , At this time , I'll go to the personal Center , Go to the order list page to view my order . How to query this order list page , According to my users id Take out the order separately 1 Libraries and orders 2 Go to the warehouse to find out the order , Then merge into a list , Is it too much trouble . therefore , When we split the data , Be sure to combine business , Select the splitting rules that are suitable for the current business scenario . Then according to the user id Is it reasonable to split the data ？ Not necessarily. , such as ： Our identity has changed , Not a buyer , It's the seller , My seller has a lot of orders , The seller's background system also has an order list page , How to check the order list page ？ Do you also need to check in all order Libraries , Then aggregate it into an order list . Look at it this way , Is it according to the user id It's unreasonable to split the order again

There are several typical fragmentation rules for horizontal splitting ：

• user id modulus , We have already mentioned
• Split data by date
• Module according to other fields , To split data

  The above figure is based on users id To find a schematic diagram of module splitting

advantage ：

• Solved single database big data 、 The performance bottleneck of high concurrency
• The splitting rules are encapsulated , Almost transparent to the application side , Developers don't need relationship splitting details
• Improved system stability and load capacity

shortcoming ：

• Splitting rules are difficult to abstract
• Fragment transaction consistency is difficult to solve
• During secondary expansion , Data migration 、 High maintenance difficulty . such as ： Let's start by user id Yes 2 modulus , But as the business grows ,2 This database is difficult to support , Or continue to split into 4 A database , Then you need to do data migration

3： Common shortcomings encountered in data segmentation

Whether it's vertical segmentation , Or horizontal segmentation , They solve the storage and access performance problems of massive data , But it also brings many new problems , Their common shortcomings are ：

• Distributed transaction problem
• Cross Library join problem
• Management of multiple data sources

Management of multiple data sources , There are two main ideas ：

1. Client mode , Within each application module , Configure the data source you need , Direct access to database , Complete data integration in each module
2. Intermediate agent mode , The intermediate agent manages all data sources in a unified way , The database layer is completely transparent to developers , Developers don't need to pay attention to the details of splitting

4： Mature solutions

Based on these two data segmentation modes , At present, there are mature third-party software , Next in our video , The representative works of these two models will be introduced to you respectively ：

• Intermediate agent mode ：MyCat
• Client mode ：sharding-jdbc