Mainstream design of database middleware

Database middleware mainstream design

1 The role of database middleware

No read / write separation is performed / In the case of sub database and sub table , We use the data source directly in the application (c3p0、druid、dbcp2 etc. ) Establish a connection to the database , Read and write , The architecture is as follows ： 

You can see that in the case of single database and single table , We use the data source directly in the application (c3p0、druid、dbcp etc. ) Establish a connection to the database , Read and write .

Most developers are familiar with the architecture of applications that access a single library . But in the process of read-write separation / After the sub-library sub-table , There will be multiple database instances at the bottom , One in the case of read / write separation master Multiple slave; In the case of sub database and sub table , There are many different sub Libraries .

From an application perspective , In addition to establishing connections with multiple different databases , You also need to handle sub database and sub table / The problem of separation of reading and writing in a specific scenario ：

• stay Read / write separation Under the circumstances , Application needs to read sql/ Write sql Distinguish , read sql Walk from the library , Write sql Go to the main library , And consider the master-slave synchronization delay 、 A series of problems such as high availability .
• stay Sub database and sub table Under the circumstances , The application needs to be able to sql To analyze 、 rewrite 、 route 、 Set the results and wait for some operations , And distributed transactions 、 Distributed id Generator, etc .

This is undoubtedly a complicated job , The role of database middleware , It allows developers to operate the database like a single database and a single table , Shield the underlying complex implementation .

2 Database middleware design scheme

Typical database middleware design schemes include 2 Kind of ： Server agent (proxy： Proxy database )、 Client agent (datasource： Proxy data sources ). The following figure illustrates the architecture of these two schemes ：

You can see whether it is a proxy database or a proxy data source , The bottom layer operates multiple database instances . The difference is ：

• Server agent (proxy： Proxy database ) in ：  We deploy an agent service independently , Behind this proxy service, multiple database instances are managed . And in application , We use a common data source (c3p0、druid、dbcp etc. ) Establish a connection with the proxy server , be-all sql Operation statements are sent to this agent , This agent operates the underlying database , Get the result and return it to the application . Under this scheme , The logic of separating database tables from reading and writing is completely transparent to developers .

• Client agent (datasource： Proxy data sources )：  The application needs to use a specific data source , Its function is to act as an agent , Internal management of multiple common data sources (c3p0、druid、dbcp etc. ), Each common data source is connected to different libraries . Application generated sql Give it to the data source agent for processing , The data source is internal to sql Do what is necessary , Such as sql Rewriting, etc. , Then give it to each ordinary data source to execute , Combine the results , Return to app . Data source proxies often also implement JDBC Specification defined API, Therefore, it can be directly connected with orm Framework integration . Under this scheme , The user's code needs to be modified , The data source using this agent , Instead of using it directly c3p0、druid、dbcp Such a connection pool .

3 Comparison of mainstream database middleware implementation

Whether it's a proxy database , Or proxy data sources , The functions of both are similar . The existing implementations of these two schemes and their advantages and disadvantages are listed below ： 

Database agent

The current implementation schemes are ： Alibaba open source cobar,mycat Team in cobar Developed on the basis of mycat,mysql Official mysql-proxy, qihoo 360 stay mysql-proxy Basic development atlas. At present except mycat, Several other projects are basically not maintained .

advantage ： Multilingual support . in other words , Whatever you use php、java Or other languages , Can support . The reason is that the database agent itself implements mysql Communication protocol of , You can just think of it as a mysql The server .mysql The official team provides different client drivers for different languages , Such as java Linguistic mysql-connector-java,python Linguistic mysql-connector-python wait . So developers in different languages can use mysql The corresponding driver provided by the official is used to establish communication with the proxy server .

shortcoming ： The implementation is complex . Because the proxy server needs to implement mysql The communication protocol of the server , Therefore, it is difficult to realize .

Data source agent

The current implementation schemes are ： Alibaba open source tddl, Public comments are open source zebra, Dangdang open source sharding-jdbc.

advantage ： Lighter weight , Can be associated with any orm Framework integration . This scheme does not need to be implemented mysql Communication protocol of , Because the common data source of the bottom Management , You can go directly through mysql-connector-java Driving and mysql The server communicates , Therefore, the implementation is relatively simple .

shortcoming ： Only one language is supported . for example tddl、zebra、sharding-jdbc Is to use java Language development , So for users of other languages , You can't use these middleware . Version upgrade is difficult , Because the application uses the data source agent to introduce a jar Package dependency , In many applications there are versions of jar When a package generates a dependency , Once this version has bug, All applications need to be upgraded . Database agent upgrade is relatively easy , Because services are deployed separately , Just upgrade the proxy server , All applications connected to this agent are naturally upgraded .

ORM Framework agent

There are hibernate Provided hibernate-shards, It can also be done through mybatis Write in a plug-in way . Compared with the previous two schemes , This scheme can be said to have only shortcomings , No advantages .

Another rookie ORM-Bee Split table only ( Single database sub table )、 Only branch library 、 Read write separation in sub database and sub table , All support well . Compared with the previous introduction ,Bee It's easier to use , Good performance . The scheme of dividing database and table is also in progress .

zebra,sharding
1. To parse sql sentence , The supported database types are limited
2. The process is complicated
In the case of sub database and sub table , The application needs to be able to sql To analyze 、 rewrite 、 route 、 Set the results and wait for some operations , And distributed transactions 、 Distributed id Generator, etc .

And zebra,sharding comparison ,Bee

Can support implemented JDBC All data of the specification ;

The process is simple , There is no need to be right sql String parsing .

Types of sub database and sub table _abckingaa The blog of -CSDN Blog

Bee frame , You can learn it in ten minutes ORM frame --Bee_abckingaa The blog of -CSDN Blog _bee orm