Cloud database smooth disassembly scheme

background

Demand scenarios , At the beginning of the business , Many customer business teams share a database instance , With the rapid development of business , This database instance may have become the bottleneck of the business link , You need to split the instances , This needs to rely on the capabilities provided by the cloud database , from 1 Instances split 2 One or more database instances .

Tencent cloud provides DTS( Database transfer service https://cloud.tencent.com/document/product/571 ) Specified database table migration scheme , It can meet the library disassembly requirements of most user scenarios , But some users are switching between reading and writing to split the new instance , Because the configuration center is not used , There are many business clients , Wide distribution , And business continuity requires that there is no way to stop service and make changes , It is impossible to switch all client configurations to new instances at the same time , There is a risk of data double writing , To solve this problem , We sort out the relevant smooth database removal schemes .

Build scenarios and analyze

Here, the database removal instance is A, The target instance is B, You need to take the instance A(a,b,c) Among them c Library to instance B Split up , First, we need to pass DTS Configure migration tasks , Select the specified object , Total quantity + The incremental migration , This is the time , After the process runs through , example B Yes and instances A Do real-time data synchronization .

Specify object migration

Here is a brief introduction to the idea of removing libraries and modifying configurations smoothly

The idea of smooth splitting of Library tables

1、 adopt dts Remove the specified object from the instance A Move to instance B, And do real-time data synchronization

2、 Put examples B VIP Bind to instance A On the back-end network node of , In this way, no matter from the instance A Or an example B Of VIP The databases accessed by the portal are ultimately instances A The data of .

3、 The user can modify the client configuration at this time , Put the original visit C The configuration of the library is changed to access instances B Of VIP

4、 At this time, you only need to confirm to access the split C Libraries are from instances B Of VIP Come here and ask ( The following meeting will talk about using the database audit function to realize ).

4、 Confirm that the flow segmentation is clean and the instance A With examples B The synchronization relationship is equal , You can take the back-end instance B Of VIP Rebind the backend gateway node as its own instance , And interrupt the synchronization relationship ( If some nodes cannot be checked and modified , To avoid double writing , You can take examples A The corresponding library permissions of are recycled , If an error is reported , At this time, change the configuration to access the instance B).

This idea can also be used for reference in the cloud based database removal , The VPC Bind to the source instance , All the accesses are from the source instance , Then do the smooth modification of the client machine configuration , The source of the request is distinguished by capturing packets .

The key difficulty of this scheme is , In fact, the database audit records the actual data of the business client ip, In this way, there is no way to distinguish between cases through audit A Or an example B Come here and ask .

programme

After verification , There are two solutions to this problem .

Scheme 1 ： Through the ability of new account and database audit

Because of the same account number , Neither packet capturing nor auditing can distinguish which client passes the instance B Of VIP To access the split library table . Here, we need to do a full amount of business sorting , Use an independent new account to access the C library , During the smooth migration process, the old accounts are still accessed through the database audit capability C The client side of the library , Gradually change the client , Finally, switch after confirmation

On the cloud database console , Click database audit , Select an unopened audit instance , Select an audit instance to enable the audit function , By default, select full audit .

Database audit request

By splitting accounts , Gradually change the configuration of the client , Through the audit log, it can be judged that the newly dismantled database tables are accessed through the new account , At this time, you can take the example B Of VPC Re bind to the corresponding new target database , To complete the smooth split .

Audit log display

Option two ： Through the intranet CLB And the ability of database audit

Load balancing （Cloud Load Balancer） It is for multiple sets The ECS provides traffic distribution services . Load balancing can expand the external service capacity of the application system through traffic distribution , Improve the availability of application system by eliminating single point of failure .

Choose a load balancing product on the Tencent cloud official website , newly build CLB example

CLB apply

Application cross VPC visit , You can bind instances at this time B Of VIP, The client's... Obtained from the database audit log IP Is this SNAT IP, By binding database instances B Of VIP, The client changes the target address to CLB Of VIP that will do ( Note that it is not an intranet IP).

CLB Cross domain binding

How to bind instances B Of VIP Well , Click listen Manager , New listener policy , Configure access 、 Weight rule , Bind the corresponding target instance VIP,VPORT( If so x.x.x.x 3306), The binding rules will be displayed on the right after the binding is successful , If the port health status is ： health , This indicates that the database instance listening and binding is normal .

CLB Listening binding

From the back-end audit log, you can see from 9.164.156.78 Come here and ask , Explain through CLB Bind target instance B Of VIP It can meet the requirements of smooth switching ：

Audit log confirms

In this way, the user can split the items that need to be split C The client of the library is changed to correspond to CLB Of VIP,VPORT, After the audit determines that all the requests have been corrected , Make a target instance VPC The binding of , If you don't need to go through CLB To visit , The business needs to be done again VIP You can switch between .

Expand

The above-mentioned splitting scheme still has a certain degree of complexity , It is suggested that businesses should consider introducing the management mode of configuration center , Avoid the subsequent splitting requirements of such scenarios