Full link voltage test: the dispute between shadow database and shadow table

author ：

 

Ge tianmeng （ Zhi Yun ）

What is the popular full link voltage measurement in the industry

Full link voltage measurement was born in Alibaba Shuang  11  Preparation process , If you say double  11  Big promotion is the business of Alibaba “ final exam ”, The full link voltage test is before the big test “ Mock exam ”, After its birth, it is known as double  11  Stability guaranteed “ Nuclear weapons ”. The full link voltage test simulates the high traffic scenario in the production environment , Get the most real online actual carrying capacity 、 Perform accurate capacity planning , Ensure system availability .

The rapid development of distributed architecture and business has brought uncertainty to business systems . Any node in a distributed environment can become a bottleneck / Short board / problem , At the same time, with the rapid growth of business , Facing more severe challenges and uncertainties . such as ：

• Single chain road pressure measurement lacks external interference and competition of various resources , The results of single chain road pressure measurement are generally optimistic , It cannot reflect the real system carrying capacity .

• Some problems will only be exposed under real large traffic , Like network bandwidth 、 Impact between systems 、 Basic dependency, etc .

• Full link voltage measurement is more than just pressure measurement , It's more about a real preview , Plan drill 、 Current limiting verification 、 Unified acceptance of high availability schemes such as destructive drills .

The common problem of full link voltage measurement is how to achieve data isolation in the production environment ： When carrying out write pressure measurement in the production environment , It is necessary to ensure that the normal operation of online business will not be affected while the pressure test is carried out , Then we need to consider the isolation and storage of the data generated by the pressure test and the real production data , Avoid the impact of dirty data on online business . In addition to providing shadow table solutions, Alibaba cloud's full link voltage testing platform , It also provides the data isolation scheme of shadow database .

In the process of implementing full link voltage measurement in the production environment , For the two schemes mentioned above , Facing the problem of choosing data isolation scheme , This article first aims at shadow database 、 Shadow table two schemes are introduced and compared , Then for common scenarios , Give suggestions on the selection of the scheme .

Selection of full link voltage measurement data isolation scheme

At present, the full link voltage measurement platform provides

Shadow vault 、 Shadow table

etc.

Solution

. How to choose a suitable scheme ？ This paper first expounds the principles of the two schemes , Then from the performance 、 stability 、 Compare the three cost indicators .

Scheme 1 ： Shadow vault

Pictured  1  Shown , For shadow library solutions , It is to establish the corresponding shadow database on the same instance . After the full link pressure probe mounted by the user service obtains the flow meter, it carries out the corresponding bypass processing , If it is shadow traffic , Then the shadow connection will be obtained from the shadow connection pool for use by the business side , Thus, the data generated by pressure measurement flow is bypassed to the corresponding shadow Library , In order to achieve the effect of data and production library isolation , Thus, the data generated by the pressure measurement flow can avoid polluting the production warehouse .

chart  1： Basic principle of shadow library scheme

Option two ： Shadow table

Pictured  2  Shown , Similar to the shadow library scheme , For shadow table scheme , It is to establish the corresponding shadow table on the same database on the same instance . After the full link pressure probe mounted by the user service obtains the flow meter, it carries out the corresponding bypass processing , If it is shadow traffic , that , The probe will be aimed at this  DB  Call to do  SQL  Parse and replace , Thus, the data generated by pressure measurement flow is bypassed to the corresponding shadow table .

chart  2： Basic principle of shadow table scheme

Scheme comparison

This article mainly from the performance 、 stability 、 The advantages and disadvantages of the two schemes are described from three aspects of cost .

chart  3： Scheme comparison

performance

Machine specification ：4c8g

Concurrency specification ： It is necessary to simulate both normal and pressure measured flows , Here we use  2:8  In proportion to , In order to simulate the low peak period of business traffic for the full link voltage test of the production environment .

• Normal flow ：200  Concurrent

• Pressure measurement of flow ：800  Concurrent

Here we mainly analyze the monitoring of the host where the service is located and the database instance used . among , Application monitoring is mainly based on  CPU、 Memory and average  RT  Analysis of three indicators . The database instance monitors the number of slave connections 、QPS  Analyze the dimensions of the two indicators .

It can be seen from the comparison of indicators in different dimensions of the above two schemes , Shadow table scheme is right  CPU  The consumption is slightly higher , This is related to the implementation of the scheme .

stability

Speaking of stability , You can specify the number of connections from the data source instance 、 Capacity specifications 、IOPS、 Analyze network traffic and other aspects .

The above indicators , Here, take the number of connections as an example , As follows ：

For shadow library solutions . Because different databases are established on the same instance , So if you don't consider that the database instance can reach the maximum number of connections , Theoretically, shadow connection and normal connection are independent , Execution does not affect each other .

For shadow table scheme , Because different data tables are established on the same database on the same instance , Here we need to consider the connection pool configuration on the business side , Because shadow traffic involves  DB  Operation and normal flow are involved  DB  operation , Database connection used , All come from the same connection pool , So if the pressure measurement level is large , It is easy to have connection bottlenecks in connection pools .

cost

According to the table , Here, the redundancy cost and data migration cost are mainly explained , As follows ：

• Redundancy costs

For shadow library solutions , In order to ensure the accuracy of the full link voltage measurement evaluation results , We need to do a full amount of Library migration on the same instance , Including table structure and table data , This will bring about an obvious problem , The cost is relatively high , All basic read-only tables （ This type of table will not have write operations ） There should be one redundant , Unable to achieve the purpose of reuse , So for medium and large enterprises , It's hard to accept .

For shadow table scheme , Is to create a shadow table on the same database on the same instance . Then you can reuse the basic read-only table in the production Library , Just create a shadow table for the write table . The shadow table scheme reduces the cost consumption caused by data redundancy to a certain extent .

• Data migration costs

From the main process of pressure measurement , It is divided into pre pressure measurement 、 Under pressure test 、 After pressure test . among , Data preparation is at the stage before pressure measurement , Whether the pressure test is successful , It is closely related to data preparation . In the process of data migration, other table fields associated with a data table need to be migrated at the same time , This process is more complex and energy-consuming . therefore , Which option to choose , It needs to be evaluated in combination with the complexity of business data .

summary

Sum up , Which of the above two options should be selected , In fact, it is not enough to rely on only one indicator , Comprehensive evaluation should be carried out in combination with the above indicators and specific business scenarios . Here are two typical scenarios to illustrate how to choose a suitable solution , The following comments are for reference only .

scene  1： The proportion of reading tables involved is higher than that of writing tables 、 And the whole database migration cost is high , It is recommended to choose the shadow table scheme , To some extent, it can reduce the cost of complex data migration .

scene  2： The proportion of writing table is higher than that of reading table , At the same time, when the instance capacity of the production library is sufficient , It is recommended to choose the shadow library scheme , It reduces carding to a certain extent 、 Cost of configuration .

at present , Full link voltage measurement provides hybrid cloud solution output , Welcome to use . For details, please click

here

see .

More exchanges , Welcome to the nail group to communicate ,PTS  User AC nail group number ：11774967.