Dry goods ｜ what do testers need to do for a complete performance test?

One 、  The significance of standardizing the implementation process of performance test

Standardized performance test implementation process can strengthen test workflow control , Define the work to be done in each stage of performance test , Instruct the tester to be correct 、 Carry out the performance test in an orderly way , Improve the working efficiency of each role in performance testing . The performance test implementation process shared this time is the performance test ” Guidelines ”, Hope to help you become a performance test as soon as possible ” Talent show" ”.

Two 、  Performance test implementation process

The performance test process is divided into five stages , Namely 【 Demand research stage 】→【 Test preparation stage 】→【 Test execution phase 】→【 Test report stage 】→【 Test summary stage 】.

 

What to do at each stage ？ Focus on what ？

1. Demand research stage

 

1.1.  Stage Overview

The main work of the research stage is ： Set up a working group 、 Project creation 、 Demand analysis 、 model building 、 Custom performance test detailed implementation plan .

Focus on ： Demand survey 、 Need to analyze 、 model building

1.2.  Key point description

The demand survey is divided into two steps ： Demand survey 、 Demand analysis .

This work is a necessary part of performance test . The output document of the work is 《XX Project performance test requirements table 》, Such as ：《 Cloud wisdom _XXX System _XXX Module performance test requirements table 》.

This phase of model building is mainly business model building .

1.2.1 Demand survey

Ø The requirement research work is led by the performance test implementation personnel , The product manager 、 Development Engineer 、 The operation and Maintenance Engineer shall cooperate to complete ;

Ø The main content of the demand survey is ：

n Performance requirements of system online environment , For example, performance requirements 、 Reliability requirements 、 Maintainability requirements, etc ;

n Other information related to system performance requirements , Including system information （ Such as online environment hardware 、 Parameter configuration 、 System architecture and deployment 、 Associated system deployment, etc ）、 Business information （ Key business logic and process 、 Deal list 、 Trading volume information 、 Business distribution law, etc ）、 Production problems 、 Documentation, etc , And collect the collected information , Realize the overall understanding of the business and technology of the system to be tested ;

Ø Development project team 、 Demand Department 、 The proposer of test tasks such as operation and maintenance department shall fill in 《 Cloud wisdom _XXX System _XXX Module performance test requirements table 》 Medium “ Task information ” and “ Test background ” Etc , Proposed testing requirements , Simple words can't explain , Documents can be attached ;

Ø The implementation personnel of the performance test team will fill in other contents obtained from the investigation 《 Cloud wisdom _XXX System _XXX Module performance test requirements table 》;

Ø For new project approval system or new development version of the system ,《 Cloud wisdom _XXX System _XXX Module performance test requirements table 》 Should and 《 Requirements specifications 》 The performance requirements in are consistent .

1.2.2 Demand analysis

Ø The basic process of requirement analysis is ：

n First , The performance test engineer analyzes the information obtained from the demand survey , take 《 Cloud wisdom _XXX System _XXX Module performance test requirements table 》 The performance requirements in are converted into specific performance requirements index values ;

n secondly , According to the difference between test environment and online environment , The performance test engineer converts the performance requirement index value under the online environment into the performance requirement index value under the current test environment ;

for example ：TPS（Transaction per Second）： The number of transactions processed by the system per second , The derivation process is as follows ,

Current online APP1.0 The trial system is mainly for inquiry transactions , Trading of 40%, The statistics of the system's production and trading volume are 1 About three months 20W pen , hypothesis APP2.0 After the system goes online, the business volume surges to the daily query class 20W, Then the total daily trading volume T achieve :

T = 20W/40%=500000 pen / Japan

System processing power TPS deduction :APP2.0 The largest trading volume after going online 500000 pen / Japan , The system has little trading volume at night , Press 2:8 Principle extrapolation , be (500000*80%)/(8*20%*3600)=69.4 pen / second , Rounding off 70 pen / second , Grow by business volume every year 50% Calculation , After one year, the system processing capacity index is about equal to 70+70*50%=105 pen / second .

Stable trading volume derivation : Take the system's processing power 60%* Duration =105 pen / second *60%*8*3600=1814400 pen .

 

After analysis, it is summarized into test index value

Ø The main contents and normative requirements of requirement analysis are as follows ：

n Performance test requirements ： The performance test index items and demand index values shall be accurately described .

n System scope ： It shall accurately describe the test scope information on which the performance test requirement index value depends , If the logic diagram of the associated system of the test scope should be described , And the information of related systems ; When testing part of the system , It is also necessary to clarify the specific test scope , Describe the subsystem of the tested system in detail .

n Analysis of environmental differences ： It shall accurately describe the test environment information on which the performance test requirement index value depends , If it is necessary to describe the overall network topology of the test environment 、 Test environment machine configuration table （ Number 、 model 、 resources 、 operating system ）、 And the corresponding software configuration 、 Important parameter configuration, etc . At the same time, the above information of online environment shall be accurately described , And a detailed analysis of environmental differences .

The above analysis will be an important part of the performance test plan .

1.2.3 Model building, for example ： business model

according to 200X year XX month XX Japan ~200X year XX month XX Peak business day between dates 200X year XX month XX Statistics of daily business volume , After a little adjustment, the following business model , Business model transactions are required to occupy at least the online trading volume 90% above ：

 

2. Test preparation stage

2.1 Stage Overview

The test preparation stage is an important stage in the performance test . In the preparation stage , You need to build the business model to the test model 、 Performance test implementation plan preparation 、 Preparation of test environment 、 Performance test case design 、 Performance test monitoring scheme design 、 Performance test scripts , And the preparation of relevant test data , And carry out access inspection according to the test plan after the completion of the above related preparatory activities .

Focus on ： Test model building 、 The project design 、 Case design 、 Data preparation, etc

2.2 Key point description

2.2.1 Test model building

The construction of test model is completed by the performance test implementer ;

On the basis of demand analysis , Analyze and sort out the relevant data and information collected from the survey , Focus on the analysis of cross system trading path 、 Transaction related relationship 、 Data processing and flow 、 Business volume 、 Proportion of transactions 、 Typical deal , As well as the processing capacity of the system and other performance test points , Targeting multiple business scenarios , And select a specific set of business transactions for each scenario , Build the corresponding test model according to the proportion of business volume .

The output of this stage is , Each test scenario , And the typical transactions in the scenario and the ratio .

for example ： From business model to test model derivation

According to the business model , Communicate with project team and product manager , In order to determine the test model, the following should also be considered ：

（1） Considering the later upgrading of securities system database , Historical queries may affect , So this test adds a separate scenario ： Historical Commission and historical transaction inquiry 50%（ namely 0456 and 0457）. meanwhile , Considering the online environment, most of the transactions are initiated by the head office , So this test “ Historical Commission and historical transaction inquiry ” All transactions are initiated from the main center ;

（2） Increase bond issuance trading scenario , The issuance and Subscription Date of national debt is generally conducted before the counter business , In this scenario, only one transaction is selected for bond issuance and subscription ;

（3） meanwhile , During the peak trading hours of the securities system, the teller signs in 、 Teller sign off transactions account for a relatively small proportion .

Through the above analysis, the test model is 3 individual ： General day to day model 、 The issue date model of national debt 、 And historical query transaction model .

General day to day model ：

 

Savings bond trading model ：

 

Historical query transaction model ：

 

2.2.2 The project design

The compilation of performance test implementation plan is a necessary part of performance test , Its output is 《 Performance test plan 》, Such as ：《 Cloud wisdom _XXX project _XXX Function module _ Performance can test the program V1.0.xlsb》.

It needs to be described in the plan ： Test requirements 、 Start stop criteria 、 Test model design 、 The test strategy 、 Test content 、 Test environment and tool requirements , And the output documents of each stage . The schedule of performance test work shall also be stated in the scheme 、 Expected risks and risk avoidance methods, etc . The content of test model design comes from the test scenarios formed in the test model design of this stage , And the typical transactions in the scenario and the ratio .

2.2.3 Case design

In case design , Include a description of the case 、 Test environment description （ Hardware 、 Software 、 Application version 、 Test data ）、 Delay settings 、 Stress scenes 、 Executive description 、 Expected results 、 Monitoring points .

Case design is a necessary part of performance test , The output document of case design is 《 Performance test cases 》.

2.2.4 Data preparation

Environment preparation involves the preparation of basic data . Number of test data 、 Logical relations are very strict , The preparation of basic test data generally adopts self-made simulation data or online data after desensitization .

2.2.5 Test script development

Test script development work is to play LR When .

The test script is a programmatic representation of business operations , A script is generally a process description of a business . This activity is mainly for script recording （ To write ）、 Modification and commissioning work , Thus, the script of each test case can be executed correctly under the condition of single stroke and a small number of iterations before test implementation . The general steps of test script development are as follows ：

Ø By recording , Or write , Script code generation complete . When the code is generated , Mainly insert transactions according to requirements , As a unit to count the response time of transactions during the test ;

Ø According to the test requirements , Make parameterization settings ;

Ø Set checkpoints , Judge whether the transaction is executed correctly according to the message content field , That is, the checkpoint is set at the application level ;

Ø Set the assembly point according to the test requirements ;

3. Test execution phase

3.1 Stage Overview

The test execution phase is to execute a test case , Get the data of system processing capacity , The stage in which performance test defects are found . During test execution , Use test tools to execute test scenarios or test scripts , At the same time, cooperate with all kinds of monitoring tools . After execution, collect all kinds of result data for analysis . According to need , During the execution phase, you can perform system tuning and regression testing .

Focus on ： Result record 、 Test monitoring 、 Result analysis

3.2 Key point description

3.2.1 Test execution and result recording

The test execution process has the corresponding priority strategy , Based on the priority of the test case , Give priority to higher level test cases . During the test , The analysis of each test result determines whether to repeat the current case or to execute a new test case ; Usually, when bottleneck problems are found, they are adjusted immediately and test cases are re executed , Until the current case passes .

In the execution phase , Test execution 、 Analysis tuning 、 Regression testing is more repetitive , All execution processes and results must be recorded carefully , Execution result data is the main basis of bottleneck analysis .

3.2.2 Test monitoring

The monitoring work of the test is carried out synchronously with the execution work , When the scenario or script starts executing , At the same time, start the monitoring program （ It can be used nmon Or system command top/vmstat/iostat etc. ）, Of course, we can also use the monitoring treasure and perspective treasure of cloud intelligence to work together , Monitoring treasure can monitor websites ／ Web Performance ／Ping/DNS/FTP/UDP/TCP/SMTP etc. IT Performance indicators of infrastructure , Perspective treasure can find host resources 、Web application 、 browser 、APP And so on , As shown in the figure below ：

 

Monitoring treasure monitoring page

 

Perspective treasure host resource monitoring page

After execution , Stop test monitoring , And extract the monitoring result data .

3.2.3 Analysis of test results

During the test, the results are displayed according to the front-end performance test tool 、 Comprehensive analysis of monitoring results of test problems .

for example ：

The test team is running “ General day to day trading model ” The load test 570TPS Under pressure , Database monitoring found deadlock imagination , As follows ：

 

Problem analysis ： Through analysis with development , Here's why ： Flow control information collection procedure （pltflowGthDaemon） At the same teller 、 When doing transactions concurrently in milliseconds plt_flowgather Watch deadlocks . Online transactions in the test environment are initiated with the same teller number , So the probability of occurrence is high .

4. Test report stage

 

4.1 Stage Overview

Start writing performance test report after test execution . The performance test report needs to be reviewed before release .

4.2 Key point description

4.2.1 Report writing

The performance test report shall include ： Test purpose 、 Scope and method 、 Description of the environment 、 Description of test results 、 Result analysis 、 Conclusions and suggestions, etc .

4.2.2 Description of test results

Description of test results , The execution process of performance test shall be reflected , Such as ： In the capacity test result display of mixed scenario , It is necessary to describe the test results and monitoring results under various concurrent gradients ; In the result record in digital form , The decimal point is required to be accurate 3 Significant digits .

4.2.3 Test defects and problems

In the performance test analysis report, the defects and problems found in the test process shall be described , Risk assessment for items identified as test defects , And give risk tips .

4.2.4 Analysis of final results

Analysis of final test results , This part should be comprehensive 、 thorough 、 It's easy to understand and more intuitive to express through charts .

for example ：

 

4.2.5 Test conclusion

The test conclusion is the content that the performance test analysis report must include . The conclusion of the test must be clear 、 Accurately answer all indexes described in performance test requirements , Test requirements need to be fully covered .

5. Test summary stage

 

5.1 Stage Overview

Summary of performance test , It mainly summarizes the testing process and testing technology of this task . The performance test work has entered the summary stage , It also means that the performance test is coming to an end . At this stage , When time permits, all important test assets shall be archived and kept .