Record one time of Tencent Test Development Engineer's automation interface test practice experience

bosses ：faithchen, tencent PCG Test Development Engineer

Catalog

One 、 background

Two 、 Traffic recording

3、 ... and 、 Generating use cases

Four 、 Traffic frequency filtering strategy

5、 ... and 、 Summary and others

Learning resource sharing

---

One 、 background

Automated testing is important for us to improve R & D efficiency 、CI/CD( Continuous integration / Continuous delivery ) Is an integral part . In background automated testing , Interface testing is particularly important , It can ensure the quality of the tested background service , And the correctness of interface logic , Help us quickly test functions 、 Improve test coverage 、 Control quality risk, etc .

1.1 Background interface test

Interface test is a kind of function test , It is a kind of test to test the interface between system components , The key point is to verify the correctness of data exchange for service interface , Generally all rely on real links , When testing, you need to start the tested service . The figure below shows a Server A The interface test ：

However , Students who have written interface tests may be troubled by complex data structures and complex assertions , It may take more than half a day to write an interface test case , Wasted a lot of valuable time ！ Based on the above considerations , In order to improve the efficiency of writing interface test cases , We want to help developers or testers do these time-consuming and labor-consuming things automatically , The idea is to record traffic , Then automatically generate interface test cases through recorded traffic . Based on the above ideas , We offer from Recording traffic To Generate interface test cases Complete link tools , Assist development students to write interface test cases quickly . Simply speaking , You only need the following three steps to access ：

Next , We go through **“ Traffic recording -> Generating use cases ” To illustrate our implementation method and access mode . We will explain common platforms such as stke、sumeru Service access process , because 123 The platform is right goreplay Wait for better support , In addition to the general process, it will use 123 Take the platform as an example . Finally, it will explain Flow frequency screening scheme ** Help you understand our process .

Two 、 Traffic recording

2.1 Introduction to traffic recording

2.1.1 What is traffic recording

What we hear more is “ Recording and playback ”, But at present, only the recording function is needed , The playback function may be added later . seeing the name of a thing one thinks of its function , Traffic recording refers to the recording of the expected interface data response、request、 Protocols and other stored operations , It can be data from a formal environment or a test environment , After the recording function is turned on , As long as the service corresponding to the environment has traffic , It can be captured and stored , There are many ways to achieve this function .

After comparing several schemes , We decided to adopt goreplay The way , Its open source code is goreplay , be based on goreplay 1.2.0 After the version transformation, it has supported trpc, http, gofree, wup, videopacket, We will continue to support other agreements later , use goreplay Its main advantage is that it can analyze and record service traffic without affecting the service , And the access process is relatively simple .

goreplay The biggest advantage of the solution is that it is decoupled from the business code , No intrusion code is required .

At the transport layer goreplay The recording method is ：

2.1.2 Why traffic recording

Interface testing is testing in a real running service , If you want to automate the generation of use cases , ideally , That is to automatically generate interface test cases for real data , So the first thing you need is Real interface data , Where does the data come from ？ spontaneous , We'll think of recording on the line （ Whether it's a test environment or a formal environment ）.

The following is 「 General process 」 Talk about how to access services goreplay Flow recording process .

2.2 「 General process 」 Service traffic recording

2.2.1 「 General process 」 Upload pb File to the protocol console for parsing binary traffic

First, we need a system that can manage protocols , It can access protocol files （ Such as pb Documents, etc. ） Conduct management , So when the agreement is updated , It can still parse the traffic generated by the new protocol .

To be able to goreplay The recorded binary data is parsed ：

Just put the dependent proto Package files （ Note that it depends on multiple proto Keep the relative directory correct when file ） And upload it to the protocol console , And generate the corresponding version pb Version number of , For our subsequent generation goreplay command .

2.2.2 「 General process 」 Generate and execute goReplay command

After the platform obtains the version number in the above agreement , We know the background service ip And port , The latest version of the protocol has been . Fill these in our goreplay In command , As shown in the figure below ：

Execute the above command in the corresponding container .

Some parameter descriptions ：

--input-raw-logreplay//  This is actually rawInput Plug in parameters ,  When this parameter is turned on ,  must ip and port You have to fill in 
--output-logreplay-cache-size 200// LogReplayOutPut plug-in unit   Required cache size ,  You can leave it blank ,  Have default values 
--output-logreplay-track-response//  Follow --input-raw-track-response Same meaning 
--input-raw-protocol //  Required agreement to be resolved 
--output-logreplay-protocol-service-name "app.server" //  The service name of the protocol file pair 
--output-logreplay-commitid "1.0.1" //  Required , If it is http agreement ,  You can customize 
--output-logreplay-qps-limit 100//  Limit recorded qps,  Not required ,  Have default values 
--output-logreplay-env formal //  optional , Default formal, Need to register to logreplay The test environment can be filled in test

Execute in container ps -ef|grep goreplay Seeing the process as shown in the figure indicates success ：

2.2.3「 General process 」 View recording traffic

When the service has traffic in the corresponding environment , Soon we can see through goreplay Recorded and parsed the successful traffic .

Inside request and respond When the data are clear at a glance （ Here is demo So there is little data , In real environment , There may be many parameters and data ）, Based on these recorded data , After that, we can automatically generate interface test cases .

3、 ... and 、 Generating use cases

When the traffic is recorded , We will have a filtering mechanism to filter traffic , However, this operation is imperceptible to the user . For the user , Then you can automation Generated traffic .

3.1 px-cmdline Tool introduction

We provide px-cmdline Tools , This is a command line tool , It can be goreplay Recorded traffic , adopt proto file , Automatically obtain relevant dependencies and generate interface test files that meet code specifications .

3.2 px-cmdline Use

It's also easy to use , The command line format is ：

px-cmdline generate -p <proto gomod> [-f] [--env test]

Introduction to common parameters :

• -p Appoint proto Depending on the path , from go.mod Middle copy

• -f Force update , After opening , The interface test file with the same name to be generated will be replaced

• --env Specify the environment variable for pulling traffic , Default test Environmental Science

• --dest Specify the generated directory , Default tests Folder

• -j Specifies whether to switch json File mode . By default json File independent mode ; After opening , Use cases are written to *_test.go in

Example ：

px-cmdline generate -p git.code.oa.com/xxxxprotocol/hello/world

After operation , If no exception occurs , Will generate... In the current directory tests subdirectories , According to each RPC Interface to generate independent __test.go file , And a unique suite_bvt_base_test.go Used to start interface testing .testdata Directory to store json Use case data in . Pictured ：

When a service has multiple running nodes , It also supports the selection of running nodes , To request this target when debugging interface use cases locally . It is recommended to give priority to test Environment or developer's feature environment

[Info]  Please select a local test target service ： You can select only the environment , Or specify a node 
[Info] (1)  Test environment  test --------
[Info] (└ 11 ) 9.2.3.4:16916, Development,  The weight 100,  health 
[Info] (2)  Characteristic environment  sdfdfe --------
[Info] (└ 21 ) 11.1.2.3:11389, Development,  The weight 100,  health 
[Info] (3)  Characteristic environment  dfadfb --------
[Info] (└ 31 ) 9.3.4.5:11042, Development,  The weight 100,  health 
[Info] (4)  Characteristic environment  c9d193e8 --------
[Info] (└ 41 ) 11.3.4.5:11222, Development,  The weight 100,  health 

In order to comply with the code specification , The use case data is currently in json Stored in the form of data , Show one of the data as follows ：

{
"request": "{\"pp\":[\"1dfa\"]}",
"response": "{\"sdf\":{\"TPDKH\":{\"basic\":{},\"cosume\":{},\"data\":{\"type\":\"NotSettle\",\"pll\":[733],\"tpl\":\"UG\"}",
"trace_id": "394729374872934"
 },

In this way, the interface test cases are successfully generated , When we debug locally OK after , You can put it mr Enter the code trunk , The interface can be tested automatically during daily pipeline operation .

Yes, of course , If some students like to use more intuitive IDE Tools , We also provide IDE Plug in for everyone to use .

3.3 px-cmdline Tools （IDE edition ） Jetbrains The plug-in USES

If the user wants to go through IDE plug-in unit , You can do the following

First step ： We have relevant plug-ins for installation

 

Select the downloaded... In the pop-up window ZIP package , Then determine ;

The second step ： Restart after installation IDE, To enable the plugin By simply right clicking the plug-in to generate the interface test file, you can achieve the same effect as the command-line tool ：

img

Then you can successfully generate interfaces and use cases in specific folders , Used for interface test execution .

Four 、 Traffic frequency filtering strategy

This chapter supplements , When we have a large amount of data , How to filter and analyze , Retain the relatively valuable traffic for generating interface test cases .

4.1 Traffic filtering background

This chapter describes our traffic filtering strategy , This part of the content is imperceptible to users , But it helps to have a better understanding of the complete link of our tool .

When we access for services goreplay after , There are thousands of traffic recorded on the service , Sometimes even more , So how do you know which traffic is repeated , What traffic is worth recording to generate use cases ？

Due to the large online environment traffic , Many traffic data may have high repetition rate and similarity , Regarding this , We will not simply store all traffic directly into the data for subsequent generation of interface tests （ After all, we can't test a piece of data with thousands of interface use cases ）, But through certain strategies Frequency screening , We will store the traffic that meets our requirements in our database .

The benefits of this are ：

1. Prevent excessive traffic of some services , The stored data is too large .

2. Eliminate duplicate traffic , The traffic with high similarity is not stored repeatedly , At the same time, it ensures the data diversity of interface test .

4.2 Similarity comparison

4.2.1 The overall process of similarity comparison

Here's the picture , There are two flows in the example response, How we treat response perhaps request The data for similarity calculation , The general process is ：

• take body part , Abandon header

• Calculate level 1 key Similarity degree , Class A key We are more strict in similarity , In general , If level one key If it's different, you can think of these two body The difference is large , As shown on the left side of the figure below response The first level of key Yes Msg、ProcId、TaskOutput, On the right side response The first level of key Yes Msg、ProcId、Code, obviously , We will think that the similarity between the two is low , It should be the flow with large difference .

• If level one key Exactly the same , Obviously, further recursive comparison , But the higher the level, the smaller the influence factor we will set , The recursive calculation of similarity is completed .

4.2.2 String similarity

It is generally calculated to level II key in the future , We will start to care about the similarity of strings , The algorithm we use here is Levenshtein distance  , The definition is as follows ：

In a nutshell , Is to calculate how many times a string needs to be edited to become another string , Combine their character length with the number of edits , We can calculate the similarity between the two .

for example , take “kitten” Turn the word into “sitting” The levinstein distance of is 3：

1. k itten → s itten （k→s）

2. sitt e n → sitt i n （e→i）

3. sittin → sittin g （ Insert g）

After obtaining the string similarity , Then we can calculate json The similarity of is response and request The similarity , Combined with multi-level key Then the similarity between the two traffic is calculated .

4.3 Screening strategy

After obtaining the similarity , We will also filter through multi-layer strategies .

• Set up response and request The proportion of , Because usually response Better reflect interface differences , Setting the specific gravity can better measure the similarity between flows , Conducive to subsequent screening .

• Set multi-layer dynamic threshold , As the amount of data increases , Dynamically adjust the similarity threshold that can be stored , In order to better control the number of recorded flows , Ensure that the recorded flow number converges in the later stage , Will not increase indefinitely .

• Record the number of flow repetitions , It can be used for subsequent generation of use cases , We believe that the more traffic is repeated, the more important it is .

Through multiple strategies , To ensure that we can screen out “ High quality ” The flow is used for subsequent interface testing .

5、 ... and 、 Summary and others

At present, this scheme has served many （ Including micro vision 、 Penguin, etc ） Generate multiple interface test cases , Has more than 1000 Integration into service on a daily basis mr Interface test in .

 

Learning resource sharing

Finally, thank everyone who reads my article carefully , Watching the rise and attention of fans all the way , Reciprocity is always necessary , Although it's not very valuable , If you can use it, you can take it

These materials , For thinking 【 Advanced automated testing 】 For our friends, it should be the most comprehensive and complete war preparation warehouse , This warehouse also accompanied me through the most difficult journey , I hope it can help you ！ Everything should be done as soon as possible , Especially in the technology industry , We must improve our technical skills . I hope that's helpful …….