Ideal interface automation project

How to evaluate interface automation project

My ideal interface automation project , There is a script , can 100% Cover core business scenarios , Be able to return to the frequently occurring problems on the line , It can support scenarios that cannot be verified manually , Be able to throw occasional problems , It can contain boundary scenes that are easy to miss 、 Abnormal scenario , It can run regularly every day , Issue a report , Give feedback .

Use the actual 「 Index quantification 」 Next ：

• Function test automation rate = Number of automated test cases / Total number of test cases

• Automated discovery bug Proportion = Automated discovery bug/ total bug Count

• Automated discovery bug effectiveness = The effectiveness of automated discovery bug/ Automated submission bug total

• Automated execution success rate = The number of successful automated executions / Total number of automated executions

• Proportion of weekly active automation use cases = (∑ Automated test cases executed weekly )/(∑ Weekly dimension automated test cases )

• Apply dimension automation code coverage = The number of lines of code covered by automated use cases / Total rows of application dimension code

What is the status quo (Situation)

I am responsible for the invoice middle office , The simplified data flow is shown in the following figure ：

In the invoice, the console provides... To the caller RPC service , There is reading and writing at the bottom MySQL、 Sync ES、 Message queue 、 Asynchronous tasks and other services , Upstream are users and orders , Account number required 、 Qualification of increasing votes 、 Orders and other materials ; Downstream is finance , Push application form to finance department to issue invoice, etc .

The interface automation of invoices was written many years ago , use Java+TestNG+RPC Localized a framework to implement , Because of employee resignation and business change , When this set of automation came into my hands , It's no longer usable , Localized interface automation projects cannot do data measurement , Only the automated use cases written by the interface test platform can be counted .

What is the task (Task)

I set my own performance at Q2 complete 95 Automation use cases , Implement invoice interface automation according to a standardized automation project , Combined with the test environment 、 Test material 、 Assembly line 、 Precise analysis , Create a stable deposit of individuals , Whether it's promotion or job hunting , Can be used to tell some stories .

What is action (Action)

「 How to write automated use cases 」

1、 The ideal way , It is able to record online traffic and directly convert it into automated use cases , Just modify it . Although we have a traffic recording and playback platform , But it is not connected with the interface automation platform , It's a pity that we can't do this .

2、 Refer to the online traffic log , Grab interface 、 Enter the reference , Manual conversion to automated use cases .

3、 Reference interface code design , such as controller/provider/sdk Application interfaces are defined , You can know what covers , What needs to be added .

4、 Construct parameters according to the input template . Too many parameters are troublesome .

I mainly use the 2 Species and 3 Two ways to write automated use cases .

「 Typical automation use cases 」

The first step ： Initialization data , Update the order invoice type to 3.

Preset variables ： Provide parameterized data .

Execution steps ： Call the modify order invoice type interface , The invoice type 3 Change it to 2, The input parameter refers to the preset variable .

Financial interface ： The dependent test material cannot meet , adopt Mock To return template data , Let the interface go on .

Basic assertion ： Interface response status code 200.

Business assertion ： Call the order list query interface , use JSONPath Extract the invoice type in the response , The assertion has been changed to 2.

Scenario use cases

Data preparation ： Create order data through material tools , write in MySQL In the database .

to update MySQL data ： Update some field data of the order , Trigger binlake, Synchronize data to ES, Because submitting Invoicing requisition is read ES data ; Get... From preset variables orderId.

Submit Invoicing requisition ： from ES Read order data , Submit Invoicing requisition ; Get... From preset variables orderId and userId.

Query the application No ： Query the requisition number according to the order number ; Get... From preset variables orderId and userId. use JSONPath Withdrawal requisition No , Stored in dynamic variables .

Push Finance ： Push the application No. to the finance department ; Get... From dynamic variables applyId.

Cover more code

① Look at the code ：

stay Java In the project Controller/Provider Find all interfaces , Analysis can be used as an interface for automation , Design automation use cases to cover .

② Code coverage ：

With the help of jacoco, After executing the automation use case , Run out the code coverage , Analyze uncovered code , Add more 「 normal / abnormal / The border 」 Automation use cases for . That is, precision testing .

Automated inspection

Every day in the morning 8 spot , Timed task execution use case set ; Send E-mail , Copy to the test team 、 Development Group 、 Leader , Let test 、 Both the development and leaders know about the automation construction ; If something goes wrong , It must be solved on the same day , And reply the reason of the problem by email ; Ensure that the automated use case results for the day are correct ; The next day, the patrol inspection continued .

Assembly line

In continuous integration , Introduce automated use case links , Take the automation case run through as the test card point , Failure to pass may result in rejection .

What is the output (Result)

Qualitatively ：

「 Combined with the test environment 、 Test material 、 automation 、 Assembly line 、 Precision testing , Completed a stable patrol inspection for environment and business , Be able to carry out quality access control for R & D and testing , Interface automation projects that can analyze the code coverage of applications .」

Quantitatively ：

• Function test automation rate = Number of automated test cases / Total number of test cases  「95/100=95%」

• Automated discovery bug Proportion = Automated discovery bug/ total bug Count  「2/53=3.7%」

• Automated discovery bug effectiveness = The effectiveness of automated discovery bug/ Automated submission bug total  「2/2=100%」

• Automated execution success rate = The number of successful automated executions / Total number of automated executions  「5396/5621=96%」

• Proportion of weekly active automation use cases = (∑ Automated test cases executed weekly )/(∑ Weekly dimension automated test cases ) 「77%」

• Apply dimension automation code coverage = The number of lines of code covered by automated use cases / Total rows of application dimension code  「39%」

This Q It mainly completes the automation use cases of core processes and important interfaces , You also need to continuously supplement scenario use cases and exceptions / Automated use cases for boundaries .

Learning resource sharing

Finally, thank everyone who reads my article carefully , Watching the rise and attention of fans all the way , Reciprocity is always necessary , Although it's not very valuable , If you can use it, you can take it

These materials , For those who want to advance 【 automated testing 】 For our friends, it should be the most comprehensive and complete war preparation warehouse , This warehouse also accompanied me through the most difficult journey , I hope it can help you ！ Everything should be done as soon as possible , Especially in the technology industry , We must improve our technical skills . I hope that's helpful …….