Cucumber test practice

source ：https://cucumber.io/docs/guides/overview/

As QA, Maybe we will encounter some problems in the iteration ：

1. Development and refactoring have a large impact , A large number of regression tests are required each time, which is time-consuming and labor-consuming
2. For some technical cards, if the test does not know the specific impact scope , I always feel uneasy without testing
3. Some customers will ask for test results such as test cases or test reports , But these may not be necessary in agile processes , It would be troublesome to prepare alone

These problems Cucumber Testing practice will provide some solutions , And far more than that .

One 、 Change of thinking

1、 To cultivate CICD consciousness

CICD, Continuous integration continuous deployment / Delivery is not a new concept , But up to now, for testers, few people have this awareness of testing . Testers in agile processes are still on a step-by-step basis Issue The content of the card conceives the test scope 、 Design test scenarios 、 Execute test case , If you do better, you may add simple automated tests later . Then there will be a deviation in rhythm , Agile processes are often accompanied by a large number of 、 Changes in a short time , If the tester follows the above process to deal with these changes , This means a lot of repetitive work . Let's take the simplest example , An authenticated system optical login is necessary for each test . therefore , When a lot of changes 、 Large scale refactoring occurs during iterations , This means that the workload of the tester will be the same as before Issue Sum of cards , It may be necessary to take measures such as reducing some test scenarios in order to compromise .

Imagine if our testing work is also made into a continuously integrated product ？ Plan all the tests we have done 、 Regular rubbing down . For tests previously performed , After that, we just need one click That is, it can execute , For expanding business needs , Just expand on the existing grammar . Deliver products iteratively , The test set is also iterating . This will not only save the testing workload, but also make QA There is an overall control over the whole product quality framework .

2、 Try to reduce “ Bare handed ” test

When we consciously make our tests integrate and execute continuously , We will realize “ Bare handed ” Testing needs to be reduced （ But it is also necessary in some scenarios ）. Because bare handed testing means one-time and relatively inefficient , Even if the rubbings are made, these free hand tests are irregular and can not be expanded .

3、 Actions and assertions need to be closed-loop , Test scenarios need idempotence

Just like the functions completed in the development, they all need to have Issue The record is the same ,QA The test behavior of should be rubbed down as much as possible . However, not all actions can be called actions , There is a need to realize that there is a difference between behavior and implementation , We want to document specific user behavior, not every step of the behavior . At the same time, try to ensure that each behavior can be self verified , Among them, the appropriate assertion is the top priority , We need to record as many assertions as we can be sure of what we are looking for , Make the recorded behavior as consistent as possible with the verification items of our free hand test .

When we record one test after another , Form test scenarios . These scenarios need to satisfy idempotency . This is also the premise that our test set can be integrated and executed continuously .

Two 、Cucumber Test practice

1、 Not at all BDD

According to Wikipedia ,BDD It is a kind of for TDD An attempt to improve agile software development , The main purpose is to make DEV、QA、BA、PO Form a common understanding of how programs work . At present, our Issue Cards are basically similar BDD Gherkin The grammar of , and kick off and desk check These two links need DEV、QA、BA Wait for different roles to complete , This is similar to a kind of BDD Practice of closed loop verification .

However , Our goal is to integrate all our behaviors in the testing process 、 Assertions are recorded by the program , therefore Cucumber It is used as a scripting tool to complete the test practice . In this scenario, we are testing a developed code , It's not a kind of BDD. Use Cucumber It doesn't mean to use BDD. So you don't need to give Cucumber The test adds an awkward layer BDD The appearance of , Instead, it is used as a scripting tool to uniformly implement the test execution behavior , Form a dictionary of test behavior .

2、 Write well Gherkin

Cucumber The execution process is as follows

source ：https://cucumber.io/docs/guides/overview/

Finally came Cucumber The practical operation of , First we need to write Gherkin, That's what I think Cucumber The most difficult piece of . I personally write Gherkin Feature The style of the document has just come into contact Cucumber Up to now, there is a big difference .Gherkin The compilation of is the whole Cucumber The script program can be maintained 、 Scalable 、 Easy to understand 、 The key to reusability , Also avoid Cucumber goes bad The key to . For details, please refer to ：https://cucumber.io/docs/bdd/better-gherkin/ and https://cucumber.io/docs/guides/overview/.

3、Step Definitions Assertions are important

as for Step Definitions That's right. coding Ability is related , But as a scripting tool, you need to lighten the project as much as possible 、 Portability . What I do is whatever it is Cucumber-jvm still Cucumber-js It will be based on the database involved in the project 、 Background or big data components to write some tool classes , Assemble through these Step Definitions. In order to prevent Cucumber goes bad, Every Step Definition It needs to be as concise as possible and not contain too much logic , But it needs to reflect the test logic we designed . Therefore, the assertion granularity must be close to our manual testing , So that we can trust our script enough , So that the script can really work .

The design of assertions is basically an effort to copy the human eye , What you see is what you get , What we see is where we assert . But it is not limited to this , For many scenes, human eye observation is limited , For example, the comparison of a large number of data 、 Simulation of various random scenes , These scripts can often be written beyond the human eye .

To make a long story short Step Definitions The principle to follow is light weight 、 It can be maintained, but we should complete our actions and assertions as carefully as possible . This sounds like a paradox , But it is our goal .

4、 Two test scenario construction ideas

If you read “ Change of thinking ” This chapter , Then we may have a feeling that it is no different from the feeling of designing test ideas or test cases when we test at ordinary times , Just add some restrictions . This feeling is actually quite correct . We are based on Gherkin Of Feature A file is actually a set of test cases . At the same time, I always think that using Cucumber Or other tools to test , It's all based on QA Test design . Test design is the core of our testing work , therefore , The construction idea of the first test scenario naturally arises ： Rewrite the test idea or test case to be based on Gherkin Linguistic Feature file , The conversion method is shown in the following figure ：

When we build some test scenarios according to the above construction idea, we get some Steps after , We naturally have such an idea ： If we put aside test design and test cases , Starting from the practical application scenarios , utilize Steps Build into a real possible scenario to form a test Feature file . Because we are designing and writing Steps I have followed the principles of behavior , And realize the self verification of each behavior , Can that prove that when I set up this Feature If the file runs through, I will have verified the whole scene , As shown in the figure below ：

3、 ... and 、 About E2E test

1、Cucumber And E2E Combination is not a good practice

stay github On the search Cucumber Related open source projects ,95% All of the above will Cucumber and E2E Combined use of testing tools . from Cucumber+WebdriverIO To the nearest Cucumber+Cypress and Cucumber+Testcafe. In terms of implementation , There seems to be no problem with this combination , It's even easier to use at the beginning . However, there are several problems ：

（1） Not good Gherkin How to write it As shown in the figure below , This is the official website for Gherkin An example of writing , This is also mentioned earlier Gherkin The behavior should be recorded, not the practice corresponding to the behavior . But almost all Cucumber E2E Projects are written in the following way . Such an approach may be less or better in terms of implementation , But the original intention can not be fulfilled ideologically . For example , In the login scenario ,Gherkin Steps It should be written like this :When "Bob" logs in instead of ：

 Given I visit "/login"
  When I enter "Bob" in the "user name" field
    And I enter "tester" in the "password" field
    And I press the "login" button
  Then I should see the "welcome" page

（2） And PageModel perhaps PageObject Not a good combination Cucumber E2E The core idea is to integrate various operation behaviors 、 Assertions are abstracted to form Steps. But when these become Steps Then it's hard to be someone Page Properties of . The key problem is ,PageObject The page has been defined as the body , That is to say Gherkin Medium Given, Different test contents Description It can be seen as different Scenarios, So there's no need to Cucumber Another layer of packaging , The semantic naming of functions can fully express the meaning .

（3） It usually takes time and effort I always think so in principle , You should write fewer E2E test . End to end testing is inherently slow , Therefore, the number of tests should be much lower than that of other tests .Cucumber It usually requires a lot of compatibility and adaptation of behaviors , These will consume a lot of energy . meanwhile UI Testing due to a large number of intelligence deficiencies , It is difficult to match the granularity and effect of human eye testing , So you can see a lot of UI Testing is a constant repetition of a fixed process , Many errors are also reported from the script itself rather than the product itself . On the whole, no matter what you do UI The test cost performance is very low .

2、 Personal solutions

For having Browser Or other Client Project , I will adopt the scheme of separation test . First, simulate yourself as each end contact to access the corresponding back end , use Cucumber Test the backend separately , This block will undergo fine-grained testing , Ensure the accuracy of functions and data .

about Browser and Client The terminal adopts lightweight E2E The script is used to confirm that the main functions are available 、 The console is normal 、 Simple popup assertion , These are what scripts can do . Of course, manual testing is essential , Because in Browser and Client Dimensions , Judging from the current technology, the assertion of the human eye and the judgment of the human brain are far more than the script . in addition Bug Bash And so on Browser and Client Demonstrate a good solution for testing .

Four 、Cucumber Related information

Books ：《The cucumber book》 Official documents ：https://cucumber.io/docs/cucumber/Cucumber-js: https://github.com/cucumber/cucumber-jsCucumber-jvm: https://github.com/cucumber/cucumber-jvm A simple one IDE：http://cuketest.com/

The copyright of this article belongs to Thoughtworks Owned by the company , If you need to reprint, please leave a message in the background .