Significance of unit testing

1. Unit test concept

Unit testing is a means of quality assurance in the process of software development . The original source is to imitate the unit test of hardware chips , Small software units can also be tested in software , So as to ensure the correctness of a partial design in the software .

1.1. Traditional unit test definitions

Traditional software unit testing defines the granularity of the tested unit as the smallest functional module in the software . about C Language usually refers to a function , about Java perhaps C++ Language usually refers to a class .

The traditional method is to carry out various white box tests for the implementation details of the unit under test , That is, carry out various branch tests and coverage tests for the implementation logic of the tested code .

Traditional unit testing lacks the support of automation tools , Often in the test, the test results are printed out , Manually compare whether each test is successful .

1.2. Definition of modern unit testing

With the progress of technology and the development of software unit testing methods , The definition of modern unit testing has changed a lot .

• The granularity of unit testing is based on the loose coupling boundary of software design , It doesn't have to be limited to the small granularity of functions and classes . For example C++ Class of is only used for public Test the interface ,private The interface is not tested . about C The language can only test the interface of each file to provide services , Private auxiliary functions in the file may not be tested . About what to test and what not to test , The ultimate principle to follow is to maximize revenue while reducing the cost of unit testing .
• The best unit test is the black box test for the tested unit , In this way, the implementation details of the tested code will be reduced , Lead to the frequency of unit test linkage modification .
• With the help of modern unit testing framework , Unit tests can be run automatically and repeatedly with one click . The success or failure of the use case execution result is completely judged by the computer , There is no need for human participation . Thanks to the modern unit testing framework , Therefore, the writing of use cases needs to follow the requirements of the test framework .

To sum up ： We think the definition of modern unit testing should be ： A black box test at the software unit level that meets the one click fully automated operation .

2. The value of unit testing

We believe that the unit testing process follows modern unit testing best practices , It can bring the following value to the software team ：

• Unit testing can make software faults be found as soon as possible . According to statistics , The later the software fault is found , Costs are rising exponentially . Good unit testing allows faults to be found at the first time , Avoid more losses caused by the difficulty of locating and repairing the fault left in the later stage .
• Regressability of unit tests , It provides a layer of safety net for the software . This layer of security protection network provides security guarantee for the subsequent reconstruction and modification of the software .
• Unit testing is how software units are used , A code sample user manual document is provided naturally .
• If you can test drive development （Test Driven Development, abbreviation TDD） Unit testing in the way of , Then you can turn unit testing into a design behavior , Can drive a more loosely coupled code design and implementation .

3. Unit test requirements

We believe that qualified unit tests should meet the following requirements ：

• Test cases should be able to run automatically with one click and judge the results automatically ;
• Test cases should not be interdependent and interfere with each other , That is, each use case can run independently ;
• Test cases are reproducible , That is to say, when the tested code remains unchanged , The execution results of test cases should be consistent . Testing should not rely on unstable factors ： For example, timer 、 Thread scheduling and so on ;
• Test cases should be simple and easy to understand , Test cases should be readable , In this way, the test case can be used as an interface document at the same time ;

4. Unit test tools

As the technology matures , Unit testing tools have now become easy to obtain and use . since Kent Beck( Agile software development method leader , Author of extreme programming and test driven development ) by Java Language development and open source JUnit After the framework , It brings unit testing to a new level . Then other languages followed JUnit Launched its own open source unit testing framework . Later, people gave a unified name to this series of frameworks of all programming languages , be called xUnit The test framework .

4.1. judge xUnit Standards for testing frameworks

At present, for any programming language , You can find several open source xUnit The test framework , So how to compare and choose suitable and easy-to-use xUnit What about the framework ？ Generally, it is evaluated from the following dimensions .

• Support automatic detection of registered use cases ： Whether the framework can support simple construction of use cases and automatic registration of test cases into the test framework ;
• Support testing Fixture： That is, whether it supports the establishment of a unified scaffold for a group of test cases , Facilitate the context construction of test cases ;
• Powerful assertion system ： Whether to provide a powerful assertion system , For users to describe expectations in use cases ;
• agile Test Suite Definition ： It can support flexible grouping of test cases ;
• Test ability ： Whether to support exception testing and parameter testing ;
• test filter Definition ： It can support flexible command line parameters , Group and filter the running cases ;
• Test results and report generation ： Whether easy to read test result reports and report files can be generated ;
• Use case dependency management ： Whether it supports editing the dependencies of use cases , Let use cases combine with each other , But without destroying the independence of each use case ;
• Sandbox mode ： Whether the sandbox mode of test cases is supported , Reduce the work of context cleaning for each test case ;
• Open source or not , Including whether the public documents and community support are comprehensive ;

4.2. Main stream C/C++ xUnit Test framework comparison

According to the judgment dimension mentioned above , Let's analyze and compare the current mainstream C/C++ xUnit The test framework .

Test framework features	Boost Test	CppUnit	Gtest	TestNgpp
Open source or not	yes	yes	yes	yes
Automatically detect registration	good	Bad	optimal	optimal
Assertion capability	good	Weak	optimal	optimal
Support Fixture	Support	Support	Support	Support
Support Suite grouping	Support	Support	Support	Support
Support case filtering	Support	Support	Support	Support
Test report	I won't support it	Support	Support	Support
Test ability	good	good	optimal	optimal
Use case dependency management	I won't support it	I won't support it	I won't support it	Support
Sandbox mode	I won't support it	I won't support it	I won't support it	Support
Community usage	low	commonly	It's highly used	commonly

From the above analysis, we can see , Mainstream C++ xUnit Testing frameworks are open source . among TestNgpp The most powerful , But there are fewer users .Google To launch the Gtest The framework is the most widely used , Community support is also the best , In terms of function, it is easy to use , As a starting frame, it is most suitable . Due to various defects, other frameworks are not recommended .

about C ++, There are many established frameworks , Include （ But not limited to, ）,Google Test,Boost.Test,CppUnit,Cute, Many or even more .catch.hpp The main feature is that there is only one header file , It's very simple to join the project to realize unit testing . So in a small project, we can simply use this header file to complete the basic unit test function .

4.3. Mock Frame recommendation

When doing unit tests, you cannot avoid piling piles for the tested code , and mock The main purpose of the frame is to simplify the piling process . Use mock Frameworks make piling code very easy to write , And will not invade the implementation code . For example, two test cases need the same pile function ： The function declaration is the same but the return value is different . In the absence of mock It is very troublesome to solve such problems in the case of framework , and mock The framework can easily deal with such problems .

Mock In addition to the function of piling, the frame , It also provides other more powerful functions . Example how to monitor the user's calling behavior of piling code , And monitor whether these behaviors meet expectations .

about Java language , Usable mock There are various frames , The choice is very wide . But for C++ language , There are only two easy to use mock frame ：gmock and mockcpp. Both are open source software , After use and comparison ,mockcpp Powerful and user experience better than gmock, So there is basically nothing to compare and recommend , If you need to go directly mockcpp Just fine .

5. Unit test process

Based on xUnit The test framework , The process of unit testing for code is generally divided into the following main steps ：

5.1. Construction of unit test environment

This step is to build a unit test environment on each developer's machine . The steps to be taken are as follows ：

• download gtest and mockcpp Source code , according to gtest and mockcpp Build installation manual , Build and install ;
• Build tool chain and directory structure for the current project , Write a build script for unit tests . The script should be able to integrate the tested code with gtest、mockcpp And compile the test case code to build a software program . The script needs to be able to compile with one click 、 Link and execute the generated software program ;
• After the environment is set up , Write some simple examples to test run , Determine the environment installation OK;

5.2. Test the writing process

When the unit test tool has been built on the developer's machine . Then you can unit test the code .

In general use xUnit There are mainly the following processes for unit testing of the framework ：

Create a unit test code file , If it is C++ Words , That is an ordinary cpp The source code file ;

Select the object code to be tested , For example, an interface function or a class . Include the header file of the code to be tested in the test file .

Write test cases in test files , Test cases generally include the following main parts ：

• Prepare the context for the code to be tested . Generally, it is to prepare the initial conditions for the code to be tested to be called , For example, prepare parameters 、 Create class objects, and so on ;
• Call the interface of the tested code , Pass in the corresponding prepared parameters ;
• Write assertions based on observable returns , Describe what you expect to happen correctly . For example, what is the return value of the interface , Or the result of the change that should happen to a certain resource .
• Clean up context . Generally, the context prepared for the test is cleared , This is mainly done for the independence and non-interference of each test , Avoid the next test being affected by the context of the previous test .

After writing the use cases , Call the build script of the test case , Compile and execute use cases , See whether the use case passes .

If the use case fails, it depends on whether it is the use case or the tested code , Fix until the use case passes .

Submit the written use cases and modified code to the code management warehouse .

5.3. Deploy through continuous integration

Generally, a large software team is developed by multiple people , At this time, it will be coordinated through the public code management warehouse . This item ensures the security of each modification of the code , You need to build a continuous integration server . Continuous integration server is installed with continuous integration software （ Open source, for example Jenkins Software ） Machine . The machine will monitor the code management warehouse in real time , Once you find a new code submission , A series of user-defined continuous integration tasks will be triggered （ Take part in the diagram below ）.

  With Jenkins for instance , Common configurable continuous integration tasks include ：

    Compiling and constructing ;
    PCLint Check ;
    Run all unit tests ;
    Code test coverage report generation ;
    Run other automated test cases ： For example, component testing or system testing ;

Due to continuous integration, the server constantly monitors the code management warehouse , Once new code is merged, the corresponding task will be executed immediately ： For example, compile 、 structure 、 Execute all unit test cases, etc . Continuous integration tools support the configuration of result notification , When a task fails, the designated person in charge will be notified by Kanban or email , In this way, once the code submitted by someone causes compilation and construction failure or unit test failure , Will be found immediately . This avoids the low-quality software from entering the code warehouse , Problems that can't be known until very late .
Test coverage statistics

One of the most relevant to unit testing is the generation of test coverage reports . about C/C++, There are not many optional test coverage tools , See the table below .

Tools	platform	Open source or not
Coverage Validator	windows	commercial
OpenCppCoverage	windows	Open source （ Only support VS2013 Above version ）
gcov + lcov	linux	Open source

The test coverage tool is generally installed and deployed on the corresponding machine of continuous integration , So every time the continuous integration server runs the test case , The test coverage results of all codes will be automatically calculated according to the current test run , You can see the coverage of each line of code in detail . The generated report can be automatically published into a web page , Everyone in the project can see .

6. Unit test considerations

Previously, we introduced the tools and implementation process of unit testing , Next, let's take a look at some things we should pay attention to when doing unit testing well .

6.1. Common misconceptions about

In the process of practice , It is found that there are often teams that have developed a large number of unit tests , But the effectiveness of unit testing is very low , Paid a lot of the costs but did not get benefits of the unit testing . After summary, there are mainly the following reasons ：

• Insufficient coverage of exception testing ; We don't need to test all possible inputs of the tested object , But it needs to be divided into equivalence classes , At least one test is required for each equivalent class . A common mistake is to always test only normal scenarios , There are few tests for abnormal scenarios .
• The test is missing assertions ; After each test, you need to use assertions to set the correct expected results . If the assertion is not completely written , Then important checkpoints must be missed , It's equivalent to tearing a hole in the safety net . I have seen some extreme scenes , Developers gather the number of test cases in order to complete the test case index , All use cases are without assertions . In this way, we can see that there are many test cases that have passed the execution , The test coverage is also very good , But they are all invalid use cases .
• Insufficient test design ability , Test coverage is not planned . Ideally, the coverage of each test on the tested code should be orthogonal , Each test case covers a part of the product code , On the whole, it protects all . This requires top-level test design , Especially for the later unit test , The top-level test design can plan priorities and cover key areas . A common misconception is that developers add unit tests separately , But the coverage of important areas is omitted .
• Product code design problem , Physical or logical dependencies are too complex , Unit tests are hard to write . At this time, it is necessary to refactor the original code and supplement unit tests . So whether unit testing can be done well , It's not just about testing , Even if you don't use TDD We also have to optimize the unreasonable design in the product code , To make unit testing more effective .

7. How to reduce the cost of unit testing

In the long run , Reducing the cost of unit testing does not lie in using better unit testing tools , Instead, it is to reduce the frequency of unit tests changing due to changes in the tested code . The reason why software is different from hardware lies in its software , Its existence value is to cope with changes . And the software's variability tends to be more intense as it is transmitted inward , This leads to software unit level design often in the core vortex of change . So I often see some software teams , Once the demand changes quickly, the construction period is tight , Unit testing was soon abandoned . The change of the tested code leads to the change of the unit test, which cannot happen , But we should reduce the probability of this linkage change through design , In this way, the maintenance cost of unit testing can be reduced .

Let unit tests be maintained at a lower cost , There are some things to pay attention to ：

• Unit tests should be black box tests at the unit level as far as possible . White box testing is coupled with code implementation details , Once the code is modified, the test should be changed , Cause repeated workload .
• Before unit testing, we should clarify the coupling relationship of the tested object . If the coupling relationship of the tested object is complex , Then the test case needs to simulate all the coupling relationships of the tested object , In this way, once the dependency of the tested object changes , The test should also be changed together . At this time, it is best to do some decoupling and reconstruction work on the tested code first .
• If possible, try to learn and master TDD How to do it , Try to adopt TDD, Let unit tests drive better code implementation , In turn, it also drives relatively more stable test cases .
• The code design ability of developers determines the fundamental quality of unit testing , It is necessary to continuously improve the software coding ability of developers .

By the visible on , Doing unit testing well is not just about mastering the use of unit testing tools . The ability of developers needs to be improved , It mainly includes ：

• How to design software reasonably , Divide software units , Design low coupling system ;
• How to design automated unit test cases close to the black box level ;
• How to decouple and reconstruct the legacy system ;
• Master and implement TDD The ability of ;
• Design a reasonable continuous integration strategy ;

8. Combine unit testing with overall testing strategy

Unit testing is just a part of software testing strategy , Others are system testing , Integration testing , Component testing, etc . Each level of testing has its value and shortcomings , Therefore, the overall testing strategy needs to focus on how to integrate these testing strategies , Make the overall cost-benefit ratio best . Therefore, it is fundamentally necessary to plan the overall testing strategy based on the overall situation , You can refer to the test quadrant and pyramid model theory of agile testing , Then formulate a reasonable overall testing strategy according to the actual situation of the project .