The significance of code coverage testing to programming white and its application

Active address ： Graduation season · The technique of attack er

ask ： What open source projects have you used that you can't help sharing with friends ？
answer ： Code coverage testing . You are new to the workplace , perky , imbued with a spirit that can conquer mountains and rivers ！ after 4 Years and even 7 Years of painstaking efforts , Study hard , You've already got “ Call beast ” Abbot's biography . Little “ Jingwu Hall ” I can't hold your ordinary heart which is ready to move , So you sneak down the mountain , The abbot will say “ The tiger ”！ finally , You understand a truth ——“ The tiger at the foot of the mountain can't eat people at all , But she will still kill you ！”, The only way to procrastinate is to feed the tiger a food called Money Things that are . But this Money It is not easy to find , The person who owns it needs you to keep moving bricks for him . So you volunteered to join a mysterious organization , And get “ Code the agriculture ” Title . Since then, I have embarked on a road of no return . The so-called newborn calves are not afraid of tigers , With the martial arts learned from Abbot beast calling , You think it's easy to move this little brick . but , Jianghu is always more dangerous than you think . One is too voluminous ！ You move at one time 10 block , Nash moved at one time 15 block , What is more hateful is the old Wang next door , In order to win the hearts of other people's Tigers , I moved in the daytime , Move at night , It also has a foreign digital name 996,007. I don't know , all 007 了 , There is no time to play with tigers ？？？ The second is too realistic , People will not take more because of you Money Go back to the tiger , The tiger is going to eat you , To teach you what sword manual to use will move a lot of bricks faster . But there are good people , Universal Linux Bodhisattva set up a guild , It's called the open source community . Where many kind-hearted people will share their unique sword manual with everyone , Anyone can use . But there is a condition —— If you practice this skill, you must “ autocastration ”................... If not “ autocastration ”, You can also practice this skill ！！！ I don't know , Almost ruined my first intention to go down the mountain . As a prostitute of the guild , I brought you a sword manual from someone else's family today —— Code coverage testing . This tool is so old , He can help you count how many bricks you move in a day , Those bricks were not counted by the supervisor , With it , Mother no longer has to worry about my study ！——“ ha-ha , Can you understand what I said ？ Understand nature and understand ！ I don't understand , I'm getting it ！”

• Don't fart much , Don't talk much ！ Take off your pants , Go straight to the topic ！

Code coverage testing

• Code coverage is the ratio and degree of the code that has been executed in the test process to the total code prepared for the test , It focuses on executing use cases , What code has been executed to , What code has not been executed to . The main meaning is that you can analyze the uncovered part of the code , It is necessary to test whether the design is sufficient in the early stage ？ Whether the code not covered is the blind spot of test design ？ It's demand / The design is not clear enough ？ Or is the test design misunderstood ？

• Code coverage testing helps to find code that cannot be covered by multiple test cases , You can reverse the confusion in code design , Reminder design / Developers sort out the code logic , Improve code quality , At the same time, it also provides a basis for obsolete code .

• Types of code coverage tests

  1. Function coverage ： A function that describes how much proportion the program executes when it runs .
  2. Statement coverage ： A statement that describes how much of a program is executed when it runs .
  3. Branch coverage ： Describes how many branch statements are executed when the program runs （if-else、case、while-do etc. ）.
  4. Conditional coverage ： Condition judgment that describes how much proportion is executed when the program is running （if、while etc. ）.

• At present , stay GNU In the environment , Generally speaking, there are three common tools ：gcov、lcov、gcovr

  • gcov： yes gcc Part of , No separate installation is required , But only plain text code coverage files can be generated ;
  • lcov： Independent of gcc Of , Separate installation required , Cross platform performance is not good , But it can generate HTML Format code coverage file ;
  • gcovr： Independent of gcc Of , Separate installation required , But cross platform is good , The instructions are simple , It can also generate HTML Format code coverage file , Highly recommended ;

gcov

• Introduce ：gcov By gcc Code coverage generation tool provided by tool chain , It can be very convenient and GCC The compiler is used in conjunction with , Usually , Direct installation gcc Tool chain , It also includes gcov Command line tools .

• For the work done by the code coverage tool , It can be simply understood as ： Mark a run in progress , What code has been executed , Which are not implemented . therefore , Even without testing code , You can also get the code coverage by directly running the compiled products . It's just , Usually, the coverage is low .

• gcov And what you get is Text form Of , And different source files need to be executed one by one gcov command , It is inconvenient for large projects , We hope to get more beautiful and easy to browse results .

• Instructions for use ：

  1. adopt gcc Compiler options for （-fprofile-arcs -ftest-coverage） Get coverage information file . there “ Generate coverage information ” The steps of are exactly the same for any coverage analysis tool .
  2. use gcov collect 、 Analyze coverage information files and generate code coverage reports in plain text format .

• Use examples ：

  1. Code under test ：

     /* file_name: profiling.c */
     
     #include <stdio.h>
     #define MAX 50
     
     float sum = 0.1234;
     float A[MAX] = {
             1, 2, 3, 4, 5, 6, 7};       // Array
     
     
     float f(int x)
     {
             
        int i, sum;
     
        sum = 1;
        for (i = 0; i < 100*x; i++)
           sum = sum + i;
        return(sum);
     }
     
     float g(float x)
     {
             
        int i, sum;
     
        sum = 0;
        for (i = 0; i < 10*x; i++)
           sum = sum + f(i);
        return(sum);
     }
     
     int main(int argc, char *argv[])
     {
             
     
        int i, j;
     
         for (i = 0; i < MAX; i = i + 1)
         {
             
             j = i;         
             A[i] = 1/g(i+1);
         }
     
         sum = 0.0;
         for (i = 0; i < MAX; i = i + 1)
         {
             
             sum = sum + A[i];
         }
     
         printf("sum = %f\n",sum);
     }
     

  2. compile ： To generate a coverage information file , You must add -fprofile-arcs -ftest-coverage Compilation options ：

     gcc -fprofile-arcs -ftest-coverage profiling.c
     

     • After the compilation , We will get a **“ reform ” An executable program a.out, When the program is compiled , Some records are inserted in its assembly file The number of executions per line Instructions , And one. The suffix is .gcno** The file of profiling.gcno, It is be gcov One of the key data files referenced .
     • -ftest-coverage： Add... To the source code Record the number of single line code executions Instructions ;
     • -fprofile-arc： Add instructions to the source code to record the execution times of each branch statement .

  3. Run the program ：

     • Run the executable generated in the previous step on the command line ：./a.out
     • After the program runs , Get one profiling.gcda The file of , It and profiling.gcno The same is about to be gcov Referenced data files （ Code coverage information file ）.

  4. Generate code coverage reports

     • Run the coverage generation command on the command line ：

       $ gcov profiling.c
       File 'profiling.c'
       Lines executed:100.00% of 18
       Creating 'profiling.c.gcov'
       

       After execution ,gcov The previous two data files will be referenced （profiling.gcno and profiling.gcda） Give a total coverage value , And generate a code coverage report profiling.c.gcov：

     • In the file profiling.c.gcov View source code lines in 、 Functions 、 Execution times of each branch, etc ：

        		-:    0:Source:profiling.c
               -:    0:Graph:profiling.gcno
               -:    0:Data:profiling.gcda
               -:    0:Runs:1
               -:    1:#include <stdio.h>
               -:    2:
               -:    3:#define MAX 50
               -:    4:
               -:    5:float sum = 0.1234;
               -:    6:float A[MAX] = {
                 1, 2, 3, 4, 5, 6, 7};       // Array
               -:    7:
               -:    8:
           12750:    9:float f(int x)
               -:   10:{
                 
               -:   11:   int i, sum;
               -:   12:
           12750:   13:   sum = 1;
       214000250:   14:   for (i = 0; i < 100*x; i++)
       213987500:   15:      sum = sum + i;
           12750:   16:   return(sum);
               -:   17:}
               -:   18:
              50:   19:float g(float x)
               -:   20:{
                 
               -:   21:   int i, sum;
               -:   22:
              50:   23:   sum = 0;
           12800:   24:   for (i = 0; i < 10*x; i++)
           12750:   25:      sum = sum + f(i);
              50:   26:   return(sum);
               -:   27:}
               -:   28:
               -:   29:
               1:   30:int main(int argc, char *argv[])
               -:   31:{
                 
               -:   32:
               -:   33:   int i, j;
               -:   34:
              51:   35:    for (i = 0; i < MAX; i = i + 1)
               -:   36:    {
                 
              50:   37:        j = i; 
              50:   38:        A[i] = 1/g(i+1);
               -:   39:    }
               -:   40:
               1:   41:    sum = 0.0;
              51:   42:    for (i = 0; i < MAX; i = i + 1)
               -:   43:    {
                 
              50:   44:        sum = sum + A[i];
               -:   45:    }
               -:   46:   
               1:   47:    printf("sum = %f\n",sum);
               -:   48:}
       

       • If a statement is not overwritten （ perform ） To , Will adopt “#####” Symbol to identify .
       • It can be seen that , Direct use gcov The generated code coverage report is not very intuitive .

lcov

• Introduce ：lcov yes gcov The graphical front end of the tool , Collection of multiple source files gcov data , Generate... That describes coverage HTML page . The generated results will include an overview page , Aspect browsing .

• install ：Ubuntu The installation command in the environment is ：sudo apt-get install lcov

• Usage method ：

  1. In the same way , Generate **.gcno and .gcda** Data files ：profiling.gcno and profiling.gcda

  2. Use lcov The command calls the two data files mentioned above , obtain **.info** file ：

     $ lcov --capture --directory . --output-file profiling.info
     Capturing coverage data from .
     Found gcov version: 10.3.0
     Using intermediate gcov format
     Scanning . for .gcda files ...
     Found 1 data files in .
     Processing profiling.gcda
     Finished .info-file creation
     
     $ ls
     a.out  profiling.c  profiling.c.gcov  profiling.gcda  profiling.gcno  profiling.info
     

     • Or in short ：lcov -c -d . -o profiling.info

  3. Use genhtml take coverage.info Turn into HTML file , Coexist in the specified directory ：

     $ genhtml profiling.info -o html
     Reading data file ./profiling.info
     Found 1 entries.
     Found common filename prefix "/home/leon1124"
     Writing .css and .png files.
     Generating output.
     Processing file gcov/profiling.c
     Writing directory view page.
     Overall coverage rate:
       lines......: 100.0% (18 of 18 lines)
       functions..: 100.0% (3 of 3 functions)
     $ ls
     a.out  html  profiling.c  profiling.c.gcov  profiling.gcda  profiling.gcno  profiling.info
     
     $ cd html;ls
     amber.png    gcov      glass.png          index-sort-l.html  ruby.png  updown.png
     emerald.png  gcov.css  index-sort-f.html  index.html         snow.png
     

     

• lcov Introduction to common command options （ Can also pass –help Check it out. ）

  • -c perhaps --capture ： Specifies that coverage information is collected from the compiled product .

  • -d DIR perhaps --directory DIR ： Specify the path to the compiled product .

  • -e FILE PATTERN perhaps --extract FILE PATTERN ： From the specified file according to PATTERN Filter results .

  • -o FILENAME perhaps --output-file FILENAME ： Specify the file name of the coverage output .

  • –rc lcov_branch_coverage=1 ： Turn on branch coverage calculation （lcov Branch coverage is not turned on by default ）

    • Be careful ： In order to make html The report format also reflects the branch coverage results , In execution genhtml On command , You also need to add this option ：

      $ lcov -c -d . --rc lcov_branch_coverage=1 -o profiling.info
      $ genhtml --rc lcov_branch_coverage=1 -o html ./profiling.info
      Reading data file ./profiling.info
      Found 1 entries.
      Found common filename prefix "/home/leon1124"
      Writing .css and .png files.
      Generating output.
      Processing file gcov/profiling.c
      Writing directory view page.
      Overall coverage rate:
        lines......: 100.0% (18 of 18 lines)
        functions..: 100.0% (3 of 3 functions)
        branches...: 100.0% (8 of 8 branches)
      

      

gcovr

• gcovr It's about C/C++ Code coverage and support in multiple ways （ Including list mode 、XML File mode 、HTML Web page mode, etc ） The tools shown , It can be used by some integration tools to parse .

• gcovr Yes, it is Python Written , This means that as long as there is Python The environment can be used gcovr, Whether it's WINDOWS still LINUX.

• install ：

  • Python Environmental use ：pip install gcovr
  • Linux Environmental use ：sudo apt install gcovr

• Usage method ：

  1. Compile and generate **.gcda** Data files ：

     $ gcc --coverage -g -O0 profiling.c
     $ ls
     a.out  profiling.c  profiling.gcno
     

  2. Run the generated executable , obtain **.gcda** file ：

     $ ./a.out
     sum = 0.000001
     $ ls
     a.out  profiling.c  profiling.gcda  profiling.gcno
     

  3. Run directly under the current directory gcovr command , Get coverage report results in text form ：

     $ gcovr
     ------------------------------------------------------------------------------
                                GCC Code Coverage Report
     Directory: .
     ------------------------------------------------------------------------------
     File                                       Lines    Exec  Cover   Missing
     ------------------------------------------------------------------------------
     profiling.c                                   18      18   100%
     ------------------------------------------------------------------------------
     TOTAL                                         18      18   100%
     ------------------------------------------------------------------------------
     

  4. By adding options , send gcovr The command to get html Coverage report results in the form of (coverage.html coverage.profiling.c.html)：

     $ gcovr --html-details -o coverage.html
     a.out  coverage.html  coverage.profiling.c.html  profiling.c  profiling.gcda  profiling.gcno
     

     

     ---

     

• Common compilation options ：

  • -r RootPath perhaps --root RootPath, namely r Followed by the root directory of your project , The default is ’.', The current directory .

  • -b perhaps --branches Report as branch coverage .

    $ gcovr -b
    ------------------------------------------------------------------------------
                               GCC Code Coverage Report
    Directory: .
    ------------------------------------------------------------------------------
    File                                    Branches   Taken  Cover   Missing
    ------------------------------------------------------------------------------
    profiling.c                                    8       8   100%
    ------------------------------------------------------------------------------
    TOTAL                                          8       8   100%
    ------------------------------------------------------------------------------
    

  • -x perhaps --xml The format of the specified report is XML.

    $ gcovr --xml -o coverage.xml
    

  • -o OUTPUT perhaps --output OUTPUT Specify the file name of the coverage output .

  • –html The format of the specified report is HTML.

Active address ： Graduation season · The technique of attack er