Pytest parameterization some tips you don't know / pytest you don't know

Preface

unittest Unit test framework uses DDT Do data-driven testing , So as a more powerful and flexible Pytest How can a framework not have the concept of data-driven ？ Actually Pytest It's using @pytest.mark.parametrize Decorator to achieve data-driven testing , So today, let's talk about how it does data-driven testing .

Decoration test class

"""
import pytest

data_1 = [
    (1, 2, 3),
    (4, 5, 9)
]


def add(a, b):
    return a + b


@pytest.mark.parametrize('a, b, expect', data_1)
class TestParametrize(object):

    def test_parametrize_1(self, a, b, expect):
        print('\n Test functions 1 The test data is \n{}-{}'.format(a, b))
        assert add(a, b) == expect

    def test_parametrize_2(self, a, b, expect):
        print('\n Test functions 2 The data is \n{}-{}'.format(a, b))
        assert add(a, b) == expect


if __name__ == '__main__':
    pytest.main(['-sv'])

Output

collecting ... collected 4 items

test_parametrize.py::TestParametrize::test_parametrize_1[1-2-3] 
 Test functions 1 The test data is 
1-2
PASSED
test_parametrize.py::TestParametrize::test_parametrize_1[4-5-9] 
 Test functions 1 The test data is 
4-5
PASSED
test_parametrize.py::TestParametrize::test_parametrize_2[1-2-3] 
 Test functions 2 The data is 
1-2
PASSED
test_parametrize.py::TestParametrize::test_parametrize_2[4-5-9] 
 Test functions 2 The data is 
4-5
PASSED

========================== 4 passed in 0.21 seconds ===========================

Process finished with exit code 0

explain

When decorators decorate test classes , The data set will be passed to all methods of the class

Decorate test functions

Single data

import pytest

data = [1, 2]


@pytest.mark.parametrize('a', data)
def test_parametrize(a):
    print('\n The loaded test data is \n{}'.format(a))


if __name__ == '__main__':
    pytest.main(['-s'])

Output

============================= test session starts =============================
platform win32 -- Python 3.7.2, pytest-4.3.1, py-1.8.0, pluggy-0.9.0
rootdir: E:\CnblogCode\pytest_parametrize, inifile:
plugins: rerunfailures-7.0, metadata-1.8.0, html-1.20.0
collected 2 items

test_parametrize.py 
 The loaded test data is 
1
.
 The loaded test data is 
2
.

========================== 2 passed in 0.16 seconds ===========================

Process finished with exit code 0

explain

When a test case only needs one parameter , We store data in a list of unordered nested sequences ,@pytest.mark.parametrize('a', data) The first argument to the decorator also requires only one variable to receive each element in the list , The second parameter passes the list of stored data , Then the test case needs to receive the test data using a string with the same name ( In the instance a) And how many elements of the list will generate and execute how many test cases

A set of data

import pytest

data = [
    [1, 2, 3],
    [4, 5, 9]
]  #  List nested list 
# data_tuple = [
#     (1, 2, 3),
#     (4, 5, 9)
# ]  #  List nested tuples 


@pytest.mark.parametrize('a, b, expect', data)
def test_parametrize_1(a, b, expect):  #  A parameter receives a data 
    print('\n The test data is \n{},{},{}'.format(a, b, expect))
    actual = a + b
    assert actual == expect


@pytest.mark.parametrize('value', data)
def test_parametrize_2(value):  #  A parameter receives a set of data 
    print('\n The test data is \n{}'.format(value))
    actual = value[0] + value[1]
    assert actual == value[2]


if __name__ == '__main__':
    pytest.main(['-s'])

Output

============================= test session starts =============================
platform win32 -- Python 3.7.2, pytest-4.3.1, py-1.8.0, pluggy-0.9.0
rootdir: E:\CnblogCode\pytest_parametrize, inifile:
plugins: rerunfailures-7.0, metadata-1.8.0, html-1.20.0
collected 4 items

test_parametrize.py 
 The test data is 
1,2,3
.
 The test data is 
4,5,9
.
 The test data is 
[1, 2, 3]
.
 The test data is 
[4, 5, 9]
.

========================== 4 passed in 0.17 seconds ===========================

Process finished with exit code 0

explain

When a test case needs multiple data , We can use nested sequences ( Nested tuples & Nested list ) To store test data

Decorator @pytest.mark.parametrize() You can use a single variable to receive data , Multiple variables can also be used to receive , Again , Test case functions also need to be consistent with them

When receiving with a single variable , When the test data is passed inside the test function, it is every element or small list in the list , You need to index every data

When multiple variables are used to receive data , Then each variable receives each element in a small list or tuple

How many groups of small lists or tuples are nested in a list , Test how many test cases are generated

Diagram correspondence

Combined data

"""
import pytest

data_1 = [1, 2]  
data_2 = [3, 4]  


@pytest.mark.parametrize('a', data_1)
@pytest.mark.parametrize('b', data_2)
def test_parametrize_1(a, b):
    print('\n The test data is \n{},{}'.format(a, b))


if __name__ == '__main__':
    pytest.main(['-s'])

Output

============================= test session starts =============================
platform win32 -- Python 3.7.2, pytest-4.3.1, py-1.8.0, pluggy-0.9.0
rootdir: E:\CnblogCode\pytest_parametrize, inifile:
plugins: rerunfailures-7.0, metadata-1.8.0, html-1.20.0
collected 4 items

test_parametrize.py 
 The test data is 
1,3
.
 The test data is 
2,3
.
 The test data is 
1,4
.
 The test data is 
2,4
.

========================== 4 passed in 0.24 seconds ===========================

Process finished with exit code 0

explain

Pass the test results , It's not difficult for us to analyze , A test function can also be decorated by multiple parameterized decorators at the same time , Then the data in multiple decorators will be cross combined and passed to the test function , And then generate n*n Test cases , This also provides convenience for our test design

Tag use cases

You can mark test cases directly , Parametric decorators can also identify ( Mark use case failure or skip )

Marked as unconditionally skipped ( Mark failure as xfail, Try it yourself )

"""
import pytest

data_1 = [
    [1, 2, 3],
    pytest.param(3, 4, 8, marks=pytest.mark.skip)
]


def add(a, b):
    return a + b


@pytest.mark.parametrize('a, b, expect', data_1)
def test_parametrize_1(a, b, expect):
    print('\n The test data is \n{},{}'.format(a, b))
    assert add(a, b) == expect


if __name__ == '__main__':
    pytest.main(['-vs'])

Output

============================= test session starts =============================
platform win32 -- Python 3.7.2, pytest-4.3.1, py-1.8.0, pluggy-0.9.0 -- C:\Programs\Python\Python37-32\python.exe
cachedir: .pytest_cache
metadata: {'Python': '3.7.2', 'Platform': 'Windows-7-6.1.7601-SP1', 'Packages': {'pytest': '4.3.1', 'py': '1.8.0', 'pluggy': '0.9.0'}, 'Plugins': {'rerunfailures': '7.0', 'metadata': '1.8.0', 'html': '1.20.0'}, 'JAVA_HOME': 'D:\\JDK'}
rootdir: E:\CnblogCode\pytest_parametrize, inifile:
plugins: rerunfailures-7.0, metadata-1.8.0, html-1.20.0
collecting ... collected 2 items

test_parametrize.py::test_parametrize_1[1-2-3] 
 The test data is 
1,2
PASSED
test_parametrize.py::test_parametrize_1[3-4-8] SKIPPED

===================== 1 passed, 1 skipped in 0.17 seconds =====================

Process finished with exit code 0

explain

The output shows that 2 A use case , One passes , One was skipped , When we don't want to execute a certain set of test data , We can mark skip or skipif; When we expect a set of data to fail , We can mark it as xfail etc.

Nested Dictionary

"""
import pytest

data_1 = (
    {
        'user': 1,
        'pwd': 2
     },
    {
        'user': 3,
        'pwd': 4
    }
)


@pytest.mark.parametrize('dic', data_1)
def test_parametrize_1(dic):
    print('\n The test data is \n{}'.format(dic))


if __name__ == '__main__':
    pytest.main(['-s'])

Output

============================= test session starts =============================
platform win32 -- Python 3.7.2, pytest-4.3.1, py-1.8.0, pluggy-0.9.0
rootdir: E:\CnblogCode\pytest_parametrize, inifile:
plugins: rerunfailures-7.0, metadata-1.8.0, html-1.20.0
collected 2 items

test_parametrize.py 
 The test data is 
{'user': 1, 'pwd': 2}
.
 The test data is 
{'user': 3, 'pwd': 4}
.

========================== 2 passed in 0.20 seconds ===========================

Process finished with exit code 0

Increase readability

Use ids Parameters

The parametric decorator has an extra parameter ids, Each test case can be identified , Customize the display of test data results , To increase readability , We can mark the test data used by each test case , Add some descriptions appropriately

You need to know before using ,ids The parameter should be a list of strings , Must be consistent with the length of the data object list , We can try to use ids, Look at the effect

"""
import pytest

data_1 = [
    (1, 2, 3),
    (4, 5, 9)
]


ids = ["a:{} + b:{} = expect:{}".format(a, b, expect) for a, b, expect in data_1]

def add(a, b):
    return a + b


@pytest.mark.parametrize('a, b, expect', data_1, ids=ids)  
class TestParametrize(object):

    def test_parametrize_1(self, a, b, expect):
        print('\n Test functions 1 The test data is \n{}-{}'.format(a, b))
        assert add(a, b) == expect

    def test_parametrize_2(self, a, b, expect):
        print('\n Test functions 2 The data is \n{}-{}'.format(a, b))
        assert add(a, b) == expect


if __name__ == '__main__':
    pytest.main(['-v'])  # -v ：  More detailed output test results 

Output

Decorator does not pass ids Output of parameters

collecting ... collected 4 items

test_parametrize.py::TestParametrize::test_parametrize_1[1-2-3] PASSED   [ 25%]
test_parametrize.py::TestParametrize::test_parametrize_1[4-5-9] PASSED   [ 50%]
test_parametrize.py::TestParametrize::test_parametrize_2[1-2-3] PASSED   [ 75%]
test_parametrize.py::TestParametrize::test_parametrize_2[4-5-9] PASSED   [100%]

========================== 4 passed in 0.16 seconds ===========================

Process finished with exit code 0

Decorator transfer ids Output of parameters

collecting ... collected 4 items

test_parametrize.py::TestParametrize::test_parametrize_1[a:1 + b:2 = expect:3] PASSED [ 25%]
test_parametrize.py::TestParametrize::test_parametrize_1[a:4 + b:5 = expect:9] PASSED [ 50%]
test_parametrize.py::TestParametrize::test_parametrize_2[a:1 + b:2 = expect:3] PASSED [ 75%]
test_parametrize.py::TestParametrize::test_parametrize_2[a:4 + b:5 = expect:9] PASSED [100%]

========================== 4 passed in 0.20 seconds ===========================

Process finished with exit code 0

explain

To execute the command, I used -v, The output results will be displayed in more detail , You can see that the use cases in all data results are clearly marked by a list , And it can be seen more intuitively through this kind of mark , The data name and test content used in each test case

Customize id Make a mark

Besides using ids Parameters increase output readability , We can also define a next to the parameter in the parameter list id Value for identification , See the following example

"""
import pytest

data_1 = [
    pytest.param(1, 2, 3, id="(a+b):pass"),  # id The value of can be customized ,  As long as it is convenient to understand what each use case does 
    pytest.param(4, 5, 10, id="(a+b):fail")
]


def add(a, b):
    return a + b


class TestParametrize(object):

    @pytest.mark.parametrize('a, b, expect', data_1)
    def test_parametrize_1(self, a, b, expect):
        assert add(a, b) == expect


if __name__ == '__main__':
    pytest.main(['-v'])

Output

test_parametrize.py::TestParametrize::test_parametrize_1[(a+b):pass] PASSED [ 50%]
test_parametrize.py::TestParametrize::test_parametrize_1[(a+b):fail] FAILED [100%]

================================== FAILURES ===================================
_______________ TestParametrize.test_parametrize_1[(a+b):fail] ________________

self = <pytest_parametrize.test_parametrize.TestParametrize object at 0x000001D7BFC4C748>
a = 4, b = 5, expect = 10

    @pytest.mark.parametrize('a, b, expect', data_1)
    def test_parametrize_1(self, a, b, expect):
>       assert add(a, b) == expect
E       assert 9 == 10
E         -9
E         +10

test_parametrize.py:28: AssertionError
===================== 1 failed, 1 passed in 0.35 seconds ======================

Process finished with exit code 0

explain

If you use this method to mark test cases , Be sure to use it in strict accordance with the format I write , The grammar is pytest.param(value, id='somthing')

summary

Pytest That's how data-driven is implemented in

master

1. Decorators and test cases use a single variable to receive multiple groups of data and multiple variables to receive multiple data access methods

2. Different test data forms ( List nested tuples , list , Dictionary, etc ) when , How to transfer and access data

3. Decorators decorate the differences between test classes and test functions ： When decorating a test class , All methods in the class must receive and send test data , Otherwise, an error will be reported , It's flexible when decorating test functions , If functions don't use data, they don't need decoration

4. For readability of output results , You can choose to use ids Parameters , And defined in test data id Parameter values to identify test cases

Be careful

1. The first parameter of the decorator is a string parameter in the form of a list "a, b, c" Can not write "a", "b", "c"

2. ids It's a list of strings , Its length should be consistent with the length of the test data list

Finally, that's all for today's article , Favorite friends can like collection comments and pay attention .