pytest-idempotent
Pytest plugin for testing the idempotency of a function.
Usage
pip install pytest-idempotent
Documentation
Suppose we had the following function, that we (incorrectly) assume is idempotent (AKA we should be able to run it more than once without any adverse effects).
from pytest_idempotent import idempotent # or use your own decorator!
@idempotent
def func(x: list[int]) -> None:
x += [9]
Note: this function is not idempotent because calling it on the same list x grows the size of x by 1 each time.
We can write an idempotency test for this function as follows:
# idempotency_test.py
import pytest
@pytest.mark.test_idempotency
def test_func() -> None:
x: list[int] = []
func(x)
assert x == [9]
Adding the @pytest.mark.test_idempotency mark automatically splits this test into two - one that tests the regular behavior and one that tests that the function can be called twice without adverse effects.
❯❯❯ pytest
================= test session starts ==================
platform darwin -- Python 3.9.2, pytest-6.2.5
collected 2 items
tests/idempotency_test.py .F [100%]
===================== FAILURES ========================
------------- test_func[idempotency-check] -------------
@pytest.mark.test_idempotency
def test_func() -> None:
x: list[int] = []
func(x)
> assert x == [9]
E assert [9, 9] == [9]
E Left contains one more item: 9
E Use -v to get the full diff
tests/idempotency_test.py:19: AssertionError
=============== short test summary info ================
FAILED tests/idempotency_test.py::test_func[idempotency-check]
- assert [9, 9] == [9]
============= 1 failed, 1 passed in 0.16s ==============
How It Works
Idempotency is a difficult pattern to enforce. To solve this issue, pytest-idempotent takes the following approach:
-
Introduce a decorator,
@idempotent, to functions.- This decorator serves as a visual aid. If this decorator is commonly used in the codebase, it is much easier to consider idempotency for new and existing functions.
- At runtime, this decorator is a no-op.
- At test-time, if the feature is enabled, we will run the decorated function twice with the same parameters in all test cases.
-
For all tests marked with
@pytest.mark.test_idempotency, we run each test twice: once normally, and once with the decorated function called twice.- Both runs need to pass all assertions.
- We return the first result because the first run should complete the processing. The second will either return exact the same result or be a no-op.
- We can also assert that the second run returns the same result as an additional parameter.
@idempotent decorator
By default, the @idempotent decorator does nothing during runtime. We do not want to add overhead to production code to run tests.
from typing import Any, Callable, TypeVar
_F = TypeVar("_F", bound=Callable[..., Any])
def idempotent(func: _F) -> _F:
"""
No-op during runtime.
This marker allows pytest-idempotent to override the decorated function
during test-time to verify the function is idempotent.
"""
return func
To use your own @idempotent decorator, you can override the pytest_idempotent_decorator function in your conftest.py to return the module path to your implementation.
# conftest.py
# You can define this to ensure the plugin is correctly installed
pytest_plugins = ["pytest_idempotent"]
def pytest_idempotent_decorator() -> str:
"""This links to my custom implementation of @idempotent."""
return "src.utils.idempotent"
12 Dec 4, 2022
1 Jan 22, 2022
1.1k Dec 31, 2022
218 Jan 3, 2023
174 Dec 16, 2022
191 Dec 21, 2022
10 May 23, 2022
170 Dec 02, 2022
963 Dec 28, 2022
27 Jan 07, 2023
75 Dec 19, 2022
3 Jan 18, 2022
3.7k Jan 04, 2023
31 Dec 24, 2022
1 Nov 29, 2021
14 Jan 03, 2023
226 Dec 30, 2022
14 Dec 06, 2022
32 Jul 23, 2022
70 Jan 03, 2023
3.3k Jan 05, 2023
0 Oct 01, 2022
2 Nov 23, 2021
941 Jan 03, 2023
1 Nov 11, 2021
3 Dec 04, 2021