Python testing with Pytest: Order of test functions - fixtures
After getting started with pytest we will soon have a few questions. For example the order of the tests and how to factor out common code from the test functions.
Printing from tests
By default pytest will hide any output generated during the tests. So if you have a print statement in our code, we will not see its output during normal test run.
examples/python/pt2/test_some.py
import sys def test_hello(): print("hello testing") sys.stderr.write("stderr during testing\n")
We can use the -s option to tell pytest to display the output that should have gone to the standard output (stdout) or standerd error (stderr):
$ pytest test_some.py -s ===================== test session starts ====================== platform darwin -- Python 3.5.2, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 rootdir: /examples/python/pt2, inifile: collected 1 items test_some.py hello testing stderr during testing . =================== 1 passed in 0.02 seconds ===================
In order to avoid mixing with the regular output of pytest one might need to to add a newline infront of each string being printed.
Order of test function
Pytest runs the test in the same order as they are found in the test module:
examples/python/pt2/test_order.py
def test_one(): assert True def test_two(): assert True def test_three(): assert True
We can see the actual order by including the verbose flag on the command line: -v
$ pytest test_order.py -v ===================== test session starts ====================== platform darwin -- Python 3.5.2, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 -- /examples/python/pt/venv3/bin/python3.5 cachedir: .cache Using --random-order-bucket=module Using --random-order-seed=771694 rootdir: /examples/python/pt2, inifile: plugins: random-order-0.5.4 collected 3 items test_order.py::test_one PASSED test_order.py::test_two PASSED test_order.py::test_three PASSED =================== 3 passed in 0.01 seconds ===================
Randomizing tests
While normally you'd probably want to run simpler towards more complex test, sometimes it is useful to run the tests in random order. That might reveal some interdependency among tests.
In order to do this we can install the pytest-random-order plugin using pip install pytest-random-order
Once we install it, it is immediately enabled and if we run pytest -v several times we'll see the order changing.
Default order
In order to get back to the default order we can turn off the plugin we've just installed. For that we need to add the following code to our test file:
import pytest pytestmark = pytest.mark.random_order(disabled=True)
examples/python/pt2/test_default_order.py
import pytest pytestmark = pytest.mark.random_order(disabled=True) def test_one(): assert True def test_two(): assert True def test_three(): assert True
Fixture - setup and teardown
Unless we are testing something really simple, we will need to set up some environment before we can check the actual functionality. This set up can be as simple as creating an instance object, but it can also involve setting up one ore databases, servers etc.
This setup is usually called fixture among the testers.
In any case if several test functions need the same setup we can factor that out. To make this standardized, pytest designated a pair of functions names for this. If we implement the function called setup_function it will be called before every test function and if implement the teardown_function then that will be called after every test function.
examples/python/pt2/test_fixture.py
def setup_function(): print("\nsetup_function") def teardown_function(): print("\nteardown_function") def test_one(): assert True print("test_one") def test_two(): assert True print("test_two") def test_three(): assert True print("test_three")
The execution will show the order:
$ pytest test_fixture.py -s ===================== test session starts ====================== platform darwin -- Python 3.5.2, pytest-3.0.7, py-1.4.33, pluggy-0.4.0 Using --random-order-bucket=module Using --random-order-seed=100711 rootdir: /examples/python/pt2, inifile: plugins: random-order-0.5.4 collected 3 items test_fixture.py setup_function test_one . teardown_function setup_function test_three . teardown_function setup_function test_two . teardown_function =================== 3 passed in 0.02 seconds ===================
Published on 2018-04-09