Testing with Pytest
Pytest is a testing framework for Python. It makes testing very easy, without a lot of boilerplate. Its advantages lie in its simplicity, automatic test discovery, modular fixtures and intelligent error output.
This blog post is intended to provide an overview and easy introduction to testing with Pytest. It first gives an overview on the use of Pytest, then provides details on how to make resources available for the tests and concludes with a few tips and tricks.
Pytest greatly simplifies the writing of tests. The only requirement is the assert statement included in Python. In order for Pytest to be able to find the test, the
test_ prefix is placed in front of the test function:
assert 1 > 0
That’s all that is needed for testing!
After installation via pip (
pip install pytest) or through the package manager of choice, Pytest can basically be started in two ways:
- using the filename as the argument:
- without argument. In this case, Pytest searches for all files with the format *\*\_test.py* or *test\_\*.py* and tests these
If the test is successful, an overview of the test result is issued:
======= test session starts =======
platform linux -- Python 3.5.2, pytest-3.0.5, py-1.4.31, pluggy-0.4.0
>rootdir: /home/sh/pytest_blog, inifile:
>collected 1 items
======= 1 passed in 0.00 seconds =======
But Pytest shows its true strength with tests that do not work, by providing detailed output on exactly what has not passed, as far as possible. A good example is that of two lists which do not contain the same elements:
assert [1, 2] == [1, 2, 3]
> assert [1, 2] == [1, 2, 3]
E assert [1, 2] == [1, 2, 3]
E Right contains more items, first extra item: 3
E Use -v to get the full diff
=============== 1 failed in 0.01 seconds ================
As you can see, Pytest immediately delivers the element which is causing the error!
The following are some of the important options of Pytest, which offer significant assistance with using Pytest:
-sprevents the capturing of input/output. This is very important, as the output of print statements is not displayed in normal mode or the debugger cannot be started otherwise
-k <string>is used to filter tests. Starts only those tests which contain
stringas a substring
-xterminates on the first failed test
Standard values for Pytest can be configured using the pytest.ini file.
Tests frequently require some resources, which need to be available for the test. Examples of this are database connections, config files, the browser for UI tests etc. Pytest provides the concept of fixtures for this purpose. Fixtures can be envisioned as having some similarities with
tearDown() functions of familiar Unit Test Frameworks. However, Pytest fixtures can be loaded much more dynamically on a per test basis: to use a fixture in a test, it simply has to be specified as an argument in the function:
browser.visit(foo) # browser is now a loaded fixture for this test
@pytest.fixture is used to create a fixture. The required value is returned with
yield. This makes it possible for the fixture to be removed again after the yield statement. An example for clarification:
1 import os
2 import sqlite3
4 import pytest
7 def db(tmpdir):
8 file = os.path.join(tmpdir.strpath, "test.db")
10 conn = sqlite3.connect(file)
11 conn.execute("CREATE TABLE blog (id, title, text)")
13 yield conn
17 def test_entry_creation(db):
18 query = ("INSERT INTO blog "
19 "(id, title, text)"
20 "VALUES (?, ?, ?)")
21 values = (1,
23 "This is a blog entry")
25 db.execute(query, values)
Here, the fixture provides a DB connection for the test and also creates a table which is required for testing.
Lines 6 and 7 are of interest: In line 6, the
scope parameter is used to specify the frequency with which a fixture is run.
session are available. If the
module scope is selected, for example, the fixture for the entire module is created only once and then reused for each test. This is very useful for resources that do not necessarily have to be recreated for each test and can therefore be reused.
In line 7, an additional fixture is loaded in our fixture. This is very straightforward. In this example, this a fixture that is supplied by Pytest. It returns a temporary directory that is unique for each test call.
Other important features
Tests with different parameters can be created with the decorator
test_entry_creation test used above is expanded as an example:
(1, "House Stark", "Winter is coming"),
(2, "House Lannister", "Hear me Roar"),
(3, "House Martell", "Unbowed, Unbent, Unbroken")
def test_parametrized_entry_creation(id, title, text, db):
query = ("INSERT INTO blog "
"(id, title, text)"
"VALUES (?, ?, ?)")
values = (id, title, text)
Three tests are now created, each with a different set of parameters.
Skipping, failing and marking
Tests can be marked in different ways:
@pytest.mark.skip(reason="Not implemented yet")causes Pytest to skip the test
@pytest.mark.skipif(not os.environ.get('CI', False))causes Pytest to skip the test if the condition is met
@pytest.mark.xfail(reason="Can't work yet")expects a test to fail
@pytest.mark.abcother marker. This makes it possible to filter tests when they are called. Example: run all tests that are marked with abc:
pytest -m abc. The
notoperator can be used to run all tests which are *not* marked with abc:
pytest -m 'not abc'
If exceptions are expected from the code tested, this is handled in Pytest using a
Autouse of fixtures
Finally, a fixture can be passed as a parameter in decorator so it is automatically available in every test. The following code could be used an example for a login taking place for every test in the browser:
def session(login_page, browser):
And last but not least…
At the time of this post, there are somewhat more than 260 Plugins available for Pytest. They extend Pytest in a wide range of aspects, from amusing extensions such as the emoji Plugin, which makes the test output a bit more fun, to advanced checks such as import order (Isort) and Syntax Linting mit Flake8, to extensions such as the integration of Selenium or Splinter. If a feature is missing in Pytest, it is definitely worth taking a look at the plugin list!
All in all, Pytest makes testing so easy that there are no excuses for tests to be neglected!