What is pytest

pytest is a Python testing framework and a tool for testing Python code. Below is a description of pytest's main features.

• Simple and easy to use
  pytest allows you to write tests in concise, easy-to-read code. In addition, features such as automatic test detection and parameterization make it easier to write tests.

• Fixture
  pytest provides a feature called fixture, which is used to perform test pre-processing, post-processing, etc., to reduce duplication of test code.

• Plugin architecture
  pytest employs a plugin architecture that allows users to create their own plugins. This allows them to extend pytest's functionality and add features appropriate to their own projects.

• Test runner
  pytest provides a runner to run multiple tests. The test runner collects tests, executes them, and generates reports.

• Various test styles
  pytest supports a variety of test styles. For example, it supports unittest-style testing, BDD-style testing, doctest-style testing, etc.

• Multilingual support
  pytest supports tests not only in Python, but also in other languages. For example, there are plugins to run C and C++ tests.

• Test coverage
  pytest also provides a plugin for measuring test coverage. Test coverage is a measure of which parts of the code have been tested. The higher the test coverage, the higher the quality of the code.

• Class and method level scopes
  pytest allows for flexible scoping of tests. For example, you can define class-level fixtures, method-level tests, etc.

• Execute tests in parallel
  pytest can run tests in parallel. This can reduce test execution time.

• Integration with external test runners
  pytest can be integrated with external test runners. For example, it can be integrated with CI/CD tools such as Jenkins or Travis CI.

• Parameterized testing
  pytest supports parameterized tests. Parameterized testing is a method of exhaustive testing by running the same test against multiple input values. pytest allows you to define parameterized tests using the @pytest.mark.parametrize decorator.

• Skipping tests and setting skip conditions
  pytest allows you to skip tests. You can also specify skip conditions. This is useful, for example, if you want to run tests only on a specific OS or Python version.

• Hook function
  pytest allows hook functions to be executed before or after tests are run. This allows for flexibility in setting up pre- and post-processing of tests.

• Custom assertions
  pytest allows you to define your own assertions. This allows you to define assertions tailored to the specifications of your application or library.

• Excellent reporting
  pytest provides extensive reporting capabilities. The results of test runs can be output in HTML, XML, JSON, or other formats, allowing you to visually verify test results.

pytest is widely used for testing Python applications and libraries and is supported by many developers. pytest can also be integrated with other Python testing frameworks (e.g. unittest and nose).

How to use pytest

To run tests on your Python project using pytest, follow these steps

1. Install pytest
2. Create a test file
3. Run the test
4. Use options

Install pytest

Install pytest with the following command

$ pip install pytest

Create a test file

Test files must have a filename beginning with "test*" and ending with .py. Also, the test function must begin with "test*". For example, the following test file is created.

def test_addition():
    assert 1 + 2 == 3

def test_subtraction():
    assert 5 - 3 == 2

Run the test

On the command line, run the pytest command as follows: pytest will automatically find and run all test files under the current directory.

$ pytest

If the test runs successfully, the following output is displayed.

============================= test session starts =============================
platform win32 -- Python 3.7.6, pytest-6.2.2, py-1.10.0, pluggy-0.13.1
rootdir: C:\projects\pytest
collected 2 items

test_sample.py ..                                                       [100%]

============================== 2 passed in 0.04s ==============================

In this example, two tests were run and all passed.

Use options

pytest offers a variety of options. For example, you can change the report format by specifying the following options

$ pytest --junitxml=junit.xml

This example writes the test results to a file called junit.xml. This file can be used by CI/CD tools.

Parameterize the test

pytest allows you to run the same test multiple times. For example, you can run the same test multiple times using different arguments. For example, the same test can be run multiple times using different arguments as follows

import pytest

@pytest.mark.parametrize("dividend, divisor, expected_quotient", [(10, 2, 5), (12, 3, 4)])
def test_division(dividend, divisor, expected_quotient):
    assert dividend / divisor == expected_quotient

This example uses the @pytest.mark.parametrize decorator to specify three arguments (dividend, divisor, and expected_quotient), each with a tuple of values. The test is run multiple times using the value of each tuple.

fixture

pytest provides a feature called fixture to increase reusability of test code. fixture can define pre-processing and post-processing of test code and can provide objects to be used in the test code.

A fixture is defined using the pytest.fixture decorator. The following is an example of a simple fixture

import pytest

@pytest.fixture
def my_fixture():
    preprocess() # preprocess test code within this function
    yield obj # obj is the object used in the test code
    postprocess() # post-process the test code within this function

This example defines a fixture named my_fixture. This fixture returns an object named obj in the yield statement. The test code can take my_fixture as an argument.

def test_my_test(my_fixture):
    # test using the object provided by my_fixture

This example defines a test code named test_my_test. The test code takes a fixture named my_fixture as an argument. The object provided by my_fixture can be used for testing.

The fixture is automatically called before pytest executes the test code to preprocess the test code. After the test code is completed, fixture automatically performs post-processing. fixture can share objects between test codes, which can increase code reusability.

Parameterize the fixture

Fixture can also be parameterized. This allows multiple objects with different parameters to be returned.

import pytest

@pytest.fixture(params=[1, 2, 3])
def my_fixture(request):
    return request.param

In this example, a fixture named my_fixture is defined. This fixture can have three parameters: 1, 2, and 3, using the params option. The test code can take my_fixture as an argument. This fixture returns an object for each parameter.

def test_my_test(my_fixture):
    assert my_fixture > 0

This example defines a test code named test_my_test. The test code takes a fixture named my_fixture as an argument. Since my_fixture returns an object for each of the three parameters, 1, 2, and 3, the test code is executed three times.

Dependencies of fixture

A fixture can depend on other fixtures. This allows you to provide multiple objects needed by the test code at once.

import pytest

@pytest.fixture
def obj1():
    return ...

@pytest.fixture
def obj2(obj1):
    return ...

@pytest.fixture
def obj3(obj1, obj2):
    return ...

In this example, three fixtures are defined. obj1 is an independent fixture, obj2 is a fixture that depends on obj1. And obj3 is a fixture that depends on obj1 and obj2. The test code can take obj3 as an argument. The obj3 is generated using the objects provided by obj1 and obj2.

def test_my_test(obj3):
    ...

This example defines a test code named test_my_test. The test code takes a fixture named obj3 as an argument. The obj3 is generated using objects provided by obj1 and obj2.

Scope of fixture

A fixture can be reused in different parts of the test set. fixture has the following scopes.

To specify the scope, use the scope option to fixture.

import pytest

@pytest.fixture(scope="module")
def my_fixture():
    return ...

This example defines a fixture named my_fixture. This fixture has a scope option of "module". This means that this fixture will be reused by all test code in the same module.

Cache fixture

A fixture may be called multiple times in the same test code. pytest caches the result of the first call instead of executing the same fixture multiple times. This saves fixture execution time.

To disable fixture caching, use the autouse option.

import pytest

@pytest.fixture(autouse=True)
def my_fixture():
    return ...

This example defines a fixture named my_fixture. This fixture has the autouse option set to True. This means that this fixture does not need to be explicitly passed to the test code as an argument, but will be executed automatically. Also, this fixture is not cached.

Automatic detection of fixture

pytest automatically detects a file named conftest.py by default. The fixture defined in conftest.py can be referenced by all test code in the directory containing that file. This allows for greater fixture reusability.

Plugins

pytest provides a variety of plug-ins that are freely available to users. Plugins are used to extend or customize the functionality of pytest. The following is a description of some typical pytest plug-ins.

• pytest-cov
  pytest-cov is a plugin for measuring code coverage. pytest-cov will retrieve coverage information when you run tests with pytest. pytest-cov can output reports in HTML format, allowing you to visually check code coverage.

• pytest-html
  pytest-html is a plugin for outputting test results in HTML format, allowing you to visually check test results by outputting them in HTML format. The report can also include screenshots, logs, and other information.

• pytest-xdist
  pytest-xdist is a plugin for running tests in parallel on multiple processes or machines. This reduces test execution time.

• pytest-flake8
  pytest-flake8 is a plugin to verify the quality of your code, using a static analysis tool called flake8 to check for code problems. pytest can display the results of flake8 when you run tests with pytest.

• pytest-django
  pytest-django is a plugin to support testing with the Django framework, allowing you to test Django models, views, etc. with pytest.

• pytest-selenium
  pytest-selenium is a plugin to run tests for web applications using Selenium WebDriver. It allows you to test the behavior of your web application by automatically operating the browser. It can also take screenshots.

• pytest-mock
  pytest-mock is a plugin for running tests using mocks, using the standard library unittest.mock to generate mock objects for testing.

How to use the plugin

The pytest plugin can be installed using the pip command. Specifically, execute the following command

$ pip install pytest-cov

This will install the pytest-cov plugin. pytest-cov plugin can be used by specifying the --cov option as a command line argument to pytest. After the --cov option, specify the package or module for which you want to measure coverage.

$ pytest --cov=myapp tests/

You can now measure the coverage of the code under the myapp package. The coverage information will be output to the terminal. You can also generate a report in HTML format by specifying the --cov-report option.

$ pytest --cov=myapp --cov-report=html tests/

This will output coverage information in HTML format.

If you want to use the pytest-html plugin, install it with the same pip command.

$ pip install pytest-html

To output test results in HTML format, specify the --html option.

$ pytest --html=report.html tests/

This will output the test results in HTML format in a file called report.html. The pytest-html plugin can also combine the resources required for an HTML file into a single file by specifying the --self-contained-html option.

$ pytest --html=report.html --self-contained-html tests/

This will output the required resources in a single file called report.html.

Other plug-ins can also be used by specifying options to the pytest command line arguments, although they are installed and used in different ways.

Measure test coverage

The pytest-cov plugin allows you to measure the coverage of your tests. Here is an example of a test using the pytest-cov plugin.

$ pytest --cov=my_module tests/

In this example, the --cov option is used to measure the coverage of a module named my_module. The pytest-cov plugin will run the test, calculate the coverage of the module, and generate a report.

Skip tests

If a test cannot be run, it can be skipped. For example, you can skip a test under certain conditions by this.

import pytest

def test_division():
    divisor = 0
    dividend = 10
    if divisor == 0:
        pytest.skip("Can't divide by zero")
    assert dividend / divisor == 5

In this example, if divisor is 0, the test is skipped by the pytest.skip() function.

References

https://docs.pytest.org/en/7.2.x/
https://github.com/pytest-dev/pytest