Test Suite - build 1.0.3

Hide navigation sidebar
Hide table of contents sidebar
Back to top
Toggle table of contents sidebar

# Test Suite

Due to its nature, `build` has a somewhat complex test suite, which we will try to go through in this document.

Firstly, there are two set of tests, unit tests and integration tests. In unit tests, we test the actual code implementation. In integration tests, we test `build` on a few real world projects; this is mostly a sanity test.

Integration tests take a long time to run, and are not very helpful tracking down issues, so they are disabled by default. They can be enabled by passing either `--run-integration` or `--only-integration` arguments to pytest, where the latter will disable the unit tests and only run the integration ones. Even though these tests are disabled by default, they will be run in CI, where test suite run durations are not a big issue.

To run the test suite we use `tox`, which automates running the test suite on different environments:

    tox

You can find out more about how to run `tox` and its arguments in the tox documentation.

We have a fairly large environment matrix. We run tests for all supported Python versions and implementations, and with the module being invoked from path, sdist install, or wheel install. Additionally, we have an environment for type checking, and one to produce the documentation. There are some other extra environments, like checking the code with the minimum version of each dependency.

Some examples commands for this project:

  - Run type checking: `tox -e type`

  - Only run unit tests against Python 3.9: `tox -e py39`

  - Run both unit and integration tests: `tox -- --run-integration`

  - Only run integration tests: `tox -- --only-integration`

  - Only run integration tests with parallel tasks: `tox -- -n auto --only-integration`

  - Only run unit tests against Python 3.9 with the module installed via wheel: `tox -e py39-wheel`

We have CI testing, where we the test suite across all supported operating systems, and have test coverage reports.