Contribution Guide

The Push Hook

Here at Desupervised we try to hit good coding and testing standards. For this reason, we’ve implemented a ‘git push hook’, which means that your code must pass certain checks before it can be pushed to the Desupervised repository (unless you specify to skip the CI build). This requires you to run the following command from the root of the cloned repository before making your first push:

make push-hook

Running Tests

Testing Locally

There are various make targets available for running tests and linting. For an overview of those available, run make help.

When running tests discovered by pytest, the option to include/exclude certain tests is available by specifying them within the environment variable TEST_ARGS before using any make target which includes the rule test.

For example, to run only integration tests, you can execute:

TEST_ARGS="-m integration" make test

Conversely, to exclude integration tests it would be

TEST_ARGS='-m "not integration"' make test

For more information on filtering tests with markers see https://docs.pytest.org/en/latest/example/markers.html

Currently, the most common test markers are:

  • download: A test which requires downloading files over the internet.

  • integration: A test which relies on the integration of multiple objects within the framework.

  • memory_intensive: A test which is memory intensive.

  • predefined: A test which involves a predefined model (can be computationally expensive).

  • tutorial: A test based on running a tutorial script.

For a full list run:

pytest --markers

NB any call to test will also run a coverage analysis on borch.

Finally, note that both make targets and test markers can be combined. For example, the push hook mentioned earlier runs the following make command:

TEST_ARGS='-m "not integration and not predefined"' make lint test

Which runs linting and tests (excluding those listed in TEST_ARGS.

Note that the tutorials also have tests associated with them. The test class can have a classmethod called before_script. The before_script will be run before the tutorials and documentation are created. This makes it possible to create artifacts in the before_script that then will be used in the tutorial. It’s recommended to be aware of artifacts created by other tutorials. Reusing the same required artifacts (e.g. CIFAR10) will save a lot of time. See one of the tutorial tests under the tutorials/ for example usages, and see the class ScriptTest for notes on how it should be used.

Testing on a GPU

To submit tests to run on a GPU run the script scripts/paperspace-test.sh. This requires that you have installed the paperspace-cli, have valid paperspace credentials, and supply a valid GitLab registry username and password.