Contribution Guide¶
The Push Hook¶
Here at Desupervised we try to hit good coding and testing standards. For this reason, we’ve implemented a ‘git push hook’, which means that your code must pass certain checks before it can be pushed to the Desupervised repository (unless you specify to skip the CI build). This requires you to run the following command from the root of the cloned repository before making your first push:
make push-hook
Running Tests¶
Testing Locally¶
There are various make targets available for running tests and linting. For an
overview of those available, run make help
.
When running tests discovered by pytest, the option to include/exclude certain
tests is available by specifying them within the environment variable TEST_ARGS
before using any make target which includes the rule test
.
For example, to run only integration tests, you can execute:
TEST_ARGS="-m integration" make test
Conversely, to exclude integration tests it would be
TEST_ARGS='-m "not integration"' make test
For more information on filtering tests with markers see https://docs.pytest.org/en/latest/example/markers.html
Currently, the most common test markers are:
download
: A test which requires downloading files over the internet.integration
: A test which relies on the integration of multiple objects within the framework.memory_intensive
: A test which is memory intensive.predefined
: A test which involves a predefined model (can be computationally expensive).tutorial
: A test based on running a tutorial script.
For a full list run:
pytest --markers
NB any call to test
will also run a coverage analysis on borch
.
Finally, note that both make targets and test markers can be combined. For example, the push hook mentioned earlier runs the following make command:
TEST_ARGS='-m "not integration and not predefined"' make lint test
Which runs linting and tests (excluding those listed in TEST_ARGS
.
Note that the tutorials
also have tests associated with them. The test class
can have a classmethod
called before_script
. The before_script
will be run
before the tutorials and documentation are created. This makes it possible to
create artifacts in the before_script
that then will be used in the tutorial.
It’s recommended to be aware of artifacts created by other tutorials. Reusing
the same required artifacts (e.g. CIFAR10) will save a lot of time. See one of
the tutorial tests under the tutorials/
for example usages, and see the class
ScriptTest
for notes on how it should be used.
Testing on a GPU¶
To submit tests to run on a GPU run the script scripts/paperspace-test.sh
.
This requires that you have installed the paperspace-cli
, have valid
paperspace credentials, and supply a valid GitLab registry username and
password.