Contribution Guide¶
The development workflow is all centered around the Makefile, so to get a full
list of the available tools please run
make help
Installation¶
When installing borch we normally use a virtual environment to manage the Python version dependencies. Two good ones are virtualenv and conda. Look at them, pick one to use and follow their documentation to create and activate an environment.
NB: All installations of python packages should be placed in the correct environment. Installing packages in the global python interpreter can result in unexpected behavior, where global packages may be used in favor of local packages.
Install locally¶
Once an appropriate conda environment has been created, run
make install
to install a production version of borch with support for a GPU, or
ARCH=cpu make install
for a version that only supports a CPU.
To install in development mode on machine (with no GPU support) run
ARCH=cpu make install-dev
and for GPU support use
make install-dev
Docker¶
All borch Docker images are found at (gitlab.com/desupervised/borch/container_registry)[https://gitlab.com/desupervised/borch/container_registry/].
Build¶
Currently, all borch docker images are based on Ubuntu 18.04. By setting
--build-arg ARCH=gpu to either gpu, cpu it will install either install all
dependencies needed to run on gpu or to only run on cpu. If not provided it will
fall back to the standard pytorch installation.
The GPU image can be built using:
docker build --build-arg ARCH=gpu .
And the CPU image using:
docker build --build-arg ARCH=cpu .
Merge requests¶
Please refer to a Gitlab issues that it closes and have a clear description.
In order for the CI to pass for a merge request it must pass linting and tests. It is recommended to get in the habit of running
make lint test
before creating the merge request such that the ci will pass.
Tip: make format will help fixing potential lint issues.
Running Tests¶
Testing Locally¶
There are various make targets available for running tests and linting. For an
overview of those available, run make help.
When running tests discovered by pytest, the option to include/exclude certain
tests is available by specifying them within the environment variable
TEST_ARGS before using any make target which includes the rule test.
For example, to run only integration tests, you can execute:
TEST_ARGS="-m integration" make test
Conversely, to exclude integration tests it would be
TEST_ARGS='-m "not integration"' make test
For more information on filtering tests with markers see https://docs.pytest.org/en/latest/example/markers.html
Currently, the most common test markers are:
tutorial: A test based on running a tutorial script.
For a full list run:
pytest --markers
NB any call to test will also run a coverage analysis on borch.
Finally, note that both make targets and test markers can be combined. For example, the push hook mentioned earlier runs the following make command:
TEST_ARGS='-m "not integration and not predefined"' make lint test
Which runs linting and tests (excluding those listed in TEST_ARGS.
Note that the tutorials also have tests associated with them. The test class
can have a classmethod called before_script. The before_script will be run
before the tutorials and documentation are created. This makes it possible to
create artifacts in the before_script that then will be used in the tutorial.
It’s recommended to be aware of artifacts created by other tutorials. Reusing
the same required artifacts (e.g. CIFAR10) will save a lot of time. See one of
the tutorial tests under the tutorials/ for example usages, and see the class
ScriptTest for notes on how it should be used.
The Push Hook¶
At Desupervised we try to hit good coding and testing standards. For this reason, we’ve implemented a ‘git push hook’, which means that your code must pass certain checks before it can be pushed to the Desupervised repository (unless you specify to skip the CI build). This requires you to run the following command from the root of the cloned repository before making your first push:
make push-hook