This post adds guidance for Python libraries to the fantastic write-up, Better Package Management.

As a maintainer of a Python library, you can break down managing dependencies into 3 concerns.

  1. How do I specify required and optional dependencies for consumers of my library?
  2. How do I specify dependencies required for contributors of my library?
  3. How do I make sure my builds are predictable and deterministic?

The answer to all three of these concerns lies in the proper usage of pip-tools.

Specifying dependencies

For Python applications that install your library, you'll need to specify the library's depencies in -

from setuptools import setup

        'thisoptionalfeature': [

Keep version constraints in this file as loose as possible, eg tornado>=4.2<5 instead of tornado==4.2.3; a good rule of thumb is to use Semantic Versioning, locking to the latest major release, aka mydep>=1,<2.

For library maintainers

For contributors/maintainers who are actually working on the Python library - you'll want to provide additional dependencies to enable development. Extend the deps specified in in a file -

# extend deps in
-e .
-e .[thisoptionalfeature]

# debugging

# linting

# releasing

Now we can compile the into a requirements.txt file like so -

(env) $ pip install pip-tools
(env) $ pip-compile

The above gives us the following fully-pinned requirements.txt -

appnope==0.1.0            # via ipython
backports-abc==0.4        # via tornado
backports.ssl-match-hostname==  # via tornado
certifi==2016.2.28        # via tornado
colorama==0.3.7           # via zest.releaser
decorator==4.0.9          # via ipython, traitlets
gnureadline==6.3.3        # via ipython
ipython-genutils==0.1.0   # via traitlets
mccabe==0.4.0             # via flake8            # via pickleshare
pep8==1.7.0               # via flake8
pexpect==4.0.1            # via ipython
pickleshare==0.6          # via ipython
pkginfo==1.2.1            # via twine
ply==3.8                  # via thriftrw
ptyprocess==0.5.1         # via pexpect
pyflakes==1.0.0           # via flake8
requests-toolbelt==0.6.0  # via twine
requests==2.9.1           # via requests-toolbelt, twine
simplegeneric==0.8.1      # via ipython
singledispatch==   # via tornado
six==1.10.0               # via singledispatch, thriftrw, zest.releaser
tornado==4.3              # via threadloop
traitlets==4.2.1          # via ipython
twine==1.6.5              # via zest.releaser

This can be used with pip install -r requirements.txt, or with pip-sync as detailed in the following section.

Installing dependencies

Now that we have our compiled requirements.txt, we can make sure our virtualenv matches it using pip-sync -

(env) $ pip-sync

This makes sure our virtualenv contains exactly the dependencies listed in requirements.txt, and guarantees a deterministic build. Let's be nice and write a target in our Makefile as a peace-offering to future contributors -

.PHONY: install
    pip install pip-tools
    python develop

Which should be ran from a virtualenv like so -

$ virtualenv env
$ source env/bin/activate
(env) $ make install