This page documents some common workflows for maintaining marbles. Most of these are also useful when developing marbles.
$ pip install --user pipenv
Creating your environment¶
In the marbles codebase, create a development virtualenv, and enter it:
$ pipenv install --dev $ pipenv shell
Inside this environment, you should have everything you need to work on marbles. See the other sections in this doc for common activities.
Adding new packages¶
If you want to add a new package for development, install it with pipenv:
$ pipenv install --dev pylint
This will update the
pylint as a new
development dependency, and will update
Pipfile.lock with the
exact version you installed. If this dependency is needed for
something you added to marbles, you should include the changes to both
Pipfile.lock in your pull
request. Otherwise, please don’t include the changes to
Pipfile.lock in your pull request.
$ python marbles/core/setup.py test $ python marbles/mixins/setup.py test
$ tox -e coverage
If you want to look at the source code annotated with coverage metrics, this produces an HTML report you can view, by loading file:///path/to/marbles/build/coverage/html/index.html in your browser.
You can build the docs and view them locally:
$ python setup.py build_sphinx
Then, load file:///path/to/marbles/build/sphinx/html/index.html in
your browser. If you make changes to just docstrings, but not
.rst files, Sphinx may not rebuild those docs, you can
embolden it to do so with these options:
$ python setup.py build_sphinx -Ea
We use tox to run continuous integration builds for multiple versions of Python, and to run each piece of our continuous integration in a separate virtualenv. You can do this locally too, to make sure your change will build cleanly on Travis CI.
We’ve configured tox to be able to:
- Run all the tests with Python 3.5 and 3.6
- Measure and report on code coverage
- Lint the code with flake8
- Build the documentation
If you just run tox by itself, it will do all of the above, each in its own virtualenv:
You can also run a subset of these with
$ tox -e docs $ tox -e py36 $ tox -e flake8,coverage
Maintaining the Changelog¶
Most pull requests should add an item to the changelog, at the top, either a bug, feature, or support note.
releases is clear about the distinction between bugs and other
release notes. Bugs are included in the next patch version that
appears above them, while features aren’t included until the next
major or minor version above them. The decision of whether to note
a change as a bug, feature, or support item will affect where it
appears in the log, though this can be controlled with the keywords
major (put bugs in the next major or minor release), and
backported (put features in the next bugfix release).
See Release organization for details.
Right before releasing a new version of marbles, add a release item to the top of the changelog noting the version string and release date, then follow the below instructions on Releasing a new version.
Releasing a new version¶
The marbles meta-package and subpackage version strings are stored in a few different locations, due to the namespace package setup:
In addition, when we bump the version, we do so in an isolated commit, and tag that commit with the version number as well.
You can increase either the
$ bumpversion major $ bumpversion minor $ bumpversion patch
This will update the version strings in all the above files and commit
that change, but won’t tag it. You should create a pull request for
the version update, merge it (without squashing it into other
commits), and then tag it once it’s on the
You can read a digression about why we bump all the versions at the same time below, in Versioning philosophy.
Uploading to PyPI¶
Once you’ve tagged the latest version of marbles, pull from GitHub to
make sure your clone is up to date and clean, build both
wheel packages for all three packages, and upload them with
twine. We have a tox rule to automate building and uploading:
$ tox -e pypi
Marbles publishes two subpackages,
marbles.mixins, and a metapackage depending on both,
marbles. This allows users to install or depend on only one of
the subpackages, and also suggests that anyone can publish their own
This raises the question of how to version each of these three packages.
- Release new versions of
marbles.mixinsindependently, and have the
marblespackage basically only ever have one release,
1.0.0, since it doesn’t actually change over time.
- Give the
marblespackage a new version each time either subpackage gets one, to make it feel like we’re moving forward.
- Release all three packages with the same version string each time any of them gets a new release.
Jupyter takes the first approach, but keep in mind that Jupyter is a much larger project with distinct teams working on each component, so allowing subpackages to have independent release schedules makes more sense for that community.
The second approach has the problem that if we release the subpackages independently, it’s unclear how to version the metapackage when that happens. Taking the max of the subpackage version strings doesn’t work if the subpackage with a lower one gets an update by itself. There are a couple other possiblities here, but none of them seemed right.
The third approach, updating everything in lock-step, is what we’ve
chosen. This will create multiple versions of one or the other package
that are identical, in some cases, which is a little odd. However, it
has the benefit of documenting which versions of
marbles.mixins were reviewed and tested together and
therefore can be expected to work together. It still allows users to
install (and update) them independently, but encourages users of both
to update them together.