Building and Uploading to PyPi

Learning Objectives

In the following section we will …

  • Review the packaging terminology
  • Understand how to build, package and publish a python package

Packaging Terminology 101


This section reviews the key python packaging concepts and definitions.


PyPI is the default Package Index for the Python community. It is open to all Python developers to consume and distribute their distributions.

There are two instances of the Package Index:

  • PyPI: Python Package Index hosted at
  • TestPyPI: a separate instance of the Python Package Index (PyPI) that allows you to try out the distribution tools and process without worrying about affecting the real index. TestPyPI is hosted at



The PyPA recommended tool for installing Python packages.

A multi-faceted tool:

  • It is an integration frontend that takes a set of package requirements (e.g. a requirements.txt file) and attempts to update a working environment to satisfy those requirements. This may require locating, building, and installing a combination of distributions.
  • It is a build frontend that can takes arbitrary source trees or source distributions and builds wheels from them.



The Python Packaging Authority (PyPA) is a working group that maintains many of the relevant projects in Python packaging.

The associated website references the PyPA Goals, Specifications and Roadmap as well as Python Packaging User Guide, a collection of tutorials and references to help you distribute and install Python packages with modern tools.


Source distribution

  • Synonyms: sdist, Source release
  • provides metadata + source files
  • needed for installing
    • by a tool like pip
    • or for generating a Built Distribution


Built Distribution

  • Synonyms: bdist
  • provides metadata + pre-built files
  • only need to be moved (usually by pip) to the correct locations on the target system


Python Distribution: pure vs non-pure

  • non-pure
    • ABI
    • Platform specific


Binary Distribution

  • is a Built Distribution
  • is non-pure
  • uses platform-specific compiled extensions



  • a Built Distribution
  • a ZIP-format archive with .whl extension
    • {distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl
  • described by PEP 427


Wheels vs. Conda packages

Wheels Conda packages
Employed by pip, blessed by PyPA Foundation of Anaconda ecosystem
Used by any python installation Used by conda python installations
Mostly specific to Python ecosystem General purpose (any ecosystem)
Good mechanism for specifying range of python compatibility Primitive support for multiple python versions (noarch)
Depends on static linking or other system package managers to provide core libraries Can bundle core system-level shared libraries as packages, and resolve dependencies

To learn more about Conda, see Conda Packages section.

Virtual Environment

An isolated Python environment that allows packages to be installed for use by a particular application, rather than being installed system wide.

Learn more reading Creating Virtual Environments

Build system

Synonym: Build backend

  • setuptools associated with the wheel package form the default build system. They support the creation of source and built distributions based on a and optionally a setup.cfg file.
  • flit is an alternative backend allowing to also create (and also publish) built distributions.

Python Package Lifecycle




This section discusses how to build python packages (or distributions) and publish them in a central repository to streamline their installation. Finally, we conclude with exercises where we publish a package with the Test Python Package Index.

Creating an environment

Before developing or building your distribution, we highly recommend to create a dedicated environment. This is supported by both conda and pip.

Building a source distribution

By leveraging the script, setuptools can build a source distribution (a tar archive of all the files needed to build and install the package):

$ python sdist

$ ls -1 dist

Building a wheel

$ pip wheel . -w dist

$ ls -1 dist

This is equivalent to:

$ python bdist_wheel

Installing a wheel

  • Install a package from PyPI:
$ pip install SomePackage
Successfully installed SomePackage
  • Install a package from TestPyPI:
$ pip install -i SomePackage
Successfully installed SomePackage
  • Install a package file:
$ pip install SomePackage-1.0-py2.py3-none-any.whl
Successfully installed SomePackage

For more details, see QuickStart guide from pip documentation.

Installing a source distribution

$ pip install SomePackage-1.0.tar.gz
Successfully installed SomePackage

It transparently builds the associated wheel and install it.

Publishing to PyPI

twine utility is used for publishing Python packages on PyPI.

It is available as both a conda and a pypi package.

Learn more reading Using TestPiPY.


Exercise 1: Prepare environment

  • In the context of this tutorial, because participants already installed miniconda, we will create a conda environment and install packages using conda install SomePackage.
# create and activate a dedicated environment named "hello-pypi"
conda create -n hello-pypi -y -c conda-forge
conda activate hello-pypi

# install pip, wheel and twine
conda install -y twine wheel pip

Exercise 2: Build source distribution and wheel

  • Download (or checkout using git) the sources of our hello-pypi sample project:
conda install -y wget
  • Extract sources
conda install -y unzip
cd hello-pypi-master
  • Modify package name so that it is unique
  • Then, build the source distribution:
$ python sdist
  • And finally, build the wheel:
$ pip wheel . -w dist
  • Make sure artifacts have been generated in the dist subdirectory.

Exercise 3: Publish artifacts on PyPI

$ twine upload --repository-url dist/*

Bonus Exercise 4: Publish artifacts automating authentication

  • Delete hello-pypi-master directory and extract archive again.
  • Update name of package and rebuild source distribution and wheel.
  • Create file .pypirc in your home directory with the following content:

username: your testpypi username
password: your testpypi password

username: your testpypi username
password: your testpypi password
  • Publish package on TestPyPI:
$ twine upload --repository testpypi dist/*

Omitting the -repository testpypi argument allows to upload to the regular PyPI server.

Bonus Exercise 5: Setting up continuous integration

  • See branch master-with-ci branch associated with hello-pypi example.


Where do I go to figure this out?

This is a really good guide:

Python Packaging User Guide:

and a more detailed tutorial:

Follow one of them

There is a sample project here:

(this has all the complexity you might need…)

You can use this as a template for your own packages.

Here is an opinionated update – a little more fancy, but some good ideas:

Rather than doing it by hand, you can use the nifty “cookie cutter” project:

And there are a few templates that can be used with that.

The core template written by the author:

And one written by the author of the opinionated blog post above:

Either are great starting points.