PyPI Externally Hosted Files


What Should I Do?


  1. Log into PyPI
  2. Navigate to each of your packages (e.g. http://pypi.python.org/pypi/your-package)
  3. Use the "urls" link at the top of the page.

    Note: This will look version specific but it is not. You only need to do it once per project.

  4. Change your Hosting Mode, you Probably want the first option

  5. Remove any urls that you don't want pip or easy_install to try and install. Files uploaded to PyPI will always be available for install

  6. Rejoice in speedier installs!

What Does All this Mean?

When pip or easy_install attempt to install a package they go to a special page on PyPI called the simple index (e.g. https://pypi.python.org/simple/apathy/).

Once they go to the simple index for the desired package they read the links on that page. Any link with rel="homepage" or rel="download" on this simple index will also be visited.

Once they have the contents of the simple index, and the contents of all the homepage and download links they process those contents looking for links that look like they are to installable files.

The rules for what is an installable file are pretty arcane, but generally it is any url that looks like it's a download for a format the installer supports (e.g. http://example.com/apathy-1.0.tar.gz).

Once it's found all the installable urls, it picks the best version it's found that matches what you've told it to install, downloads it, and installs it.

Obviously scraping all of those homepage and download pages for external urls can take quite a long time. When selecting your Hosting Mode the first and second option tell PyPI to change the rel attributes to ext-homepage and ext-download which pip and easy_install will not scrape. This offers a pretty massive speedup in many cases.

Whenever you upload a package to PyPI it processes the long_description or README(.rst) file and attempts to find urls. Any url it finds it will list on the packages simple index and if those urls look installable then installers will attempt to install them.

The first Hosting Mode tells PyPI to no longer scan and automatically add urls to the simple index and to instead only list urls that you have explicitly told it too (and the uploaded files are always listed).

The list of urls (if you have any) will contain all of the links that PyPI has scanned and extracted from your long_description and README(.rst) over time.


Fork me on GitHub