Security Developer-in-Residence – Weekly Report #4

Published 2023-07-21 by Seth Larson
Reading time: 5 minutes

This critical role would not be possible without funding from the OpenSSF Alpha-Omega Project. Massive thank-you to Alpha-Omega for investing in the security of the Python ecosystem!

Sigstore signed Python releases

Ever since the 3.11.0 release of Python, all Python release tarballs are signed using Sigstore. You can read this page for more information about Sigstore signatures and how to verify the signatures using the Python Sigstore client.

The gist is each Release Manager for a given Python release (ie 3.7, 3.8, etc) has an identity (email address with @python.org) defined in the Sigstore information page. This is the identity used to verify who signed the release. Sigstore uses OpenID Connect which means we also need to designate an identity provider (IdP) to verify signatures.

# Currently documented example of how to verify Python signatures
$ python -m sigstore verify identity \
  --certificate Python-3.11.0.tgz.crt \
  --signature Python-3.11.0.tgz.sig \
  --cert-identity pablogsal@python.org \
  --cert-oidc-issuer https://accounts.google.com \

It was reported that the instructions for verifying signatures weren't consistent with the reality of how the artifacts had been signed. I put together a few simple scripts which downloaded and attempted to verify every Python release artifact against its signatures and published the results. These scripts found a few issues:

With these findings the release managers took the following steps to make the signature verifications consistent for consumers:

You can see the change in state in this commit to the above dataset. From here all that's needed is fixing Łukasz's identity on signatures which may prove difficult as using GitHub as an IdP only allows the primary email address as an identity rather than all configured email addresses. William Woodruff opened an issue after discussion of this any other GitHub IdP use-cases on Sigstore's Slack channel. I've also opened a PR in Python's release-tool to keep Sigstore signing consistent with the documentation for future Python releases.

All of this highlights how even though Sigstore is much more ergonomic signing experience than GPG, there are still little things that can create an inconsistent experience for consumers. I'll be updating the Sigstore documentation once we've completed this task. Thanks to all the release managers for working on this issue!

PEP 710 (Package Provenance Info)

I stumbled on the discuss thread for PEP 710 written by Fridolín Pokorný which proposes a new metadata file provenance_url.json to be placed in an installed Python distribution's .dist-info directory if the package was installed via an index with resolution (ie python -m pip install urllib3). You can read the PEP for the exact structure of the document which includes the original download URL and a list of available hashes for the distribution like SHA-256, etc.

This PEP would allow tools to inspect an already installed Python environment and be able to recreate exact download URLs for installed distributions along with hashes in order to ensure integrity of the distributions. This makes the installed environment more reproducible if the download URLs are still intact.

Having this information available also enables tools to create a Software Bill of Materials (SBOM) from a pre-installed environment. I created an experimental tool that uses the reference implementation of PEP 710 for pip to create SPDX and CycloneDX SBOM documents.

From sharing this thread on Twitter I was also pointed to pip installation reports, a feature I wasn't aware of that's new in pip v22.2. (thanks Stéphane Bidoul!) This feature lets you hook into the results of pip's dependency resolution for a given CLI input to pip (ie pip install -r requirements.txt). I'll be experimenting with this feature in the future.

After reviewing the PEP draft I made a few suggestions including upgrading the "SHOULD" to a "MUST" for installing a resolved distribution from the installers' cache and clarifying if there was any case where either provenance_url.json or PEP 610's direct_url.json wouldn't be installed for a given distribution. If provenance_url.json can be made required for the inverse case of PEP 610 it would mean that all installed distributions would have a metadata file describing the distribution's origin.

I also suggested adding another key index_url which would contain the URL of the index that the distribution was resolved from. This is useful for auditing which indices were used for which packages to arrive at the final download URL.

I am hopeful that the amount of positive interest in this PEP will result in its acceptance. If you're interested in this PEP please participate in the discussion thread.

Other projects

That's all for this week! 👋 If you're interested in more you can read next week's report or last week's report.

Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.

This work is licensed under CC BY-SA 4.0