Security Developer-in-Residence Weekly Report #23

Published 2023-12-28 by Seth Larson
Reading time: 3 minutes

This critical role would not be possible without funding from the OpenSSF Alpha-Omega project. Massive thank-you to Alpha-Omega for investing in the security of the Python ecosystem!

It was a short week this week due to Christmas and New Years holidays in the United States.

This week I spent time learning about the current state of proposals for solving the "index trust" problem for the Python Package Index. There are a few different proposals that have happened in the past being PEP 458 and RSTUF. This learning was to better review a draft proposal for publish provenance (stay tuned!) that also contained some mentions of the index trust problem. Some outcomes we're looking for when solving the index trust problem:

Back to publish provenance, I learned more about how NPM is handling their own provenance and made some recommendations about future provenance attestations that might come after publish provenance (like build provenance with the builder workflow ID, git commit, and repo being verifiable!). After thinking about build provenance and how to keep the choice of which builder workflows are "trusted" in the hands of consumers instead of PyPI I created the following quick mockup:

PyPIOIDC ProviderBuilder Workflow (Reusable Workflow)Publishing Workflow (User-controlled)PyPIOIDC ProviderBuilder Workflow (Reusable Workflow)Publishing Workflow (User-controlled)loop[For each distribution ...]Call Builder workflow with git repo, commit SHA parametersProvide OIDC token for signing provenanceBuild Python distributionsSubmit name and digest of distribution and build provenanceVerify provenance is validStore build provenance document awaiting distributionHTTP 200 OKReturn distributions to Publishing WorkflowUpload distributions to PyPIAssociate build provenance with distributionsMake build provenance available programmatically and in the UI
This diagram doesn't represent a concrete proposal.

The only standards that would need agreeing on would be the upload API, the build provenance statement itself, and which parameters have special behavior/meaning for build provenance parameters (ie git repository, git commit SHA). This architecture would allow arbitrary build workflows provenance statements to be submitted to PyPI and verified.

Build provenance provides different guarantees to publish provenance, publish provenance only proves that a given artifact was at one point in the "hands" of the workflow but can't assert where the artifact actually came from without deeper inspection of the workflow. In a way, publish provenance is only making public and verifiable the configuration of a projects' Trusted Publisher at the time of release.

Build provenance from an isolated and trusted build workflow can assert that the artifact was built using a known set of steps and dependencies. It's also likely that there will be far fewer build workflows than projects, meaning that the list of trusted build workflows will be more surmountable to review compared to every publishing workflow of every project and dependency that would be required to review if relying on publish provenance for build integrity.

There's lots of cool stuff coming in this space next year, look forward to more from me and others.

Other items

That's all for this year! 👋 If you're interested in more you can read next week's report or last week's report.

Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.

This work is licensed under CC BY-SA 4.0