Proposal for Software Bill-of-Materials for CPython

Published 2023-11-22 by Seth Larson
Reading time: 4 minutes

This critical role would not be possible without funding from the OpenSSF Alpha-Omega project. Massive thank-you to Alpha-Omega for investing in the security of the Python ecosystem!

This week I created the top-level tracking GitHub issue for being able to create and distribute Software Bill-of-Materials documents alongside Python artifacts on python.org/downloads. To completely implement SBOMs for CPython it will require changes to multiple repositories (cpython, release-tools, cpython-source-deps, pythondotorg, ...) so this issue will be the central tracking location.

I've also created a PR which adapts my experiments in creating SBOMs for past and current CPython releases and trying those SBOMs with current scanners. The PR adds a new Makefile target regen-sbom which updates the dependencies-only SBOM in the CPython codebase.

This proposal doesn't modify Python itself but does have some implications for Python core developers due to needing to update SBOM metadata whenever vendored dependencies are upgraded (including those in cpython-source-deps) to ensure the data is consistent. The method I've proposed uses SBOM file checksums to check whether a file has been updated and if so, to flag to the author of the PR that they need to update SBOM metadata.

CPython Base SBOM
CPython Base...
CPython Source SBOM
CPython Sourc...
CPython Windows SBOM
CPython Windo...
CPython macOS SBOM
CPython macOS...
CPython VEX Statements
CPython VEX S...
CPython Source Deps SBOMs
CPython Sourc...
Vuln Scanner
Vuln Scanner
External Ref
External Ref
Text is not SVG - cannot display

Above is the architecture diagram for how the final released SBOMs will be created. The blue elements above denote metadata that would need to be manually updated and green elements (within the release-tools area) would be generated as a result of automation using the base CPython SBOM and source deps SBOMs as starting points. The green SBOM elements would be published to python.org/downloads.

This proposal is also likely to carry additional maintenance burden, especially for Python Security Response Team and Release Manager members. Having more visibility into the software that users are running combined with vulnerability scanners means there will be more folks knocking on the door whenever a vulnerability in one of our dependencies is published.

I am hoping that there is a systematic way of reducing this effort especially when it comes to vulnerabilities in subcomponents which aren't exploitable (which is common for OpenSSL, which we only use TLS). My current approach is to embed a link (marked as External Ref above) to Vulnerability Exploitability Exchange (VEX) statements that can be controlled by core developers. I'm currently researching this for both SBOM formats and scanners.

Because of this additional burden to core developers and to field questions I have opened a Discourse topic for this work. Please leave a comment or question and I will reply!

Memory safety hardening through compiler options

The OpenSSF Best Practices working group recently completed a C and C++ hardening guide using compiler options. Michael Droettboom and I were invited to discuss this new guide and Python. I mentioned it'd be interesting to see what the current CPython build would produce with all the options enabled and Michael was successfully sniped 😉. Michael helpfully created an issue with the complete build log and all the warnings. The build was able to complete successfully along with passing all test cases.

The options currently being considered are:

-O2 -Wall -Wformat=2 -Wconversion -Wtrampolines -Wimplicit-fallthrough \
-fstrict-flex-arrays=3 \
-fstack-clash-protection -fstack-protector-strong \
-Wl,-z,nodlopen -Wl,-z,noexecstack \
-Wl,-z,relro -Wl,-z,now \
-fPIE -pie -fPIC

We'll likely be picking through each option and adopting the ones that make sense for CPython. There was already a PR created to fix the single issue found by -Wmaybe-uninitialized.

The guide itself has a section going into the behavior and requirements of each option.

Other items

Thursday and Friday are holidays in the United States and from Monday to Thursday I'll be traveling to meet up with some of my colleagues at the Python Software Foundation. Due to this, expect there to not be much to report for next week. See you in December!

That's all for this week! 👋 If you're interested in more you can read next week's report or last week's report.

Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.

This work is licensed under CC BY-SA 4.0