Security Developer-in-Residence – Weekly Report #7

This critical role would not be possible without funding from the OpenSSF Alpha-Omega Project. Massive thank-you to Alpha-Omega for investing in the security of the Python ecosystem!

The past few weeks have been fairly light as I've taken time off to get married! 🎉 The usual weekly cadence will resume next week.

RFI on OSS Security by US National Cyber Director§

The biggest news in the open source security space right now is the Request for Information (RFI) titled "Request for Information on Open Source Software Security: Areas of Long-Term Focus and Prioritization". This announcement means that the US Government is soliciting ideas from the broader community on where to focus and what to do to improve the security of open source software.

I've been catching up this initiative since returning from time away, you can read the fact sheet about the RFI and Tidelift's summary blog post for more information.

Certifi and Truststore§

Back in late July, certifi published a GHSA advisory about the removal of the e-Tugra root certificate. This advisory had an associated CVE ID so should eventually make its way into the PyPA advisory database but the automation wasn't able to import it automatically. After that issue was resolved, pip was able to land a PR upgrading certifi to the latest version.

Certifi is a critical package in the Python ecosystem as it's the most common way that SSLContext instances are configured due to strong ties to the OpenSSL library by Python's ssl module. A consequence of certifi's use in pip in particular due to bundling causes a chain of events whenever there's a security issue with a root CA:

• Removal occurs in Mozilla Root CA Bundle
• Certifi bundles the Mozilla Root CA Bundle, requires an update, advisory, and new release of certifi.
• Pip bundles certifi, requires a new release of pip.
• New pip releases require a release of get-pip.
• Python bundles a default version of pip and it's preferable to bundle a version of pip that isn't vulnerable to any CVEs (but this issue is fixable by users).

This chain of updates causes a lot of churn any time there's a removed CA in certifi and doesn't account for all the upgrades that need to happen in individual application lock files.

Truststore is a library authored by myself and David Glick which is aiming to remove the need for certifi by using system trust stores instead of hardcoded bundle which puts the onus of keeping the trust store up-to-date on the system itself (which for macOS and Windows can be automatically updated in the background!)

Truststore has recently received a large amount of passive users thanks to PDM adopting the library when installed via pdm[truststore] or pdm[all]. The code path is automatically used if truststore is installed which means we can be sure that the library is working as intended for a wide variety of configurations. I've been monitoring PDM's issue tracker for issues related to truststore and so far there aren't any after being installed ~2,000 times per day.

Pip currently supports using truststore if its already installed and I have an outstanding PR for adding truststore support to pip without the need to install the library separately.

Below is a list of projects (ordered by downloads) which directly depend on certifi that also would be candidates to switch from certifi to truststore but don't have the same issue of bundling certifi that pip does:

• requests
• msrest
• snowflake-connector-python
• httpcore
• opensearch-py
• httpx
• sentry-sdk
• selenium
• pipenv
• pyproj
• uamqp
• fiona
• datadog-api-client
• elastic-transport
• netcdf4
• rasterio
• launchdarkly-server-sdk
• minio
• yt-dlp

This list was generated from the following SQL query on the pypi-data dataset.

SELECT packages.name FROM deps JOIN packages ON deps.package_name = packages.name
WHERE dep_name = 'certifi' AND extra IS NULL
ORDER BY packages.downloads DESC LIMIT 20;

I'm hopeful we can move away from certifi to reduce the amount of churn generated from using PyPI as a CA distribution channel.

Other items§

• Discussed with PyPI's new Safety and Security Engineer Mike Fiedler about Open Source Vulnerability format and its ability for representing malware or typo-squatted releases on PyPI. After seeing that the OSV database now contains malware advisories, I checked with Oliver Cheng and Caleb Brown on the expected volume of malicious packages being taken down by PyPI (~1,300 per month) to verify the volume matched their expectations. Watch this space for more, especially regarding the PyPI Malware Detection project.
• Reviewed the blog post for mandatory 2FA enforcement for new users of PyPI.
• Drafted a blog post for the PSF CNA work.
• Created a fix in OSV tooling that was affecting pypa/advisory-database.
• Created an issue for additional hardening of PyPI projects using Trusted Publishers that want to relinquish the ability to publish through other means.
• Created a feature request for GitHub regarding default collaborators on GitHub Security Advisories.

That's all for this week! 👋 If you're interested in more you can read next week's report or last week's report.

Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.