PEP: 644 Title: Require OpenSSL 1.1.1 or newer Author: Christian Heimes
<christian@python.org> Discussions-To:
https://discuss.python.org/t/pep-644-require-openssl-1-1-or-newer/5584
Status: Final Type: Standards Track Content-Type: text/x-rst Created:
27-Oct-2020 Python-Version: 3.10 Post-History: 27-Oct-2020, 03-Mar-2021,
17-Mar-2021, 17-Apr-2021 Resolution:
https://mail.python.org/archives/list/python-dev@python.org/message/INLCO2EZVQW7R7J2OL6HWVLVU3TQRAZV/

Abstract

This PEP proposes for CPython’s standard library to support only OpenSSL
1.1.1 LTS or newer. Support for OpenSSL versions past end-of-lifetime,
incompatible forks, and other TLS libraries are dropped.

Motivation

Python makes use of OpenSSL in hashlib, hmac, and ssl modules. OpenSSL
provides fast implementations of cryptographic primitives and a full TLS
stack including handling of X.509 certificates. The ssl module is used
by standard library modules like urllib and 3rd party modules like
urllib3 to implement secure variants of internet protocols. pip uses the
ssl module to securely download packages from PyPI. Any bug in the ssl
module's bindings to OpenSSL can lead to a severe security issue.

Over time OpenSSL's public API has evolved and changed. Version 1.0.2
introduced new APIs to verify and match hostnames. OpenSSL 1.1.0 made
internal structs opaque and introduced new APIs that replace direct
access of struct members. Version 3.0.0 will deprecate more APIs due to
internal reorganization that moves cryptographic algorithms out of the
core and into providers. Forks like LibreSSL and BoringSSL have diverged
in different directions.

Currently Python versions 3.6 to 3.9 are compatible with OpenSSL 1.0.2,
1.1.0, and 1.1.1. For the most part Python also works with LibreSSL >=
2.7.1 with some missing features and broken tests.

Due to limited resources and time it becomes increasingly hard to
support multiple versions and forks as well as test and verify
correctness. Besides multiple incompatible APIs there are build time
flags, distribution-specific patches, and local crypto-policy settings
that add to plethora of combinations. On the other hand, the Python core
team has only a couple of domain experts who are familiar with TLS and
OpenSSL internals and even fewer who are active maintainers.

Requiring OpenSSL 1.1.1 would allow us to give the vast majority of
users a better experience, reduce our maintenance overhead and thus free
resources to implement new features. Users would be able to rely on the
presence of new features and consistent behavior, ultimately resulting
in a more robust experience.

Impact

OpenSSL 1.1.1 is the default variant and version of OpenSSL on almost
all supported platforms and distributions. It’s also the only version
that still receives security support from upstream[1].

No macOS and Windows user will be affected by the deprecation. The
python.org installer and alternative distributions like Conda ship with
most recent OpenSSL version.

As of October 2020 and according to DistroWatch[2] most current BSD and
Linux distributions ship with OpenSSL 1.1.1 as well. Some older releases
of long-term support (LTS) and enterprise distributions have older
versions of OpenSSL or LibreSSL. By the time Python 3.10 will be
generally available, several of these distributions will have reached
end of lifetime, end of general support, or moved from LibreSSL to
OpenSSL.

Other software has dropped support for OpenSSL 1.0.2 as well. For
example, PyCA cryptography 3.2 (2020-10-25) removed compatibility with
OpenSSL 1.0.2.

OpenSSL 1.0.2 LTS

released: 2015-02 end of lifetime: 2019-12

OpenSSL 1.0.2 added hostname verification, ALPN support, and elliptic
curves.

-   CentOS 7 (EOL 2024-06)
-   Debian 8 Jessie (EOL 2020-07)
-   Linux Mint 18.3 (EOL 2021-04)
-   RHEL 7 (full support ends 2019-08, maintenance 2 support ends
    2024-06)
-   SUSE Enterprise Linux 12-SP5 (general supports ends 2024-10)
-   Ubuntu 16.04 LTS / Xenial (general support ends 2021-04)

OpenSSL 1.1.0

released: 2016-08 end of lifetime: 2019-09

OpenSSL 1.1.0 removed or disabled insecure ciphers by default and added
support for ChaCha20-Poly1305, BLAKE2 (basic features), X25519 and CT.
The majority of structs were made opaque and new APIs were introduced.
OpenSSL 1.1.0 is not API compatible with 1.0.2.

-   Debian 9 Stretch (security support ended 2020-07, LTS until 2022-06)
-   Ubuntu 18.04 LTS / Bionic (general support ends 2023-04)

OpenSSL 1.1.1 LTS

released: 2018-08 end of lifetime: 2023-09 (planned)

OpenSSL 1.1.1 added TLS 1.3, SHA-3, X448 and Ed448.

-   Alpine (switched back to OpenSSL in 2018[3])
-   Arch Linux current
-   CentOS 8.0+
-   Debian 10 Buster
-   Debian 11 Bullseye (ETA 2021-06)
-   Fedora 29+
-   FreeBSD 11.3+
-   Gentoo Linux stable (dropped LibreSSL as alternative in January
    2021[4])
-   HardenedBSD (switched back to OpenSSL in 2018[5])
-   Linux Mint 19.3+
-   macOS (python.org installer)
-   NetBSD 8.2+
-   openSUSE 15.2+
-   RHEL 8.0+
-   Slackware current
-   SUSE Enterprise Linux 15-SP2
-   Ubuntu 18.10+
-   Ubuntu 20.04 LTS / Focal
-   VoidLinux (switched back to OpenSSL in March 2021[6])
-   Windows (python.org installer, Conda)

Major CI providers provide images with OpenSSL 1.1.1.

-   AppVeyor (with image Ubuntu2004)
-   CircleCI (with recent cimg/base:stable or cimg/base:stable-20.04)
-   GitHub Actions (with runs-on: ubuntu-20.04)
-   Giblab CI (with Debian Stretch, Ubuntu Focal, CentOS 8, RHEL 8, or
    Fedora runner)
-   Packit
-   TravisCI (with dist: focal)
-   Zuul

OpenSSL 3.0.0

released: n/a (planned for mid/late 2021)

OpenSSL 3.0.0 is currently under development. Major changes include
relicensing to Apache License 2.0 and a new API for cryptographic
algorithms providers. Most changes are internal refactorings and don’t
affect public APIs.[7]

LibreSSL

created: 2014-04 (forked from OpenSSL 1.0.1g)

-   DragonFly BSD
-   Hyperbola GNU/Linux-libre
-   OpenBSD
-   OpenELEC (discontinued)
-   TrueOS (discontinued)

Some distributions like FreeBSD and OPNsense also feature LibreSSL
instead of OpenSSL as non-standard TLS libraries. Gentoo discontinued
LibreSSL as an alternative to OpenSSL in January 2021[8] due to
compatibility issues and little testing.

OpenBSD ports has a port security/openssl/1.1 which is documented as
"[...] is present to provide support for applications which cannot be
made compatible with LibReSSL"[9]. The package could be used by OpenBSD
to provide a working ssl module.

BoringSSL

created: 2014-06

BoringSSL is Google’s fork of OpenSSL. It’s not intended for general use
and therefore not supported by Python. There are no guarantees of API or
ABI stability. Vendored copies of BoringSSL are used in Chrome/Chromium
browser, Android, and on Apple platforms[10].

Benefits

TLS 1.3

OpenSSL 1.1.1 introduced support for the new TLS 1.3 version. The latest
version of the TLS protocol has a faster handshake and is more secure
than the previous versions.

Thread and fork safety

Starting with release 1.1.0c, OpenSSL is fully fork and thread safe.
Bindings no longer need any workarounds or additional callbacks to
support multithreading.

SHA-3

Since 1.1.0, OpenSSL ships with SHA-3 and SHAKE implementations.
Python's builtin SHA-3 support is based on the reference implementation.
The internal _sha3 code is fairly large and the resulting shared library
close to 0.5 MB. Python could drop the builtin implementation and rely
on OpenSSL's libcrypto instead.

So far LibreSSL upstream development has refused to add SHA-3
support.[11]

Compatibility

OpenSSL downstream patches and options

OpenSSL features more than 70 configure and build time options in the
form of OPENSSL_NO_* macros. Around 60 options affect the presence of
features like cryptographic algorithms and TLS versions. Some
distributions apply patches to alter settings. Furthermore, default
values for settings like security level, ciphers, TLS version range, and
signature algorithms can be set in OpenSSL config file.

The Python core team lacks resources to test all possible combinations.
This PEP proposes that Python only supports OpenSSL builds that have
standard features enabled. Vendors shall disable deprecated or insecure
algorithms and TLS versions with build time options like
OPENSSL_NO_TLS1_1_METHOD or OpenSSL config options like
MinProtocol = TLSv1.2.

Python assumes that OpenSSL is built with

-   hashlib’s default algorithms such as MD5, SHA-1, SHA-2 family,
    SHA-3/SHAKE family, BLAKE2
-   TLS 1.2 and TLS 1.3 protocols
-   current key agreement, signature, and encryption algorithms for TLS
    1.2 and 1.3 (ECDH, RSA, ECDSA, Curve25519, AES, Poly1309-ChaCha20,
    ...)
-   threading, file I/O, socket I/O, and error messages

Weak algorithms (MD5, SHA-1 signatures) and short keys (RSA < 2024 bits)
may be disabled at runtime. Algorithms may also be blocked when they are
disabled by a crypto policy such as FIPS. The PEP is not more specific
on purpose to give room for new features as well as countermeasures
against vulnerabilities. As a rule of thumb, Python should be able to
connect to PyPI and the test suite should pass.

LibreSSL support

LibreSSL is a fork of OpenSSL. The fork was created off OpenSSL 1.0.1g
by members of the OpenBSD team in 2014 in light of the heartbleed
vulnerability. Since its inception several features deemed problematic
or insecure were removed or replaced (SSL 2.0, SSL 3.0, improved CPRNG)
or backported from OpenSSL and BoringSSL.

At the moment LibreSSL is not fully API compatible with OpenSSL 1.1.1.
The latest release LibreSSL 3.3.2 is missing features and behaves
differently in some cases. Mentionable missing or incompatible features
include

-   SHA-3, SHAKE, BLAKE2
-   SSL_CERT_* environment variables
-   security level APIs
-   session handling APIs
-   key logging API
-   verified cert chain APIs
-   OPENSSL_VERSION macro

This PEP proposed to remove any and all LibreSSL related workarounds
from Python. In the future Python will not actively prohibit LibreSSL
support with configure and compile time checks. But Python will not
accept patches that add non-trivial workarounds or disable tests either.

BoringSSL

There are currently no plans to support BoringSSL.

Rejected Ideas

Formalize supported OpenSSL versions

This PEP does not provide a set of formal rules and conditions under
which an OpenSSL version is supported.

In general Python aims to be compatible with commonly used and
officially supported OpenSSL versions. Patch releases of Python may not
be compatible with new major releases of OpenSSL. Users should not
expect that a new major or minor release of Python works with an OpenSSL
version that is past its end-of-lifetime. Python core development may
backport fixes for new releases or extend compatibility with EOLed
releases as we see fit.

The new ABI stability and LTS policies of OpenSSL[12] should help, too.

Keep support for OpenSSL 1.1.0

It was suggested to keep support for OpenSSL 1.1.0 for compatibility
with Debian 9 (Stretch). The proposal was rejected since it would
complicated code cleanup and testing. Stretch is already out of regular
security support and close to end of long-term support. By the time of
Python 3.10 final release, Debian Buster and Debian Bullseye will be
available.

Instead Python 3.10 will gain additional documentation and a new
configure option --with-openssl-rpath=auto to simplify use of custom
OpenSSL builds[13].

Backwards Compatibility

Python 3.10 will no longer support TLS/SSL and fast hashing on platforms
with OpenSSL 1.0.2 or LibreSSL. The first draft of this PEP was
published at the beginning of the 3.10 release cycles to give vendors
like Linux distributors or CI providers sufficient time to plan.

Python's internal copy of the Keccak Code Package and the internal _sha3
module will be removed. This will reduce source code size by about 280kB
and code size by roughly 0.5MB. The hashlib will solely rely on
OpenSSL's SHA-3 implementation. SHA-3 and SHAKE will no longer be
available without OpenSSL.

Disclaimer and special thanks

The author of this PEP is a contributor to OpenSSL project and employed
by a major Linux distributor that uses OpenSSL.

Thanks to Alex Gaynor, Gregory P. Smith, Nathaniel J. Smith, Paul
Kehrer, and Seth Larson for their review and feedback on the initial
draft.

References

Copyright

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.

[1] https://www.openssl.org/policies/releasestrat.html

[2] https://distrowatch.com/

[3] https://lists.alpinelinux.org/~alpine/devel/%3CCA%2BT2pCGFeh30aEi43hAvJ3yoHBijABy_U62wfjhVmf3FmbNUUg%40mail.gmail.com%3E

[4] https://www.gentoo.org/support/news-items/2021-01-05-libressl-support-discontinued.html

[5] https://hardenedbsd.org/article/shawn-webb/2018-04-30/hardenedbsd-switching-back-openssl

[6] https://voidlinux.org/news/2021/02/OpenSSL.html

[7] https://www.openssl.org/docs/OpenSSL300Design.html

[8] https://www.gentoo.org/support/news-items/2021-01-05-libressl-support-discontinued.html

[9] https://openports.se/security/openssl/1.1

[10] https://forums.swift.org/t/rfc-moving-swiftnio-ssl-to-boringssl/18280

[11] https://github.com/libressl-portable/portable/issues/455

[12] https://www.openssl.org/policies/releasestrat.html

[13] https://bugs.python.org/issue43466