PEP: 599 Title: The manylinux2014 Platform Tag Version: $Revision$
Last-Modified: $Date$ Author: Dustin Ingram <di@python.org> Sponsor:
Paul Moore <p.f.moore@gmail.com> BDFL-Delegate: Paul Moore
<p.f.moore@gmail.com> Discussions-To:
https://discuss.python.org/t/the-next-manylinux-specification/1043
Status: Superseded Type: Informational Topic: Packaging Content-Type:
text/x-rst Created: 29-Apr-2019 Post-History: 29-Apr-2019 Superseded-By:
600 Resolution:
https://discuss.python.org/t/the-next-manylinux-specification/1043/199

Abstract

This PEP proposes the creation of a manylinux2014 platform tag to
succeed the manylinux2010 tag introduced by PEP 513. It also proposes
that PyPI and pip both be updated to support uploading, downloading, and
installing manylinux2014 distributions on compatible platforms.

Rationale

CentOS 6 is now the oldest supported CentOS release, and will receive
maintenance updates through November 30th, 2020,[1] at which point it
will reach end-of-life, and no further updates such as security patches
will be made available. All wheels built under the manylinux2010 images
will remain at obsolete versions after that point.

Therefore, we propose the continuation of the existing manylinux
standard, and that a new PEP 425-style platform tag called manylinux2014
be derived from CentOS 7 and that the manylinux toolchain, PyPI, and pip
be updated to support it.

Similar to how PEP 571 and PEP 513 drew allowed shared libraries and
their symbol versions from CentOS 5.11 and CentOS 6, respectively, a
manylinux2014 platform tag will draw its libraries and symbol versions
from CentOS 7, which will reach end-of-life on June 30th, 2024.[2]

The manylinuxYYYY pattern has a number of advantages that motivate
continuing with the current status quo:

-   Well-defined Docker images with clearly specified compatible
    libraries;
-   No need to survey for compatibility issues across multiple releases;
-   A single build image and auditwheel profile per architecture.

There are also some disadvantages:

-   Requires drafting a new PEP for every new standard;
-   Requires adding the new platform tag to installers (e.g., pip);
-   Installers are unable to install a platform tag which predates a
    given release.

There are also challenges which would exist for any proposal, including
the time and effort it takes to define, prepare and release the Docker
images and corresponding auditwheel profiles. These challenges were
experienced in the long rollout period for manylinux2010, which took
approximately 1 year from PEP acceptance to compatible build environment
published.[3]

However, if this PEP can be an indicator, the process is now
well-defined and easily repeatable, which should increase the timeline
for rollout of a newer, updated platform tag.

The manylinux2014 policy

The following criteria determine a linux wheel's eligibility for the
manylinux2014 tag:

1.  The wheel may only contain binary executables and shared objects
    compiled for one of the following architectures supported by CentOS
    7, or a CentOS 7 compatible base image (such as ubi7):[4] :

        x86_64
        i686
        aarch64
        armv7l
        ppc64
        ppc64le
        s390x

    This list adds support for ARMv7 (armv7l), ARMv8 (aarch64) and
    PowerPC (ppc64, ppc64le) architectures supported by the CentOS
    Alternative Architecture Special Interest Group, as well as the IBM
    Z (s390x) architecture.[5]

2.  The wheel's binary executables or shared objects may not link
    against externally-provided libraries except those in the following
    list: :

        libgcc_s.so.1
        libstdc++.so.6
        libm.so.6
        libdl.so.2
        librt.so.1
        libc.so.6
        libnsl.so.1
        libutil.so.1
        libpthread.so.0
        libresolv.so.2
        libX11.so.6
        libXext.so.6
        libXrender.so.1
        libICE.so.6
        libSM.so.6
        libGL.so.1
        libgobject-2.0.so.0
        libgthread-2.0.so.0
        libglib-2.0.so.0

    This list is identical to the externally-provided libraries
    originally allowed for manylinux2010, with one exception:
    libcrypt.so.1 was removed due to being deprecated in Fedora 30.
    libpythonX.Y remains ineligible for inclusion for the same reasons
    outlined in PEP 513.

    On Debian-based systems, these libraries are provided by the
    packages:

      Package        Libraries
      -------------- ----------------------------------------------------------------------------------------------------------
      libc6          libdl.so.2, libresolv.so.2, librt.so.1, libc.so.6, libpthread.so.0, libm.so.6, libutil.so.1, libnsl.so.1
      libgcc1        libgcc_s.so.1
      libgl1         libGL.so.1
      libglib2.0-0   libgobject-2.0.so.0, libgthread-2.0.so.0, libglib-2.0.so.0
      libice6        libICE.so.6
      libsm6         libSM.so.6
      libstdc++6     libstdc++.so.6
      libx11-6       libX11.so.6
      libxext6       libXext.so.6
      libxrender1    libXrender.so.1

    On RPM-based systems, they are provided by these packages:

      Package      Libraries
      ------------ ----------------------------------------------------------------------------------------------------------
      glib2        libglib-2.0.so.0, libgthread-2.0.so.0, libgobject-2.0.so.0
      glibc        libresolv.so.2, libutil.so.1, libnsl.so.1, librt.so.1, libpthread.so.0, libdl.so.2, libm.so.6, libc.so.6
      libICE       libICE.so.6
      libX11       libX11.so.6
      libXext:     libXext.so.6
      libXrender   libXrender.so.1
      libgcc:      libgcc_s.so.1
      libstdc++    libstdc++.so.6
      mesa         libGL.so.1

3.  If the wheel contains binary executables or shared objects linked
    against any allowed libraries that also export versioned symbols,
    they may only depend on the following maximum versions:

        GLIBC_2.17
        CXXABI_1.3.7, CXXABI_TM_1 is also allowed
        GLIBCXX_3.4.19
        GCC_4.8.0

    As an example, manylinux2014 wheels may include binary artifacts
    that require glibc symbols at version GLIBC_2.12, because this an
    earlier version than the maximum of GLIBC_2.17.

4.  If a wheel is built for any version of CPython 2 or CPython versions
    3.0 up to and including 3.2, it must include a CPython ABI tag
    indicating its Unicode ABI. A manylinux2014 wheel built against
    Python 2, then, must include either the cpy27mu tag indicating it
    was built against an interpreter with the UCS-4 ABI or the cpy27m
    tag indicating an interpreter with the UCS-2 ABI. (PEP 3149[6])

5.  A wheel must not require the PyFPE_jbuf symbol. This is achieved by
    building it against a Python compiled without the --with-fpectl
    configure flag.

Compilation of Compliant Wheels

Like manylinux1, the auditwheel tool adds manylinux2014 platform tags to
linux wheels built by pip wheel or bdist_wheel in a manylinux2014 Docker
container.

Docker Images

A manylinux2014 Docker image based on CentOS 7 x86_64 should be provided
for building binary linux wheels that can reliably be converted to
manylinux2014 wheels. This image will come with a full compiler suite
installed (gcc, g++, and gfortran 4.8.5) as well as the latest releases
of Python and pip.

Auditwheel

The auditwheel tool will also be updated to produce manylinux2014
wheels.[7] Its behavior and purpose will be otherwise unchanged from PEP
513.

Platform Detection for Installers

Platforms may define a manylinux2014_compatible boolean attribute on the
_manylinux module described in PEP 513. A platform is considered
incompatible with manylinux2014 if the attribute is False.

If the _manylinux module is not found, or it does not have the attribute
manylinux2014_compatible, tools may fall back to checking for glibc. If
the platform has glibc 2.17 or newer, it is assumed to be compatible
unless the _manylinux module says otherwise.

Specifically, the algorithm we propose is:

    def is_manylinux2014_compatible():
        # Only Linux, and only supported architectures
        from distutils.util import get_platform

        if get_platform() not in [
            "linux-x86_64",
            "linux-i686",
            "linux-aarch64",
            "linux-armv7l",
            "linux-ppc64",
            "linux-ppc64le",
            "linux-s390x",
        ]:
            return False

        # Check for presence of _manylinux module
        try:
            import _manylinux

            return bool(_manylinux.manylinux2014_compatible)
        except (ImportError, AttributeError):
            # Fall through to heuristic check below
            pass

        # Check glibc version. CentOS 7 uses glibc 2.17.
        # PEP 513 contains an implementation of this function.
        return have_compatible_glibc(2, 17)

Backwards compatibility with manylinux2010 wheels

As explained in PEP 513, the specified symbol versions for manylinux1
allowed libraries constitute an upper bound. The same is true for the
symbol versions defined for manylinux2014 in this PEP. As a result,
manylinux1 and manylinux2010 wheels are considered manylinux2014 wheels.
A pip that recognizes the manylinux2014 platform tag will thus install
manylinux2010 wheels for manylinux2014 platforms -- even when explicitly
set --when no manylinux2014 wheels are available.

PyPI Support

PyPI should permit wheels containing the manylinux2014 platform tag to
be uploaded in the same way that it permits manylinux2010.

If technically feasible, PyPI should attempt to verify the compatibility
of manylinux2014 wheels, but that capability is not a requirement for
adoption of this PEP.

Package authors should not upload non-compliant manylinux2014 wheels to
PyPI, and should be aware that PyPI may begin blocking non-compliant
wheels from being uploaded.

References

Acceptance

PEP 599 was accepted by Paul Moore on July 31, 2019.

Copyright

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.

[1] CentOS Product Specifications
(https://wiki.centos.org/About/Product)

[2] CentOS Product Specifications
(https://wiki.centos.org/About/Product)

[3] Tracking issue for manylinux2010 rollout
(https://github.com/pypa/manylinux/issues/179)

[4] Red Hat Universal Base Image 7
(https://access.redhat.com/containers/?tab=overview#/registry.access.redhat.com/ubi7)

[5] The CentOS Alternative Architecture Special Interest Group
(https://wiki.centos.org/SpecialInterestGroup/AltArch)

[6] SOABI support for Python 2.X and PyPy
(https://github.com/pypa/pip/pull/3075)

[7] auditwheel (https://github.com/pypa/auditwheel/)