PEP: 699 Title: Remove private dict version field added in PEP 509
Author: Ken Jin <kenjin@python.org> Discussions-To:
https://discuss.python.org/t/pep-699-remove-private-dict-version-field-added-in-pep-509/19724
Status: Accepted Type: Standards Track Content-Type: text/x-rst Created:
03-Oct-2022 Python-Version: 3.12 Post-History: 05-Oct-2022 Replaces: 509
Resolution:
https://discuss.python.org/t/pep-699-remove-private-dict-version-field-added-in-pep-509/19724/13

Abstract

PEP 509 introduced a private ma_version_tag field for dictionaries to
allow optimizations in CPython and extension libraries. This PEP
proposes to rescind PEP 509 and declare the field an implementation
detail, as it has already been superseded by alternatives. This will
further allow the field to be reused for future optimization.

Motivation

PEP 509 introduced the ma_version_tag field to dictionaries. This 64-bit
field stored a version counter that updates every time a dictionary was
modified. The original proposal was to use this version counter as a
guard for optimizations.

Since CPython 3.11, this field has become unused by internal
optimization efforts. PEP 659-specialized instructions use other methods
of verifying that certain optimizations are safe.

To enable further optimizations in CPython, this PEP proposes that the
ma_version_tag field no longer conform to the PEP 509 specification.
This will allow the CPython developers to store other optimization
information, such as dictionary write watchers.

Rationale

This PEP does not specify what the field may be used for in the future.
This is intentional, as implementation details are subject to change,
and the field should be used only for internal consumption by CPython.

Specification

This specification rescinds that in PEP 509. The ma_version_tag field of
the Python dict class is declared to be an internal implementation
detail and may be removed altogether, or may have a different
representation. C extensions should not rely on this field.

Backwards Compatibility

Certain extensions use ma_version_tag for fast dictionary or globals
lookups. For example, Cython uses the field for fast dynamic module
variable lookups.

This PEP proposes to emit a compiler warning when accessing
ma_version_tag. After two consecutive version releases with warnings,
ma_version_tag will be removed, in line with PEP 387.

The biggest user the author could find for this field was Cython.
Discussions with a Cython maintainer indicated that removing support for
it from Cython is trivial.

Security Implications

PEP 509 was concerned with integer overflow. However, this PEP does not
introduce any additional security concerns.

Rejected Ideas

A possible alternative is to preserve the field for backwards
compatibility. This PEP rejects that alternative as future optimizations
will consume more memory, and the field was always considered private
and undocumented aside from the PEP, with no backward compatibility
guarantees. Dictionaries in Python are commonly used, and any increase
in their memory consumption will adversely affect Python’s performance.

Reference Implementation

A reference implementation can be found in python/cpython#101193.

Special Thanks

Thanks to C.A.M. Gerlach for edits and wording changes to this document.
Thanks also to Mark Shannon and Kumar Aditya for providing possible
implementations.

Copyright

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.