PEP: 611 Title: The one million limit Author: Mark Shannon
<mark@hotpy.org> Status: Withdrawn Type: Standards Track Content-Type:
text/x-rst Created: 05-Dec-2019 Post-History:

Abstract

This PR proposes a soft limit of one million (1 000 000), and a larger
hard limit for various aspects of Python code and its implementation.

The Python language does not specify limits for many of its features.
Not having any limit to these values seems to enhance programmer
freedom, at least superficially, but in practice the CPython VM and
other Python virtual machines have implicit limits or are forced to
assume that the limits are astronomical, which is expensive.

This PR lists a number of features which are to have a limit of one
million.

For CPython the hard limit will be eight million (8 000 000).

Motivation

There are many values that need to be represented in a virtual machine.
If no limit is specified for these values, then the representation must
either be inefficient or vulnerable to overflow. The CPython virtual
machine represents values like line numbers, stack offsets and
instruction offsets by 32 bit values. This is inefficient, and
potentially unsafe.

It is inefficient as actual values rarely need more than a dozen or so
bits to represent them.

It is unsafe as malicious or poorly generated code could cause values to
exceed 2³².

For example, line numbers are represented by 32 bit values internally.
This is inefficient, given that modules almost never exceed a few
thousand lines. Despite being inefficient, it is still vulnerable to
overflow as it is easy for an attacker to created a module with billions
of newline characters.

Memory access is usually a limiting factor in the performance of modern
CPUs. Better packing of data structures enhances locality and reduces
memory bandwidth, at a modest increase in ALU usage (for shifting and
masking). Being able to safely store important values in 20 bits would
allow memory savings in several data structures including, but not
limited to:

-   Frame objects
-   Object headers
-   Code objects

There is also the potential for a more efficient instruction format,
speeding up interpreter dispatch.

Is this a worthwhile trade off?

The downside of any form of limit is that it might potentially make
someone's job harder, for example, it may be harder to write a code
generator that keeps the size of modules to one million lines. However,
it is the author's opinion, having written many code generators, that
such a limit is extremely unlikely to be a problem in practice.

The upside of these limits is the freedom it grants implementers of
runtimes, whether CPython, PyPy, or any other implementation, to improve
performance. It is the author's belief, that the potential value of even
a 0.1% reduction in the cost of running Python programs globally will
hugely exceed the cost of modifying a handful of code generators.

Rationale

Imposing a limit on values such as lines of code in a module, and the
number of local variables, has significant advantages for ease of
implementation and efficiency of virtual machines. If the limit is
sufficiently large, there is no adverse effect on users of the language.

By selecting a fixed but large limit for these values, it is possible to
have both safety and efficiency whilst causing no inconvenience to human
programmers and only very rare problems for code generators.

One million

The value "one million" is very easy to remember.

The one million limit is mostly a limit on human generated code, not
runtime sizes.

One million lines in a single module is a ridiculous concentration of
code; the entire Python standard library is about 2/3rd of a million
lines, spread over 1600 files.

The Java Virtual Machine (JVM)[1] specifies a limit of 2¹⁶-1 (65535) for
many program elements similar to those covered here. This limit enables
limited values to fit in 16 bits, which is a very efficient machine
representation. However, this limit is quite easily exceeded in practice
by code generators and the author is aware of existing Python code that
already exceeds 2¹⁶ lines of code.

The hard limit of eight million fits into 23 bits which, although not as
convenient for machine representation, is still reasonably compact. A
limit of eight million is small enough for efficiency advantages (only
23 bits), but large enough not to impact users (no one has ever written
a module that large).

While it is possible that generated code could exceed the limit, it is
easy for a code generator to modify its output to conform. The author
has hit the 64K limit in the JVM on at least two occasions when
generating Java code. The workarounds were relatively straightforward
and wouldn't have been necessary with a limit of one million bytecodes
or lines of code.

Where necessary, the soft limit can increased for those programs that
exceed the one million limit.

Having a soft limit of one million provides a warning of problematic
code, without causing an error and forcing an immediate fix. It also
allows dynamic optimizers to use more compact formats without inline
checks.

Specification

This PR proposes that the following language features and runtime values
have a soft limit of one million.

-   The number of source code lines in a module
-   The number of bytecode instructions in a code object.
-   The sum of local variables and stack usage for a code object.
-   The number of classes in a running interpreter.
-   The recursion depth of Python code.

It is likely that memory constraints would be a limiting factor before
the number of classes reaches one million.

Recursion depth

The recursion depth limit only applies to pure Python code. Code written
in a foreign language, such as C, may consume hardware stack and thus be
limited to a recursion depth of a few thousand. It is expected that
implementations will raise an exception should the hardware stack get
close to its limit. For code that mixes Python and C calls, it is most
likely that the hardware limit will apply first. The size of the
hardware recursion may vary at runtime and will not be visible.

Soft and hard limits

Implementations should emit a warning whenever a soft limit is exceeded,
unless the hard limit has the same value as the soft limit. When a hard
limit is exceeded, then an exception should be raised.

Depending on the implementation, different hard limits might apply. In
some cases the hard limit might be below the soft limit. For example,
many micropython ports are unlikely to be able to support such large
limits.

Introspecting and modifying the limits

One or more functions will be provided in the sys module to introspect
or modify the soft limits at runtime, but the limits may not be raised
above the hard limit.

Inferred limits

These limits are not part of the specification, but a limit of less than
one million can be inferred from the limit on the number of bytecode
instructions in a code object. Because there would be insufficient
instructions to load more than one million constants or use more than
one million names.

-   The number of distinct names in a code object.
-   The number of constants in a code object.

The advantages for CPython of imposing these limits:

Line of code in a module and code object restrictions.

When compiling source code to bytecode or modifying bytecode for
profiling or debugging, an intermediate form is required. By limiting
operands to 23 bits, instructions can be represented in a compact 64 bit
form allowing very fast passes over the instruction sequence.

Having 23 bit operands (24 bits for relative branches) allows
instructions to fit into 32 bits without needing additional EXTENDED_ARG
instructions. This improves dispatch, as the operand is strictly local
to the instruction. It is unclear whether this would help performance,
it is merely an example of what is possible.

The benefit of restricting the number of lines in a module is primarily
the implied limit on bytecodes. It is more important for implementations
that it is instructions per code object, not lines per module, that is
limited to one million, but it is much easier to explain a one million
line limit. Having a consistent limit of one million is just easier to
remember. It is mostly likely, although not guaranteed, that the line
limit will be hit first and thus provide a simpler to understand error
message to the developer.

Total number of classes in a running interpreter

This limit has to the potential to reduce the size of object headers
considerably.

Currently objects have a two word header, for objects without references
(int, float, str, etc.) or a four word header for objects with
references. By reducing the maximum number of classes, the space for the
class reference can be reduced from 64 bits to fewer than 32 bits
allowing a much more compact header.

For example, a super-compact header format might look like this:

    struct header {
        uint32_t gc_flags:6; /* Needs finalisation, might be part of a cycle, etc. */
        uint32_t class_id:26; /* Can be efficiently mapped to address by ensuring suitable alignment of classes */
        uint32_t refcount; /* Limited memory or saturating */
    }

This format would reduce the size of a Python object without slots, on a
64 bit machine, from 40 to 16 bytes.

Note that there are two ways to use a 32 bit refcount on a 64 bit
machine. One is to limit each sub-interpreter to 32Gb of memory. The
other is to use a saturating reference count, which would be a little
bit slower, but allow unlimited memory allocation.

Enforcement

Python implementations are not obliged to enforce the limits. However,
if a limit can be enforced without hurting performance, then it should
be.

It is anticipated that CPython will enforce the limits as follows:

-   The number of source code lines in a module: version 3.9 onward.
-   The number of bytecode instructions in a code object: 3.9 onward.
-   The sum of local variables and stack usage for a code object: 3.9
    onward.
-   The number of classes in a running interpreter: probably 3.10
    onward, maybe warning in 3.9.

Hard limits in CPython

CPython will enforce a hard limit on all the above values. The value of
the hard limit will be 8 million.

It is hypothetically possible that some machine generated code exceeds
one or more of the above limits. The author believes that to be
incredibly unlikely and easily fixed by modifying the output stage of
the code generator.

We would like to gain the benefit from the above limits for performance
as soon as possible. To that end, CPython will start applying limits
from version 3.9 onward. To ease the transition and minimize breakage,
the initial limits will be 16 million, reducing to 8 million in a later
version.

Backwards Compatibility

The actual hard limits enforced by CPython will be:

+---------------+--------------+
| Version       | Hard limit   |
+===============+==============+
|   3.9         |   16 million |
+---------------+--------------+
|   3.10 onward |   8 million  |
+---------------+--------------+

Given the rarity of code generators that would exceed the one million
limits, and the environments in which they are typically used, it seems
reasonable to start issuing warnings in 3.9 if any limited quantity
exceeds one million.

Historically the recursion limit has been set at 1000. To avoid breaking
code that implicitly relies on the value being small, the soft recursion
limit will be increased gradually, as follows:

+---------+-------------+
| Version | Soft limit  |
+=========+=============+
|   3.9   |   4 000     |
+---------+-------------+
|   3.10  |   16 000    |
+---------+-------------+
|   3.11  |   64 000    |
+---------+-------------+
|   3.12  |   125 000   |
+---------+-------------+
|   3.13  |   1 million |
+---------+-------------+

The hard limit will be set to 8 million immediately.

Other implementations

Implementations of Python other than CPython have different purposes, so
different limits might be appropriate. This is acceptable, provided the
limits are clearly documented.

General purpose implementations

General purpose implementations, such as PyPy, should use the one
million limit. If maximum compatibility is a goal, then they should also
follow CPython's behaviour for 3.9 to 3.11.

Special purpose implementations

Special purpose implementations may use lower limits, as long as they
are clearly documented. An implementation designed for embedded systems,
for example MicroPython, might impose limits as low as a few thousand.

Security Implications

Minimal. This reduces the attack surface of any Python virtual machine
by a small amount.

Reference Implementation

None, as yet. This will be implemented in CPython, once the PEP has been
accepted.

Rejected Ideas

Being able to modify the hard limits upwards at compile time was
suggested by Tal Einat. This is rejected as the current limits of 2³²
have not been an issue, and the practical advantages of allowing limits
between 2²⁰ and 2³² seem slight compared to the additional code
complexity of supporting such a feature.

Open Issues

None, as yet.

References

https://docs.oracle.com/javase/specs/jvms/se8/jvms8.pdf

Copyright

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.

[1] The Java Virtual Machine specification