PEP: 726 Title: Module __setattr__ and __delattr__ Author: Sergey B
Kirpichev <skirpichev@gmail.com> Sponsor: Adam Turner
<python@quite.org.uk> Discussions-To:
https://discuss.python.org/t/32640/ Status: Rejected Type: Standards
Track Content-Type: text/x-rst Created: 24-Aug-2023 Python-Version: 3.13
Post-History: 06-Apr-2023, 31-Aug-2023, Resolution:
https://discuss.python.org/t/32640/32

Abstract

This PEP proposes supporting user-defined __setattr__ and __delattr__
methods on modules to extend customization of module attribute access
beyond PEP 562.

Motivation

There are several potential uses of a module __setattr__:

1.  To prevent setting an attribute at all (i.e. make it read-only)
2.  To validate the value to be assigned
3.  To intercept setting an attribute and update some other state

Proper support for read-only attributes would also require adding the
__delattr__ function to prevent their deletion.

It would be convenient to directly support such customization, by
recognizing __setattr__ and __delattr__ methods defined in a module that
would act like normal python:object.__setattr__ and
python:object.__delattr__ methods, except that they will be defined on
module instances. Together with existing __getattr__ and __dir__ methods
this will streamline all variants of customizing module attribute
access.

For example

    # mplib.py

    CONSTANT = 3.14
    prec = 53
    dps = 15

    def dps_to_prec(n):
        """Return the number of bits required to represent n decimals accurately."""
        return max(1, int(round((int(n)+1)*3.3219280948873626)))

    def prec_to_dps(n):
        """Return the number of accurate decimals that can be represented with n bits."""
        return max(1, int(round(int(n)/3.3219280948873626)-1))

    def validate(n):
        n = int(n)
        if n <= 0:
            raise ValueError('Positive integer expected')
        return n

    def __setattr__(name, value):
        if name == 'CONSTANT':
            raise AttributeError('Read-only attribute!')
        if name == 'dps':
            value = validate(value)
            globals()['dps'] = value
            globals()['prec'] = dps_to_prec(value)
            return
        if name == 'prec':
            value = validate(value)
            globals()['prec'] = value
            globals()['dps'] = prec_to_dps(value)
            return
        globals()[name] = value

    def __delattr__(name):
        if name in ('CONSTANT', 'dps', 'prec'):
            raise AttributeError('Read-only attribute!')
        del globals()[name]

    >>> import mplib
    >>> mplib.foo = 'spam'
    >>> mplib.CONSTANT = 42
    Traceback (most recent call last):
      ...
    AttributeError: Read-only attribute!
    >>> del mplib.foo
    >>> del mplib.CONSTANT
    Traceback (most recent call last):
      ...
    AttributeError: Read-only attribute!
    >>> mplib.prec
    53
    >>> mplib.dps
    15
    >>> mplib.dps = 5
    >>> mplib.prec
    20
    >>> mplib.dps = 0
    Traceback (most recent call last):
      ...
    ValueError: Positive integer expected

Existing Options

The current workaround is assigning the __class__ of a module object to
a custom subclass of python:types.ModuleType (see[1]).

For example, to prevent modification or deletion of an attribute we
could use:

    # mod.py

    import sys
    from types import ModuleType

    CONSTANT = 3.14

    class ImmutableModule(ModuleType):
        def __setattr__(name, value):
            raise AttributeError('Read-only attribute!')

        def __delattr__(name):
            raise AttributeError('Read-only attribute!')

    sys.modules[__name__].__class__ = ImmutableModule

But this variant is slower (~2x) than the proposed solution. More
importantly, it also brings a noticeable speed regression (~2-3x) for
attribute access.

Specification

The __setattr__ function at the module level should accept two
arguments, the name of an attribute and the value to be assigned, and
return None or raise an AttributeError.

    def __setattr__(name: str, value: typing.Any, /) -> None: ...

The __delattr__ function should accept one argument, the name of an
attribute, and return None or raise an AttributeError:

    def __delattr__(name: str, /) -> None: ...

The __setattr__ and __delattr__ functions are looked up in the module
__dict__. If present, the appropriate function is called to customize
setting the attribute or its deletion, else the normal mechanism
(storing/deleting the value in the module dictionary) will work.

Defining module __setattr__ or __delattr__ only affects lookups made
using the attribute access syntax --- directly accessing the module
globals (whether by globals() within the module, or via a reference to
the module's globals dictionary) is unaffected. For example:

    >>> import mod
    >>> mod.__dict__['foo'] = 'spam'  # bypasses __setattr__, defined in mod.py

or

    # mod.py

    def __setattr__(name, value):
       ...

    foo = 'spam'  # bypasses __setattr__
    globals()['bar'] = 'spam'  # here too

    def f():
        global x
        x = 123

    f()  # and here

To use a module global and trigger __setattr__ (or __delattr__), one can
access it via sys.modules[__name__] within the module's code:

    # mod.py

    sys.modules[__name__].foo = 'spam'  # bypasses __setattr__

    def __setattr__(name, value):
        ...

    sys.modules[__name__].bar = 'spam'  # triggers __setattr__

This limitation is intentional (just as for the PEP 562), because the
interpreter highly optimizes access to module globals and disabling all
that and going through special methods written in Python would slow down
the code unacceptably.

How to Teach This

The "Customizing module attribute access"[2] section of the
documentation will be expanded to include new functions.

Reference Implementation

The reference implementation for this PEP can be found in CPython PR
#108261.

Backwards compatibility

This PEP may break code that uses module level (global) names
__setattr__ and __delattr__, but the language reference explicitly
reserves all undocumented dunder names, and allows "breakage without
warning"[3].

The performance implications of this PEP are small, since additional
dictionary lookup is much cheaper than storing/deleting the value in the
dictionary. Also it is hard to imagine a module that expects the user to
set (and/or delete) attributes enough times to be a performance concern.
On another hand, proposed mechanism allows to override setting/deleting
of attributes without affecting speed of attribute access, which is much
more likely scenario to get a performance penalty.

Discussion

As pointed out by Victor Stinner, the proposed API could be useful
already in the stdlib, for example to ensure that sys.modules type is
always a dict:

    >>> import sys
    >>> sys.modules = 123
    >>> import asyncio
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "<frozen importlib._bootstrap>", line 1260, in _find_and_load
    AttributeError: 'int' object has no attribute 'get'

or to prevent deletion of critical sys attributes, which makes the code
more complicated. For example, code using sys.stderr has to check if the
attribute exists and if it's not None. Currently, it's possible to
remove any sys attribute, including functions:

    >>> import sys
    >>> del sys.excepthook
    >>> 1+  # notice the next line
    sys.excepthook is missing
     File "<stdin>", line 1
       1+
        ^
    SyntaxError: invalid syntax

See related issue for other details.

Other stdlib modules also come with attributes which can be overridden
(as a feature) and some input validation here could be helpful.
Examples: threading.excepthook, warnings.showwarning,
io.DEFAULT_BUFFER_SIZE or os.SEEK_SET.

Also a typical use case for customizing module attribute access is
managing deprecation warnings. But the PEP 562 accomplishes this
scenario only partially: e.g. it's impossible to issue a warning during
an attempt to change a renamed attribute.

Footnotes

Copyright

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.

[1] Customizing module attribute access
(https://docs.python.org/3.11/reference/datamodel.html#customizing-module-attribute-access)

[2] Customizing module attribute access
(https://docs.python.org/3.11/reference/datamodel.html#customizing-module-attribute-access)

[3] Reserved classes of identifiers
(https://docs.python.org/3.11/reference/lexical_analysis.html#reserved-classes-of-identifiers)