PEP: 702 Title: Marking deprecations using the type system Author: Jelle
Zijlstra <jelle.zijlstra@gmail.com> Discussions-To:
https://discuss.python.org/t/pep-702-marking-deprecations-using-the-type-system/23036
Status: Final Type: Standards Track Topic: Typing Created: 30-Dec-2022
Python-Version: 3.13 Post-History: 01-Jan-2023, 22-Jan-2023 Resolution:
07-Nov-2023

typing:deprecated and @warnings.deprecated<warnings.deprecated>

Abstract

This PEP adds an @warnings.deprecated() decorator that marks a class or
function as deprecated, enabling static checkers to warn when it is
used. By default, this decorator will also raise a runtime
DeprecationWarning.

Motivation

As software evolves, new functionality is added and old functionality
becomes obsolete. Library developers want to work towards removing
obsolete code while giving their users time to migrate to new APIs.
Python provides a mechanism for achieving these goals: the
DeprecationWarning warning class, which is used to show warnings when
deprecated functionality is used. This mechanism is widely used: as of
the writing of this PEP, the CPython main branch contains about 150
distinct code paths that raise !DeprecationWarning. Many third-party
libraries also use !DeprecationWarning to mark deprecations. In the top
5000 PyPI packages, there are:

-   1911 matches for the regex warnings\.warn.*\bDeprecationWarning\b,
    indicating use of !DeprecationWarning (not including cases where the
    warning is split over multiple lines);
-   1661 matches for the regex ^\s*@deprecated, indicating use of some
    sort of deprecation decorator.

However, the current mechanism is often insufficient to ensure that
users of deprecated functionality update their code in time. For
example, the removal of various long-deprecated unittest features had to
be reverted from Python 3.11 to give users more time to update their
code. Users may run their test suite with warnings disabled for
practical reasons, or deprecations may be triggered in code paths that
are not covered by tests.

Providing more ways for users to find out about deprecated functionality
can speed up the migration process. This PEP proposes to leverage static
type checkers to communicate deprecations to users. Such checkers have a
thorough semantic understanding of user code, enabling them to detect
and report deprecations that a single grep invocation could not find. In
addition, many type checkers integrate with IDEs, enabling users to see
deprecation warnings right in their editors.

Rationale

At first glance, deprecations may not seem like a topic that type
checkers should touch. After all, type checkers are concerned with
checking whether code will work as is, not with potential future
changes. However, the analysis that type checkers perform on code to
find type errors is very similar to the analysis that would be needed to
detect usage of many deprecations. Therefore, type checkers are well
placed to find and report deprecations.

Other languages already have similar functionality:

-   GCC supports a deprecated attribute on function declarations. This
    powers CPython's Py_DEPRECATED macro.
-   GraphQL supports marking fields as @deprecated.
-   Kotlin supports a Deprecated annotation.
-   Scala supports an @deprecated annotation.
-   Swift supports using the @available attribute to mark APIs as
    deprecated.
-   TypeScript uses the @deprecated JSDoc tag to issue a hint marking
    use of deprecated functionality.

Several users have requested support for such a feature:

-   typing-sig thread
-   Pyright feature request
-   mypy feature request

There are similar existing third-party tools:

-   Deprecated provides a decorator to mark classes, functions, or
    methods as deprecated. Access to decorated objects raises a runtime
    warning, but is not detected by type checkers.
-   flake8-deprecated is a linter plugin that warns about use of
    deprecated features. However, it is limited to a short, hard-coded
    list of deprecations.

Specification

A new decorator @deprecated() is added to the warnings module. This
decorator can be used on a class, function or method to mark it as
deprecated. This includes typing.TypedDict and typing.NamedTuple
definitions. With overloaded functions, the decorator may be applied to
individual overloads, indicating that the particular overload is
deprecated. The decorator may also be applied to the overload
implementation function, indicating that the entire function is
deprecated.

The decorator takes the following arguments:

-   A required positional-only argument representing the deprecation
    message.
-   Two keyword-only arguments, category and stacklevel, controlling
    runtime behavior (see under "Runtime behavior" below).

The positional-only argument is of type str and contains a message that
should be shown by the type checker when it encounters a usage of the
decorated object. Tools may clean up the deprecation message for
display, for example by using inspect.cleandoc or equivalent logic. The
message must be a string literal. The content of deprecation messages is
up to the user, but it may include the version in which the deprecated
object is to be removed, and information about suggested replacement
APIs.

Type checkers should produce a diagnostic whenever they encounter a
usage of an object marked as deprecated. For deprecated overloads, this
includes all calls that resolve to the deprecated overload. For
deprecated classes and functions, this includes:

-   References through module, class, or instance attributes
    (module.deprecated_object, module.SomeClass.deprecated_method,
    module.SomeClass().deprecated_method)
-   Any usage of deprecated objects in their defining module
    (x = deprecated_object() in module.py)
-   If import * is used, usage of deprecated objects from the module
    (from module import *; x = deprecated_object())
-   from imports (from module import deprecated_object)
-   Any syntax that indirectly triggers a call to the function. For
    example, if the __add__ method of a class C is deprecated, then the
    code C() + C() should trigger a diagnostic. Similarly, if the setter
    of a property is marked deprecated, attempts to set the property
    should trigger a diagnostic.

If a method is marked with the typing.override decorator from PEP 698
and the base class method it overrides is deprecated, the type checker
should produce a diagnostic.

There are additional scenarios where deprecations could come into play.
For example, an object may implement a typing.Protocol, but one of the
methods required for protocol compliance is deprecated. As scenarios
such as this one appear complex and relatively unlikely to come up in
practice, this PEP does not mandate that type checkers detect them.

Example

As an example, consider this library stub named library.pyi:

    from warnings import deprecated

    @deprecated("Use Spam instead")
    class Ham: ...

    @deprecated("It is pining for the fiords")
    def norwegian_blue(x: int) -> int: ...

    @overload
    @deprecated("Only str will be allowed")
    def foo(x: int) -> str: ...
    @overload
    def foo(x: str) -> str: ...

    class Spam:
        @deprecated("There is enough spam in the world")
        def __add__(self, other: object) -> object: ...

        @property
        @deprecated("All spam will be equally greasy")
        def greasy(self) -> float: ...

        @property
        def shape(self) -> str: ...
        @shape.setter
        @deprecated("Shapes are becoming immutable")
        def shape(self, value: str) -> None: ...

Here is how type checkers should handle usage of this library:

    from library import Ham  # error: Use of deprecated class Ham. Use Spam instead.

    import library

    library.norwegian_blue(1)  # error: Use of deprecated function norwegian_blue. It is pining for the fiords.
    map(library.norwegian_blue, [1, 2, 3])  # error: Use of deprecated function norwegian_blue. It is pining for the fiords.

    library.foo(1)  # error: Use of deprecated overload for foo. Only str will be allowed.
    library.foo("x")  # no error

    ham = Ham()  # no error (already reported above)

    spam = library.Spam()
    spam + 1  # error: Use of deprecated method Spam.__add__. There is enough spam in the world.
    spam.greasy  # error: Use of deprecated property Spam.greasy. All spam will be equally greasy.
    spam.shape  # no error
    spam.shape = "cube"  # error: Use of deprecated property setter Spam.shape. Shapes are becoming immutable.

The exact wording of the diagnostics is up to the type checker and is
not part of the specification.

Runtime behavior

In addition to the positional-only message argument, the @deprecated
decorator takes two keyword-only arguments:

-   category: A warning class. Defaults to DeprecationWarning. If this
    is set to None, no warning is issued at runtime and the decorator
    returns the original object, except for setting the __deprecated__
    attribute (see below).
-   stacklevel: The number of stack frames to skip when issuing the
    warning. Defaults to 1, indicating that the warning should be issued
    at the site where the deprecated object is called. Internally, the
    implementation will add the number of stack frames it uses in
    wrapper code.

If the decorated object is a class, the decorator wraps the __new__
method such that instantiating the class issues a warning. If the
decorated object is a callable, the decorator returns a new callable
that wraps the original callable but raises a warning when called.
Otherwise, the decorator raises a TypeError (unless category=None is
passed).

There are several scenarios where use of the decorated object cannot
issue a warning, including overloads, Protocol classes, and abstract
methods. Type checkers may show a warning if @deprecated is used without
category=None in these cases.

To accommodate runtime introspection, the decorator sets an attribute
__deprecated__ on the object it is passed, as well as on the wrapper
callables it generates for deprecated classes and functions. The value
of the attribute is the message passed to the decorator. Decorating
objects that do not allow setting this attribute is not supported.

If a Protocol with the @runtime_checkable decorator is marked as
deprecated, the __deprecated__ attribute should not be considered a
member of the protocol, so its presence should not affect isinstance
checks.

For compatibility with typing.get_overloads, the @deprecated decorator
should be placed after the @overload decorator.

Type checker behavior

This PEP does not specify exactly how type checkers should present
deprecation diagnostics to their users. However, some users (e.g.,
application developers targeting only a specific version of Python) may
not care about deprecations, while others (e.g., library developers who
want their library to remain compatible with future versions of Python)
would want to catch any use of deprecated functionality in their CI
pipeline. Therefore, it is recommended that type checkers provide
configuration options that cover both use cases. As with any other type
checker error, it is also possible to ignore deprecations using
# type: ignore comments.

Deprecation policy

We propose that CPython's deprecation policy (PEP 387) is updated to
require that new deprecations use the functionality in this PEP to alert
users about the deprecation, if possible. Concretely, this means that
new deprecations should be accompanied by a change to the typeshed repo
to add the @deprecated decorator in the appropriate place. This
requirement does not apply to deprecations that cannot be expressed
using this PEP's functionality.

Backwards compatibility

Creating a new decorator poses no backwards compatibility concerns. As
with all new typing functionality, the @deprecated decorator will be
added to the typing_extensions module, enabling its use in older
versions of Python.

How to teach this

For users who encounter deprecation warnings in their IDE or type
checker output, the messages they receive should be clear and
self-explanatory. Usage of the @deprecated decorator will be an advanced
feature mostly relevant to library authors. The decorator should be
mentioned in relevant documentation (e.g., PEP 387 and the
DeprecationWarning documentation) as an additional way to mark
deprecated functionality.

Reference implementation

A runtime implementation of the @deprecated decorator is available in
the typing-extensions library since version 4.5.0. The pyanalyze type
checker has prototype support for emitting deprecation errors, as does
Pyright.

Rejected ideas

Deprecation of modules and attributes

This PEP covers deprecations of classes, functions and overloads. This
allows type checkers to detect many but not all possible deprecations.
To evaluate whether additional functionality would be worthwhile, I
examined all current deprecations in the CPython standard library.

I found:

-   74 deprecations of functions, methods and classes (supported by this
    PEP)
-   28 deprecations of whole modules (largely due to PEP 594)
-   9 deprecations of function parameters (supported by this PEP through
    decorating overloads)
-   1 deprecation of a constant
-   38 deprecations that are not easily detectable in the type system
    (for example, for calling asyncio.get_event_loop without an active
    event loop)

Modules could be marked as deprecated by adding a __deprecated__
module-level constant. However, the need for this is limited, and it is
relatively easy to detect usage of deprecated modules simply by
grepping. Therefore, this PEP omits support for whole-module
deprecations. As a workaround, users could mark all module-level classes
and functions with @deprecated.

For deprecating module-level constants, object attributes, and function
parameters, a Deprecated[type, message] type modifier, similar to
Annotated could be added. However, this would create a new place in the
type system where strings are just strings, not forward references,
complicating the implementation of type checkers. In addition, my data
show that this feature is not commonly needed.

Features for deprecating more kinds of objects could be added in a
future PEP.

Placing the decorator in the typing module

An earlier version of this PEP proposed placing the @deprecated
decorator in the typing module. However, there was feedback that it
would be unexpected for a decorator in the typing module to have runtime
behavior. Therefore, the PEP now proposes adding the decorator the
warnings module instead.

Acknowledgments

A call with the typing-sig meetup group led to useful feedback on this
proposal.

Copyright

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.