PEP: 696 Title: Type Defaults for Type Parameters Author: James
Hilton-Balfe <gobot1234yt@gmail.com> Sponsor: Jelle Zijlstra
<jelle.zijlstra@gmail.com> Discussions-To:
https://discuss.python.org/t/pep-696-type-defaults-for-typevarlikes/22569
Status: Final Type: Standards Track Topic: Typing Created: 14-Jul-2022
Python-Version: 3.13 Post-History: 22-Mar-2022, 08-Jan-2023, Resolution:
https://discuss.python.org/t/pep-696-type-defaults-for-typevarlikes/22569/34

typing:type_parameter_defaults and type-params

Abstract

This PEP introduces the concept of type defaults for type parameters,
including TypeVar, ParamSpec, and TypeVarTuple, which act as defaults
for type parameters for which no type is specified.

Default type argument support is available in some popular languages
such as C++, TypeScript, and Rust. A survey of type parameter syntax in
some common languages has been conducted by the author of PEP 695 and
can be found in its
Appendix A <695#appendix-a-survey-of-type-parameter-syntax>.

Motivation

    T = TypeVar("T", default=int)  # This means that if no type is specified T = int

    @dataclass
    class Box(Generic[T]):
        value: T | None = None

    reveal_type(Box())                      # type is Box[int]
    reveal_type(Box(value="Hello World!"))  # type is Box[str]

One place this regularly comes up is Generator. I propose changing the
stub definition to something like:

    YieldT = TypeVar("YieldT")
    SendT = TypeVar("SendT", default=None)
    ReturnT = TypeVar("ReturnT", default=None)

    class Generator(Generic[YieldT, SendT, ReturnT]): ...

    Generator[int] == Generator[int, None] == Generator[int, None, None]

This is also useful for a Generic that is commonly over one type.

    class Bot: ...

    BotT = TypeVar("BotT", bound=Bot, default=Bot)

    class Context(Generic[BotT]):
        bot: BotT

    class MyBot(Bot): ...

    reveal_type(Context().bot)         # type is Bot  # notice this is not Any which is what it would be currently
    reveal_type(Context[MyBot]().bot)  # type is MyBot

Not only does this improve typing for those who explicitly use it, it
also helps non-typing users who rely on auto-complete to speed up their
development.

This design pattern is common in projects like:

-   discord.py — where the example above was taken from.
-   NumPy — the default for types like ndarray's dtype would be float64.
    Currently it's Unknown or Any.
-   TensorFlow — this could be used for Tensor similarly to
    numpy.ndarray and would be useful to simplify the definition of
    Layer.

Specification

Default Ordering and Subscription Rules

The order for defaults should follow the standard function parameter
rules, so a type parameter with no default cannot follow one with a
default value. Doing so should ideally raise a TypeError in
typing._GenericAlias/types.GenericAlias, and a type checker should flag
this as an error.

    DefaultStrT = TypeVar("DefaultStrT", default=str)
    DefaultIntT = TypeVar("DefaultIntT", default=int)
    DefaultBoolT = TypeVar("DefaultBoolT", default=bool)
    T = TypeVar("T")
    T2 = TypeVar("T2")

    class NonDefaultFollowsDefault(Generic[DefaultStrT, T]): ...  # Invalid: non-default TypeVars cannot follow ones with defaults


    class NoNonDefaults(Generic[DefaultStrT, DefaultIntT]): ...

    (
        NoNoneDefaults ==
        NoNoneDefaults[str] ==
        NoNoneDefaults[str, int]
    )  # All valid


    class OneDefault(Generic[T, DefaultBoolT]): ...

    OneDefault[float] == OneDefault[float, bool]  # Valid
    reveal_type(OneDefault)          # type is type[OneDefault[T, DefaultBoolT = bool]]
    reveal_type(OneDefault[float]()) # type is OneDefault[float, bool]


    class AllTheDefaults(Generic[T1, T2, DefaultStrT, DefaultIntT, DefaultBoolT]): ...

    reveal_type(AllTheDefaults)                  # type is type[AllTheDefaults[T1, T2, DefaultStrT = str, DefaultIntT = int, DefaultBoolT = bool]]
    reveal_type(AllTheDefaults[int, complex]())  # type is AllTheDefaults[int, complex, str, int, bool]
    AllTheDefaults[int]  # Invalid: expected 2 arguments to AllTheDefaults
    (
        AllTheDefaults[int, complex] ==
        AllTheDefaults[int, complex, str] ==
        AllTheDefaults[int, complex, str, int] ==
        AllTheDefaults[int, complex, str, int, bool]
    )  # All valid

With the new Python 3.12 syntax for generics (introduced by PEP 695),
this can be enforced at compile time:

    type Alias[DefaultT = int, T] = tuple[DefaultT, T]  # SyntaxError: non-default TypeVars cannot follow ones with defaults

    def generic_func[DefaultT = int, T](x: DefaultT, y: T) -> None: ...  # SyntaxError: non-default TypeVars cannot follow ones with defaults

    class GenericClass[DefaultT = int, T]: ...  # SyntaxError: non-default TypeVars cannot follow ones with defaults

ParamSpec Defaults

ParamSpec defaults are defined using the same syntax as TypeVar s but
use a list of types or an ellipsis literal "..." or another in-scope
ParamSpec (see Scoping Rules).

    DefaultP = ParamSpec("DefaultP", default=[str, int])

    class Foo(Generic[DefaultP]): ...

    reveal_type(Foo)                  # type is type[Foo[DefaultP = [str, int]]]
    reveal_type(Foo())                # type is Foo[[str, int]]
    reveal_type(Foo[[bool, bool]]())  # type is Foo[[bool, bool]]

TypeVarTuple Defaults

TypeVarTuple defaults are defined using the same syntax as TypeVar s but
use an unpacked tuple of types instead of a single type or another
in-scope TypeVarTuple (see Scoping Rules).

    DefaultTs = TypeVarTuple("DefaultTs", default=Unpack[tuple[str, int]])

    class Foo(Generic[*DefaultTs]): ...

    reveal_type(Foo)               # type is type[Foo[DefaultTs = *tuple[str, int]]]
    reveal_type(Foo())             # type is Foo[str, int]
    reveal_type(Foo[int, bool]())  # type is Foo[int, bool]

Using Another Type Parameter as default

This allows for a value to be used again when the type parameter to a
generic is missing but another type parameter is specified.

To use another type parameter as a default the default and the type
parameter must be the same type (a TypeVar's default must be a TypeVar,
etc.).

This could be used on builtins.slice where the start parameter should
default to int, stop default to the type of start and step default to
int | None.

    StartT = TypeVar("StartT", default=int)
    StopT = TypeVar("StopT", default=StartT)
    StepT = TypeVar("StepT", default=int | None)

    class slice(Generic[StartT, StopT, StepT]): ...

    reveal_type(slice)  # type is type[slice[StartT = int, StopT = StartT, StepT = int | None]]
    reveal_type(slice())                        # type is slice[int, int, int | None]
    reveal_type(slice[str]())                   # type is slice[str, str, int | None]
    reveal_type(slice[str, bool, timedelta]())  # type is slice[str, bool, timedelta]

    T2 = TypeVar("T2", default=DefaultStrT)

    class Foo(Generic[DefaultStrT, T2]):
        def __init__(self, a: DefaultStrT, b: T2) -> None: ...

    reveal_type(Foo(1, ""))  # type is Foo[int, str]
    Foo[int](1, "")          # Invalid: Foo[int, str] cannot be assigned to self: Foo[int, int] in Foo.__init__
    Foo[int]("", 1)          # Invalid: Foo[str, int] cannot be assigned to self: Foo[int, int] in Foo.__init__

When using a type parameter as the default to another type parameter,
the following rules apply, where T1 is the default for T2.

Scoping Rules

T1 must be used before T2 in the parameter list of the generic.

    T2 = TypeVar("T2", default=T1)

    class Foo(Generic[T1, T2]): ...   # Valid
    class Foo(Generic[T1]):
        class Bar(Generic[T2]): ...   # Valid

    StartT = TypeVar("StartT", default="StopT")  # Swapped defaults around from previous example
    StopT = TypeVar("StopT", default=int)
    class slice(Generic[StartT, StopT, StepT]): ...
                      # ^^^^^^ Invalid: ordering does not allow StopT to be bound

Using a type parameter from an outer scope as a default is not
supported.

Bound Rules

T1's bound must be a subtype of T2's bound.

    T1 = TypeVar("T1", bound=int)
    TypeVar("Ok", default=T1, bound=float)     # Valid
    TypeVar("AlsoOk", default=T1, bound=int)   # Valid
    TypeVar("Invalid", default=T1, bound=str)  # Invalid: int is not a subtype of str

Constraint Rules

The constraints of T2 must be a superset of the constraints of T1.

    T1 = TypeVar("T1", bound=int)
    TypeVar("Invalid", float, str, default=T1)         # Invalid: upper bound int is incompatible with constraints float or str

    T1 = TypeVar("T1", int, str)
    TypeVar("AlsoOk", int, str, bool, default=T1)      # Valid
    TypeVar("AlsoInvalid", bool, complex, default=T1)  # Invalid: {bool, complex} is not a superset of {int, str}

Type Parameters as Parameters to Generics

Type parameters are valid as parameters to generics inside of a default
when the first parameter is in scope as determined by the previous
section.

    T = TypeVar("T")
    ListDefaultT = TypeVar("ListDefaultT", default=list[T])

    class Bar(Generic[T, ListDefaultT]):
        def __init__(self, x: T, y: ListDefaultT): ...

    reveal_type(Bar)                    # type is type[Bar[T, ListDefaultT = list[T]]]
    reveal_type(Bar[int])               # type is type[Bar[int, list[int]]]
    reveal_type(Bar[int]())             # type is Bar[int, list[int]]
    reveal_type(Bar[int, list[str]]())  # type is Bar[int, list[str]]
    reveal_type(Bar[int, str]())        # type is Bar[int, str]

Specialisation Rules

Type parameters currently cannot be further subscripted. This might
change if Higher Kinded TypeVars are implemented.

Generic TypeAliases

Generic TypeAliases should be able to be further subscripted following
normal subscription rules. If a type parameter has a default that hasn't
been overridden it should be treated like it was substituted into the
TypeAlias. However, it can be specialised further down the line.

    class SomethingWithNoDefaults(Generic[T, T2]): ...

    MyAlias: TypeAlias = SomethingWithNoDefaults[int, DefaultStrT]  # Valid
    reveal_type(MyAlias)          # type is type[SomethingWithNoDefaults[int, DefaultStrT]]
    reveal_type(MyAlias[bool]())  # type is SomethingWithNoDefaults[int, bool]

    MyAlias[bool, int]  # Invalid: too many arguments passed to MyAlias

Subclassing

Subclasses of Generics with type parameters that have defaults behave
similarly to Generic TypeAliases. That is, subclasses can be further
subscripted following normal subscription rules, non-overridden defaults
should be substituted in, and type parameters with such defaults can be
further specialised down the line.

    class SubclassMe(Generic[T, DefaultStrT]):
        x: DefaultStrT

    class Bar(SubclassMe[int, DefaultStrT]): ...
    reveal_type(Bar)          # type is type[Bar[DefaultStrT = str]]
    reveal_type(Bar())        # type is Bar[str]
    reveal_type(Bar[bool]())  # type is Bar[bool]

    class Foo(SubclassMe[float]): ...

    reveal_type(Foo().x)  # type is str

    Foo[str]  # Invalid: Foo cannot be further subscripted

    class Baz(Generic[DefaultIntT, DefaultStrT]): ...

    class Spam(Baz): ...
    reveal_type(Spam())  # type is <subclass of Baz[int, str]>

Using bound and default

If both bound and default are passed default must be a subtype of bound.
Otherwise the type checker should generate an error.

    TypeVar("Ok", bound=float, default=int)     # Valid
    TypeVar("Invalid", bound=str, default=int)  # Invalid: the bound and default are incompatible

Constraints

For constrained TypeVars, the default needs to be one of the
constraints. A type checker should generate an error even if it is a
subtype of one of the constraints.

    TypeVar("Ok", float, str, default=float)     # Valid
    TypeVar("Invalid", float, str, default=int)  # Invalid: expected one of float or str got int

Function Defaults

In generic functions, type checkers may use a type parameter's default
when the type parameter cannot be solved to anything. We leave the
semantics of this usage unspecified, as ensuring the default is returned
in every code path where the type parameter can go unsolved may be too
hard to implement. Type checkers are free to either disallow this case
or experiment with implementing support.

    T = TypeVar('T', default=int)
    def func(x: int | set[T]) -> T: ...
    reveal_type(func(0))  # a type checker may reveal T's default of int here

Defaults following TypeVarTuple

A TypeVar that immediately follows a TypeVarTuple is not allowed to have
a default, because it would be ambiguous whether a type argument should
be bound to the TypeVarTuple or the defaulted TypeVar.

    Ts = TypeVarTuple("Ts")
    T = TypeVar("T", default=bool)

    class Foo(Generic[Ts, T]): ...  # Type checker error

    # Could be reasonably interpreted as either Ts = (int, str, float), T = bool
    # or Ts = (int, str), T = float
    Foo[int, str, float]

With the Python 3.12 built-in generic syntax, this case should raise a
SyntaxError.

However, it is allowed to have a ParamSpec with a default following a
TypeVarTuple with a default, as there can be no ambiguity between a type
argument for the ParamSpec and one for the TypeVarTuple.

    Ts = TypeVarTuple("Ts")
    P = ParamSpec("P", default=[float, bool])

    class Foo(Generic[Ts, P]): ...  # Valid

    Foo[int, str]  # Ts = (int, str), P = [float, bool]
    Foo[int, str, [bytes]]  # Ts = (int, str), P = [bytes]

Subtyping

Type parameter defaults do not affect the subtyping rules for generic
classes. In particular, defaults can be ignored when considering whether
a class is compatible with a generic protocol.

TypeVarTuples as Defaults

Using a TypeVarTuple as a default is not supported because:

-   Scoping Rules does not allow usage of type parameters from outer
    scopes.
-   Multiple TypeVarTuples cannot appear in the type parameter list for
    a single object, as specified in
    646#multiple-type-variable-tuples-not-allowed.

These reasons leave no current valid location where a TypeVarTuple could
be used as the default of another TypeVarTuple.

Binding rules

Type parameter defaults should be bound by attribute access (including
call and subscript).

    class Foo[T = int]:
        def meth(self) -> Self:
            return self

    reveal_type(Foo.meth)  # type is (self: Foo[int]) -> Foo[int]

Implementation

At runtime, this would involve the following changes to the typing
module.

-   The classes TypeVar, ParamSpec, and TypeVarTuple should expose the
    type passed to default. This would be available as a __default__
    attribute, which would be None if no argument is passed and NoneType
    if default=None.

The following changes would be required to both GenericAliases:

-   logic to determine the defaults required for a subscription.
-   ideally, logic to determine if subscription (like
    Generic[T, DefaultT]) would be valid.

The grammar for type parameter lists would need to be updated to allow
defaults; see below.

A reference implementation of the runtime changes can be found at
https://github.com/Gobot1234/cpython/tree/pep-696

A reference implementation of the type checker can be found at
https://github.com/Gobot1234/mypy/tree/TypeVar-defaults

Pyright currently supports this functionality.

Grammar changes

The syntax added in PEP 695 will be extended to introduce a way to
specify defaults for type parameters using the "=" operator inside of
the square brackets like so:

    # TypeVars
    class Foo[T = str]: ...

    # ParamSpecs
    class Baz[**P = [int, str]]: ...

    # TypeVarTuples
    class Qux[*Ts = *tuple[int, bool]]: ...

    # TypeAliases
    type Foo[T, U = str] = Bar[T, U]
    type Baz[**P = [int, str]] = Spam[**P]
    type Qux[*Ts = *tuple[str]] = Ham[*Ts]
    type Rab[U, T = str] = Bar[T, U]

Similarly to the bound for a type parameter <695-scoping-behavior>,
defaults should be lazily evaluated, with the same scoping rules to
avoid the unnecessary usage of quotes around them.

This functionality was included in the initial draft of PEP 695 but was
removed due to scope creep.

The following changes would be made to the grammar:

    type_param:
        | a=NAME b=[type_param_bound] d=[type_param_default]
        | a=NAME c=[type_param_constraint] d=[type_param_default]
        | '*' a=NAME d=[type_param_default]
        | '**' a=NAME d=[type_param_default]

    type_param_default:
        | '=' e=expression
        | '=' e=starred_expression

The compiler would enforce that type parameters without defaults cannot
follow type parameters with defaults and that TypeVars with defaults
cannot immediately follow TypeVarTuples.

Rejected Alternatives

Allowing the Type Parameters Defaults to Be Passed to type.__new__'s **kwargs

    T = TypeVar("T")

    @dataclass
    class Box(Generic[T], T=int):
        value: T | None = None

While this is much easier to read and follows a similar rationale to the
TypeVar unary syntax, it would not be backwards compatible as T might
already be passed to a metaclass/superclass or support classes that
don't subclass Generic at runtime.

Ideally, if PEP 637 wasn't rejected, the following would be acceptable:

    T = TypeVar("T")

    @dataclass
    class Box(Generic[T = int]):
        value: T | None = None

Allowing Non-defaults to Follow Defaults

    YieldT = TypeVar("YieldT", default=Any)
    SendT = TypeVar("SendT", default=Any)
    ReturnT = TypeVar("ReturnT")

    class Coroutine(Generic[YieldT, SendT, ReturnT]): ...

    Coroutine[int] == Coroutine[Any, Any, int]

Allowing non-defaults to follow defaults would alleviate the issues with
returning types like Coroutine from functions where the most used type
argument is the last (the return). Allowing non-defaults to follow
defaults is too confusing and potentially ambiguous, even if only the
above two forms were valid. Changing the argument order now would also
break a lot of codebases. This is also solvable in most cases using a
TypeAlias.

    Coro: TypeAlias = Coroutine[Any, Any, T]
    Coro[int] == Coroutine[Any, Any, int]

Having default Implicitly Be bound

In an earlier version of this PEP, the default was implicitly set to
bound if no value was passed for default. This while convenient, could
have a type parameter with no default follow a type parameter with a
default. Consider:

    T = TypeVar("T", bound=int)  # default is implicitly int
    U = TypeVar("U")

    class Foo(Generic[T, U]):
        ...

    # would expand to

    T = TypeVar("T", bound=int, default=int)
    U = TypeVar("U")

    class Foo(Generic[T, U]):
        ...

This would have also been a breaking change for a small number of cases
where the code relied on Any being the implicit default.

Allowing Type Parameters With Defaults To Be Used in Function Signatures

A previous version of this PEP allowed TypeVarLikes with defaults to be
used in function signatures. This was removed for the reasons described
in Function Defaults. Hopefully, this can be added in the future if a
way to get the runtime value of a type parameter is added.

Allowing Type Parameters from Outer Scopes in default

This was deemed too niche a feature to be worth the added complexity. If
any cases arise where this is needed, it can be added in a future PEP.

Acknowledgements

Thanks to the following people for their feedback on the PEP:

Eric Traut, Jelle Zijlstra, Joshua Butt, Danny Yamamoto, Kaylynn Morgan
and Jakub Kuczys

Copyright

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.