PEP: 3115 Title: Metaclasses in Python 3000 Author: Talin
<viridia@gmail.com> Status: Final Type: Standards Track Content-Type:
text/x-rst Created: 07-Mar-2007 Python-Version: 3.0 Post-History:
11-Mar-2007, 14-Mar-2007

Abstract

This PEP proposes changing the syntax for declaring metaclasses, and
alters the semantics for how classes with metaclasses are constructed.

Rationale

There are two rationales for this PEP, both of which are somewhat
subtle.

The primary reason for changing the way metaclasses work, is that there
are a number of interesting use cases that require the metaclass to get
involved earlier in the class construction process than is currently
possible. Currently, the metaclass mechanism is essentially a
post-processing step. With the advent of class decorators, much of these
post-processing chores can be taken over by the decorator mechanism.

In particular, there is an important body of use cases where it would be
useful to preserve the order in which a class members are declared.
Ordinary Python objects store their members in a dictionary, in which
ordering is unimportant, and members are accessed strictly by name.
However, Python is often used to interface with external systems in
which the members are organized according to an implicit ordering.
Examples include declaration of C structs; COM objects; Automatic
translation of Python classes into IDL or database schemas, such as used
in an ORM; and so on.

In such cases, it would be useful for a Python programmer to specify
such ordering directly using the declaration order of class members.
Currently, such orderings must be specified explicitly, using some other
mechanism (see the ctypes module for an example.)

Unfortunately, the current method for declaring a metaclass does not
allow for this, since the ordering information has already been lost by
the time the metaclass comes into play. By allowing the metaclass to get
involved in the class construction process earlier, the new system
allows the ordering or other early artifacts of construction to be
preserved and examined.

There proposed metaclass mechanism also supports a number of other
interesting use cases beyond preserving the ordering of declarations.
One use case is to insert symbols into the namespace of the class body
which are only valid during class construction. An example of this might
be "field constructors", small functions that are used in the creation
of class members. Another interesting possibility is supporting forward
references, i.e. references to Python symbols that are declared further
down in the class body.

The other, weaker, rationale is purely cosmetic: The current method for
specifying a metaclass is by assignment to the special variable
__metaclass__, which is considered by some to be aesthetically less than
ideal. Others disagree strongly with that opinion. This PEP will not
address this issue, other than to note it, since aesthetic debates
cannot be resolved via logical proofs.

Specification

In the new model, the syntax for specifying a metaclass is via a keyword
argument in the list of base classes:

    class Foo(base1, base2, metaclass=mymeta):
        ...

Additional keywords will also be allowed here, and will be passed to the
metaclass, as in the following example:

    class Foo(base1, base2, metaclass=mymeta, private=True):
        ...

Note that this PEP makes no attempt to define what these other keywords
might be - that is up to metaclass implementors to determine.

More generally, the parameter list passed to a class definition will now
support all of the features of a function call, meaning that you can now
use *args and **kwargs-style arguments in the class base list:

    class Foo(*bases, **kwds):
        ...

Invoking the Metaclass

In the current metaclass system, the metaclass object can be any
callable type. This does not change, however in order to fully exploit
all of the new features the metaclass will need to have an extra
attribute which is used during class pre-construction.

This attribute is named __prepare__, which is invoked as a function
before the evaluation of the class body. The __prepare__ function takes
two positional arguments, and an arbitrary number of keyword arguments.
The two positional arguments are:

  ------- --------------------------------------
  name    the name of the class being created.
  bases   the list of base classes.
  ------- --------------------------------------

The interpreter always tests for the existence of __prepare__ before
calling it; If it is not present, then a regular dictionary is used, as
illustrated in the following Python snippet.

    def prepare_class(name, *bases, metaclass=None, **kwargs):
        if metaclass is None:
            metaclass = compute_default_metaclass(bases)
        prepare = getattr(metaclass, '__prepare__', None)
        if prepare is not None:
            return prepare(name, bases, **kwargs)
        else:
            return dict()

The example above illustrates how the arguments to 'class' are
interpreted. The class name is the first argument, followed by an
arbitrary length list of base classes. After the base classes, there may
be one or more keyword arguments, one of which can be metaclass. Note
that the metaclass argument is not included in kwargs, since it is
filtered out by the normal parameter assignment algorithm. (Note also
that metaclass is a keyword-only argument as per PEP 3102.)

Even though __prepare__ is not required, the default metaclass ('type')
implements it, for the convenience of subclasses calling it via super().

__prepare__ returns a dictionary-like object which is used to store the
class member definitions during evaluation of the class body. In other
words, the class body is evaluated as a function block (just like it is
now), except that the local variables dictionary is replaced by the
dictionary returned from __prepare__. This dictionary object can be a
regular dictionary or a custom mapping type.

This dictionary-like object is not required to support the full
dictionary interface. A dictionary which supports a limited set of
dictionary operations will restrict what kinds of actions can occur
during evaluation of the class body. A minimal implementation might only
support adding and retrieving values from the dictionary - most class
bodies will do no more than that during evaluation. For some classes, it
may be desirable to support deletion as well. Many metaclasses will need
to make a copy of this dictionary afterwards, so iteration or other
means for reading out the dictionary contents may also be useful.

The __prepare__ method will most often be implemented as a class method
rather than an instance method because it is called before the metaclass
instance (i.e. the class itself) is created.

Once the class body has finished evaluating, the metaclass will be
called (as a callable) with the class dictionary, which is no different
from the current metaclass mechanism.

Typically, a metaclass will create a custom dictionary - either a
subclass of dict, or a wrapper around it - that will contain additional
properties that are set either before or during the evaluation of the
class body. Then in the second phase, the metaclass can use these
additional properties to further customize the class.

An example would be a metaclass that uses information about the ordering
of member declarations to create a C struct. The metaclass would provide
a custom dictionary that simply keeps a record of the order of
insertions. This does not need to be a full 'ordered dict'
implementation, but rather just a Python list of (key,value) pairs that
is appended to for each insertion.

Note that in such a case, the metaclass would be required to deal with
the possibility of duplicate keys, but in most cases that is trivial.
The metaclass can use the first declaration, the last, combine them in
some fashion, or simply throw an exception. It's up to the metaclass to
decide how it wants to handle that case.

Example

Here's a simple example of a metaclass which creates a list of the names
of all class members, in the order that they were declared:

    # The custom dictionary
    class member_table(dict):
        def __init__(self):
            self.member_names = []

        def __setitem__(self, key, value):
            # if the key is not already defined, add to the
            # list of keys.
            if key not in self:
                self.member_names.append(key)

            # Call superclass
            dict.__setitem__(self, key, value)

    # The metaclass
    class OrderedClass(type):

        # The prepare function
        @classmethod
        def __prepare__(metacls, name, bases): # No keywords in this case
            return member_table()

        # The metaclass invocation
        def __new__(cls, name, bases, classdict):
            # Note that we replace the classdict with a regular
            # dict before passing it to the superclass, so that we
            # don't continue to record member names after the class
            # has been created.
            result = type.__new__(cls, name, bases, dict(classdict))
            result.member_names = classdict.member_names
            return result

    class MyClass(metaclass=OrderedClass):
        # method1 goes in array element 0
        def method1(self):
            pass

        # method2 goes in array element 1
        def method2(self):
            pass

Sample Implementation

Guido van Rossum has created a patch which implements the new
functionality: https://bugs.python.org/issue1681101

Alternate Proposals

Josiah Carlson proposed using the name 'type' instead of 'metaclass', on
the theory that what is really being specified is the type of the type.
While this is technically correct, it is also confusing from the point
of view of a programmer creating a new class. From the application
programmer's point of view, the 'type' that they are interested in is
the class that they are writing; the type of that type is the metaclass.

There were some objections in the discussion to the 'two-phase' creation
process, where the metaclass is invoked twice, once to create the class
dictionary and once to 'finish' the class. Some people felt that these
two phases should be completely separate, in that there ought to be
separate syntax for specifying the custom dict as for specifying the
metaclass. However, in most cases, the two will be intimately tied
together, and the metaclass will most likely have an intimate knowledge
of the internal details of the class dict. Requiring the programmer to
insure that the correct dict type and the correct metaclass type are
used together creates an additional and unneeded burden on the
programmer.

Another good suggestion was to simply use an ordered dict for all
classes, and skip the whole 'custom dict' mechanism. This was based on
the observation that most use cases for a custom dict were for the
purposes of preserving order information. However, this idea has several
drawbacks, first because it means that an ordered dict implementation
would have to be added to the set of built-in types in Python, and
second because it would impose a slight speed (and complexity) penalty
on all class declarations. Later, several people came up with ideas for
use cases for custom dictionaries other than preserving field orderings,
so this idea was dropped.

Backwards Compatibility

It would be possible to leave the existing __metaclass__ syntax in
place. Alternatively, it would not be too difficult to modify the syntax
rules of the Py3K translation tool to convert from the old to the new
syntax.

References

[1] [Python-3000] Metaclasses in Py3K (original proposal)
https://mail.python.org/pipermail/python-3000/2006-December/005030.html

[2] [Python-3000] Metaclasses in Py3K (Guido's suggested syntax)
https://mail.python.org/pipermail/python-3000/2006-December/005033.html

[3] [Python-3000] Metaclasses in Py3K (Objections to two-phase init)
https://mail.python.org/pipermail/python-3000/2006-December/005108.html

[4] [Python-3000] Metaclasses in Py3K (Always use an ordered dict)
https://mail.python.org/pipermail/python-3000/2006-December/005118.html

Copyright

This document has been placed in the public domain.