PEP: 3114 Title: Renaming iterator.next() to iterator.__next__() Author:
Ka-Ping Yee <ping@zesty.ca> Status: Final Type: Standards Track
Content-Type: text/x-rst Created: 04-Mar-2007 Python-Version: 3.0
Post-History:

Abstract

The iterator protocol in Python 2.x consists of two methods: __iter__()
called on an iterable object to yield an iterator, and next() called on
an iterator object to yield the next item in the sequence. Using a for
loop to iterate over an iterable object implicitly calls both of these
methods. This PEP proposes that the next method be renamed to __next__,
consistent with all the other protocols in Python in which a method is
implicitly called as part of a language-level protocol, and that a
built-in function named next be introduced to invoke __next__ method,
consistent with the manner in which other protocols are explicitly
invoked.

Names With Double Underscores

In Python, double underscores before and after a name are used to
distinguish names that belong to the language itself. Attributes and
methods that are implicitly used or created by the interpreter employ
this naming convention; some examples are:

-   __file__ - an attribute automatically created by the interpreter
-   __dict__ - an attribute with special meaning to the interpreter
-   __init__ - a method implicitly called by the interpreter

Note that this convention applies to methods such as __init__ that are
explicitly defined by the programmer, as well as attributes such as
__file__ that can only be accessed by naming them explicitly, so it
includes names that are used or created by the interpreter.

(Not all things that are called "protocols" are made of methods with
double-underscore names. For example, the __contains__ method has double
underscores because the language construct x in y implicitly calls
__contains__. But even though the read method is part of the file
protocol, it does not have double underscores because there is no
language construct that implicitly invokes x.read().)

The use of double underscores creates a separate namespace for names
that are part of the Python language definition, so that programmers are
free to create variables, attributes, and methods that start with
letters, without fear of silently colliding with names that have a
language-defined purpose. (Colliding with reserved keywords is still a
concern, but at least this will immediately yield a syntax error.)

The naming of the next method on iterators is an exception to this
convention. Code that nowhere contains an explicit call to a next method
can nonetheless be silently affected by the presence of such a method.
Therefore, this PEP proposes that iterators should have a __next__
method instead of a next method (with no change in semantics).

Double-Underscore Methods and Built-In Functions

The Python language defines several protocols that are implemented or
customized by defining methods with double-underscore names. In each
case, the protocol is provided by an internal method implemented as a C
function in the interpreter. For objects defined in Python, this C
function supports customization by implicitly invoking a Python method
with a double-underscore name (it often does a little bit of additional
work beyond just calling the Python method.)

Sometimes the protocol is invoked by a syntactic construct:

-   x[y] --> internal tp_getitem --> x.__getitem__(y)
-   x + y --> internal nb_add --> x.__add__(y)
-   -x --> internal nb_negative --> x.__neg__()

Sometimes there is no syntactic construct, but it is still useful to be
able to explicitly invoke the protocol. For such cases Python offers a
built-in function of the same name but without the double underscores.

-   len(x) --> internal sq_length --> x.__len__()
-   hash(x) --> internal tp_hash --> x.__hash__()
-   iter(x) --> internal tp_iter --> x.__iter__()

Following this pattern, the natural way to handle next is to add a next
built-in function that behaves in exactly the same fashion.

-   next(x) --> internal tp_iternext --> x.__next__()

Further, it is proposed that the next built-in function accept a
sentinel value as an optional second argument, following the style of
the getattr and iter built-in functions. When called with two arguments,
next catches the StopIteration exception and returns the sentinel value
instead of propagating the exception. This creates a nice duality
between iter and next:

  iter(function, sentinel) <--> next(iterator, sentinel)

Previous Proposals

This proposal is not a new idea. The idea proposed here was supported by
the BDFL on python-dev[1] and is even mentioned in the original iterator
PEP, PEP 234:

    (In retrospect, it might have been better to go for __next__()
    and have a new built-in, next(it), which calls it.__next__().
    But alas, it's too late; this has been deployed in Python 2.2
    since December 2001.)

Objections

There have been a few objections to the addition of more built-ins. In
particular, Martin von Loewis writes[2]:

    I dislike the introduction of more builtins unless they have a true
    generality (i.e. are likely to be needed in many programs). For this
    one, I think the normal usage of __next__ will be with a for loop, so
    I don't think one would often need an explicit next() invocation.

    It is also not true that most protocols are explicitly invoked through
    builtin functions. Instead, most protocols are can be explicitly invoked
    through methods in the operator module. So following tradition, it
    should be operator.next.

    ...

    As an alternative, I propose that object grows a .next() method,
    which calls __next__ by default.

Transition Plan

Two additional transformations will be added to the 2to3 translation
tool[3]:

-   Method definitions named next will be renamed to __next__.
-   Explicit calls to the next method will be replaced with calls to the
    built-in next function. For example, x.next() will become next(x).

Collin Winter looked into the possibility of automatically deciding
whether to perform the second transformation depending on the presence
of a module-level binding to next[4] and found that it would be "ugly
and slow". Instead, the translation tool will emit warnings upon
detecting such a binding. Collin has proposed warnings for the following
conditions[5]:

-   Module-level assignments to next.
-   Module-level definitions of a function named next.
-   Module-level imports of the name next.
-   Assignments to __builtin__.next.

Approval

This PEP was accepted by Guido on March 6, 2007[6].

Implementation

A patch with the necessary changes (except the 2to3 tool) was written by
Georg Brandl and committed as revision 54910.

References

Copyright

This document has been placed in the public domain.

[1] Single- vs. Multi-pass iterability (Guido van Rossum)
https://mail.python.org/pipermail/python-dev/2002-July/026814.html

[2] PEP: rename it.next() to it.__next__()... (Martin von Loewis)
https://mail.python.org/pipermail/python-3000/2007-March/005965.html

[3] 2to3 refactoring tool
https://github.com/python/cpython/tree/ef04c44e29a8276a484f58d03a75a2dec516302d/Lib/lib2to3

[4] PEP: rename it.next() to it.__next__()... (Collin Winter)
https://mail.python.org/pipermail/python-3000/2007-March/006020.html

[5] PEP 3113 transition plan
https://mail.python.org/pipermail/python-3000/2007-March/006044.html

[6] PEP: rename it.next() to it.__next__()... (Guido van Rossum)
https://mail.python.org/pipermail/python-3000/2007-March/006027.html