Iterators
Iterators
A for loop in Python is not special syntax over indexes. It asks an object for an iterator, then calls next until the iterator is empty. The boring default is: use for on a list, tuple, or dict. Learn iter / next so you can write a stream when a full list would be waste, and so you stop being surprised when a stream is empty the second time you walk it.
Mental model
An iterable is anything iter(x) accepts: a list of tickets, a dict of tables, a file, a custom class with __iter__. That call returns an iterator.
An iterator is a stateful cursor. It has __next__. Each call yields one item. When there is nothing left it raises StopIteration. A for loop catches that exception and ends. You almost never catch StopIteration yourself.
iter(x) on an iterator usually returns the same object. iter(x) on a list returns a new cursor each time. That is why you can loop a list twice and a generator once.
Worked examples
Case 1: iter and next
Save as iter_next.py. Three tickets sit on the rail. iter hands you a cursor; next pulls one ticket at a time.
# iter_next.py
def main():
tickets = ["T-11", "T-12", "T-13"]
cursor = iter(tickets)
print(type(cursor).__name__)
print(next(cursor))
print(next(cursor))
print(next(cursor))
if __name__ == "__main__":
main()Run:
uv run python iter_next.pyOutput:
list_iterator
T-11
T-12
T-13
The list is still intact. The cursor moved. tickets is the iterable; cursor is the iterator.
Case 2: StopIteration and a default
Save as next_default.py. A fourth next has nothing to return. Bare next raises. next(cursor, default) does not.
# next_default.py
def main():
cursor = iter(["T-11"])
print(next(cursor))
print(next(cursor, "window closed"))
cursor = iter([])
try:
next(cursor)
except StopIteration:
print("empty window")
if __name__ == "__main__":
main()Run:
uv run python next_default.pyOutput:
T-11
window closed
empty window
Use the default form at a boundary (a queue that may be empty). Do not wrap every next in try. A for loop already does that job.
Case 3: What for actually does
Save as for_protocol.py. The loop on the left is the protocol on the right. Same tickets, same order.
# for_protocol.py
def walk(tickets):
cursor = iter(tickets)
while True:
try:
ticket = next(cursor)
except StopIteration:
break
print(f"called {ticket}")
def main():
tickets = ["T-11", "T-12"]
for ticket in tickets:
print(f"for {ticket}")
walk(tickets)
if __name__ == "__main__":
main()Run:
uv run python for_protocol.pyOutput:
for T-11
for T-12
called T-11
called T-12
Write the while version only when you are teaching the protocol, or when you need next with a sentinel. At the desk, write for.
Case 4: A custom iterator class
Save as ticket_window.py. The window holds a list and a position. __iter__ returns self, so the object is its own iterator. __next__ raises StopIteration when the rail is empty.
# ticket_window.py
class TicketWindow:
def __init__(self, tickets):
self._tickets = list(tickets)
self._i = 0
def __iter__(self):
return self
def __next__(self):
if self._i >= len(self._tickets):
raise StopIteration
ticket = self._tickets[self._i]
self._i += 1
return ticket
def main():
window = TicketWindow(["T-21", "T-22", "T-23"])
print(next(window))
for ticket in window:
print(f"served {ticket}")
if __name__ == "__main__":
main()Run:
uv run python ticket_window.pyOutput:
T-21
served T-22
served T-23
for ticket in window did not restart. next had already taken T-21. The class is a cursor, not a catalog.
The trap
Save as twice.py. A list can be walked twice because each for calls iter and gets a fresh cursor. A TicketWindow returns itself, so the second loop is already at the end.
# twice.py
class TicketWindow:
def __init__(self, tickets):
self._tickets = list(tickets)
self._i = 0
def __iter__(self):
return self
def __next__(self):
if self._i >= len(self._tickets):
raise StopIteration
ticket = self._tickets[self._i]
self._i += 1
return ticket
def main():
rail = ["T-31", "T-32"]
print("list", list(rail), list(rail))
window = TicketWindow(rail)
print("window", list(window), list(window))
if __name__ == "__main__":
main()Run:
uv run python twice.pyOutput:
list ['T-31', 'T-32'] ['T-31', 'T-32']
window ['T-31', 'T-32'] []
The empty second walk is not a bug in list. The iterator was spent. If you need two independent walks, either keep the source list, or make __iter__ return a new cursor (or a generator, next chapter).
The boring rule
- Prefer
for item in ticketsover indexes and over hand-writtennextloops. iter(x)gives a cursor.next(cursor)pulls one item.StopIterationmeans empty.next(cursor, default)is for a boundary that may be empty.- A custom class with
__iter__returningselfis a one-shot cursor. Document that, or do not write the class. - Do not catch
StopIterationin ordinary desk code. Letfordo it.
Try this
- In
iter_next.py, printlist(cursor)after the threenextcalls. Then printlist(iter(tickets)). - In
ticket_window.py, add apeekmethod that returns the next ticket without advancing, orNoneat the end. - Change
TicketWindow.__iter__so it returns a freshTicketWindow(self._tickets)instead ofself. Runtwice.pyagain. - Pass a dict of
{table: ticket}tofor. Print both the key and the value. Then iterate.values().