Generator functions and yield

Some series of values are long, and some never end: all the odd numbers, the next ticket number at a counter, the lines of a huge file. Building the whole series in a list before using it wastes time and memory, and for an endless series it's impossible. A generator solves this by producing the values one at a time, only when they're asked for.

yield

A function that contains the keyword yield is a generator function. yield hands out one value, much like return, but the function doesn't end there. It pauses, and when the next value is asked for, it continues from the same spot with the same variables. A for loop asks for values until the function ends:

def countdown(start):
    number = start
    while number > 0:
        yield number
        number -= 1


for number in countdown(3):
    print(number)
print("Liftoff!")

3
2
1
Liftoff!

countdown(3) doesn't return 3, 2 and 1 at once. It returns a generator: an object that produces the values on request. The loop asks it for a value three times. The fourth time, the while condition is false, the function ends, and so does the loop.

Pausing and continuing

A few prints inside a generator function show exactly when its code runs:

def steps():
    print("start")
    yield "one"
    print("after one")
    yield "two"
    print("done")


for value in steps():
    print("got", value)

start
got one
after one
got two
done

Calling steps() runs none of its code yet. When the loop asks for the first value, the function runs up to the first yield and pauses there. The loop prints got one and asks again, and the function continues right after that yield. When the function's code runs out, the generator is finished and the loop ends.

That's the difference from return, which ends the call for good and forgets the function's variables. yield keeps them for the next value.

Endless generators, and lists

A generator doesn't have to end. This one gives odd numbers forever, so the loop needs a break, or it would never stop:

def odd_numbers():
    number = 1
    while True:
        yield number
        number += 2


for number in odd_numbers():
    if number > 7:
        break
    print(number)

1
3
5
7

A generator that does end can be turned into a list with list(), which asks for every value. Printing the generator itself shows no values, only something like <generator object countdown at 0x7f3a2c>, since the values don't exist yet:

>>> def countdown(start):
...     number = start
...     while number > 0:
...         yield number
...         number -= 1
...
>>> list(countdown(5))
[5, 4, 3, 2, 1]