Python Generators – A complete guide to create and use generators

In our Python Iterators article, we create our own iterators.

Generators are also used to create functions that behave like iterators. It is a different approach to create iterators.

People often misunderstood this topic as a hard one because it looks different than the usual one but in fact, it is very easy. This article will teach you about Python generators, how you can create and effectively use generators in your code.

We will also see its applications so let’s get started.

What are Python Generator Functions?

Generator functions are special kind of functions that returns an iterator and we can loop it through just like a list, to access the objects one at a time.

Python generator functions are a simple way to create iterators. They solve the common problem of creating iterable objects. The traditional way was to create a class and then we have to implement __iter__() and __next__() methods.

We also have to manage the internal state and raise the StopIteration exception when the generator ends.

How to Create a Python Generator?

A Python generator is created like a normal function but we do not use the return statement in generators.

For generating a value, we use the yield keyword. When we use the yield keyword inside a function, it automatically becomes a generator function.

Let us understand the working of a generator with a simple generator.

def mygen():
    yield 10
    yield 20
    yield 40
f
gen = mygen()

print(next(gen))
print(next(gen))
print(next(gen))
print(next(gen))

Output:

Here, the mygen() function is a generator function in which we used 3 yield statements.

The next() function used on the generator object returns the value of the first yield statement. Again calling the next() will then return the value from the second function and so on. When it gets exhausted the generator function raises the StopIteration exception.

We can also use the for loop for easy traversing the elements.

def mygen():
    n = 1
    yield n
    n+=2
    yield n
    n+=3
    yield n
    yield "End"

for n in mygen():
    print(n)

Output:

One thing to notice here is that when the generator yields the values it remembers the value of the variable n between each call.

Python Generators with Loops

The above examples were simple, only for understanding the working of the generators. Now we will see generators with a loop that is more practically applicable for creating customized iterable objects. We can use for-loop to yield values.

Let us generate a sequence of Fibonacci numbers:

def fib_sequence(n):
   a,b = 0,1
   for i in range(n):
       yield a
       a,b = b,a+b

for i in fib_sequence(10):
    print(i)

Output:

Using Generators with List() Function

We can use the lits() function to yield all the values into a list. This is very handy sometimes. We can try this by using the same fib_sequence() generator function we made.

print( list(fib_sequence(20)) )

Output:

Python Generator Expressions

If you are familiar with list comprehensions then this would be very easy for you to understand. We have even a more shorthand technique to create python generators.

In list comprehensions we use [] brackets while in generator expressions we use () parenthesis. They are used at places where we quickly want to use a generator right away.

gen = (x**2 for x in range(10))

for x in gen:
  print(x)

Output:

Here, we created a generator that is used to generate a square of 0-9 numbers. If we print the gen object then we see that it is a generator object.

print(gen)

Output:

Why Generators are Used?

• Generators are used to create iterable objects.
• They are easier to implement than the creation of iterable objects which we saw in the iterables article.
• The generators yield one value at a time from a set of items.
• So they are memory efficient as they require less space and we don’t have to store everything at once.
• It is also easier to implement an infinite generation of series.
• We cannot store infinite series of a sequence in memory so with generators we can continuously generate values or infinite values(theoretically)
• Generators are also helpful in reading and performing operations on large files.
• We can use generators to effectively write code for our programs.

Summary

Python Generators are often misinterpreted as a hard topic but in fact, they are very useful and easy, once you start understanding how it works. We saw what Python generators are, how to create them and iterate over generators with for loops.

Moreover, we saw an even more easy way to create generators i.e. generators expression.

In the end, we saw the advantages of python generators.