Generators in Python
Before starting with this tutorial you should learn what is yield Keyword and what are iterables & iterators as those concepts will be used in here.
A Generator is nothing but a function which returns value using the
yield keyword and not using the
If we have to write a custom iterator you will have to implement the
__next__() method, define the mechanism of returning the next value by maintaining the state and raise the
StopIteration exception when no more values are left for iterating.
Well, generator makes it super easy to create an iterator in which state is maintained, all you have to do is use the
yield keyword to return data.
Any function that contains the
yield keyword becomes a generator and will return an iterator object which can be iterated over using a
Creating a Python Generator
The magic recipe to convert a simple function into a generator function is the
yield keyword. We can have a single or multiple
yield statements to return some data from the generator where each time the generator is called the
yield statement stores the state of the local variables and yields a result.
Here is a simple example,
In above example, we have defined a python generator called
new_gen() in which we take a variable
n assign it some value, print something and then yield a value. Inside the function, we have repeated it 3 times.
If we try to call a generator function like a normal function it won't get executed because it returns an iterator object. Hence we have taken a variable
x to store the iterator object and then used the
next() method to execute the method. And every time the execution of the generator starts from where it last left i.e. the last
Yielding into Python List
We can directly use a generator to form a list by generating values as shown in the example below, where we have used a generator to generate a series of values and stored the values yielded by the generator into the list
# defining a generator
x = 0
while x < 5:
x += 1
# yielding values directly into a list
mylist = list(generator())
# use for loop to print mylist items
for x in mylist:
Generator vs. Normal Function vs. Python List
The major difference between a generator and a simple function is that a generator yields values instead of returning values. In simpler words, a simple function just returns a single variable and then it ends but a generator can yield a series of values by providing an iterator object on which we can iterate.
A generator must have a
yield statement, else it will just be a normal function.
As the generator yields it stores its local variable state and returns the control to the caller. And when it is again called, it starts from where it left in the last call. A normal function doesn't support this behaviour.
Now let's talk about the differences between a generator and a list. The biggest difference which is also the biggest advantage of a generator is that the generator never stores any values; it just yields values one by one therefore saving space as well as time, which is not in the case of a list.
Python Generators with a Loop
As a generator return an iterator object hence we can use loops like
for loop to iterate over it and print the yielded values.
Let's take a simple example:
Generator Expressions in Python
Just like list comprehension, we can use expressions to create python generator's shorthand. For example,
data = [0, 1, 2, 3, 4]
# creating a generator using generator expression
new_generator = (x*x for x in range(5))
for each in new_generator:
In the above example, the generator expression generates an iterator which is similar to the list
data with each of its values squared. Instead of creating a list of squares we have directly printed the result without the need to store a single value.