Generators in Python - Part 2

In the previous part of python generators, we understood what a generator is, how it can be used with for loops and how the generator method is different from a normal method. In this post, we will understand generator expression.

Why are generator expressions needed?

They help the process of the creation of generators and make it easy. A generator expression creates an anonymous generator function. Creating a generator expression is similar to creating a list comprehension in Python. The only difference between list comprehension and a generator expression is that a generator expression produces one data element at a time whereas a list comprehension produces the entire list.

Since the elements are produced only when they are needed, the amount of memory consumed when a generator expression is created is much less, in comparison to other options. This means generator expressions are memory-friendly.

Time for an example:

The below example shows the difference in output while using list comprehension and a generator expression.

my_list = [1, 12, 7, 89, 45]
print("The below is a list comprehension")
print([element**3 for element in my_list])  # list comprehension
print("\n")
print("The below is a generator expression")
(element**2 for element in my_list)  # generator expression

Output:

The below is a list comprehension
[1, 1728, 343, 704969, 91125]

The below is a generator expression
Out[6]: <generator object <genexpr> at 0x00000263B7991390>

The above code clearly shows that a list comprehension generated a list based on a specific operation, whereas a generator expression resulted in the creation of a generator object. This generator object returns data elements only when needed. This can be achieved with the help of the __next__ method.

Time for an example:

my_list = [1, 12, 7, 89, 45]
generator = (element**2 for element in my_list)  # generator expression
print(next(generator))
print(next(generator))
print(next(generator))
print(next(generator))
print(next(generator))
next(generator)

Output:

1
144
49
7921
2025
Traceback (most recent call last):

  File "<ipython-input-8-a66965a2b6ca>", line 8, in <module>
    next(generator)

StopIteration

Using generator expression with a built-in function

The generator expression can be used with built-in methods like sum, min, and max. Below is an example demonstrating the same:

my_list = [1, 12, 7, 89, 45]
generator_one = sum(element**2 for element in my_list)  # generator expression
generator_two = min(element**2 for element in my_list)  # generator expression
generator_three = max(element**2 for element in my_list)  # generator expression
print(generator_one)
print(generator_two)
print(generator_three)

Output:

10140
1
7921

Why use generators at all?

1. They are simple and easy to implement.

2. They keep all the bookkeeping automatically (creation of __iter__ and __next__ methods).

3. They are memory-friendly since the data element is generated on demand (one at a time), otherwise, only the generator object is the result.

4. They go hand-in-hand when dealing with large amounts of streaming data or bookkeeping data.

5. They are easier to understand and write as well!

Conclusion

In this post, we saw how generators are needed, how they can be used with built-in functions and their significance.