When an iterator is used in Python, it results in overhead, which sometimes may prove to be not-so time-effective. This is because when an iterator is used, the __iter__ and the __next__ methods need to be implemented in a class. In addition to this, a tab needs to be kept on the internal states and a StopIteration error needs to be raised when no value is returned from the iterator.
This is where a generator comes into play and has an upper hand when it comes to performace and maintaining state.
A generator is one of the simplest ways to create iterators which can maintain state. Instead of us keeping a tab on the various processes, the generator automatically handles all the overhead for us. It is a method (function) that iterates over an iterable and returns a generator object. They can be put into use by calling the next method on a generator object or a for loop.
Creating a generator in Python
The process of creating a generator function is a simple process. If a function contains at least one yield statement, it is considered to be a generator function. In addition to containing one yield statement, it can contain other return or yield statements as well. The return value from yield and return are the same. The only difference between the yield and return statements is that the yield statement pauses the method and saves all the states, later continuing the execution from the last saved state whereas the return statement terminates the method in its entirety.
Time for an example:
The below example creates a generator method (since it has a yield statement) and runs a for loop to print the value by calling the generator function.
def GeneratorMethod(x):
yield x
for value in GeneratorMethod(12):
print(value)
Output:
12
Using the next method with generators
The below example shows how the generator function can be created and used in conjunction with __next__ method. It also shows how the for loop can be used with the generator method.
Time for an example:
def fibonacci_seq(value):
a, b = 0, 1
while a < value:
yield a
a, b = b, a + b
x = fibonacci_seq(4) # creating a generator object
print("Using _next__()")
print(x.__next__());
print(x.__next__());
print(x.__next__());
print(x.__next__());
print("\n")
print("Using for loop with generator")
for i in fibonacci_seq(4):
print(i)
Output:
Using _next__()
0
1
1
2
Using for loop with generator
0
1
1
2
3
Differences between a generator function and a normal function
-
The generator function returns using a yield statement.
-
The __iter__ and __next__ methods are implemented automatically in generator methods. It can be iterated over using the __next__ method.
-
It returns a generator object but doesn't start execution right afterward.
-
Once the yield statement is executed inside the generator method, the function is paused and the control gets transferred to the caller.
-
The state of the local variables is remembered between 2 successive calls.
-
The StopIteration is automatically called when the function gets terminated.
Application of generators in Python
Used in stream processing to handle large data files which could potentially be log files.
Conclusion
In this post, we understood the importance of generators, creating a generator and the differences between a generator method and a normal method. Don't forget to execute the code in IDE and meddle with the code.
