Python Generator and Iterator

Iterator

Iterator is an object which allows to traverse through all the elements of a collection.

In Python, an iterator is an object which implements the iterator protocol.

The iterator protocol consists of two methods.

1. The iter() method, which must return the iterator object,
2. the next() method, which returns the next element from a sequence.

Iterators have several advantages:

• can work with infinite sequences
• save resources

Python has several built-in objects examples lists, tuples, strings, dictionaries or files, which implement the iterator protocol.

## Define Custom Iterator using Iterator Protocol.

class krange(object):
    
    def __init__(self, n):
        self.i = 0
        self.n = n
        
    def __iter__(self):
        return self
    
    def next(self):
        if self.i < self.n:
            i = self.i
            self.i += 1
            return i
        else:
            raise StopIteration()

k = krange(5)

print(k.next())
print(k.next())
print(k.next())
print(k.next())

0
1
2
3

Example: Write an iterator class reverse_iter, that takes a list and iterates it from the reverse direction.

class reverse_iter(object):
    
    def __init__(self, alist):
        self.alist = alist
        self.n = len(alist) -1 
        
    def __iter__(self):
        return self
    
    def next(self):
        if self.n > 0:
            item = self.alist[self.n]
            self.n -= 1
            return item
        else:
            raise StopIteration()
        

abc = reverse_iter([1,2,3,4,5])

print(abc.next())
print(abc.next())
print(abc.next())
print(abc.next())

5
4
3
2

Python Generators

Generator is a special routine that can be used to control the iteration behavior of a loop. A generator is similar to a function returning an array.
Generators are defined similar to a function and have the same properties like has parameters, can be called. But unlike functions, which return a whole array, a generator yields one value at a time. This requires less memory.

Generators in Python:

• Use the yield keyword and it May use several yield keywords
• Return an iterator

Generators simplifies creation of iterators. A generator is a function that produces a sequence of results instead of a single value.

So a generator is also an iterator. Here You don’t have to worry about the iterator protocol.

def gen(alist):
   for k in alist:
    yield k

g = gen([1,2,3,4,5])

print(next(g))
print(next(g))
print(g.__next__()) # g.next() in python2
print(g.__next__())

1
2
3
4

Can you think about how it is working internally?

When a generator function is called, it returns a generator object without even beginning execution of the function. When next method is called for the first time, the function starts executing until it reaches yield statement. The yielded value is returned by the next call.

Let’s check an example:

def foo():
    print("start Foo")
    for i in range(5):
        print(f"Before Yield i -- {i}")
        yield i
        print(f"After Yield i -- {i}")
    print("End Foo")

f = foo()
print(f.__next__())

start Foo
Before Yield i -- 0
0

print(f.__next__())

After Yield i -- 0
Before Yield i -- 1
1

print(f.__next__())

After Yield i -- 1
Before Yield i -- 2
2

print(f.__next__())

After Yield i -- 2
Before Yield i -- 3
3

print(f.__next__())

After Yield i -- 3
Before Yield i -- 4
4

print(f.__next__())

After Yield i -- 4
End Foo



---------------------------------------------------------------------------

StopIteration                             Traceback (most recent call last)

<ipython-input-72-2f4b6c2bfb23> in <module>
----> 1 print(f.__next__())


StopIteration: 

Python generator expression

A generator expression is created with round brackets.

aa = (a for a in range(1000)) # this return a generator
ab = [b for b in range(100)] # this return a list

print(next(aa))
print(next(aa))
print(next(aa))
print(next(aa))

print(next(aa))
print(next(aa))
print(next(aa))
print(next(aa))

0
1
2
3
4
5
6
7

Further Reading: http://www.dabeaz.com/generators-uk/