Building Custom Iterators with __iter__ and __next__

Hello! Welcome back to your second lesson on asyncio and async iterators.

Introduction

In our previous lesson, we established the fundamental difference between iterables and iterators. We saw that an iterable is a data source (like a list) with an __iter__ method, while an iterator is a stateful helper object with both an __iter__ method and a __next__ method that retrieves items and signals completion by raising StopIteration.

Today, we move from theory to practice. This lesson addresses the next learning outcome: implementing a custom iterator class using the __iter__ and __next__ methods.

By building your own iterator, you gain complete control over the iteration logic. This skill is not only powerful in itself but also serves as the direct foundation for understanding more advanced Python concepts like generators and, ultimately, the asynchronous iterators that are your end goal.

The Anatomy of a Custom Iterator Class

To create a class that functions as an iterator, it must adhere to the iterator protocol we discussed. This means implementing specific "dunder" methods:

1. __init__(self, ...): The standard class constructor. For an iterator, this is where you'll typically initialize its state, such as the data to be iterated over and a starting position (e.g., an index).

2. __iter__(self): As per the protocol, this method must return an iterator object. When you build a class that manages its own state and iteration logic, the simplest and most common pattern is for this method to just return self.

3. __next__(self): This is the core of the iterator. On each call, it should:

   • Calculate and return the next value in the sequence.
   • Update its internal state to point to the subsequent value.
   • When there are no more values to return, it must raise the StopIteration exception.

Guided Example: A Custom range Iterator

Let's build a custom iterator that mimics Python's built-in range() function. This is a classic example that clearly demonstrates state management and the iterator protocol.

The following video segment from Corey Schafer walks through creating exactly this type of class.

Here is the code from the video, with added comments to highlight the key mechanics we've discussed:

class MyRange:
    # 1. The constructor sets up the initial state.
    def __init__(self, start, end):
        self.value = start
        self.end = end

    # 2. This makes the class an iterable. It returns an iterator object,
    #    which in this case is the instance itself.
    def __iter__(self):
        return self

    # 3. This makes the class an iterator. It defines the logic for
    #    producing the next value.
    def __next__(self):
        # a. Check the termination condition first.
        if self.value >= self.end:
            raise StopIteration

        # b. If not finished, store the current value.
        current = self.value

        # c. Update the state for the *next* call.
        self.value += 1

        # d. Return the stored current value.
        return current

This simple class now fully implements the iterator protocol.

Using Your Custom Iterator

You can now use MyRange just like any other iterable, most commonly in a for loop. The for loop transparently handles calling iter() to get the iterator and catching the StopIteration exception to end the loop.

# The for loop works seamlessly.
nums = MyRange(1, 5)
for num in nums:
    print(num)

# Expected Output:
# 1
# 2
# 3
# 4

To reinforce what's happening under the hood, let's use it manually, just as we did in the last lesson:

# 1. Create an instance of our iterable class
nums_iterable = MyRange(1, 4)

# 2. Get the iterator from it (which is just the object itself)
nums_iterator = iter(nums_iterable)
print(f"Iterator object: {nums_iterator}")

# 3. Call next() repeatedly
print(next(nums_iterator)) # Output: 1
print(next(nums_iterator)) # Output: 2
print(next(nums_iterator)) # Output: 3

# 4. The next call will raise StopIteration
try:
    next(nums_iterator)
except StopIteration:
    print("StopIteration was raised, as expected.")

Another Perspective: Iterating Over Existing Data

The MyRange example generates new data. A more common use case is to create an iterator that wraps an existing data structure to provide a custom iteration order or logic.

The Real Python article you reviewed in the last lesson has an excellent example of this. Please read the following section, which implements an iterator that simply yields items from a sequence it is given.

The SequenceIterator follows the exact same protocol but for a different purpose. Its state is an index (_index) into the sequence it holds. This pattern is fundamental for tasks where you need to process elements of a collection in a specific, stateful way.

Practical Exercise: Build Your Own Iterator

Now it's your turn to apply these concepts. Your task is to create an iterator that traverses a list in reverse.

Challenge:

1. Create a class named ReversedList.
2. The __init__ method should accept a list as an argument.
3. Implement the iterator protocol (__iter__ and __next__) so that when used in a for loop, the class yields items from the list in reverse order.

For example:

my_list = [10, 20, 30, 40]
rev_list_iterator = ReversedList(my_list)

for item in rev_list_iterator:
    print(item)

# Expected Output:
# 40
# 30
# 20
# 10

Take some time to implement this yourself. When you're ready, you can check your work against the solution below.

class ReversedList:
    def __init__(self, data_list):
        self.data = data_list
        # Initialize the index to the last element.
        self.index = len(data_list) - 1

    def __iter__(self):
        return self

    def __next__(self):
        # The termination condition is when the index goes below 0.
        if self.index < 0:
            raise StopIteration
        
        # Get the item at the current index.
        result = self.data[self.index]
        
        # Decrement the index for the next iteration.
        self.index -= 1
        
        return result

# --- Testing the solution ---
my_list = [10, 20, 30, 40]
rev_list_iterator = ReversedList(my_list)

for item in rev_list_iterator:
    print(item)

# You can also verify that it's a one-time-use iterator.
# If you try to loop again, it will be empty because self.index is now -1.
print("\nTrying to iterate a second time:")
for item in rev_list_iterator:
    print(item) # This loop will produce no output.

Conclusion

In this lesson, you've moved from understanding the iterator protocol to actively implementing it. This is a significant step towards demystifying Python's iteration mechanics.

Key Takeaways:

• A custom iterator is a class that implements __init__, __iter__, and __next__.
• __init__ is for setting up the initial state of the iterator.
• __iter__ typically returns self, making the object both an iterable and its own iterator.
• __next__ contains the core logic: it returns the next item, updates the internal state, and raises StopIteration to signal the end.

Next Up:
We have now used the StopIteration exception to terminate our loops. In the next lesson, we will take a closer look at its specific role, exploring how Python uses this exception not as an error, but as a standard control flow mechanism within the iterator protocol. This will complete our foundational understanding of synchronous iteration.