Special Methods, OOP, and Iteration Python Tutorial

Welcome to part 21 of the intermediate Python programming tutorial series. In this tutorial, we're going to take a unique opportunity to mesh OOP, iterables, iterators, special methods, AND generators. It's an exciting day.

There are all sorts of special methods that we can add to our classes, but one common one might be to either add iteration functionality, or even to modify how something is iterated over. Let's revisit iterables.

When we do something like:

x = range(5)

What is x? Is it an iterator? Is it iterable? Is it a Generator? ...is it a range object?

Is there a difference between x = range(5) and x = (i for i in range(5)) in what x is? Your homework is to answer that question.

Let's say:

x = (i for i in range(5))

Now, how do we iterate over this?

for i in x:
    print(i)

Output:

0
1
2
3
4

What happens here, I mean *really*? As you iterate over an iterable, which is a Python object, you're actually calling the next method to run. If this is news to you, check this out:

x = (i for i in range(5))
next(x)
next(x)
for i in x:
    print(i)

Output:

2
3
4

With next, we move the iterator (think of it like a "selector") explicitly, before iterating over whatever was left. Now comes the OOP part. Python is such a high level language, and there's so much going on, that even the most basic of scripts is actually pretty complex Object Oriented Programming, even when the author has no idea. In our case, it wouldn't appear to us that we've got any OOP going on here, but we do. Everything's an object. That's why Python's Pickle is so darned useful. Rather than using next(), we can also actually just use the raw method, a special method: __next__.

x = (i for i in range(5))
x.__next__()
x.__next__()
for i in x:
    print(i)

Output:

2
3
4

Works the same way. Jeez x, what other methods are you hiding from us?! Given that __next__ exists, we can rest assured __iter__ also exists. That's it x, you're getting searched. We'll use Python's built-in dir function to return the attributes of this x object:

>>> print(dir(x))
['__class__', '__del__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__lt__', '__name__', '__ne__', '__new__', '__next__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'close', 'gi_code', 'gi_frame', 'gi_running', 'gi_yieldfrom', 'send', 'throw']

!!!!!!!!

Look at all of those dunders that x has been hiding from us. Never trust an x. Checking range is a similar story:

>>> print(dir(range))
['__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'count', 'index', 'start', 'step', 'stop']

Sheesh. We can trust no one. We need to handle this iteration stuff on our own, clearly.

Let's make a new class, called range_examp, and build the __init__ method:

class range_examp:
    
    def __init__(self, end, step=1):
        self.current = 0
        self.end = end
        self.step = step

Any good range function is going to need to know where it is and where to head next. Other than that, it doesn't need much else, but we'll also give it a stopping point. Now, let's do the entirely challenging task of writing the __iter__ method...

    def __iter__(self):
        return self

Whew, that was hard. Now we need to write the dunder next method.

    def __next__(self):
        if self.current >= self.end:
            raise StopIteration()
        else:
            return_val = self.current
            self.current += self.step
            return return_val

Simple enough above. If the current value is at or beyond the end value, then we're done, and we raise a StopIteration(). Otherwise, we set the return value to be the current value, we add the step, and return the current value. Now, we can do:

for i in range_examp(5):
    print(i)

0
1
2
3
4

Fancy! We can also illustrate the use of __next__:

i = range_examp(5)

i.__next__()
i.__next__()

for j in i:
    print(j)

Output:

2
3
4

Of course, we still have a lot of hidden attributes to even this new class that we just made...

>>> print(dir(range_examp))
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__next__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__']

...not to mention that StopIteration, given its TitleCasing is obviously also a class, with a bunch of methods and attributes of its own. Alright, that's it, let's just use a generator for this range stuff:

def range_gen(end):
    current = 0
    while current < end:
        yield current
        current += 1

Perfect, good ol' functions. No classes here. We can do something like:

for i in range_gen(5):
    print(i)

Output:

0
1
2
3
4

But then again...

>>> print(dir(range_gen(5)))
['__class__', '__del__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__lt__', '__name__', '__ne__', '__new__', '__next__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'close', 'gi_code', 'gi_frame', 'gi_running', 'gi_yieldfrom', 'send', 'throw']

There's no getting away from Object Oriented Programming in Python 3!

The next tutorial: Logging Python Tutorial