More on Generators with Python

Welcome to part 9 of the intermediate Python programming tutorial series. We're going to be revisiting the topic of generators again. In the previous generators tutorial, we simply wrote generator expressions, which is a "lazy" way to work with generators.

In many cases, you really don't need to actually write a generator over just writing a generator expression. It's also more Pythonic to go ahead and just write the generator expression since it's simpler. That said, you should still know how to write a generator, since it will give you a bit more insight into how generators work, and what iterable objects really are, since, despite looking a lot like list comprehension, generators are very unique in how they act. Also, you will want to be able to know a generator when you see it.

Generators don't return things, they yield things, in a stream. This can be done on iterables, which has been already illustrated, but also can be based purely in logic, whatever you want. For example:

def simple_gen():
    yield 'Oh'
    yield 'hello'
    yield 'there'

There's our generator, super basic. Now we can actually iterate over it:

for i in simple_gen():
    print(i)

Oh
hello
there

You can add logic to the yield statements, whatever you want. Let's consider a scenario where we want to find the combination to a lock, one of the ones where you spin the dials, and each dial has 0-9 on it, and we need to get the perfect combination to unlock the lock. How might a beginner approach this problem? I know how I would have started:

CORRECT_COMBO = (3, 6, 1)

for c1 in range(10):
    for c2 in range(10):
        for c3 in range(10):
            if (c1, c2, c3) == CORRECT_COMBO:
                print('Found the combo:{}'.format((c1, c2, c3)))

Sure enough, we do get the right answer:

Found the combo:(3, 6, 1)

So what's the problem? Well, even after we've found the combo, we continue to iterate through *all* of the combinations. Luckily for us, the calculation is super cheap, but this code doesn't scale at all. Even as beginners though, we know about the break statement. Can't we just call it like:

for c1 in range(10):
    for c2 in range(10):
        for c3 in range(10):
            if (c1, c2, c3) == CORRECT_COMBO:
                print('Found the combo:{}'.format((c1, c2, c3)))
                break

You can, but you're only saving yourself the processing of less than 10 out of a thousand operations. Hmm. Okay, we might be beginners, but we're still crafty. We'll have breaking logic for all of the lines. To do this, we're going to introduce a new var: found_combo, which we'll set to False to start, but then update to True if we find the combo. Then, we'll have an if statement to see if we found the combo at each step, and then break if that's the case!

found_combo = False
for c1 in range(10):
    if found_combo:
        break
    for c2 in range(10):
        if found_combo:
            break
        for c3 in range(10):
            if (c1, c2, c3) == CORRECT_COMBO:
                print('Found the combo:{}'.format((c1, c2, c3)))
                found_combo = True
                break

Great, we've done it! Is this Pythonic though? It meets PEP 8 standards...but this is one of the reasons why you need to take PEP 8 with a grain of salt, or, better put, think about PEP 8 only as you're writing code, but, after that, we need to assess whether or not it's *pythonic.* There's a better way. Since this is a generator tutorial, I suppose you might already know the answer is going to be a generator.

For the generator, we really need to just iterate through the combinations, the logic is nothing new or surprising:

def combo_gen():
    for c1 in range(10):
        for c2 in range(10):
            for c3 in range(10):
                yield (c1, c2, c3)

Then, to actually make use of it:

for (c1, c2, c3) in combo_gen():
    print(c1, c2, c3)
    if (c1, c2, c3) == CORRECT_COMBO:
        print('Found the combo:{}'.format((c1, c2, c3)))
        break

Once we break, the generator stream stops, and we're done.

This code is only 2 lines of actual code shorter (1 if we count white space), but it's easier to read and understand, and is certainly more beautiful.

Next up, let's talk about how to really amplify our Pythonic superpowers with multiprocessing.

The next tutorial: Multiprocessing with Python intro