Python: Common Newbie Mistakes, Part 1

In the past few months I’ve been helping some people who are new to Python to get to know the language. I found that there are some pitfalls that almost everyone meet when they’re new to the language, so I decided to share my advice with you. Each part of this series will focus on a different common mistake, describe what causes it and offer a solution.

Using a Mutable Value as a Default Value

This one definitely deserves its place at the top of the list. Not only is the reason for this “bug” subtle, it’s also very hard to debug. Consider the following snippet1:

def foo(numbers=[]):
    numbers.append(9)
    print numbers

So we take a list (defaults to an empty list), add 9 to it and print it.

>>> foo()
[9]
>>> foo(numbers=[1,2])
[1, 2, 9]
>>> foo(numbers=[1,2,3])
[1, 2, 3, 9]

Seems good, right? Except this is what happens when we call foo without numbers:

>>> foo() # first time, like before
[9]
>>> foo() # second time
[9, 9]
>>> foo() # third time...
[9, 9, 9]
>>> foo() # WHAT IS THIS BLACK MAGIC?!
[9, 9, 9, 9]

So what’s happening here? Our instincts tell us that whenever we call foo without numbers, then numbers will be assigned with an empty list. This is wrong. Default values for functions in Python are instantiated when the function is defined, not when it’s called.

Still, you’d ask, why isn’t the same pre-calculated value reassigned every time I call the function? Well, every time you specify a default value for a function, Python stores that value. If you call the function and override that default, the stored value is not used. When you don’t override it (like in our example), Python makes the name you defined (numbers, in our case) reference the stored value. It does not copy the stored value to your variable. This may be a difficult concept for beginners, so imagine that instead of two distinct variables (one - the internal, initial default value and the other - the current, local variable), what we have in fact is two names that allow us to interact with the same value. So whenever numbers changes, it also changes Python’s memory of what the initial value is.

The solution here is this2:

def foo(numbers=None):
    if numbers is None:
        numbers = []
    numbers.append(9)
    print numbers

More often than not, when people hear about this, they ask how come other default values do work as expected. Consider this:

def foo(count=0):
    count += 1
    print count

When we run it, it works percisely as expected:

>>> foo()
1
>>> foo()
1
>>> foo(2)
3
>>> foo(3)
4
>>> foo()
1

Why is that? The “secret” here is not in the default value assignment, but in the value itself. An integer is an immutable type. While, like the empty list, it is executed in the definition of the function, it cannot be changed. When we do count += 1 we do not change the original value of count. We simply make the name count point to a different value. However, when we do numbers.append(9) we do change the original list. Hence, the confusion.

There is another variety of this same problem when you try to call a function as the default value:

def print_now(now=time.time()):
    print now

As before, while the value of time.time() is immutable, it is calculated only when the function is defined, so it’ll always return the same time - the time when the function code was parsed by Python:

>>> print_now()
1373121487.91
>>> print_now()
1373121487.91
>>> print_now()
1373121487.91

Read the rest of the series:

  1. This problem and its solution are relevant in both Python 2.x and 3.x, with the sole difference that with Python 3.x, the print statements should be function calls (e.g., print(numbers)). ↩︎

  2. You’ll notice that I used if numbers is None in the solution and not if not numbers. This is another common mistake which I already dedicated a post to, and you can find it here ↩︎

Discuss this post at the comment section below.
Follow me on Twitter , Facebook or Google+

Similar Posts