Lies My Teacher Told Me (About Python) Three Places Where Basic Concepts Break Down

07 Aug 2016

(1) “When you call the function without a parameter, your default argument gets passed in.”

I think nearly every Python developer has gotten burned on this one. The key mistake is that default arguments are evaluated when the function is defined, not when the function is called. If you set a list or some other mutable data structure as a default argument, what actually gets passed in isn’t going to be your list as declared in the source code. It’s actually going to be the object that your code evaluated to earlier on, which may have been mutated several times since then. Beginners often write functions like this, which mutate their default argument:

def cool_list(list_of_items = []):
    list_of_items.append("cool")
    return list_of_items

The issue, of course, is that there is only going to be one list object created when the function is defined. So when you call cool_func repeatedly, it’s going to keep appending to that single list:

>>> cool_list()
['cool']
>>> cool_list()
['cool', 'cool']
>>> cool_list()
['cool', 'cool', 'cool']

The solution is to use a “dummy” default argument, which you can check in the body of the function. So we can rewrite the above as:

def cooler_list(list_of_items = None):
    if not list_of_items:
        list_of_items = []
    list_of_items.append("cooler")
    return list_of_items

Now repeated calls of cooler_list will operate on new list objects, rather than all mutating the same one.

(2) “The import statement reads a module’s source code into memory.”

The first time you execute an import statement, this is true of course. But have you ever tried to reload a module in the REPL, after making code changes? You won’t see your changes because Python actually keeps a cache of modules that have already been imported. To force a module to reload, you actually have to call importlib.reload() (for Python 3.4+) or imp.reload (for earlier versions). However, there are pitfalls with using these functions. For example, they re-use the same dictionary as the module that was originally imported so recently removed methods or attributes will still appear. The accepted wisdom is to avoid dynamically reloading modules altogether. For more info on the eccentricities of the Python import statement see David Beazley’s tutorial entitled “Modules and Packages: Live and Let Die!”

(3) “The del statement can remove dictionary keys.”

Here’s something that may be surprising: Python dictionaries have tombstones! When you “delete” a key from a Python dictionary, it doesn’t revert to a pre-insert state but instead marks the key as removed. This is necessary because Python uses a rehashing strategy. That is to say, when hash collisions occur Python rehashes the key to determine a backup slot to insert into. If that backup is also filled, this process gets repeated. So when you perform a lookup, Python often needs to follow a trail of collisions to actually determine if the key exists. If Python completely erased a key in the middle of the collision trail, it would hit an empty slot before finding the key at the end. Thus we need a tombstone, just to tell the dictionary to keep looking. For more details on how dictionaries work, watch Brandon Rhodes’ fantastic talk “The Mighty Dictionary.”