Understanding Python Class Instantiation
Let’s say you have a class
What happens when you instantiate it (create an instance of that class)?
That call to
Foo - what function or method is being called there? Most beginners and even many experienced Python programmers will immediately answer that
__init__ is called. If you stop to think about it for a second, this is far from being a correct answer.
__init__ doesn’t return an object, but calling
Foo(1, y=2) does return an object. Also,
__init__ expects a
self parameter, but there is no such parameter when calling
Foo(1, y=2). There is something more complex at work here. In this post we’ll investigate together what happens when you instantiate a class in Python.
Instantiating an object in Python consists of a few stages, but the beauty of it is that they are Pythonic in themselves - understanding the steps gives us a little bit more understanding of Python in general.
Foo is a class, but classes in Python are objects too! Classes, functions, methods and instances are all objects and whenever you put parentheses after their name, you invoke their
__call__ method. So
Foo(1, y=2) is equivalent to
Foo.__call__(1, y=2). That
__call__ is the one defined by
Foo’s class. What is
Foo is an object of type
type and calling
__call__ returns an object of class
Foo. Next, let’s look at what the
__call__ method for
type looks like. This method is fairly complicated, but we’ll try to simplify it. Below I have pasted both the CPython
C and the PyPy Python implementation. I find that looking at the original source code is very interesting, but feel free to skip to my simplification of it below:
Link to source.
Link to source.
If we ignore error checking for a minute, then for regular class instantiation this is roughly equivalent to:
__new__ allocates memory for the object, constructs it as an “empty” object and then
__init__ is called to initialize it.
Foo(*args, **kwargs)is equivalent to
Foois an instance of
type.__call__(Foo, *args, **kwargs).
type.__call__(Foo, *args, **kwargs)calls
type.__new__(Foo, *args, **kwargs)which returns
objis then initialized by calling
Now we turn our attention to the
__new__ method. Essentially, it is the method responsible for actual object creation. We won’t go in detail into the base implementation of
__new__. The gist of it is that it allocates space for the object and returns it. The interesting thing about
__new__ is that once you realize what it does, you can use it to customize instance creation in interesting ways. It should be noted that while
__new__ is a static method, you don’t need to declare it with
@staticmethod - it is special-cased by the Python interpreter.
A nice example of the power of
__new__ is using it to implement a Singleton class:
Notice that in this Singleton implementation,
__init__ will be called each time we call
Singleton(), so care should be taken.
Another similar example is implementing the Borg design pattern:
One final note - the examples above show the power of
__new__, but just because you can use it, doesn’t mean you should:
__new__is one of the most easily abused features in Python. It’s obscure, riddled with pitfalls, and almost every use case I’ve found for it has been better served by another of Python’s many tools. However, when you do need
__new__, it’s incredibly powerful and invaluable to understand.
– Arion Sprague, Python’s Hidden New
It is rare to come across a problem in Python where the best solution was to use
__new__. The trouble is that if you have a hammer, every problem starts to look like a nail - and you might “suddenly” come across many problem that
__new__ can solve. Always prefer a better design over a shiny new tool.
__new__ is not always better.
Follow me on Twitter and Facebook
Thanks to Hannan Aharonov, Yonatan Nakar and Ram Rachum for reading drafts of this.