Python Course

Home Python 2 Tutorial Python 3 Tutorial Advanced Topics Numerical Programming Machine Learning Tkinter Tutorial Contact

Previous Chapter: Multiple Inheritance

Next Chapter: OOP, Inheritance Example

Magic Methods and Operator Overloading

Follow Bernd Klein,

the author of this
Introduction website, at Google+:
Bernd Klein on
Python 3
Google
Tutorial The so-called magic methods have nothing to do with wizardry. You have already seen them in previous chapters of our tutorial. They are special methods with fixed
names. They are the methods with this clumsy syntax, i.e. the double underscores at the beginning and the end. They are also hard to talk about. How do you
Bernd Klein on
The Origins of pronounce or say a method name like __init__? "Underscore underscore init underscore underscore" sounds horrible and is nearly a tongue twister. "Double
Facebook
Python underscore init double underscore" is a lot better, but the ideal way is "dunder init dunder"1 That's why magic methods methods are sometimes called dunder
methods!
Starting with
Python: The Search this website:
So what's magic about the __init__ method? The answer is, you don't have to invoke it directly. The invocation is realized behind the scenes. When you create an
Interactive Shell instance x of a class A with the statement "x = A()", Python will do the necessary calls to __new__ and __init__.
Executing a Go
Script Nearly at the end of this chapter of our tutorial we will introduce the __call__ method. It is overlooked by many beginners and even advanced programmers of
Indentation Python. It is a functionality which many programming languages do not have, so programmers are generally not looking for. The __call__ method enables Python This topic in German
Data Types and programmers to write classes where the instances behave like functions. Both functions and the instances of such classes are called callables. / Deutsche
Variables Übersetzung:
We have encountered the concept of operator overloading many times in the course of this tutorial. We had used the plus sign to add numerical values, to Magische Methoden
Operators
concatenate strings or to combine lists: und Operator-
Sequential Data
Überladung
Types: Lists and
Strings >>> 4 + 5
9
Classroom
List
>>> 3.8 + 9 Training
Manipulations Courses
12.8
Shallow and >>> "Peter" + " " + "Pan"
Deep Copy 'Peter Pan' The goal of this
Dictionaries >>> [3,6,8] + [7,11,13]
website is to provide
[3, 6, 8, 7, 11, 13]
Sets and Frozen >>> educational material,
Sets allowing you to learn
An Extensive Python on your own.
Example Using It's even possible to overload the "+" operator as well as all the other operators for the purposes of your own class. To do this, you need to understand the underlying mechanism. Nevertheless, it is
There is a special (or a "magic") method for every operator sign. The magic method for the "+" sign is the __add__ method. For "-" it is "__sub__" and so on. We have a complete faster and more
Sets
listing of all the magic methods a little bit further down. efficient to attend a
input via the
"real" Python course
keyboard in a classroom, with
The mechanism works like this: If we have an expression "x + y" and x is an instance of class K, then Python will check the class definition of K. If K has a method __add__ it will
Conditional be called with x.__add__(y), otherwise we will get an error message. an experienced
Statements trainer. So why not
Loops, while attend one of the live
Loop Traceback (most recent call last): Python courses in
File "<stdin>", line 1, in <module> Strasbourg, Paris,
For Loops TypeError: unsupported operand type(s) for +: 'K' and 'K'
Difference Luxembourg,
Amsterdam, Zürich /
between
Zurich, Vienna /
interators und
Wien, London, Berlin,
Iterables Munich, Hamburg,
Output with Print Overview of Magic Methods Frankfurt or Lake
Formatted output Constance by Bernd
with string Klein, the author of
Binary Operators this tutorial?
modulo and the
format method Operator Method
Functions + object.__add__(self, other)
Recursion and Onsite Training
- object.__sub__(self, other) Courses
Recursive
Functions * object.__mul__(self, other)
// object.__floordiv__(self, other) Let us come to your
Parameter
company,
Passing in / object.__truediv__(self, other) organization or
Functions % object.__mod__(self, other) institute and train
Namespaces your employees, as
** object.__pow__(self, other[, modulo])
Global and Local we've done it many
Variables << object.__lshift__(self, other) times in Amsterdam
Decorators >> object.__rshift__(self, other) (The Netherlands),
Memoization with Berlin (Germany),
& object.__and__(self, other)
Decorators Bern (Switzerland),
^ object.__xor__(self, other) Basel (Switzerland),
Read and Write
| object.__or__(self, other) Zurich (Switzerland),
Files
Locarno
Modular (Switzerland), Den
Programming Extended Assignments Haag (The Hague),
and Modules Hamburg (Germany),
Operator Method Frankfurt (Germany),
Packages in
Python += object.__iadd__(self, other) Toronto (Canada),
Regular -= object.__isub__(self, other) Edmonton (Canada),
Munich (Germany)
Expressions *= object.__imul__(self, other) and many other
Regular
/= object.__idiv__(self, other) cities. We do training
Expressions, courses in England,
Advanced //= object.__ifloordiv__(self, other)
Switzerland,
Lambda %= object.__imod__(self, other) Liechtenstein,
Operator, Filter, **= object.__ipow__(self, other[, modulo]) Austria, Germany,
Reduce and Map France, Belgium, the
<<= object.__ilshift__(self, other)
List Netherlands,
>>= object.__irshift__(self, other) Luxembourg, Poland,
Comprehension
&= object.__iand__(self, other) UK, Italy and other
Iterators and
locations in Europe
Generators ^= object.__ixor__(self, other)
and in Canada.
Exception |= object.__ior__(self, other)
Handling This way you will get
Tests, DocTests, a perfect training up
UnitTests
Unary Operators to your needs and it
Object Oriented will be extremely cost
Operator Method
Programming efficient as well.
- object.__neg__(self) Contact us so we can
Class and
+ object.__pos__(self) define and find the
Instance
best course
Attributes abs() object.__abs__(self)
curriculum to meet
Properties vs. ~ object.__invert__(self) your needs, and
getters and complex() object.__complex__(self) schedule course
setters sessions to be held at
int() object.__int__(self)
Inheritance your location.
Multiple long() object.__long__(self)
Inheritance float() object.__float__(self)
Magic Methods oct() object.__oct__(self) Skilled Python
and Operator Programmers
hex() object.__hex__(self
Overloading
You are looking for
OOP, Inheritance
Comparison Operators experienced Python
Example
developers or
Slots programmers? We
Operator Method
Classes and can help you, please
< object.__lt__(self, other)
Class Creation contact us.
Road to <= object.__le__(self, other)
Metaclasses == object.__eq__(self, other) Quote of the
Metaclasses != object.__ne__(self, other) Day:
Metaclass Use
>= object.__ge__(self, other)
Case: Count "Many people tend to
> object.__gt__(self, other) look at programming
Function Calls
Abstract Classes styles and languages
like religions: if you
belong to one, you
Example class: Length cannot belong to
Some Magic
others. But this
analogy is another
"And above all, watch We will demonstrate in the following Length class, how you can overload the "+" operator for your own class. To do this, we have to overload the __add__ method. Our class contains the __str__ and __repr__
fallacy." (Niklaus
with glittering eyes methods as well. The instances of the class Length contain length or distance information. The attributes of an instance are self.value and self.unit.
Wirth)
the whole world
around you because This class allows us to calculate expressions with mixed units like this one:
the greatest secrets
are always hidden in 2.56 m + 3 yd + 7.8 in + 7.03 cm
the most unlikely
places. Those who The class can be used like this:
Data Protection
don't believe in magic
Declaration
will never find it."
>>> from unit_conversions import Length
(Roald Dahl)
>>> L = Length Data Protection
>>> print(L(2.56,"m") + L(3,"yd") + L(7.8,"in") + L(7.03,"cm")) Declaration
"It's still magic even 5.57162
if you know how it's >>>
done." (Terry
Pratchett, A Hat Full
of Sky) The listing of the class:

Japanese Food
class Length:
and OOP
__metric = {"mm" : 0.001, "cm" : 0.01, "m" : 1, "km" : 1000,
"I think both Object- "in" : 0.0254, "ft" : 0.3048, "yd" : 0.9144,
Oriented "mi" : 1609.344 }
Programming and
def __init__(self, value, unit = "m" ):
Japanese food are
self.value = value
significant aesthetic self.unit = unit
and stylistic
achievements - def Converse2Metres(self):
they're both about return self.value * Length.__metric[self.unit]
taking simple meager
ingredients and def __add__(self, other):
l = self.Converse2Metres() + other.Converse2Metres()
making something
return Length(l / Length.__metric[self.unit], self.unit )
remarkable.", Sean
M. Burke def __str__(self):
return str(self.Converse2Metres())

Object-oriented def __repr__(self):

return "Length(" + str(self.value) + ", '" + self.unit + "')"
Programming
if __name__ == "__main__":
"Certainly not every x = Length(4)
good program is print(x)
object-oriented, and y = eval(repr(x))
not every object-
z = Length(4.5, "yd") + Length(1)
oriented program is print(repr(z))
good." print(z)
(Bjarne Stroustrup,
Danish computer
scientist, best known
for the creation and If we start this program, we get the following output:
the development of
the widely used C++
programming 4
language.) Length(5.593613298337708, 'yd')
5.1148
"Object-oriented
programming is an
We use the method__iadd__ to implement the extended assignment:
exceptionally bad
idea which could only
have originated in def __iadd__(self, other):
California." l = self.Converse2Metres() + other.Converse2Metres()
(Edsger Dijkstra, self.value = l / Length.__metric[self.unit]
(Dutch computer return self
Scientist, 1930-2002)

Dijkstra also said:

"... what society Now we are capable to write the following assignments:
overwhelmingly asks
for is snake oil. Of
x += Length(1)
course, the snake oil
x += Length(4, "yd")
has the most
impressive names -
otherwise you would
We have added 1 metre in the example above by writing "x += Length(1))". Most certainly, you will agree with us that it would be more convenient to simply write "x += 1" instead. We also want to treat
be selling nothing -
expressions like "Length(5,"yd") + 4.8" similarly. So, if somebody uses a type int or float, our class takes it automatically for "metre" and converts it into a Length object. It's easy to adapt our __add__ and
like "Structured
"__iadd__" method for this task. All we have to do is to check the type of the parameter "other":
Analysis and Design",
"Software
Engineering", def __add__(self, other):
"Maturity Models", if type(other) == int or type(other) == float:
"Management l = self.Converse2Metres() + other
Information else:
Systems", l = self.Converse2Metres() + other.Converse2Metres()
return Length(l / Length.__metric[self.unit], self.unit )
"Integrated Project
Support def __iadd__(self, other):
Environments" if type(other) == int or type(other) == float:
"Object Orientation" l = self.Converse2Metres() + other
and "Business else:
Process Re- l = self.Converse2Metres() + other.Converse2Metres()
engineering" self.value = l / Length.__metric[self.unit]
return self

This website is It's a safe bet that if somebody works for a while with adding integers and floats from the right sight that he or she wants to the same from the left side! So let's try it out:
supported by:
>>> from unit_conversions import Length
Linux and Python >>> x = Length(3, "yd") + 5
>>> x = 5 + Length(3, "yd")
Courses and
Traceback (most recent call last):
Seminars File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'Length'
>>>

Of course, the left side has to be of type "Length", because otherwise Python tries to apply the __add__ method from int, which can't cope with Length objects as second arguments!

Python provides a solution for this problem as well. It's the __radd__ method. It works like this: Python tries to evaluate the expression "5 + Length(3, 'yd')". First it calls int.__add__(5,Length(3, 'yd')), which will
raise an exception. After this it will try to invoke Length.__radd__(Length(3, "yd"), 5). It's easy to recognize that the implementation of __radd__ is analogue to __add__:

def __radd__(self, other):

if type(other) == int or type(other) == float:
l = self.Converse2Metres() + otherLength.__radd__(Length(3, "yd"), 5)
else:
l = self.Converse2Metres() + other.Converse2Metres()
return Length(l / Length.__metric[self.unit], self.unit )

It's advisable to make use of the __add__ method in the __radd__ method:

def __radd__(self, other):

return Length.__add__(self,other)

The following diagram illustrates the relationship between __add__ and __radd__:

The __call__ method

The __call__ method can be used to turn the instances of the class into callables. Functions are callable objects. A callable object is an object which can be used and behaves like a function but might not be a
function. By using the __call__ method it is possible to define classes in a way that the instances will be callable objects. The __call__ method is called, if the instance is called "like a function", i.e. using brackets.
The following example defines a class with which we can create abitrary polynomial functions:

class Polynomial:

def __init__(self, *coefficients):

self.coefficients = coefficients[::-1]

def __call__(self, x):

res = 0
for index, coeff in enumerate(self.coefficients):
res += coeff * x** index
return res

# a constant function
p1 = Polynomial(42)

# a straight Line
p2 = Polynomial(0.75, 2)

# a third degree Polynomial

p3 = Polynomial(1, -0.5, 0.75, 2)

for i in range(1, 10):

print(i, p1(i), p2(i), p3(i))

These are the results of the previous function:

1 42 2.75 3.25
2 42 3.5 9.5
3 42 4.25 26.75
4 42 5.0 61.0
5 42 5.75 118.25
6 42 6.5 204.5
7 42 7.25 325.75
8 42 8.0 488.0
9 42 8.75 697.25

You will find further interesting examples of the __call__ function in our tutorial in the chapters Decorators and Memoization with Decorators. You may also consult our chapter on Polynomials

Standard Classes as Base Classes

It's possible to use standard classes - like int, float, dict or lists - as base classes as well.

We extend the list class by adding a push method:

class Plist(list):

def __init__(self, l):

list.__init__(self, l)

def push(self, item):

self.append(item)

if __name__ == "__main__":
x = Plist([3,4])
x.push(47)
print(x)

This means that all the previously introduced binary and extended assignment operators exist in the "reversed" version as well:

__radd__
__rsub__
__rmul__
...
and so on

Exercises

1. Write a class with the name Ccy, similar to the previously defined Length class.

Ccy should contain values in various currencies, e.g. "EUR", "GBP" or "USD". An instance should contain the amount and the currency unit.

The class, you are going to design as an excercise, might be best described with the following example session:

>>> from currencies import Ccy

>>> v1 = Ccy(23.43, "EUR")
>>> v2 = Ccy(19.97, "USD")
>>> print(v1 + v2)
32.89 EUR
>>> print(v2 + v1)
31.07 USD
>>> print(v1 + 3) # an int or a float is considered to be a EUR value
27.43 EUR
>>> print(3 + v1)
27.43 EUR
>>>

Solutions to our Exercises

1. First exercise:

"""

The class "Ccy" can be used to define money values in various currencies. A Ccy instance has the string attributes 'unit' (e.g. 'CHF', 'CAD' od 'EUR' and the 'value' as a
float.
A currency object consists of a value and the corresponding unit.

"""

class Ccy:

currencies = {'CHF': 1.0821202355817312,

'CAD': 1.488609845538393,
'GBP': 0.8916546282920325,
'JPY': 114.38826536281809,
'EUR': 1.0,
'USD': 1.11123458162018}

def __init__(self, value, unit="EUR"):

self.value = value
self.unit = unit

def __str__(self):
return "{0:5.2f}".format(self.value) + " " + self.unit

def changeTo(self, new_unit):

"""
An Ccy object is transformed from the unit "self.unit" to "new_unit"
"""
self.value = (self.value / Ccy.currencies[self.unit] * Ccy.currencies[new_unit])
self.unit = new_unit

def __add__(self, other):

"""
Defines the '+' operator.
If other is a CCy object the currency values
are added and the result will be the unit of
self. If other is an int or a float, other will
be treated as a Euro value.
"""
if type(other) == int or type(other) == float:
x = (other * Ccy.currencies[self.unit])
else:
x = (other.value / Ccy.currencies[other.unit] * Ccy.currencies[self.unit])
return Ccy(x + self.value, self.unit)

def __iadd__(self, other):

"""
Similar to __add__
"""
if type(other) == int or type(other) == float:
x = (other * Ccy.currencies[self.unit])
else:
x = (other.value / Ccy.currencies[other.unit] * Ccy.currencies[self.unit])
self.value += x
return self

def __radd__(self, other):

res = self + other
if self.unit != "EUR":
res.changeTo("EUR")
return res

# __sub__, __isub__ and __rsub__ can be defined analogue

x = Ccy(10,"USD")
y = Ccy(11)
z = Ccy(12.34, "JPY")
z = 7.8 + x + y + 255 + z
print(z)

lst = [Ccy(10,"USD"), Ccy(11), Ccy(12.34, "JPY"), Ccy(12.34, "CAD")]

z = sum(lst)

print(z)

The program returns:

282.91 EUR
28.40 EUR

Another interesting aspect of this currency converter class in Python can be shown, if we add multiplication. You will easily understand that it makes no sense to allow expressions like "12.4 € * 3.4
" (orinpraef ixnotation :" €12.4∗ 3.4"), but it makes perfect sense to evaluate "3 * 4.54 €". You can find the new currency converter class with the newly added methods for __mul__, __imul__ and __rmul__

in the following listing:

"""

The class "Ccy" can be used to define money values in various currencies. A Ccy instance has the string attributes 'unit' (e.g. 'CHF', 'CAD' od 'EUR' and the 'value' as a
float.
A currency object consists of a value and the corresponding unit.

"""

class Ccy:

currencies = {'CHF': 1.0821202355817312,

'CAD': 1.488609845538393,
'GBP': 0.8916546282920325,
'JPY': 114.38826536281809,
'EUR': 1.0,
'USD': 1.11123458162018}

def __init__(self, value, unit="EUR"):

self.value = value
self.unit = unit

def __str__(self):
return "{0:5.2f}".format(self.value) + " " + self.unit

def __repr__(self):
return 'Ccy(' + str(self.value) + ', "' + self.unit + '")'

def changeTo(self, new_unit):

"""
An Ccy object is transformed from the unit "self.unit" to "new_unit"
"""
self.value = (self.value / Ccy.currencies[self.unit] * Ccy.currencies[new_unit])
self.unit = new_unit

def __add__(self, other):

"""
Defines the '+' operator.
If other is a CCy object the currency values
are added and the result will be the unit of
self. If other is an int or a float, other will
be treated as a Euro value.
"""
if type(other) == int or type(other) == float:
x = (other * Ccy.currencies[self.unit])
else:
x = (other.value / Ccy.currencies[other.unit] * Ccy.currencies[self.unit])
return Ccy(x + self.value, self.unit)

def __iadd__(self, other):

"""
Similar to __add__
"""
if type(other) == int or type(other) == float:
x = (other * Ccy.currencies[self.unit])
else:
x = (other.value / Ccy.currencies[other.unit] * Ccy.currencies[self.unit])
self.value += x
return self

def __radd__(self, other):

res = self + other
if self.unit != "EUR":
res.changeTo("EUR")
return res

# __sub__, __isub__ and __rsub__ can be defined analogue

def __mul__(self, other):

"""
Multiplication is only defined as a scalar multiplication,
i.e. a money value can be multiplied by an int or a float.
It is not possible to multiply to money values
"""
if type(other)==int or type(other)==float:
return Ccy(self.value * other, self.unit)
else:
raise TypeError("unsupported operand type(s) for *: 'Ccy' and " + type(other).__name__)

def __rmul__(self, other):

return self.__mul__(other)

def __imul__(self, other):

if type(other)==int or type(other)==float:
self.value *= other
return self
else:
raise TypeError("unsupported operand type(s) for *: 'Ccy' and " + type(other).__name__)

Assuming that you have saved the class under the name currency_converter, you can use it in the following way in the command shell:

>>> from currency_converter import Ccy

>>> x = Ccy(10.00, "EUR")
>>> y = Ccy(10.00, "GBP")
>>> x + y
Ccy(21.215104685942173, "EUR")
>>> print(x + y)
21.22 EUR
>>> print(2*x + y*0.9)
30.09 EUR
>>>

We can further improve our currency converter class by using a function get_currencies, which downloads the latest exchange rates from finance.yahoo.com. This function returns an exchange rates dictionary
in our previously used format. The function is in a module called exchange_rates.py This is the code of the function exchange_rates.py:

from urllib.request import urlopen

from bs4 import BeautifulSoup

def get_currency_rates(base="USD"):
""" The file at location url is read in and the exchange rates are extracted """
url = "https://finance.yahoo.com/webservice/v1/symbols/allcurrencies/quote"
data = urlopen(url).read()
data = data.decode('utf-8')
soup = BeautifulSoup(data, 'html.parser')
data = soup.get_text()

flag = False
currencies = {}
for line in data.splitlines():
if flag:
value = float(line)
flag = False
currencies[currency] = value
if line.startswith("USD/"):
flag = True
currency = line[4:7]

currencies["USD"] = 1 # we must add it, because it's not included in file

if base != "USD":
base_currency_rate = currencies[base]
for currency in currencies:
currencies[currency] /= base_currency_rate

return currencies

We can import this function from our module. (You have to save it somewhare in your Python path or the directory where your program runs):

from exchange_rates import get_currency_rates

class Ccy:

currencies = get_currencies()

# continue with the code from the previous version

We save this version as currency_converter_web.

>>> from currency_converter_web import Ccy

>>> x = Ccy(1000, "JPY")
>>> y = Ccy(10, "CHF")
>>> z = Ccy(15, "CAD")
>>> print(2*x + 4.11*y + z)
7722.98 JPY
>>>

Footnotes

1 as suggested by Mark Jackson

Previous Chapter: Multiple Inheritance

Next Chapter: OOP, Inheritance Example

© 2011 - 2018, Bernd Klein, Bodenseo; Design by Denise Mitchinson adapted for python-course.eu by Bernd Klein