Measuring Polymorphism in Python Programs

Beatrice Åkerblom Tobias Wrigstad

Stockholm University, Sweden Uppsala University, Sweden
beatrice@dsv.su.se tobias.wrigstad@it.uu.se

Abstract 1. Introduction
Following the increased popularity of dynamic languages and The increasing use of dynamic languages in critical applica-
their increased use in critical software, there have been many tion domains [21, 25, 30] has prompted academic research on
proposals to retrofit static type system to these languages to “retrofitting” dynamic languages with static typing. Examples
improve possibilities to catch bugs and improve performance. include using type inference or programmer declarations for
A key question for any type system is whether the types Self [1], Scheme [36], Python [4, 5, 29], Ruby [3, 15], JavaS-
should be structural, for more expressiveness, or nominal, to cript [17, 35], and PHP [14].
carry more meaning for the programmer. For retrofitted type Most mainstream programming languages use static typ-
systems, it seems the current trend is using structural types. ing from day zero, and thus naturally imposed constraints
This paper attempts to answer the question to what extent on the run-time flexibility of programs. For example, strong
this extra expressiveness is needed, and how the possible static typing usually guarantees that a well-typed x.m() at
polymorphism in dynamic code is used in practise. compile-time will not fail at run-time due to a “message not
We study polymorphism in 36 real-world open source understood”. This constraint restricts developers to updates
Python programs and approximate to what extent nominal that grow types monotonically.
and structural types could be used to type these programs. Retrofitting a static type system on a dynamic language
The study is based on collecting traces from multiple runs where the definitions of classes, and even individual objects,
of the programs and analysing the polymorphic degrees of may be arbitrarily redefined during runtime poses a signific-
targets at more than 7 million call-sites. ant challenge. In previous work for Ruby and Python, for
Our results show that while polymorphism is used in all example, restrictions have been imposed om the languages to
programs, the programs are to a great extent monomorphic. simplify the design of type systems. The simplifications con-
The polymorphism found is evenly distributed across librar- cern language features like dynamic code evaluation [15, 29],
ies and program-specific code and occur both during program the possibility to make dynamic changes to definitions of
start-up and normal execution. Most programs contain a few classes and methods [4], and possibility to remove methods
“megamorphic” call-sites where receiver types vary widely. [15]. Recent research [2, 24, 27] has shown that the use of
The non-monomorphic parts of the programs can to some such dynamic language features is rare—but non-negligible.
extent be typed with nominal or structural types, but none of Apart from the inherent plasticity of dynamic languages
the approaches can type entire programs. described above, a type system designer must also consider
the fact that dynamic typing gives a language unconstrained
polymorphism. In Python, and other dynamic languages,
Categories and Subject Descriptors D.3 Programming there is no static type information that can be used to control
Languages [D.3.3 Language Constructs and Features]: Poly- polymorphism e.g., for method calls or return values.
morphism Previous retrofitted type systems use different approaches
to handle ad-hoc polymorphic variables. Some state pre-
requisites disallowing polymorphic variables [5, 35], assum-
Keywords Python, dynamic languages, polymorphism, trace- ing that polymorphic variables are rare [29]. Others use a
based analysis flow-sensitive analysis to track how variables change types
[11, 15, 18]. Disallowing polymorphic variables is too re-
strictive as it rules out polymorphic method calls [19, 24, 27].
There are not many published results on the degree of
polymorphism or dynamism in dynamic languages [2, 8, 19,
24, 27]. This makes it difficult to determine whether or not
relying on the absence of, or restricting, some dynamic beha-
viour is possible in practise, and whether certain techniques
for handling difficulties arising due to dynamicity is prefer- 2.1 Polymorphism and Types
able over others. Most definitions of object-oriented programming lists poly-
This article presents the results of a study of the runtime morphism—the ability of an object of type T to appear as of
behaviour of 36 open source Python programs. We inspect another type T 0 —as one of its cornerstones.
traces of runs of these programs to determine the extent to In dynamically typed languages, like Python, polymorph-
which method calls are polymorphic in nature, and the nature ism is not constrained by static checking and error-checking
of that polymorphism, ultimately to find out if programs’ is deferred to the latest possible time for maximal flexibility.
polymorphic behaviour can be fitted into a static type. This means that T and T 0 from above need not be explicitly
1.1 Contributions related (through inheritance or other language mechanisms).
It also means that fields can hold values of any type and still
This paper presents the results of a trace-based study of a function normally (without errors) as long as all uses con-
corpus of 36 open-source Python programs, totalling oven 1 form to the run-time type of the current object they store.
million LOC. Extracting and analysing over 7 million call- This kind of typing/polymorphic behaviour is commonly re-
sites in over 800 million events from trace-logs, we report ferred to as “duck typing” [23].
several findings – in particular: Subtype polymorphism in statically typed languages is
– A study of the run-time types of receiver variables that bounded by the requirements needed for static checking (e.g.,
shows the extent to which the inherently polymorphic that all well-typed method calls can be bound to suitable
nature of dynamic typing is used in practise. methods at run-time). This leads to restrictions for how T and
T 0 may be related. In a nominal system this may mean that
We find that variables are predominantly monomorphic,
the classes used to define T and T 0 must have an inheritance
i.e., only holds values of a single type during a program.
relation. A nominal type is a type that is based on names, that
However, most programs have a few places which are
is that type equality for two objects requires that the name
megamorphic, i.e., variables containing values of many
of the types of the objects is the same. In a structural type
different types at different times or in different contexts.
system, type equivalence and subtyping is decided by the
Hence, a retrofitted type system should consider both these
definition of values’ structures. For example, in OCaml and
circumstances.
Strongtalk, type equivalence is determined by comparing the
– An approximation of the extent to which a program can be fields and methods of two objects and also comparing their
typed using nominal or structural types using three type- signatures (method arguments and return values).
ability metrics for nominal types, nominal types with para- Strachey [33] separates the polymorphism of functions
metric polymorphism, and structural types. We consider into two different categories: ad hoc and parametric. The
both individual call-sites and clusters of call-sites inside a main difference between the categories is that ad-hoc poly-
single source file. morphism lacks the structure brought by parameterisation
We find that, because of monomorphism, most programs and that there is no unified method that makes it possible to
can be typed to a large extent using simple type systems. predict the return type from an ad-hoc polymorphic function
Most polymorphic and megamorphic parts of programs are based on the arguments passed in as would be the case for
not typeable by nominal or structural systems, for example the parametric polymorphic function [33]. As an example of
due to use of value-based overloading. Structural typing is ad hoc polymorphism, consider overloading of / for combin-
only slightly better than nominal typing at handling non- ations of integers and reals always yielding a real.
monomorphic program parts. Cardelli and Wegner [10] further divide polymorphism
into two categories at the top level: universal and ad-hoc.
Our trace data and a version of this article with larger figures
Universal polymorphism corresponds to Strachey’s paramet-
is available from dsv.su.se/~beatrice/python.
ric polymorphism together with call inclusion polymorphism,
Outline The paper is organised as follows. § 2 gives a which includes object-oriented polymorphism (subtypes and
background on polymorphism and types. § 3 describe the inheritance). The common factor for universal polymorphism
motivations and goals of the work. § 4 accounts for how the is that it is based on a common structure (type) [10]. Ad-hoc
work was conducted. § 5 presents the results. § 7 discusses polymorphism, on the other hand, is divided into overload-
related research and finally in § 8 we present our conclusions ing and coercion, where overloading allows using the same
and present ideas for future work. name for several functions and coercion allowing polymorph-
ism in situations when a type can automatically be translated
2. Background to another type [10].
Using the terms from above, “duck typing” can be de-
We start with a background and overview of polymorphism
scribed as a lazy structural typing [23] (late type checking)
and types (§ 2.1) followed by a quick overview of the Python
and is a subcategory of ad-hoc polymorphism [10].
programming language (§ 2.2). A reader with a good under-
standing of these areas may skip over either or both part(s).
2.2 Python 2.3 Measuring Polymorphism
Python is a class based language, but Python’s classes are When the code below is run, a class Foo is first defined
far less static than classes normally found in statically typed containing two methods; init and bar, both expecting
systems. Class definitions are executed during runtime much one argument. The init method creates the instance
like any other code, which means that a class is not available variable a and assigns the expected argument to it. In the
until its definition has been executed. Class definition may bar method a call is made to the method foo on the instance
appear anywhere, e.g., in a subroutine or within one branch variable a and then a call is made to the method baz on the
of a conditional statement. If two class definitions with the argument variable b.
same name are executed within the same name-space, the last 01 10 f = Foo(...)
definition will replace the first (although already created ob- 02 class Foo: 11
jects will keep the old class definition). If a class is reloaded, 03 def __init__(self, a): 12 for e in range(0,100):
it might have been reloaded with a different set of methods 04 self.a = a 13 class Bar:
05 14 def baz(self):
than the original one. Given this possibility to reload classes, 06 def bar(self, b): 15 pass
the same code creating objects from the class C may end up 07 self.a.foo() 16
creating objects of different classes at different times during 08 b.baz() 17 f.bar(Bar())
execution, objects that may have a different set of methods. 09 18
Python allows multiple inheritance, i.e., a class may have After the class definition is finished, a variable f is created
many superclasses [28]. Subclasses may override methods in and it is assigned with a new object of the class Foo.
its superclass(es) and may call methods in its superclass(es). On line 12–17, follows a for loop that will iterate 100
Python’s built-in classes can be used as superclasses. times and for every iteration the class Bar is created with
Python classes are represented as objects at runtime. Class a method baz that has no body. On the last line in the for
objects can contain attributes and methods. All members in a loop a call is made to the method bar for the Foo object in f
Python class (attributes and methods) are public. Methods al- (from line 10) passing a new object of the current Bar class
ways take the receiver of the method call as the first argument. as an argument.
It must be explicitly included in the method’s parameter list Several lines in the code above (7, 8 and 17), contain
but is passed in implicitly in the method call. method calls. These lines are call-sites.
There are two different types of classes available in Py-
thon up to version 3.0: old-style/classic classes and new-style D E F I N I T I O N 1 (Call-site). A call-site in a program is a
classes. The latter were introduced in Python 2.2 (released point (on a line in a Python source file) where a method call
in 2001) to unify class and type hierarchies of the language is made.
and they also (among other things) brought a new method Every call-site has two points, the receiver and the argu-
resolution order for multiple inheritance. From Python 3.0, ment(s), where types may vary depending on the path taken
all classes are new-style classes. through the program up to the call-site. In the analyses made
Python objects are essentially hash tables in that attributes for this paper, the focus has been on the receiver types. Argu-
and methods and their names may be regarded as key-value ments will generally become receivers at a later point in the
pairs. Both attributes and methods may be added, replaced program execution, which means that also that polymorph-
and entirely removed also after initialisation. For an object ism will get captured by the logging.
foo, we can add an attribute bar by simply assigning to On line 17, a call is made to the method bar, where the
that name, i.e. foo.bar = ’Baz’. The same attribute may receiver will always be an object of the class Foo, since the
then be removed, e.g., by the statement del foo.bar, which assignment to f is made from a call to the constructor of
removes both key and value. Foo on line 10. This means that the call-site f.bar(...) on
Classes in Python are thus less templates for object cre- line 10 is monomorphic and will always resolve to the same
ation than what we may be used to from statically typed method at run-time.
languages, but more like factories creating objects–objects
that may later change independent of their class and the other D E F I N I T I O N 2 (Monomorphic). A call-site that has the
objects created from the same class. This more dynamic ap- same receiver type in all observations is monomorphic.
proach to classes has implications on and may increase pro-
The call-site on line 7 may be monomorphic, but that can-
gram polymorphism.
not be concluded from the static information in the available
In nominally typed language, a type Sub is a subtype of
code. The type of the receiver on line 7 depends on the type
another type Sup only if the class Sup is explicitly declared
of the argument to the constructor when the object was cre-
to be its supertype. In some languages, Python for example,
ated. If objects are created storing objects of different types in
this declaration may be updated and changed during runtime.
the instance variable a, the line 7 will potentially be executed
with more than one receiver type, that is, it is polymorphic.
If the number of receiver types is very high, the call-site is
instead megamorphic. Following Agesen [1] we count a call- 2. Extent and degree
site as megamorphic if it has been observed with six or more (a) What is the proportion between monomorphic and
receiver types. polymorphic call-sites?
D E F I N I T I O N 3 (Polymorphic). A call-site that has 2–5 (b) What is the average, median and maximum degrees
different receiver types in all observations is polymorphic. of polymorphism and megamorphism (that is, number
of receiver types) of non-monomorphic call-sites?
(c) To what extent are non-monomorphic call-sites
D E F I N I T I O N 4 (Megamorphic). A call-site that has six “megamorphic”?
or more receiver types in all observations is megamorphic. (d) Does the degree of polymorphism and
Line 8 in the code above shows an example of a mega- megamorphism differ between library and program or
morphic call-site with a call to the method baz for the object between start-up and normal runtime?
in the variable b. The value of b depends on what is passed (e) What types are seen at extremely megamorphic
as the argument with the method call to bar, made on line call-sites (e.g., with 350 different receiver types)?
17. The loop on line 12–17 runs the class definition of Bar
3. Typeability
in every iteration, which means that every call to the method
baz will be made to an object of a new class. Nevertheless, (a) How do types at polymorphic and megamorphic
since the class always has the same name and contains the call-sites and clusters relate to each other in terms of
same fields and methods, the classes created here should be inheritance and overridden methods?
regarded as the same class. This megamorphism is false and (b) To what extent is it possible to find a common super
will not be considered as such by our analysis. type for all the observed receiver types that makes it
possible to fit the polymorphism into a nominal static
3. Motivation and Research Questions type?
A plethora of proposals for static type systems for dynamic (c) To what extent is it possible to find a common super
languages exist [1, 3–5, 15, 17, 29, 35, 36]. The inherent type for all the observed receiver types if the nominal
plasticity of the dynamic languages (for example, the possib- types are extended with parametric polymorphism?
ility to add and remove fields and methods and change an (d) To what extent do receiver types in clusters contain
object’s class at run-time) is a major obstacle for designers all the methods that are called at the call-sites of the
of type systems but the use of these possibilities have been cluster? That is, to what extent can we find a common
shown to be infrequent [2, 19, 24, 27]. Additionally, a type structural type for all the receiver types found in
system designer must also take duck typing into consider- clusters?
ation, where objects of statically unrelated classes may be
used interchangeably in places where common subsets of Following [2, 19, 24, 27] we also examine the applicability
their methods are used. of the phenomenon of Folklore, put forward by Richards et
We examine several aspects of Python programs of in- al [27] which states that there is an initialisation phase that is
terest to designers of type systems for dynamic languages more dynamic than other phases of the runtime. We compare
in general and for Python specifically. These aspects of pro- if there are differences in the use of polymorphism depending
gram dynamicity may also be used to enable comparisons of on where we find the method calls; during start-up vs. during
different proposed type system solutions. normal execution and also if there are differences between
We study Python’s unlimited polymorphism—duck typing— libraries and program-specific code.
in particular the degree of polymorphism in receivers of
method calls in typical programs: How many different types 4. Methodology
are used and how related the receivers’ types are e.g., in Studying how polymorphism is used in Python programs
terms of inheritance. We study how the underlying dynamic necessitates studying real programs. We discarded static ap-
nature of Python affects the polymorphism of programs due proaches such as program analysis and abstract interpretation
to classes being dynamically created and possibly modified because of their over-approximate nature. Instead, we base
at run-time. our study on traces of running programs obtained by an in-
Analysis Questions Our questions belong to three categor- strumented version of the standard CPython interpreter that
ies: program structure, extent and degree and typeability: saves data about all method calls made throughout a program
run. Our instrumented interpreter is based on CPython 2.6.6
1. Program structure because of Debian packaging constraints, which was import-
(a) How many classes do Python programs use/create at ant to study certain proprietary code which in the end did not
run-time? How often are classes redefined? end up in this study.
(b) How many methods do Python classes have and how The results are obtained from in total 522 runs of 36 open
many methods are overridden in subclasses? source Python programs (see Table 1) collected from Source-
Forge [32]. Selection was based on programs’ popularity b a
(>1,000 downloads), that the program was still maintained B B
(updated during the last 12 months) and was classified as f: A f: C
stable, i.e., had been under development for some time. For
pragmatic reasons, we excluded programs that used C exten- A ≮∶ C
sions, and programs that for various reasons would not run C≮∶ A
under Debian. For equally pragmatic reasons, we excluded
Figure 1. Parametric polymorphism. Different instances of
plugins (e.g., to web browsers), programs that required spe-
B hold objects of different types in the f fields.
cific hardware (e.g., microscopes, network equipment or serv-
ers) and software that required subscriptions (e.g., poker site
accounts). based on how many receiver types were found according to
To separate events in the start-up phase from ”normal the following categories:
Wednesday 28 January 15

program run-time” in our analyses, we followed the example

Single-call The call-site was only executed once. It is there-
of Holkner and Harland [19] and placed markers in the source
fore trivially monomorphic, but we conservatively refrain
of all programs at the point where the start-up phase finished.
from classifying it any further.
This would typically be at the point where the graphical user
Monomorphic The call-site was monomorphic and ex-
interface had finished loading and just before entering the
ecuted more than once, so it is observably monomorphic.
main loop of the program.
“Observably” refers to the nature of our trace-based
We have chosen to include libraries in our study to make it
method, which does not exclude the possibility that a
possible to compare the library code to program specific code
different run of the same program might observe poly-
to see if we find any difference in polymorphic behaviour. To
morphic behaviour for the same call-site.
separate the events originating in library code from those
Polymorphic The call-site was observed with between two
originating in program specific code in our analyses, a fully
and five different receiver types.
qualified file name was saved for all events.
Megamorphic The call-site was observed with more than
Command line programs were run using commands given
five different receiver types.
in official tutorials and manuals to capture the execution of all
standard expected use cases. Libraries were used in a similar
Typeability The questions in § 3 § 3a – § 3d are all con-
way with examples from official tutorials. Depending on the
cerned with to what extent the polymorphism found in real
availability of examples, command line programs and some
Python programs could be retrofitted with a type system.
libraries shared between multiple programs were run over
All monomorphic call-sites are always typeable with a
100 times.
nominal or a structural type. Receivers at a specific call-site
For applications with a GUI the official tutorials and ex-
in isolation will always have the same structural type (see
amples were followed by hand and care was taken to ensure
§ 2.1). For a polymorphic call-site to be nominally typeable,
that each menu alternative and button was used. The interact-
all receivers must share a common supertype that defines the
ive GUI applications were run for 10–15 minutes between 2
method in question.
and 12 times depending on the number of functions available.
We define a metric, N-typeable to approximate static type-
The Python interpreter we used was instrumented to trace
ability with a hypothetical simple nominal type system:
all method calls (including calls caused by internal Python
constructs, like the use of operators, etc.) and all loaded D E F I N I T I O N 5 (N-typeable). A polymorphic call-site is
class definitions. For all method calls made, we logged the N-typeable if there is, for all its receiver types, a common
call-site’s location in the source files, the receiver type and superclass that contains the method called at the call-site.
identity, the method name, the identity of the calling context
(self when the method call was made), the arguments’ types Nominal typing could be extended with parametric poly-
and return types. Every time a class definition was executed, morphism (see § 2.1 to increase the flexibility to account for
we logged the class name, names of superclasses and the different types being used in the same source locations across
names of the methods. different run-time contexts. In that case, a call-site can be
typed for unrelated receiver types given that it is N-typeable
Program Structure To answer our questions on program for each sender identity (that is the value of self when the
structure from § 3, we collect data about classes loaded at run- call was made).
time. We count recurrences of class definitions and compare This would mean that the receiver was typeable for all
their sets of methods. calls that were executed inside some specific object, as is
illustrated in Figure 1 with objects a and b, both instances
Extent and Degree (of Polymorphism) To answer our of the class B. The field f in a holds an instance of the class
questions in § 3 § 2a – § 2e, we collect receiver type inform- C, while the field f in b holds an instance of the class A. A
ation found at each call-site, and categorise the call-sites call-site in the code of the class B, that has the field f as a
receiver would in this case always have the same type for all For the cluster to be typeable with a structural type, all the
calls made in the same caller context. types (T and T’) seen at all call-sites (on line 3, 5 and 7) must
For all polymorphic and megamorphic call-sites we also contain all the methods that were called at all call-sites in the
examine if they are NPP-typeable: cluster (foo(), bar() and baz().
D E F I N I T I O N 6 (NPP-typeable). A polymorphic call-site D E F I N I T I O N 9 (S-typeable Cluster). A cluster is S-type-
is NPP-typeable if it is N-typeable or, if the receiver types able iff the intersection of all types of all its receivers contains
were grouped by the identity of the sender (self when the all the methods called at all call-sites in the cluster.
call was made), we find a common supertype for each group
that contains the method called at the call-site. Whereas considering individual call-sites may be overly
optimistic, considering clusters of call-sites may be overly
The typeability considered so far has been based on indi- pessimistic. For the code example above, for example, we
vidual call-sites (i.e., individual source locations). This might would conclude that the cluster was neither N-typeable nor
lead to an over-estimate of the typeability of programs. For S-typeable, since there exists no type T’’ that contain all
example, in the code example below, calls are made to the the three methods called at the cluster’s call-sites. A more
method example(a, b) with a first argument of either the powerful type system might be able to capture this value-
type T or T’, where T has the methods foo() and bar() but based overloading, such as a system with refinement types.
not the method baz() and where T’ has the method foo() Whether such a system used nominal or structural types is
and baz() but not the method bar(). The second argument insignificant in this case.
for the method calls is always a boolean; a boolean that is
always True when a is of the type T and False when a is of 5. Results
the type T’ (so-called value-based overloading).
This section presents the results from analysing 528 program
02 def example(a, b): traces of the 36 Python programs in our corpus. The results
03 a.foo() are grouped into the same categories that were presented in
04 if b: § 3; Program structure, Extent and degree and Typeability.
05 a.bar()
06 else: 5.1 Program Structure
07 a.baz()
Classes in Python Programs The underlying dynamic
Considering each call-site in isolation, the call-sites on nature of Python affects the polymorphism of programs in
line 5 and 7 are typeable since they will always have the same that classes are dynamically created and possibly modified
receiver type. However, giving a static type to the program at runtime. The possibility to reload a class with a differ-
without significant rewrite would assign a single type to a ent definition during runtime and the possibility that the
which means typing line 3, 5 and 7 with a single static type. path taken through the program affects the numbers and/or
To assign types to co-dependent source locations, we versions of classes that are loaded all contribute to the poly-
cluster call-sites connected by the same receiver values (i.e., morphism of Python programs. This polymorphism makes it
3 & 5 and 3 & 7) plus transitivity (i.e., 5 & 7, indirectly via more difficult to predict statically what types will be needed
3). We then attempt to type the cluster as a whole. to type the the program the next time it runs.
Our traces contained 31,941 unique classes. The source
D E F I N I T I O N 7 (Cluster). A cluster is a set of call-sites, code of the 36 programs contained the definition of 11,091
from the same source file, connected by the receivers they classes (libraries uncounted). The source of the individual
have seen. For all pairs of call-sites A and B in a cluster, programs contain between 4 and 1,839 class definitions with
they have either seen the same receiver or there exists a third an average of 308 classes and a median of 129 classes and
call-site C that has seen the same receiver as both A and B. (see Table 2).
Typing the cluster in the code example above, we search With only three exceptions (Pychecker, Docutils and Eric4),
for a common supertype of T and T’ that contains all of the number of classes loaded by the program was larger than
foo(), bar() and baz(), i.e., the union of the call-sites’ the number of classes defined in its source code. The number
methods in the cluster. If such a type does not exist, the of declared classes found in the source code can be found as
cluster can not be typed. It can be argued that rejecting the the first figure in the column titled “Class defs. top/nested”
cluster in its entirety is a better approximation than claiming in Table 2. That the number of classes used in a program
66% of the method’s call-sites typeable. is larger than the number defined in the program’s code is
what should be expected since Python comes with a large eco-
D E F I N I T I O N 8 (N-typeable Cluster). A cluster is N-type- system of libraries containing important utilities. The loading
able iff T’, the most specific common supertype of the types of these library modules leads to loading and creation of
of all receivers in all call-sites in the cluster, contains all the classes; classes that can not be found in the current program’s
methods called at all call-sites in the cluster. source code. The exceptions (Pychecker, Docutils and Eric3)
Table 1. A list of the programs included in the study, sorted on size (see Table 2. A list of the programs included in the study sorted on size
Table 2). The third column contains the share of the call-sites that were (LOC from the second column) with the smallest one at the top. The third
polymorphic + megamorphic (P+M), and the fourth one the share of these column shows the range (min-max number) of unique classes loaded when
P+M that were N-typeable (P+M N-t). The fifth column contains the share the programs were run. The fourth column contains the number of class
of all call-sites that were N-typeable (N-t). Column 3-5 all contain figures definitions found in the source code of the programs and the number of class
for whole programs. Column 6-7 contain P+M and N-t for program startup, definitions that were found in a nested environment (e.g. inside a method)
column 8-9 P+M and N-t for runtime, column 10 P+M for library code and the fifth the average number of method definitions loaded during the
and finally column 11 P+M for program specific code. All figures denote program runs. The sixth column contains the average number of method
the share of call-sites compared with the total numbers of call-sites in the definitions loaded during a program run that were redefinitions of inherited
program traces, except column 4 (Typeable Poly (%)). Program version methods. The seventh column contains the number of classes that were found
numbers can be found in Table 4. defined with more than one interface (set of methods). The eighth and last
column contains the the percent of all classes that use multiple inheritance.
Whole Startup Runtime Lib. Prog.
P+M Typeable N-t P+M N-t P+M N-t P+M P+M
No. Name (%) Poly (%) (%) (%) (%) (%) (%) (%) (%) #Classes Class defs. Avg.# Avg.# Int. Mult.
1. Pdfshuffler 2.5 32.7 0.8 0.6 0.0 4.2 0.5 2.0 4.6 Program LOC (range) top/nested meth. overr. diff. inh.(%)
2. PyTruss 1.6 3.9 0.1 - - - - - - 1. PDF-Shuffler 1.0K 181-181 4/0 1.6K 0.2K 0 11.0
3. Radiotray 1.6 18.8 0.3 1.5 0.0 1.7 0.4 1.8 0.0 2. PyTruss 1.5K 731-745 19/0 11.4K 1.7K 0 3.1
4. Gimagereader 3.0 4.4 0.1 0.9 0.1 7.6 0.1 3.2 2.0 3. Radiotray 1.5K 353-353 25/0 3.1K 0.4K 0 5.8
5. Ntm 1.1 3.8 0.0 1.2 0.0 1.0 0.0 1.3 0.2 4. GImageReader 2.2K 361-361 15/0 2.8K 0.4K 0 3.5
6. Torrentsearch 12.4 4.6 0.6 4.9 0.5 15.1 0.6 - - 5. Ntm 2.8K 239-239 10/0 2.0K 0.3K 0 5.4
7. Brainworkshop 1.0 20.9 0.2 0.6 0.1 2.7 0.4 0.6 3.6 6. TorrentSearch 3.0K 471-479 63/0 5.1K 0.5K 0 1.7
8. Bleachbit 4.2 6.8 0.3 3.4 0.3 7.5 0.2 2.5 9.7 7. BrainWorkshop 3.6K 673-677 43/0 6.0K 0.7K 0 2.4
9. Diffuse 1.5 0.6 0.0 0.8 0.0 2.1 0.0 12.4 2.5 8. BleachBit 4.1K 249-250 39/2 2.0K 0.2K 0 5.9
10. Photofilmstrip 3.6 37.5 1.3 0.6 0.0 5.3 1.0 4.0 1.8 9. Diffuse 5.6K 154-154 47/24 1.4K 0.1K 1 3.9
11. Comix 3.5 4.1 0.1 0.7 0.0 4.9 0.1 4.3 1.4 10. PhotoFilmStrip 6.1K 791-795 66/0 12.3K 1.9K 0 3.0
12. Pmw 3.1 49.0 1.7 - - - - - - 11. Comix 7.7K 287-308 45/0 2.3K 0.3K 0 3.3
13. Requests 2.8 24.9 0.6 - - - - 3.3 2.9 12. Pmw 10.3K 97-113 41/0 1.2K 0.1K 0 8.5
14. Virtaal 2.5 18.1 0.5 1.4 0.0 3.0 0.4 2.5 2.5 13. Requests 11.2K 366-423 109/6 2.9K 0.5K 0 10.2
15. Pychecker 1.5 8.7 0.3 - - - - 1.8 0.7 14. Virtaal 11.4K 644-654 133/18 13.7K 2.4K 0 3.2
16. Idle 5.6 56.0 3.2 1.1 0.4 7.7 4.2 3.7 8.1 15. Pychecker 12.7K 82-2180 311/35 2.0K 0.7K 0 5.6
17. Fretsonfire 2.2 18.3 0.4 1.2 0.0 3.7 1.0 1.7 3.3 16. Idle 13.0K 285-311 146/10 3.0K 0.4K 0 6.1
18. PyPe 2.5 17.8 0.4 1.3 0.7 4.5 1.1 2.1 4.8 17. FretsOnFire 14.0K 772-797 365/8 2.8K 0.8K 1 7.2
19. PyX 3.5 33.9 1.2 - - - - 1.3 4.5 18. PyPe 15.3K 891-891 320/30 7.2K 1.5K 1 5.8
20. Pyparsing 5.7 72.0 4.1 - - - - 1.6 11.9 19. PyX 15.8K 409-453 303/15 3.5K 0.5K 0 7.9
21. Rednotebook 1.4 3.7 0.1 1.2 0.0 1.8 0.0 1.5 1.3 20. Pyparsing 16.6K 111-160 109/4 3.8K 0.6K 0 5.9
22. Linkchecker 6.6 2.6 0.2 1.2 0.0 13.7 0.1 5.3 8.8 21. RedNotebook 17.4K 485-513 123/7 5.9K 1.2K 0 6.1
23. Solfege 2.8 41.4 1.2 1.2 0.0 3.6 1.5 1.3 3.9 22. LinkChecker 20.6K 891-891 235/9 3.9K 0.7K 1 6.3
24. Chilsdplay 4.1 33.9 1.4 0.9 0.0 6.3 3.3 1.7 8.5 23. Solfege 20.7K 489-502 248/7 11.4K 2.4K 1
25. Scikitlearn 3.1 60.9 2.1 - - - - - - 24. Childsplay 22.0K 929-957 233/16 8.2K 1.3K 0 8.4
26. Mnemosyne 3.0 57.2 1.8 1.2 0.2 3.1 2.0 2.8 3.6 25. ScikitLearn 22.5K 403-1208 184/11 8.9K 1.9K 4 3.0
27. Youtube-dl 1.2 11.6 0.1 - - - - - - 26. Mnemosyne 26.8K 1237-1237 125/1 0.8K 0.2K 4 5.3
28. Docutils 6.2 31.7 2.0 - - - - 2.2 8.7 27. Youtube-dl 28.5K 672-702 416/8 4.9K 1.3K 0 3.2
29. Pymol 8.6 0.6 0.1 - - - - 10.7 4.4 28. Docutils 32.1K 45-1239 541/14 4.3K 1.7K 0 14.1
30. Timeline 2.0 21.1 0.4 0.5 0.0 2.8 0.7 - - 29. PyMol 35.2K 276-281 46/10 3.3K 0.4K 0 3.2
31. DispcalGUI 2.9 15.5 0.4 0.8 0.0 4.1 0.6 2.1 4.2 30. Timeline 42.3K 819-944 769/12 13.0K 2.1K 0 4.7
32. Pysolfc 4.3 40.7 1.8 1.0 0.4 9.4 3.9 3.1 4.9 31. DispcalGUI 44.1K 1030-1030 180/11 14.9K 2.2K 0 3.9
33. Wikidpad 3.9 23.5 0.9 2.6 1.1 5.3 0.5 3.8 6.7 32. PySolFC 61.9K 2143-2156 1839/7 14.1K 5.6K 0 3.0
34. Task Coach 6.4 37.1 2.4 - - - - 3.6 8.4 33. WikidPad 84.9K 1185-1292 845/34 17.0K 2.7K 0 4.3
35. SciPy 6.8 42.4 2.8 - - - - 3.8 7.8 34. TaskCoach 101.5K 1848-2301 1230/69 22.7K 4.1K 2 9.3
36. Eric4 2.2 37.0 0.8 1.7 0.6 3.2 0.8 1.6 2.5 35. SciPy 130.6K 1030-1777 1074/91 15.7K 2.9K 10 2.4
Average 3.9 25.0 0.96 1.35 0.18 5.18 0.98 3.12 4.61 36. Eric4 177.3K 804-980 989/12 9.3K 1.1K 0 13.2
Averages 28.6K 623-793 314/13 6.9K 1.3K 0.7 5.8

runtime. This make types difficult to predict statically and

complicates typing of Python programs.
may be explained by the fact that each example that was
run for Pychecker and Docutils was small and focused on Old Style vs. New Style Classes As a result of the introduc-
explaining some specific part of the program functionality tion of new-style classes, a Python class hierarchy has two
and thus did not run all of the the programs. Eric4, in turn, possible root classes. If old style classes can be found in cur-
is an interactive program with large functionality and all rent programs, it would mean that the development of a type
functions were not executed in each run of the program. system for Python needs to account for both of these root
In most programs, one or a few of the classes were loaded classes. Python 3 abolishes old-style classes but has failed to
several times, but only in 9 of them, at least one reloaded achieve the popularity of Python 2.6/7, possibly because of
class had more than one set of defined methods (shown in its several backwards incompatibilities.
Col. “Int. diff. in Table 2). Out of these, only 4 had more In our program traces, 22% of all classes were old style
than 1 redefined class with more than one set of methods. classes. The programs were all but five initiated after 2001,
Scipy had 10 classes with multiple interfaces, SciKitLearn the year of the release of Python 2.2 which introduced the
and Mnemosyne had 4 each and TaskCoach had 2. new-style classes as a parallel hierarchy. As shown in Fig-
The dynamism of Python classes usually does not change ure 5, there seems to be no correlation between the program’s
the interfaces of classes, but sometimes classes change during age and the percentage of old-style classes in the program.
Figure 2. For all programs the number of classes for which the class definition has been loaded more than once. Programs
sorted on size in LOC.
Figure 3. For all programs the average shares (in %) of the clusters that were single call and monomorphic.
Figure 4. Call-sites/receiver types.
Figure 5. The percentage of traced classes that were “old style”. Programs sorted on age with the oldest to the left and the
youngest to the right.
For the programs started before 2001, this likely means that
many old style classes have been changed into new style Single call 50.6%
equivalents (the use of old style classes has been strongly dis- Monomorphic 45.4%
couraged). Many of the old-style classes were imported from Polymorphic 4%
libraries, both standard libraries and third party libraries.
A pattern to reduce the amount of old-style classes found Figure 6. Distribution of call- sites between polymorphic, single call and
in several programs in our corpus is the insertion of an monomorphic in whole programs.
explicit derivation from object in addition to its old style
superclasses, which increases the use of multiple inheritance. ods inherited from some superclass. This suggests that our
We conclude that the use of old-style and new-style Python programs are quite object-oriented, and use its object-
classes in parallel means that a type system for Python has oriented concepts similar to statically typed languages like
two choices: it either must account for two root classes, or it Java.
must exclude (support for) old libraries and require changes
to commonly more than a fifth of all classes. Individual Call-Site Polymorphism To give a high-level
overview of the polymorphism of a program, we classify call-
Use of Multiple Inheritance All programs use multiple in- sites depending its measured degree of polymorphism. A call-
heritance, ranging from 2.4% to 17.5% of all classes with an site is either monomorphic, polymorphic or megamorphic. A
average of 5.9% (see Col. “Multiple Inheritance” in Table 2). fourth category, single call, was added to avoid classifying
These are the figures after removing any multiple inheritance call-sites observed only once as monomorphic.
due to the pattern for making old-style classes into new-style For all program runs, the share of monomorphic call-sites
classes mentioned above in Section § 5.1. Multiple inherit- (including single call) ranged between 88–99% with an av-
ance is found both in library classes and program-specific erage of 96% (see Figure 6). This means that in most pro-
classes. Classes used as superclasses in multiple inheritance grams only a very small share of the call-sites exhibits any
are also both library classes and program-specific classes. receiver-polymorphic behaviour at all. To avoid wrongful
5.2 Extent and Degree of Polymorphism classifications due to bad input or non-representative runs,
all programs were run multiple times. The amount of mono-
Overridding In our analysis to decide if a call-site is N- morphic and single call call-sites did not vary significantly
typeable or NPP-typeable, (see Def. 5, Def. 6) we first look between different runs of the same program, including uses
for a common super type for all receiver types. If such a of the same library by different programs, as shown by the
type is found, the second step is to check if the method error bars in Figure 8.
called at the call-site can be found in that type. Thus, to be Single call call-sites accounted for 27–81% of the total
be N-typeable or NPP-typeable, the program needs method number of call-sites for all runs of all programs with an
overriding. Such overriding is at times required in statically average of 51% and a median at 49%.
typed code leading to the insertion of abstract methods to be Monomorphic call-sites are always typeable since all
allowed to call methods on a polymorphic type1 . Since there receivers have the same run-time type. Single call call-sites
is no such need in dynamically typed programs, this analysis are typeable for the same reason, at least for that run of the
is in this respect a conservative approximation. program. Many call-sites would still be single call even if
If the method has been overridden in all subclasses, execu- input was increased/made more complex, etc.
tion of the call-site will lead to execution of different methods The table below shows the degree of monomorphism,
with potentially different behaviour for every receiver type. A polymorphism and megamorphism for all the programs sor-
program designed in this way is arguably more polymorphic ted by increasing size (in terms of lines of code). There seems
than if all executions of the call-site leads to a call to the same to be no correlation between program size and the ration
method in the superclass. On the downside, method overrid- of monomorphism, polymorphism and megamorphism. The
ing makes code harder to read, understand and debug due to polymorphism for the smaller programs (numbers 1–18) is
the increased complexity of the control flow. similar to the polymorphism in the larger programs (num-
Column 6 (“Avg. # overr.”) in Table 2 shows the aver- bers 19–36). We perform a t-test (two-tailed, independent,
age number of overridden methods per program, that is the equal sample sizes, unequal variance) with null hypothesis
number of methods that are redefinition of inherited methods. that the average degree of polymorphism is the same in the
Comparing with column 5 in the same table (“Avg. # meth.) small programs and in the large programs. Column 5 con-
we can see that 19% of all methods are re-definitions of meth- tains the result, confirming the hypothesis for all degrees of
1 In a statically typed language, classes B and a class C both with a method polymorphism. All values are lower than (α=0.05,d.f.=17) =
m() with a common supertype A, the supertype could be used as a static 2.110.
type for objects of B and C, but we could not make calls to m() through a
Figure 7 shows the maximal polymorphic degree for all
variable declared as A unless A also contains a definition of m(). This way,
overriding is necessary in statically typed languages in a way that it is not runs of all programs, ranging from 2 to 356 receiver types.
in a dynamic language. The average maximal polymorphic degree in the programs in
Figure 7. For all programs, the polymorphism max values.
Figure 8. For all programs the average shares (in %) of the call-sites that were single call and monomorphic. Error bars shows
the distance between max and min values. Sorted on size in LOC.
our corpus was 75 and the median 27. The blue dotted line Table 4. A list of the programs, sorted on size (see Table 2), followed
marks the border between polymorphism and megamorphism by 7 columns showing the percent of the total amount of call-sites that were
at 5 receiver types. Only 3 programs contain no megamorphic single-call (S-C), monomorphic (Mono), or polymorphic to different degrees
up to megamorphic (types >5). Finally, in column 8, also the percent of the
call-sites at all (Ntm 2, Comix 4 and RedNotebook 5).
megamorphic call-sites for every program that was found in library code.
7 of 36 programs had at least one call-site with a very high
Call-sites with N receiver types
number of receiver types—close to or above 10 times the Program name S-C Mono 2 3 4 5 >5 %Lib.
average maximum. The maximal degree of polymorphism in 1. PDF-Shuffler 0.6.0 38% 59% 2% 0 <1% <1% 2% 100
2. PyTruss 80% 19% 1% <1% <1% <1% 1% 100
these programs (PyTruss 355, Torrentsearch 279, Pychecker 3. Radiotray 0.6 56% 43% 1% <1% <1% 0 <1% 100
355, Fretsonfire 356, Youtube-dl 321, TaskCoach 305 and 4. GImageReader 0.9 51% 46% 2% 1% <1% <1% <1% 100
5. Ntm 1.3.1 56% 43% 1% 0 0 0 0 -
SciPy 253 respectively) was much higher than in the other 6. Torrent Search 0.11-2 27% 61% 7% 4% 1% <1% 1% 43
programs. There seems to be no correlation between program 7. Brain Workshop 4.8.1 68% 31% 1% <1% <1% 0 <1% 100
8. BleachBit 0.8.0 46% 50% 3% <1% <1% <1% <1% 0
size and the programs with high degrees of polymorphism. 9. Diffuse 0.4.3 38% 61% 1% <1% 0 <1% <1% 0
The programs that contained the highest polymorphism are 10. PhotoFilmStrip 1.5.0 53% 44% 3% <1% <1% <1% <1% 100
11. Comix 4.0.4 48% 49% 3% <1% <1% 0 0 -
distributed evenly over Table 1 which is sorted on program 12. Python megawidgets 49% 48% 2% <1% <1% <1% <1% 99/-
size, although the concentration is somewhat higher at the 13. Requests 2.2.1 58% 39% 3% <1% <1% <1% <1% 1
14. Virtaal 0.6.1 47% 50% 2% <1% <1% <1% <1% 95
bottom of the table (larger programs). The programs with 15. Pychecker 0.8.18-7 56% 42% 2% <1% <1% <1% <1% 100
highest maximum polymorphism are number 17, 15, 2, 27, 16. Idle 2.6.6-8 37% 57% 5% 1% <1% <1% <1% 100
17. Frets on fire 1.3.110 59% 39% 1% <1% <1% <1% 1% 89
34, 6 and 35 (descending). 18. PyPe 2.9.4 54% 43% 2% <1% <1% <1% <1% 58
Column 2 of Table 1, “Whole – P+M %”, shows the pro- 19. PyX 0.10-2 53% 43% 3% <1% <1% <1% <1% 0
20. Python parsing 1.5.2-2 42% 52% 2% 1% <1% <1% 2% 0
portions of the call-sites that were polymorphic and mega- 21. RedNotebook 1.0.0 50% 49% 1% <1% <1% <1% 0 -
morphic for each program (program averages). There is no 22. Link checker 5.2 48% 45% 4% 1% <1% <1% <1% 28
23. Solfege 3.16.4-2 39% 58% 2% <1% <1% <1% <1% 6
strong correlation between the size of the program and the 24. Childsplay 1.3 45% 51% 3% 1% <1% <1% <1% 19
degree of polymorphism. The average of the upper half of 25. Scikit Learn 0.8.1 54% 43% 2% <1% <1% <1% <1% 1
26. Mnemosyne 2.1 57% 40% 2% <1% <1% <1% <1% 84
the table is 3.1%, the average for the lower part of the table 27. Youtube-dl 2013.01.02 76% 22% 1% <1% <1% 0 <1% 69
is 4.0% and the average for the whole is 3.5%. Which means 28. Docutils 0.7-2 43% 49% 5% 1% <1% <1% 1% 1
29. PyMol 1.2r2-1.1+b1 40% 50% 7% <1% 2% <1% <1% 100
that the larger programs contain more polymorphism but the 30. Timeline 1.1.0 47% 51% 1% <1% <1% <1% <1% 100
difference is only 25.4%. Both the programs with the highest 31. DispcalGUI 1.2.7.0 47% 50% 2% <1% <1% <1% <1% 50
32. PySolFC 2.0 56% 40% 2% 1% <1% 1% 1% 26
share of polymorphic and megamorphic call-sites (Torrent- 33. WikidPad 2.1-01 45% 51% 3% <1% <1% <1% <1% 10
search, 12%) and the program with the lowest share (Brain- 34. Task Coach 1.3.22 42% 51% 4% 1% <1% <1% 1% 10
35. SciPy 0.7.2+dfsg1-1 44% 49% 5% 1% <1% <1% 1% 54
workshop, 1%) are small programs. They both have less than 36. Eric4 4.5.12 62% 36% 2% <1% <1% <1% <1% 0
5K lines of code, which is well below both the average and Averages 50% 46% 2.6% <1% <1% <1% <1% 52
the median sizes.
Cluster Polymorphism We apply the same classification
from 46% in Pychecker to 77% in Youtube-dl. The single call
for individual call-sites to clusters. This reduces the size
is lower for the cluster analysis and the monomorphic higher,
of the single call category, as call-sites involving the same
compared to the call-site analysis. The overall result is that
receiver will be placed in a single cluster. The size of the
the monomorphic share of is slightly lower for clusters than
category is still large, which could suggest that it is common
for call-sites, on average 95.2% (0.8%).
to create objects and operate on them only once.
Most clusters are small, 59% contained only 1 call-site2
On average, 35% of all clusters are single call, ranging
(which may have been observed multiple times with possibly
from 20% in Youtube-dl to 58% in Pytruss as shown in Fig-
different receiver types). The largest cluster had 2.720 call-
ure 3. The monomorphic clusters, shown in the same figure,
sites. The average cluster size was 5.
were on average 61% of all clusters for the programs, ranging
Degree of Polymorphism at Individual Call-sites The de-
gree of polymorphism at a call-site is the number of different
Table 3. Polymorphism of small/large programs in Table 2 receiver types we observed at that call-site.
Student’s t-test Figure 4 shows the degree of polymorphism for all poly-
(α=0.05,d.f.=17)
Prog. 1–18 Prog. 19–36 All = 2.110 morphic and megamorphic call-sites. In Figure 7 and in Fig-
Single call 49.7 50.4 50.1 -0.06 ure 4, the border between polymorphism and megamorphism
Monom. 46.9 45.3 46.1 0.02 is represented by the dotted line. The vast majority, 88%, of
Polym. (2) 2.3 2.9 2.6 0.02 all polymorphic and megamorphic call-sites are not mega-
Polym. (3) 0.52 0.45 0.47 0.001 morphic (69,367 polymorphic against 7,870 megamorphic).
Polym. (4) 0.17 0.27 0.23 0.005
Polym. (5) 0.10 0.10 0.14 0.003
Megam. 0.34 0.30 0.38 0.01 2 Thismeans that in a source file there was just one single place that
manipulated (a) certain value(s).
Figure 9. Polymorphic degree of the clusters of polymorphic and megamorphic clusters.
Figure 10. N-Typeable call-sites.
Figure 11. N-Typeable, single call and monomorphic call-sites.
Figure 12. The % of all clusters that were S-Typeable.
78% of the polymorphic call-sites had a polymorphic degree megamorphic. 42% of the polymorphic call-sites had a poly-
of 2, that is two different receiver types. morphic degree of 2 (i.e., two different receiver types).
While these numbers show that megamorphic call-sites
are relatively rare, they are not concentrated to specific pro- Polymorphism in Library Code vs. Program-specific The
grams. Almost all programs (33 of 36) exhibited some form columns under “Lib.” and “Prog.” in Table 1 shows the
of megamorphic behaviour, see Table 4, Column 9, “Call- share of all call-sites from library code and program specific
sites with N receiver types >5”. The programs without mega- code that were polymorphic or megamorphic. Assuming that
morphic call-sites were Ntm, Mcomix and Rednotebook. In polymorphism on average does not differ between library
83% of all programs (30 out of 36), 1% or less of all call- code and program-specific code we ran a statistical test (a
sites were megamorphic. The largest share of megamorphic Student’s t-test, two-tailed, independent, equal sample sizes,
call-sites, 2%, were seen in PdfShuffler and Python parsing. unequal variance) comparing all data all 28 programs where
the separation of libraries and program-specific code was
Manual Inspection To better understand their nature, we made. The result was that the hypothesis holds for all of the
investigated the receiver types of the extremely megamorphic programs. For α=0.05, and a degree of freedom that ranges
call-sites for the five programs with the highest mega- from 1 to 56, and a p-value ranging from 1.98 to 12.706, the
morphic maximum value (Pytruss, Torrentsearch, Frets on t-values ranged from -0.44 to 0.12.
fire, Youtube-dl and Scipy) To uncover differences between megamorphism in lib-
In two of these programs (Pytruss, Frets on fire), the same raries and program code, we manually inspected all mega-
OpenGL library was the main cause of megamorphism and morphic call-sites of all programs to see if they were found
all call-sites of degree >10 (Pytruss) or >50 (Frets on fire) in libraries or in program-specific code. No clear pattern
originated from calls on OpenGL objects. Frets on fire also emerged and, on average, 59% of the megamorphic call-sites
had some very program-specific receivers in megamorphic originated from library code. As shown in the last column
call-sites such as songs, menus, etc., related to the game. of table Table 1, the share varied from 0 to 100%. For 10 of
For Torrentsearch and Youtube-dl, the megamorphism the programs (28%) all megamorphism stemmed from lib-
stemmed from the singleton class implementation of the rary code. Only 5 programs had none of their megamorphic
representation of different torrent sites or sites from which call-sites in the library code (14%).
content could be downloaded. For Youtube-dl, the mega-
morphism also varied a lot between runs. Polymorphism at Start-up vs. Runtime Using the marker
The very high megamorphic values in SciPy with degrees we inserted in all programs to separate the programs’ start-
>100 were all caused by testing frameworks (the built-in up time from the actual runtime, we separated the trace data
unittest or the nose unit test extension framework). All gathered during start-up from that gathered during “normal
call-sites with a megamorphic degree <100 and >22 were program execution”. This was only done for interactive pro-
either part of the testing frameworks or used to create distri- grams (24/36) as it was relatively easy to identify the end of
butions using generator classes. The call-sites with a mega- the start-up for those programs as the time control is handed
morphic degree <23 and >5 often used arguments of differ- over to the main event loop waiting for user input. Remain-
ent classes to handle different shapes or components used to ing programs are marked with a “–” in the columns under
create plots. “Startup” and “Runtime” in Table 1. Assuming first that poly-
From the manual inspection, it seemed that to a large ex- morphism does not differ between start-up and runtime we
tent, megamorphism is due to patterns emerging from con- ran a statistical test comparing all runtime data to all start-up
venience (the simplicity to create specific, singleton classes data for all 24 programs where the separation of runtime and
at run-time in Python), and not from an actual need to cre- start-up data was done (a Student’s t-test, two-tailed, inde-
ate widely different unrelated classes. Thus, in many cases, pendent, equal sample sizes, unequal variance). This test, and
it is possible to reduce megamorphism by redesigning how the assumption, fails for all but one of the tested programs.
classes are used. Nevertheless, the proliferation of mega- Since the t-values in all cases except one (23/24) are negative
morphism (by convenience or not) must be considered by we conclude that the average polymorphism during startup is
retrofitted type systems for Python. lower than the average polymorphism during runtime.
Notably, all program traces contain a lower degree of poly-
Degree of Polymorphism in Clusters The degree of poly- morphic and megamorphic call-sites during start-up com-
morphism in a cluster is the number of receiver types we pared the whole program run. This is an interesting find given
observed at that call-site. that RPython [4] is based on the idea that programs are more
Figure 9, shows the degree of polymorphism for all poly- dynamic during start-up, limiting the use of the more dy-
morphic and megamorphic clusters. The border between namic features of Python to an initial, bootstrapping phase.
polymorphism and megamorphism is represented by the About 1% of the call-sites seen at start-up were polymorphic
dotted line. Similar to the call-sites, the majority, 67%, or megamorphic. During normal program execution, on aver-
of all polymorphic and megamorphic clusters are non- age, 5% of all call-sites were polymorphic or megamorphic.
Figure 13. NPP-Typeable call-sites.
Figure 14. NPP-Typeable, single call and monomorphic call-sites.
Figure 15. The % of all clusters that were N-Typeable.
5.3 Typeability between 88.9% and 100% of the call-sites, with an average
We applied our three metrics for approximating typeability at 97.8%.
using nominal, nominal and parametrically polymorphic, and By extending the nominal type system with parametric
structural typing to the call-sites and clusters in our trace logs. polymorphism, we can type more call-sites for all programs.
For one program, Frets on Fire, we could even type all
N-Typeable Call-sites Figure 10 shows the percentage call-sites. But for the rest of the programs, our simple and
of all call-sites that were polymorphic or megamorphic conservative nominal types are not powerful enough even
and N-typeable. In Figure 10 as well as in column 4 when extended with parametric polymorphism.
(“Whole program”/“N-t”) of Table 1, all programs contain N-Typeable Clusters The figures reported in this section
call-sites that are N-typeable, although the N-typeable share to this point are optimistic as they only consider individual
of the call-sites is always low. In column 3 (“Whole pro- call-sites. We apply the same analysis to clusters of call-sites
gram”/“Typeable Poly”) of Table 1 we see the N-typeable as discussed in § 4.
shares of the non-monomorphic parts of the programs. The For all the polymorphic and megamorphic clusters, we
dashed line in Figure 10 marks the average value at 0.96%. search for a common supertype among the receiver types
The program with the highest amount of N-typeable call- that contains all methods called in the call-sites of the cluster.
sites was Pyparsing with 4.11%, which also had the highest If the methods were found, the cluster is N-typeable (see
N-typeability share, 72.0%, when considering only non- Def. 8).
monomorphic call-sites. Figure 15, shows the results of applying our N-typeability
The average of the upper half of Table 1 (the smaller analysis on all polymorphic or megamorphic clusters. The
programs), was 0.9%. The programs in the lower half of the staples represent the % of all clusters that were N-typeable
same table (the larger programs) had a slightly higher average and the dashed blue line marks the average at 0.4%. The
for the N-typeability (1.1%). highest typeability, 1.1% was found in Torrentsearch, and the
Figure 11 shows the amount of N-typeable call-sites on lowest, 0% in both Mnemosyne and Comix. This result is, as
top of the shares of monomorphic (which are always type- expected, lower compared to the N-typeability analysis we
able) and the single call (which are typeable for this run of the made for call-sites.
program). In combination, they can be used to type between When we combine the N-typeability with the single call
88.1% and 99.9% of the call-sites, with an average at 97.4%. and monomorphic clusters, as shown in Figure 16, between
In conclusion, most call-sites in Python programs are not 91.9% and 97.8% of the call-sites are typeable in the 36
polymorphic or megamorphic, but when they are, our simple programs, with an average at 95.6%. This result is lower than
and conservative nominal types cannot in general be used to the typeability we reached for call-sites, as expected.
type them. In conclusion, clusters in the programs of our corpus are
predominately monomorphic or single call. When a cluster
NPP-typeable Call-sites All call-sites that are N-typeable
is polymorphic or megamorphic, nominal types cannot in
(see Def. 5) are also NPP-typeable (see Def. 6), but the
general be used to type them.
NPP-typeability analysis increases our possibilities to find
typeable call-sites. S-Typeable Clusters The shares of program clusters that
All call-sites in the programs, that were not N-typeable, were S-typeable (Def. 9) are shown in Figure 12. On average,
were sorted and separated on the identity of the caller, that is 1.6% of all the clusters are S-typeable, with a minimum at
the identity of self at the time when the call was made. After 0.4% in Pype and a maximum at 3.7% in Diffuse.
this separation, we again search for a common supertype for As with nominal typing, we can combine the S-typeable
all receiver types and if found check if that supertype contains clusters with single call and monomorphic clusters to find
the method called at the call-site. out how large parts of the programs we could type in total.
Figure 13, shows for each program the percentage of all Figure 17 show the results. Combined, these three typeable
call-sites that were polymorphic or megamorphic and NPP- shares give a typeability of 96.7%, on average. Lowest in
typeable. All programs contain call-sites that are N-typeable, BleachBit with 94.8% and highest in PyChecker with 98.4%.
and we for all programs find that they are NPP-typeable to Unsurprisingly S-typeability analysis for clusters gives a
a higher extent than they are N-typeable. The dashed line in higher overall typeability (12.0% higher) than achieved with
Figure 13 marks the average value at 1.34%. As was the case N-typeability, but no program can be typed to more than
for our N-typeable analysis, the program with the highest 98.4%. Again, monomorphism dominates the programs. The
amount call-sites was Pyparsing also for the NPP-typeability small parts that are polymorphic and megamorphic cannot be
analysis, with 4.31%. typed entirely using a structural approach.
Figure 14 shows the amount of NPP-typeable call-sites
on top of the shares of monomorphic (which are always 6. Threats to Validity
typeable) and the single call (which are typeable for this run Validity of our findings is affected by several decisions and
of the program). In combination, they can be used to type choices. The program selection was not made entirely at ran-
Figure 16. N-Typeable, single call and monomorphic clusters.
Figure 17. S-Typeable, single call and monomorphic clusters.
dom, which might lead to that the programs are not repres- do not generally contain much use of dynamic features and
entative for Python programs in general. Generating repres- if use of dynamic features can be found it will be easy to
entative runs for a particular program requires representative rewrite in a more static style, and second that if use of dy-
input. Many programs are very large, and since we do not namic features is found it will be found during start-up. The
have coverage measurements, we do not know to what extent first assumption was found to be false and the second to be
the source code of the programs was executed. Especially true. Their study was trace-based and operated at byte-code
program parts that are used less frequently e.g., parts of pro- level, whereas our approach has been to modify the inter-
grams checking for updates and installing updates have prob- preter to produce log files in plain text format. In comparison
ably not been run. An approximate coverage measurement with the work done by Holkner and Harland our approach
would be to compare the number of call-sites in the code has no noticeable impact noticeable impact on the programs’
with the number of call-sites visited by a trace, but it would performance and the logs produced are manageable in size.
be very rough so we have not included it. Both performance and log file sizes were problematic for
Our typeability analyses are based on receiver types at call- Holkner’s & Harland’s study. Their goal is similar to what
sites and clusters. Individual call-sites is very fine-grained we are aiming to achieve with this study, but do they not
and clusters may be too coarse-grained. The individual call- consider method polymorphism.
site analysis will likely over-estimate typeability since it does Another study with a similar goal has been made for
not consider connected call-sites. Similarly, cluster analysis Smalltalk where Callaú et al [8] first made a static analysis
will under-estimate typeability by connecting call-sites too of 1.000 Smalltalk programs to see which dynamic features
greedily (e.g., due to value-based overloading) forcing them were used, how frequent the use is and where in the programs
to be typed together. We believe that the two approached the use could be found. They then studied code to understand
function as upper and lower bound for (our definitions of) why the features were used. Their results were that dynamic
typeability. features are used sparsely although not seldom enough to be
Static typing is difficult in the presence of powerful reflect- disregarded, that the use of dynamic features is more com-
ive support for non-monotonic changes to classes at run-time. mon in some application areas, that the dynamic Smalltalk
Although we found no use of this in our corpus, Python, for features that have been included also in statically typed lan-
example, allows assigning the __class__ variable of an ob- guages like Java are the most popular features, and that use of
ject, thereby changing its class. Prior work by ourselves [2] dynamic features to some extent can be replaced with more
and others [19] investigate the actual usage of such mechan- statically checkable code. The studies of code revealed that
isms in Python and conclude that, although not very common, the majority of the use of dynamic features was benefiting
programs actually contain code of this kind. Our typeability from their dynamic nature and would be impossible to re-
analyses do not consider this, leaving this for future work. place with more static code, but some use of dynamic features
In this paper we set out to understand how structural and were really a sign of limitations in the language design that
nominal types can be used to type untyped Python code with programmers solved by using unnecessarily dynamic solu-
respect to polymorphism. Clearly, an actual implementation tions. These cases could be rewritten without dynamic fea-
of static typing must consider use of reflection. We leave the tures but the code would get more complex. Yet other uses of
decision of what strategy to employ (run-time detection, con- dynamic features could be replaced by less dynamic code. In
straining of Python’s reflective protocol, etc.) to the designers this study, most of the programs were only studied statically
of such systems. and polymorphism was never considered.
Lebresne et al. [24] and Richards et al. [27] have done a
similar study for JavaScript where the interaction with 103
7. Related Work
web sites was traced and three common benchmark suites
The idea to study real programs to understand how languages analysed. Common assumptions about JavaScript programs
are used and then use the knowledge for designing better are listed and the goal of the paper is to find support for or
languages and tools is not a new one. In 1971, Knuth studied invalidate these assumptions. Results from the analysis show
how Fortran programs were written to make new guidelines that the programs use dynamic features and that the divi-
for compiler designers [22]. sion into an initialisation phase and a division of program
Many efforts have later been presented, for several dif- runs into different phases is less applicable for JavaScript,
ferent dynamic languages, to increase our understanding for since e.g., objects are observed to have properties added or re-
how dynamic languages are used in practice. We have seen moved long after initialisation. One of the assumptions used
the study of use of dynamic features by Holkner and Harland as a starting point for the study was that call-site polymorph-
[19], where they study the use of e.g., reflection, changes ism would be low. Their result was that 81% of all call-sites
made to objects at runtime (adding/removing attributes, etc.), were monomorphic, which is less than the 96% we have ob-
variables that are used to point out objects of different types, served for Python programs (see Table 1). Our study was also
and dynamic code evaluation in 24 Python programs. This
work is based on two assumptions; first that Python programs
similar to theirs in that they also examined the receivers of Recently, the work on type inference for Python has been
the call-sites. dominated by the PyPy initiative, originally aiming to imple-
Method polymorphism in particular has been studied in ment Python in Python. PyPy uses type inference in RPython,
the context of inline caching and polymorphic inline caching the restricted version of the language that is used to imple-
in Smalltalk [12] and Self [20]. Polymorphic inline caching ment the interpreter [4].
in Smalltalk has a reported 95% hit frequency with a size Types inferred for object-oriented languages are often
one cache [12], suggesting that Smalltalk call-sites are either nominal [4, 9, 26] but there are other solutions, like Strongtalk
relatively monomorphic, or that call-sites are executed often [6] that infers structural types and DRuby where class (nom-
between changes to receiver types. (However unlikely, a inal) types are combined with object (structural) types.
Smalltalk program may have a 95% hit frequency while still
having 100% megamorphic call-sites.) Our Python-specific
result is similar, and has a stronger bearing on the typeability 8. Conclusions
of whole programs as we consider polymorphism in clusters Our results show that while Python’s dynamic typing allows
of call-sites. In Hölzle’s et al. work on Self [20], the number unbounded polymorphism, Python programs are predomin-
of receiver types in a call-site are usually lower than 10. In antly monomorphic, that is, variables only hold values of a
our study, 88% of all call-sites have 5 or fewer receiver types. single type. This is true for program start-up and normal
Other related work include initiatives to build type or type runtime, in library code and in program-specific code.
inference systems for dynamic languages starting in func- Nevertheless, most programs have a few places which
tional languages based on the work of Hindley, Milner and are megamorphic, meaning that variables in those places
Damas [13]. The use of inferred types has been successfully contain values of many different types at different times or in
implemented in languages like ML and Haskell. different contexts. Smaller programs do not generally differ
Type inference for object-oriented languages, on the from larger programs in this.
other hand, turned out to be more complex and computa- Because of the high degree of monomorphism, most pro-
tionally demanding as in Suzuki’s case where the infer- grams can be typed to a large extent using a very simple type
ence algorithm [34] failed because of the Smalltalk envir- systems. Our findings show that the receiver in 97.4% of all
onment’s restriction on the number of live objects. His work call-sites in the average program can be described by a single
was followed up more successfully by others both in Small- static type using a conservative nominal type system using
talk [6, 16, 26] and other object-oriented languages. Type single inheritance. If we add parametric polymorphism to
inference for Smalltalk was mainly motivated by increased the type system, we increase the typeability to 97.9% of all
reliability although readability and performance often also call-sites for the average program.
are mentioned as other expected improvements. For clusters, the receiver objects are typeable using a
Following Smalltalk, Diamondback Ruby [15] focuses conservative nominal type system using single inheritance
on finding and isolating errors by combining inferred static to 95.6% (on average). If we instead use a structural type, the
types with type annotations made by the programmer where typeability increases somewhat to 96.7% (on average).
annotated code is excluded from type inference and its types Most polymorphic and megamorphic parts of programs
will be checked at runtime. The type system was tested on a are not typeable by nominal or structural systems, for ex-
set of benchmark programs of 29–1,030 LOC and proved to ample due to use of value-based overloading. Structural typ-
be useful for finding errors. ing is only slightly better than nominal typing at handling
Type inference systems implemented for Python have non-monomorphic program parts. This suggests that nominal
often focused on improving performance rather than program and structural typing is not a deciding factor in type system
quality aspects as reliability or readability. design if typing polymorphic code is desirable. More power-
Aycock’s aggressive type inference [5] was designed as ful constructs are needed in these cases, such as refinement
a first step towards translating Python code to Perl. The types. We will investigate this in future research.
aggressiveness is expressed in that the program has to adhere
to the restriction rules for how Python programs may be
written for the type inference to work. e.g., runtime code
5. REFERENCES
generation is not allowed, and types for local variables must [1] O. Agesen, “The Cartesian Product Algorithm: Simple and
not change during execution. Precise Type Inference Of Parametric Polymorphism”, Proc.
Following this, also targeting performance but without re- ECOOP’95, pp 2–26, 1995.
strictions for the language, Cannon first [9] thoroughly dis- [2] B. Åkerblom, J. Stendahl, M. Tumlin and T. Wrigstad,
cusses difficulties met when implementing type inference “Tracing Dynamic Features in Python Programs”, Proc.
for Python and then presents a system for inferring types in MSR’14, 2014.
a local name-space. Tests show that performance improve- [3] J.D. An, A. Chaudhuri, J.S. Foster, and M. Hicks, “Dynamic
ments were around 1%. inference of static types for Ruby”, In POPL’11, pp 459–472,
2011.
 [4] D. Ancona, M. Ancona, A. Cuni, and N. Matsakis. “RPython: [25] J. McCauley. “About POX”. 2013.
Reconciling Dynamically and Statically Typed OO http://www.noxrepo.org/pox/about-pox/
Languages”, In DLS’07, 2007. [26] J. Palsberg and M. I. Schwartzbach, “Object-Oriented Type
[5] J. Aycock, “Aggressive Type Inference”, pp 11–20, Proc. of Inference”, In OOPSLA’91, pp. 146–161, 1991.
the 8th International Python Conference, 2000. [27] G. Richards, S. Lebresne, B. Burg and J. Vitek, “An Analysis
[6] G. Bracha and D. Griswold, “Strongtalk: Typechecking of the Dynamic Behavior of JavaScript Programs” , In
Smalltalk in a Production Environment”, In Proc. PLDI’10, 2010.
OOPSLA’93, pp. 215–230, 1993. [28] G. van Rossum and F.L. Drake, “PYTHON 2.6 Reference
[7] F. Brito e Abreu, “The MOOD Metrics Set,” Proc. ECOOP’95 Manual”, CreateSpace, Paramount, CA, 2009.
Workshop on Metrics, 1995. [29] M. Salib, “Faster than C: Static type inference with
[8] O. Callaú, R. Robbes, É. Tanter, and D. Röthlisberger, “How Starkiller”, In PyCon Proceedings, Washington DC, 2004.
Developers Use the Dynamic Features of Programming [30] Securities and Exchange Commission. Release Nos. 33-9117;
Languages: The Case of Smalltalk”, In MSR’11, 2011. 34-61858; File No. S7-08-10. 2010.
[9] B. Cannon, “Localized Type Inference of Atomic Types in http://www.sec.gov/rules/proposed/2010/33-9117.pdf
Python”, Master Thesis, California Polytechnic State [31] J. Siek, and W. Taha, “Gradual Typing for Objects”, in Proc.
University, San Luis Obispo, 2005. ECOOP’07, Berlin, Germany, July 2007.
[10] L. Cardelli, and P. Wegner, “On understanding types, data [32] SourceForge, http://sourceforge.net/.
abstraction, and polymorphism”, ACM Computing Surveys
[33] C. Strachey, “Fundamental Concepts in Programming
Volume 17, pp 471-222, 1985.
Languages”, in Higher Order Symbol. Comput., pp. 11–49,
[11] R. Chugh, D. Herman, and R. Jhala, “Dependent Types for 2000.
JavaScript”, In SIGPLAN Not., Vol 47, No. 10 Oct 2012.
[34] N. Suzuki, “Inferring types in smalltalk”, in Proceedings
[12] L. P. Deutsch and A. M. Schiffman, “Efficient POPL’81, pp. 187–199, New York, USA, 1981.
Implementation of the Smalltalk-80 System”, In POPL 1984.
[35] P. Thiemann, “Towards a type system for analyzing JavaScript
[13] L. Damas and R. Milner, “Principal Type-schemes for programs”, in Proceedings of ESOP’05, pp. 408–422, 2005.
Functional Programs”, In POPL’82, pp. 207–212, 1982.
[36] S. Tobin-Hochstadt and M. Felleisen, “Interlanguage
[14] Facebook, Inc., “Specification for Hack”, Facebook, Inc., migration: from scripts to programs”, in DLS’06, 2006.
https://github.com/hhvm/hack-langspec, 2015. [37] L. Tratt, “Dynamically Typed Languages”, Advances in
[15] M. Furr, J.D. An, J.S. Foster, and M. Hicks, “Static type Computers 77. Vol, pp. 149-184, ed. Marvin V. Zelkowitz,
inference for Ruby”, In the 2009 ACM Symposium on 2009.
Applied Computing, 2009.
[16] J. O. Graver and R. E. Johnson, “A Type System for
Smalltalk”, In Proc. POPL’90, pp. 136–150, 1990.
[17] B. Hackett and S. Guo, “Fast and Precise Hybrid Type
Inference for JavaScript”, In PLDI’12, pp. 239–250, 2012.
[18] P. Heidegger and P. Thiemann, “Recency types for analyzing
scripting languages”, In ECOOP’10, pp. 200–224, 2010.
[19] A. Holkner and J. Harland, “Evaluating the Dynamic
Behaviour of Python Applications”, In ACSC’09, pp. 19-28,
2009.
[20] U. Hölzle and C. Chambers and D. Ungar, “Optimizing
Dynamically-Typed Object-Oriented Languages With
Polymorphic Inline Caches”, In ECOOP ’91, LNCS 512, July,
1991.
[21] S. Karabuk and F.H. Grant, “A common medium for
programming operations-research models”. IEEE Software,
24(5):39-47, 2007.
[22] D. E. Knuth: “An Empirical Study of FORTRAN Programs.
Software”, Practice and Experience, 1(2): 105-133, 1971.
[23] A. Lamaison, “Inferring Useful Static Types for Duck Typed
Languages”, Ph.D. Thesis, Imperial College, London, U.K.,
2012.
[24] S.Lebresne, G.Richards, J.Östlund, T.Wrigstad, J.Vitek,
“Understanding the dynamics of JavaScript”, In Script to
Program Evolution, 2009.