Learning Python Testing

Daniel Arbuckle

Chapter No. 2
"Working with doctest"

In this package, you will find:

The authors biography
A preview chapter from the book, Chapter no.2 "Working with doctest"
A synopsis of the books content
Information on where to buy this book

About the Author

Daniel Arbuckle received his PhD. degree in Computer Science from the University of
Southern California in 2007. He is an active member of the Python community and an
avid unit tester.
I would like to thank Grig, Titus, and my family for their companionship
and encouragement along the way.

For More Information:

www.packtpub.com/application-development/learning-python-testing

Learning Python Testing

In this book, you'll learn about the major tools, techniques, and skills of automated
testing in the Python 3 language. You'll learn about the tools that are included in
Python's standard library, such as doctest, unittest, and unittest.mock. You'll also learn
about useful nonstandard tools such as Nose and coverage.py. As we're talking about
these tools, we'll also be discussing the philosophy and best practices of testing, so when
you're done you'll be ready to use what you've learned in real-world projects.
This book is a successor to an earlier book, Python Testing: Beginner's Guide, Daniel
Arbuckle, Packt Publishing which only covered Python up to version 2.6. Python 3 and
its related tools are just slightly too different to justify calling this book a second edition.
If you've read the earlier book and parts of this book seem familiar to you, it's because the
two books are in fact similar.

What This Book Covers

Chapter 1, Python and Testing, provides an introduction to formalized and automated
testing in Python.
Chapter 2, Working with doctest, teaches you to use doctest, a tool that integrates testing
and documentation.
Chapter 3, Unit Testing with doctest, helps you understand how to apply doctest to the
discipline of unit testing.
Chapter 4, Decoupling Units with unittest.mock, teaches you to create and use
mock objects.
Chapter 5, Structured Testing with unittest, helps you to build more structured test suites
with unittest.
Chapter 6, Running Your Tests with Nose, helps you run your doctests, unittests, and
more with one command.
Chapter 7, Test-driven Development Walk-through, takes you step by step through the
test-driven development process.
Chapter 8, Integration and System Testing, teaches you how to test the interactions
between units of code.
Chapter 9, Other Tools and Techniques, helps you learn about continuous integration,
version control hooks, and other useful things that are related to testing.

For More Information:

www.packtpub.com/application-development/learning-python-testing

Working with doctest

The first testing tool we're going to look at is called doctest. The name is short for
"document testing" or perhaps a "testable document". Either way, it's a literate tool
designed to make it easy to write tests in such a way that computers and humans
both benefit from them. Ideally, doctest tests both, informs human readers, and
tells the computer what to expect.
Mixing tests and documentation helps us:

Keeps the documentation up-to-date with reality

Make sure that the tests express the intended behavior

Reuse some of the efforts involved in the documentation and test creation

Where doctest performs best

The design decisions that went into doctest make it particularly well suited to
writing acceptance tests at the integration and system testing levels. This is because
doctest mixes human-only text with examples that both humans and computers can
read. This structure doesn't support or enforce any of the formalizations of testing,
but it conveys information beautifully and it still provides the computer with the
ability to say that works or that doesn't work. As an added bonus, it is about the easiest
way to write tests you'll ever see.
In other words, a doctest file is a truly excellent program specification that you
can have the computer check against your actual code any time you want. API
documentation also benefits from being written as doctests and checked alongside
your other tests. You can even include doctests in your docstrings.
The basic idea you should be getting from all this is that doctest is ideal for uses
where humans and computers will both benefit from reading them.

For More Information:

www.packtpub.com/application-development/learning-python-testing

Working with doctest

The doctest language

Like program source code, doctest tests are written in plain text. The doctest
module extracts the tests and ignores the rest of the text, which means that the tests
can be embedded in human-readable explanations or discussions. This is the feature
that makes doctest suitable for uses such as program specifications.

Example creating and running a simple

doctest
We are going to create a simple doctest file, to show the fundamentals of using the
tool. Perform the following steps:
1. Open a new text file in your editor, and name it test.txt.
2. Insert the following text into the file:
This is a simple doctest that checks some of Python's arithmetic
operations.
>>> 2 + 2
4
>>> 3 * 3
10

3. We can now run the doctest. At the command prompt, change to the
directory where you saved test.txt. Type the following command:
$ python3 -m doctest test.txt

4. When the test is run, you should see output like this:

[ 12 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Chapter 2

Result three times three does not equal ten

You just wrote a doctest file that describes a couple of arithmetic operations, and
ran it to check whether Python behaved as the tests said it should. You ran the tests
by telling Python to execute doctest on the file containing the tests.
In this case, Python's behavior differed from the tests because, according to the
tests, three times three equals ten. However, Python disagrees on that. As doctest
expected one thing and Python did something different, doctest presented you with
a nice little error report showing where to find the failed test, and how the actual
result differed from the expected result. At the bottom of the report is a summary
showing how many tests failed in each file tested, which is helpful when you have
more than one file containing tests.

The syntax of doctests

You might have already figured it out from looking at the previous example:
doctest recognizes tests by looking for sections of text that look like they've been
copied and pasted from a Python interactive session. Anything that can be expressed
in Python is valid within a doctest.
Lines that start with a >>> prompt are sent to a Python interpreter. Lines that start
with a ... prompt are sent as continuations of the code from the previous line,
allowing you to embed complex block statements into your doctests. Finally, any
lines that don't start with >>> or ..., up to the next blank line or >>> prompt,
represent the output expected from the statement. The output appears as it would in
an interactive Python session, including both the return value and anything printed
to the console. If you don't have any output lines, doctest assumes it to mean that
the statement is expected to have no visible result on the console, which usually
means that it returns None.
The doctest module ignores anything in the file that isn't part of a test, which means
that you can put explanatory text, HTML, line-art diagrams, or whatever else strikes
your fancy in between your tests. We took advantage of this in the previous doctest
to add an explanatory sentence before the test itself.

Example a more complex test

Add the following code to your test.txt file, separated from the existing code by at
least one blank line:
Now we're going to take some more of doctest's syntax for a spin.
>>> import sys
[ 13 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Working with doctest

>>> def test_write():
...
sys.stdout.write("Hello\n")
...
return True
>>> test_write()
Hello
True

Now take a moment to consider before running the test. Will it pass or fail? Should it
pass or fail?

Result five tests run

Just as we discussed before, run the test using the following command:
python3 -m doctest test.txt

You should see a result like this:

Because we added the new tests to the same file containing the tests from before, we
still see the notification that three times three does not equal 10. Now, though, we
also see that five tests were run, which means our new tests ran and were successful.
Why five tests? As far as doctest is concerned, we added the following three tests to
the file:

The first one says that, when we import sys, nothing visible should happen

The second test says that, when we define the test_write function, nothing
visible should happen

The third test says that, when we call the test_write function, Hello and
True should appear on the console, in that order, on separate lines

[ 14 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Chapter 2

Since all three of these tests pass, doctest doesn't bother to say much about them.
All it did was increase the number of tests reported at the bottom from two to five.

Expecting exceptions
That's all well and good for testing that things work as expected, but it is just as
important to make sure that things fail when they're supposed to fail. Put another
way: sometimes your code is supposed to raise an exception, and you need to be
able to write tests that check that behavior as well.
Fortunately, doctest follows nearly the same principle in dealing with exceptions
as it does with everything else; it looks for text that looks like a Python interactive
session. This means it looks for text that looks like a Python exception report and
traceback, and matches it against any exception that gets raised.
The doctest module does handle exceptions a little differently from the way it
handles other things. It doesn't just match the text precisely and report a failure
if it doesn't match. Exception tracebacks tend to contain many details that are
not relevant to the test, but that can change unexpectedly. The doctest module
deals with this by ignoring the traceback entirely: it's only concerned with the first
line, Traceback (most recent call last):, which tells it that you expect an
exception, and the part after the traceback, which tells it which exception you expect.
The doctest module only reports a failure if one of these parts does not match.
This is helpful for a second reason as well: manually figuring out what the traceback
will look like, when you're writing your tests, would require a significant amount of
effort and would gain you nothing. It's better to simply omit them.

Example checking for an exception

This is yet another test that you can add to test.txt, this time testing some code
that ought to raise an exception.
Insert the following text into your doctest file, as always separated by at least one
blank line:
Here we use doctest's exception syntax to check that Python is
correctly enforcing its grammar. The error is a missing ) on the def
line.
>>> def faulty(:
...
yield from [1, 2, 3, 4, 5]
Traceback (most recent call last):
SyntaxError: invalid syntax
[ 15 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Working with doctest

The test is supposed to raise an exception, so it will fail if it doesn't raise the
exception or if it raises the wrong exception. Make sure that you have your mind
wrapped around this: if the test code executes successfully, the test fails, because
it expected an exception.
Run the tests using the following doctest:
python3-mdoctesttest.txt

Result success at failing

The code contains a syntax error, which means this raises a SyntaxError exception,
which in turn means that the example behaves as expected; this signifies that the
test passes.

When dealing with exceptions, it is often desirable to be able to use a wildcard

matching mechanism. The doctest provides this facility through its ellipsis
directive that we'll discuss shortly.

Expecting blank lines

The doctest uses the first blank line after >>> to identify the end of the expected
output, so what do you do when the expected output actually contains a blank line?
The doctest handles this situation by matching a line that contains only the text
<BLANKLINE> in the expected output with a real blank line in the actual output.

[ 16 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Chapter 2

Controlling doctest behavior with

directives
Sometimes, the default behavior of doctest makes writing a particular test
inconvenient. For example, doctest might look at a trivial difference between the
expected and real outputs and wrongly conclude that the test has failed. This is
where doctest directives come to the rescue. Directives are specially formatted
comments that you can place after the source code of a test and that tell doctest
to alter its default behavior in some way.
A directive comment begins with # doctest:, after which comes a comma-separated
list of options that either enable or disable various behaviors. To enable a behavior,
write a + (plus symbol) followed by the behavior name. To disable a behavior, white
a (minus symbol) followed by the behavior name. We'll take a look at the several
directives in the following sections.

Ignoring part of the result

It's fairly common that only part of the output of a test is actually relevant to
determining whether the test passes. By using the +ELLIPSIS directive, you can
make doctest treat the text ... (called an ellipsis) in the expected output as a
wildcard that will match any text in the output.
When you use an ellipsis, doctest will scan until it finds text matching whatever
comes after the ellipsis in the expected output, and continue matching from there.
This can lead to surprising results such as an ellipsis matching against a 0-length
section of the actual output, or against multiple lines. For this reason, it needs to
be used thoughtfully.

Example ellipsis test drive

We're going to use the ellipsis in a few different tests to better get a feel of how it
works. As an added bonus, these tests also show the use of doctest directives.
Add the following code to your test.txt file:
Next up, we're exploring the ellipsis.
>>> sys.modules # doctest: +ELLIPSIS
{...'sys': <module 'sys' (built-in)>...}
>>> 'This is an expression that evaluates to a string'
... # doctest: +ELLIPSIS
[ 17 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Working with doctest

'This is ... a string'
>>> 'This is also a string' # doctest: +ELLIPSIS
'This is ... a string'
>>> import datetime
>>> datetime.datetime.now().isoformat() # doctest: +ELLIPSIS
'...-...-...T...:...:...'

Result ellipsis elides

The tests all pass, where they would all fail without the ellipsis. The first and last
tests, in which we checked for the presence of a specific module in sys.modules and
confirmed a specific formatting while ignoring the contents of a string, demonstrate
the kind of situation where ellipsis is really useful, because it lets you focus on the
part of the output that is meaningful and ignore the rest of the test. The middle tests
demonstrate how different outputs can match the same expected result when ellipsis
is in play.
Look at the last test. Can you imagine any output that wasn't an ISO-formatted time
stamp, but that would match the example anyway? Remember that the ellipsis can
match any amount of text.

Ignoring white space

Sometimes, white space (spaces, tabs, newlines, and their ilk) is more trouble than
it's worth. Maybe you want to be able to break a single line of expected output across
several lines in your test file, or maybe you're testing a system that uses lots of white
space but doesn't convey any useful information with it.
The doctest gives you a way to "normalize" white space, turning any sequence of
white space characters, in both the expected output and in the actual output, into
a single space. It then checks whether these normalized versions match.

Example invoking normality

We're going to write a couple of tests that demonstrate how whitespace
normalization works.
Insert the following code into your doctest file:
Next, a demonstration of whitespace normalization.
>>> [1, 2, 3, 4, 5, 6, 7, 8, 9]
[ 18 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Chapter 2
...
[1,
4,
7,

# doctest: +NORMALIZE_WHITESPACE
2, 3,
5, 6,
8, 9]

>>> sys.stdout.write("This text\n contains weird

... # doctest: +NORMALIZE_WHITESPACE
This text contains weird spacing.
39

spacing.\n")

Result white space matches any other

white space
Both of these tests pass, in spite of the fact that the result of the first one has been
wrapped across multiple lines to make it easy for humans to read, and the result
of the second one has had its strange newlines and indentations left out, also for
human convenience.
Notice how one of the tests inserts extra whitespace in the expected output, while
the other one ignores extra whitespace in the actual output? When you use
+NORMALIZE_WHITESPACE, you gain a lot of flexibility with regard to how things
are formatted in the text file.
You may have noted the value 39 on the last line of the last
example. Why is that there? It's because the write() method
returns the number of bytes that were written, which in this
case happens to be 39. If you're trying this example in an
environment that maps ASCII characters to more than one byte,
you will see a different number here; this will cause the test to
fail until you change the expected number of bytes.

Skipping an example
On some occasions, doctest will recognize some text as an example to be checked,
when in truth you want it to be simply text. This situation is rarer than it might at first
seem, because usually there's no harm in letting doctest check everything it can. In
fact, usually it's very helpful to have doctest check everything it can. For those times
when you want to limit what doctest checks, though, there's the +SKIP directive.

[ 19 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Working with doctest

Example humans only

Append the following code to your doctest file:
Now we're telling doctest to skip a test
>>> 'This test would fail.' # doctest: +SKIP
If it were allowed to run.

Result it looks like a test, but it's not

Before we added this last example to the file, doctest reported thirteen tests when
we ran the file through it. After adding this code, doctest still reports thirteen tests.
Adding the skip directive to the code completely removed it from consideration by
doctest. It's not a test that passes, nor a test that fails. It's not a test at all.

The other directives

There are a number of other directives that can be issued to doctest, should you find
the need. They're not as broadly useful as the ones already mentioned, but the time
might come when you require one or more of them.
The full documentation for all of the doctest directives can
be found at http://docs.python.org/3/library/
doctest.html#doctest-options.

The remaining directives of doctest in the Python 3.4 version are as follows:

DONT_ACCEPT_TRUE_FOR_1: This makes doctest differentiate between

boolean values and numbers

DONT_ACCEPT_BLANKLINE: This removes support for the

<BLANKLINE> feature

IGNORE_EXCEPTION_DETAIL: This makes doctest only care that an exception

is of the expected type

Strictly speaking, doctest supports several other options that can be set using
the directive syntax, but they don't make any sense as directives, so we'll ignore
them here.

[ 20 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Chapter 2

The execution scope of doctest tests

When doctest is running the tests from text files, all the tests from the same file are
run in the same execution scope. This means that, if you import a module or bind a
variable in one test, that module or variable is still available in later tests. We took
advantage of this fact several times in the tests written so far in this chapter: the sys
module was only imported once, for example, although it was used in several tests.
This behavior is not necessarily beneficial, because tests need to be isolated from each
other. We don't want them to contaminate each other because, if a test depends on
something that another test does, or if it fails because of something that another test
does, these two tests are in some sense combined into one test that covers a larger
section of your code. You don't want that to happen, because then knowing which
test has failed doesn't give you as much information about what went wrong and
where it happened.
So, how can we give each test its own execution scope? There are a few ways to
do it. One would be to simply place each test in its own file, along with whatever
explanatory text that is needed. This works well in terms of functionality, but
running the tests can be a pain unless you have a tool to find and run all of them
for you. We'll talk about one such tool (called Nose) in a later chapter. Another
problem with this approach is that this breaks the idea that the tests contribute to
a human-readable document.
Another way to give each test its own execution scope is to define each test within
a function, as follows:
>>> def test1():
...
import frob
...
return frob.hash('qux')
>>> test1()
77

By doing this, the only thing that ends up in the shared scope is the test function
(named test1 here). The frob module and any other names bound inside the
function are isolated with the caveat that things that happen inside imported
modules are not isolated. If the frob.hash() method changes a state inside the
frob module, that state will still be changed if a different test imports the frob
module again.
The third way is to exercise caution with the names you create, and be sure to set
them to known values at the beginning of each test section. In many ways this is the
easiest approach, but this is also the one that places the most burden on you, because
you have to keep track of what's in the scope.

[ 21 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Working with doctest

Why does doctest behave in this way, instead of isolating tests from each other?
The doctest files are intended not just for computers to read, but also for humans.
They often form a sort of narrative, flowing from one thing to the next. It would
break the narrative to be constantly repeating what came before. In other words, this
approach is a compromise between being a document and being a test framework,
a middle ground that works for both humans and computers.
The other framework that we will study in depth in this book (called simply
unittest) works at a more formal level, and enforces the separation between tests.

Check your understanding

Once you've decided on your answers to these questions, check them by writing
a test document and running it through doctest:

How does doctest recognize the beginning of a test in a document?

How does doctest know when a test continues to further lines?

How does doctest recognize the beginning and end of the expected output
of a test?

How would you tell doctest that you want to break the expected output
across several lines, even though that's not how the test actually outputs it?

Which parts of an exception report are ignored by doctest?

When you assign a variable in a test file, which parts of the file can actually
see that variable?

Why do we care what code can see the variables created by a test?

How can we make doctest not care what a section of output contains?

Exercise English to doctest

Time to stretch your wings a bit. I'm going to give you a description of a single
function in English. Your job is to copy the description into a new text file, and
then add tests that describe all the requirements in a way that the computer can
understand and check.
Try to make the doctests so that they're not just for the computer. Good doctests tend
to clarify things for human readers as well. By and large, this means that you present
them to human readers as examples interspersed with the text.

[ 22 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Chapter 2

Without further ado, here is the English description:

The fib(N) function takes a single integer as its only parameter N.
If N is 0 or 1, the function returns 1. If N is less than 0, the
function raises a ValueError. Otherwise, the function returns the sum
of fib(N 1) and fib(N 2). The returned value will never be less
than 1. A nave implementation of this function would get very slow as
N increased.

I'll give you a hint and point out that the last sentence about the function being slow,
isn't really testable. As computers get faster, any test you write that depends on an
arbitrary definition of "slow" will eventually fail. Also, there's no good way to test
the difference between a slow function and a function stuck in an infinite loop, so
there's not much point in trying. If you find yourself needing to do that, it's best to
back off and try a different solution.
Not being able to tell whether a function is stuck or just slow is
called the halting problem by computer scientists. We know that
it can't be solved unless we someday discover a fundamentally
better kind of computer. Faster computers won't do the trick, and
neither will quantum computers, so don't hold your breath.

The next-to-last sentence also provides some difficulty, since to test it completely
would require running every positive integer through the fib() function, which
would take forever (except that the computer will eventually run out of memory and
force Python to raise an exception). How do we deal with this sort of thing, then?
The best solution is to check whether the condition holds true for a random sample
of viable inputs. The random.randrange() and random.choice() functions in the
Python standard library make that fairly easy to do.

Embedding doctests into docstrings

It's just as easy to write doctests into docstrings as it is to write them into
documentation files.
For those who don't know, docstrings are a Python feature that
allows programmers to embed documentation directly into
their source code. The Python help() function is powered by
docstrings. To learn more about docstrings, you can start with
the Python tutorial section at https://docs.python.org/3/
tutorial/controlflow.html#documentation-strings.

[ 23 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Working with doctest

When written in docstrings, doctests serve a slightly different purpose. They still let
the computer check that things work as expected, but the humans who see them will
most often be coders who use the Python interactive shell to work on an idea before
committing it to code, or whose text editor pops up docstrings as they work. In that
context, the most important thing a doctest can do is be informative, so docstrings
aren't usually a good place for checking picky details. They're a great place for a
doctest to demonstrate the proper behavior of a common case, though.
The doctests embedded in docstrings have a somewhat different execution scope than
doctests in text files do. Instead of having a single scope for all of the tests in the file,
doctest creates a single scope for each docstring. All of the tests that share a docstring
also share an execution scope, but they're isolated from tests in the other docstrings.
The separation of each docstring into its own execution scope often means that we
don't need to put much thought into isolating doctests when they're embedded
in docstrings. This is fortunate, since docstrings are primarily intended for
documentation, and the tricks required to isolate the tests might obscure the meaning.

Example a doctest in a docstring

We're going to embed a test right inside the Python source file that it tests, by placing
it inside a docstring.
Create a file called test.py containing the following code:
def testable(x):
r"""
The `testable` function returns the square root of its
parameter, or 3, whichever is larger.
>>> testable(7)
3.0
>>> testable(16)
4.0
>>> testable(9)
3.0
>>> testable(10) == 10 ** 0.5
True
"""
if x < 9:
return 3.0
return x ** 0.5
[ 24 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Chapter 2

Notice the use of a raw string for the docstring (denoted by the
r character before the first triple quote). Using raw strings for
your docstrings is a good habit to get into, because you usually
don't want escape sequencesfor example, \n for newline to
be interpreted by the Python interpreter. You want them to be
treated as text, so that they are correctly passed on to doctest.

Running these tests is just as easy as running the tests in a doctest document:
python3 -m doctest test.py

Since all the tests pass, the output of this command is nothing at all. We can make it
more interesting by adding the verbose flag to the command line:
python3 -m doctest -v test.py

Result the code is now self-documenting

and self-testable
When we run the Python file through doctest with the verbose flag, we see the
output, as shown in the the following screenshot:

[ 25 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Working with doctest

We put the doctest code right inside the docstring of the function it was testing.
This is a good place for tests that also show a programmer how to do something. It's
not a good place for detailed, low-level tests (the doctest in the docstring example
code, which was quite detailed for illustrative purposes, is perhaps too detailed),
because docstrings need to serve as API documentationyou can see the reason for
this just by looking at the example, where the doctests take up most of the room in
the docstring without telling the readers any more than they would have learned
from a single test.
Any test that will serve as good API documentation is a good candidate for including
in the docstrings of a Python file.
You might be wondering about the line that reads 1 items had no tests, and the
following line that just reads test. These lines are referring to the fact that there are
no tests written in the module-level docstring. That's a little surprising, since we didn't
include such a docstring in our source code at all, until you realize that, as far as Python
(and thus doctest) is concerned, no docstring is the same as an empty docstring.

Putting it into practice an AVL tree

We're going to walk step-by-step through the process of using doctest to create a
testable specification for a data structure called an AVL tree. An AVL tree is a way to
organize key-value pairs so that they can be quickly located by key. In other words,
it's a lot like Python's built-in dictionary type. The name AVL references the initials
of the people who invented this data structure.
While AVL trees are similar to Python dictionaries, they have
some significantly different properties. For one thing, the keys
stored in an AVL tree can be iterated over in a sorted order with
no overhead. Another difference is that, while inserting and
removing objects in an AVL tree is slower than a Python dict
in many cases, it's faster in the worst case.

As its name suggests, an AVL tree organizes the keys that are stored in it into a tree
structure, with each key having up to two child keys one child key that is less
than the parent key by comparison, and one that is more. In the following figure,
the Elephant key has two child keys, Goose has one, and Aardvark and Frog both
have none.

[ 26 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Chapter 2

The AVL tree is special because it keeps one side of the tree from getting much
taller than the other, which means that users can expect it to perform reliably and
efficiently no matter what. In the following figure, the AVL tree will reorganize to
stay balanced if Frog gains a child:

"Frog"
Lesser
"Aardvark"

"Goose"
Lesser
Greater
"Elephant"

We're going to write tests for an AVL tree implementation here, rather than writing
the implementation itself, so we're going to gloss over the details of how an AVL tree
works, in favor of looking at what it should do when it works right.
If you want to know more about AVL trees, you will
find many good references on the Internet. Wikipedia's
entry on this subject is a good place to start with:
http://en.wikipedia.org/wiki/AVL_tree.

We're going to start with a plain-language specification, and then interject tests
between the paragraphs. You don't have to actually type all of this into a text file;
it is here for you to read and to think about.

English specification
The first step is to describe what the desired result should be, in normal language.
This might be something that you do for yourself, or it might be something that
somebody else does for you. If you're working for somebody, hopefully you and
your employer can sit down together and work this part out.
In this case, there's not much to work out, because AVL trees have been fully
described for decades. Even so, the description here isn't quite like the one you'd find
elsewhere. This capacity for ambiguity is exactly the reason why a plain-language
specification isn't good enough. We need an unambiguous specification, and that's
exactly what the tests in a doctest file can give us.
[ 27 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Working with doctest

The following text goes in a file called AVL.txt, (that you can find in its final form in
the accompanying code archive; at this stage of the process, the file contains only the
normal language specification):
An AVL Tree consists of a collection of nodes organized in a binary
tree structure. Each node has left and right children, each of which
may be either None or another tree node. Each node has a key, which
must be comparable via the less-than operator. Each node has a value.
Each node also has a height number, measuring how far the node is from
being a leaf of the tree -- a node with height 0 is a leaf.
The binary tree structure is maintained in ordered form, meaning that
of a node's two children, the left child has a key that compares
less than the node's key and the right child has a key that compares
greater than the node's key.
The binary tree structure is maintained in a balanced form, meaning
that for any given node, the heights of its children are either the
same or only differ by 1.
The node constructor takes either a pair of parameters representing
a key and a value, or a dict object representing the key-value pairs
with which to initialize a new tree.
The following methods target the node on which they are called, and
can be considered part of the internal mechanism of the tree:
Each node has a recalculate_height method, which correctly sets the
height number.
Each node has a make_deletable method, which exchanges the positions
of the node and one of its leaf descendants, such that the tree
ordering of the nodes remains correct.
Each node has rotate_clockwise and rotate_counterclockwise methods.
Rotate_clockwise takes the node's right child and places it where
the node was, making the node into the left child of its own former
child. Other nodes in the vicinity are moved so as to maintain
the tree ordering. The opposite operation is performed by rotate_
counterclockwise.
Each node has a locate method, taking a key as a parameter, which
searches the node and its descendants for a node with the specified
key, and either returns that node or raises a KeyError.
The following methods target the whole tree rooted at the current
node. The intent is that they will be called on the root node:
[ 28 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Chapter 2
Each node has a get method taking a key as a parameter, which locates
the value associated with the specified key and returns it, or raises
KeyError if the key is not associated with any value in the tree.
Each node has a set method taking a key and a value as parameters, and
associating the key and value within the tree.
Each node has a remove method taking a key as a parameter, and
removing the key and its associated value from the tree. It raises
KeyError if no value was associated with that key.

Node data
The first three paragraphs of the specification describe the member variables of an
AVL tree node, and tell us what the valid values for the variables are. They also tell
us how the tree height should be measured and define what a balanced tree means.
It's our job now to take these ideas, and encode them into tests that the computer can
eventually use to check our code.
We can check these specifications by creating a node and then testing the values,
but that would really just be a test of the constructor. It's important to test the
constructor, but what we really want to do is to incorporate checks that the node
variables are left in a valid state into our tests of each member function.
To that end, we'll define functions that our tests can call to check that the state of
a node is valid. We'll define these functions just after the third paragraph, because
they provide extra details related to the content of the first three paragraphs:
Notice that the node data test is written as if the AVL tree
implementation already existed. It tries to import an avl_tree
module containing an AVL class, and it tries to use the AVL class
in specific ways. Of course, at the moment there is no avl_tree
module, so the tests will fail. This is as it should
be. All that the failure means is that, when the time comes
to implement the tree, we should do so in a module called avl_
tree, with contents that function as our tests assume. Part of the
benefit of testing like this is being able to test-drive your code
before you even write it.
>>> from avl_tree import AVL
>>> def valid_state(node):
...
if node is None:
...
return
...
if node.left is not None:
[ 29 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Working with doctest

...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...

assert isinstance(node.left, AVL)

assert node.left.key < node.key
left_height = node.left.height + 1
else:
left_height = 0
if node.right is not None:
assert isinstance(node.right, AVL)
assert node.right.key > node.key
right_height = node.right.height + 1
else:
right_height = 0
assert abs(left_height - right_height) < 2
node.key < node.key
node.value

>>> def valid_tree(node):

...
if node is None:
...
return
...
valid_state(node)
...
valid_tree(node.left)
...
valid_tree(node.right)

Notice that we didn't actually call these functions yet. They aren't tests, as such, but
tools that we'll use to simplify writing tests. We define them here, rather than in the
Python module that we're going to test, because they aren't conceptually part of the
tested code, and because anyone who reads the tests will need to be able to see what
the helper functions do.

Testing the constructor

The fourth paragraph describes the constructor of the AVL class. According to
this paragraph, the constructor has two modes of operation: it can create a single
initialized node, or it can create and initialize a whole tree of nodes based on the
contents of a dictionary.
The test for the single node mode is easy. We'll add it after the fourth paragraph:
>>> valid_state(AVL(2, 'Testing is fun'))

We don't even have to write an expected result, since we wrote the function to
raise an AssertionError if there's a problem and to return None if everything is
fine. AssertionError is triggered by the assert statement in our test code, if the
expression in the assert statement produces a false value.
[ 30 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Chapter 2

The test for the second mode looks just as easy, and we'll add it right after the other:
>>> valid_tree(AVL({1: 'Hello', 2: 'World', -3: '!'}))

There's a bit of buried complexity here, though. In all probability, this constructor
will function by initializing a single node and then using that node's set method
to add the rest of the keys and values to the tree. This means that our second
constructor test isn't a unit test, it's an integration test that checks the interaction
of multiple units.
Specification documents often contains integration-level and system-level tests, so
this isn't really a problem. It's something to be aware of, though, because if this test
fails it won't necessarily show you where the problem really lies. Your unit tests will
do that.
Something else to notice is that we didn't check whether the constructor fails
appropriately when given bad inputs. These tests are very important, but the English
specification didn't mention these points at all, which means that they're not really
among the acceptance criteria. We'll add these tests to the unit test suite instead.

Recalculating height
The recalculate_height() method is described in the fifth paragraph of the
specification. To test it, we're going to need a tree for it to operate on, and we don't
want to use the second mode of the constructor to create it after all, we want this
test to be independent of any errors that might exist there. We'd really prefer to make
the test entirely independent of the constructor but, in this case, we need to make
a small exception to the rule, since it's mighty difficult to create an object without
calling its constructor in some way.
What we're going to do is define a function that builds a specific tree and returns it.
This function will be useful in several of our later tests as well:
>>> def make_test_tree():
...
root = AVL(7, 'seven')
...
root.height = 2
...
root.left = AVL(3, 'three')
...
root.left.height = 1
...
root.left.right = AVL(4, 'four')
...
root.right = AVL(10, 'ten')
...
return root

[ 31 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Working with doctest

Now that we have the make_test_tree() function, testing recalculate_height()

is easy:
>>>
>>>
>>>
>>>
2

tree = make_test_tree()
tree.height = 0
tree.recalculate_height()
tree.height

Making a node deletable

The sixth paragraph of the specification described the make_deletable() method.
You can't delete a node that has children, because that would leave the node's
children disconnected from the rest of the tree. Consider the tree with animal names
in it that we looked at earlier. If we delete the Elephant node from the bottom of the
tree, what do we do about Aardvark, Goose, and Frog? If we delete Goose, how do
we find Frog afterwards?

"Frog"
Lesser
"Aardvark"

"Goose"
Lesser
Greater
"Elephant"

The way around that is to have the node swap places with its largest leaf descendant
on the left side (or its smallest leaf descendant on the right side, but we're not doing
it that way).
We'll test this by using the same make_test_tree() function that we defined
earlier to create a new tree to work on, and then check whether make_deletable()
swaps correctly:
>>> tree = make_test_tree()
>>> target = tree.make_deletable()
>>> (tree.value, tree.height)
('four', 2)
>>> (target.value, target.height)
('seven', 0)
[ 32 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Chapter 2

Rotation
The two rotate functions, described in paragraph seven of the specification, perform
a somewhat tricky manipulation of the links in a tree. You probably found the plain
language description of what they do a bit confusing. This is one of those times when
a little bit of code makes a whole lot more sense than any number of sentences.
While tree rotation is usually defined in terms of rearranging the links between
nodes in the tree, we'll check whether it worked by looking at the values rather than
by looking directly at the left and right links. This allows the implementation to swap
the contents of nodes, rather than the nodes themselves, when it wishes. After all, it's
not important to the specification which operation happens, so we shouldn't rule out
a perfectly reasonable implementation choice:
>>> tree = make_test_tree()
>>> tree.value
'seven'
>>> tree.left.value
'three'
>>> tree.rotate_counterclockwise()
>>> tree.value
'three'
>>> tree.left is None
True
>>> tree.right.value
'seven'
>>> tree.right.left.value
'four'
>>> tree.right.right.value
'ten'
>>> tree.right.left.value
'four'
>>> tree.left is None
True
>>> tree.rotate_clockwise()
>>> tree.value
'seven'
>>> tree.left.value
'three'
>>> tree.left.right.value
'four'
>>> tree.right.value
'ten'
>>> tree.right.left is None
True
>>> tree.left.left is None
True
[ 33 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Working with doctest

Locating a node
According to the eighth paragraph of the specification, the locate() method is
expected to return a node, or raise a KeyError exception, depending on whether
the key exists in the tree or not. We'll use our specially built testing tree again, so
that we know exactly what the tree's structure looks like:
>>> tree = make_test_tree()
>>> tree.locate(4).value
'four'
>>> tree.locate(17) # doctest: +ELLIPSIS
Traceback (most recent call last):
KeyError: ...

Downloading the example code

You can download the example code files for all
Packt books you have purchased from your account
at http://www.packtpub.com. If you purchased
this book elsewhere, you can visit http://www.
packtpub.com/support and register to have the
files e-mailed directly to you.

The rest of the specification

The remaining paragraphs of the specification describe higher-level functions that
operate by calling the already described functions. This means that, until we learn
the tricks of mock objects in Chapter 4, Decoupling Units with unittest.mock, we're stuck
with writing integration-level tests here. As I mentioned earlier, this is not a terrible
thing to do in a specification document, so we'll go ahead and do it:
Each node has a get method taking a key as a parameter, which locates
the value associated with the specified key and returns it, or raises
KeyError if the key is not associated with any value in the tree.
>>> tree = make_test_tree()
>>> tree.get(10)
'ten'
>>> tree.get(97) # doctest: +ELLIPSIS
Traceback (most recent call last):
KeyError: ...
Each node has a set method taking a key and a value as parameters, and
associating the key and value within the tree.

[ 34 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Chapter 2
>>> tree = make_test_tree()
>>> tree.set(10, 'foo')
>>> tree.locate(10).value
'foo'
Each node has a remove method taking a key as a parameter, and
removing the key and its associated value from the tree. It raises
KeyError if no values was associated with that key.
>>> tree = make_test_tree()
>>> tree.remove(3)
>>> tree.remove(3) # doctest: +ELLIPSIS
Traceback (most recent call last):
KeyError: ...

Summary
We learned the syntax of doctest, and went through several examples describing
how to use it. After that, we took a real-world specification for the AVL tree,
and examined how to formalize it as a set of doctests, so that we could use it
to automatically check the correctness of an implementation.
Specifically, we covered doctest's default syntax and the directives that alter it, how
to write doctests in text files, how to write doctests in Python docstrings, and what it
feels like to use doctest to turn a specification into tests.
Now that we've learned about doctest, we're ready to talk about how to use
doctest to do unit testingthe topic of the next chapter.

[ 35 ]

For More Information:

www.packtpub.com/application-development/learning-python-testing

Where to buy this book

You can buy Learning Python Testing from the Packt Publishing website:
.
Free shipping to the US, UK, Europe and selected Asian countries. For more information, please
read our shipping policy.

Alternatively, you can buy the book from Amazon, BN.com, Computer Manuals and
most internet book retailers.

www.PacktPub.com

For More Information:

www.packtpub.com/application-development/learning-python-testing