Testing and debugging

in Python

Sukrit Gupta

November 16, 2023

Sukrit Gupta Intro to Computing and Data Structures 1/31

Outline

1 Real world scenario in software development

2 Testing

3 Bugs and debugging

4 Exceptions

Sukrit Gupta Intro to Computing and Data Structures 2/31

Acknowledgement and disclaimer
All mistakes (if any) are mine.
I have used several other sources which I have referred to in the
appropriate places.

Sukrit Gupta Intro to Computing and Data Structures 3/31

 Section 1

Real world scenario in software development

Sukrit Gupta Intro to Computing and Data Structures 4/31

Let’s look for software around us

If we look at AirBnB, it employed close to 1000 software

professionals1 .
How do so many people manage to collaborate, maintain and
create new features on the code repository?
More importantly how do they ensure that the product that they
launch is not buggy.

1
source: LinkedIn, 1st January, 2022
Sukrit Gupta Intro to Computing and Data Structures 5/31
Ensuring product quality

Often developers who have written the code are biased or are
unaware of the real world scenario in which the code will be used.
They are, therefore, not the best people to judge whether a code
works as expected.
Therefore, in each organization there is a dedicated team of
professionals that test the product.
Testing has been evolved to cater to different types of errors that
creep into software.

Sukrit Gupta Intro to Computing and Data Structures 6/31

 Section 2

Testing

Sukrit Gupta Intro to Computing and Data Structures 7/31

Testing
“Program testing can be used to show the presence of bugs, but never
to show their absence!” - Edsger Dijkstra

“No amount of experimentation can ever prove me right; a single

experiment can prove me wrong.” - Albert Einstein

Testing is the process of running a program to try and ascertain

whether or not it works as intended.
The purpose of testing is to show that bugs exist, not to show that
a program is bug-free.
Why is this so? Even the simplest of programs has billions of
possible inputs.
The best we can do is to run it on inputs that have a reasonable
probability of producing the wrong answer.

Sukrit Gupta Intro to Computing and Data Structures 8/31

Test Suite

def factorial(n):
fact = 1
while n>1:
fact *= n
n -=1
return fact

Test suite: A collection of inputs that has a high likelihood of

revealing bugs, yet does not take too long to run.
Step 1: Partition the input space into subsets that provide
equivalent information about the correctness of the program.
Step 2: Then construct a test suite that contains one input from
each partition.
Running the code snippet above on all integers would be tedious
(and impossible!). We can, however, run it on integers that have a
reasonable probability of producing the wrong answer.
Sukrit Gupta Intro to Computing and Data Structures 9/31
Test Partition

A disjoint partition of the input set. Consider, for example,

factorial(x). The set of possible test inputs is all integers.

One way to partition this set is into these subsets:

x positive
x negative
x=0
If one tested the implementation on at least one value from each of
these subsets, there would be reasonable probability (but no guarantee)
of exposing a bug if one exists.

Let us look at the different types of testing.

Sukrit Gupta Intro to Computing and Data Structures 10/31

Black Box Testing

Black-box tests are constructed without looking at the code to be

tested.

Sukrit Gupta Intro to Computing and Data Structures 11/31

Black Box Testing

Black-box testing allows testers and implementers to be drawn

from separate populations.
The independence reduces the likelihood of generating test suites
that exhibit mistakes that are correlated with mistakes in the code.
If the same people construct the test suite for the program, they
would likely bring assumptions/biases while designing the test
suite.
Black-box testing is robust with respect to implementation
changes.

Sukrit Gupta Intro to Computing and Data Structures 12/31

Glass Box Testing

Sukrit Gupta Intro to Computing and Data Structures 13/31

Glass Box Testing
Glass-box testing tests the internal structures or workings of an
application, as opposed to its functionality.
Without looking at the internal structure of the code, it is
impossible to know which test cases are likely to provide new
information.
Black-box testing should never be skipped, but it is rarely
sufficient. Glass-box test suites are usually much easier to
construct than black-box test suites.
Glass-box testing usually works at the following levels:
Unit testing: Testing on individual code files to ensure that the
code is working as intended, before integration happens with
previously tested code. Catches many defects early on and aids in
addressing defects that happen later on.
Integration testing: Examines the correctness of the behaviour in an
open environment for any interactions of interfaces that are known
to the programmer.

Sukrit Gupta Intro to Computing and Data Structures 14/31

There are a few rules of thumb that are usually worth following:
Exercise both branches of all if statements.
For each for loop, have test cases in which:
The loop is not entered,
The loop is executed exactly once, and
The loop is executed more than once.
For recursive functions, include test cases that cause the function
to return with no recursive calls, exactly one recursive call, and
more than one recursive call.

Sukrit Gupta Intro to Computing and Data Structures 15/31

 Section 3

Bugs and debugging

Sukrit Gupta Intro to Computing and Data Structures 16/31

Debugging

Let us say that you perform testing and you find that the program
does not work as intended. What do you do?
You De-bug.
Debugging is the process of trying to fix a program that you
already know does not work as intended.
To make the program easier to test and debug, good programmers
break the program up into separate components that can be
implemented, tested, and debugged independently of other
components.
Assumption is that syntax errors and semantic errors have already
been tackled.

Sukrit Gupta Intro to Computing and Data Structures 17/31

Runtime bugs can be categorized along two dimensions:
Overt → covert:
An overt bug has an obvious manifestation. For example: the
program crashes or takes far longer to run than it should.
A covert bug has no obvious manifestation. The program may run
to conclusion with no problem—other than providing an incorrect
answer.
Many bugs fall between the two extremes, and whether or not the
bug is overt can depend upon how carefully one examines the
behavior of the program.
Persistent → intermittent:
A persistent bug occurs every time the program is run with the
same inputs.
An intermittent bug occurs only some of the time, even when the
program is run on the same inputs (known as race condition) and
seemingly under the same conditions.

Sukrit Gupta Intro to Computing and Data Structures 18/31

Overt and persistent (Easiest to detect)

Easily detected. Developers can be under no illusion about the

advisability of deploying the program.
Good programmers try to write their programs in such a way that
programming mistakes lead to bugs that are both overt and
persistent. This is often called defensive programming.
We will study about how exceptions and assertions can be used to
alert the users or the developers about the bugs.

Sukrit Gupta Intro to Computing and Data Structures 19/31

Overt but intermittent

Boeing 737 Max: Boeing employees “deceived the FAA,”

misleading the safety regulators about a new flight control system
on the 737 Max called MCAS. That system activated erroneously,
contributing to the deadly crashes, by forcing the planes into
nosedives that the pilots could not pull the aircraft out of 2 .
Can lead to deployment of systems with the flawed program, but
sooner or later the bug will manifest.
If the conditions prompting the bug to become manifest are easily
reproducible, it is often relatively easy to track down and repair
the problem. Otherwise, things are much harder.

2
www.npr.org
Sukrit Gupta Intro to Computing and Data Structures 20/31
Both covert and intermittent

A radiation therapy machine that delivers a little more or a little

less radiation than intended can be the difference between life and
death for a person with cancer.
We are increasingly dependent on software to perform critical
computations that are beyond the ability of humans to carry out
or even check for correctness. This makes it hard to debug the
code (because you don’t know the expected output).
Therefore, a program can provide undetected fallacious answer for
long periods of time. Such programs can cause a lot of damage.
Bugs that are both covert and intermittent are almost always the
hardest to find and fix. Programs that fail in covert ways are often
highly dangerous.

Sukrit Gupta Intro to Computing and Data Structures 21/31

Debugging process

Ask: ‘Why is the program behaving like this?’

Rule out blocks of code where the bug can be. Either line by line
or function by function.
Sometimes there’s a fault with the logic itself.
Stop debugging and start writing documentation.
Walk away, and try again tomorrow.
When you have found the bug, think about all the places that the bug
can manifest and update your code accordingly.

Sukrit Gupta Intro to Computing and Data Structures 22/31

 Section 4

Exceptions

Sukrit Gupta Intro to Computing and Data Structures 23/31

Exceptions in Python

Most of the built-in exceptions of Python deal with situations in

which a program has attempted to execute a statement with no
appropriate semantics.
When an exception is raised, the program terminates (crashes),
and we go back to our code and attempt to figure out what went
wrong.
When an exception is raised that causes the program to terminate,
we say that an unhandled exception has been raised.

Sukrit Gupta Intro to Computing and Data Structures 24/31

Handling exceptions

An exception does not need to lead to program termination.

Exceptions, when raised, can be handled by the program.
Sometimes an exception is raised because there is a bug in the
program (like accessing a variable that doesn’t exist).
However, many times, an exception is something the programmer
can and should anticipate.

Sukrit Gupta Intro to Computing and Data Structures 25/31

try-except block

def divide(x, y):

try:
val = x/y
except ZeroDivisionError: #or except Exception e:
val = None
print('Cannot divide by zero')
return val

The try block lets you test a block of code for errors.
The except block lets you handle the error.
If the programmer forgets to include the except block of code and
the exception is raised, the program will halt immediately.
This is good because it creates an overt bug and makes it easy to
debug the program.

Sukrit Gupta Intro to Computing and Data Structures 26/31

Raising exceptions

Exceptions can be used as a convenient flow-of control mechanism

to simplify programs.
Functions can raise an exception when it cannot produce a result
that is consistent with the function’s specification.

1 raise exceptionName(arguments)

The exceptionName is usually one of the built-in exceptions. You

can use Python’s built-in exceptions (Link) or define your own
exceptions.

Sukrit Gupta Intro to Computing and Data Structures 27/31

Example: Raising exceptions

def divide(x, y):

if y == 0:
val = None
raise ZeroDivisionError("divide by zero")
else:
val = x/y
return val

Sukrit Gupta Intro to Computing and Data Structures 28/31

Assertions

An assertion is a sanity-check that can replace the if-raise statement.

An expression is tested, and if the result comes up false, an
exception (AssertionError) is raised.

def divide(x, y):

assert y != 0, 'Cannot divide by zero'
val = x/y
return val

Sukrit Gupta Intro to Computing and Data Structures 29/31

What did we learn today?

1 Real world scenario in software development

2 Testing

3 Bugs and debugging

4 Exceptions

Sukrit Gupta Intro to Computing and Data Structures 30/31

 Thank you!

Sukrit Gupta Intro to Computing and Data Structures 31/31