Testing and Debugging: Chapter Goals

Testing and Debugging
8
CHAPTER
Chapter Goals
◆ To learn how to design test harnesses for testing components of your programs
in isolation
◆ To understand the principles of test case selection and evaluation
◆ To learn how to use assertions
◆ To become familiar with the debugger
◆ To learn strategies for effective debugging
313
314 Chapter 8. Testing and Debugging
A complex program never works right the first time; it will contain errors, commonly
called bugs, and will need to be tested. It is easier to test a program if it has been de-
signed with testing in mind. This is a common engineering practice: On television
circuit boards or in the wiring of an automobile, you will find lights and wire connec-
tors that serve no direct purpose for the TV or car but are put in place for the repair
person in case something goes wrong. In the first part of this chapter you will learn
how to instrument your programs in a similar way. It is a little more work up-front,
but that work is amply repaid by shortened debugging times.
In the second part of this chapter you will learn how to run the debugger to cope
with programs that don’t do the right thing.
8.1 Unit Tests
The single most important testing tool is the unit test of a method or a set of co-
operating methods. For this test, the methods are compiled outside the program
in which they will be used, together with a test harness that feeds parameters to the
methods.
The test arguments can come from one of three sources: from user input, by run-
ning through a range of values in a loop, and as random values.
In the following sections, we will use a simple example for a method to test, namely
an approximation algorithm to compute square roots that was known to the ancient
Greeks. The algorithm starts by guessing a value x that might be somewhat close to
the desired square root 冪a. The initial value doesn’t have to be very close; x ⳱ a is a
perfectly good choice. Now consider the quantities x and a/x. If x ⬍ 冪a, then a/x ⬎
a/ 冪a ⳱ 冪a. Similarly, if x ⬎ 冪a, then a/x ⬍ a/ 冪a ⳱ 冪a. That is, 冪a lies between
x and a/x. Make the midpoint of that interval our improved guess of the square root, as
shown in Figure 1. Therefore set xnew ⳱ (x Ⳮ a/x)/2 and repeat the procedure—that
is, compute the average of xnew and a/xnew . Stop when two successive approximations
differ from each other by a very small amount.
This method converges very rapidly. To compute 冪400, only 10 steps are re-
quired:
400
200.5
101.24750623441396
52.599110411804922
30.101900881222353
21.695049123587058
20.06621767747577
20.000109257780434
20.000000000298428
20
8.1 Unit Tests 315
Figure 1 a/x a x
Approximation of the Square Midpoint

Root
Here is a static method sqrt in a class MathAlgs that implements this algorithm.
class MathAlgs
{ public static double sqrt(double a)
{ if (a <= 0) return 0;
double xnew = a;
double xold;
do
{ xold = xnew;
xnew = (xold + a / xold) / 2;
} while (!Numeric.approxEqual(xnew, xold));
return xnew;
}
}
Does this method work correctly? Let us approach this question systematically. First,
let us write a test harness to supply individual values to the method to be tested.
Program SqrtTest1.java
public class SqrtTest1
{ public static void main(String[] args)
{ ConsoleReader console = new ConsoleReader(System.in);
boolean done = false;
while (!done)
{ String inputLine = console.readLine();
if (inputLine == null) done = true;
else
{ double x = Double.parseDouble(inputLine);
double y = MathAlgs.sqrt(x);
System.out.println("square root of " + x

+ " = " + y);
}
}
}
}
When you run this test harness, you need to enter inputs and to force an end of input
when you are done, by typing a key such as Ctrl+Z or Ctrl+D (see Section 6.5.1). You
can store the test data in a file and use redirection (see Productivity Hint 6.1):
java SqrtTest1 < test1.in
For each test case, the harness code calls the MathAlgs.sqrt method and prints the
result. You can then manually check the computations. Once you have confidence
that the method works correctly, you can plug it into your program.
It is also possible to generate test cases automatically. If there are few possible
inputs, it is feasible to run through a representative number of them with a loop:
{ for (double x = 0; x <= 10; x = x + 0.5)
{ double y = MathAlgs.sqrt(x);
+ " = " + y);
}
}
}
Note that we purposefully test boundary cases (zero) and fractional numbers.
Unfortunately, this test is restricted to only a small subset of values. To overcome
that limitation, random generation of test cases can be useful:
import java.util.Random;

{ Random generator = new Random();
for (int i = 1; i <= 100; i++)
{ // generate random test value
double x = 1.0E6 * generator.nextDouble();


+ " = " + y);
}
}
}
No matter how you generate the test cases, the important point is that you test the
method thoroughly before you put it into the program. If you ever put together a
computer or fixed a car, you probably followed a similar process. Rather than simply
throwing all the parts together and hoping for the best, you probably first tested
each part in isolation. It takes a little longer, but it greatly reduces the possibility of
complete failure once the parts are put together.
8.2 Selecting Test Cases 317
8.2 Selecting Test Cases

Selecting good test cases is an important skill for debugging programs. Of course,
you want to test your program with inputs that a typical user might supply.
You should test all program features. In the program that prints English names of
numbers, you should check typical test cases such as 5, 19, 29, 1093, 1728, 30000.
These tests are positive tests. They consist of legitimate inputs, and you expect the
program to handle them correctly.
Next, you should include boundary cases. Test what happens if the input is 0 or
-1. Boundary cases are still legitimate inputs, and you expect that the program will
handle them correctly—usually in some trivial way.
Finally, gather negative test cases. These are inputs that you expect the program to
reject. Examples are inputs in the wrong format, such as five.
How should you collect test cases? This is easy for programs that get all their in-
put from standard input. Just make each test case into a file—say, test1.in, test2.in,
test3.in. These files contain the keystrokes that you would normally type at the key-
board when the program runs. Feed the files into the program to be tested, using
redirection:
java Program < test1.in > test1.out
Then study the outputs and see whether they are correct.
Keeping a test case in a file is smart, because you can then use it to test every
version of the program. In fact, it is a common and useful practice to make a test
file whenever you find a program bug. You can use that file to verify that your bug
fix really works. Don’t throw it away; feed it to the next version after that and all
subsequent versions. Such a collection of test cases is called a test suite.
You will be surprised how often a bug that you fixed will reappear in a future ver-
sion. This is a phenomenon known as cycling. Sometimes you don’t quite understand
the reason for a bug and apply a quick fix that appears to work. Later, you apply a
different quick fix that solves a second problem but makes the first problem appear
again. Of course, it is always best to really think through what causes a bug and fix
the root cause instead of doing a sequence of “Band-Aid” solutions. If you don’t suc-
ceed in doing that, however, at least you want to have an honest appraisal of how
well the program works. By keeping all old test cases around and testing them all
against every new version, you get that feedback. The process of testing against a
set of past failures is called regression testing.
Testing the functionality of the program without consideration of its internal
structure is called black-box testing. That is an important part of testing, because, after
all, the users of a program do not know its internal structure. If a program works
perfectly on all positive inputs and fails gracefully on all negative ones, then it does
its job.
However, it is impossible to ensure absolutely that a program will work correctly
on all inputs just by supplying a finite number of test cases. As the famous computer
scientist Edsger Dijkstra pointed out, testing can show only the presence of bugs—
not their absence. To gain more confidence in the correctness of a program, it is
useful to consider its internal structure. Testing strategies that look inside a program
are called white-box testing. Performing unit tests of each method is a part of white-box
testing.
You want to make sure that each part of your program is exercised at least once
by one of your test cases. This is called test coverage. If some code is never executed
by any of your test cases, you have no way of knowing whether that code would
perform correctly if it ever were executed by user input. That means that you need to
look at every if/else branch to see that each of them is reached by some test case.
Many conditional branches are in the code only to take care of strange and abnormal
inputs, but they still do something. It is a common phenomenon that they end up
doing something incorrect, but that those faults are never discovered during testing,
because nobody supplied the strange and abnormal inputs. Of course, these flaws
become immediately apparent when the program is released and the first user types
in a bad input and is incensed when the program crashes. A test suite should ensure
that each part of the code is covered by some input.
For example, in testing the getTax method of the tax program in Chapter 5, you
want to make sure that every if statement is entered for at least one test case. You
should test both single and married taxpayers, with incomes in each of the three tax
brackets.
It is a good idea to write the first test cases before the program is written completely.
Designing a few test cases can give you insight into what the program should do,
which is valuable for implementing it. You will also have something to throw at the
program when it compiles for the first time. Of course, the initial set of test cases
will be augmented as the debugging process progresses.
Modern programs can be quite challenging to test. In a program with a graphical
user interface, the user can click random buttons with a mouse and supply input in
random order. Programs that receive their data through a network connection need
to be tested by simulating occasional network delays and failures. All this is much
harder, since you cannot simply place keystrokes in a file. You need not worry about
these complexities as you study this book, and there are tools to automate testing in
these scenarios. The basic principles of regression testing (never throwing a test case
away) and complete coverage (executing all code at least once) still hold.
8.3 Test Case Evaluation
In the last section we worried about how to get test inputs. Now let us consider what
to do with the outputs. How do you know whether the output is correct?
Sometimes you can verify the output by calculating the correct values by hand.
For example, for a payroll program you can compute taxes manually.
Sometimes a computation does a lot of work, and it is not practical to do the com-
putation manually. That is the case with many approximation algorithms, which
8.3 Test Case Evaluation 319
may run through dozens or hundreds of iterations before they arrive at the final an-
swer. The square root method of Section 8.1 is an example of such an approximation.
How can you test that the square root method works correctly? You can supply
test inputs for which you know the answer, such as 4 and 900, and also 25 4 , so that
you don’t just restrict the inputs to integers.
Alternatively, you can write a test harness that verifies that the output values fulfill
certain properties. For the square root program you can compute the square root,
compute the square of the result, and verify that you obtain the original input:

for (int i = 1; i <= 100; i++)

// check that test value fulfills square property
if (Numeric.approxEqual(y * y, x))
System.out.println("Test passed.");
else
System.out.println("Test failed.");
System.out.println("square root of " + x + " = " + y);
}
}
}
Finally, there may be a less efficient way of computing the same value that a method
produces. You can then run a test harness that computes the method to be tested,
together with the slower process, and compares the answers. For example, 冪x ⳱
x 1/2 , so you can use the slower Math.pow method to generate the same value. Such
a slower but reliable method is called an oracle.

for (int i = 1; i <= 100; i++)

// compare against oracle
if (Numeric.approxEqual(y, Math.pow(x, 0.5)))

System.out.println("Test passed. ");
else
System.out.println("Test failed. ");
System.out.println("square root of " + x + " = " + y);
}
}
}
Productivity Hint 8.1

◆ Batch Files and Shell Scripts
◆
◆ If you need to perform the same tasks repeatedly on the command line, then it is worth learn-
◆ ing about the automation features offered by your operating system.
◆ Under DOS, you use batch files to execute a number of commands automatically. For ex-
◆ ample, suppose you need to test a program with three inputs:
◆
◆ java Program < test1.in
◆
Then you find a bug, fix it, and run the tests again. Now you need to type the three commands
◆
◆ once more. There has to be a better way. Under DOS, put the commands in a text file and call
◆ it test.bat:
◆
◆ File test.bat
◆
◆
Then you just type
◆
◆ test
◆
◆ and the three commands in the batch file execute automatically.
◆ It is easy to make the batch file more useful. If you are done with Program and start working
◆ on Program2, you can of course write a batch file test2.bat, but you can do better than that.
◆ Give the test batch file a parameter. That is, call it with
◆
◆ test Program
◆
or
◆
◆ test Program2
◆
8.4 Program Traces 321
◆ You need to change the batch file to make this work. In a batch file, %1 denotes the first string
◆ that you type after the name of the batch file, %2 the second string, and so on:
◆
◆ File test.bat
◆
◆ java %1 < test1.in
◆
◆ What if you have more than three test files? DOS batch files have a very primitive for loop:
◆
◆ File test.bat
◆
◆ for %%f in (test*.in) do java %1 < %%f
◆ If you work in a computer lab, you will want a batch file that copies all your files onto a floppy
◆ disk when you are ready to go home. Put the following lines in a file gohome.bat:
◆
◆
◆
File gohome.bat
◆ copy *.java a:
◆ copy *.txt a:
◆ copy *.in a:
◆
◆ There are lots of uses for batch files, and it is well worth it to learn more about them.
◆ Batch files are a feature of the DOS operating system, not of Java. On a UNIX system, shell
◆ scripts are used for the same purpose.
8.4 Program Traces
Sometimes you run a program and you are not sure where it spends its time. To get
a printout of the program flow, you can insert trace messages into the beginning and
end of every method:
public static double sqrt(double a)
{ System.out.println("Entering MathAlgs.sqrt. a = " + a);
. . .
System.out.println
("Leaving MathAlgs.sqrt. Return value = " + xnew);
return xnew;
}
To get a proper trace, you must locate each method exit point. Place a trace message
before every return statement and at the end of the method:
public static int factorial(int n)
{ System.out.println("Entering factorial. n = " + n);
if (n < 0) throw new IllegalArgumentException();
if (n == 0)
{ System.out.println
("Exiting factorial. Return value = " + 1);
return 1;
}
else
{ int result = n * factorial(n-1);
System.out.println
("Leaving factorial. Return value = " + result);
return result;
}
}
You aren’t restricted to “enter/exit” messages. You can report on progress inside a
method:
public static double sqrt(double a)
{ . . .
do
{ xold = xnew;
System.out.println("MathAlgs.sqrt. xold = " + xold +
", xnew = " + xnew);
} while (!Numeric.approxEqual(xnew, xold));
. . .
}
Program traces can be useful to analyze the behavior of a program, but they have
some definite disadvantages. It can be quite time-consuming to find out which trace
messages to insert. If you insert too many messages, you produce a flurry of output
that is hard to analyze; if you insert too few, you may not have enough information to
spot the cause of the error. When you are done with the program, you need to remove
all trace messages. If you find another error, however, you need to stick the print
statements back in. If you find that a hassle, you are not alone. Most professional
programmers use a debugger, not trace messages, to locate errors in their code. The
debugger is covered later in this chapter.
8.5 Asserting Conditions
Programs often contain implicit assumptions. For example, denominators need to be

nonzero; salaries should not be negative. Sometimes the iron force of logic ensures
that these conditions are satisfied. If you divide by 1 + x * x, then that value will
never be zero, and you need not worry. Negative interest rates on savings accounts,
however, are not necessarily ruled out by logic but merely by convention. Surely no-
body would ever deposit money in an account that earns negative interest, but such
a value might creep into a program due to an input or processing error. In practice
the “impossible” happens with distressing regularity.
8.6 The Debugger 323
An assumption that you believe to be true is called an assertion. Testing for asser-
tions is a powerful quality control mechanism. It is useful to build a special class for
testing assertions. Here is an example of such a class:
public class Assertion
{ public static void check(boolean b)
{ if (!b)
{ System.out.println("Assertion failed.");
// construct a Throwable object to get a stack trace
new Throwable().printStackTrace();
System.exit(1);
}
}
}
This particular implementation prints a stack trace, a printout of all pending methods,
if it finds the assertion has been violated. Here is a typical stack trace:
java.lang.Exception
at Assertion.check(Assertion.java:6)
at DentalRobot.computeIntersection(DentalRobot.java:37)
at RobotTest.testDrill(RobotTest.java:93)
at RobotTest.main(RobotTest.java:10)
The line just below Assertion.check indicates the culprit.
Here is a typical assertion check:
public void computeIntersection()
{ . . .
double y = r * r - (x - a) * (x - a);
Assertion.check(y >= 0);
. . .
root = Math.sqrt(y);
. . .
}
In this program excerpt, the programmer expects that the quantity y can never be
negative. When the assertion is correct, no harm is done, and the program works in
the normal way. If, for some reason, the assertion fails, then the programmer would
rather have the program terminate than go on, compute the square root of a negative
number, and cause greater harm later.
Assertions are different from trace messages in one important respect. You can
leave them in your code when testing is complete.
8.6 The Debugger
As you have undoubtedly realized by now, computer programs rarely run perfectly
the first time. At times, it can be quite frustrating to find the bugs. Of course, you
can insert trace messages to show the program flow as well as the values of key
variables, run the program, and try to analyze the printout. If the printout does not
clearly point to the problem, you may need to add and remove print commands and
run the program again. That can be a time-consuming process.
Modern development environments contain special programs, called debuggers,
that help you locate bugs by letting you follow the execution of a program. You
can stop and restart your program and see the contents of variables whenever your
program is temporarily stopped. At each stop, you have the choice of what variables
to inspect and how many program steps to run until the next stop.
Some people feel that debuggers are just a tool to make programmers lazy. Admit-
tedly some people write sloppy programs and then fix them up with the debugger,
but the majority of programmers make an honest effort to write the best program
they can before trying to run it through the debugger. These programmers realize
that the debugger, while more convenient than print statements, is not cost-free. It
does take time to set up and carry out an effective debugging session.
In actual practice, you cannot avoid using the debugger. The larger your programs
get, the harder it is to debug them simply by inserting print statements. You will
find that the time investment to learn about the debugger is amply repaid in your
programming career.
Random Fact 8.1

◆ The First Bug
◆
◆ According to legend, the first bug was one found in 1947 in the Mark II, a huge electromechan-
◆ ical computer at Harvard University. It really was caused by a bug—a moth was trapped in a
◆ relay switch. Actually, from the note that the operator left in the log book next to the moth
◆ (see Figure 2), it appears as if the term “bug” had already been in active use at the time.
◆
◆
◆
◆
◆
◆
◆
◆
◆
◆
◆
◆
◆
◆
◆ Figure 2
◆
◆
◆ The First Bug
◆
◆
◆ The pioneering computer scientist Maurice Wilkes wrote: “Somehow, at the Moore School
◆ and afterwards, one had always assumed there would be no particular difficulty in getting
◆ programs right. I can remember the exact instant in time at which it dawned on me that a
◆ great part of my future life would be spent finding mistakes in my own programs.”
8.6.1 Using a Debugger

Like compilers, debuggers vary widely from one system to another. On some sys-
tems they are quite primitive and require you to memorize a small set of arcane
commands; on others they have an intuitive window interface.
You will have to find out how to prepare a program for debugging and how to start
the debugger on your system. If you use an integrated development environment,
which contains an editor, compiler, and debugger, this step is usually very easy. You
just build the program in the usual way and pick a menu command to start debugging.
On many UNIX systems, you must manually build a debug version of your program
and invoke the debugger.
Once you have started the debugger, you can go a long way with just three debug-
ging commands: “run until this line”, “step to next line”, and “inspect variable”. The
names and keystrokes or mouse clicks for these commands differ widely between
debuggers, but all debuggers support these basic commands. You can find out how,
either from the documentation or a lab manual, or by asking someone who has used
the debugger before.
The “run until this line” command is the most important. Many debuggers show
you the source code of the current program in a window. Select a line with the mouse
or cursor keys. Then hit a key or select a menu command to run the program to the
selected line. On other debuggers, you have to type in a command or a line number.
In either case, the program starts execution and stops as soon as it reaches the line
you selected (see Figure 3). Of course, you may have selected a line that will not be
reached at all during a particular program run. Then the program terminates in the
normal way. The very fact that the program has or has not reached a particular line
can be valuable information.
The “step to next line” command executes the current line and stops at the next
program line. Once the program has stopped, you can look at the current values
of variables. Again, the method for selecting the variables differs among debuggers.
Some debuggers always show you a window with the current local variables (see
Figure 4). On other debuggers you issue a command such as “inspect variable” and
type in or click on the variable. The debugger then displays the contents of the vari-
able. If all variables contain what you expected, you can run the program until the
next point where you want to stop.
When inspecting objects, you often need to give a command to “open up” the
object, for example by clicking on a tree node. Once the object is opened up, you see
its instance variables.
Finally, when the program has completed running, the debug session is also fin-
ished. You can no longer inspect variables. To run the program again, you may be
Figure 3
Debugger Stopped at Selected Line
Figure 4
Inspecting Variables
able to reset the debugger, or you may need to exit the debugging program and start
over. Details depend on the particular debugger.
Productivity Hint 8.2

◆ Inspect this in the Debugger
◆
It is a good idea to inspect the this object frequently. Then you know the state of the implicit
◆
parameter of the method through which you are currently tracing. If your debugger has a
◆ “watch” window where you can place values that you always want to have displayed, add
◆ this to the watch window!
8.6.2 A Sample Debugging Session

Consider the following program, whose purpose is to compute all prime numbers
up to a number n. An integer is defined to be prime if it is not evenly divisible by any
number except by 1 and itself. Also, mathematicians find it convenient not to call 1
a prime. Thus, the first few prime numbers are 2, 3, 5, 7, 11, 13, 17, 19.
Program PrimeBug.java
public class PrimeBug // 1
{ /** // 2
Tests whether an integer is prime // 3
@param n any positive integer // 4
@return true iff n is a prime // 5
**/ // 6
// 7
public static boolean isPrime(int n) // 8
{ if (n == 2) return true; // 9
if (n % 2 == 0) return false; // 10
int k = 3; // 11
while (k * k < n) // 12
{ if (n % k == 0) return false; // 13
k = k + 2; // 14
} // 15
return true; // 16
} // 17
// 18
public static void main(String[] args) // 19
{ ConsoleReader console // 20
= new ConsoleReader(System.in); // 21
System.out.println // 22
("Please enter the upper bound:"); // 23
int n = console.readInt(); // 24
// 25
for (int i = 1; i <= n; i = i + 2) // 26
{ if (isPrime(i)) // 27
System.out.println(i); // 28
} // 29
} // 30
} // 31
We numbered the lines so that we can refer to them in this example. Probably your
debugger won’t actually number them.
When you run this program with an input of 10, then the output is
1
3
5
7
9
That is not very promising; it looks as if the program just prints all odd numbers. Let
us find out what it does wrong, by using the debugger. (Actually, for such a simple
program, it is easy to correct mistakes simply by looking at the faulty output and the
program code. However, we want to learn to use the debugger.)
Let us first go to line 27. On the way, the program will stop to read the input into
n. Supply the input value 10.
int n = console.readInt(); // 24
// 25
for (int i = 1; i <= n; i = i + 2) // 26
{ if (isPrime(i)) // 27
System.out.println(i); // 28
} // 29
Now we wonder why the program treats 1 as a prime. Go to line 9.
int k = 3; // 11
while (k * k < n) // 12
k = k + 2; // 14
} // 15
return true; // 16
} // 17
Convince yourself that the parameter n of isPrime is currently 1, by inspecting its
value in the debugger. Then execute the “run to next line” command. You will notice
that the program goes to lines 10 and 11 and then directly to line 16.
int k = 3; // 11
while (k * k < n) // 12
k = k + 2; // 14
} // 15
return true; // 16
} // 17
Inspect the value of k. It is 3, and therefore the while loop was never entered. It
looks as though the isPrime method needs to be rewritten to treat 1 as a special
case.
Next, we would like to know why the program doesn’t print 2 as a prime even
though the isPrime method does recognize that 2 is a prime, whereas all other even
numbers are not. Go again to line 9, the next call of isPrime. Inspect n; you will note
that n is 3. Now it becomes clear: The for loop in main tests only odd numbers. The
main should either test both odd and even numbers or, better, just handle 2 as a
special case.
Finally, we would like to find out why the program believes 9 is a prime. Go again
to line 9 and inspect n; it should be 5. Repeat that step twice until n is 9. (With some
debuggers, you may need to go from line 9 to line 10 before you can go back to
line 9.) Now use the “run to next line” command repeatedly. You will notice that the
program again skips past the while loop; inspect k to find out why. You will find that
k is 3. Look at the condition in the while loop. It tests whether k * k < n. Now k
* k is 9 and n is also 9, so the test fails. Actually, it does make sense to test divisors
only up to 冪n; if n has any divisors except 1 and itself, at least one of them must
be less than 冪n. However, actually that isn’t quite true; if n is a perfect square of
a prime, then its sole nontrivial divisor is equal to 冪n. That is exactly the case for
9 ⳱ 32 .
By running the debugger, we have now discovered three bugs in the program:
◆ isPrime falsely claims 1 to be a prime.

◆ main doesn’t handle 2.
◆ The test in isPrime should be while (k * k <= n).
Here is the improved program:
Program GoodPrime.java
public class GoodPrime

{ /**
Tests whether an integer is a prime
@param n any positive integer
@return true iff n is a prime
*/
public static boolean isPrime(int n)

{ if (n == 1) return false;
if (n == 2) return true;
if (n % 2 == 0) return false;
int k = 3;
while (k * k <= n)
{ if (n % k == 0) return false;
k = k + 2;
}
return true;
}
public static void main(String[] args)

{ ConsoleReader console
= new ConsoleReader(System.in);
System.out.println
("Please enter the upper bound:");
int n = console.readInt();
if (n >= 2) System.out.println(2);
for (int i = 1; i <= n; i = i + 2)
{ if (isPrime(i))
System.out.println(i);
}
}
}
Is our program now free from bugs? That is not a question the debugger can answer.
Remember: Testing can show only the presence of bugs, not their absence.
8.6.3 Stepping through a Program

You have learned how to run a program until it reaches a particular line. Variations
of this strategy are often useful.
There are two methods of running the program in the debugger. You can tell it to
run to a particular line; then it gets speedily to that line, but you don’t know how
it got there. You can also single-step with the “run to next line” command. Then you
know how the program flows, but it can take a long time to step through it.
Actually, there are two kinds of single-stepping commands, often called “step
over” and “step into”. The “step over” command always goes to the next program
line. The “step into” command steps into method calls. For example, suppose the
current line is
String name = console.readLine();
System.out.println("Hello " + name + "! I'm glad to meet you.");
When you “step over” method calls, you get to the next line:
String name = console.readLine();
System.out.println("Hello " + name + "! I'm glad to meet you.");
However, if you “step into” method calls, you enter the first line of the readLine
method.
public String readLine()
{ String inputLine = "";
. . .
}
You should step into a method to check whether it carries out its job correctly. You
should step over a method if you know it works correctly.
If you single-step past the last line of a method, either with the “step over” or the
“step into” command, you return to the line in which the method was called.
You should not step into system methods like println. It is easy to get lost in
them, and there is no benefit in stepping through system code. If you do get lost,
there are three ways out. You can just choose “step over” until you are finally again
in familiar territory. Many debuggers have a command “run until method return”
that executes to the end of the current method, and then you can select “step over”
to get out of the method. Finally, most debuggers can show you a call stack: a listing
of all currently pending method calls. On the one end of the call stack is main, on the
other the method that is currently executing (see Figure 5). Sometimes, the debug-
ger shows you multiple threads—you should ignore those threads that don’t have
the methods that you are debugging. By selecting another method in the middle
of the call stack, you can jump to the code line containing that method call. Then
move the cursor to the next line and choose “run until this line”. That way, you get
out of any nested morass of method calls.
The techniques you saw so far let you trace through the code in various incre-
ments. All debuggers support a second navigational approach: You can set so-called
breakpoints in the code. Breakpoints are set at specific code lines, with a command
“add breakpoint here”; again, the exact command depends on the debugger. You can
set as many breakpoints as you like. When the program reaches any one of them,
execution stops and the breakpoint that causes the stop is displayed. Breakpoints
stay active until you remove them.
Breakpoints are particularly useful when you know the point at which your pro-
gram starts doing the wrong thing. You can set a breakpoint, have the program run at
full speed to the breakpoint, and then start tracing slowly to observe the program’s
behavior.
Figure 5
Call Stack Display

Some debuggers let you set conditional breakpoints. A conditional breakpoint stops
the program only when a certain condition is met. You could stop at a particular line
only if a variable n has reached 0, or if that line has been executed for the twentieth
time. Conditional breakpoints are an advanced feature that can be indispensable in
knotty debugging problems.
Common Error 8.1

◆ Tracing through Recursive Methods
◆
◆ When you set a breakpoint in a recursive method, the program stops as soon as that pro-
◆ gram line is encountered in any call to the recursive method. Suppose you want to debug the
◆ factorial method:
◆
public static int factorial(int n) // 1
◆
{ if (n < 0) throw new IllegalArgumentException(); // 2
◆
◆ if (n == 0) // 3
◆ return 1; // 4
◆ else // 5
◆ { int f = factorial(n - 1); // 6
◆ int result = n * f; // 7
◆ return result; // 8
◆ } // 9
◆ } //10
◆
◆ Suppose you inspect n in line 2, and it is 4. You tell the debugger to run until line 7. When you
◆ inspect n again, its value is 1! That makes no sense. There was no instruction that changed the
◆ value of n! Is that a bug with the debugger?
◆ No. The program stopped in the first recursive call to factorial that reached line 7. If you
◆ are confused, try this and look at the call stack. You will see that four calls to factorial are
◆ pending.
◆ You can debug recursive methods with the debugger. You just need to be particularly care-
◆ ful, and watch the call stack frequently.
8.7 Debugging Strategies
Now you know about the mechanics of debugging, but all that knowledge may still
leave you helpless when you fire up the debugger to look at a sick program. There
are a number of strategies that you can use to recognize bugs and their causes.
8.7.1 Reproduce the Error

As you test your program, you notice that your program sometimes does something
wrong. It gives the wrong output, it seems to print something completely random, it
8.7 Debugging Strategies 333
goes in an infinite loop, or it crashes. Find out exactly how to reproduce that behavior.
What numbers did you enter? Where did you click with the mouse?
Run the program again; type in exactly the same answers and click with the mouse
on the same spots (or as close as you can get). Does the program exhibit the same
behavior? If so, then you are ready to fire up the debugger to study this particular
problem. Debuggers are good for analyzing particular failures. They aren’t terribly
useful for studying a program in general.
8.7.2 Divide and Conquer

Now that you have a particular failure, you want to get as close to the failure as pos-
sible. Suppose your program dies with a division by 0. Since there are many division
operations in a typical program, it is often not feasible to set breakpoints to all of
them. Instead, use a technique of divide and conquer. Step over the methods in main,
but don’t step inside them. Eventually, the failure will happen again. Now you know
which method contains the bug: It is the last method that was called from main be-
fore the program died. Restart the debugger and go back to that line in main, then
step inside that method. Repeat the process.
Eventually, you will have pinpointed the line that contains the bad division.
Maybe it is completely obvious from the code why the denominator is not correct.
If not, you need to find the location where it is computed. Unfortunately, you can’t
go back in the debugger. You need to restart the program and move to the point
where the denominator computation happens.
8.7.3 Know What Your Program

Should Do
The debugger shows you what the program does do. You must know what the pro-
gram should do, or you will not be able to find bugs. Before you trace through a loop,
ask yourself how many iterations you expect the program to make. Before you in-
spect a variable, ask yourself what you expect to see. If you have no clue, set aside
some time and think first. Have a calculator handy to make independent computa-
tions. When you know what the value should be, inspect the variable. This is the
moment of truth. If the program is still on the right track, then that value is what
you expected, and you must look further for the bug. If the value is different, you
may be on to something. Double-check your computation. If you are sure your value
is correct, find out why your program comes up with a different value.
In many cases, program bugs are the result of simple errors such as loop termi-
nation conditions that are off by 1. Quite often, however, programs make computa-
tional errors. Maybe they are supposed to add two numbers, but by accident the code
was written to subtract them. Unlike your calculus instructor, programs don’t make
a special effort to ensure that everything is a simple integer. You will need to make
some calculations with large integers or nasty floating-point numbers. Sometimes
these calculations can be avoided if you just ask yourself, “Should this quantity be
positive? Should it be larger than that value?” Then inspect variables to verify those
theories.
8.7.4 Look at All Details

When you debug a program, you often have a theory about what the problem is.
Nevertheless, keep an open mind and look around at all details. What strange mes-
sages are displayed? Why does the program take another unexpected action? These
details count. When you run a debugging session, you really are a detective who
needs to look at every clue available.
If you notice another failure on the way to the problem that you are about to pin
down, don’t just say, “I’ll come back to it later”. That very failure may be the original
cause for your current problem. It is better to make a note of the current problem,
fix what you just found, and then return to the original mission.
8.7.5 Understand Each Error Before You

Fix It
Once you find that a loop makes too many iterations, it is very tempting to apply
a “Band-Aid” solution and subtract 1 from a variable so that the particular problem
doesn’t appear again. Such a quick fix has an overwhelming probability of creating
trouble elsewhere. You really need to have a thorough understanding of how the
program should be written before you apply a fix.
It does occasionally happen that you find bug after bug and apply fix after fix, and
the problem just moves around. That usually is a symptom of a larger problem with
the program logic. There is little you can do with the debugger. You must rethink the
program design and reorganize it.
Random Fact 8.2

◆ The Therac-25 Incidents
◆
◆ The Therac-25 is a computerized device to deliver radiation treatment to cancer patients (see
◆ Figure 6). Between June 1985 and January 1987, several of these machines delivered serious
◆ overdoses to at least six patients, killing some of them and seriously maiming the others.
◆ The machines were controlled by a computer program. Bugs in the program were directly
◆ responsible for the overdoses. According to [1], the program was written by a single program-
◆ mer, who had since left the manufacturing company producing the device and could not be
◆ located. None of the company employees interviewed could say anything about the educa-
◆ tional level or qualifications of the programmer.
◆ The investigation by the federal Food and Drug Administration (FDA) found that the pro-
◆ gram was poorly documented and that there was neither a specification document nor a formal
◆ test plan. (This should make you think. Do you have a formal test plan for your programs?)
◆ The overdoses were caused by an amateurish design of the software that had to control
◆ different devices concurrently, namely the keyboard, the display, the printer, and of course
◆ the radiation device itself. Synchronization and data sharing between the tasks were done in
◆ an ad hoc way, even though safe multitasking techniques were known at the time. Had the
◆ programmer enjoyed a formal education that involved these techniques, or taken the effort
◆ to study the literature, a safer machine could have been built. Such a machine would have
Chapter Summary 335
◆ Room Motion Therapy room

◆ emergency power switch intercom
◆ TV switches
camera
◆
◆ Therac-25 unit
◆ Beam
◆ on/off light
◆
Door
◆
interlock
◆ switch
◆
◆ Room
◆ emergency
◆ switch
◆
◆ Treatment table
Display
◆ terminal
◆ TV monitor Printer Turntable
◆ Motion enable Control position
◆ switch (footswitch) console monitor
◆
◆ Figure 6
◆
◆
◆ Typical Therac-25 Facility
◆
◆ probably involved a commercial multitasking system, which might have required a more ex-
◆ pensive computer.
◆
The same flaws were present in the software controlling the predecessor model, the
◆
Therac-20, but that machine had hardware interlocks that mechanically prevented overdoses.
◆
The hardware safety devices were removed in the Therac-25 and replaced by checks in the
◆
◆ software, presumably to save cost.
◆ Frank Houston of the FDA wrote in 1985 [1]: “A significant amount of software for life-
◆ critical systems comes from small firms, especially in the medical device industry; firms that
◆ fit the profile of those resistant to or uninformed of the principles of either system safety or
◆ software engineering.”
◆ Who is to blame? The programmer? The manager who not only failed to ensure that the
◆ programmer was up to the task but also didn’t insist on comprehensive testing? The hospitals
◆ that installed the device, or the FDA, for not reviewing the design process? Unfortunately,
◆ even today there are no firm standards of what constitutes a safe software design process.
Chapter Summary
1. Use unit tests to test each key method in isolation. Write a test harness to feed
test data to the method being tested. Select test cases that cover each branch of the
method.
2. Use assertions to document assumptions that you are making in your programs.
3. You can debug a program by inserting trace printouts, but that gets quite tedious
for even moderately complex debugging situations. You should learn to use the de-
bugger.
4. You can make effective use of the debugger by mastering just three commands:
“run until this line”, “step to next line”, and “inspect variable”. The names and
keystrokes or mouse clicks for these commands differ between debuggers.
5. There are three windows in the debugger to which you should pay attention:
the window that shows your source code and, in particular, the currently executed
instruction; the window that shows the values of program variables; and the call
stack, which shows which method calls are currently “stacked up”.
6. Use the “divide-and-conquer” technique to locate the point of failure of a program.

Inspect variables and compare their actual contents against the values that you know
they should have.
7. The debugger can be used only to analyze the presence of bugs, not to show that
a program is bug-free.
Further Reading
[1] Nancy G. Leveson and Clark S. Turner, “An Investigation of the Therac-25 Accidents,” IEEE
Computer, July 1993, pp. 18–41.
Classes, Objects, and Methods

Introduced in This Chapter
java.lang.Throwable
printStackTrace
Review Exercises
Exercise R8.1. Define the terms unit test and test harness.
Exercise R8.2. If you want to test a program that is made up of four different meth-
ods, one of which is main, how many test harness programs do you need?
Exercise R8.3. What is an oracle?
Exercise R8.4. Define the terms regression testing and test suite.
Exercise R8.5. What is the debugging phenomenon known as “cycling”? What can
you do to avoid it?
Review Exercises 337
Exercise R8.6. The arc sine function is the inverse of the sine function. That is,
y ⳱ arcsin x if x ⳱ sin y. It is defined only if ⫺1 ⱕ x ⱕ 1. Suppose you need to
write a Java method to compute the arc sine. List five positive test cases with their
expected return values and two negative test cases with their expected outcomes.
Exercise R8.7. What is a program trace? When does it make sense to use a program
trace, and when does it make more sense to use a debugger?
Exercise R8.8. Explain the differences between these debugger operations:
◆ Stepping into a method

◆ Stepping over a method
◆ Running until the current line

◆ Setting a breakpoint to the current line
◆ Inspecting a variable
◆ Watching a variable
Exercise R8.11. What is a call stack display in the debugger? Give two debugging
scenarios in which the call stack display is useful.
Exercise R8.12. Explain in detail how to inspect the information stored in a

Point2D.Double object in your debugger.
Exercise R8.13. Explain in detail how to inspect the string stored in a String object
in your debugger.
Exercise R8.14. Explain in detail how to use your debugger to inspect a string stored
in a BankAccount object.
Exercise R8.15. Explain the “divide-and-conquer” strategy to get close to a bug in

the debugger.
Exercise R8.16. True or false:
◆ If a program has passed all tests in the test suite, it has no more bugs.
◆ If a program has a bug, that bug always shows up when running the program
through the debugger.
◆ If all methods in a program were proven correct, then the program has no
bugs.
Programming Exercises
Exercise P8.1. The arc sine function is the inverse of the sine function. That is,
y ⳱ arcsin x
if
x ⳱ sin y
For example,
arcsin(0) ⳱ 0
arcsin(0.5) ⳱ ␲ /6
arcsin( 冪2/2) ⳱ ␲ /4
arcsin( 冪3/2) ⳱ ␲ /3
arcsin(1) ⳱ ␲ /2
arcsin(⫺1) ⳱ ␲ /2
The arc sine is defined only for values between ⫺1 and 1. This function is also often
called sin⫺1 x. Note, however, that it is not at all the same as 1/ sin x. There is a
Java standard library method to compute the arc sine, but you should not use it for
this exercise. Write a Java method that computes the arc sine from its Taylor series
expansion
arcsin x ⳱ x Ⳮ x 3 /3! Ⳮ x 5 ⭈ 32 /5! Ⳮ x 7 ⭈ 32 ⭈ 52 /7! Ⳮ x 9 ⭈ 32 ⭈ 52 ⭈ 72 /9! Ⳮ ⭈⭈⭈
You should compute the sum until a new term is ⬍ 10⫺6 . This method will be used
in subsequent exercises.
Exercise P8.2. Write a simple test harness for the arcsin method that reads floating-
point numbers from standard input and computes their arc sines, until the end of the
input is reached. Then run that program and verify its outputs against the arc sine
function of a scientific calculator.
Exercise P8.3. Write a test harness that automatically generates test cases for the
arcsin method, namely numbers between ⫺1 and 1 in a step size of 0.1.
Exercise P8.4. Write a test harness that generates 10 random floating-point numbers
between ⫺1 and 1 and feeds them to arcsin.
Exercise P8.5. Write a test harness that automatically tests the validity of the
arcsin method by verifying that Math.sin(arcsin(x)) is approximately equal to
x. Test with 100 random inputs.
Programming Exercises 339
Exercise P8.6. The arc sine function can be computed from the arc tangent function,
according to the formula
冢
arcsin x ⳱ arctan x/冪1 ⫺ x 2 冣
Use that expression as an oracle to test that your arc sine method works correctly.
Test your method with 100 random inputs and verify against the oracle.
Exercise P8.7. The domain of the arc sine function is ⫺1 ⱕ x ⱕ 1. Supply an ex-
ception to your arcsin method that ensures that the input is valid. Test your method
by computing arcsin(1.1). What happens?
Exercise P8.8. Place trace messages into the loop of the arc sine method that com-
putes the power series. Print the value of n, the value of the current term, and the
current approximation to the result. What trace output do you get when you com-
pute arcsin(0.5)?
Exercise P8.9. Add trace messages to the beginning and end of the isPrime method
in the buggy prime program. Also put a trace message as the first statement of the
while loop in the isPrime method. Print relevant values such as method parameters,
return values, and loop counters. What trace do you get when you compute all primes
up to 20? Are the messages informative enough to spot the bug?
Exercise P8.10. Run a test harness of the arcsin method through the debugger.
Step inside the computation of arcsin(0.5). Step through the computation until
the x 7 term has been computed and added to the sum. What is the value of the
current term and of the sum at this point?
Exercise P8.11. Run a test harness of the arcsin method through the debugger.
Step inside the computation of arcsin(0.5). Step through the computation until
the x n term has become smaller than 10⫺6 . Then inspect n. How large is it?
Exercise P8.12. The following method has a subtle bug:

class MathAlgs
{ public static double sqrt(double a)
{ if (a < 0) return 0;
double xnew = a / 2;
double xold;
do
{ xold = xnew;
}
while (!Numeric.approxEqual(xnew, xold));
return xnew;
}
}
Create a series of test cases to flush out the bug. Then run a debugging session to
find it. Run the debugger to the line in which the bug manifests itself. What are the
values of all local variables at that point?
Exercise P8.13. Write a program that tests the recursive factorial method from
Chapter 7. Compute factorial(6). Step inside recursive calls until you arrive at
factorial(3). Then display the call stack. Which calls are currently pending?
Exercise P8.14. Find all bugs in the following version of a factorial method. De-
scribe how you found the bugs.
class Numeric
{ public int factorial(int n)
{ int p = 1;
int i = n;
// compute p = n * (n-1) * (n-2) * . . . * 2
while (i >= 2);

{ i--;
p = p * i;
}
return i;
}
}

Testing and Debugging: Chapter Goals

Uploaded by

Copyright:

Available Formats

Testing and Debugging: Chapter Goals

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Testing and Debugging: Chapter Goals

Uploaded by

Copyright:

Available Formats

Testing and Debugging

◆ To understand the principles of test case selection and evaluation

◆ To learn how to use assertions

◆ To become familiar with the debugger

◆ To learn strategies for effective debugging

8.1 Unit Tests

Approximation of the Square Midpoint

System.out.println("square root of " + x

public class SqrtTest3

double x = 1.0E6 * generator.nextDouble();

System.out.println("square root of " + x

8.2 Selecting Test Cases

8.3 Test Case Evaluation

public class SqrtTest4

double x = 1.0E6 * generator.nextDouble();

// check that test value fulfills square property

public class SqrtTest5

double x = 1.0E6 * generator.nextDouble();

// compare against oracle

if (Numeric.approxEqual(y, Math.pow(x, 0.5)))

Productivity Hint 8.1

8.4 Program Traces

8.5 Asserting Conditions

Programs often contain implicit assumptions. For example, denominators need to be

8.6 The Debugger

Random Fact 8.1

8.6.1 Using a Debugger

Debugger Stopped at Selected Line

Productivity Hint 8.2

8.6.2 A Sample Debugging Session

◆ isPrime falsely claims 1 to be a prime.

◆ The test in isPrime should be while (k * k <= n).

Here is the improved program:

public class GoodPrime

public static boolean isPrime(int n)

public static void main(String[] args)

8.6.3 Stepping through a Program

Call Stack Display

Common Error 8.1

8.7 Debugging Strategies

8.7.1 Reproduce the Error

8.7.2 Divide and Conquer

8.7.3 Know What Your Program

8.7.4 Look at All Details

8.7.5 Understand Each Error Before You

Random Fact 8.2

◆ Room Motion Therapy room

6. Use the “divide-and-conquer” technique to locate the point of failure of a program.

Classes, Objects, and Methods

Exercise R8.3. What is an oracle?

Exercise R8.8. Explain the differences between these debugger operations:

◆ Stepping into a method

Exercise R8.9. Explain the differences between these debugger operations:

◆ Running until the current line

Exercise R8.10. Explain the differences between these debugger operations:

Exercise R8.12. Explain in detail how to inspect the information stored in a

Exercise R8.15. Explain the “divide-and-conquer” strategy to get close to a bug in

Exercise R8.16. True or false: