check for expected results.

If you get a different answer, that is

a failure. An error is more catastrophic it indicates an error
condition that you didn’t check for.

Unittasting
I recommend that developers write their own unit tests, one per
class. The framework supports the writing of suites of tests,

Simple whkh can be attached to a class. I recommend that all classes

respond to the message testSuite, returning a suite containing
the unit tests. I recommend that developers spend 25–50% of

Smalltalk their time developing tests.

Integration
tasting

KENT BECK
testing 1 recommend that an independent tester write integration tests.
Where should the integration tests go? The recent movement
of user-interface frameworks to better facilitate programmatic
access provides one answer-drive the user interface, but do it

Y
ou can’t argue with inspiration (or deadlines). I started with the tests. In VisualWorks (the dialect used in the imple-
to write the final column in the sequence about using mentation below), you can open an ApplicationModel and
patterns for design, but what came out was this. lt begin stuffing values into its ValueHolders, causing all sorts of
describes some work I have been doing with a framework that havoc, with very little trouble.
takes the tedium out of vvriting tests. 1’11get back to the pat-
tern sti in the next issue. Running tasts
SmsIltalk has suffered because it lacks a testing culture. This One final bit of philosophy. It is tempting to set up a bunch of
column describes a simple testing strategy and a framework to test data, then run a bunch of tests, then clean up. In my experi-
support
.. it. The testing strate~—. and framework are not intended ence, thk procedure always causes more problems hat it is
to be complete solutions, but, rather, are intended to be starting worth. Tests end up interacting with one another, and a failure
points from which industrial strength tools and procedures can in one test can prevent subsequent tests from running. The test-
be constructed. ing ilamework makes it easy to set up a common set of test
The article is divided into four sections: data, but the data will be created and thrown away for each test.
- Philomphy. Describes the phdosophy of writing and running The potential performance problems with this approach should-
tests embodied by the framework. Read this section for gen- n’t be a big deal, because suites of tests can run unobserved.
eral background.
● Framework. A literate program version of the testing frame- FRAMEWORK
work. Read this for in-depth knowledge of how the frame- The smallest unit of testing is the TestCase. When a TestCase
work operates. runs, it sets up its test data, runs a test method, then discards
● Example. An example of using the testing framework to test the test data. Because many cases may want to share the same
part of the methods in Set. test data, TestCase chooses which method to run with the
● Cookbook.A simple cookbook for writing yow own tests. instance variable selector, which will be pefiormed to run the
test method.
PHILOSOPHY class:TestCase
The most radical philosophy espoused here is a rejection of superclass:Object
user-interface-based tests. In my experience, tests based on user instance variables: selector
interface scripts are too brittle to be useful. Testers spend more classvariable: FailedChecMignal
time keeping the tests up to date and tracking down false fail- TestCases are always created with a selector. The class method
ures and false successes than they do writing new tests. selector: ensures thk.
My solution is to write the tests (and check results) in TestCaseclass>>selecto~asyrrrbol
Smslltalk. Although thk approach has the disadvantage that ‘self new setselecto~ a$nnbol
your testers need to be able to write simple Smalkslk programs, TestCase>>setSelectora$nnbol
the resulting tests are much more stable. selector:= a$nrrbol
The simplest way to run a TestCase is just to send it the message
Failurss and errors run. run invokes the set up code, performs the selector, then
The framework distinguishes between failures and errors. A runs the tear-down code. Notice that the tear-down code is run
failure is an anticipated problem. When you write tests, you regardless of whether there is an error in performing the test.
TestCase>>run
self setUp.
[selfperforrnTest]
vahmNorvOrOnUnwindDo:
[selftearDown]!

16 The Smalltalk Repoti

Subclasses of TestCase are errpected to create and destroy test The default TestResult is constmcted by the TestSuite:
fktures in setUp and tearDown, respectively. TestCase itself pro- TestSuite>>defaultTestResuk
vides stubs for these methods that do nothing: ‘self dei%ultTeatResuMlass test self
TestCase~>setUp TestSuite>>defaultTestResuMlass
‘Runwhatwer codeyou need to get readyfor the test to run.” ‘TestResuh
TestCase~>tearDown A TestResult is always created on a TestSuite:
“Releasewhatwer resources you used for the test.” TestResultclass>>testiaTest
PerformTest just performs the selectoc “selfnewsetTest:aTest
TestCaae~~erformTest TestResul@>setTest aTest
self perform selector testName:=aTestname.
A single TestCase is hardly ever interesting once you have got- failures:=OrderedCollefionnew.
ten it running. In production, you will want to run suites of errors:=OrderedCollection new
TestCases. Aggregating TestCases is the job of the TestSuite: TestResults are timestamped by sending them the messages
Class:TestSuite start and stop:
super class: Object TestResul*>start
instance variables: name testCases sttie :=DatedateAndTinwNoW
When a TestSuite is created, it is initiahzed to prepare it to TestResulb>stop
hold TestCases. TestSuites are also named, so you can identi@ stopl%ne:=DatedateAndTimeNow
them even if you have, for example, read them in from sec- When a TestSuite runs for a given TestResult, it simply runs
ondary storage: each of its TestCases with that TestResult
TestSuiteclase>rwmed:aStcing TestSuite>~run:aTestResult
“selfnewsethne: aString testCasesdo: [:each I eachrum aTestResuk]
TestSuite>>setName: a.shing Because the selector run: is the same in both TestSuite and
name:= aShing. TestCase, it is trivial to construct TestSuites which contain other
testCases:=OrderedCollectionnew TestSuites, instead of or in addition to containing TestCases.
TestSuites have an accessing method for their name in anticipa- When a TestCase runs for a given TestResult, it should either
tion of user intefiaces that will have to display them: silently run correctly, add an error to the TestResult, or add a
TestSuite>>name failure to the TestResult. Catching errora is simple-use the
“name system-supplied errorSignal. Catching failures must be support-
TestSuites have protocol for adding one or more TestCases: ed by the TestCase itself Fkst, we need a SignaL
TestSuite>>addTestCaae: aTestCase TestCaseclas->initialize
testCasesadd aTeatCase FailedCheckSigml:=selferrorSignalnewsignal
TestSuite>>addTestCases: aCollection notierString: ‘Checkfailed -‘;
aColle&ondo: [:each I selfaddTestCaseeach] nameclass: self message: #checkSignal
When you run a TestSuite, you’d like all of its TestCases to run. Now we need a way of accessing it
It’s not quite that simple, though. Running a suite is different TestCase>>failedCheckSignal
from running a single test case. For example, if you have a suite ‘FailedCheckSignal
that represents the acceptance test for your application, after it Now, when the TestCase runs with a TestResult, it must catch
runs, you’d like to know how long the suite ran and which of errors and failures and inform the TestResult, and it must run
the cases had problems. This is information you would like to the tearDown code regardless of whether the test executed cor-
be able to store away for fhture reference. rectly. This results in the ugliest method in the framework
Te.stResuk solves this problem. Running a TestCase just exe- because there are two nested error handlers and
cutes the test method and returns the TestCase. Running a valueNowOrOnUnwindDo: in one method:
TestSuite, however, returns a TestResult that records the infor- TestCase>~rurraTestResuk
mation described above- the start and stop times of the run, the self setUp.
name of the suite, and any failures or errors: [self errorSignal
Class:TestResult handle: [:esr I aTestResukerro~ ex errorSting ti. sew
superclass:Object do: [self failedCheckSigml
instancevariables:startTimestopllmetestNamet%ilureserrors handle: [:ex I aTesLResukfailure: ex errorStiing im selfl
When you run a TestSuite, it creates a TestResult, which is time do: [self performTest]]]
stamped before and after the TestCases are run: valueNowOrOnUnwindD o:
TestSuite>Xon [selftearDown]
I result I When a TestResult is told that an error or failure happened, it
result:= selfdefauM’estResult. records that fact in one of its two collections. For simplicity,
result start. the record is just a two element array but it probably should
self run: result. be a first-class object with a time stamp and more details of
result stop. the blowup:
“result TestResulb>erroca.stringin: aTestCase

October 1994 17
 errors add: (Arraywith: aTestCasewitlu aSh’ing)
TestResul&>faihuc aShing im aTestCase
failures add (Arraywith aTestCasewith: XYzirrg)
The error case gets invoked if there is ever an uncaught error
(for example, message not understood) in the testing method.
How do the fadures get invoked? TestCase provides two meth-
ods that simpliij checking for fdure. The first, should: aBIoek, SetTestCase r
selector #testiHegal

u
signala a failure if the evaluation of aBlock returns false. The \
second, shouldnt: aBlock, does just the opposite. full
should:aBlock empty
tilock value ifTalse:[self failedCheckSignalraise]
shouldrmfllOCk
aBlockvalue Whua [self failedCheckSignalraise]
Testing methods wiU likely run some code, then check the
==7
startTime #(26 August 19942:09:00 am)
results inside should: and shoukink blocks.
stopTime #(26 August 19942:09:01 am)
EKAMPLE
Okay that’s how it worka, but how do you use it? Here’s a short
testName
I
failuresOrderedCollection ()
example that tests a few of the messages supported by Sets. errorsOrderedCollection ()
First we subclass TestCase, because we’ll always want a couple
of interesting Sets around to play with:
Class:SetTestCase
Figurs 1.
superclass TestCase
instancevariables:emptyfull
Now we need to initialize these variables, so we subclass setUp. functionfllty in Set. Writing tests for all the public methods in
SetTestCase>>setUp Set is a daunting task. However, as Hal Hddebrand told me
empty:= Set new. after using an e~lier version of this i%rnework, “If the underly-
full:= Set viiti #abc rvith: 5 ing objects don’t work, notlsing else matters. You have to write
Now we need a testing method. Let’s test to see if adding an the tests to make sure everything is working.”
element to a Set really works:
SetTestCaae>>testAdd COOKBOOK
empty add: 5. Here are simple steps you can follow to write your own tests:
self should: [emptyincludes: 5] 1. Scope the tests. Decide whether you want to test one object
Now we can run a test case by evaluating: or several cooperating objects. Subclass TestCase, prei%ring
(SetTestCaseselector #testAdd) run. the name of your test scope.
Here’s a case that uses shouldnt:. It reads ‘after removing 5 2. Create test ilxture. Find a few configurations of your objects
from W, full should include #abc and it shouldn’t include 5.” that will exercise the code you want to test. Add an instance
SetTestCase>>testRemove variable to your TestCase subclass for each configuration.
full remove: 5. Override setUp to create the configurations.
self should [fullinchrdex#abc]. 3. Write the test methods. Write a method for each distinct
selfshouldnk [fullincludes:5] test you want to perform. Use should and shouldnt wher-
Here’s one that makes sure an error is signaled if you try to do ever you can check for comect or incorrect results. As you
keyed access: write each method, test it by creating an instance of your
SetTestCase>*stIllegal TestCase subclass and sending it nm.
selfshould [selferrorSignal 4. Create the suite. Write a method in one of the classes you
handle: [:ex I true] do: [empty ak 5. false]] are testing that collects all of the TestCases into a TestSuite
Now we can put together a TestSuite. and returns it.
I suite I
suite:= TestSuite named: ‘SetTesta’. CONCLUSION
suite addTestCase:(SetTestCaseselecto~ #testAdd). This column has presented the philosophy and implementation
suite addTestCase:(SetTestCaseselecto~ #testRemove). of a simple testing stiate~ for Smalltalk. The strate~ has the
suite addTestCase:(SetTestCaseselectoc #testIllegal). advantages that it is simple, lightweight, and produces tests that
‘suite are stable. It is not complete or perfect, but it’s a whole lot better
Figure 1 showa an Object Explorer picture of the suite and of than no programmatic tests at all. As ahvays, if you have com-
the TestResult we get back when we run it. ments, plesse pass them on to me at
The test methods shown above only cover a fraction of the 70761 .1216@compuserve.com. q

18 The Smalltalk Report