Journal of Universal Computer Science, vol. 3, no. 10 (1997), 1126-1132
submitted: 2/10/97, accepted: 24/10/97, appeared: 28/10/97 Springer Pub. Co.
On Two Aspects of Improving Web-Based Training
H. Maurer
Graz University of Technology
hmaurer@iicm.edu
Abstract: In this paper we discuss two novel ideas to improve teaching and informa-
tion transfer between persons in general. The rst aspect is centered around proposing
to use the \Tamagotchi craze" for teaching purposes, the second deals with new ways of
unobtrusively collecting data on the subjective satisfaction of persons with information
and teaching material o ered on the Web.
Key Words: Training, teaching, learning, courseware, web-based training, evaluation
techniques
Category: J. 7, K 3, K.4
1 Harnessing the Tamagotchi Craze
The Tamagotchi craze that started in 1997 has driven many kids, teachers and
parents to the brink of desperation. An estimated 20 million Tamagotchis or
clones thereof were sold within one year, resulting in hundred millions of hours
spent on keeping the virtual beings happy. It is the aim of this paper to propose
to harness the appeal of caring for such virtual beings for teaching and training
purposes: rather than taking care of Tamagotchis by \feeding", \cleaning" and
\entertaining" them we propose to implement what we call VR-Friends (virtual
friends) that di er in three crucial ways from traditional Tamagotchis: (i) they
are kept happy if their owners answer questions correctly; (ii) they are implemented in software, not hardware and (iii) they live on the Web, in WWW
servers. In a way, VR-Friends can be considered to be special types of Avatars
(as often mentioned in connection with VR games), or intelligent agents ([7],[8]).
Psychologists have not been able yet to fathom the real reason why Tamagotchis have been successful in such an unprecedented way. But no matter
what the explanation is, it seems worthwhile to exploit what are seen as major reasons for the success for the education of primary and secondary school
children.
1.1
Introduction
VR-Friends are implemented as programs that run on WWW servers and can
be accessed by ordinary Web browsers from e.g. Netscape or Microsoft. They
come in various behavioral and knowledge domain classes in the sense that they
act as instructional agents for di erent ages and subject matters. Their success
as \teachers" is assumed to come from the fact that they are seen as live beings
that thrive only if continually \fed" with correct answers to questions they keep
asking. I.e. VR-Friends want to \learn" and in doing so - it is actually their
owners that learn!
In what follows we present the basic ideas of how such VR-Friends will work.
We do this in three subsections. In section 1.2 we describe a typical scenario. In
Maurer H.: On Two Aspects of Improving Web-Based Training
1133
section 1.3 we explain the main implementational aspects. And in section 1.4 we
argue why VR-Friends have indeed a chance to be successful yet why they may
fail, and what practical use and further developments might look like.
1.2 A typical scenario
So far, twelve year old Mark has shown very little interest in learning basic geographical facts: Although the importance of factual knowledge and rote memorising has been much challenged by educationalists over the last few decades there
is still a wide-spread feeling that some basic facts have to be stored inside our
brain (rather than on easily accessible storage devices) as basis for \associative"
and \intuitive" thinking.
To improve Mark's performance and following a counselor's recommendation, the geography teacher has activated an appropriate (i.e. certain level of
geography) VR-Friend on the school's Web server. Mark has chosen the name
VR- Nick and certain features (like a special T-shirt and trousers) for his VRFriend when rst logging in. Now, whenever Mark logs in on the school server
for whatever activities the rst thing Mark sees is VR-Nick. VR-Nick's appearance, his attitude, and what he says (yes, VR-Nick speaks) depends on Mark's
past performance and on how much work Mark has \cared for" VR-Nick. Sometimes VR-Nick will be smiling, \Hey, great you are looking me up again", or
sulky maybe even tearful (\You are really neglecting me"). Overall, if VR-Nick
is treated well he stays healthy, becomes more and more friendly, will show Mark
a cartoon, a joke, or tell him some titbits (e.g. pointing him to an interesting
URL on the Web), otherwise, VR-Nick will look more and more sick, he will
sit in a sad posture in a not so good-looking neighbourhood and, if seriously
\neglected" will actually die. On the other hand, if Mark learns well (treats VRNick well), VR-Nick will eventually congratulate Mark and tell him that he has
to move on, but will stay in contact with him: and he will once in a while (with
a decreasing tendency) brie y show up when Mark logs in, tell him some titbits,
and disappear again. Note that Mark may have a growing number of VR-Friends
that way, that entertain Mark a bit here and there as he does other Web stu .
The actual heart of the conversation between Mark and VR-Nick (who was
after all created to teach some geographic facts) consists of questions asked by
VR-Nick that have to be answered by Mark. (Mark can exit \no time right now"
any time he wants). A typical \geography" VR-Nick may help the student (Mark)
to learn about the location of cities, countries, etc. on the globe. Questions that
VR-Nick might ask are \show me the location of xxx" (where xxx is a city, a
country, a mountain, etc.) Mark's reaction is to click at the right place on the
globe or map shown. The system is very patient: if Mark clicks on Antarctica
when asked for the location of Vienna, the system (NOT VR-Nick) will explain
that this is Antarctica, some facts about it, and will show the real location of
Vienna and information on it and Austria. However, the system records the
information (\Vienna, totally wrong") in such case. A second totally wrong
answer to Vienna's location would give a somewhat more stern reply to Mark,
and the system would record again (\Vienna, totally wrong"). A third complete
failure of Mark would result in the system storing (\just fooling around"), etc.
Thus, the system basically records for each session between Mark and VR-Nick
ve numbers (a,b,c,d,t). Here a is the number of correct answers, b the number
of totally wrong answers, c the number of approximately correct answers, d the
1134
Maurer H.: On Two Aspects of Improving Web-Based Training
number of times Mark just seems to have been fooling around; t is the point
of time when the session takes place. The \development" of VR-Nick depends
on how often and lengthy sessions between Mark and VR-Nick take place; it is
up to the teacher to set parameters like: \at least one session with 10 questions
every second school day and all questions answered correctly twice within a total
of three months".
Above is clearly just one typical scenario. Other questions (requiring e.g.
text input or such) are of course conceivable. Finally, VR-Nick has always some
surprises \up his sleeve": like instead of VR-Nick appearing, Mark nds a sign
\Out right now. Back in 2 minutes", or nds some other VR-Friend, etc.
The main point is that although VR-Nick forces Mark to learn (unless Mark
accepts that VR-Nick deteriorates rather than prospers) this learning is fun and
full of surprises.
1.3 The implementation
The main idea of the implementation is to have two completely separate modules
called development and learning.
The learning module is clearly domain dependent, i.e. di ers from subject to
subject and takes the age of the learner into account. However, learning modules
are created readily with teaching wizards: the most simple form of a teaching
wizard just helps preparing problems as follows: each problem consisting of a
question and possibly a picture (image), and the answer is a click in a certain
position, where a click is recorded as \correct", \close" or \wrong" dependent
on how far from the current position the clock occurs. Note that the system will
keep track of \close" or \wrong" answers to be able to decide if a student is
seriously trying or just fooling around. Other teaching wizards that e.g. allow
textual answers can also be used.
The main point is that the teaching module communicates to the development module only ve numbers (a,b,c,d,t) as explained in section 1. The reaction
of the VR-Friend as carried out by the development module only depends on the
sequence of \answer-quintuples" (a,b,c,d,t) obtained. To keep the system simple
there is no (or no substantive) data- ow from the development module to the
teaching module.
Depending on the answer-quintuples the state of the VR-Friend (and hence
its appearance and utterances) within the development module are changing.
A random element makes sure that the VR-Friend behaves somewhat unpredictably. A number of VR-Friends (age, looks, clothing, ...) with certain developmental paths are built into the system, so new subject domains do not
necessitate changes or additions of the development module, although tools to
introduce new VR-Friends may well be provided in the future.
The development module and its functionality is clearly very much open
ended. On a rst and primitive level for each \state" of the VR-Friend one of
a number of facial expressions, gestures, background images and utterances is
combined in a random fashion. This leads to a fairly complex \behaviour" at
moderate development cost.
It remains to be mentioned that both teachers and students can set certain
environmental parameters as mentioned in passing: the teachers can de ne time
intervals that are \acceptable", students will choose sex, name and age of their
VR-Friends.
Maurer H.: On Two Aspects of Improving Web-Based Training
1.4
1135
Other issues
The most burning issue is certainly if VR-Friends will be interesting enough to
indeed motivate students to learn. There are strong arguments both pro and
con, and only experiments can answer this question.
On the one hand, Tamagotchis have been amazingly successful, and VRFriends are similar in a number of important ways: once \born" they cannot be
\stopped" but either develop well (if treated well) or else they visibly deteriorate, even to the extent that they die. On the other hand, while Tamagotchis
are \omnipresent" and keep reminding users of their existence by beeping at appropriate or less appropriate times, VR-Friends only show up when a user logs
into the correct server, typically the main server of a school or school-district.
There is another crucial issue. Kids who love to play and love to learn all
kinds of irrelevant information seem to lose interest in games once they realise
that they are actually learning useful information (!). It has been theorised that
the association \learning is work, work is unpleasant" (a product it seems of
our less than optimal school systems) is suciently strong to turn kids o . A
typical case supporting above arguments is the huge success of the \Space Quest"
adventure games vs. the meagre sales of the \Goldrush" adventures game, all by
Sierra: all have the same interface and the same kind of story: users have to help
the hero to discover certain facts: Those facts are \useless" in \Space Quest"
but contain real historic information in \Goldrush". It is vexing to see that sales
of Space Quest have been hundred-fold better than those of Goldrush!
However, there have been some educational games that have been successful (like the classic arithmetic learning game \How the West was won" or the
geography/history games \Where in the world (where in history) is Carmen
San Diego".) Thus, the nal verdict whether playing cannot be combined with
teaching is still out!
There is also another, rather opposite issue that can only be settled experimentally: can it happen that kids get attached to their VR-friends to an extent
that their departure, let alone death, will cause serious emotional turmoil? Note
that even Tamagotchis have turned out to be fairly addictive: What will a more
human-like, reality-like VR-Friend do? Whatever the answer is it is clear that
VR-Friends programmes have to make sure that users will not develop guilt
feelings, e.g. when a VR-Friend gets sick or dies since its owner was sick, on
vacations, or such.
The most interesting aspects of VR-Friends is the fact that they are on the
Web. We leave a detailed discussion of the implications of this for a future paper. But observe that VR-Friends can potentially communicate, compete, or
temporarily be taken care of by \Baby-sitters" via the Web. Note further that
VR-Friends can use and point out relevant information (i.e. URL's) on the Web,
lead to discussion forums, on-line chats between owners of similar VR-Friends,
etc. Indeed the dialogue of users and their VR-Friends may lead to questions
posed by other VR-Friends to other users. It should be clear that the complex
interaction of VR-Friends will have to based on what has been learnt and developed in the area of intelligence agents as mentioned earlier.
The notion of Web-based VR-Friends used for educational applications opens
a new and potentially very powerful way to communicate factual knowledge and maybe even more. Experiments will have to demonstrate the viability of
the approach. At the time of writing no polished version of VR-Friends software
1136
Maurer H.: On Two Aspects of Improving Web-Based Training
exists, but we expect to have one ready by mid 1998.
Note also that successful implementations of VR-Friends will be much eased
by using powerful Web-servers such as Hyperwave [1], [2], [3].
2
Unobtrusively Collecting Data on Web-Based Material
When o ering educational material on the Web it is of crucial importance to
obtain feedback on e ectiveness and user satisfaction. In this note we focus
on one sub-aspect, obtaining data on subjective user satisfaction. We discuss
\traditional" techniques, none of which has worked well in the past. We then
present novel techniques that we believe could provide e ective tools desperately
needed to assure continuous quality improvement of Web based training material.
2.1
Classical ways to collect satisfaction data
There are two main classical ways to collect data on user satisfaction with educational material available on the Web: (a) the use of questionnaires and (b)
evaluation of detailed log les.
The problem with questionnaires is that there is much user resistance to
lling them out carefully, thus creating lots of statistical \noise". Even \willing"
students are often at a loss of how to answer questions properly that refer to
material that was worked through some time back.
The other approach, the evaluation of detailed log les is problematic for two
reasons: the HTTP protocol used on the Web is \stateless", i.e. it is not possible
to record the \trail" of users but only how often each page has been hit. Using
more sophisticated techniques such as session oriented protocols, or simulating
them by using \cookies" or \session keys" as is done e.g. in Hyperwave ([1], [2],
[3]) it is possible to keep more detailed records of usage of educational material.
However, such detailed records have already been kept in non-Web based systems
as early as in Plato [4], but have rarely been terribly helpful. It seems that e orts
to extract interesting information from a huge mass of log les have never been
entirely successful.
Without going into further detail it should be clear that new techniques to
judge subjective users satisfaction are needed for use on the Web which is becoming a major tool of teacher-student communication and course administration.
We propose three such techniques in the next section.
2.2
Novel ways to judge subjective user satisfaction
The rst alternative we suggest to use instead of questionnaires is the use \of
questionnairelets", Q-lets for short. Such a Q-let consists of a single question that
can be answered within a few seconds. Q-lets are presented to users in a random
fashion but never more than a few per session. As little bonus for answering a
question an optional cartoon or joke is shown to encourage answering the Q-let.
Cartoons or jokes shown must be chosen so as not to disrupt the ow of learning.
Note that an option \skip question" is presented, i.e. the answering of Q-lets is
not enforced. However, clicking at the Q-let \skip" button will often be the same
amount of work as clicking at one of the choices (e.g. radio buttons) o ered in
the Q-let.
Maurer H.: On Two Aspects of Improving Web-Based Training
1137
Observe that users in a Q-let environment do not even need to ll out a complete questionnaire in total, yet may actually answer the same Q-let more than
once. Di erent answers will thus be a good indication of the (un)\anonymously
identi ed" across session boundaries in the sense of [5], i.e. users have (selfchosen) names and passwords, so that detailed record can be kept for each
\anonymous" user, yet the system does not know the real identity of any of
the names chosen. Experimentation on the distribution and frequency of Q-lets
used will have to be carried out.
An alternative to Q-lets (or an additional feature) is a so-called \Feedback
Button ". When clicked at, a form appears where users can click at any of the
presented checkboxes to voice their opinion about the current \page" presented,
or even about a certain part or aspect of it.
We feel that collecting individual answers at points where they are relevant
is both less bothersome for the user, and leads to more reliable results at the
same time.
The second alternative to evaluate subjective user satisfaction comes from
the approach used in GENTLE [6] to questions posed by students: at any point
while working through some material students can \ask a question" by inserting
a question mark anywhere on the screen. This action triggers the sending of
an appropriate message to a certain user group (i.e. tutors), who will answer
the question (synchronously or asynchronously) at which point the question
mark turns into an exclamation mark; also, the answer is mailed to the person
having asked the question, and other users seeing an exclamation mark can
retrieve the \question/answer" dialogue that occurred earlier by just clicking at
the exclamation mark.
By evaluating pages with such question marks and exclamation marks much
valuable feedback can be collected. In rst experiments it was observed that
questions were not only asked when material was badly explained but actually
even more frequently when the material interested the users to the extent that
they wished further explanations!
The third alternative is called \sensitive button". After all, each \page" of
courseware leads to one of a number of further pages clicking at some navigational button. Users are informed that the position where they click the button
will be seen as the expression of an opinion concerning the current \page" at
issue. There are many alternatives. However, to be speci c, here is one scenario:
clicking at a button on the left could mean \I don't like this page" (and upper
left \bad explanation" middle left \I don't like this page but I don't specify any
reason" and lower left \The explanation is too terse"), while clicking on the middle of the button may mean \my feelings are neutral" and clicking on the right
\I like this page" (where upper right could indicate \I like the way things are
presented", middle right \I like this page but I don't specify any reasons" and
lower right \I like this page because of the contents provided".) Clearly many
alternatives are possible and need careful testing.
We feel that sensitive buttons (although they will generate some statistical
noise) will work well after some initial period (during which the user may be
asked \do you really mean xyz?"): some \click" to navigate is necessary anyway,
so users \may as well" signal their level of satisfaction.
In this section we have argued that new ways of collecting feedback on user
satisfaction with educational material is of great importance for optimising WBT
(Web Based Teaching/Training). We have proposed a few novel techniques but
1138
Maurer H.: On Two Aspects of Improving Web-Based Training
believe that many other ways can be thought of and should become elds of
serious experimentation.
References
[1] Maurer, H.: HyperWave: The Next Generation Web Solution; (Ed.), AddisonWesley Longman, London
[2] Maurer, H., Mayrhofer, V.: Handling Large Web Sites on Internet and Intranets The Ocial Guide to Hyperwave, dpunkt, Heidelberg (1998), to appear.
[3] Maurer, H.: What We Want from WWW as Distributed Multimedia System; Proc.
VSMM'97, Geneva. IEEE (1997), 148-155.
[4] Bitzer, D.: The Wide World of Computer Based Education; Advances in Computers, Academic Press, 15 (1976), 239-283.
[5] Flinn, B., Maurer, H.: Levels of Anonymity , JUCS 1, 1 (1995), 35-47.
[6] Maurer, H., Dietinger, Th.: How Modern WWW Systems Support Teaching and
Training; ICCE'97, Kuching, Malaysia (1997), to appear.
[7] Riecken, D.: Intelligent Agents; Special Issue of the C.ACM 37,7 (1994).
[8] Maes, P.: Agents That Reduce Work and Information Overload; C.ACM 37,7
(1994), 30-41.