Psychology of Aesthetics, Creativity, and the Arts Copyright 2007 by the American Psychological Association

2007, Vol. 1, No. 2, 114 –120 1931-3896/07/$12.00 DOI: 10.1037/1931-3896.1.2.114

An Electronic Museum Guide in Real Use

Christoph Bartneck Aya Masuoka, Toru Takahashi, and Takugo Fukaya

Eindhoven University of Technology ATR Media Information Science Laboratory

Within the Contextual Model of Learning framework, the authors conducted a study with electronic
handheld guides at the Van Abbe Museum. The authors offered the general electronic tour using the
ubiNext system for the duration of two weeks. This study investigated whether the use of multimedia
content would increase the usage of the system compared to only visual content. In addition the authors
were interested if the age of the visitors would influence the system’s usage. The authors also investigated
whether individual visitors use the system more compared to visitors who experience the museum in
couples. More than half of the visitors only requested information for five or less objects independently
of the visitor’s age. Technical problems and usability issues might have caused many visitors to stop
using ubiNext. Among those participants who heavily used the system, the display of multimedia content
lead to an increased usage compared to visual content only.

Keywords: museum, electronic, guide, pda, learning

Museums increasingly acknowledge the need to provide ad- recommendation system and fixed tours that have been developed
equate educational services to their visitors. To guide the mu- by the museum staff. After the museum visit, the users can login
seums’ educational efforts first theoretical frameworks were at the website again to review the art objects they visited.
developed. The Contextual Model of Learning framework is In an earlier study, this system was tested at the Van Abbe
among the most renowned (Falk & Dierking, 2000). It identifies Museum in Eindhoven, the Netherlands (Bartneck, Masuoka,
three overlapping contexts: the personal context, the sociocul- Takahashi, & Fukaya, 2006). Several school classes were asked
tural context, and the physical context. All three contribute to in the framework of their art classes to plan their tour at the
and influence the interactions and experiences that people have website and then to visit the museum itself. This article presents
when engaging in free-choice learning activities such as visiting the follow-up study that tries to extend the original work while
museums. The personal context consists of four factors: Moti- maintaining most of the methodology to ensure comparable
vation and Expectations, Interest, Prior Knowledge and Expe- results.
rience, Choice and Control. The sociocultural context contains Several choices in the methodology of the original experi-
three factors: Within-Group Sociocultural Mediation, Facili- ment limit the generalizability of its results. School classes are
tated Mediation by Others and Culture. The physical context an essential part of the educational efforts of the Van Abbe
includes four factors: Advance Preparation, Setting, Design, Museum, but it forms a very homogenous age group. These
and Subsequent Reinforcing Events and Experiences. Together, teenagers grew up with information technology and frequently
these 11 factors offer a conceptual framework that describes use computers and the Internet. The majority of the general
the influences and interactions between the museum and its museum visitors are much older and have less or no experiences
visitors. with computers and the Internet. For this study, we therefore
This Contextual Model of Learning framework formed the base invited the general public to participate.
upon which the ubiNext museum guide system was developed. It Another limiting factor has been that the students were required
allows the users to plan a tour on a website prior to their actual to participate in the study within the framework of their art classes.
museum visit. At the museum the system guides the users through Even thought many students enjoyed the museum visit, they did
the museum using personal digital assistants (PDA). Besides of- not volunteer, and they simply did not complete their tour planning
fering the users their preplanned tours, it also offers an algorithmic homework. This mild pressure from the school might have influ-
enced the results. In our current study we therefore only used
participants that voluntarily decided to join.
Christoph Bartneck, Department of Industrial Design, Eindhoven Uni- Furthermore, the previous participants were always asked to
versity of Technology; Aya Masuoka, Toru Takahashi, and Takugo form couples, so that their conversation could be recorded. The
Fukaya, ATR Media Information Science Laboratory. general museum visitors may also come in couples, but many visit
We would like to thank Willem-Jan Renders, Loes Janssen and Marjon the museum alone. Sharing a PDA with another person might have
de Groot for their support. This research was partly supported by the
been a different experience from operating it alone; therefore, the
National Institute of Information and Communications Technology (NICT)
participants in this study were free to either use one PDA together
of Japan.
Correspondence concerning this article should be addressed to or to have one individually.
Christoph Bartneck, Department of Industrial Design, Eindhoven Univer- Lastly, the results of the previous study showed that the
sity of Technology, Den Dolech 2, 5600MB Eindhoven, the Netherlands. ubiNext system did not significantly perform better compared
E-mail: christoph@bartneck.de to the traditional paper guide. We had concluded that the PDA

114
 AN ELECTRONIC MUSEUM GUIDE IN REAL USE 115

had only shown text and pictures and had not used the full vice or not prepare their visit. We could not manipulate this
potential of a PDA. In this study we added audio commentary factor. In the following text, the italic style is used to highlight
and movies to the tour, while also offering the old version for the conditions’ names.
comparison.
Similar to our previous experiment we attempted to measure
the visitors’ learning experience to determine the quality of the Measurements
system. It has to be acknowledged that the concept of “learning
experience” remains on an abstract level. Not just the repeatable The following measurements can be considered indicators for
knowledge of art, such as the recognition of art objects and their the visitor’s learning experience.
key data, is considered a learning experience, but also personal Demographics. Age and gender of the participants were re-
experiences that might influence the visitors’ behavior in the corded. For the couples, their average age was recorded, and the
future (Falk & Dierking, 2000). A visitor might, for example, gender of the couple was recorded as male, mixed, or female.
remember the atmosphere of a Picasso painting years after Conversation. The participants’ utterances were recorded on a
visiting the exhibition and use this knowledge to interpret small audio recorder that was hung around the neck of one par-
another art object or decide to spend a vacation in France. Falk ticipant. This recording was optional since it poses an intrusion
and Dierking suggest, at the risk of oversimplifying the com- into the visitors’ privacy. After the experiment, the utterances were
plexity of the learning process, that the nature of learning in analyzed. First, the recording was cropped from the point that the
free-choice settings, such as museums, can be described in three participants entered the exhibition to when they exited the exhibi-
simple statements: learning begins with the individual, learning tion. The duration was noted as speechTotal. The utterances in the
involves others, and learning takes place somewhere. This recording were then grouped into the following categories and the
definition is so broad that it comes close to having to admit that duration of the categories was noted. Therefore instead the number
it is not clear at all what learning experience is. Anything could of utterances in a category the sum of how long participants talked
be a learning experience.
about each category was measured.
The operationalization of such an abstract concept is diffi-
• Art—the participants talked about art objects, related concepts
cult. Still, certain behaviors of the visitors are fairly good
and their reaction to it.
indicators for it. Clearly, the longer visitors indulge in an
• Handheld—the participants discussed the functioning of the
exhibition to see many art objects, the higher the chance they
had a learning experience (Smith & Smith, 2003). Furthermore, handheld device.
the more discussions and reflections the visitors have with each • Navigation—the participants conversed about where they are
other about the exhibited art the more likely they had a learning and where they want to go.
experience (Smith & Smith, 2003). This indicator is explicitly • NotAudible— utterances which could not be understood, for
mentioned as factor five (Within Group Sociocultural Media- example when they were too silent.
tion) in the Contextual Model of Learning framework. • OffTopic—the participants talked about topics that were not
Based on the extensions mentioned above we define four re- connected to the exhibition or the study. For example, the participants
search questions for this study: talked about how much or how little they liked a certain neighbor.
• Silence—a pause of two seconds and longer was defined as
1. Do visitor from different age groups have different learn- silence. This is a standard threshold in speech analysis. Pauses
ing experiences using the ubiNext system? shorter than two seconds are therefore considered to be part of the
conversation flow in the other categories.
2. Does the presentation of audiovisual media improve the
• Troubleshooting—the participants talked about technical
visitors’ learning experience compared to the presenta-
problems and may have asked the experimenter for help.
tion of visual media only?
When two or more groups of participants talked to each
3. Do visitors who operate a PDA by themselves have a other, the utterances were categorized as Group Interaction.
different learning experience compared to couples that Within Group Interaction, the same subcategories exist as used
share a PDA? above: Group Art, Group Handheld, Group Navigation, Group
Not-Audible, Group OffTopic, Group Silence, and Group Trou-
4. Do visitors who planned their tour prior to the actual bleshooting.
museum visit have a better learning experience compared Behavior. Every time the participants executed an action on the
to visitors who did not plan their tour? handheld device, such as requesting information about an art
object, a log file entry on the server was created automatically.
Method Based on this log file data, several measurements are available:
We conducted a 2 (media) ⫻ 2 (group) between participants • DurationOfUse—the duration from the first user action to the
experiment. The two media conditions were visual or audiovi- last user action.
sual. In the visual condition only text and still images were • NumberOfObjects—the number of objects that the participant
displayed, and in the audiovisual condition movies and audio requested information about.
commentary were presented. The two group conditions con- These measurements contain a certain number of errors, since a
sisted of individual and couple. In addition, the participants participant could request the same documents several times. Due to
could either prepare their visit using the ubiNext Internet ser- technical problems, a request might have also been reloaded by the
116 BARTNECK, MASUOKA, TAKAHASHI, AND FUKAYA

system. The size of this error is not clear, but it can be assumed
that it is evenly distributed.
Qualitative. To gain a more qualitative insight into the visitor’s
experiences with the system, we offered the visitors to write their
comments and ideas into a feedback book.

Setup
The experiment was conducted in January 2006 at the Van Abbe
Museum in Eindhoven, the Netherlands. Thirty art objects in the
museum were entered into the ubiNext system. Eleven object
information screens consisting of still images and text were ac-
companied by audio commentary and another 11 featured video
clips while only eight were limited to the basic visual information.
The 30 art objects were spread across 12 rooms on three floors, and
labels containing the identification number of the object were
attached below the regular exhibition labels.
Figure 1. The Dell Axim C51v with two headsets.
Participants
The age of the 189 people who participated in the study ranged The museum volunteers guarding the museum were trained to
between 12 and 75 (M ⫽ 46, SD ⫽ 16.5). Seventy of the partic- help the visitor with their PDAs, but for serious technical problems
ipants were female and 59 were male. Forty-three couples agreed the visitors had to return to the information desk in the entrance
to have their conversations recorded. hall. Once the visitors were ready to leave the museum, they would
return their PDAs and receive a postcard on which the address of
Materials the website together with their login and password were noted.

The 20 Dell Axim C51v PDAs featured a 3.7 in color display Results
that displayed 480 ⫻ 640 pixels. They wirelessly communicated
with the ubiNext server using the 802.11b standard. Stereo head- The participants either used the PDA by themselves or in
phones were attached to the PDAs to play the audio content (see couples. One couple used the system far beyond any other partic-
Figure 1). Due to the framgmented architecture of the Van Abbe ipant, and it is likely that they individually used the PDA after each
Museum, it was necessary to install eight access points to enable other and hence doubling the number of log entries on the server.
the PDAs to wirelessly communication with the server. The server Since the actions of this couple have been extremely out of the
itself was equipped with two network cards. One would be as- general trend, they were considered an outliner and excluded from
signed to the museum internal communication with the PDAs and the further analysis.
a second would provide the web service to the Internet users. From the remaining 187 people 71 participants used the PDA
The conversations of the participants were recorded using a individually while 116 used them in couples. The data recorded for
small MP3 player (Creative MuVo V200) that was hung around the couples were treated as being one participant which results in
their necks. The devices’ built-in microphone provided sufficient
audio quality for the recording. The recordings were analyzed after
the experiment using Noldus Observer.

Procedure
Prior to the experiment, the museum advertised the ubiNext
service on its website and through newspapers and TV news. This
should attract a sufficient number of participants visiting the tour
planning website. After visitors entered the museum and purchased
their tickets, they would pass the information desk, on which the
ubiNext handout took place.
They were offered a free ubiNext tour, and after making a
deposit they would be given a PDA. If they had planned a tour
prior to their arrival at the museum, they could login the system
with their existing user ID and password. If they did not plan their
own tour, then they would receive a login and password at the
information desk. After a short introduction given by the staff, they
entered the museum and could experience the museum with the
ubiNext system as long as they desired (see Figure 2). Figure 2. Visitors using ubiNext in the museum.
 AN ELECTRONIC MUSEUM GUIDE IN REAL USE 117

a total number of 129 (71 ⫹ 116/2) participants. Sixty-five par- Table 1

ticipants were using the PDA in the Visual condition while 64 Mean NumberOfObjects Across Age Groups
participants used the PDA in the audiovisual condition. Sixty-
seven people prepared their visit by creating a tour using the Age range M N
ubiNext web service with an average of 5 objects per tour (SD ⫽ 10–19 8.88 6
5.54). Only 11 from these 67 showed up at the museum and 20–29 10.68 23
walked with their preplanned tour. One hundred eighteen partici- 30–39 8.43 17
pants did not prepare a tour. 40–49 7.95 22
50–59 8.10 28
From these 129 participants 14 did not use the system after it
60–69 6.33 29
had been given to them and no data was recorded for them. Since 70–79 10.33 4
they did not actually experience the system, they were excluded
from the further analysis which brings the number of participants
available for the analysis to 115. The participants were reasonable
equally distributed across the four conditions (see Figure 3). quality or the participants switched the recording device off. Table
An initial analysis of covariance (ANCOVA) in which Media 4 shows that there is a large duration difference between the
and Group were the independent variables and Age and Prepara- recorded conversations (SpeechTotal) and the duration of PDA use
tion the covariants did not result in any significant differences in (DurationOfUse). Most participants worked with the PDA for an
the dependent variables NumberOfObjects and DurationOfUse. average of 20 minutes before they would either bring the PDA
Table 1 shows that the age of the participants did not result in back to the front desk or continue their museum tour without using
significant differences in the NumberOfObjects. the PDA.
We then analyzed the distribution of participants across the On average more than half of the audio was recorded while the
NumberOfObjects they viewed, and Table 2 shows that more than PDA was not used. This strongly biases the measurements as
half of the participants looked at five or less objects. Furthermore, presented in Table 5. An ANCOVA with Media as the independent
when mapping the number of information request for each art variable, Age and Preparation as the covariant was conducted, but
object onto its location in the museum, it became clear that the no significant difference was detected.
majority of the participants used the system in rooms that were
close to the entrance on floor zero (see Appendix). Considerable Technical Problems
fewer objects were accessed on the other floors.
During the experiment we encountered some practical and tech-
It can be assumed that one would have to at least use the PDA
nical problems which might also be interesting for future museum
to request information about 10 objects to fully experience the
guide developers. First, the network connection between the PDAs
system. Therefore we limited our further analysis to the 39 par-
and the access points failed. We could identify two possible
ticipants whose NumberOfObjects were equal or greater than 10.
reasons. Sometimes the visitors would simply be out of range from
Figure 4 shows that there are too few participants in the Couple/
the access points, such as when they entered the cafeteria. This
Audiovisual condition to gain meaningful statistical results. We
problem could be relatively easy to overcome by installing more
therefore excluded the Group factor from the following analysis,
access points. A more serious problem is the roaming function of
which resulted in 25 participants being in the Visual condition and
the PDAs. While the PDA’s operating system switches its com-
14 in the Audiovisual condition.
munication from one access point to another, no communication
An ANCOVA with Media as the independent factor and Age
with the server is possible. If the visitor tried to access information
and Preparation as covariants was conducted. Preparation had no
during this period, they would receive an error message, which
significant influence on the measurements while Age had an al-
caused several visitors to return to the information desk.
most significant, F(1, 35) ⫽ 4.002, p ⫽ .053 influence on Dura-
tionOfUse. Table 3 shows that there was little difference for the
NumberOfObjects across the age groups. Media had a significant, Discussion and Conclusions
F(1, 35) ⫽ 4.363, p ⫽ .044 influence on the NumberOfObjects. More than half of the participants only requested information
Participants viewed on average 15 objects in the Visual condition about five or less art objects. On average the participants used the
and 19 in the Audiovisual condition. system for about 20 minutes and stayed in the museum for around
The conversations of 43 participating couples were recorded. one hour. In our previous study the participants used the system
Eight of the recordings had severe problems, such as low audio

Table 2
Distribution of Participants Across the NumberOfObjects

NumberOfObjects Number of participants

0–5 70
6–10 22
11–15 20
16–20 6
21–25 7
26–30 4
Figure 3. Distribution of the 115 participants across the four conditions.
118 BARTNECK, MASUOKA, TAKAHASHI, AND FUKAYA

Table 4
Duration of Usage and Duration of Recorded Speech

M Mdn SD

DurationOfUse 00:20:52 00:20:08 00:15:47

SpeechTotal 01:03:50 01:05:23 00:38:00
Difference 00:38:41 00:33:37 00:36:50

Figure 4. Distribution of the 39 participants across the four conditions.

terial justifies the use of PDAs in museums compared to traditional
paper guides.
much more. The reason might be that in our present study the It was not possible for us to properly test if visitors who operate
participants volunteered to use the PDA and were free to use it as a PDA by themselves have a different learning experience com-
long as they wanted to. In our previous study we asked and pared to couples that shared a PDA. The numbers of true ubiNext
supervised the usage far more, and the study was executed in the users was insufficient to test this hypothesis.
framework of the participants’ high school art class. Even though the study was advertised in newspapers and even
Through the mapping of the art object’s popularity on their on TV, only 11 out of 67 people who preplanned their museum
location (see Appendix), it became clear that most visitors used the tour using the ubiNext web service actually visited the museum. It
system in the rooms close to the entrance and far less in other appears as if the 2-week duration of the study was too short to
rooms. It appears as if many participants tried out the system on allow many people to preplan their tour and visit the museum in
level zero after entering the museum and slowly stopped using the the given time frame. It is unlikely that the majority of the visitors
PDA further on in the building. came to the museum to use the PDA system. Instead, those people
Surprisingly, the age of the participants did not influence the who already wanted to come to the museum anyway might have
usage. We expected that the young participants who are more also used the ubiNext web service before. Due to the low number
familiar with computers and PDAs would use the system more. of participants that preplanned their tour, it was not possible for us
But the elderly people used the system just as much. Some elderly to test if these participants had a higher learning experience com-
museum visitors even mentioned that they preferred the PDA tour pared to the ones that did not prepare their visit.
over normal audio tours since they were not able to hear well. A possible indication for the low usage of the system could be
Elderly people with poor vision usually do not come to the mu- found in the qualitative feedback we received through the feedback
seums since they cannot see the art objects, but elderly people with book. The complaints of the users focused on the reliability of the
hearing problems do come. The relatively high average age of 46 service and the system’s ease of use. Several users encountered
years already indicates that the majority of the visitors, in partic- technical problems, which lead them to stop using the system.
ular during the working week, are senior citizens. The main Many users also commented that the user interface was too diffi-
challenge for this group of users is the small screen size of the cult and had too many features. The ubiNext system required too
PDA. The text information displayed on it was on the edge of what much attention that they would prefer to spend on the art objects.
they could read. Several elderly visitors returned to the wardrobe The PDA guides were given out free of charge, and many
lockers to get their reading glasses before using the PDA. museum visitors might have been tempted to try out the system
Whatever made people stop using the PDA strongly biased the
measurements. Only among those who really used the system, a
significant difference across the media condition could be found. Table 5
Among those participants that truly used the system and accessed Means and Standard Deviations of Speech Categories
more than 10 art objects the audiovisual material increased their
Visual Audiovisual
usage. Clearly, these participants preferred the audio and video
material compared to the presentation of pictures and text only. Measurement M SD M SD
This indicates that only the presentation of audio and video ma-
Art 00:13:01 00:11:07 00:09:26 00:08:26
Handhelt 00:08:58 00:06:26 00:10:05 00:13:55
Navigation 00:05:58 00:17:16 00:05:05 00:10:03
Table 3 NotAudible 00:05:28 00:09:59 00:07:01 00:11:51
Means and Standard Deviations Across Age OffTopic 00:05:51 00:10:15 00:04:04 00:03:48
Silence 00:18:52 00:18:21 00:25:35 00:23:30
NumberOfObjects Duration Of Use Troubleshooting 00:00:52 00:01:42 00:00:50 00:01:48
Undetermined 00:00:20 00:00:44 00:00:21 00:00:45
Age N M SD M SD Grouptotal 00:02:50 00:04:48 00:03:06 00:06:27
Groupart 00:01:49 00:03:20 00:00:33 00:01:02
10–19 2 12 1 0:36:20 0:15:36 Grouphandhelt 00:00:50 00:01:24 00:01:35 00:02:45
20–29 10 18 7 0:20:37 0:10:10 Groupnavigation 00:00:00 00:00:00 00:00:05 00:00:12
30–39 6 12 2 0:31:30 0:05:40 GroupnotAudible 00:00:00 00:00:02 00:00:15 00:00:51
40–49 7 16 7 0:29:37 0:19:30 GroupoffTopic 00:00:09 00:00:22 00:00:30 00:02:01
50–59 8 18 6 0:31:18 0:16:43 Groupsilence 00:00:00 00:00:00 00:00:03 00:00:15
60–69 5 17 4 0:36:20 0:14:37 Grouptroubleshooting 00:00:00 00:00:00 00:00:01 00:00:05
70–79 1 22 0:59:35 Groupundetermined 00:00:00 00:00:00 00:00:00 00:00:00
 AN ELECTRONIC MUSEUM GUIDE IN REAL USE 119

without really having an interest in it. It would be interesting to see should be replaced by a number pat on which the visitors have to
how the usage of the PDAs would change if the visitors would enter the art object’s number indicated its exhibition label. The
have to pay for the usage. It can be speculated that the PDAs would visitors could then no longer easily preview art objects without
be used more, but also that the visitors might be more frustrated if actually standing in front of them. A reliable and valid visualiza-
the system malfunctions. tion of the visitors paths through the museum could help the
museum to better design their exhibition space.
Future Work
The appendix is a first attempt to visualize the paths that the References
visitors take through the museum. With the available data it could
already be extended to also visualize the sequence in which the art Bartneck, C., Masuoka, A., Takahashi, T., & Fukaya, T. (2006). The
objects were viewed. However, at this point in time, the visitor can learning experience with electronic museum guides. Psychology of Aes-
thetics, Creativity, and the Arts, 2.
choose the art object on the PDA using a pull down menu. This
Falk, J. H., & Dierking, L. D. (2000). Learning from museums: Visitor
enables them to preview art objects before actually going there. experiences and the making of meaning. Walnut Creek, CA: AltaMira
This compromises the assumption that the visitor is standing in Press.
front of the art object when requesting information about it and Smith, J. K., & Smith, L. F. (2003). Origins of Impressionism: Relating
hence the visualization of the visitors paths. To get more reliable visitor behavior to perceived learning. Bulletin of Psychology and the
data on the visitors position in the museum the pull down menu Arts, 4, 80 – 85.

(Appendix follows)
120 BARTNECK, MASUOKA, TAKAHASHI, AND FUKAYA

Appendix

Received May 12, 2006

Revision received July 27, 2006
Accepted July 31, 2006 䡲