Searchling: User-Centered Evaluation of a Visual Thesaurus-Enhanced Interface for Bilingual Digital Libraries

Amy Stafford2, Ali Shiri3, Stan Ruecker1, Matthew Bouchard1, Paras Mehta4, Karl Anvik1, and Ximena Rossello1
Humanities Computing Program Department of English and Film Studies 3 School of Library and Information Studies 4 Faculty of Medicine University of Alberta {als2,ashiri,sruecker}@ualberta.ca, {matt.bouchard@gmail.com}
2 1

Abstract. In this paper, we describe a qualitative user study of Searchling an experimental visual interface that allows users to leverage a bilingual thesaurus for query formulation and enhancement. The design of Searchling is based on theories of thesaurus-based interface design from Shiri et al. [1], combined with the principles of rich-prospect browsing [2]. The Searchling interface provides the user with three working spaces on one screen: the Thesaurus space, Query space, and Document space. We interviewed 15 graduate and faculty researchers at the University of Alberta, who carried out three structured tasks in a thinkaloud protocol, with simultaneous audio recording and screen capture. These participants identified a number of significant advantages to the researcher provided by Searchling, including the value of having an interface that could help with identifying search terms, suggesting preferred terms, and giving bilingual search support. They also suggested areas for future improvement, primarily related to our assumption that common knowledge of thesauri would be sufficient to make the various features clear if they were described using standard vocabulary from the thesaurus field. Keywords: Visual Interfaces, Thesauri, Multilingual Digital Libraries, Information Retrieval, User Evaluation.

1 Introduction
Visual interfaces are finding their way into digital libraries as well as into a wide range of information retrieval systems. Visual interfaces enhanced with a thesaurus can be designed to support a variety of information seeking tasks, namely searching, browsing and navigation. More specifically, depending on the level of sophistication that can be offered in the interface, visual interfaces have the potential to support such interactive tasks as query formulation, modification or expansion. In this project, our aim was to develop and evaluate a thesaurus-enhanced bilingual visual interface that will support users information search and retrieval experience within the context of
B. Christensen-Dalsgaard et al. (Eds.): ECDL 2008, LNCS 5173, pp. 117121, 2008. Springer-Verlag Berlin Heidelberg 2008

118

A. Stafford et al.

digital libraries. For the current phase, we used the Government of Canada Core Subject Thesaurus. Previous research in thesaurus-based query formulation and expansion has shown promising results. For instance, Beaulieu [3] reported that both the explicit and implicit use of a thesaurus (using interactive or automatic query expansion respectively) can be beneficial. Efthimiadis [4] has suggested that during query expansion, a thesaurus could be used to display the relationships of the selected terms to other terms. Shiri and Revie [5] in a study of end-user search term selection and query expansion behaviour within a thesaurus-enhanced search system, found that in 50% of the cases where users found additional thesaurus terms, they were not aware of those terms at the beginning of the search. More recently, Shiri [6] has examined 21 metadata-enhanced digital library visual interfaces from the following perspectives: a) information access and retrieval features supported; b) metadata elements used; c) visualization techniques and metaphors utilized. The results show that visual interfaces to digital libraries enhanced with metadata are becoming more widespread as they provide richer representation of digital collections.

2 Theoretical Framework and Design of Searchling

The design of our user interface, called Searchling, was based on two key concepts. The first is the idea of rich-prospect interfaces, in which individual representations of every item in a collection are combined with emergent tools [2]. Using this conceptual framework, Ruecker et al. have subsequently developed a number of metadata enhanced visual interfaces to support users information search and exploration activities (e.g. [7], [8]). The second set of principles we applied are the design ideas for thesaurus-based search interfaces suggested by Shiri et al. [1], including: Providing hierarchical and alphabetical lists to support different strategies. Allowing flexible ways of choosing terms. Facilitating moving between a descriptor and its hierarchical structure. Catering for the selection of alternative Boolean operators. Providing a term pool option for saving the descriptors. Integrating thesaurus and retrieved documents displays. Making thesaurus options available in all stages of the search process.

The interface provides the user with the following three spaces within a single screen: the thesaurus space, the query space, and the document space (Figure 1). The Thesaurus space is on the left. It includes a browsable side panel of high-level categories, next to a list of thesaurus terms. Each term has a number beside it, which indicates how many documents in the collection contain the term. When a term is queried or clicked, it moves to the top of the list and all related terms from the thesaurus appear below it. The table to the right of the Thesaurus list indicates related terms that are broader, narrower, preferred or non-preferred compared with the selected term; the user can also sort by these categories. Finally, there is a language switch at the top of the Thesaurus list. The Query space is located in the right panel of the screen. Users can search for a single term in the thesaurus by entering it in the query box, choosing a language, and

Searchling: User-Centered Evaluation of a Visual Thesaurus-Enhanced Interface

119

Fig. 1. Searchling v1.1, showing the Thesaurus, Query Formulation, and Document spaces

clicking the button labeled Find in Thesaurus. If the term is entered in English but the user selects French as the query language, Searchling will search for the corresponding French term, but the English term will also always be visible as a microtext satellite below the query term. When users decide to add a term to their query, they do so by checking the box next to it in the thesaurus list, and it is added to the Selected Terms list on the lower right side panel. This area serves as the term pool described by Shiri et al. [1]. The Document space forms the third section of the screen, running across the bottom. The documents are represented by standard bibliographic information; they can be sorted in various ways, and each item serves as a link to the actual document.

3 Methodology
The methodology and the technologies used to develop the visual user interface have been detailed in a previous paper [9]. The focus of this paper is to report on a usercentred evaluation of the interface.

120

A. Stafford et al.

Fifteen participants for this study were recruited by snowball sampling of researchers at the University of Alberta, including in particular the bilingual researchers from the Department of Modern Languages and Cultural Studies. Data from the interviews were collected verbally, digitally and in written form. The tasks in this user study were designed to test the users initial response to the Searchling interface; we were interested in how easy or difficult it is for new users to understand the relationship between the different groups of information on the page and whether they found this grouping helpful in their search. In other words, the assigned tasks asked users to try out as many of the interface features as possible, while requiring very little focus on the actual content of the database. In addition to the qualitative feedback, we also had users rank a number of items on a 5-point Likert scale.

4 Results
The most promising finding for Searchling is that it has the potential to solve the greatest problem that people encounter with other search tools, namely formulating queries. When asked about difficulties with their current search tools, the study participants unanimously cited the inability to isolate keyword search terms for a specific topic as the single most frustrating problem. One participant saw immediate implications for her own research. She studies non-canonical elements, which often use multiple names, so it would be very helpful. Other users said they would find Searchling most useful at the beginning of a research project on an unfamiliar topic, because they could start by browsing through general categories for relevant terms and the Thesaurus could help them narrow or broaden their search. There were also positive responses to specific aspects of the interface, such as the numbers next to each term in the Thesaurus, which indicate how many documents in the collection contain that term. Most users also appreciated that they could keep as many items as they like on deck in the Selected Terms list, without immediately adding them to the search for documents.

References
1. Shiri, A.A., Revie, C., Chowdhury, G.: Thesaurus-Enhanced Search Interfaces. Journal of Information Science 28(2), 111122 (2002) 2. Ruecker, S.: Affordances of Prospect for Academic Users of Interpretively-tagged Text Collections. Ph.D. Dissertation. Edmonton, University of Alberta (2003) 3. Beaulieu, M.: Experiments of interfaces to support query expansion. Journal of Documentation 53(1), 819 (1997) 4. Efthimiadis, E.N.: Interactive query expansion, A user-based evaluation in a relevance feedback environment. Journal of the American Society for Information Science 51(11), 9891003 (2000) 5. Shiri, A., Revie, C.: Query Expansion Behaviour Within a Thesaurus-enhanced Search Environment, A User-centred Evaluation. Journal of the American Society for Information Science and Technology 57(4), 462478 (2006)

Searchling: User-Centered Evaluation of a Visual Thesaurus-Enhanced Interface

121

6. Shiri, A.: The Use of Metadata in Visual Interfaces to Digital Libraries. In: Proceedings of the 11th European Conference on Research and Advanced Technology for Digital Libraries, Budapest, Hungary, September 16-21. LNCS. Springer, Heidelberg (2007) 7. Ruecker, S., Lewcio, M., Plouffe, M., Wynne, M.: I never forget a face, a rich-prospect image browser for conferences. In: The Society for Digital Humanities (SDH/SEMI) conference, May 29-31. York University, Toronto (2006c) 8. Ruecker, S., Given, L., Simpson, H., Sadler, E., Ruskin, A.: Design of a Rich-Prospect Browsing Interface for Seniors, A Qualitative Study of Image Similarity Clustering. Visible Language 41(1), 422 (2007) 9. Shiri, A., Ruecker, S., Rossello, X., Bouchard, M., Mehta, P.: Development of a Thesaurusenhanced Visual Interface for Multilingual Digital Libraries. In: Dalkir, K., Arsenault, C. (eds.) Proceedings of the 35th Annual Conference of the Canadian Association for Information Science, Information Sharing in a Fragmented World, Crossing Boundaries, Montreal, May 10-12 (2007)