research-article

How usable are operational digital libraries: a usability evaluation of system interactions

Authors:

Xiangmin Zhang,

Ying ZhangAuthors Info & Claims

EICS '09: Proceedings of the 1st ACM SIGCHI symposium on Engineering interactive computing systems

Pages 177 - 186

https://doi.org/10.1145/1570433.1570468

Published: 15 July 2009 Publication History

Abstract

This paper reports a usability evaluation of three operational digital libraries (DLs): the ACM DL, the IEEE Computer Society DL, and the IEEE Xplore DL. An experiment was conducted in a usability lab and 35 participants completed the assigned tasks. The results demonstrate that all three DLs have more or less usability problems by various measures. Searching in Xplore by inexperienced users was problematic, and browsing in IEEE CS was extremely difficult for all users. Interaction design features that caused these usability problems were identified and discussed. The study implies there is still large room for operational DLs to improve in order to provide more satisfactory services.

References

[1]

Belkin, N.J., Marchetti, P.G., Cool, C. 1993. Braque: design of an interface to support user interaction in information retrieval. Information Processing&Management. 29(3), 325--344.

Digital Library

[2]

Chang, S. L. 1995. Toward a multidimensional framework for understanding browsing. Unpublished doctoral dissertation, Rutgers, The State University of New Jersey, New Brunswick, NJ.

[3]

Chen, S. Y., Magoulas, G. D., Dimakopoulos, D. 2005. A flexible interface design for Web directories to accommodate different cognitive styles. Journal of the American Society for Information Science and Technology. 56(1), 70--83.

Digital Library

[4]

Chimera, R., Shneiderman, B. 1994. An exploratory evaluation of three interfaces for browsing large hierarchical tables of contents. ACM Transactions on Information Systems (TOIS). 12(4), 383--406.

Digital Library

[5]

Chin, J., Diehl, V. A.,&Norman, K. L. 1988. Development of an instrument measuring user satisfaction of the human-computer interface. Proceedings of the SIGCHI conference on Human factors in computing systems (pp.213--218). Washington, D.C., May 15--19, 1988.

Digital Library

[6]

Chrzastowski, T. E.,&Scheeline, A. 2006. ASDL: The Analytical Sciences Digital Library taking the next steps. Science&Technology Libraries, 26(3/4), 79--94.

[7]

Dakka, W., Ipeirotis, P. G., Wood, K. R. 2005. Automatic construction of multifaceted browsing interfaces, in: Chowdhury, A., Fuhr, N., Ronthaler, M., Schek, H.-J., Teiken, W. (Eds.), Proceedings of the 14th ACM international conference on Information and knowledge management CIKM '05. ACM Press, New York, pp. 768--775.

Digital Library

[8]

Davis, F.D. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 3, 319--340.

Digital Library

[9]

Doll, W.J.; Hendrickson, A.; and Deng, X. 1998. Using Davis's perceived usefulness and ease of use instruments for decision making: A confirmatory and multigroup invariance analysis. Decision Sciences, 29, 4, 839--869.

[10]

Egan, D.E., Remde, J.R., Landauer, T.K. 1989. Behavioral evaluation and analysis of a hypertext browser, in: Bice, K., Lewis, C. (Eds.), Proceedings of CHI'89 Conference on Human Factors in Computing Systems. ACM Press, New York, pp.205--210.

Digital Library

[11]

Hartson, H. R., Shivakumar, P.,&Perez-Quinones, M. A. 2004. Usability inspection of digital libraries: a case study. International Journal of Digital Library, 4, 108--123.

Digital Library

[12]

Hearst, M., Elliot, A., English, J., Sinha, R., Swearingen, K.,&Yee, K.P. 2002. Finding the flow in Website search. Communications of the ACM, 45(9):42--49.

Digital Library

[13]

Hertzum, M., Frøkjær, E. 1996. Browsing and querying in online documentation: a study of user interfaces and the interaction process. ACM Transactions on Computer-Human Interaction (TOCHI). 3(2), 136--161.

Digital Library

[14]

Hill, L.L., Carver, L., Larsgaard, M., Dolin, R., Smith, T. R., Frew, J.,&Rae, M.-A. 2000. Alexandria Digital Library: User Evaluation Studies and System Design. Journal of the American Society for Information Science. 51(3):246--259.

Digital Library

[15]

Hinton, P. R., Brownlow, C., McMurray, I.,&Cozens, B. 2004. SPSS Explained. New York, NY: Routledge, Taylor&Francis Group.

Digital Library

[16]

Hong, W., Thong, J. Y.L., Wong, W.-M.,&Tam, K.-Y. 2001--2002. Determinants of user acceptance of digital libraries: An empirical examination of individual differences and system characteristics. Journal of Management Information Systems, 18(3), 97--124.

Digital Library

[17]

Kani-Zabihi, E., Ghinea, G., Chen, S. Y. 2006. Digital libraries: what do users want? Online Information Review, 30(4), 395--412.

[18]

Kengeri, R., Seals, C. D., Harley, H. D., Reddy, H. R.,&Fox, E. A. 1999. Usability study of digital libraries: ACM, IEEE-CS, NCSTRL, NDLTD. International Journal of Digital Library, 2, 157--169.

Digital Library

[19]

Lesk, M. 2005. Understanding Digital Libraries, 2nd ed. San Francisco, CA: Morgan Kaufmann Publishers.

Digital Library

[20]

Marchionini, G. 1995. Information Seeking in Electronic Environments. Cambridge University Press, United Kingdom.

Digital Library

[21]

Marchionini, G., Norman, K.,&Morrell, K. 1992. Final project report: Computer interface design for intermediate results for Grateful Med. Report to the National Library of Medicine. Bethesda, MD: NLM.

[22]

Notess, M., Kouper, I.,&Swan, M. B. 2005. Designing effective tasks for digital library user tests: lessons learned. OCLC Systems&Services, 21(4), 300--310.

[23]

Paynter, G. W., Witten, I. H., Cunningham, S. J., Buchanan, G. 2000. Scalable browsing for large collections: a case study, in: Nürnberg, P. J., Hicks, D. L., Furuta, R. (Eds.), Proceedings of the fifth ACM Conference on Digital libraries. ACM Press, New York, pp.215--223.

Digital Library

[24]

Preece, J., Rogers, Y.,&Sharp, H. 2007. Interaction design: beyond human-computer interaction. New York: John&Wiley Sons, Inc.

Digital Library

[25]

Resnick, M.L.&Vaughan, M.W. 2006. Best practices and future visions for search user interfaces. Journal of the American Society for Information Science&Technology, 57(6), 781--787.

Digital Library

[26]

Saracevic, T. 2000. Digital library evaluation: towards an evolution of concepts. Library Trends, 49(3): 350--369.

[27]

Shiri, A. and Molberg, K. 2005. Interfaces to knowledge organization systems in Canadian digital library collections. Online information Review, 29(6), 604--620.

[28]

Shneiderman, B.,&Plaisant C. 2005. Designing the user interface. Boston, MA: Pearson Education, Inc.

Digital Library

[29]

Shneiderman, B., Byrd, D.,&Croft, B. 1997. Clarifying search: A user-interface framework for text searches. D-Lib Magazine, January 1997. Retrieved January 20, 2008, from http://www.dlib.org/dlib/january97/retrieval/01shneiderman.html.

Digital Library

[30]

Tague-Sutcliffe, J. 1992. The pragmatics of information retrieval experimentation, revisited. Information Processing&Management, 28(4), 467--490.

Digital Library

[31]

Theng, Y. L. 1999. "Lostness" and digital libraries. Proceedings of the fourth ACM conference on Digital libraries, 250--251.

Digital Library

[32]

Thong, J. Y. L., Hong, W.,&Tam, K.-Y. 2002. Understanding user acceptance of digital libraries: what are the roles of interface characteristics, organizational context, and individual differences. International Journal of Human-Computer Studies, 57, 215--242.

Digital Library

[33]

Tsakonas, G.,&Papatheodorou, C. 2006. Analysing and evaluating usefulness and usability in electronic information services. Journal of Information Science, 32 (5), 400--419.

Digital Library

[34]

Tsakonas, G.,&Papatheodorou, C. 2008. Exploring usefulness and usability in the evaluation of open access digital libraries. Information Processing and Management, 44, 1234--1250.

Digital Library

[35]

Yee, K.P., Swearingen, K., Li, K., Hearst, M. 2003. Searching and organizing: Faceted metadata for image search and browsing, in: Cockton, G., Korhonen, P. (Eds.), Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press, New York, pp. 401--408.

Digital Library

[36]

Xie, H. 2006. Evaluation of digital libraries: Criteria and problems from users' perspectives. Library&Information Science Research, 28, 433--452.

[37]

Zhang, J., Marchionini, G. 2005. User interfaces for integrating web-based information: Evaluation and evolution of a browse and search interface: relation browser, in: Delcambre, L. M. L., Giuliano, G. (Eds.), Proceedings of the 2005 National Conference on Digital Government Research. Digital Government Research Center, California, pp.179--188.

Digital Library

[38]

Zhang, X., Li, Y., Liu, J.,&Zhang, Y. 2008. Effects of interaction design in digital libraries on user interactions. Journal of Documentation, 64(3), 438--463.

Cited By

Peker SDalveren G(2023)Usability assessment of scholarly publishers’ online journal interfacesJournal of Data, Information and Management10.1007/s42488-023-00110-z5:4(363-374)Online publication date: 2-Dec-2023
https://doi.org/10.1007/s42488-023-00110-z
Mahdi MAhmad ANatiq HSubhi MQassim Q(2021)Comprehensive Review and Future Research Directions on Dynamic Faceted SearchApplied Sciences10.3390/app1117811311:17(8113)Online publication date: 31-Aug-2021
https://doi.org/10.3390/app11178113
Gaona-García PMartin-Moncunill DMontenegro-Marin C(2017)Trends and challenges of visual search interfaces in digital libraries and repositoriesThe Electronic Library10.1108/EL-03-2015-004635:1(69-98)Online publication date: 6-Feb-2017
https://doi.org/10.1108/EL-03-2015-0046
Show More Cited By

Index Terms

How usable are operational digital libraries: a usability evaluation of system interactions
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

Digital competencies for developing and managing digital libraries

PurposeThe purpose of this study was to explore the essential digital competencies for developing and managing digital libraries. The study identified useful training programs for university librarians to acquire digital competencies. It examined their ...
Digital Preservation, Archival Science and Methodological Foundations for Digital Libraries

Digital libraries, whether commercial, public, or personal, lie at the heart of the information society. Yet, research into their long-term viability and the meaningful accessibility of their contents remains in its infancy. In general, as we have ...
Worksets Expand the Scholarly Utility of Digital Libraries
JCDL '18: Proceedings of the 18th ACM/IEEE on Joint Conference on Digital Libraries

Scholars using digital libraries and archives routinely create worksets-aggregations of digital objects-as a way to segregate resources of interest for in-depth scrutiny. To illustrate how worksets can enhance the scholarly utility of digital library ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

EICS '09: Proceedings of the 1st ACM SIGCHI symposium on Engineering interactive computing systems

July 2009

348 pages

ISBN:9781605586007

DOI:10.1145/1570433

General Chair:
T.C. Nicholas Graham
Queen's University, Canada
,
Program Chairs:
Gaëlle Calvary
University of Grenoble, France
,
Philip Gray
University of Glasgow, United Kingdom

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 July 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

EICS'09

Sponsor:

EICS'09: Engineering Interactive Computing Systems

July 15 - 17, 2009

PA, Pittsburgh, USA

Acceptance Rates

Overall Acceptance Rate 73 of 299 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
498
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Peker SDalveren G(2023)Usability assessment of scholarly publishers’ online journal interfacesJournal of Data, Information and Management10.1007/s42488-023-00110-z5:4(363-374)Online publication date: 2-Dec-2023
https://doi.org/10.1007/s42488-023-00110-z
Mahdi MAhmad ANatiq HSubhi MQassim Q(2021)Comprehensive Review and Future Research Directions on Dynamic Faceted SearchApplied Sciences10.3390/app1117811311:17(8113)Online publication date: 31-Aug-2021
https://doi.org/10.3390/app11178113
Gaona-García PMartin-Moncunill DMontenegro-Marin C(2017)Trends and challenges of visual search interfaces in digital libraries and repositoriesThe Electronic Library10.1108/EL-03-2015-004635:1(69-98)Online publication date: 6-Feb-2017
https://doi.org/10.1108/EL-03-2015-0046
Aljohani MBlustein J(2015)Heuristic Evaluation of University Institutional Repositories Based on DSpaceDesign, User Experience, and Usability: Interactive Experience Design10.1007/978-3-319-20889-3_12(119-130)Online publication date: 21-Jul-2015
https://doi.org/10.1007/978-3-319-20889-3_12
Heuwing BMandl TWomser-Hacker CElsweiler DLudwig BAzzopardi LWilson M(2014)Evaluating a tool for the exploratory analysis of usability information using a cognitive walkthrough methodProceedings of the 5th Information Interaction in Context Symposium10.1145/2637002.2637033(243-246)Online publication date: 26-Aug-2014
https://dl.acm.org/doi/10.1145/2637002.2637033
Yuan XZhang XTrofimovsky ABelkin NKelly D(2010)Testing visualization on the use of information systemsProceedings of the third symposium on Information interaction in context10.1145/1840784.1840840(365-370)Online publication date: 18-Aug-2010
https://dl.acm.org/doi/10.1145/1840784.1840840

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents