research-article

Understanding usability practices in complex domains

Authors:

Parmit K. Chilana,

Jacob O. Wobbrock,

Amy J. KoAuthors Info & Claims

CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 2337 - 2346

https://doi.org/10.1145/1753326.1753678

Published: 10 April 2010 Publication History

Abstract

Although usability methods are widely used for evaluating conventional graphical user interfaces and websites, there is a growing concern that current approaches are inadequate for evaluating complex, domain-specific tools. We interviewed 21 experienced usability professionals, including in-house experts, external consultants, and managers working in a variety of complex domains, and uncovered the challenges commonly posed by domain complexity and how practitioners work around them. We found that despite the best efforts by usability professionals to get familiar with complex domains on their own, the lack of formal domain expertise can be a significant hurdle for carrying out effective usability evaluations. Partnerships with domain experts lead to effective results as long as domain experts are willing to be an integral part of the usability team. These findings suggest that for achieving usability in complex domains, some fundamental educational changes may be needed in the training of usability professionals.

References

[1]

Alexander, P. (1992) Domain Knowledge: Evolving Themes and Emerging Concerns. Ed Psychologist 27 (1), 33--51.

[2]

Beyer, H. and Holtzblatt, K. (1998) Contextual design: Defining customer-centered systems. San Francisco: Morgan Kaufmann.

Digital Library

[3]

Bias, R. (1994) The pluralistic usability walkthrough: coordinated empathies. In: Nielsen, J. and Mack, R. (eds.) Usability Inspection Methods. New York: Wiley.

Digital Library

[4]

Bodker, S. (1989) A Human Activity Approach to User Interfaces. Human-Computer Interaction 4 (3), 171--195.

Digital Library

[5]

Cummings, J. and Kiesler, S. (2005) Collaborative research across disciplinary and organizational boundaries. Social Studies of Science 35 (5), 703.

[6]

Dumas, J. and Redish, J. (1999) A practical guide to usability testing. Portland: Intellect Books.

Digital Library

[7]

Fischer, G. and Scharff, E. (2000) Meta-design: design for designers. Proc DIS'00, 396--405.

Digital Library

[8]

Flanagan, J. (1954) The critical incident technique. Psych bulletin 51 (4), 327--358.

[9]

Folstad, A. (2007) Work--Domain Experts as Evaluators: Usability Inspection of Domain-Specific Work-Support Systems. IJHCI 22 (3), 217--245.

[10]

Frøkjær, E. and Hornbæk, K. (2005) Cooperative usability testing: complementing usability tests with user-supported interpretation sessions. Proc CHI'05, 1383--1386.

Digital Library

[11]

Gabbard, J., Hix, D., Swan II, J., Livingston, M., Höllerer, T., Julier, S., Brown, D. and Baillot, Y. (2003) Usability Engineering for Complex Interactive Systems Development. Proc Human Systems Integration Symposium, 1--13.

[12]

Gould, J. and Lewis, C. (1985) Designing for usability: key principles and what designers think. Comm ACM 28 (3), 300--311.

Digital Library

[13]

Gulliksen, J., Boivie, I. and Göransson, B. (2006) Usability professionals-current practices and future development. Interacting with Computers 18 (4), 568--600.

Digital Library

[14]

Gulliksen, J. and Sandblad, B. (1995) Domain-specific design of user interfaces. IJCHI 7 (2), 135--151.

Digital Library

[15]

Hewett, T., Baecker, R., Card, S., Carey, T., Gasen, J., Mantei, M., Perlman, G., Strong, G. and Verplank, W. (1996) ACM SIGCHI curricula for human-computer interaction (Technical Report). New York: ACM Press.

[16]

Hudson, W. (May 3, 2000) User-Centered Survey Results email posting to [email protected].

[17]

Millen, D. Rapid ethnography: Time deepening strategies for HCI field research. Proc DIS'00, 280--286.

Digital Library

[18]

Mirel, B. (2004) Interaction design for complex problem solving: Developing useful and usable software. Boston: Elsevier.

[19]

Muller, M., Matheson, L., Page, C. and Gallup, R. (1998) Methods & tools: participatory heuristic evaluation. interactions 5 (5), 13--18.

Digital Library

[20]

Nielsen, J. (2008) 25 Years in Usability. Jakob Nielsen's Alertbox, April 21, 2008.

[21]

Nielsen, J. (1993) Usability Engineering. Boston: Morgan Kaufmann.

[22]

Redish, J. (2007) Expanding Usability Testing to Evaluate Complex Systems. J Usability Studies 2 (3), 102--111.

Digital Library

[23]

Roesler, A. and Woods, D. (2007) Designing for Expertise. In: Schifferstein, H. and Hekkert, P. (eds.) Product Experience. San Diego: Elsevier.

[24]

Rubin, H.J. and Rubin, I. (2005) Qualitative Interviewing: The Art of Hearing Data. Thousand Oaks, CA: Sage Publications.

[25]

Schuler, D. and Namioka, A. (1993) Participatory Design: Principles and Practices. Mahwah, NJ: Lawrence Erlbaum.

Digital Library

[26]

Scott, K.M. (2009) Is usability obsolete? interactions 16 (3), 6--11.

Digital Library

[27]

Strauss, A. and Corbin, J. (1998) Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. New York: Sage Publications.

[28]

Suchman, L. (1995) Making work visible. Comm ACM 38 (9), 56--64.

Digital Library

[29]

Vicente, K. (1999) Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work. Mahwah, NJ: Lawrence Erlbaum.

[30]

Viitanen, J., Karjalainen, S., Kautonen, H. and Laukkanen, A. (2005) Challenges in Usability Evaluation of Expert Domain Products. Proc Intl Conf HCI '05.

[31]

Vredenburg, K., Mao, J., Smith, P. and Carey, T. (2002) A survey of user-centered design practice. Proc CHI'02, 471--478.

Digital Library

[32]

Wixon, D.R., Ramey, J., Holtzblatt, K., Beyer, H., Hackos, J., Rosenbaum, S., Page, C., Laakso, S.A. and Laakso, K.-P. (2002) Usability in practice: field methods evolution and revolution. Proc CHI '02 Extended Abstracts, 880--884.

Digital Library

Cited By

Ashtari NChilana P(2024)How New Developers Approach Augmented Reality Development Using Simplified Creation Tools: An Observational StudyMultimodal Technologies and Interaction10.3390/mti80400358:4(35)Online publication date: 22-Apr-2024
https://doi.org/10.3390/mti8040035
Liu ZYu X(2024)Development of a T2D App for Elderly Users: Participatory Design Study via Heuristic Evaluation and Usability TestingElectronics10.3390/electronics1319386213:19(3862)Online publication date: 29-Sep-2024
https://doi.org/10.3390/electronics13193862
Turbeville KMuengtaweepongsa JStevens SMoss JPon ALee KMehra CGutierrez Villalobos JKumar R(2024)LLM-powered Multimodal Insight Summarization for UX TestingProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685701(4-11)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3678957.3685701
Show More Cited By

Index Terms

Understanding usability practices in complex domains
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

On user behaviour adaptation under interface change
IUI '14: Proceedings of the 19th international conference on Intelligent User Interfaces

Different interfaces allow a user to achieve the same end goal through different action sequences, e.g., command lines vs. drop down menus. Interface efficiency can be described in terms of a cost incurred, e.g., time taken, by the user in typical ...
A comparison of synchronous remote and local usability studies for an expert interface
CHI EA '04: CHI '04 Extended Abstracts on Human Factors in Computing Systems

Synchronous remote usability studies can be a convenient and cost-effective alternative to conventional local usability studies. Although they are common in the field, there has been little research comparing synchronous remote usability studies with ...
User recalled occurrences of usability errors: implications on the user experience
CHI EA '03: CHI '03 Extended Abstracts on Human Factors in Computing Systems

Usability testing determines what problems thwart goal attainment, but what problems shape the user experience? This study gathered users recalled instances of frustration from using various technologies and categorized those frustrating incidents with ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2010

2690 pages

ISBN:9781605589299

DOI:10.1145/1753326

General Chair:
Elizabeth Mynatt
Georgia Institute of Technology
,
Program Chairs:
Geraldine Fitzpatrick
Vienna University of Technology
,
Scott Hudson
Carnegie Mellon University
,
Keith Edwards
Georgia Tech
,
Tom Rodden
University of Nottingham

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 April 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '10

Sponsor:

SIGCHI

CHI '10: CHI Conference on Human Factors in Computing Systems

April 10 - 15, 2010

Georgia, Atlanta, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

54
Total Citations
View Citations
1,330
Total Downloads

Downloads (Last 12 months)57
Downloads (Last 6 weeks)9

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ashtari NChilana P(2024)How New Developers Approach Augmented Reality Development Using Simplified Creation Tools: An Observational StudyMultimodal Technologies and Interaction10.3390/mti80400358:4(35)Online publication date: 22-Apr-2024
https://doi.org/10.3390/mti8040035
Liu ZYu X(2024)Development of a T2D App for Elderly Users: Participatory Design Study via Heuristic Evaluation and Usability TestingElectronics10.3390/electronics1319386213:19(3862)Online publication date: 29-Sep-2024
https://doi.org/10.3390/electronics13193862
Turbeville KMuengtaweepongsa JStevens SMoss JPon ALee KMehra CGutierrez Villalobos JKumar R(2024)LLM-powered Multimodal Insight Summarization for UX TestingProceedings of the 26th International Conference on Multimodal Interaction10.1145/3678957.3685701(4-11)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3678957.3685701
Hertzum M(2024)Concurrent or Retrospective Thinking Aloud in Usability Tests: A Meta-Analytic ReviewACM Transactions on Computer-Human Interaction10.1145/366532731:3(1-29)Online publication date: 17-May-2024
https://dl.acm.org/doi/10.1145/3665327
Kuang ELi MFan MShinohara K(2024)Enhancing UX Evaluation Through Collaboration with Conversational AI Assistants: Effects of Proactive Dialogue and TimingProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642168(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642168
Aly YMunteanu C(2024)Privacy-Conscious Design Requirements to Support Older Adults’ Health Information SeekingHCI for Cybersecurity, Privacy and Trust10.1007/978-3-031-61379-1_13(188-211)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-61379-1_13
Kuang E(2023)Crafting Human-AI Collaborative Analysis for User Experience EvaluationExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3577042(1-6)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3577042
Kuang EJahangirzadeh Soure EFan MZhao JShinohara K(2023)Collaboration with Conversational AI Assistants for UX Evaluation: Questions and How to Ask them (Voice vs. Text)Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581247(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581247
Kuang EJin XFan M(2022)“Merging Results Is No Easy Task”: An International Survey Study of Collaborative Data Analysis Practices Among UX PractitionersProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517647(1-16)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517647
Soure EKuang EFan MZhao J(2022)CoUX: Collaborative Visual Analysis of Think-Aloud Usability Test Videos for Digital InterfacesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.311482228:1(643-653)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1109/TVCG.2021.3114822
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents