Article

Predictive human performance modeling made easy

Authors:

Bonnie E. John,

Konstantine Prevas,

Dario D. Salvucci,

Ken KoedingerAuthors Info & Claims

CHI '04: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 455 - 462

https://doi.org/10.1145/985692.985750

Published: 25 April 2004 Publication History

Abstract

Although engineering models of user behavior have enjoyed a rich history in HCI, they have yet to have a widespread impact due to the complexities of the modeling process. In this paper we describe a development system in which designers generate predictive cognitive models of user behavior simply by demonstrating tasks on HTML mock-ups of new interfaces. Keystroke-Level Models are produced automatically using new rules for placing mental operators, then implemented in the ACT-R cognitive architecture. They interact with the mock-up through integrated perceptual and motor modules, generating behavior that is automatically quantified and easily examined. Using a query-entry user interface as an example [19], we demonstrate that this new system enables more rapid development of predictive models, with more accurate results, than previously published models of these tasks.

References

[1]

Anderson, J. R., and Lebiere, C. The Atomic Components of Thought. Lawrence Erlbaum Associates, Hillsdale, NJ, USA, 1998.

[2]

Baumeister, L., John, B. E. and Byrne, M. A Comparison of Tools for Building GOMS Models. CHI 2000, ACM Conference on Human Factors in Computing Systems, CHI Letters 2(1), 502--509.

Digital Library

[3]

Beard, David V., Smith, Dana K. and Denelsbeck, Kevin M. Quick and Dirty GOMS: A Case Study of Computed Tomography, Human-Computer Interaction 11, 2 (1996), 157--180.

Digital Library

[4]

Byrne, M. D. ACT-R/PM and menu selection: Applying a cognitive architecture to HCI. International Journal of Human-Computer Studies, 55 (2001), 41--84.

Digital Library

[5]

Byrne, M. D., Wood, S. D., Sukaviriya, P. N., Foley, J. D. and Kieras, D. Automating Interface Evaluation, Proceedings of CHI 1994, ACM Press (1994), 232--237.

Digital Library

[6]

Card, S. K., Moran, T.P. and Newell, A. The keystroke-level model for user performance time with interactive systems. Communications of the ACM 23, 7 (1980), 396--410.

Digital Library

[7]

Card, S. K., Moran, T.P. and Newell, A. The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates, Hillsdale, NJ, USA (1983).

Digital Library

[8]

Dix, A., Finlay, J., Abowd, G., and Beale, R. Human-Computer Interaction (2nd ed.) Prentice Hall Europe, London, UK, 1998.

Digital Library

[9]

Gray, W. D., John, B. E., and Atwood, M. E. Project Ernestine: A validation of GOMS for prediction and explanation of real-world task performance. Human-Computer Interaction, 8 (1993), 237--309.

Digital Library

[10]

Hudson, S. E., John, B. E., Knudsen, K., and Byrne, M. D. A tool for creating predictive performance models from user interface demonstrations. UIST'99: Proceedings of the ACM Symposium on User Interface Software and Technology, CHI Letters 1(1), 93--102.

Digital Library

[11]

Hudson, S., and Stasko, J. Animation Support in User Interface Toolkits: Flexible, Robust, and Reusable Abstractions. Proceedings of the ACM Symposium on User Interface Software and Technology, ACM Press (1995), 57--67.

Digital Library

[12]

John, B. E. Toward a deeper comparison of methods: A reaction to Nielsen and Phillips and new data. Proceedings Companion of CHI 1994, ACM Press (1994), 285--286.

Digital Library

[13]

John, B. E. TYPIST: A Theory of Performance In Skilled Typing. Human-Computer Interaction 11, 4 (1996), 321--355.

Digital Library

[14]

John, B. E., and Kieras, D. E. Using GOMS for user interface design and evaluation: Which technique? ACM Transactions on Computer-Human Interaction 3, 4 (1996), 287--319.

Digital Library

[15]

John, B., Vera, A., Matessa, M., Freed, M., and Remington, R. Automating CPM-GOMS. CHI 2002, ACM Conference on Human Factors in Computing Systems, CHI Letters 4(1), 147--154.

Digital Library

[16]

Kieras, D. E., Wood, S. D., Abotel, K., and Hornof, A. GLEAN: A Computer-Based Tool for Rapid GOMS Model Usability Evaluation of User Interface. Proceedings of the ACM Symposium on User Interface Software and Technology, ACM Press (1995), 91--100

Digital Library

[17]

Koedinger, K. R., Aleven, V., and Heffernan, N. Toward a rapid development environment for Cognitive Tutors. Artificial Intelligence in Education: Shaping the Future of Learning through Intelligent Technologies, Proceedings of AI-ED 2003, IOS Press (2003), 455--457.

[18]

Lane, D. M., Napier, H. A., Batsell, R. R. and Naman, J. Predicting the skilled use of hierarchical menus with the keystroke-level model. Human-Computer Interaction 8, 2 (1993), 185--192.

Digital Library

[19]

Nielsen, J., and Phillips, V. A. Estimating the relative usability of two interfaces: heuristic, formal, and empirical methods compared. Proceedings of CHI 1993, ACM Press (1993), 214--221.

Digital Library

[20]

Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., and Carey, T. Human-Computer Interaction. Addison Wesley, Wonkingham, UK, 1994.

Digital Library

[21]

Raskin, J. The Humane Interface. Addison Wesley, Boston, MA, USA, 2000

[22]

Salvucci, D. D., and Lee, F. J. Simple cognitive modeling in a complex cognitive architecture. CHI 2003, ACM Conference on Human Factors in Computing Systems, CHI Letters 5(1), 265--272.

Digital Library

[23]

Shneiderman, B. Designing the User Interface (3rd ed.). Addison Wesley, Reading, MA, USA, 1998.

Digital Library

[24]

Williams, K. E. Automating the cognitive task modeling process: An extension to GOMS for HCI. Proceedings of the Fifth International Conference on Human-Computer Interaction Poster Sessions: Abridged Proceedings, 3 (1993), 182.

Cited By

von Janczewski NKraus JEngeln ABaumann M(2024)IVIPAT: an in-vehicle information processing analysis tool to optimize user interaction flowsCognition, Technology and Work10.1007/s10111-024-00752-y26:2(247-265)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10111-024-00752-y
Kalenderian EWalji M(2024)Applying HCI Principles in Designing Usable Systems for DentistryHuman Computer Interaction in Healthcare10.1007/978-3-031-69947-4_14(345-370)Online publication date: 14-Nov-2024
https://doi.org/10.1007/978-3-031-69947-4_14
Ritter FYeh MStager SMcDermott AWeyhrauch P(2023)The Effect of Task Fidelity on Learning Curves: A Synthetic AnalysisInternational Journal of Human–Computer Interaction10.1080/10447318.2022.216186339:11(2253-2267)Online publication date: 29-Jan-2023
https://doi.org/10.1080/10447318.2022.2161863
Show More Cited By

Index Terms

Predictive human performance modeling made easy
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

The GOMS family of user interface analysis techniques: comparison and contrast

Sine the publication of The Psychology of Human-Computer Interaction, the GOMS model has been one of the most widely known theoretical concepts in HCI. This concept has produced severval GOMS analysis techniques that differ in appearance and form, ...
Using GOMS for user interface design and evaluation: which technique?

Since the seminal book, The Psychology of Human-Computer Interaction, the GOMS model has been one of the few widely known theoretical concepts in human-computer interaction. This concept has spawned much research to verify and extend the original work ...
Toward Cognitive Modeling for Predicting Usability
Proceedings of the 13th International Conference on Human-Computer Interaction. Part I: New Trends

Historically, predictive human performance modeling has been successful at predicting the task execution time of skilled users on a desktop computer. More recent work has predicted novice behavior in web searches. This paper reports on a collaborative ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '04: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2004

742 pages

ISBN:1581137028

DOI:10.1145/985692

Conference Chairs:
Elizabeth Dykstra-Erickson
Kinoma
,
Manfred Tscheligi
CURE, Austria

Copyright © 2004 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

CHI04

Sponsor:

CHI04: CHI 2004 Conference on Human Factors in Computing Systems

April 24 - 29, 2004

Vienna, Austria

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

152
Total Citations
View Citations
2,759
Total Downloads

Downloads (Last 12 months)60
Downloads (Last 6 weeks)11

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

von Janczewski NKraus JEngeln ABaumann M(2024)IVIPAT: an in-vehicle information processing analysis tool to optimize user interaction flowsCognition, Technology and Work10.1007/s10111-024-00752-y26:2(247-265)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10111-024-00752-y
Kalenderian EWalji M(2024)Applying HCI Principles in Designing Usable Systems for DentistryHuman Computer Interaction in Healthcare10.1007/978-3-031-69947-4_14(345-370)Online publication date: 14-Nov-2024
https://doi.org/10.1007/978-3-031-69947-4_14
Ritter FYeh MStager SMcDermott AWeyhrauch P(2023)The Effect of Task Fidelity on Learning Curves: A Synthetic AnalysisInternational Journal of Human–Computer Interaction10.1080/10447318.2022.216186339:11(2253-2267)Online publication date: 29-Jan-2023
https://doi.org/10.1080/10447318.2022.2161863
Koca EGreen P(2022)Databases for Estimating Task Element Times: An OverviewProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/107118132266131466:1(1591-1595)Online publication date: 27-Oct-2022
https://doi.org/10.1177/1071181322661314
Zhou XTeng FDu XLi JJin MXue C(2022)H-GOMS: a model for evaluating a virtual-hand interaction system in virtual environmentsVirtual Reality10.1007/s10055-022-00674-y27:2(497-522)Online publication date: 22-Jul-2022
https://doi.org/10.1007/s10055-022-00674-y
Pandey STaffese T(2021)Using Performance Predictions to Evaluate Two-factor Authentication Setup ProcessesProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/107118132165132865:1(999-1003)Online publication date: 12-Nov-2021
https://doi.org/10.1177/1071181321651328
Park JMcKenzie JShahini FZahabi M(2021)Application of Cognitive Performance Modeling for Usability Evaluation of Emergency Medical Services In-Vehicle TechnologyProceedings of the Human Factors and Ergonomics Society Annual Meeting10.1177/107118132064102164:1(72-76)Online publication date: 9-Feb-2021
https://doi.org/10.1177/1071181320641021
Yuan HLi SRusconi P(2021)CogTool+ACM Transactions on Computer-Human Interaction10.1145/344753428:2(1-38)Online publication date: 17-Apr-2021
https://dl.acm.org/doi/10.1145/3447534
Cunha DDuarte RCunha C(2021)KLM-GOMS Detection of Interaction Patterns Through the Execution of Unplanned TasksComputational Science and Its Applications – ICCSA 202110.1007/978-3-030-86960-1_15(203-219)Online publication date: 11-Sep-2021
https://doi.org/10.1007/978-3-030-86960-1_15
Theofilou SVardas NKatsanos C(2021)FLM-2A: Towards Automated HCI Modeling of Android Applications Based on a Modified Version of the Keystroke Level ModelHuman-Computer Interaction. Theory, Methods and Tools10.1007/978-3-030-78462-1_25(329-342)Online publication date: 3-Jul-2021
https://doi.org/10.1007/978-3-030-78462-1_25
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents