research-article

Frustration: Still a Common User Experience

Authors:

Morten Hertzum,

Kasper HornbækAuthors Info & Claims

ACM Transactions on Computer-Human Interaction, Volume 30, Issue 3

Article No.: 42, Pages 1 - 26

https://doi.org/10.1145/3582432

Published: 10 June 2023 Publication History

Abstract

When computers unexpectedly delay or thwart goal attainment, frustration ensues. The central studies of the extent, content, and impact of such frustration were done more than 15 years ago. We revisit this issue after computers have become more mature and computer use is more extensive. To this end, we had 234 crowdsourced participants log the frustrating episodes they experienced with their computers during one hour of computer use. The average time lost due to frustrating episodes was between 11% and 20% of the one-hour period. Though this is less time lost than in the earlier studies, frustration remains a common user experience. While shorter, the median level of frustration during the episodes was high (7 on a 9-point scale). The frustration level correlated with task importance and time lost but was unaffected by computer experience and largely unaffected by computer self-efficacy. In addition, participants indicated that 84% of the episodes had happened before, that 87% could happen again, and that they were unable to resolve 26% of the episodes. This high rate of recurrence and lack of control likely added to the frustration level. The episodes spanned various issues pertaining to performance (49%), usability (36%), and utility (16%).

References

[1]

David Akers, Matthew Simpson, Robin Jeffries, and Terry Winograd. 2009. Undo and erase events as indicators of usability problems. In Proceedings of the CHI2009 Conference on Human Factors in Computing Systems. ACM, New York, 659–668. DOI:

Digital Library

[2]

Blake Alcott. 2005. Jevons’ paradox. Ecological Economics 54, 1 (2005), 9–21. DOI:

[3]

Ohoud Alharbi and Wolfgang Stuerzlinger. 2022. Auto-cucumber: The impact of autocorrection failures on users’ frustration. In Proceedings of the Graphics Interface 2022 Conference. Canadian Information Processing Society, Mississauga, ON.

[4]

Abram Amsel. 1992. Frustration theory - Many years later. Psychological Bulletin 112, 3 (1992), 396–399. DOI:

[5]

Mara S. Aruguete, Ho Huynh, Blaine L. Browne, Bethany Jurs, Emilia Flint, and Lynn E. McCutcheon. 2019. How serious is the ‘carelessness’ problem on Mechanical Turk? International Journal of Social Research Methodology 22, 5 (2019), 441–449. DOI:

[6]

Ryan S. J. d. Baker, Sidney K. D'Mello, Ma Mercedes T. Rodrigo, and Arthur C. Graesser. 2010. Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive-affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies 68, 4 (2010), 223–241. DOI:

Digital Library

[7]

Roy F. Baumeister, Ellen Bratslavsky, Catrin Finkenauer, and Kathleen D. Vohs. 2001. Bad is stronger than good. Review of General Psychology 5, 4 (2001), 323–370. DOI:

[8]

Leonard Berkowitz. 1989. Frustration-aggression hypothesis: Examination and reformulation. Psychological Bulletin 106, 1 (1989), 59–73. DOI:

[9]

Katherine Bessière, Irina Ceaparu, Jonathan Lazar, John Robinson, and Ben Shneiderman. 2003. Social and psychological influences on computer user frustration. In Media Access: Social and Psychological Dimensions of New Technology Use. Erik P. Bucy and John E. Newhagen (Eds.), Routledge, New York, 91–103. DOI:

[10]

Katie Bessière, John E. Newhagen, John P. Robinson, and Ben Shneiderman. 2006. A model for computer frustration: The role of instrumental and dispositional factors on incident, session, and post-session frustration and mood. Computers in Human Behavior 22, 6 (2006), 941–961. DOI:

[11]

Mark Blythe and Andrew Monk. 2018. Funology 2: From Usability to Enjoyment. Second edition. Springer, Cham.

[12]

Kirsten Boehner, Rogério DePaula, Paul Dourish, and Phoebe Sengers. 2007. How emotion is made and measured. International Journal of Human-Computer Studies 65, 4 (2007), 275–291. DOI:

Digital Library

[13]

Johannes Breuer and Malte Elson. 2017. Frustration-aggression theory. In The Wiley Handbook of Violence and Aggression, Peter Sturney (Ed.), Wiley, New York, 40:01–40:12. DOI:

[14]

Anders Bruun, Effie Lai-Chong Law, Matthias Heintz, and Lana H. A. Alkly. 2016. Understanding the relationship between frustration and the severity of usability problems: What can psychophysiological data (not) tell us? In Proceedings of the CHI2016 Conference on Human Factors in Computing Systems. ACM, New york, 3975–3987. DOI:

Digital Library

[15]

Irina Ceaparu, Jonathan Lazar, Katherine Bessiere, John Robinson, and Ben Shneiderman. 2004. Determining causes and severity of end-user frustration. International Journal of Human-Computer Interaction 17, 3 (2004), 333–356. DOI:

[16]

Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20, 1 (1960), 37–46. DOI:

[17]

Mihaly Csikszentmihalyi and Reed Larson. 2014. Validity and reliability of the experience-sampling method. In Flow and the Foundations of Positive Psychology, Mihaly Csikszentmihalyi (Ed.), Springer, Netherlands, 35–54.

[18]

John Dollard, Neal E. Miller, Leonard W. Doob, Orval H. Mowrer, and Robert R. Sears. 1939. Frustration and Aggression. Yale University Press, New Haven, CT.

[19]

Netia K. Dor-Shav and Mario Mikulincer. 1990. Learned helplessness, causal attribution, and response to frustration. The Journal of General Psychology 117, 1 (1990), 47–58. DOI:

[20]

Joe S. Dumas. 1989. Stimulating change through usability testing. ACM SIGCHI Bull. 21, 1 (1989), 37–44. DOI:

Digital Library

[21]

Nina Ferreri and Christopher B. Mayhorn. 2021. Individual differences in frustration and performance with online shopping activities. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 65, 1 (2021), 148–153. DOI:

[22]

Atul Gawande. 2018. The Upgrade: Why doctors hate their computers. New Yorker November 12 (2018), 62–73. Retrieved from https://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-computers. Accessed February 12, 2023.

[23]

Donald Golden. 1980. A plea for friendly software. ACM SIGSOFT Software Engineering Notes 5, 4 (1980), 4–5. DOI:

Digital Library

[24]

Lee Hadlington and Mark O. Scase. 2018. End-user frustrations and failures in digital technology: Exploring the role of fear of missing out, internet addiction and personality. Heliyon 4, 11 (2018), e00872. DOI:

[25]

Chris Harrison, Zhiquan Yeo, and Scott E. Hudson. 2010. Faster progress bars: Manipulating perceived duration with visual augmentations. In Proceedings of the CHI2010 Conference on Human Factors in Computing Systems. ACM, New York, 1545–1548. DOI:

Digital Library

[26]

Marc Hassenzahl. 2004. The interplay of beauty, goodness, and usability in interactive products. Human-Computer Interact. 19, 4 (2004), 319–349. DOI:

Digital Library

[27]

Morten Hertzum. 2010. Frustration: A common user experience. In DHRS2010: Proceedings of the 10th Danish Human-Computer Interaction Research Symposium. Computer Science Research Report #132, Morten Hertzum and Magnus Hansen (Eds.). Roskilde University, Roskilde, DK, 11–14.

[28]

Morten Hertzum. 2021. Reference values and subscale patterns for the task load index (TLX): A meta-analytic review. Ergonomics 64, 7 (2021), 869–878. DOI:

[29]

Morten Hertzum. 2022. Associations among workload dimensions, performance, and situational characteristics: A meta-analytic review of the task load index. Behaviour and Information Technology 41, 16 (2022), 3506–3518. DOI:

[30]

Weiyin Hong, Traci J. Hess, and Andrew Hardin. 2013. When filling the wait makes it feel longer: A paradigm shift perspective for managing online delay. MIS Quarterly. 37, 2 (2013), 383–406. Retrieved from https://www.jstor.org/stable/43825915.

Digital Library

[31]

Daniel Kahneman, Alan B. Krueger, David A. Schkade, Norbert Schwarz, and Arthur A. Stone. 2004. A survey method for characterizing daily life experience: The day reconstruction method. Science 306, 5702 (2004), 1776–1780. DOI:

[32]

Noriaki Kano. 1984. Attractive quality and must-be quality. Journal of the Japanese Society for Quality Control 14, 2 (1984), 39–48.

[33]

Jesper Kjeldskov, Mikael B. Skov, and Jan Stage. 2010. A longitudinal study of usability in health care: Does time heal? International Journal of Medical Informatics 79, 6 (2010), e135–e143. DOI:

[34]

Thomas K. Landauer. 1995. The Trouble with Computers: Usefulness, Usability, and Productivity. MIT Press, Cambridge, MA.

Digital Library

[35]

Jonathan Lazar, Jinjuan H. Feng, and Harry Hochheiser. 2017. Research Methods in Human-Computer Interaction (2nd ed.). Morgan Kaufmann, Cambridge, MA.

[36]

Jonathan Lazar, Adam Jones, Mary Hackley, and Ben Shneiderman. 2006. Severity and impact of computer user frustration: A comparison of student and workplace users. Interacting with Computers 18, 2 (2006), 187–207. DOI:

Digital Library

[37]

Jonathan Lazar, Adam Jones, and Ben Shneiderman. 2006. Workplace user frustration with computers: An exploratory investigation of the causes and severity. Behaviour and Information Technology 25, 3 (2006), 239–251. DOI:

[38]

Valerie Mendoza and David G. Novick. 2005. Usability over time. In Proceedings of the SIGDOC2005 Conference on Design of Communication. ACM, New York, 151–158. DOI:

Digital Library

[39]

Helena Mentis and Geri Gay. 2003. User recalled occurrences of usability errors: Implications on the user experience. In Proceedings of the CHI2003 Conference on Human Factors in Computing Systems: Extended Abstracts. ACM, New York, 736–737. DOI:

Digital Library

[40]

David G. Novick, Edith Elizalde, and Nathaniel Bean. 2007. Toward a more accurate view of when and how people seek help with computer applications. In Proceedings of the SIGDOC2007 Conference on Design of Communication. ACM, New York, 95–102. DOI:

Digital Library

[41]

David G. Novick, Baltazar Santaella, Aaron Cervantes, and Carlos Andrade. 2012. Short-term methodology for long-term usability. In Proceedings of the SIGDOC2012 Conference on Design of Communication. ACM, New York, 205–211. DOI:

Digital Library

[42]

Agnieszka Piasna, Wouter Zwysen, and Jan Drahokoupil. 2022. The Platform Economy in Europe: Results from the Second ETUI Internet and Platform Work Survey. Working paper 2022.05. European Trade Union Institute, Brussels.

[43]

Lisa Posch, Arnim Bleier, Fabian Flöck, Clemens M. Lechner, Katharina Kinder-Kurlanda, Denis Helic, and Markus Strohmaier. 2022. Characterizing the global crowd workforce: A cross-country comparison of crowdworker demographics. Human Computation 9, 1 (2022), 22–57. DOI:

[44]

Judith Ramsay, Alessandro Barbesi, and Jenny Preece. 1998. A psychological investigation of long retrieval times on the world wide web. Interacting with Computers 10, 1 (1998), 77–86. DOI:

[45]

Kendra Reed, D. Harold Doty, and Douglas R. 2005. The impact of aging on self-efficacy and computer skill acquisition. Journal of Managerial Issues 17, 2 (2005), 212–228. Retrieved from https://www.jstor.org/stable/40604496.

[46]

Pertti Saariluoma and Jussi P. P. Jokinen. 2014. Emotional dimensions of user experience: A user psychological analysis. International Journal of Human-Computer Interaction 30, 4 (2014), 303–320. DOI:

[47]

Steven C. Seow. 2008. Designing and Engineering Time: The Psychology of Time Perception in Software. Addison-Wesley, Upper Saddle River, NJ. DOI:

[48]

Brian Shackel. 1985. Human factors and usability - Whence and whither? In Software-Erqonomie’85: Mensch-Computer-lnteraktion. 13–31.

[49]

Manoj K. Sharma, Shweta Sunil, B. N. Roopesh, Preeti M. Galagali, Nitin Anand, Pranjali C. Thakur, Priya Singh, S. J. Ajith, and Keshava D. Murthy. 2020. Digital failure: An emerging reason of anger expression among adolescents. Industrial Psychiatry Journal 29, 2 (2020), 335–338. DOI:

[50]

Saul Shiffman, Arthur A. Stone, and Michael R. Hufford. 2008. Ecological momentary assessment. Annual Review of Clinical Psychology 4, (2008), 1–32. DOI:

[51]

Ben Shneiderman. 2000. Universal usability. Communications of the ACM 43, 5 (2000), 84–91. DOI:

Digital Library

[52]

Donna Spencer. 2009. Card sorting: Designing Usable Categories. Rosenfeld, New York.

[53]

Rosemary Spencer, Enrico Coiera, and Pamela Logan. 2004. Variation in communication loads on clinical staff in the emergency department. Annals of Emergency Medicine 44, 3 (2004), 268–273. DOI:

[54]

Alexandre N. Tuch and Kasper Hornbæk. 2015. Does Herzberg's notion of hygienes and motivators apply to user experience? ACM Transactions on Computer-Human Interaction 22, 4 (2015), 1–24. DOI:

Digital Library

[55]

Rongjun Yu. 2016. The neural basis of frustration state. In Neuroimaging Personality, Social Cognition, and Character, John R. Absher and Jasmin Cloutier (Eds.). Academic Press, Amsterdam, 223–243. DOI:

[56]

Nichole K. Zimmerman, Everett Sambrook, and Jonathan S. Gore. 2014. The effects of a computer malfunction on subsequent task performance. Behaviour and Information Technology 33, 9 (2014), 874–881. DOI:

Digital Library

Cited By

Tan YLeow MOng L(2024)Evaluation of the extended TAM for digital signage augmented roadshow (DiSAR) using PLS-SEMFrontiers in Computer Science10.3389/fcomp.2024.14499086Online publication date: 10-Dec-2024
https://doi.org/10.3389/fcomp.2024.1449908
Krogstad A(2024)Se deg om i vredeNorsk sosiologisk tidsskrift10.18261/nost.8.6.28:6(9-26)Online publication date: 25-Nov-2024
https://doi.org/10.18261/nost.8.6.2
Seifi HBhatia AHornbæk K(2024)Charting User Experience in Physical Human–Robot InteractionACM Transactions on Human-Robot Interaction10.1145/365905813:2(1-29)Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3659058
Show More Cited By

Index Terms

Frustration: Still a Common User Experience
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies

Recommendations

The effectiveness of social agents in reducing user frustration
CHI EA '06: CHI '06 Extended Abstracts on Human Factors in Computing Systems

A study was conducted to evaluate the effectiveness of social agents in reducing user frustration. The particular type of agent studied reacted to users' facial expressions while they browsed through a shopping website. While highly frustrated users ...
Empathic agents to reduce user frustration: The effects of varying agent characteristics

There is now growing interest in the development of computer systems which respond to users' emotion and affect. We report three small-scale studies (with a total of 42 participants), which investigate the extent to which affective agents, using ...
Emotional user experience

Emotional experience has become an important topic in human-technology interaction research and design. Nevertheless, such research and design often lacks a proper explanatory basis and methodologically robust operationalisation. In this article, a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer-Human Interaction

ACM Transactions on Computer-Human Interaction Volume 30, Issue 3

June 2023

544 pages

ISSN:1073-0516

EISSN:1557-7325

DOI:10.1145/3604411

Editors:
Kristina Höök
KTH Royal Institute of Technology, Sweden
,
Kasper Hornbæk
University of Copenhagen, Denmark

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 June 2023

Online AM: 30 January 2023

Accepted: 09 January 2023

Revised: 20 December 2022

Received: 11 October 2022

Published in TOCHI Volume 30, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
1,788
Total Downloads

Downloads (Last 12 months)768
Downloads (Last 6 weeks)69

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tan YLeow MOng L(2024)Evaluation of the extended TAM for digital signage augmented roadshow (DiSAR) using PLS-SEMFrontiers in Computer Science10.3389/fcomp.2024.14499086Online publication date: 10-Dec-2024
https://doi.org/10.3389/fcomp.2024.1449908
Krogstad A(2024)Se deg om i vredeNorsk sosiologisk tidsskrift10.18261/nost.8.6.28:6(9-26)Online publication date: 25-Nov-2024
https://doi.org/10.18261/nost.8.6.2
Seifi HBhatia AHornbæk K(2024)Charting User Experience in Physical Human–Robot InteractionACM Transactions on Human-Robot Interaction10.1145/365905813:2(1-29)Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3659058
Wester JSchrills TPohl Hvan Berkel N(2024)“As an AI language model, I cannot”: Investigating LLM Denials of User RequestsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642135(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642135
Benramdane MKornyshova E(2024)Reinforcement Learning to Personalize User eXperience within Digital Business Ecosystems2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC61105.2024.00085(584-593)Online publication date: 2-Jul-2024
https://doi.org/10.1109/COMPSAC61105.2024.00085
Hertzum M(2024)Inferior, Yet Transformative: the User Experience with Robotic Vacuum CleanersInteracting with Computers10.1093/iwc/iwae00336:1(16-29)Online publication date: 27-Feb-2024
https://doi.org/10.1093/iwc/iwae003
Chhabra SKaushal VGirija S(2024)Determining the causes of user frustration in the case of conversational chatbotsBehaviour & Information Technology10.1080/0144929X.2024.2362956(1-19)Online publication date: 13-Jun-2024
https://doi.org/10.1080/0144929X.2024.2362956
Lintunen ESalmela VJarre PHeikkinen TKilpeläinen MJokela MOulasvirta A(2024)Cognitive abilities predict performance in everyday computer tasksInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103354192(103354)Online publication date: Dec-2024
https://doi.org/10.1016/j.ijhcs.2024.103354
Desai SSharma TSaha P(2023)Using ChatGPT in HCI Research—A TrioethnographyProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3603755(1-6)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3571884.3603755

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents