Cited By
View all- Zhang GHansen SBehrens OHansen J(2024)Saccade response testing during teleoperations with a head-mounted displayCognition, Technology and Work10.1007/s10111-023-00750-626:1(127-138)Online publication date: 1-Feb-2024
Measuring cognitive workload in automated knowledge work environments: a systematic literature review
Pages 557 - 587
Abstract
Traditionally, automation was introduced to alleviate workload associated with tedious and repetitive tasks. Recently, automation is being used to augment knowledge work, which includes high-level cognitive activities. As automated systems are being extended to perform skill-based tasks, the work required of humans may be altered, potentially affecting their cognitive workloads. Researchers have investigated the influences of automation on cognitive workload across different domains and tasks by assessing changes in task performance, perceived (subjective) workload, and physiological states. A major challenge in comparing results and drawing inferences across studies is that a profusion of measures is often used to assess cognitive workload. The experimental tasks employed across many domains further complicates synthesizing findings. Thus, the aim of this review is to examine how cognitive workload is assessed when at least two different measures of cognitive workload are used in research focused on human-automated knowledge work. To accomplish this aim, the various approaches employed to measure cognitive workload were first summarized. Then, automated and cognitive experimental tasks were classified, utilizing existing frameworks, to identify associations, dissociations, and insensitivities across task types. Finally, recommendations were provided for aligning task types, study designs, and measurement selections, along with expanding the types of tasks and measures used when studying automation applications supporting knowledge work.
References
[1]
Annett J Subjective rating scales: science or art? Ergonomics 2002 45 14 966-987
[2]
Beer RD, Rieth CA, Tran R, Cook MB (2017) Framework for multi-human multi-robot interaction: impact of operational context and team configuration on interaction task demands. In: 2017 AAAI spring symposium—technical report, SS-17-01-, pp 287–294
[3]
Biondi FN, Balasingam B, and Ayare P On the cost of detection response task performance on cognitive load Hum Factors 2021 63 5 804-812
[4]
Boehm U, Matzke D, Gretton M, Castro S, Cooper J, Skinner M, Heathcote A, et al. Real-time prediction of short-timescale fluctuations in cognitive workload Cognit Res Princ Implic 2021 6 1 1-29
[5]
Brookhuis KA, van Driel CJG, Hof T, van Arem B, and Hoedemaeker M Driving with a congestion assistant; mental workload and acceptance Appl Ergon 2009 40 6 1019-1025
[6]
Brown SD and Heathcote A The simplest complete model of choice response time: linear ballistic accumulation Cogn Psychol 2008 57 3 153-178
[7]
Brünken R, Plass JL, and Leutner D Direct measurement of cognitive load in multimedia learning Educ Psychol 2003 38 1 53-61
[8]
Castro SC, Strayer DL, Matzke D, and Heathcote A Cognitive workload measurement and modeling under divided attention J Exp Psychol Hum Percept Perform 2019 45 6 826-839
[9]
Charles RL and Nixon J Measuring mental workload using physiological measures: a systematic review Appl Ergon 2019 74 May 2018 221-232
[10]
Chen SI, Visser TAW, Huf S, and Loft S Optimizing the balance between task automation and human manual control in simulated submarine track management J Exp Psychol Appl 2017 23 3 240-262
[11]
Colcombe S and Kramer AF Fitness effects on the cognitive function of older adults: a meta-analytic study Perspect Psychol Sci 2003 13 2 213-217
[12]
De Visser E and Parasuraman R Adaptive aiding of human-robot teaming: effects of imperfect automation on performance, trust, and workload J Cognit Eng Decis Mak 2011 5 2 209-231
[13]
Endsley MR and Kaber DB Level of automation effects on performance, situation awareness and workload in a dynamic control task Ergonomics 1999 42 3 462-492
[14]
Endsley MR and Kiris EO The out-of-the-loop performance problem and level of control in automation Hum Fact 1995 37 2 381-394
[15]
Evans DC and Fendley M A multi-measure approach for connecting cognitive workload and automation Int J Hum Comput Stud 2017 97 182-189
[16]
Fasola J and Mataric M A socially assistive robot exercise coach for the elderly J Hum Robot Interact 2013
[17]
Galy E, Cariou M, and Mélan C What is the relationship between mental workload factors and cognitive load types? Int J Psychophysiol 2012 83 3 269-275
[18]
Ghani U, Signal N, Niazi IK, and Taylor D ERP based measures of cognitive workload: a review Neurosci Biobehav Rev 2020 118 18-26
[19]
Hameed S, Sarter N (2009) Context-sensitive information presentation: Integrating adaptive and adaptable approaches to display design. In: Proceedings of the 53rd annual meeting of the human factors and ergonomics society. Human Factors and Ergonomics Society, Santa Monica, pp 1694–1698.
[20]
Hancock PA On the process of automation transition in multitask human–machine systems IEEE Trans Syst Man Cybern Part A Syst Hum 2007 37 4 586-598
[21]
Hancock PA Procedure and dynamic display relocation on performance in a multitask environment IEEE Trans Syst Man Cybern Part A Syst Hum 2007 37 1 47-57
[22]
Hancock PA Whither workload? Mapping a path for its future development Commun Comput Inf Sci 2017 726 3-17
[23]
Hancock PA and Matthews G Workload and performance: associations, insensitivities, and dissociations Hum Factors 2019 61 3 374-392
[24]
Hancock PA and Scallen SF The performance and workload effects of task re-location during automation Displays 1997 17 2 61-68
[25]
Hart SG and Staveland LE Hancock PA and Meshkati N Development of NASA-TLX (Task Load Index): results of empirical and theoretical research Human mental workload 1988 Amsterdam North Holland Press
[26]
Hughes AM, Hancock GM, Marlow SL, Stowers K, and Salas E Cardiac measures of cognitive workload: a meta-analysis Hum Factors 2019 61 3 393-414
[27]
Jiang X, Gramopadhye AK, Melloy BJ, and Grimes LW Evaluation of best system performance: human, automated, and hybrid inspection systems Hum Factors Ergon Manuf 2003 13 2 137-152
[28]
Jou Y-T, Yenn T-C, Lin CJ, Yang C-W, and Chiang C-C Evaluation of operators’ mental workload of human–system interface automation in the advanced nuclear power plants Nucl Eng Design 2009 239 11 2537-2542
[29]
Jovanović M, De Angeli A, McNeill A, and Coventry L User requirements for inclusive technology for older adults Int J Hum Comput Interact 2021 37 20 1947-1965
[30]
Kaber DB and Endsley MR The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task Theor Issues Ergon Sci 2004 5 2 113-153
[31]
Kaber DB, Onal E, and Endsley MR Design of automation for telerobots and the effect on performance, operator situation awareness, and subjective workload Hum Factors Ergon Manuf 2000 10 4 409-430
[32]
Kaber DB, Wright MC, and Sheik-Nainar MA Investigation of multi-modal interface features for adaptive automation of a human-robot system Int J Hum Comput Stud 2006 64 6 527-540
[33]
Kaber D, Perry C, Segall N, and Sheik-Nainar M Workload state classification with automation during simulated air traffic control Int J Aviat Psychol 2007 17 4 371-390
[34]
Li H, Wickens CD, Sarter N, and Sebok A Stages and levels of automation in support of space teleoperations Hum Factors 2014 56 6 1050-1061
[35]
Lohani M, Payne BR, and Strayer DL A review of psychophysiological measures to assess cognitive states in real-world driving Front Hum Neurosci 2019 13 57
[36]
Lorenz B, Di Nocera F, Röttger S, and Parasuraman R Automated fault-management in a simulated spaceflight micro-world Aviat Space Environ Med 2002 73 9 886-897
[37]
Manzey D, Reichenbach J, and Onnasch L Human performance consequences of automated decision aids: the impact of degree of automation and system experience J Cognit Eng Decis Mak 2012 6 1 57-87
[38]
Marmeleira J, Ferreira I, Melo F, and Godinho M Associations of physical activity with driving-related cognitive abilities in older drivers: an exploratory study Percept Motor Skills 2012 115 2 521-533
[39]
Masalonis AJ, Duley JA, and Parasuraman R Effects of manual and autopilot control on mental workload and vigilance during simulated general aviation flight Transp Hum Factors 1999 1 2 187-200
[40]
Matthews G, Reinerman-Jones LE, Barber DJ, and Abich J The psychometrics of mental workload: multiple measures are sensitive but divergent Hum Factors 2015 57 1 125-143
[41]
Matthews G, De Winter J, and Hancock PA What do subjective workload scales really measure? Operational and representational solutions to divergence of workload measures Theor Issues Ergon Sci 2020 21 4 369-396
[42]
McDonald AD, Alambeigi H, Engström J, Markkula G, Vogelpohl T, Dunne J, and Yuma N Toward computational simulations of behavior during automated driving takeovers: a review of the empirical and modeling literatures Hum Factors 2019 61 4 642-688
[43]
Metzger U and Parasuraman R Automation in future air traffic management: effects of decision aid reliability on controller performance and mental workload Hum Factors 2005 47 1 35-49
[44]
Mitchell GW Effects of automation on the structure and functioning of banking Am Econ Assoc 1966 56 1/2 159-166
[45]
Moher D, Liberati A, Tetzlaff J, Altman DG, Altman D, Antes G, and Tugwell P Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement PLoS Med 2009
[46]
Navarro J, Heuveline L, Avril E, and Cegarra J Influence of human-machine interactions and task demand on automation selection and use Ergonomics 2018 61 12 1601-1612
[47]
Newlin Lew K, Nowlin S, Chyun D, and Melkus GD State of the science: diabetes self-management interventions West J Nurs Res 2014 36 9 1111-1157
[48]
Nisser T and Westin C Human factors challenges in unmanned aerial vehicles: a literature review 2006 Ljungbyhed School of Aviation of the Lund University
[49]
Onnasch L, Wickens CD, Li H, and Manzey D Human performance consequences of stages and levels of automation Hum Factors 2014 56 3 476-488
[50]
Paas FG and Van Merrienboer JJG The efficiency of instructional conditions: an approach to combine mental effort and performance measures Hum Factors 1993 35 4 737-743
[51]
Paas FG, Van Merriënboer JJ, and Adam JJ Measurement of cognitive load in instructional research Percept Mot Skills 1994 79 1 419-430
[52]
Parasuraman R, Sheridan TB, and Wickens CD A model for types and levels of human interaction with automation IEEE Trans Syst Man Cybern Part A Syst Hum 2000 30 3 286-297
[53]
Parasuraman R, Cosenzo KA, and De Visser E Adaptive automation for human supervision of multiple uninhabited vehicles: effects on change detection, situation awareness, and mental workload Mil Psychol 2009 21 2 270-297
[54]
Prinzel LJ III, Freeman FG, Scerbo MW, Mikulka PJ, and Pope AT Effects of a psychophysiological system for adaptive automation on performance, workload, and the event-related potential P300 component Hum Factors 2003 45 4 601-614
[55]
Reid GB and Nygren TE The subjective workload assessment technique: a scaling procedure for measuring mental workload Advances in psychology 1988 Amsterdam North Holland Press 185-218
[56]
Rouse WB Human–computer interaction in the control of dynamic systems Comput Surv 1981 13 1 71-99
[57]
Rovira E, McGarry K, and Parasuraman R Effects of imperfect automation on decision making in a simulated command and control task Hum Factors 2007 49 1 76-87
[58]
Sauer J, Nickel P, and Wastell D Designing automation for complex work environments under different levels of stress Appl Ergon 2013 44 1 119-127
[59]
Scerbo MW Parasuraman R and Rizzo M Adaptive automation Neuroergonomics: the brain at work 2007 New York Oxford University Press 239-252
[60]
Sehrawat, Choudhury TA, Raj G (2017) Surveillance drone for disaster management and military security. In: 2017 international conference on computing, communication and automation (ICCCA), pp 470–475.
[61]
Sheridan TB and Parasuraman R Nickerson RS Human-automation interaction Reviews of human factors and ergonomics 2006 Santa Monica Human Factors and Ergonomics Society 89-129
[62]
Sheridan TB and Verplank WL Human and computer control of undersea teleoperators (Technical Report) 1978 Cambridge MIT Man-Machine Systems Laboratory
[63]
Squire PN and Parasuraman R Effects of automation and task load on task switching during human supervision of multiple semi-autonomous robots in a dynamic environment Ergonomics 2010 53 8 951-961
[64]
Stapel J, Mullakkal-Babu FA, and Happee R Automated driving reduces perceived workload, but monitoring causes higher cognitive load than manual driving Transp Res Part F 2019 60 590-605
[65]
Stojmenova K and Sodnik J Detection-response task-uses and limitations Sensors 2018 18 2 594
[66]
Van Acker BB, Parmentier DD, Vlerick P, and Saldien J Understanding mental workload: from a clarifying concept analysis toward an implementable framework Cogn Technol Work 2018 20 3 351-365
[67]
Vanderhaegen F and Carsten O Can dissonance engineering improve risk analysis of human–machine systems? Cognit Technol Work 2017 19 1 1-12
[68]
Vanderhaegen F, Crevits I, Debernard S, and Millot P Human–machine cooperation: toward an activity regulation assistance for different air traffic control levels Int J Hum Comput Interact 1994 6 1 65-104
[69]
Vu K-PL, Strybel TZ, Battiste V, Lachter J, Dao A-QV, Brandt S, Ligda S, and Johnson W Pilot performance in trajectory-based operations under concepts of operation that vary separation responsibility across pilots, air traffic controllers, and automation Int J Hum Comput Interact 2012 28 2 107-118
[70]
Weinstein R RFID: a technical overview and its application to the enterprise IT Prof 2005 7 3 27-33
[71]
Wilson GF and Russell CA Performance enhancement in an uninhabited air vehicle task using psychophysiologically determined adaptive aiding Hum Factors 2007 49 6 1005-1018
[72]
Woods DD Parasuraman R and Mouloua M Decomposing automation: apparent simplicity, real complexity Automation and human performance: theory and applications 1996 Mahwah Erlbaum 3-17
[73]
Zhang W, Feltner D, Shirley J, Swangnetr M, Kaber D (2016) Unmanned aerial vehicle control interface design and cognitive workload: a constrained review and research framework. In: 2016 IEEE international conference on systems, man, and cybernetics (SMC), pp 001821–001826.
[74]
Zijlstra FRH Efficiency in work behaviour: a design approach for modern tools 1993 Delft Delft University Press
Index Terms
- Measuring cognitive workload in automated knowledge work environments: a systematic literature review
Index terms have been assigned to the content through auto-classification.
Recommendations
A Survey on Measuring Cognitive Workload in Human-Computer Interaction
The ever-increasing number of computing devices around us results in more and more systems competing for our attention, making cognitive workload a crucial factor for the user experience of human-computer interfaces. Research in Human-Computer Interaction ...
Automation of systematic literature reviews: A systematic literature review
Abstract ContextSystematic Literature Review (SLR) studies aim to identify relevant primary papers, extract the required data, analyze, and synthesize results to gain further and broader insight into the investigated domain. ...
Knowledge Work Activities of a Systematic Review Task
ABSTRACTMost workers' activities in the workplace can be classed as knowledge work (KW). Previous research has described workers' tasks very broadly, making it difficult to know the actual work. We hypothesise that deconstructing what goes on inside a ...
Comments
Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2022.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 01 November 2022
Accepted: 13 June 2022
Received: 02 April 2022
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 14 Jan 2025
Other Metrics
Citations
Cited By
View all- Zhang GHansen SBehrens OHansen J(2024)Saccade response testing during teleoperations with a head-mounted displayCognition, Technology and Work10.1007/s10111-023-00750-626:1(127-138)Online publication date: 1-Feb-2024