research-article

Psycho-physiological measures for assessing cognitive load

Authors:

Eija Haapalainen,

Jodi F. Forlizzi,

Anind K. DeyAuthors Info & Claims

UbiComp '10: Proceedings of the 12th ACM international conference on Ubiquitous computing

Pages 301 - 310

https://doi.org/10.1145/1864349.1864395

Published: 26 September 2010 Publication History

Abstract

With a focus on presenting information at the right time, the ubicomp community can benefit greatly from learning the most salient human measures of cognitive load. Cognitive load can be used as a metric to determine when or whether to interrupt a user. In this paper, we collected data from multiple sensors and compared their ability to assess cognitive load. Our focus is on visual perception and cognitive speed-focused tasks that leverage cognitive abilities common in ubicomp applications. We found that across all participants, the electrocardiogram median absolute deviation and median heat flux measurements were the most accurate at distinguishing between low and high levels of cognitive load, providing a classification accuracy of over 80% when used together. Our contribution is a real-time, objective, and generalizable method for assessing cognitive load in cognitive tasks commonly found in ubicomp systems and situations of divided attention.

References

[1]

}}Ackerman, P.L. and Cianciolo, A.T. (2000) Cognitive, perceptual-speed, and psychomotor determinants of individual differences during skill acquisition. Journal of Experimental Psychology: Applied, 6(4):259--290.

[2]

}}Bailey, B.P., Konstan, J.A. and Carlis, J.V. (2001) The Effects of Interruptions on Task Performance, Annoyance, and Anxiety in the User Interface. In Proc. Interact 2001, 593--601.

[3]

}}Beatty, J. and Lucero-Wagoner, B. (2000) The pupillary system. In Cacioppo, J.T., Tassinary, L.G. and. Berntson, G.G. (Eds.), Handbook of psychophysiology, Cambridge University Press, Hillsdale, NJ, USA, 142--162.

[4]

}}Caroll, J.B. (1993) Human Cognitive Abilities: A Survey of Factor-Analytic Studies. Cambridge University Press, Cambridge, UK.

[5]

}}Edwards, A.L. (1962) Experimental Design in Psychological Research. Holt, Rinehart and Winston, New York, NY, USA.

[6]

}}Ekstrom, R.B., French, J.W., Harman, H.H. and Dermen, D. (1976) Manual for KIT of factor-referenced cognitive tests. Educational Testing Service, Princeton, NJ, USA.

[7]

}}Ekstrom, R.B., French, J.W. and Harman, H.H. (1979) Cognitive factors: Their identification and replication. Multivariate Behavioral Research Monographs, 79(2): 3--84.

[8]

}}Eliot, J. and Smith, I.M. (Eds). (1983) An International directory of spatial tests. NFER Nelson, Windsor, England.

[9]

}}Field, G. (1987) Experimentus Interruptus. ACM SIGCHI Bulletin 19(2): 42--46.

Digital Library

[10]

}}Fredericks, T.K., Choi, S.D., Hart J., Butt, S.E. and Mital, A. (2005) An investigation of myocardial aerobic capacity as a measure of both physical and cognitive workloads. International. Journal of Industrial Ergonomics, 35(12): 1097--1107.

[11]

}}1French, J.W. (1951) The description of aptitude and achievement tests in terms of rotated factors. Psychometric Monograph, 5, University of Chicago Press, Chicago, IL, USA.

[12]

}}French, J.W., Ekstrom, R.B. and Price, L.A. (1963) Manual and kit of reference tests for cognitive factors. Educational Testing Service, Princeton, NJ, USA.

[13]

}}Golledge, R.G. (Ed.). (1999) Wayfinding behavior: Cognitive mapping and other spatial processes. Johns Hopkins University Press, Baltimore, MD, USA.

[14]

}}Grandhi, S. and Jones, Q. (2010) Technology-mediated interruption management. International Journal of Human-Computer Studies, 68(5): 288--306.

Digital Library

[15]

}}Gray, W.D., John, B.E. and Atwood, M.E. (1992) The Precis of Project Ernestine or an overview of a validation of GOMS. In Proc. CHI 1992, ACM Press, 307--312.

Digital Library

[16]

}}Hancock, P.A. and Chignell, M.H. (1986) Toward a Theory of Mental Work Load: Stress and Adaptability in Human-Machine Systems. In Proc. IEEE SMC 1986, 378--383.

[17]

}}Hart, S.G. and Staveland, L.E. (1988) Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. In Hancock, P.A. and Meshkati, N. (Eds.), Human Mental Workload. Amsterdam, North-Holland, 139--183.

[18]

}}Healey, J.A. and Picard, R.W. (2005) Detecting stress during real-world driving tasks using physiological sensors. IEEE Transactions of Intelligent Transportation Systems, 6(2): 156- 166.

Digital Library

[19]

}}Horvitz, E. and Apacible, J. (2003) Learning and Reasoning About Interruption. Proc. ICMI 2003, 20--27.

Digital Library

[20]

}}Hudson, S.E., Fogarty, J., Atkeson, C.G., Avrahami, D., Forlizzi, J., Kiesler, S., Lee, J.C. and Yang, J. (2003) Predicting Human Interruptibility with Sensors: A Wizard of Oz Feasibility Study. In Proc. CHI 2003, ACM Press, 257--264.

Digital Library

[21]

}}Ikehara, C.S. and Crosby, M.E. (2005) Assessing Cognitive Load with Physiological Sensors. Proc. HICSS 2005, 295--303.

Digital Library

[22]

}}Iqbal, S.T., Adamczyk, P.D., Zheng, X.S., and Bailey, B.P. (2005) Towards an index of opportunity: understanding changes in mental workload during task execution. In Proc. CHI 2005, ACM Press, 311--320.

Digital Library

[23]

}}Kilseop, R. and Rohae, M. (2005) Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic. International Journal of Industrial Ergonomics 35(11): 991--1009.

[24]

}}Kreifeldt, J.G. and McCarthey, M.E. (1981) Interruption as a Test of the User-Computer Interface. In Proc. Conference on Manual Control, California Institute of Technology, JPL Publication, 81--95, 655--667.

[25]

}}Lohman, D.F. (1988) Spatial abilities as traits, processes, and knowledge. In Sternberg, R.J. (Ed.), Advances in the psychology of human intelligence, 40. Erlbaum, Hillsdale, NJ, USA, 181--248.

[26]

}}McFarlane, D.C. (1999) Coordinating the Interruption of People in Human-Computer Interaction. In Proc. Interact 1999, 295--303.

[27]

}}McGrew, K.S. (2009) CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37, 1--10.

[28]

}}Mital, A. and Govindaraju, M. (1999) Is it possible to have a single measure for all work?. International Journal of Industrial Engineering Theory, 6, 190--195.

[29]

}}Mulder, L.J.M. (1992) Measurement and analysis methods of heart rate and respiration for use in applied environments. Biological Psychology 34, 205--236.

[30]

}}Nasoz, F., Alvarez, K., Lisetti, L. and Finkelstein, N. (2004) Emotion recognition from physiological signals using wireless sensors for presence technologies. Cognition, Technology and Work, 6(1): 4--14.

Digital Library

[31]

}}Nettelbeck, T. and Lally, M. (1976) Inspection Time and Measured Intelligence. British Journal of Psychology, 67, 17--22.

[32]

}}Paas, F. and Merrienboer, J.V. (1993) The efficiency of instructional conditions: an approach to combine mental effort and performance measures. Human Factors, 35, 737--743.

[33]

}}Paas, F. and van Merriënboer, J.J.G. (1994) Instructional control of cognitive load in the training of complex cognitive tasks. Educational Psychology Review, 6, 51--71.

[34]

}}Paas, F., Tuovinen, J.E., Tabbers, H.K. and Van Gerven, P.W.M. (2003) Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38(1): 63--71.

[35]

}}Petrill, S.A. and Deary, I.J. (2001) Inspection time and intelligence: Celebrating 25 years of research. Intelligence, 29(6): 441--442.

[36]

}}Ritter, F.E. and Avraamides, M.N. (2000) Steps towards including behavioural moderators in human performance models in synthetic environments. Tech Report No. 2000--1. Applied Cognitive Science Lab, Penn State University, PA, USA.

[37]

}}Ritter, F.E. and Young, R.M. (2001) Embodied models as simulated users: introduction to this special issue on using cognitive models to improve interface design. International Journal of Human-Computer Studies, 55(1): 1--14.

Digital Library

[38]

}}Salthouse, T.A. (1992) What do adult age differences in the Digit Symbol Substitution Test Reflect?. Journal of Gerontology, 47(3): 121--128.

[39]

}}Salthouse, T.A. (1996) The processing-speed theory of adult age differences in cognition. Psychological Review, 103(3): 403--428.

[40]

}}Salthouse, T.A. (2000) Aging and measures of processing speed. Biological Psychology, 54, 35--54.

[41]

}}Salvucci, D.D. (2001) Predicting the effects of in-car interface use on driver performance: An integrated model approach. International Journal of Human-Computer Studies, 55(1): 85--107.

Digital Library

[42]

}}Salvucci, D.D. (2006) Modeling Driver Behavior in a Cognitive Architecture. Human Factors, 48(2): 362--380.

[43]

}}Shi, Y., Ruiz, N., Taib, R., Choi, E. and Chen, F. (2007) Galvanic skin response (GSR) as an index of cognitive load. Ext. Abstracts CHI 2007, ACM Press, 2651--2656.

Digital Library

[44]

}}Simon, H.A. (1971) Designing organizations for an information rich world. In Greenberger, M. (Ed.), Computers, communications, and the public interest. Johns Hopkins University Press, Baltimore, MD, USA, 37--72.

[45]

}}Speier, C., Valacich, J.S. and Vessey, I. (1999) The Influence of Task Interruption on Individual Decision Making: An Information Overload Perspective. Decision Sciences, 30(2): 337--360.

[46]

}}Stramler, J.H. (1993) The Dictionary for Human Factors / Ergonomics. CRC Press, Inc., Boca Raton, FL, USA.

[47]

}}Street, R.F. (1931) A gestalt completion test: a study of a cross section of intellect. Contributions to Education, 481, Teacher's College, Columbia University, New York, NY, USA.

[48]

}}Treisman, A. and Gelade, G. (1980) A feature integration theory of attention. Cognitive Psychology, 12, 97--136.

[49]

}}Welford, A.T. (1986) Mental workload as a function of demand, capacity, strategy and skill. Ergonomics, 21, 151--176.

[50]

}}Wickens, C.D. (1991) Processing resources and attention. In Multiple Task Performance. D.L. Damos (Ed.), Taylor and Francis, Ltd., Bristol, UK, 3--34.

[51]

}}Wickens, C.D. and McCarley, J. (2008) Applied attention theory. Taylor and Francis, Boca Raton, FL, USA.

[52]

}}Wilson, G.F. (2002) An analysis of mental workload in pilots during flight using multiple psychophysiological measures. International Journal of Aviation Psychology, 12(1): 3--18.

[53]

}}Yin, B., Chen, F., Ruiz, N. and Ambikairajah, E. (2008) Speech-based cognitive load monitoring system. In Proc. IEEE ICASSP 2008, 2041--2044.

Cited By

Nandipati KPal SMitra R(2024)Electrodermal Activity (EDA) Morphologies and Prediction of Engagement with Simple Moving Average Crossover: A Mixed-Method StudySensors10.3390/s2414456524:14(4565)Online publication date: 14-Jul-2024
https://doi.org/10.3390/s24144565
Ahmadi MMichalka SNajafabadi MWünsche BBillinghurst M(2024)EEG, Pupil Dilations, and Other Physiological Measures of Working Memory Load in the Sternberg TaskMultimodal Technologies and Interaction10.3390/mti80400348:4(34)Online publication date: 19-Apr-2024
https://doi.org/10.3390/mti8040034
Foltyn ADeuschel JLang-Richter NHolzer NOppelt M(2024)Evaluating the robustness of multimodal task load estimation modelsFrontiers in Computer Science10.3389/fcomp.2024.13711816Online publication date: 10-Apr-2024
https://doi.org/10.3389/fcomp.2024.1371181
Show More Cited By

Index Terms

Psycho-physiological measures for assessing cognitive load
1. Human-centered computing

Recommendations

Using stroop task to assess cognitive load
ECCE '10: Proceedings of the 28th Annual European Conference on Cognitive Ergonomics

Motivation -- Assessment of cognitive load on user tasks is useful for characterizing user interfaces and tasks with respect to their demands on the user's mental effort.

Research approach -- We conducted a controlled experiment with 48 subjects. The ...
Detecting Users’ Cognitive Load by Galvanic Skin Response with Affective Interference

Experiencing high cognitive load during complex and demanding tasks results in performance reduction, stress, and errors. However, these could be prevented by a system capable of constantly monitoring users’ cognitive load fluctuations and adjusting its ...
Towards measuring cognitive load through multimodal physiological data
Abstract
Cognitive load plays an important role during learning and working, as it has been linked to well-functioning cognitive processes, performance, burnout and depression. Nonetheless, attempts to assess cognitive load in real-time by means of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '10: Proceedings of the 12th ACM international conference on Ubiquitous computing

September 2010

366 pages

ISBN:9781605588438

DOI:10.1145/1864349

General Chairs:
Jakob E. Bardram
IT University of Copenhagen, Denmark
,
Marc Langheinrich
University of Lugano, Switzerland
,
Program Chairs:
Khai N. Truong
University of Toronto, Canada
,
Paddy Nixon
University College Dublin, Ireland

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

University of Florida: University of Florida

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 September 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

Ubicomp '10

Sponsor:

Ubicomp '10: The 2010 ACM Conference on Ubiquitous Computing

September 26 - 29, 2010

Copenhagen, Denmark

Acceptance Rates

UbiComp '10 Paper Acceptance Rate 39 of 202 submissions, 19%;

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Upcoming Conference

UBICOMP '24

Sponsor:
sigchi
sigchi

The 2024 ACM International Joint Conference on Pervasive and Ubiquitous Computing

October 5 - 9, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

232
Total Citations
View Citations
3,989
Total Downloads

Downloads (Last 12 months)354
Downloads (Last 6 weeks)30

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nandipati KPal SMitra R(2024)Electrodermal Activity (EDA) Morphologies and Prediction of Engagement with Simple Moving Average Crossover: A Mixed-Method StudySensors10.3390/s2414456524:14(4565)Online publication date: 14-Jul-2024
https://doi.org/10.3390/s24144565
Ahmadi MMichalka SNajafabadi MWünsche BBillinghurst M(2024)EEG, Pupil Dilations, and Other Physiological Measures of Working Memory Load in the Sternberg TaskMultimodal Technologies and Interaction10.3390/mti80400348:4(34)Online publication date: 19-Apr-2024
https://doi.org/10.3390/mti8040034
Foltyn ADeuschel JLang-Richter NHolzer NOppelt M(2024)Evaluating the robustness of multimodal task load estimation modelsFrontiers in Computer Science10.3389/fcomp.2024.13711816Online publication date: 10-Apr-2024
https://doi.org/10.3389/fcomp.2024.1371181
Stolp FStellmacher MArnrich Bd'Amorim M(2024)CognitIDE: An IDE Plugin for Mapping Physiological Measurements to Source CodeCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663805(592-596)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663805
Hashky ARheault BAhsan ABenda BAudino TLonneman SRagan E(2024)Multi-Modal User Modeling for Task Guidance: A Dataset for Real-Time Assistance with Stress and Interruption Dynamics2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00105(544-550)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00105
Cauquis JPeillard EDominjon LDuval TMoreau G(2024)Investigating Whether the Mass of a Tool Replica Influences Virtual Training Learning OutcomesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337204130:5(2411-2421)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372041
Wang RJo WZhao DWang WGupte AYang BChen GMin B(2024) Husformer : A Multimodal Transformer for Multimodal Human State Recognition IEEE Transactions on Cognitive and Developmental Systems10.1109/TCDS.2024.335761816:4(1374-1390)Online publication date: Aug-2024
https://doi.org/10.1109/TCDS.2024.3357618
Amadori PDemiris Y(2024)User-Aware Multilevel Cognitive Workload Estimation From Multimodal Physiological SignalsIEEE Transactions on Cognitive and Developmental Systems10.1109/TCDS.2023.334213916:4(1212-1222)Online publication date: Aug-2024
https://doi.org/10.1109/TCDS.2023.3342139
Daza RGomez LFierrez JMorales ATolosana ROrtega-Garcia J(2024)DeepFace-Attention: Multimodal Face Biometrics for Attention Estimation With Application to e-LearningIEEE Access10.1109/ACCESS.2024.343729112(111343-111359)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3437291
Eesee AJaskó SEigner GAbonyi JRuppert T(2024)Extension of HAAS for the Management of Cognitive LoadIEEE Access10.1109/ACCESS.2024.335990212(16559-16572)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3359902
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents