research-article

This is your brain on interfaces: enhancing usability testing with functional near-infrared spectroscopy

Authors:

Leanne M. Hirshfield,

Rebecca Gulotta,

Stuart Hirshfield,

Matthew Russell,

Robert JacobAuthors Info & Claims

CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 373 - 382

https://doi.org/10.1145/1978942.1978996

Published: 07 May 2011 Publication History

Abstract

This project represents a first step towards bridging the gap between HCI and cognition research. Using functional near-infrared spectroscopy (fNIRS), we introduce tech-niques to non-invasively measure a range of cognitive workload states that have implications to HCI research, most directly usability testing. We present a set of usability experiments that illustrates how fNIRS brain measurement provides information about the cognitive demands placed on computer users by different interface designs.

References

[1]

Adcock, R.A., Constable, R.T., Gore, J.C. and Goldman-Rakic, P.S. Functional neuroanatomy of executive processes involved in dual-task performance. Proceedings of the National Academy of Sciences of the United States of America, 97 (7). 3567--3572.

[2]

Anderson, E.J., Mannan, S.K., Husain, M., Rees, G., Summer, P., Mort, D.J., McRobbie, D. and Kennard, C. Involvement of prefrontal cortex in visual search. Experimental Brain Research, 180 (2). 289--302.

[3]

Baddeley, A. and Della Sala, S. Working memory and execu-tive control. Philosophical Transactions of the Royal Society of London, 351.

[4]

Berka, C., Levendowski, D., Cvetinovic, M., Petrovic, M., Davis, G., Lumicao, L., Zivkovic, V., Popovic, M. and Olmstead, R. Real-Time Analysis of EEG Indexes of Alert-ness, Cognition, and Memory Acquired With a Wireless EEG Headset. International Journal of Human Computer In-teraction, 17 (2). 151--170.

[5]

Blackmon, M.H., Kitajima, M. and Polson, P.G., Tool for accurately predicting website navigation problems, non-problems, problem severity, and effectiveness of repairs. in In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, (Portland, Oregon, USA, 2005).

Digital Library

[6]

Brumby, D.P., Salvucci, D.D. and Howes, A., Focus on driv-ing: how cognitive constraints shape the adaptation of strategy when dialing while driving. in In Proc. of the 27th international Conference on Human Factors in Computing Sys-tems, (Boston, MA, USA, 2009), ACM.

Digital Library

[7]

Buxton, R. Introduction to functional magnetic resonance imaging. Cambridge University Press, Cambridge, United Kingdom, 2002.

[8]

Chance, B., Anday, E., Nioka, S., Zhou, S., Hong, L., Worden, K., Li, C., Murray, T., Ovetsky, Y. and Thomas, R. A novel method for fast imaging of brain function, non-invasively, with light. Optics Express, 10 (2).

[9]

Czerwinski, M. and Larson, K. Cognition and the Web: Moving from Theory to Web Design. in Human Factors and Web Development, Ratner, J. (Ed.), Erlbaum: NJ, 2002, 147--165.

[10]

Eckstrom, R., French, J., Harman, H. and Derman, D. Kit of factor-referenced cognitive tests.

[11]

Gevins, A. and Smith, M. Neurophysiological Measures of Working memory and Individual Differences in Cognitive Ability and Cogntive Style. Cerebral Cortex, 10.

[12]

Grimes, D., Tan, D., Hudson, S., Shenoy, P. and Rao, R., Feasibility and Pragmatics of Classifying Working Memory Load with an Electroencephalograph. in CHI 2008 Conference on Human Factors in Computing Systems, (Florence, Italy., 2008).

Digital Library

[13]

Guhe, M., Liao, W., Zhu, Z., Ji, Q., Gray, W.D. and Schoelles, M.J., Non-intrusive measurement of workload in real-time. in 49th Annual Conference of the Human Factors and Ergonomics Society, (2005), 1157--1161.

[14]

Hart, S.G. and Staveland, L.E. Development of NASA-TLX (Task Load Index): Results of empirical and theorical research. in Hancock, P., Meshkati, N. ed. Human Mental Workload, Amsterdam, 1988, pp 139--183.

[15]

Hirshfield, L.M. Enhancing Usabiltiy Testing with Functional Near Infrared Spectroscopy Computer Science, Tufts University, Medford, MA, 2009.

[16]

Hirshfield, L.M., Solovey, E.T., Girouard, A., Kebinger, J., Jacob, R.J.K., Sassaroli, A. and Fantini, S., Brain Measure-ment for Usability Testing and Adaptive Interfaces: An Example of Uncovering Syntactic Workload in the Brain Using Functional Near Infrared Spectroscopy. in Proc. of SIGCHI, (2009).

Digital Library

[17]

Hoshi, Y. and Tamuraa, M. Near-Infrared Optical Detection of Sequential Brain Activation in the Prefrontal Cortex during Mental Tasks. NeuroImage, 5. 292--297.

[18]

Hoshi, Y., Tsoub, B., Billockc, V., Tanosakia, M., Iguchia, Y., Shimadaa, M., Shinbaa, T., Yamadad, Y. and Odae, I. Spatiotemporal characteristics of hemodynamic changes in the human lateral prefrontal cortex during working memory tasks. NeuroImage, 20. 1493--1504.

[19]

Izzetoglu, K., Bunce, S., Onaral, B., Pourrezaei, K. and Chance, B. Functional Optical Brain Imaging Using Near-Infrared During Cognitive Tasks. International Journal of Human-Computer Interaction, 17 (2).

[20]

Jaeggi, S.M., Seewer, R., Nirkko, A.C., Eckstein, D., Schroth, G., Groner, R. and Gutbrod, K. Does excessive memory load attenuate activation in the prefrontal cortex? Load-dependent processing in single and dual tasks: functional magnetic resonance imaging study. NeuroImage, 19 (2). 210--225.

[21]

Jancke, L., Brunner, B. and Esslen, M. Brain activation during fast driving in a driving simulator: the role of the lateral prefrontal cortex. Neuroreport, 19 (11).

[22]

Jasdzewski, G., Strangman, G., Wagner, J., Kwong, K., Poldrack, R. and Boas, D. Differences in the hemodynamic response to event-related motor and visual paradigms as measured by near-infrared spectroscopy. NEUROIMAGE, 20 (1). 479--488.

[23]

Joanette, Y., Ansaldo, A., de Mattos Pimenta Parente, M., Fonseca, R., Kristensen, C. and Scherer, L. Neuroimaging investigation of executive functions: evidence from fNIRS. PSICO, 39 (3).

[24]

Lee, J.C. and Tan, D.S., Using a Low-Cost Electroencephalograph for Task Classification in HCI Research. in ACM Symposium on User Interface Software and Technology, (2006).

Digital Library

[25]

Meek, J., Elwell, C., Khan, M., Romaya, J., Wyatt, J., Delpy, D. and Zeki, S. Regional Changes in Cerebral Haemodynamics as a Result of a Visual Stimulus Measured by Near Infrared Spectroscopy. Proc. Roy. Soc. London, 261. 351--356.

[26]

Muller-Plath, G. Localizing subprocesses of visual search by correlating local brain activation in fMRI with response time model parameters. Journal of Neuroscience Methods, 171 (2). 316--330.

[27]

Parasuraman, R. and Caggiano, D. Neural and Genetic Assays of Human Mental Workload. in Quantifying Human Information Processing, Lexington Books, 2005.

[28]

Scerbo, M., Frederick, G., Freeman, F. and Mikulka, P. A brain-based system for adaptive automation. Theoretical Issues in Ergonomics Science, 4 (1).

[29]

Schroeter, M.L., Zysset, S., Kupka, T., Kruggel, F. and Yves von Cramon, D. Near-Infrared Spectroscopy Can Detect Brain Activity During a Color-Word Matching Stroop Task in an Event-Related Design. Human Brain Mapping, 17 (1). 61--71.

[30]

Smith, E. and Jonides, J. Storage and Executive Processes n the Frontal Lobes. Science, 283.

[31]

Solovey, E., Girouard, A., Chauncey, K., Hirshfield, L., Sassaroli, A., Zheng, F., Fantini, S. and Jacob, R., Using fNIRS Brain Sensing in Realistic HCI Settings: Experiments and Guidelines. in ACM UIST Symposium on User Interface Software and Technology, (2009), ACM.

Digital Library

[32]

Tamborello, F. and Byrne, M.D., Information search: The in-tersection of visual and semantic space. in CHI 2005 Extended Abstracts, (Portland, OR, USA, 2005), ACM, 1821--1824.

Digital Library

[33]

Wickens, C., Lee, J., Liu, Y. and Becker, S. An Introduction to Human Factors Engineering. Pearson, 2004.

Digital Library

[34]

Wilson, G.F. and Fisher, F. Cognitive task classification based upon topographic EEG data. Biological Psychology, 40. 239--250.

[35]

Witten, I.H. and Frank, E. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann, San Francisco, 2005.

Digital Library

[36]

Ye, Z. and Zhou, X. Conflict control during sentence comprehension: fMRI evidence. Neuroimage, 48 (1). 280--290.

Cited By

Landowska AWilson MCraven MHarrington K(2024)Adaptative computerized cognitive training decreases mental workload during working memory precision task - A preliminary fNIRS studyInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103206184:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103206
Nissen ARiedl RSchütte R(2024)Users’ reactions to website designsComputers in Human Behavior10.1016/j.chb.2024.108168155:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.chb.2024.108168
Kosch TKarolus JZagermann JReiterer HSchmidt AWoźniak P(2023)A Survey on Measuring Cognitive Workload in Human-Computer InteractionACM Computing Surveys10.1145/358227255:13s(1-39)Online publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1145/3582272
Show More Cited By

Index Terms

This is your brain on interfaces: enhancing usability testing with functional near-infrared spectroscopy
1. Computing methodologies
  1. Artificial intelligence

Index terms have been assigned to the content through auto-classification.

Recommendations

Using fNIRS brain sensing to evaluate information visualization interfaces
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

We show how brain sensing can lend insight to the evaluation of visual interfaces and establish a role for fNIRS in visualization. Research suggests that the evaluation of visual design benefits by going beyond performance measures or questionnaires to ...
Brain measurement for usability testing and adaptive interfaces: an example of uncovering syntactic workload with functional near infrared spectroscopy
CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

A well designed user interface (UI) should be transparent, allowing users to focus their mental workload on the task at hand. We hypothesize that the overall mental workload required to perform a task using a computer system is composed of a portion ...
Brain Activity Changes Elicited Through Multi-session Training Assessment in the Prefrontal Cortex by fNIRS
Augmented Cognition
Abstract
This study aims to utilize a non-invasive and portable neuroimaging modality – functional near infrared spectroscopy (fNIRS) to investigate training and transfer of skills in unmanned aerial system (UAS) sensor operators (SOs). To achieve this ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

May 2011

3530 pages

ISBN:9781450302289

DOI:10.1145/1978942

General Chair:
Desney Tan
Microsoft Research
,
Program Chairs:
Geraldine Fitzpatrick
Vienna University of Technology
,
Carl Gutwin
University of Saskatchewan
,
Bo Begole
PARC
,
Wendy A. Kellogg
IBM Research

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '11

Sponsor:

SIGCHI

CHI '11: CHI Conference on Human Factors in Computing Systems

May 7 - 12, 2011

BC, Vancouver, Canada

Acceptance Rates

CHI '11 Paper Acceptance Rate 410 of 1,532 submissions, 27%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
1,009
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Landowska AWilson MCraven MHarrington K(2024)Adaptative computerized cognitive training decreases mental workload during working memory precision task - A preliminary fNIRS studyInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103206184:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103206
Nissen ARiedl RSchütte R(2024)Users’ reactions to website designsComputers in Human Behavior10.1016/j.chb.2024.108168155:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.chb.2024.108168
Kosch TKarolus JZagermann JReiterer HSchmidt AWoźniak P(2023)A Survey on Measuring Cognitive Workload in Human-Computer InteractionACM Computing Surveys10.1145/358227255:13s(1-39)Online publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1145/3582272
Hirshfield LWickens CDoherty ESpencer CWilliams THayne L(2023)Toward Workload-Based Adaptive Automation: The Utility of fNIRS for Measuring Load in Multiple Resources in the BrainInternational Journal of Human–Computer Interaction10.1080/10447318.2023.226624240:22(7404-7430)Online publication date: 23-Oct-2023
https://doi.org/10.1080/10447318.2023.2266242
Bagheri MPower S(2022)Simultaneous Classification of Both Mental Workload and Stress Level Suitable for an Online Passive Brain–Computer InterfaceSensors10.3390/s2202053522:2(535)Online publication date: 11-Jan-2022
https://doi.org/10.3390/s22020535
Putze FPutze SSagehorn MMicek CSolovey E(2022)Understanding HCI Practices and Challenges of Experiment Reporting with Brain Signals: Towards Reproducibility and ReuseACM Transactions on Computer-Human Interaction10.1145/349055429:4(1-43)Online publication date: 31-Mar-2022
https://dl.acm.org/doi/10.1145/3490554
Sommer NKakillioglu BGrant TVelipasalar SHirshfield L(2021)Classification of fNIRS Finger Tapping Data With Multi-Labeling and Deep LearningIEEE Sensors Journal10.1109/JSEN.2021.311540521:21(24558-24569)Online publication date: 1-Nov-2021
https://doi.org/10.1109/JSEN.2021.3115405
Bergen-Cico DGrant THirshfield LRazza RCosta MKilaru P(2021)Using fNIRS to Examine Neural Mechanisms of Change Associated with Mindfulness-Based Interventions for Stress and Trauma: Results of a Pilot Study for WomenMindfulness10.1007/s12671-021-01705-612:9(2295-2310)Online publication date: 30-Jul-2021
https://doi.org/10.1007/s12671-021-01705-6
Wong HEpps JChen S(2020)A Comparison of Methods for Mitigating Within-Task Luminance Change for Eyewear-Based Cognitive Load MeasurementIEEE Transactions on Cognitive and Developmental Systems10.1109/TCDS.2018.287634812:4(681-694)Online publication date: Dec-2020
https://doi.org/10.1109/TCDS.2018.2876348
Grant TDhruv KEloy LHayne LDurkee KHirshfield L(2020)A Neurophysiological Sensor Suite for Real-Time Prediction of Pilot Workload in Operational SettingsHCI International 2020 – Late Breaking Papers: Cognition, Learning and Games10.1007/978-3-030-60128-7_5(60-77)Online publication date: 4-Oct-2020
https://doi.org/10.1007/978-3-030-60128-7_5
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents