research-article

Framework for Electroencephalography-based Evaluation of User Experience

Authors:

Fabien LotteAuthors Info & Claims

CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

Pages 2283 - 2294

https://doi.org/10.1145/2858036.2858525

Published: 07 May 2016 Publication History

Abstract

Measuring brain activity with electroencephalography (EEG) is mature enough to assess mental states. Combined with existing methods, such tool can be used to strengthen the understanding of user experience. We contribute a set of methods to estimate continuously the user's mental workload, attention and recognition of interaction errors during different interaction tasks. We validate these measures on a controlled virtual environment and show how they can be used to compare different interaction techniques or devices, by comparing here a keyboard and a touch-based interface. Thanks to such a framework, EEG becomes a promising method to improve the overall usability of complex computer systems.

Supplementary Material

MP4 File (p2283-frey.mp4)

Download
265.82 MB

References

[1]

Chris Berka and DJ Levendowski. 2007. EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat Space Environ Med. 78, 5 Suppl (May 2007), B231--44.

[2]

Benjamin Blankertz, Michael Tangermann, Carmen Vidaurre, Siamac Fazli, Claudia Sannelli, Stefan Haufe, Cecilia Maeder, Lenny Ramsey, Irene Sturm, Gabriel Curio, and Klaus-Robert Müller. 2010. The Berlin Brain-Computer Interface: Non-Medical Uses of BCI Technology. Front Neurosci 4 (Jan. 2010), 198.

[3]

Christopher G. Burns and Stephen H. Fairclough. 2015. Use of auditory event-related potentials to measure immersion during a computer game. International Journal of Human-Computer Studies 73 (2015), 107--114.

[4]

Ula Cartwright-Finch and Nilli Lavie. 2007. The role of perceptual load in inattentional blindness. Cognition 102, 3 (2007), 321--340.

[5]

J T Coull. 1998. Neural correlates of attention and arousal: insights from electrophysiology, functional neuroimaging and psychopharmacology. Progress in neurobiology 55, 4 (July 1998), 343--61.

[6]

Edward Cutrell and Desney Tan. 2008. BCI for passive input in HCI. CHI '08 Workshops (2008), 1--3.

[7]

Sebastian Deterding, Dan Dixon, Rilla Khaled, and Lennart Nacke. 2011. From game design elements to gamefulness. In MindTrek '11. 9.

Digital Library

[8]

Stephen H. Fairclough. 2009. Fundamentals of physiological computing. Interacting with Comp. 21, 1--2 (Jan. 2009), 133--145.

Digital Library

[9]

Pierre W Ferrez and Jose Del R Millan. 2008. Error-related EEG potentials generated during simulated brain-computer interaction. IEEE Trans. Biomed. Eng. 55, 3 (March 2008), 923--9.

[10]

David R. Flatla, Carl Gutwin, Lennart E. Nacke, Scott Bateman, and Regan L. Mandryk. 2011. Calibration games: making calibration tasks enjoyable by adding motivating game elements. UIST '11 (2011), 403--412.

Digital Library

[11]

Jérémy Frey, Christian Mühl, Fabien Lotte, and Martin Hachet. 2014. Review of the use of electroencephalography as an evaluation method for human-computer interaction. In PhyCS '14.

[12]

David Grimes, DS Tan, and SE Hudson. 2008. Feasibility and pragmatics of classifying working memory load with an electroencephalograph. CHI '08 (2008), 835.

Digital Library

[13]

F.E. Grubbs. 1969. Procedures for Detecting Outlying Observations in Samples. Technometrics 11, 1 (1969), 1--21.

[14]

SG Hart and LE Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Human mental workload.

[15]

Leanne M. Hirshfield, Erin Treacy Solovey, Audrey Girouard, James Kebinger, Robert J.K. Jacob, Angelo Sassaroli, and Sergio Fantini. 2009. Brain measurement for usability testing and adaptive interfaces. In CHI '09. ACM Press, NY, NY, USA, 2185.

Digital Library

[16]

Ulrich Hoffmann, JM Vesin, and Touradj Ebrahimi. 2006. Spatial filters for the classification of event-related potentials. In ESANN '06.

[17]

J. Matias Kivikangas, Inger Ekman, Guillaume Chanel, Simo Järvelä, Ben Cowley, Pentti Henttonen, and Niklas Ravaja. 2010. Review on psychophysiological methods in game research. Proc. of 1st Nordic DiGRA (2010).

[18]

Olivier Ledoit and Michael Wolf. 2004. A well-conditioned estimator for large-dimensional covariance matrices. J Multivar Anal (Feb. 2004).

Digital Library

[19]

A. John Maule and Anne C Edland. 1997. The effects of time pressure on human judgment and decision making. In Decision making: Cognitive models and Explanations.

[20]

Ranjana K Mehta and Raja Parasuraman. 2013. Neuroergonomics: a review of applications to physical and cognitive work. Frontiers in human neuroscience 7, December (Jan. 2013), 889.

[21]

Ernst Niedermeyer and FH Lopes da Silva. 2005. Electroencephalography: basic principles, clinical applications, and related fields. LWW.

[22]

S Nieuwenhuis, K R Ridderinkhof, J Blom, G P Band, and A Kok. 2001. Error-related brain potentials are differentially related to awareness of response errors: evidence from an antisaccade task. Psychophysiology 38, 5 (Sept. 2001), 752--60.

[23]

Richard E. Nisbett and Timothy DeCamp Wilson. 1977. Telling more than we can know: Verbal reports on mental processes. Psychological Review 84, 3 (1977), 231--260.

[24]

Adrian M. Owen, Kathryn M. McMillan, Angela R. Laird, and Ed Bullmore. 2005. N-back working memory paradigm: A meta-analysis of normative functional neuroimaging studies. Human Brain Mapping 25, 1 (2005), 46--59.

[25]

Evan M M. Peck, Beste F. Yuksel, Alvitta Ottley, Robert J.K. Jacob, and Remco Chang. 2013. Using fNIRS brain sensing to evaluate information visualization interfaces. CHI '13 (2013), 473.

Digital Library

[26]

Rosalind W. Picard. 1995. Affective computing. Technical Report 321. MIT Media Laboratory.

[27]

Matthew Pike, ML Wilson, Anna Divoli, and Alyona Medelyan. 2012. CUES: Cognitive Usability Evaluation System. EuroHCIR '12 (2012), 1--4.

[28]

G. Michael Poor, Guy Zimmerman, Dale S. Klopfer, Samuel D. Jaffee, Laura Marie Leventhal, and Julie Barnes. 2013. Mobility Matters: Identifying Cognitive Demands That Are Sensitive to Orientation. In Interact '13, Vol. 8117 LNCS. 193--210.

[29]

Felix Putze, Christoph Amma, and Tanja Schultz. 2015. Design and Evaluation of a Self-Correcting Gesture Interface based on Error Potentials from EEG. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 3375--3384.

Digital Library

[30]

Herbert Ramoser, Johannes Müller-Gerking, and Gert Pfurtscheller. 2000. Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans Rehabil Eng 8, 4 (2000), 441--446.

[31]

S Sternberg. 1966. High-speed scanning in human memory. Science (NY, N.Y.) 153 (1966), 652--654.

[32]

Romain Trachel, Thomas Brochier, and Maureen Clerc. 2013. Enhancing visuospatial attention performance with brain-computer interfaces. CHI '13 (2013), 1245.

Digital Library

[33]

Chi Vi and Sriram Subramanian. 2012. Detecting error-related negativity for interaction design. CHI '12 (2012), 493.

Digital Library

[34]

J.A. Wilson, C. Guger, and G. Schalk. 2012. BCI hardware and software. In Brain-computer interfaces: principles and practice. 165--188.

[35]

Dennis Wobrock, Jérémy Frey, Delphine Graef, Jean-Baptiste de la Rivi'ere, Julien Castet, and Fabien Lotte. 2015. Continuous Mental Effort Evaluation during 3D Object Manipulation Tasks based on Brain and Physiological Signals. In INTERACT '15.

Cited By

Grootjen JThalhammer PKosch T(2024)Your Eyes on Speed: Using Pupil Dilation to Adaptively Select Speed-Reading Parameters in Virtual RealityProceedings of the ACM on Human-Computer Interaction10.1145/36765318:MHCI(1-17)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676531
Chiossi FGruenefeld UHou BNewn JOu CLiao RWelsch RMayer S(2024)Understanding the Impact of the Reality-Virtuality Continuum on Visual Search Using Fixation-Related Potentials and Eye Tracking FeaturesProceedings of the ACM on Human-Computer Interaction10.1145/36765288:MHCI(1-33)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676528
Zhang KCao ZZheng XBillinghurst M(2024)Identifying Hand-based Input Preference Based on Wearable EEGProceedings of the Augmented Humans International Conference 202410.1145/3652920.3653028(102-118)Online publication date: 4-Apr-2024
https://dl.acm.org/doi/10.1145/3652920.3653028
Show More Cited By

Index Terms

Framework for Electroencephalography-based Evaluation of User Experience
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

User experience evaluation methods: current state and development needs
NordiCHI '10: Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries

The recent shift of emphasis to user experience (UX) has rendered it a central focus of product design and evaluation. A multitude of methods for UX design and evaluation exist, but a clear overview of the current state of the available UX evaluation ...
Understanding, scoping and defining user experience: a survey approach
CHI '09: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Despite the growing interest in user experience (UX), it has been hard to gain a common agreement on the nature and scope of UX. In this paper, we report a survey that gathered the views on UX of 275 researchers and practitioners from academia and ...
Usability of EEG Systems: User Experience Study
PETRA '16: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments

In recent years there was a change in EEG experimental designs - from simple behavior in the lab to complex behavior outside. That change required also an adjustment of EEG systems -- from being static and sensitive to mobile and noise-resistant. The ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

May 2016

6108 pages

ISBN:9781450333627

DOI:10.1145/2858036

General Chairs:
Jofish Kaye
Yahoo
,
Allison Druin
University of Maryland / National Park Service
,
Program Chairs:
Cliff Lampe
University of Michigan
,
Dan Morris
Microsoft
,
Juan Pablo Hourcade
University of Iowa

Copyright © 2016 ACM.

Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI'16

Sponsor:

SIGCHI

CHI'16: CHI Conference on Human Factors in Computing Systems

May 7 - 12, 2016

California, San Jose, USA

Acceptance Rates

CHI '16 Paper Acceptance Rate 565 of 2,435 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

56
Total Citations
View Citations
765
Total Downloads

Downloads (Last 12 months)49
Downloads (Last 6 weeks)6

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Grootjen JThalhammer PKosch T(2024)Your Eyes on Speed: Using Pupil Dilation to Adaptively Select Speed-Reading Parameters in Virtual RealityProceedings of the ACM on Human-Computer Interaction10.1145/36765318:MHCI(1-17)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676531
Chiossi FGruenefeld UHou BNewn JOu CLiao RWelsch RMayer S(2024)Understanding the Impact of the Reality-Virtuality Continuum on Visual Search Using Fixation-Related Potentials and Eye Tracking FeaturesProceedings of the ACM on Human-Computer Interaction10.1145/36765288:MHCI(1-33)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676528
Zhang KCao ZZheng XBillinghurst M(2024)Identifying Hand-based Input Preference Based on Wearable EEGProceedings of the Augmented Humans International Conference 202410.1145/3652920.3653028(102-118)Online publication date: 4-Apr-2024
https://dl.acm.org/doi/10.1145/3652920.3653028
Katsini CEpiphaniou GMaple C(2024)Towards an Evaluation Framework for Extended Reality Authentication SchemesExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651021(1-6)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3651021
Aloi DBouzit SLi J(2024)Biometric Methods for User Research: Three Case StudiesExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3637132(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3637132
Liu YVaitheeshwari RYeh SWu E(2024)Neural Reactivity to Haptics: Virtual Tasks Versus Physical TasksIEEE Sensors Journal10.1109/JSEN.2024.336417024:7(11817-11828)Online publication date: 1-Apr-2024
https://doi.org/10.1109/JSEN.2024.3364170
Koukopoulos ZKoukopoulos DFidas C(2024)eXtended Reality Services for Outdoor Cultural Spaces: System Design and Evaluation Framework Using Unbiased Biometric Data2024 15th International Conference on Information, Intelligence, Systems & Applications (IISA)10.1109/IISA62523.2024.10786698(1-8)Online publication date: 17-Jul-2024
https://doi.org/10.1109/IISA62523.2024.10786698
Lin YYu SZhuang ALee CTing YLee SLin BKuo P(2024)Representing scentsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103357192:COnline publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1016/j.ijhcs.2024.103357
Lotte FJeunet-Kelway C(2024)Brain-Computer Interaction and NeuroergonomicsMind, Body, and Digital Brains10.1007/978-3-031-58363-6_10(141-156)Online publication date: 23-Jun-2024
https://doi.org/10.1007/978-3-031-58363-6_10
Joshi JWang KCho Y(2023)PhysioKit: An Open-Source, Low-Cost Physiological Computing Toolkit for Single- and Multi-User StudiesSensors10.3390/s2319824423:19(8244)Online publication date: 4-Oct-2023
https://doi.org/10.3390/s23198244
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents