SE-CNN Attention Structure for Quantitative EEG-Based Assessment of VR Motion Sickness
Authors: 
Mengdie Xu, Banghua Yang, Mutian Liu, Yiling Huang, Xinxing Xia, Jie ZhangAuthors Info & Claims
Mengdie Xu, Banghua Yang, Mutian Liu, Yiling Huang, Xinxing Xia, Jie ZhangAuthors Info & ClaimsPages 606 - 611
Abstract
The applications of virtual reality (VR) technology are currently numerous and promising, but motion sickness (MS) problems are affecting the development of the VR market. Questionnaires are commonly used to subjectively assess motion sickness, but they are applied before and after the user experiences VR and cannot assess the user's motion sickness in real time. In this work, this paper proposes a convolutional neural network (CNN) structure incorporating squeeze and stimulus (SE) attention mechanisms, with subjective questionnaire scores used as markers of electroencephalographic signal (EEG) for real-time prediction of VR motion sickness scores. In this thesis, EEG signals were collected from 16 subjects in a virtual reality environment and the experimental data were fed into the network structure to train the model. The experimental results showed that the root mean square error (RMSE) was 25.69, the Pearson linear correlation coefficient (PLCC) was greater than 0.85, and the Spearman rank correlation coefficient (SROCC) was also greater than 0.85, indicating a significant relationship between the prediction and subjective assessment scores (p<0.05), proving proposed method's effectiveness.
References
[1]
T. van Gemert and J. Bergström, "Evaluating VR Sickness in VR Locomotion Techniques," 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Lisbon, Portugal, 2021, pp. 380-382.
[2]
C. -T. Lin, S. -W. Chuang, Y. -C. Chen, L. -W. Ko, S. -F. Liang and T. -P. Jung, "EEG Effects of Motion Sickness Induced in a Dynamic Virtual Reality Environment," 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Lyon, France, 2007, pp. 3872-3875.
[3]
Krokos, E., Varshney, A. Quantifying VR cybersickness using EEG. Virtual Reality 26, 77–89 (2022).
[4]
Zhao, L., "EEG characteristics of motion sickness subjects in automatic driving mode based on virtual reality tests." Qinghua Daxue Xuebao/Journal of Tsinghua University (2019).
[5]
Valmaggia L R, Latif L, Kempton M J, Virtual reality in the psychological treatment for mental health problems: An systematic review of recent evidence. Psychiatry research,2016,236:189-195.
[6]
Li, X.,et al. "VR motion sickness recognition by using EEG rhythm energy ratio based on wavelet packet transform." Computer Methods and Programs in Biomedicine 188(2019):105266.
[7]
Liao, Chung Yen, "Using EEG and Deep Learning to Predict Motion Sickness Under Wearing a Virtual Reality Device." IEEE Access PP.99(2020):1-1.
[8]
Sozykin, A.V. . "An overview of methods for deep learning in neural networks." Ural Federal University (Mira str. 19 (2017):28–59.
[9]
Frank, L, "Simulator Sickness: A Reaction to a Transformed Perceptual World. 1. Scope of the Problem." (1983).
[10]
Kennedy, R. . "A simulator sickness questionnaire (SSQ) : A new method for quantifying simulator sickness." International J Aviation Psychology 3(1993).
[11]
Kim H K, Park J, Choi Y, Virtual reality sickness questionnaire (VRSQ): Motion sickness measurement index in a virtual reality environment. Applied ergonomics, 2018, 69:66-73.
[12]
Kim H, Kim D J, Chung W H, Clinical predictors of cybersickness in virtual reality (VR) among highly stressed people. Scientific reports,2021,11(1): 1-11.
[13]
Chang E, Kim H T, Yoo B. Predicting cybersickness based on user's gaze behaviors in HMD-based virtual reality.Journal of Computational Design and Engineering, 2021,8(2):728-739.
[14]
Envelope, Rsp, and D. Li . "A review study on eye-tracking technology usage in immersive virtual reality learning environments." Computers & Education (2022).
[15]
Yi L, Jia R, Liu R, Evaluation Indicators for Visually Induced Motion Sickness in Virtual Reality Environment. Space Medicine & Medical Engineering, 2018.
[16]
R. Islam, "Automatic Detection and Prediction of Cybersickness Severity using Deep Neural Networks from user's Physiological Signals," 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Porto de Galinhas, Brazil, 2020, pp. 400-411.
[17]
Garcia-Agundez A, Reuter C, Becker H, Development of a classifier to determine factors causing cybersickness in virtual reality environments. Games for health journal, 2019, 8(6): 439-444.
[18]
Lee T M, Yoon J C, Lee I K.Motion sickness prediction in stereoscopic videos using 3d convolutional neural networks. IEEE transactions on visualization and computer graphics, 2019, 25(5): 1919-1927.
[19]
Gruden T, Popović N B, Stojmenova K, Electrogastrography in Autonomous Vehicles—An Objective Method for Assessment of Motion Sickness in Simulated Driving Environments. Sensors, 2021, 21(2): 550.
[20]
Kim J,Oh H, Kim W, A Deep Motion Sickness Predictor Induced by Visual Stimuli in Virtual Reality. IEEE Transactions on Neural Networks and Learning Systems, 33(2):554-566.
[21]
S. Lee, S. Kim, H. G. Kim and Y. M. Ro, "Assessing Individual VR Sickness Through Deep Feature Fusion of VR Video and Physiological Response," in IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 5, pp. 2895-2907, May 2022.
[22]
Gramfort, A.,et al. "The MNE package for M/EEG data processing." (2013).
[23]
J. Hu, L. Shen, S. Albanie, G. Sun and E. Wu, "Squeeze-and-Excitation Networks," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 8, pp. 2011-2023, 1 Aug. 2020.
Index Terms
- SE-CNN Attention Structure for Quantitative EEG-Based Assessment of VR Motion Sickness
Recommendations
VRSA Net: VR Sickness Assessment Considering Exceptional Motion for 360° VR Video
The viewing safety is one of the main issues in viewing virtual reality (VR) content. In particular, VR sickness could occur when watching immersive VR content. To deal with the viewing safety for VR content, objective assessment of VR sickness is of ...
Measurement of exceptional motion in VR video contents for VR sickness assessment using deep convolutional autoencoder
VRST '17: Proceedings of the 23rd ACM Symposium on Virtual Reality Software and TechnologyThis paper proposes a new objective metric of exceptional motion in VR video contents for VR sickness assessment. In VR environment, VR sickness can be caused by several factors which are mismatched motion, field of view, motion parallax, viewing angle, ...
Effect of VR technology matureness on VR sickness
AbstractIn this paper relationship of perceived virtual reality (VR) sickness phenomenon with different generations of virtual reality head mounted displays (VR HMD) is presented. Action content type omnidirectional video clip was watched by means of four ...
Comments
Information & Contributors
Information
Published In
March 2023
824 pages
ISBN:9781450399029
DOI:10.1145/3594315
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 02 August 2023
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICCAI 2023
ICCAI 2023: 2023 9th International Conference on Computing and Artificial Intelligence
March 17 - 20, 2023
Tianjin, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 49Total Downloads
- Downloads (Last 12 months)32
- Downloads (Last 6 weeks)1
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format