short-paper

UASOS: An Experimental Environment for Assessing Mental Fatigue & Cognitive Flexibility during Drone Operations

Authors:

Marcel F. Hinss,

Vincenzo Maria Vitale,

Nhat Tien Phan,

Raphaëlle N. Roy,

Anke M. BrockAuthors Info & Claims

HRI '24: Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction

Pages 906 - 909

https://doi.org/10.1145/3610977.3637484

Published: 11 March 2024 Publication History

Abstract

Mental fatigue from continuous operations without breaks represents a safety issue for military drone operations, as these systems are complex and operate during long shifts. Military operations are hard to study due to their sensitive nature. The open-access program UASOS serves as a testbed to examine the effects of mental fatigue in an ecologically valid environment. UASOS recreates fundamental aspects of military drone operations in a controllable environment that is easy enough for novices to understand but demanding enough to elicit mental fatigue. Participants alternate between navigating a drone-using either a trackball/mouse or a joystick-and searching for visual targets. The protocol is set up in a way that taxes the cognitive flexibility of participants by constantly requiring them to alternate between tasks. In addition, several parameters such as difficulty, duration, questionnaires, training phases, and more can be adapted. The task also allows for synchronization with physiological data using LabStreamingLayer. Implemented in python, the code is set up to be easily installed.

References

[1]

G Robert Arrabito, Geofrey Ho, Annie Lambert, Mark Rutley, Jocelyn Keillor, Allison Chiu, HeidiAu, and Ming Hou. 2010. Human Factors Issues for Controlling Uninhabited Aerial Vehicles:. (2010), 107.

[2]

Reg Austin. 2011. Unmanned Aircraft Systems: UAVS Design, Development and Deployment. John Wiley & Sons. Google-Books-ID: 03gdqhU61C0C.

[3]

G. Bradski. 2000. The OpenCV Library. Dr. Dobb's Journal of Software Tools (2000).

[4]

Stéphane Caid, Daniel Hauret, Marion Wolf, and Régis Mollard. 2016. Fatigue study and discourse analysis of french uninhabited aerial vehicle (UAV) operators to understand operational issues. In Proceedings of the 15th Ergo'IA "Ergonomie Et Informatique Avancé" Conference on -Ergo'IA '16. ACM Press, Bidart, France, 1--8. https://doi.org/10.1145/3050385.3050399

Digital Library

[5]

Jessica R. Cauchard, Mohamed Khamis, Jérémie Garcia, and Anke M. Brock. 2021. Toward a Roadmap for Human-Drone Interaction. Interactions 28, 2 (mar 2021), 76--81. https://doi.org/10.1145/3447889

Digital Library

[6]

Jessie Y. C. Chen and Michael J. Barnes. 2014. Human--Agent Teaming for Multirobot Control: A Review of Human Factors Issues. IEEE Transactions on Human-Machine Systems 44, 1 (Feb. 2014), 13--29. https://doi.org/10.1109/THMS. 2013.2293535

[7]

Justin CTR (FAA) Durham. 2020. Literature Review and Annotated Bibliography (1990 -- 2019): Duty Time, Shift Work, and Operator Fatigue for Consideration of Unmanned Aircraft Systems in Air Carrier Operations. (2020), 70.

[8]

A. Ghanbary Sartang, M. Ashnagar, E. Habibi, and S. Sadeghi. 2016. Evaluation of Rating Scale Mental Efort (RSME) efectiveness for mental workload assessment in nurses. Journal of Occupational Health and Epidemiology 5, 4 (Oct. 2016), 211--217. https://doi.org/10.18869/acadpub.johe.5.4.211 Publisher: Journal of Occupational Health and Epidemiology.

[9]

Stefanie Giese, David Carr, and Javaan Chahl. 2013. Implications for unmanned systems research of military UAV mishap statistics. In 2013 IEEE Intelligent Vehicles Symposium (IV). IEEE, Gold Coast City, Australia, 1191--1196. https://doi.org/10. 1109/IVS.2013.6629628

[10]

Marcel F Hinss, Anke M Brock, and Raphaëlle N Roy. 2023. The double task-switching protocol: An investigation into the efects of similarity and confict on cognitive fexibility in the context of mental fatigue. Plos one 18, 2 (2023), e0279021.

[11]

Emilie S. Jahanpour, Bruno Berberian, Jean-Paul Imbert, and Raphaëlle N. Roy. 2020. Cognitive fatigue assessment in operational settings: a review and UAS implications. IFAC-PapersOnLine 53, 5 (Jan. 2020), 330--337. https://doi.org/10. 1016/j.ifacol.2021.04.188

[12]

Kosuke Kaida, Masaya Takahashi, Torbjörn Åkerstedt, Akinori Nakata, Yasumasa Otsuka, Takashi Haratani, and Kenji Fukasawa. 2006. Validation of the Karolinska sleepiness scale against performance and EEG variables. Clinical Neurophysiology 117, 7 (July 2006), 1574--1581. https://doi.org/10.1016/j.clinph.2006.03.011

[13]

Christian Kothe, David Medine, Chadwick Boulay, Matthew Grivich, and Tristan Stenner. 2014. Lab streaming layer. URL https://github.com/sccn/labstreaminglayer (2014).

[14]

Y. LeCun, Fu Jie Huang, and L. Bottou. 2004. Learning methods for generic object recognition with invariance to pose and lighting. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004., Vol. 2. IEEE, Washington, DC, USA, 97--104. https://doi.org/10.1109/ CVPR.2004.1315150

[15]

Steven Macenski, Tully Foote, Brian Gerkey, Chris Lalancette, and William Woodall. 2022. Robot Operating System 2: Design, architecture, and uses in the wild. Science Robotics 7, 66 (2022), eabm6074. https://doi.org/10.1126/scirobotics. abm6074

[16]

Jonathan Peirce, Jeremy R Gray, Sol Simpson, Michael MacAskill, Richard Höchenberger, Hiroyuki Sogo, Erik Kastman, and Jonas Kristofer Lindeløv. 2019. PsychoPy2: Experiments in behavior made easy. Behavior research methods 51 (2019), 195--203.

[17]

Vanessa A. Petruo, Moritz Mückschel, and Christian Beste. 2018. On the role of the prefrontal cortex in fatigue efects on cognitive fexibility -a system neurophysiological approach. Scientifc Reports 8, 1 (April 2018), 6395. https: //doi.org/10.1038/s41598-018--24834-w

[18]

Sherwood W Samn and Layne P Perelli. 1982. Estimating aircrew fatigue: a technique with application to airlift operations. (1982).

[19]

Teresa Scheiman, Wayne Chappelle, and Elizabeth Sanford. 2017. U.S. Air Force Special Operations Command Remotely Piloted Aircraft Operator Fatigue Levels and Compensatory Strategies. (2017), 49.

[20]

Shital Shah, Debadeepta Dey, Chris Lovett, and Ashish Kapoor. 2017. AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles. In Field and Service Robotics. arXiv:arXiv:1705.05065 https://arxiv.org/abs/1705.05065

[21]

Peter Squire, Greg Trafton, and Raja Parasuraman. 2006. Human Control of Multiple Unmanned Vehicles: Efects of Interface Type on Execution and Task Switching Times. (2006), 7.

[22]

Anthony P Tvaryanas, William Platte, Caleb Swigart, Jayson Colebank, and Nita Lewis Miller. 2021. A Resurvey of Shift Work-Related Fatigue in MQ-1 Predator Unmanned Aircraft System Crewmembers. npj Digital Medicine 4, 1 (March 2021), 1--5. https://doi.org/10.1038/s41746-021-00415--6 Number: 1 Publisher: Nature Publishing Group.

[23]

Jaimie F. Veale. 2014. Edinburgh Handedness Inventory -Short Form: a revised version based on confrmatory factor analysis. Laterality 19, 2 (2014), 164--177. https://doi.org/10.1080/1357650X.2013.783045

[24]

Qile Wang, Qinqi Zhang, Weitong Sun, Chadwick Boulay, Kangsoo Kim, and Roghayeh Leila Barmaki. 2023. A scoping review of the use of lab streaming layer framework in virtual and augmented reality research. Virtual Reality (2023), 1--16.

Digital Library

[25]

Shijing Yu, Moritz Mückschel, and Christian Beste. 2021. Event-related synchronization/desynchronization and functional neuroanatomical regions associated with fatigue efects on cognitive fexibility. Journal of Neurophysiology 126, 2 (Aug. 2021), 383--397. https://doi.org/10.1152/jn.00228.2021

[26]

Torbjörn Åkerstedt and Mats Gillberg. 1990. Subjective and Objective Sleepiness in the Active Individual. International Journal of Neuroscience 52, 1--2 (Jan. 1990), 29--37. https://doi.org/10.3109/00207459008994241 Publisher: Taylor & Francis _eprint: https://doi.org/10.3109/00207459008994241.

Cited By

Hinss MBrock ARoy R(2024)Visual alerts for operator cognitive flexibility improvement: a neuroergonomic approachProceedings of the 12th International Conference on Human-Agent Interaction10.1145/3687272.3690870(326-328)Online publication date: 24-Nov-2024
https://dl.acm.org/doi/10.1145/3687272.3690870

Recommendations

Social Proxemics of Human-Drone Interaction: Flying Altitude and Size
HRI '18: Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction

To what extent do humans comfortably approach to hovering drones? In human-robot interaction, social proxemics is somewhat known. Han & Bae showed that students usually stand as far apart as the height of tele-robot teacher [1]. As commercial drone ...
Drone Near Me: Exploring Touch-Based Human-Drone Interaction

Personal drones are becoming more mainstream and are used for a variety of tasks, such as delivery and photography. The exposed blades in conventional drones raise serious safety concerns. To address this, commercial drones have been moving towards a ...
The Learning Experience of Becoming a FPV Drone Pilot
HRI '21 Companion: Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction

An immersive drone-flying modality known as First-Person View (FPV) flying is increasing in popularity. Nonetheless, due to its recency, there is a lack of research focusing on FPV drones. Specifically, the journey FPV pilots go through when learning ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '24: Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction

March 2024

982 pages

ISBN:9798400703225

DOI:10.1145/3610977

General Chairs:
Dan Grollman
Plus One Robotics, USA
,
Elizabeth Broadbent
University of Auckland, New Zealand
,
Program Chairs:
Wendy Ju
Cornell Tech, USA
,
Harold Soh
National University of Singapore, Singapore
,
Tom Williams
Colorado School of Mines, USA

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 March 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

HRI '24

Sponsor:

HRI '24: ACM/IEEE International Conference on Human-Robot Interaction

March 11 - 15, 2024

CO, Boulder, USA

Acceptance Rates

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
111
Total Downloads

Downloads (Last 12 months)111
Downloads (Last 6 weeks)4

Reflects downloads up to 20 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hinss MBrock ARoy R(2024)Visual alerts for operator cognitive flexibility improvement: a neuroergonomic approachProceedings of the 12th International Conference on Human-Agent Interaction10.1145/3687272.3690870(326-328)Online publication date: 24-Nov-2024
https://dl.acm.org/doi/10.1145/3687272.3690870

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents