The Effect of Predictive Formal Modelling at Runtime on Performance in Human-Swarm Interaction
Authors: 
Ayodeji O. Abioye, William Hunt, Yue Gu, 
Eike Schneiders, Mohammad Naiseh, 
Joel E. Fischer, Sarvapali D. Ramchurn, Mohammad D. Soorati, Blair Archibald, Michele SevegnaniAuthors Info & Claims
Ayodeji O. Abioye, William Hunt, Yue Gu, Eike Schneiders, Mohammad Naiseh, Joel E. Fischer, Sarvapali D. Ramchurn, Mohammad D. Soorati, Blair Archibald, Michele SevegnaniAuthors Info & ClaimsPages 172 - 176
Abstract
Formal Modelling is often used as part of the design and testing process of software development to ensure that components operate within suitable bounds even in unexpected circumstances. In this paper, we use predictive formal modelling (PFM) at runtime in a human-swarm mission and show that this integration can be used to improve the performance of human-swarm teams. We recruited 60 participants to operate a simulated aerial swarm to deliver parcels to target locations. In the PFM condition, operators were informed of the estimated completion times given the number of drones deployed, whereas, in the No-PFM condition, operators did not have this information. The operators could control the mission by adding or removing drones from the mission and thereby, increasing or decreasing the overall mission cost. The evaluation of human-swarm performance relied on four metrics: the task completion time, the number of agents, the number of completed tasks, and the cost per task. Our results show that PFM modelling at runtime improves mission performance without significantly affecting the operator's workload or the system's usability.
Supplemental Material
MP4 File
Supplemental video
- Download
- 16.23 MB
References
[1]
Ayodeji O. Abioye, Mohammad Naiseh, William Hunt, Jediah Clark, Sarvapali D. Ramchurn, and Mohammad D. Soorati. 2023. The Effect of Data Visualisation Quality and Task Density on Human-Swarm Interaction. In 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). 1494--1501. https://doi.org/10.1109/RO-MAN57019.2023.10309454
[2]
Ayodeji Opeyemi Abioye, Stephen D. Prior, Peter Saddington, and Sarvapali D. Ramchurn. 2022. The performance and cognitive workload analysis of a multimodal speech and visual gesture (mSVG) UAV control interface. Robotics and Autonomous Systems, Vol. 147 (2022), 103915. https://doi.org/10.1016/j.robot.2021.103915
[3]
Ezio Bartocci, Yliès Falcone, Adrian Francalanza, and Giles Reger. 2018. Introduction to runtime verification. Lectures on Runtime Verification: Introductory and Advanced Topics (2018), 1--33.
[4]
Nelly Bencomo, Svein Hallsteinsen, and Eduardo Santana De Almeida. 2012. A view of the dynamic software product line landscape. Computer, Vol. 45, 10 (2012), 36--41.
[5]
Gordon Blair, Nelly Bencomo, and Robert B France. 2009. Models@ run. time. Computer, Vol. 42, 10 (2009), 22--27.
[6]
Ioana Boureanu, Panagiotis Kouvaros, and Alessio Lomuscio. 2016. Verifying security properties in unbounded multiagent systems. In Proceedings of the 2016 international conference on autonomous agents & multiagent systems. 1209--1217.
[7]
John Brooke. 1995. SUS: A quick and dirty usability scale. Usability Eval. Ind., Vol. 189 (11 1995).
[8]
Muffy Calder and Michele Sevegnani. 2019. Stochastic Model Checking for Predicting Component Failures and Service Availability. IEEE Transactions on Dependable and Secure Computing, Vol. 16, 1 (2019), 174--187. https://doi.org/10.1109/TDSC.2017.2650901
[9]
Jediah R Clark, Mohammad Naiseh, Joel Fischer, Marisé Galvez Trigo, Katie Parnell, Mario Brito, Adrian Bodenmann, Sarvapali D Ramchurn, and Mohammad D Soorati. 2022. Industry Led Use-Case Development for Human-Swarm Operations. In AAAI 2022 Spring Symposium Series (Putting AI in the Critical Loop: Assured Trust and Autonomy in Human-Machine Teams). AAAI, 1--6.
[10]
Agner Krarup Erlang. 1917. Solution of some problems in the theory of probabilities of significance in automatic telephone exchanges. Post Office Electrical Engineer's Journal, Vol. 10 (1917), 189--197.
[11]
Yliès Falcone, Srdjan Krstić, Giles Reger, and Dmitriy Traytel. 2021. A taxonomy for classifying runtime verification tools. International Journal on Software Tools for Technology Transfer, Vol. 23, 2 (2021), 255--284.
[12]
Yue Gu, William Hunt, Blair Archibald, Mengwei Xu, Michele Sevegnani, and Mohammad D. Soorati. 2023. Successful Swarms: Operator Situational Awareness with Modelling and Verification at Runtime. In 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). 541--548. https://doi.org/10.1109/RO-MAN57019.2023.10309626
[13]
Sandra G Hart. 1986. NASA task load index (TLX). (1986).
[14]
Maria-Theresa Oanh Hoang, Niels van Berkel, Mikael B. Skov, and Timothy R. Merritt. 2023. Challenges and Requirements in Multi-Drone Interfaces. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI EA '23). Association for Computing Machinery, New York, NY, USA, Article 65, 9 pages. https://doi.org/10.1145/3544549.3585673
[15]
William Hunt, Jack Ryan, Ayodeji O. Abioye, Sarvapali D. Ramchurn, and Mohammad D Soorati. 2023. Demonstrating Performance Benefits of Human-Swarm Teaming. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems (London, United Kingdom). IFAAMAS, Richland, SC, 3062--3064.
[16]
Panagiotis Kouvaros and Alessio Lomuscio. 2015. Verifying emergent properties of swarms. In Twenty-Fourth International Joint Conference on Artificial Intelligence.
[17]
Dasom Lee, David J. Hess, and Michiel A. Heldeweg. 2022. Safety and privacy regulations for unmanned aerial vehicles: A multiple comparative analysis. Technology in Society, Vol. 71 (2022), 102079. https://doi.org/10.1016/j.techsoc.2022.102079
[18]
Alessio Lomuscio and Edoardo Pirovano. 2021. Verifying fault-tolerance in probabilistic swarm systems. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence. 325--331.
[19]
Mingyang Lyu, Yibo Zhao, Chao Huang, and Hailong Huang. 2023. Unmanned Aerial Vehicles for Search and Rescue: A Survey. Remote Sensing, Vol. 15, 13 (2023). https://doi.org/10.3390/rs15133266
[20]
Mohammad Naiseh, Mohammad D Soorati, and Sarvapali Ramchurn. 2023. Outlining the design space of eXplainable swarm (xSwarm): experts perspective. arXiv preprint arXiv:2309.01269 (2023).
[21]
Katie J Parnell, Joel E Fischer, Jediah R Clark, Adrian Bodenmann, Maria Jose Galvez Trigo, Mario P Brito, Mohammad D Soorati, Katherine L Plant, and Sarvapali D Ramchurn. 2022. Trustworthy UAV relationships: Applying the Schema Action World taxonomy to UAVs and UAV swarm operations. International Journal of Human-Computer Interaction (2022), 1--17.
[22]
Prolific. 2023. Prolific - quickly find research participants you can trust. https://www.prolific.co/
[23]
Sarvapali D Ramchurn, Joel E Fischer, Yuki Ikuno, Feng Wu, Jack Flann, and Antony Waldock. 2015. A study of human-agent collaboration for multi-UAV task allocation in dynamic environments. In Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI). 1184--1192.
[24]
Eike Schneiders, EunJeong Cheon, Jesper Kjeldskov, Matthias Rehm, and Mikael B. Skov. 2022. Non-Dyadic Interaction: A Literature Review of 15 Years of Human-Robot Interaction Conference Publications. J. Hum.-Robot Interact., Vol. 11, 2, Article 13 (feb 2022), 32 pages. https://doi.org/10.1145/3488242
[25]
Mohammad D Soorati, Mohammad Naiseh, William Hunt, Katie Parnell, Jediah Clark, and Sarvapali D Ramchurn. 2023. Enabling Trustworthiness in Human-swarm Systems Through a Digital Twin. Putting AI in the Critical Loop (2023).
[26]
Usability.gov Improving the User Experience. 2023. System Usability Scale (SUS). https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html
[27]
James Wilson, Greg Chance, Peter Winter, Suet Lee, Emma Milner, Dhaminda Abeywickrama, Shane Windsor, John Downer, Kerstin Eder, Jonathan Ives, and Sabine Hauert. 2023. Trustworthy Swarms. In Proceedings of the First International Symposium on Trustworthy Autonomous Systems (Edinburgh, United Kingdom) (TAS '23). Association for Computing Machinery, New York, NY, USA, Article 10, 11 pages. https://doi.org/10.1145/3597512.3599705
Index Terms
- The Effect of Predictive Formal Modelling at Runtime on Performance in Human-Swarm Interaction
Recommendations
Cognitive correlates of EEG spectral power indicate human-swarm task performance
IOT '18: Proceedings of the 8th International Conference on the Internet of ThingsIn this paper, we examine whether the geometric complexity of a robotic group affects performance in a human-swarm target acquisition task, and if these changes are reflected in average neurophysiological and behavioral characteristics. This is one of ...
Flying Frustum: A Spatial Interface for Enhancing Human-UAV Awareness
HAI '15: Proceedings of the 3rd International Conference on Human-Agent InteractionWe present Flying Frustum, a 3D spatial interface that enables control of semi-autonomous UAVs (Unmanned Aerial Vehicles) using pen interaction on a physical model of the terrain, and that spatially situates the information streaming from the UAVs onto ...
Social Proxemics of Human-Drone Interaction: Flying Altitude and Size
HRI '18: Companion of the 2018 ACM/IEEE International Conference on Human-Robot InteractionTo what extent do humans comfortably approach to hovering drones? In human-robot interaction, social proxemics is somewhat known. Han & Bae showed that students usually stand as far apart as the height of tele-robot teacher [1]. As commercial drone ...
Comments
Information & Contributors
Information
Published In
March 2024
1408 pages
ISBN:9798400703232
DOI:10.1145/3610978
- General Chairs:
- Dan Grollman,
- Elizabeth Broadbent,
- Program Chairs:
- Wendy Ju,
- Harold Soh,
- Tom Williams
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 March 2024
Check for updates
Author Tags
Qualifiers
- Short-paper
Funding Sources
- UK Research and Innovation (UKRI)
- Engineering and Physical Sciences Research Council (EPSRC)
Conference
HRI '24: ACM/IEEE International Conference on Human-Robot Interaction
March 11 - 15, 2024
CO, Boulder, USA
Acceptance Rates
Overall Acceptance Rate 268 of 1,124 submissions, 24%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 117Total Downloads
- Downloads (Last 12 months)117
- Downloads (Last 6 weeks)24
Reflects downloads up to 03 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in