HIFuzz: Human Interaction Fuzzing for Small Unmanned Aerial Vehicles
Authors: 
Theodore Chambers, 
Michael Vierhauser, Ankit Agrawal, Michael Murphy, Jason Matthew Brauer, Salil Purandare, Myra B Cohen, Jane Cleland-HuangAuthors Info & Claims
Theodore Chambers, Michael Vierhauser, Ankit Agrawal, Michael Murphy, Jason Matthew Brauer, Salil Purandare, Myra B Cohen, Jane Cleland-HuangAuthors Info & ClaimsArticle No.: 266, Pages 1 - 14
Abstract
Small Unmanned Aerial Systems (sUAS) must meet rigorous safety standards when deployed in high-stress emergency response scenarios; however many reported accidents have involved humans in the loop. In this paper, we, therefore, present the HiFuzz testing framework, which uses fuzz testing to identify system vulnerabilities associated with human interactions. HiFuzz includes three distinct levels that progress from a low-cost, limited-fidelity, large-scale, no-hazard environment, using fully simulated Proxy Human Agents, via an intermediate level, where proxy humans are replaced with real humans, to a high-stakes, high-cost, real-world environment. Through applying HiFuzz to an autonomous multi-sUAS system-under-test, we show that each test level serves a unique purpose in revealing vulnerabilities and making the system more robust with respect to human mistakes. While HiFuzz is designed for testing sUAS systems, we further discuss its potential for use in other Cyber-Physical Systems.
Supplemental Material
MP4 File - Video Presentation
Video Presentation
- Download
- 27.02 MB
- Transcript
CSV File - L1 Tests
L1-TESTS (csv): A list of all L1 test cases including their contexts, configurations, actions etc.
- Download
- 72.47 KB
XLSX File - L2 Tests
L2-TESTS (xlsx): A list of all L2 test cases (subset of L1) including contexts, configurations, actions, rationale for including in L2, summarized outcome of L2, and snapshot image extracted from the L2 flight log replay along with the relevant PX4 log url which allows for inspection of log data and dynamic 3D replay.
- Download
- 13.69 MB
References
[1]
2015. Autoware - the world’s leading open-source software project for autonomous driving. https://github.com/autowarefoundation/autoware. (Accessed on 12/01/2023).
[2]
Ankit Agrawal, Sophia J. Abraham, Benjamin Burger, Chichi Christine, Luke Fraser, John M. Hoeksema, Sarah Hwang, Elizabeth Travnik, Shreya Kumar, Walter J. Scheirer, Jane Cleland-Huang, Michael Vierhauser, Ryan Bauer, and Steve Cox. 2020. The Next Generation of Human-Drone Partnerships: Co-Designing an Emergency Response System. In Proc. of CHI Conference on Human Factors in Computing Systems. ACM, New York, 1–13. https://doi.org/10.1145/3313831.3376825
[3]
Ardupilot. 2023. Flight Controller Modes. https://ardupilot.org/plane/docs/flight-modes.html. [Online; Accessed 01-07-2023].
[4]
Clint R Balog, Brent A Terwilliger, Dennis A Vincenzi, and David C Ison. 2017. Examining human factors challenges of sustainable small unmanned aircraft system (sUAS) operations. In Advances in Human Factors in Robots and Unmanned Systems: Proceedings of the AHFE 2016 International Conference on Human Factors in Robots and Unmanned Systems, July 27-31, 2016, Walt Disney World®, Florida, USA. Springer, 61–73.
[5]
Marcel Böhme, Van-Thuan Pham, Manh-Dung Nguyen, and Abhik Roychoudhury. 2017. Directed Greybox Fuzzing. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (Dallas, Texas, USA) (CCS ’17). Association for Computing Machinery, New York, NY, USA, 2329–2344. https://doi.org/10.1145/3133956.3134020
[6]
Marcel Böhme, Van-Thuan Pham, and Abhik Roychoudhury. 2016. Coverage-Based Greybox Fuzzing as Markov Chain. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (Vienna, Austria) (CCS ’16). Association for Computing Machinery, New York, NY, USA, 1032–1043. https://doi.org/10.1145/2976749.2978428
[7]
Matthew L Bolton and Ellen J Bass. 2009. A method for the formal verification of human-interactive systems. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 53. SAGE Publications Sage CA: Los Angeles, CA, 764–768.
[8]
Matthew L Bolton, Ellen J Bass, and Radu I Siminiceanu. 2013. Using formal verification to evaluate human-automation interaction: A review. IEEE Transactions on Systems, Man, and Cybernetics: Systems 43, 3 (2013), 488–503.
[9]
CARLA. 2023. Open-source simulator for autonomous driving research. https://carla.org. [Online: accessed 8-14-2023].
[10]
Linfeng Chen, Kazuki Takashima, Kazuyuki Fujita, and Yoshifumi Kitamura. 2021. Pinpointfly: An egocentric position-control drone interface using mobile ar. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.
[11]
Yuqi Chen, Bohan Xuan, Christopher M Poskitt, Jun Sun, and Fan Zhang. 2020. Active fuzzing for testing and securing cyber-physical systems. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis. 14–26.
[12]
Jane Cleland-Huang, Theodore Chambers, Sebastian Zudaire, Muhammed Tawfiq Chowdhury, Ankit Agrawal, and Michael Vierhauser. 2024. Human–machine Teaming with Small Unmanned Aerial Systems in a MAPE-K Environment. ACM Trans. Auton. Adapt. Syst. 19, 1, Article 3 (feb 2024), 35 pages. https://doi.org/10.1145/3618001
[13]
Jane Cleland-Huang, Michael Vierhauser, and Sean Bayley. 2018. Dronology: an incubator for cyber-physical systems research. In Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results. 109–112. https://doi.org/10.1145/3183399.3183408
[14]
Paul Curzon, Rimvydas Rukšėnas, and Ann Blandford. 2007. An approach to formal verification of human–computer interaction. Formal Aspects of Computing 19 (2007), 513–550.
[15]
Byron DeVries and Betty HC Cheng. 2018. Run-time monitoring of self-adaptive systems to detect n-way feature interactions and their causes. In Proceedings of the 13th International Conference on Software Engineering for Adaptive and Self-Managing Systems. 94–100.
[16]
Dronology. 2020. Research Incubator and Dataset. https://dronology.info. [Last accessed 01-01-2022].
[17]
Mica R. Endsley. 2011. Designing for Situation Awareness: An Approach to User-Centered Design, Second Edition (2nd ed.). CRC Press, Inc., Boca Raton, FL, USA.
[18]
Mica R Endsley. 2017. Autonomous driving systems: A preliminary naturalistic study of the Tesla Model S. Journal of Cognitive Engineering and Decision Making 11, 3 (2017), 225–238.
[19]
Chin-Feng Fan, Ching-Chieh Chan, Hsiang-Yu Yu, and Swu Yih. 2018. A simulation platform for human-machine interaction safety analysis of cyber-physical systems. International journal of industrial ergonomics 68 (2018), 89–100.
[20]
Andrea Fioraldi, Alessandro Mantovani, Dominik Maier, and Davide Balzarotti. 2023. Dissecting American Fuzzy Lop: A FuzzBench Evaluation. ACM Trans. Softw. Eng. Methodol. 32, 2, Article 52 (mar 2023), 26 pages. https://doi.org/10.1145/3580596
[21]
Flight Safety Foundation. 2019. Preliminary Report B737-800MAX. https://flightsafety.org/preliminary-report-b737-800max-et-avj. [Last accessed 01-01-2022].
[22]
Daniel S Fowler, Jeremy Bryans, Siraj Ahmed Shaikh, and Paul Wooderson. 2018. Fuzz testing for automotive cyber-security. In 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W). IEEE, 239–246.
[23]
FrySky. 2023. Taranis Series Handheld RC. https://www.frsky-rc.com/product-category/transmitters/taranis-series. [Online: accessed 8-14-2023].
[24]
Jia Cheng Han and Zhi Quan Zhou. 2020. Metamorphic Fuzz Testing of Autonomous Vehicles. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops (Seoul, Republic of Korea) (ICSEW’20). Association for Computing Machinery, New York, NY, USA, 380–385. https://doi.org/10.1145/3387940.3392252
[25]
Ruidong Han, Chao Yang, Siqi Ma, JiangFeng Ma, Cong Sun, Juanru Li, and Elisa Bertino. 2022. Control parameters considered harmful: Detecting range specification bugs in drone configuration modules via learning-guided search. In Proceedings of the 44th International Conference on Software Engineering. 462–473.
[26]
Chenxu Hao, Anany Dwivedi, and Philipp Beckerle. 2022. A Literature-Based Perspective on Human-Centered Design and Evaluation of Interfaces for Virtual Reality in Robotics. In Human-Friendly Robotics 2022 - HFR: 15th International Workshop on Human-Friendly Robotics, Delft, The Netherlands, 22-23 September 2022(Springer Proceedings in Advanced Robotics, Vol. 26), Pablo Borja, Cosimo Della Santina, Luka Peternel, and Elena Torta (Eds.). Springer, 1–13. https://doi.org/10.1007/978-3-031-22731-8_1
[27]
Viviane Herdel, Lee J Yamin, and Jessica R Cauchard. 2022. Above and beyond: A scoping review of domains and applications for human-drone interaction. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–22.
[28]
Md Nafee Al Islam, Muhammed Tawfiq Chowdhury, Ankit Agrawal, Michael Murphy, Raj Mehta, Daria Kudriavtseva, Jane Cleland-Huang, Michael Vierhauser, and Marsha Chechik. 2023. Configuring mission-specific behavior in a product line of collaborating Small Unmanned Aerial Systems. J. Syst. Softw. 197 (2023), 111543. https://doi.org/10.1016/J.JSS.2022.111543
[29]
Shinpei Kato, Eijiro Takeuchi, Yoshio Ishiguro, Yoshiki Ninomiya, Kazuya Takeda, and Tsuyoshi Hamada. 2015. An open approach to autonomous vehicles. IEEE Micro 35, 6 (2015), 60–68.
[30]
Tim Kelly and Rob Weaver. 2004. The Goal Structuring Notation – A Safety Argument Notation. In Proc. Dependable Syst. Networks 2004 Work. Assur. Cases.
[31]
Md Nafiz Hasan Khan and Carman Neustaedter. 2019. An exploratory study of the use of drones for assisting firefighters during emergency situations. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–14.
[32]
Hyungsub Kim, Muslum Ozgur Ozmen, Antonio Bianchi, Z Berkay Celik, and Dongyan Xu. 2021. PGFUZZ: Policy-Guided Fuzzing for Robotic Vehicles. In NDSS.
[33]
Taegyu Kim, Chung Hwan Kim, Junghwan Rhee, Fan Fei, Zhan Tu, Gregory Walkup, Xiangyu Zhang, Xinyan Deng, and Dongyan Xu. 2019. { RVFuzzer} : Finding input validation bugs in robotic vehicles through { Control-Guided} testing. In 28th USENIX Security Symposium (USENIX Security 19). 425–442.
[34]
L.T. Kohn, J.M. Corrigan, and M.s. Donaldson. 1999. To err is human, Building a safety health system. Washington, DC: National Academy Press (1999).
[35]
Wojtek J Krzanowski and YT Lai. 1988. A criterion for determining the number of groups in a data set using sum-of-squares clustering. Biometrics (1988), 23–34.
[36]
Nancy G Leveson and Peter R Harvey. 1983. Software fault tree analysis. Journal of Systems and Software 3, 2 (1983), 173–181.
[37]
Christoph Luckeneder, Michael Rathmair, and Hermann Kaindl. 2017. Investigating and coordinating safety-critical feature interactions in automotive systems using simulation. (2017).
[38]
Vasudev S Mallan, Syam Gopi, Alexander Muir, and Rao R Bhavani. 2017. Comparative empirical usability assessment of two HRI input devices for a mobile robot. In 2017 4th International Conference on Signal Processing, Computing and Control (ISPCC). IEEE, 331–337.
[39]
Jason S. Mccarley and Christopher D. Wickens. [n. d.]. Human factors concerns in UAV flight. Technical Report.
[40]
Henry Muccini and Mahyar Tourchi Moghaddam. 2018. IOT Architectural Styles. In Proc. of 2018 European Conference on Software Architecture. Springer, 68–85.
[41]
D.C. Nagel. 1998. Human error in aviation Operations. Human factors in Aviation, E.L.Weiner and E.C.Nagel (Eds)19890047069, 34 (1998), 263–303. https://doi.org/10.1109/2.910904
[42]
NASA. 2023. NASA-UTM: Unmanned Aircraft Systems Traffic Management. https://www.nasa.gov/centers-and-facilities/ames/what-is-unmanned-aircraft-systems-traffic-management. [Online: accessed 8-14-2023].
[43]
Pedro Neto, J Norberto Pires, and A Paulo Moreira. 2010. High-level programming and control for industrial robotics: using a hand-held accelerometer-based input device for gesture and posture recognition. Industrial Robot: An International Journal 37, 2 (2010), 137–147.
[44]
Jack Nicas, Natalie Kitroeff, David Gelles, and James Glanz. 2019. Boeing Built Deadly Assumptions Into 737 Max, Blind to a Late Design Change. The New York Times, https://www.nytimes.com/2019/06/01/business/boeing-737-maxcrash html [accessed: 23.01.2020] (2019).
[45]
Sara Nikula, Célia Martinie, Philippe A. Palanque, Julius Hekkala, Outi-Marja Latvala, and Kimmo Halunen. 2022. Models-Based Analysis of Both User and Attacker Tasks: Application to EEVEHAC. In Human-Centered Software Engineering - 9th IFIP WG 13.2 International Working Conference, HCSE 2022, Eindhoven, The Netherlands, August 24-26, 2022, Proceedings(Lecture Notes in Computer Science, Vol. 13482), Regina Bernhaupt, Carmelo Ardito, and Stefan Sauer (Eds.). Springer, 70–89. https://doi.org/10.1007/978-3-031-14785-2_5
[46]
Donald A. Norman and Stephen W. Draper (Eds.). 1986. User centered system design: New perspectives on human-computer interaction. Lawrence Erlbaum Associates, Hillsdale, NJ.
[47]
Mitchell Olsthoorn, Arie van Deursen, and Annibale Panichella. 2021. Generating Highly-Structured Input Data by Combining Search-Based Testing and Grammar-Based Fuzzing. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering (Virtual Event, Australia) (ASE ’20). Association for Computing Machinery, New York, NY, USA, 1224–1228. https://doi.org/10.1145/3324884.3418930
[48]
Open Robotics. 2023. Gazebo. https://gazebosim.org. [Online: accessed 8-14-2023].
[49]
Rohan Padhye, Caroline Lemieux, and Koushik Sen. 2019. JQF: Coverage-Guided Property-Based Testing in Java. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (Beijing, China) (ISSTA 2019). Association for Computing Machinery, New York, NY, USA, 398–401. https://doi.org/10.1145/3293882.3339002
[50]
Philippe Palanque and Célia Martinie. [n. d.]. Designing and Assessing Interactive Systems Using Task Models. 2016. In ACM CHI Extended Abstracts. 976–979.
[51]
PX4. 2022. jMAVSim. https://docs.px4.io/master/en/simulation/jmavsim.html. [Last accessed 01-01-2022].
[52]
PX4. 2023. Flight Controller Modes. https://docs.px4.io/main/en/flight_modes. [Online; Accessed 01-07-2023].
[53]
PX4. 2023. Flight Review Platform. https://logs.px4.io/. [Online: accessed 8-14-2023].
[54]
Balita Heriniaina Rakotonarivo, Nicolas Drougard, Stéphane Conversy, and Jérémie Garcia. 2023. Cleared for Safe Take-off? Improving the Usability of Mission Preparation to Mitigate the Safety Risks of Drone Operations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17.
[55]
Donald J. Reifer. 1979. Software Failure Modes and Effects Analysis. IEEE Trans. Reliability R-28,3 (1979), 247–249.
[56]
Shital Shah, Debadeepta Dey, Chris Lovett, and Ashish Kapoor. 2018. Airsim: High-fidelity visual and physical simulation for autonomous vehicles. In Field and Service Robotics. Springer, 621–635.
[57]
Kevin J Sullivan, Joanne Bechta Dugan, and David Coppit. 1999. The Galileo fault tree analysis tool. In Digest of Papers. Twenty-Ninth Annual International Symposium on Fault-Tolerant Computing (Cat. No. 99CB36352). IEEE, 232–235.
[58]
Yang Sun, Christopher M. Poskitt, Jun Sun, Yuqi Chen, and Zijiang Yang. 2023. LawBreaker: An Approach for Specifying Traffic Laws and Fuzzing Autonomous Vehicles. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (Rochester, MI, USA) (ASE ’22). Association for Computing Machinery, New York, NY, USA, Article 62, 12 pages. https://doi.org/10.1145/3551349.3556897
[59]
MA Syakur, BK Khotimah, EMS Rochman, and Budi Dwi Satoto. 2018. Integration k-means clustering method and elbow method for identification of the best customer profile cluster. In IOP conference series: materials science and engineering, Vol. 336. IOP Publishing, 012017.
[60]
Ari Takanen, Jared D Demott, Charles Miller, and Atte Kettunen. 2018. Fuzzing for software security testing and quality assurance. Artech House.
[61]
Michael Vierhauser, Md Nafee Al Islam, Ankit Agrawal, Jane Cleland-Huang, and James Mason. 2021. Hazard analysis for human-on-the-loop interactions in sUAS systems. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 8–19.
[62]
Herman Wijaya, Maurício Aniche, and Aditya Mathur. 2020. Domain-based fuzzing for supervised learning of anomaly detection in cyber-physical systems. In Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops. 237–244.
[63]
Mingxin Yu, Yingzi Lin, David Schmidt, Xiangzhou Wang, and Yu Wang. 2014. Human-robot interaction based on gaze gestures for the drone teleoperation. Journal of Eye Movement Research 7, 4 (2014), 1–14.
[64]
Xiaogang Zhu, Sheng Wen, Seyit Camtepe, and Yang Xiang. 2022. Fuzzing: A Survey for Roadmap. ACM Comput. Surv. 54, 11s, Article 230 (sep 2022), 36 pages. https://doi.org/10.1145/3512345
Index Terms
- HIFuzz: Human Interaction Fuzzing for Small Unmanned Aerial Vehicles
Recommendations
Towards Autonomous Autorotation Landing for Small Size Unmanned Helicopters
The consideration of safety issues in the operation of helicopters involves the capability to perform emergency descending maneuvers through the autorotation principle when the engine is no longer supplying power. When comparing manned and unmanned ...
Unmanned aerial vehicles
As unmanned aerial vehicles have become more affordable, their popularity with the general public and commercial organisations has seen significant growth in recent years. Whilst remaining a device for both the hobbyist and aircraft-enthusiast to enjoy, ...
Comments
Information & Contributors
Information
Published In
May 2024
18961 pages
ISBN:9798400703300
DOI:10.1145/3613904
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 May 2024
Check for updates
Badges
Artifacts Available / v1.1
Honorable Mention
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Data Availability
L1 Tests: L1-TESTS (csv): A list of all L1 test cases including their contexts, configurations, actions etc. https://dl.acm.org/doi/10.1145/3613904.3642958#pn6978-supplemental-material-1.csv
L2 Tests: L2-TESTS (xlsx): A list of all L2 test cases (subset of L1) including contexts, configurations, actions, rationale for including in L2, summarized outcome of L2, and snapshot image extracted from the L2 flight log replay along with the relevant PX4 log url which allows for inspection of log data and dynamic 3D replay. https://dl.acm.org/doi/10.1145/3613904.3642958#pn6978-supplemental-material-2.xlsx
L1 Tests: L1-TESTS (csv): A list of all L1 test cases including their contexts, configurations, actions etc. https://dl.acm.org/doi/10.1145/3613904.3642958#pn6978-supplemental-material-1.csv
L2 Tests: L2-TESTS (xlsx): A list of all L2 test cases (subset of L1) including contexts, configurations, actions, rationale for including in L2, summarized outcome of L2, and snapshot image extracted from the L2 flight log replay along with the relevant PX4 log url which allows for inspection of log data and dynamic 3D replay. https://dl.acm.org/doi/10.1145/3613904.3642958#pn6978-supplemental-material-2.xlsx
Funding Sources
Conference
Acceptance Rates
Overall Acceptance Rate 6,199 of 26,314 submissions, 24%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 353Total Downloads
- Downloads (Last 12 months)353
- Downloads (Last 6 weeks)44
Reflects downloads up to 11 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format