The Flexcrash Platform for Testing Autonomous Vehicles in Mixed-Traffic Scenarios
Authors: 
Alessio Gambi, Shreya Mathews, Benedikt Steininger, Mykhailo Poienko, David BobekAuthors Info & Claims
Alessio Gambi, Shreya Mathews, Benedikt Steininger, Mykhailo Poienko, David BobekAuthors Info & ClaimsPages 1811 - 1815
Abstract
Autonomous vehicles (AV) leverage Artificial Intelligence to reduce accidents and improve fuel efficiency while sharing the roads with human drivers. Current AV prototypes have not yet reached these goals, highlighting the need for better development and testing methodologies. AV testing practices extensively rely on simulations, but existing AV tools focus on testing single AV instances or do not consider human drivers. Thus, they might generate many irrelevant mixed-traffic test scenarios. The Flexcrash platform addresses these issues by allowing the generation and simulation of mixed-traffic scenarios, thus enabling testers to identify realistic critical scenarios, traffic experts to create new datasets, and regulators to extend consumer testing benchmarks.
References
[1]
Matthias Althoff, Markus Koschi, and Stefanie Manzinger. 2017. CommonRoad: Composable benchmarks for motion planning on roads. In IEEE Intelligent Vehicles Symposium, IV 2017, Los Angeles, CA, USA, June 11-14, 2017. IEEE, 719–726. https://doi.org/10.1109/IVS.2017.7995802
[2]
Avishay Artsy. [n. d.]. San Francisco’s robotaxi experiment is getting out of hand. https://www.vox.com/technology/2023/8/19/23837648/self-driving-taxis-gm-cruise-alphabet-waymo
[3]
Autoware. [n. d.]. Autoware: Open-source software for urban autonomous driving. https://github.com/CPFL/Autoware
[4]
Baidu. [n. d.]. Baidu apollo: An open autonomous driving platform. https://github.com/ApolloAuto/apollo
[5]
Sai Krishna Bashetty, Heni Ben Amor, and Georgios Fainekos. 2020. DeepCrashTest: Turning Dashcam Videos into Virtual Crash Tests for Automated Driving Systems. In 2020 IEEE International Conference on Robotics and Automation, ICRA 2020, Paris, France, May 31 - August 31, 2020. IEEE, 11353–11360. https://doi.org/10.1109/ICRA40945.2020.9197053
[6]
BeamNG GmbH. [n. d.]. BeamNG.tech. https://www.beamng.tech/
[7]
Mingfei Cheng, Yuan Zhou, and Xiaofei Xie. 2023. BehAVExplor: Behavior Diversity Guided Testing for Autonomous Driving Systems. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2023, Seattle, WA, USA, July 17-21, 2023, René Just and Gordon Fraser (Eds.). ACM, 488–500. https://doi.org/10.1145/3597926.3598072
[8]
Florian Dandl, Klaus Bogenberger, and Hani S. Mahmassani. 2019. Autonomous Mobility-on-Demand Real-Time Gaming Framework. In 6th International Conference on Models and Technologies for Intelligent Transportation Systems, MT-ITS 2019, Cracow, Poland, June 5-7, 2019. IEEE, 1–10. https://doi.org/10.1109/MTITS.2019.8883286
[9]
Wenhao Ding, Chejian Xu, Mansur Arief, Haohong Lin, Bo Li, and Ding Zhao. 2023. A Survey on Safety-Critical Driving Scenario Generation - A Methodological Perspective. IEEE Trans. Intell. Transp. Syst., 24, 7 (2023), 6971–6988. https://doi.org/10.1109/TITS.2023.3259322
[10]
Alessio Gambi, Tri Huynh, and Gordon Fraser. 2019. Generating effective test cases for self-driving cars from police reports. In Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/SIGSOFT FSE 2019, Tallinn, Estonia, August 26-30, 2019, Marlon Dumas, Dietmar Pfahl, Sven Apel, and Alessandra Russo (Eds.). ACM, 257–267. https://doi.org/10.1145/3338906.3338942
[11]
Alessio Gambi, Vuong Nguyen, Jasim Ahmed, and Gordon Fraser. 2022. Generating Critical Driving Scenarios from Accident Sketches. In IEEE International Conference On Artificial Intelligence Testing, AITest 2022, Newark, CA, USA, August 15-18, 2022. IEEE, 95–102. https://doi.org/10.1109/AITEST55621.2022.00022
[12]
Global NCAP. [n. d.]. Global New Car Assessment Programme. https://www.globalncap.org/
[13]
Yuqi Huai, Yuntianyi Chen, Sumaya Almanee, Tuan Ngo, Xiang Liao, Ziwen Wan, Qi Alfred Chen, and Joshua Garcia. 2023. Doppelgänger Test Generation for Revealing Bugs in Autonomous Driving Software. In 45th IEEE/ACM International Conference on Software Engineering, ICSE 2023, Melbourne, Australia, May 14-20, 2023. IEEE, 2591–2603. https://doi.org/10.1109/ICSE48619.2023.00216
[14]
Nidhi Kalra and Susan M. Paddock. 2016. Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and Practice, 94 (2016), 182–193. issn:0965-8564 https://doi.org/10.1016/j.tra.2016.09.010
[15]
Moritz Klischat and Matthias Althoff. 2019. Generating Critical Test Scenarios for Automated Vehicles with Evolutionary Algorithms. In 2019 IEEE Intelligent Vehicles Symposium, IV 2019, Paris, France, June 9-12, 2019. IEEE, 2352–2358. https://doi.org/10.1109/IVS.2019.8814230
[16]
Yueyuan Li, Wei Yuan, Songan Zhang, Weihao Yan, Qiyuan Shen, Chunxiang Wang, and Ming Yang. 2024. Choose Your Simulator Wisely: A Review on Open-Source Simulators for Autonomous Driving. IEEE Trans. Intell. Veh., 9, 5 (2024), 4861–4876. https://doi.org/10.1109/TIV.2024.3374044
[17]
Sebastian Maierhofer, Moritz Klischat, and Matthias Althoff. 2021. CommonRoad Scenario Designer: An Open-Source Toolbox for Map Conversion and Scenario Creation for Autonomous Vehicles. In 24th IEEE International Intelligent Transportation Systems Conference, ITSC 2021, Indianapolis, IN, USA, September 19-22, 2021. IEEE, 3176–3182. https://doi.org/10.1109/ITSC48978.2021.9564885
[18]
State of California Dept. of Motor Vehicles. [n. d.]. Autonomous Vehicle Collision Reports. https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/autonomous-vehicle-collision-reports/
[19]
Richard N Taylor, Nenad Medvidovic, and Eric M Dashofy. 2009. Software architecture: foundations, theory, and practice. Wiley Publishing.
[20]
Haoxiang Tian, Guoquan Wu, Jiren Yan, Yan Jiang, Jun Wei, Wei Chen, Shuo Li, and Dan Ye. 2022. Generating Critical Test Scenarios for Autonomous Driving Systems via Influential Behavior Patterns. In 37th IEEE/ACM International Conference on Automated Software Engineering, ASE 2022, Rochester, MI, USA, October 10-14, 2022. ACM, 46:1–46:12. https://doi.org/10.1145/3551349.3560430
[21]
Moritz Werling, Julius Ziegler, Sören Kammel, and Sebastian Thrun. 2010. Optimal trajectory generation for dynamic street scenarios in a Frenét Frame. In IEEE International Conference on Robotics and Automation, ICRA 2010, Anchorage, Alaska, USA, 3-7 May 2010. IEEE, 987–993. https://doi.org/10.1109/ROBOT.2010.5509799
[22]
Xinxin Zhang, Fei Li, and Xiangbin Wu. 2020. CSG: Critical Scenario Generation from Real Traffic Accidents. In IEEE Intelligent Vehicles Symposium, IV 2020, Las Vegas, NV, USA, October 19 - November 13, 2020. IEEE, 1330–1336. https://doi.org/10.1109/IV47402.2020.9304609
Index Terms
- The Flexcrash Platform for Testing Autonomous Vehicles in Mixed-Traffic Scenarios
Recommendations
Evaluation of ACC vehicles in mixed traffic: lane change effects and sensitivity analysis
Almost every automobile company is producing vehicles with adaptive cruise control (ACC) system onboard that enables a vehicle to do automatic vehicle following in the longitudinal direction. The ACC system is designed for driver's comfort and safety ...
Can Connected Autonomous Vehicles really improve mixed traffic efficiency in realistic scenarios?
2021 IEEE International Intelligent Transportation Systems Conference (ITSC)Connected autonomous vehicles (CAVs) can supplement the information from their own sensors with information from surrounding CAVs for decision making and control. This has the potential to improve traffic efficiency. CAVs face additional challenges in ...
Study on mixed traffic of autonomous vehicles and human-driven vehicles with different cyber interaction approaches
AbstractThe emergence of autonomous vehicles will significantly improve traffic efficiency and safety. Before the fully autonomous driving of traffic system, the mixed traffic with autonomous and human-driven vehicles will be a typical feature ...
Comments
Information & Contributors
Information
Published In
September 2024
1928 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 September 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- HORIZON EUROPE Framework Programme
- Chips Joint Undertaking
Conference
ISSTA '24
Sponsor:
ISSTA '24: 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis
September 16 - 20, 2024
Vienna, Austria
Acceptance Rates
Overall Acceptance Rate 58 of 213 submissions, 27%
Upcoming Conference
ISSTA '25
- Sponsor:
- sigsoft
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 119Total Downloads
- Downloads (Last 12 months)119
- Downloads (Last 6 weeks)23
Reflects downloads up to 03 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in