Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3365438.3410933acmconferencesArticle/Chapter ViewAbstractPublication PagesmodelsConference Proceedingsconference-collections
research-article

SceML: a graphical modeling framework for scenario-based testing of autonomous vehicles

Published: 16 October 2020 Publication History

Abstract

Ensuring the functional correctness and safety of autonomous vehicles is a major challenge for the automotive industry. However, exhaustive physical test drives are not feasible, as billions of driven kilometers would be required to obtain reliable results. Scenario-based testing is an approach to tackle this problem and reduce necessary test drives by replacing driven kilometers with simulations of relevant or interesting scenarios. These scenarios can be generated or extracted from recorded data with machine learning algorithms or created by experts. In this paper, we propose a novel graphical scenario modeling language. The graphical framework allows experts to create new scenarios or review ones designed by other experts or generated by machine learning algorithms. The scenario description is modeled as a graph and based on behavior trees. It supports different abstraction levels of scenario description during software and test development. Additionally, the graph-based structure provides modularity and reusable sub-scenarios, an important use case in scenario modeling. A graphical visualization of the scenario enhances comprehensibility for different users. The presented approach eases the scenario creation process and increases the usage of scenarios within development and testing processes.

References

[1]
Raja Ben Abdessalem, Shiva Nejati, Lionel C Briand, and Thomas Stifter. [n.d.]. Testing vision-based control systems using learnable evolutionary algorithms. In 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE) (2018). IEEE, 1016--1026.
[2]
Yasasa Abeysirigoonawardena, Florian Shkurti, and Gregory Dudek. 2019. Generating adversarial driving scenarios in high-fidelity simulators. In 2019 International Conference on Robotics and Automation (ICRA). IEEE, 8271--8277.
[3]
ASAM OpenSCENARIO. 2020. ASAM OpenSCENARIO. https://www.asam.net/standards/detail/openscenario/. [Online; accessed 17-April-2020].
[4]
Vittorio Astarita and Vincenzo Pasquale Giofré. 2019. From traffic conflict simulation to traffic crash simulation: Introducing traffic safety indicators based on the explicit simulation of potential driver errors. Simulation Modelling Practice and Theory 94 (2019), 215--236.
[5]
IPG Automotive. 2020. CarMaker: Virtual testing of automobiles and light-duty vehicles. https://ipg-automotive.com/products-services/simulation-software/carmaker/. [Online; accessed 16-April-2020].
[6]
CARLA Team. 2020. ScenarioRunner for CARLA. https://github.com/carla-simulator/scenario_runner. [Online; accessed 19-April-2020].
[7]
Michele Colledanchise and Petter Ögren. 2016. How behavior trees modularize hybrid control systems and generalize sequential behavior compositions, the subsumption architecture, and decision trees. IEEE Transactions on robotics 33, 2 (2016), 372--389.
[8]
W Damm, S Kemper, E Möhlmann, T Peikenkamp, and A Rakow. 2017. Traffic sequence charts-from visualization to semantics. Technical Report. AVACS Technical Report 117 (Oct 2017).
[9]
Microsoft Docs. 2016. Microsoft Visual Studio Visualization and Modeling SDK. https://docs.microsoft.com/en-us/visualstudio/modeling/getting-started-with-domain-specific-languages?view=vs-2019. [Online; accessed 15-April-2020].
[10]
Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. 2017. CARLA: An open urban driving simulator. arXiv preprint arXiv:1711.03938 (2017).
[11]
dSpace. 2020. dSpace - ASM Traffic. https://www.dspace.com/en/pub/home/medien/product_info/prodinf_asm_traffic.cfm. [Online; accessed 16-April-2020].
[12]
Lukas Hartjen, Robin Philipp, Fabian Schuldt, Bernhard Friedrich, and FalkHowar. 2019. Classification of Driving Maneuvers in Urban Traffic for Parametrization of Test Scenarios. In 9. Tagung Automatisiertes Fahren. Lehrstuhl für Fahrzeugtechnik mit TÜV SÜD Akademie.
[13]
Nidhi Kalra and Susan M Paddock. 2016. Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability? Transportation Research Part A: Policy and Practice 94 (2016), 182--193.
[14]
István Majzik, Oszkár Semeráth, Csaba Hajdu, Kristóf Marussy, Zoltán Szatmári, Zoltán Micskei, András Vörös, Aren A Babikian, and Dániel Varró. 2019. Towards System-Level Testing with Coverage Guarantees for Autonomous Vehicles. In 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS). IEEE, 89--94.
[15]
Ryan Marcotte and Howard J Hamilton. 2017. Behavior trees for modelling artificial intelligence in games: A tutorial. The Computer Games Journal 6, 3 (2017), 171--184.
[16]
Till Menzel, Gerrit Bagschik, and Markus Maurer. 2018. Scenarios for development, test and validation of automated vehicles. In 2018 IEEE Intelligent Vehicles Symposium (IV). IEEE, 1821--1827.
[17]
Open Autonomous Safety. [n.d.]. OAS Scenario Testing. https://oas.voyage.auto/scenarios. Accessed: 2010-09-30.
[18]
Pegasus Project. 2019. Pegasus Method - An Overview. https://www.pegasusprojekt.de/files/tmpl/Pegasus-Abschlussveranstaltung/PEGASUS-Gesamtmethode.pdf. [Online; accessed 16-April-2020].
[19]
Raphael Pfeffer, Eric Sax, and Steffen Schmidt. 2019. Realdatenbasierte simulationsgestützte Absicherung hochautomatisierter Fahrfunktionen. ATZelektronik 14, 11 (nov 2019), 24--29.
[20]
Elias Rocklage, Heiko Kraft, Abdullah Karatas, and Jörg Seewig. [n.d.]. Automated scenario generation for regression testing of autonomous vehicles. In 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC) (2017). IEEE, 476--483.
[21]
Philipp Rosenberger, Jan Timo Wendler, Martin Friedrich Holder, Clemens Linnhoff, Moritz Berghöfer, Hermann Winner, and Markus Maurer. 2019. Towards a Generally Accepted Validation Methodology for Sensor Models - Challenges, Metrics, and First Results. In Grazer Symposium Virtuelles Fahrzeug. http://tuprints.ulb.tu-darmstadt.de/8653/
[22]
VTD. 2020. VTD - Virtual Test Drive. https://vires.com/vtd-vires-virtual-test-drive/. [Online; accessed 16-April-2020].
[23]
Walther Wachenfeld, Philipp Junietz, Raphael Wenzel, and Hermann Winner. 2016. The worst-time-to-collision metric for situation identification. In 2016 IEEE Intelligent Vehicles Symposium (IV). IEEE, 729--734.
[24]
Walther Wachenfeld and Hermann Winner. 2016. The release of autonomous vehicles. In Autonomous driving. Springer, 425--449.
[25]
Carsten Wiecher, Joel Greenyer, and Jan Korte. 2019. Test-Driven Scenario Specification of Automotive Software Components. In 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C). IEEE, 12--17.
[26]
Mathew Wood, David Wittmann, Siyuan Liu, Christian Knobel, Sandro Syguda, Thomas Wiltschko, Neil Garbacik, Michael O'Brien, Udo Dannebaum, Jack Weast, and Bernd Dorniede. 2019. Safety first for automated driving. https://www.daimler.com/documents/innovation/other/safety-first-for-automated-driving.pdf
[27]
Ansar-Ul-Haque Yasar, Yolande Berbers, and Davy Preuveneers. 2008. A computational analysis of driving variations on distributed multiuser driving simulators. In IASTED International Conference on Modelling and Simulation. ACTA Press; Canada, 178--186.

Cited By

View all
  • (2024)Concretization of Abstract Traffic Scene Specifications Using Metaheuristic SearchIEEE Transactions on Software Engineering10.1109/TSE.2023.333125450:1(48-68)Online publication date: 1-Jan-2024
  • (2023)A Survey on Automated Driving System Testing: Landscapes and TrendsACM Transactions on Software Engineering and Methodology10.1145/357964232:5(1-62)Online publication date: 24-Jul-2023
  • (2022)MoDLFProceedings of the 25th International Conference on Model Driven Engineering Languages and Systems10.1145/3550355.3552453(187-198)Online publication date: 23-Oct-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MODELS '20: Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems
October 2020
406 pages
ISBN:9781450370196
DOI:10.1145/3365438
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

In-Cooperation

  • IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 October 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. automotive software engineering
  2. graph
  3. graphical modeling language
  4. modularity
  5. scenario-based
  6. testing

Qualifiers

  • Research-article

Conference

MODELS '20
Sponsor:

Acceptance Rates

MODELS '20 Paper Acceptance Rate 35 of 127 submissions, 28%;
Overall Acceptance Rate 118 of 382 submissions, 31%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)117
  • Downloads (Last 6 weeks)8
Reflects downloads up to 30 Aug 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Concretization of Abstract Traffic Scene Specifications Using Metaheuristic SearchIEEE Transactions on Software Engineering10.1109/TSE.2023.333125450:1(48-68)Online publication date: 1-Jan-2024
  • (2023)A Survey on Automated Driving System Testing: Landscapes and TrendsACM Transactions on Software Engineering and Methodology10.1145/357964232:5(1-62)Online publication date: 24-Jul-2023
  • (2022)MoDLFProceedings of the 25th International Conference on Model Driven Engineering Languages and Systems10.1145/3550355.3552453(187-198)Online publication date: 23-Oct-2022
  • (2022)Requirements engineering for autonomous vehiclesProceedings of the 37th ACM/SIGAPP Symposium on Applied Computing10.1145/3477314.3507004(1299-1308)Online publication date: 25-Apr-2022

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media