From TARA to Test: Automated Automotive Cybersecurity Test Generation Out of Threat Modeling
Authors: 
Stefan F Marksteiner, Christoph Schmittner, Korbinian Christl, Dejan Nickovic, Mikael Sjödin, Marjan SirjaniAuthors Info & Claims
Stefan F Marksteiner, Christoph Schmittner, Korbinian Christl, Dejan Nickovic, Mikael Sjödin, Marjan SirjaniAuthors Info & ClaimsArticle No.: 5, Pages 1 - 10
Abstract
The United Nations Economic Commission for Europe (UNECE) demands the management of cyber security risks in vehicle design and that the effectiveness of these measures is verified by testing. Generally, with rising complexity and openness of systems via software-defined vehicles, verification through testing becomes a very important for security assurance. This mandates the introduction of industrial-grade cybersecurity testing in automotive development processes. Currently, the automotive cybersecurity testing procedures are not specified or automated enough to be able to deliver tests in the amount and thoroughness needed to keep up with that regulation, let alone doing so in a cost-efficient manner. This paper presents a methodology to automatically generate technology-agnostic test scenarios from the results of threat analysis and risk assessment (TARA) process. Our approach is to transfer the resulting threat models into attack trees and label their edges using actions from a domain-specific language (DSL) for attack descriptions. This results in a labelled transitions system (LTS), in which every labelled path intrinsically forms a test scenario. In addition, we include the concept of Cybersecurity Assurance Levels (CALs) and Targeted Attack Feasibility (TAF) into testing by assigning them as costs to the attack path. This abstract test scenario can be compiled into a concrete test case by augmenting it with implementation details. Therefore, the efficacy of the measures taken because of the TARA can be verified and documented. As TARA is a de-facto mandatory step in the UNECE regulation and the relevant ISO standard, automatic test generation (also mandatory) out of it could mean a significant improvement in efficiency, as two steps could be done at once.
References
[1]
Amenaza Technologies Limited. 2023. SecurITree. Online. https://www.amenaza.com Accessed: 2023-10-03.
[2]
Paul Ammann, Duminda Wijesekera, and Saket Kaushik. 2002. Scalable, Graph-Based Network Vulnerability Analysis. In Proceedings of the 9th ACM Conference on Computer and Communications Security. ACM, New York, NY, USA, 217–224.
[3]
Jeremy Bryans, Hoang Nga Nguyen, and Siraj Ahmed Shaikh. 2019-01. Attack Defense Trees with Sequential Conjunction. In 2019 IEEE 19th International Symposium on High Assurance Systems Engineering (HASE). IEEE, Hangzhou, China, 247–252. https://doi.org/10.1109/HASE.2019.00045
[4]
Madeline Cheah, Hoang Nga Nguyen, Jeremy Bryans, and Siraj A. Shaikh. 2018. Formalising Systematic Security Evaluations Using Attack Trees for Automotive Applications. In Information Security Theory and Practice, Gerhard P. Hancke and Ernesto Damiani (Eds.). Vol. 10741. Springer International Publishing, Cham, 113–129. https://doi.org/10.1007/978-3-319-93524-9_7 Series Title: Lecture Notes in Computer Science.
[5]
Sebastian Chlup, Korbinian Christl, Christoph Schmittner, Abdelkader Magdy Shaaban, Stefan Schauer, and Martin Latzenhofer. 2023. THREATGET: Towards Automated Attack Tree Analysis for Automotive Cybersecurity. Inf. 14, 1 (2023), 14. https://doi.org/10.3390/info14010014
[6]
Korbinian Christl and Thorsten Tarrach. 2021. The analysis approach of ThreatGet. CoRR abs/2107.09986 (2021), 57 pages. arXiv:2107.09986https://arxiv.org/abs/2107.09986
[7]
Frédéric Cuppens and Rodolphe Ortalo. 2000. Lambda: A Language to Model a Database for Detection of Attacks. In International Workshop on Recent Advances in Intrusion Detection. Springer, Berlin, Heidelberg, 197–216.
[8]
Dag Eng. 2017. Integrated Threat Modelling. Master’s thesis. University of Olso.
[9]
Foreseeti AB. 2020. Foreseeti. Online. https://foreseeti.com/ Accessed: 2020-11-29.
[10]
Md. Shariful Haque and Travis Atkison. 2017. An Evolutionary Approach of Attack Graph to Attack Tree Conversion. International Journal of Computer Network and Information Security 9, 11 (Nov. 2017), 1–16. https://doi.org/10.5815/ijcnis.2017.11.01
[11]
Terrance R Ingoldsby. 2021. Attack Tree-Based Threat Risk Analysis. Technical Report. Amenaza Technologies Limited.
[12]
International Organization for Standardization. 2022. Information Security, Cybersecurity and Privacy Protection – Evaluation Criteria for IT Security – Part 2: Security Functional Components. ISO/IEC Standard 15408-2:2022. International Organization for Standardization.
[13]
International Organization for Standardization and Society of Automotive Engineers. 2021. Road Vehicles – Cybersecurity Engineering. ISO/SAE Standard "21434". International Organization for Standardization.
[14]
International Organization for Standardization and Society of Automotive Engineers. 2022. ISO/SAE PAS8475 (WIP) Road Vehicles – Cybersecurity Assurance Levels and Targeted Attack Feasibility - SAE International. https://www.sae.org/standards/content/iso/sae%20pas8475/.
[15]
International Organization for Standardization and Society of Automotive Engineers. 2023. ISO/SAE PAS8477 (WIP) Road Vehicles - Cybersecurity Verification and Validation - SAE International. https://www.sae.org/standards/content/iso/sae%20pas8477/.
[16]
Isograph. 2023. Isograph AttackTree. Online. https://www.isograph.com/software/attacktree/ Accessed: 2023-10-03.
[17]
Robert M. Keller. 1976. Formal Verification of Parallel Programs. Commun. ACM 19, 7 (July 1976), 371–384. https://doi.org/10.1145/360248.360251
[18]
Rafiullah Khan, Kieran McLaughlin, David Laverty, and Sakir Sezer. 2017. STRIDE-based threat modeling for cyber-physical systems. In 2017 IEEE PES Innovative Smart Grid Technologies Conference Europe (ISGT-Europe). IEEE, New York, NY, 1–6.
[19]
Barbara Kordy, Sjouke Mauw, Saša Radomirović, and Patrick Schweitzer. 2011. Foundations of Attack–Defense Trees. In Formal Aspects of Security and Trust, Pierpaolo Degano, Sandro Etalle, and Joshua Guttman (Eds.). Vol. 6561. Springer Berlin Heidelberg, Berlin, Heidelberg, 80–95. https://doi.org/10.1007/978-3-642-19751-2_6 Series Title: Lecture Notes in Computer Science.
[20]
D Richard Kuhn, Raghu N Kacker, and Yu Lei. 2010. Practical Combinatorial Testing. SP 800-142. National Institute of Standards and Technology.
[21]
Harjinder Singh Lallie, Kurt Debattista, and Jay Bal. 2020. A Review of Attack Graph and Attack Tree Visual Syntax in Cyber Security. Computer Science Review 35 (Feb. 2020), 100219. https://doi.org/10.1016/j.cosrev.2019.100219
[22]
Georg Macher, Harald Sporer, Reinhard Berlach, Eric Armengaud, and Christian Kreiner. 2015. SAHARA: A Security-Aware Hazard and Risk Analysis Method. In 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, Grenoble, France, 621–624. https://doi.org/10.7873/DATE.2015.0622
[23]
Stefan Marksteiner, Nadja Marko, Andre Smulders, Stelios Karagiannis, Florian Stahl, Hayk Hamazaryan, Rupert Schlick, Stefan Kraxberger, and Alexandr Vasenev. 2021. A Process to Facilitate Automated Automotive Cybersecurity Testing. In 2021 IEEE 93rd Vehicular Technology Conference (VTC Spring). IEEE, New York, NY, USA, 1–7.
[24]
Sjouke Mauw and Martijn Oostdijk. 2005. Foundations of Attack Trees. In Information Security and Cryptology - ICISC 2005, Dong Ho Won and Seungjoo Kim (Eds.). Vol. 3935. Springer Berlin Heidelberg, Berlin, Heidelberg, 186–198. https://doi.org/10.1007/11734727_17
[25]
C. C. Michael, Ken van Wyk, and Will Radosevich. 2005. Risk-Based and Functional Security Testing. Technical Report. U.S. Deparmtent of Homeland Security.
[26]
Cédric Michel and Ludovic Mé. 2001. ADeLe: An Attack Description Language for Knowledge-Based Intrusion Detection. In Trusted Information(IFIP International Federation for Information Processing), Michel Dupuy and Pierre Paradinas (Eds.). Springer US, Boston, MA, 353–368. https://doi.org/10.1007/0-306-46998-7_25
[27]
Carl Adam Petri. 1962. Kommunikation mit Automaten. Ph. D. Dissertation. Technische Universität Darmstadt.
[28]
Cynthia Phillips and Laura Painton Swiler. 1998. A Graph-Based System for Network-Vulnerability Analysis. In Proceedings of the 1998 Workshop on New Security Paradigms. ACM, New York, NY, USA, 71–79.
[29]
Magdy El Sadany, Christoph Schmittner, and Wolfgang Kastner. 2019. Assuring Compliance with Protection Profiles with ThreatGet. In SAFECOMP 2019 Workshops(Lecture Notes in Computer Science). Springer, Berlin, 62–73.
[30]
Christoph Schmittner, Bernhard Schrammel, and Sandra König. 2021. Asset Driven ISO/SAE 21434 Compliant Automotive Cybersecurity Analysis with ThreatGet. In Systems, Software and Services Process Improvement(Communications in Computer and Information Science), Murat Yilmaz, Paul Clarke, Richard Messnarz, and Michael Reiner (Eds.). Springer International Publishing, Cham, 548–563. https://doi.org/10.1007/978-3-030-85521-5_36
[31]
Bruce Schneier. 1999. Attack Trees. Dr. Dobb’s journal 24, 12 (1999), 21–29.
[32]
Raivo Sell, Mairo Leier, Anton Rassõlkin, and Juhan-Peep Ernits. 2020. Autonomous Last Mile Shuttle ISEAUTO for Education and Research. International Journal of Artificial Intelligence and Machine Learning 10, 1 (Jan. 2020), 18–30. https://doi.org/10.4018/IJAIML.2020010102
[33]
Adam Shostack. 2014. Threat Modeling: Designing for Security. John Wiley & Sons, Indianaplois, IN.
[34]
Tutamantic Ltd.2020. Tutamen Threat Model Automator. Online. https://www.tutamantic.com/ Accessed: 2020-11-29.
[35]
United Nations Economic and Social Council - Economic Commission for Europe. 2020. UN Regulation on Uniform Provisions Concerning the Approval of Vehicles with Regard to Cyber Security and of Their Cybersecurity Management Systems. Technical Report ECE/TRANS/WP.29/2020/79. United Nations Economic and Social Council - Economic Commission for Europe / United Nations Economic and Social Council - Economic Commission for Europe, Brussels.
[36]
Upstream Security. 2020. Upstream Security Global Automotive Cybersecurity Report. Technical Report. Upstream Security.
[37]
David Ward, Ireri Ibarra, and Alastair Ruddle. 2013. Threat Analysis and Risk Assessment in Automotive Cyber Security. SAE International Journal of Passenger Cars-Electronic and Electrical Systems 6, 2013-01-1415 (2013), 507–513.
[38]
Jan Was, Pooja Avhad, Matthew Coles, Nick Ozmore, Rohit Shambhuni, and Izar Tarandach. 2020. OWASP pytm. Online. https://owasp.org/www-project-pytm/ Accessed: 2020-11-29.
[39]
Christian Wolschke, Stefan Marksteiner, Tobias Braun, and Markus Wolf. 2021. An Agnostic Domain Specific Language for Implementing Attacks in an Automotive Use Case. In The 16th International Conference on Availability, Reliability and Security(ARES 2021). Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3465481.3470070
[40]
Mark Yampolskiy, Péter Horváth, Xenofon D. Koutsoukos, Yuan Xue, and Janos Sztipanovits. 2015. A Language for Describing Attacks on Cyber-Physical Systems. International Journal of Critical Infrastructure Protection 8 (Jan. 2015), 40–52. https://doi.org/10.1016/j.ijcip.2014.09.003
Index Terms
- From TARA to Test: Automated Automotive Cybersecurity Test Generation Out of Threat Modeling
Recommendations
Automated Security Test Generation with Formal Threat Models
Security attacks typically result from unintended behaviors or invalid inputs. Security testing is labor intensive because a real-world program usually has too many invalid inputs. It is highly desirable to automate or partially automate security-...
Threat led advanced persistent threat penetration test
Cyber security attacks have been on the rise in recent years. One of the most destructive attacks are known as advanced persistent threat (APT) attacks which can inflict massive damages to a network. A common approach of testing the security of an IT ...
Automated test-case generation by cloning
AST '12: Proceedings of the 7th International Workshop on Automation of Software TestTest cases are often similar. A preliminary study of eight open-source projects found that on average at least 8% of all test cases are clones; the maximum found was 42%. The clones are not identical with their originals -- identifiers of classes, ...
Comments
Information & Contributors
Information
Published In
December 2023
104 pages
ISBN:9798400704543
DOI:10.1145/3631204
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 05 December 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- ECSEL Joint Undertaking
Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 560Total Downloads
- Downloads (Last 12 months)560
- Downloads (Last 6 weeks)101
Reflects downloads up to 10 Aug 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in