research-article

Fault Injection for Test-Driven Development of Robust SoC Firmware

Authors:

Petra R. Maier,

Veit B. Kleeberger,

Daniel Mueller-Gritschneder,

Ulf SchlichtmannAuthors Info & Claims

ACM Transactions on Embedded Computing Systems (TECS), Volume 17, Issue 1

Article No.: 19, Pages 1 - 26

https://doi.org/10.1145/3092943

Published: 06 December 2017 Publication History

Abstract

Robustness against errors in hardware must be considered from the very beginning of safety-critical system-on-chip firmware design. Therefore, we present fault injection for test-driven development (TDD) of robust firmware. As TDD is based on instant feedback to the designer, fault injection must execute within few minutes. In contrast to state-of-the-art approaches, we avoid long simulation scenarios and runtimes by injecting faults at the unit level and utilizing host-compiled simulation. Further, three static bit-level analyses of firmware source code and hardware specification reduce the fault set significantly. This accelerates fault injection by several orders of magnitude and enables robustness-aware TDD.

References

[1]

Samar Abdi. 2010. Automatic generation of host-compiled timed TLMs for high level design. In High Level Design Validation and Test Workshop (HLDVT’10).

[2]

David Abrahams and Ralf W. Grosse-Kunstleve. 2003. Building hybrid systems with boost.python. C/C++ Users J. 21, 7 (2003).

[3]

Alfred V. Aho, Monica S. Lam, Ravi Sethi, and Jeffrey D. Ullman. 2007. Compilers: Principles, Techniques and Tools (2 ed.). Pearson Education.

Digital Library

[4]

Raul Barbosa, Jonny Vinter, Peter Folkesson, and Johan Karlsson. 2005. Assembly-level pre-injection analysis for improving fault injection efficiency. In Dependable Computing-EDCC 5. Springer.

Digital Library

[5]

Alfredo Benso, Maurizio Rebaudengo, Leonardo Impagliazzo, and Pietro Marmo. 1998. Fault-list collapsing for fault-injection experiments. In Reliability and Maintainability Symposium (RAMS’98).

[6]

Suhas Chakravarty, Zhuoran Zhao, and Andreas Gerstlauer. 2013. Automated, retargetable back-annotation for host compiled performance and power modeling. In Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS’13).

Digital Library

[7]

Steve Chamberlain. 1991. Libbfd, The Binary File Descriptor Library. Free Software Foundation.

[8]

Hyungmin Cho, Shahrzad Mirkhani, Chen-Yong Cher, Jacob A. Abraham, and Subhasish Mitra. 2013. Quantitative evaluation of soft error injection techniques for robust system design. In Design Automation Conference (DAC’13). ACM.

Digital Library

[9]

Clang. 2008. Clang: a C language family frontend for LLVM. (May 2008). Retrieved from http://clang.llvm.org.

[10]

Björn Döbel, Hermann Härtig, and Michael Engel. 2012. Operating system support for redundant multithreading. In Conference on Embedded Software (EMSOFT’12).

Digital Library

[11]

Joan G. Dyer, Mark Lindemann, Ronald Perez, Reiner Sailer, Leendert van Doorn, and Sean W. Smith. 2001. Building the IBM 4758 secure coprocessor. Computer 34, 10 (2001).

Digital Library

[12]

Mojtaba Ebrahimi, Nour Sayed, Maryam Rashvand, and Mehdi B. Tahoori. 2015. Fault injection acceleration by architectural importance sampling. In Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS’15).

Digital Library

[13]

Freescale Semiconductors. 2013. Qorivva MPC5643L microcontroller data sheet. (2013).

[14]

Darshan Gandhi, Andreas Gerstlauer, and Lidiya John. 2014. FastSpot: Host-compiled thermal estimation for early design space exploration. In International Symposium on Quality Electronic Design (ISQED’14).

[15]

Georg Georgakos, Ulf Schlichtmann, Reinhard Schneider, and Samarjit Chakraborty. 2013. Reliability challenges for electric vehicles: From devices to architecture and systems software. In Design Automation Conference (DAC’13).

Digital Library

[16]

Simon Graham. 2004. Writing drivers for reliability, robustness and fault tolerant systems. Retrieved from https://download.microsoft.com/download/5/b/5/5b5bec17-ea71-4653-9539-204a672f11cf/FTdrv.doc.

[17]

James W. Grenning. 2011. Test-driven Development for Embedded C. Pragmatic Bookshelf.

[18]

Paul Hamill. 2004. Unit Test Frameworks: Tools for High-quality Software Development. O’Reilly Media, Inc.

Digital Library

[19]

Ali Hayek and Josef Börcsök. 2014. Safety chips in light of the standard IEC 61508: survey and analysis. In International Symposium on Fundamentals of Electrical Engineering (ISFEE’14).

[20]

Carles Hernandez and Jaume Abella. 2015. Timely error detection for effective recovery in light-lockstep automotive systems. Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD’15) 34, 11 (Nov. 2015).

[21]

Andrea Höller, Georg Macher, Tobias Rauter, Johannes Iber, and Christian Kreiner. 2015. A virtual fault injection framework for reliability-aware software development. In Dependable Systems and Networks Workshops (DSN-W’15).

Digital Library

[22]

IP-XACT. 2014. IEEE standard 1685-2014 for IP-XACT. (2014).

[23]

ISO 26262. 2011. Road Vehicles—Functional Safety. International Organization for Standardization (ISO), Geneva, Switzerland.

[24]

Yue Jia and Mark Harman. 2011. An analysis and survey of the development of mutation testing. Trans. Softw. Eng. 37, 5 (2011).

Digital Library

[25]

Asim Kadav, Matthew J. Renzelmann, and Michael M. Swift. 2009. Tolerating hardware device failures in software. In Symposium on Operating Systems Principles (SOSP’09).

Digital Library

[26]

David Kammler, Junqing Guan, Gerd Ascheid, Rainer Leupers, and Heinrich Meyr. 2009. A fast and flexible platform for fault injection and evaluation in Verilog-based simulations. In Conference on Secure Software Integration and Reliability Improvement (SSIRI).

Digital Library

[27]

Michael Kerrisk. 2016. Linux Programmer’s Manual: Backtrace. Retrieved from http://man7.org/linux/man-pages/man3/backtrace_symbols.3.html.

[28]

Veit B. Kleeberger, Daniel Mueller-Gritschneder, and Ulf Schlichtmann. 2013. Technology-aware system failure analysis in the presence of soft errors by mixture importance sampling. In International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFT’13).

[29]

Veit B. Kleeberger, Stefan Rutkowski, and Ruth Coppens. 2015. Design 8 verification of automotive SoC firmware. In Design Automation Conference (DAC’15).

Digital Library

[30]

Kun Lu, Daniel Müller-Gritschneder, and Ulf Schlichtmann. 2012. Accurately timed transaction level models for virtual prototyping at high abstraction level. In Conference on Design, Automation and Test in Europe (DATE’12).

Digital Library

[31]

Lech Madeyski. 2010. Test-driven Development: An Empirical Evaluation of Agile Practice. Springer-Verlag Berlin.

[32]

Petra R. Maier, Veit Kleeberger, Daniel Mueller-Gritschneder, and Ulf Schlichtmann. 2016a. Fault injection at host-compiled level with static fault set reduction for SoC firmware robustness testing. In Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS’16).

Digital Library

[33]

Petra R. Maier, Daniel Müller-Gritschneder, Ulf Schlichtmann, and Veit B. Kleeberger. 2016b. Embedded software reliability testing by unit-level fault injection. In Asia and South Pacific Design Automation Conference (ASP-DAC’16).

[34]

MISRA-C. 2012. Guidelines for the Use of the C Language in Critical Systems. Motor Industry Software Reliability Association.

[35]

Oracle. 2010. Oracle Solaris 10 Software Developer Documentation - Writing Device Drivers. Retrieved from https://docs.oracle.com/cd/E18752_01/pdf/816-4854.pdf.

[36]

Diego Rodrigues, Ghazaleh Nazarian, Álvaro Moreira, Luigi Carro, and Georgi Gaydadjiev. 2015. A non-conservative software-based approach for detecting illegal CFEs caused by transient faults. In Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFTS’15).

[37]

Kenneth H. Rosen. 2003. Discrete Mathematics and Its Applications (5th ed.). McGraw--Hill Science.

[38]

Stefan Stattelmann, Gernot Gebhard, Christoph Cullmann, Oliver Bringmann, and Wolfgang Rosenstiel. 2012. Hybrid source-level simulation of data caches using abstract cache models. In Conference on Design, Automation and Test in Europe (DATE’12).

Digital Library

[39]

Thomas Stober and Uwe Hansmann. 2010. Agile Software Development: Best Practices for Large Software Development Projects. Springer-Verlag, Berlin.

Digital Library

[40]

Gary Stringham. 2009. Hardware/Firmware Interface Design: Best Practices for Improving Embedded Systems Development. Newnes.

Digital Library

[41]

Jun Sun, Wanghong Yuan, Mahesh Kallahalla, and Nayeem Islam. 2005. HAIL: A language for easy and correct device access. In Conference on Embedded Software (EMSOFT’05).

Digital Library

[42]

Unity. 2015. UNITY: Unit Testing for C (especially Embedded Software). Retrieved from http://www.throwtheswitch.org/unity/.

[43]

Zheng Wang, Chao Chen, and Anupam Chattopadhyay. 2013. Fast reliability exploration for embedded processors via high-level fault injection. In International Symposium on Quality Electronic Design (ISQED’13).

[44]

Vincent Zimmer, Jiming Sun, Marc Jones, and Stefan Reinauer. 2015. Embedded Firmware Solutions: Development Best Practices for the Internet of Things. Apress.

Digital Library

Cited By

Gao XZhang CLiu JZhang JYan Y(2024)Intelligent Simulation Platform for Fault Injection to Configuration Bitstream based on Multiple Cores Architecture Processor2024 20th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD)10.1109/ICNC-FSKD64080.2024.10702211(1-6)Online publication date: 27-Jul-2024
https://doi.org/10.1109/ICNC-FSKD64080.2024.10702211
Hsieh CWang Z(2021)A Two-stage Method of Synchronization Prediction Framework in TDDArabian Journal for Science and Engineering10.1007/s13369-021-06207-0Online publication date: 30-Sep-2021
https://doi.org/10.1007/s13369-021-06207-0
Werner MDevarajegowda KChaari MEcker W(2019)Increasing Soft Error Resilience by Software TransformationProceedings of the 56th Annual Design Automation Conference 201910.1145/3316781.3323479(1-4)Online publication date: 2-Jun-2019
https://dl.acm.org/doi/10.1145/3316781.3323479

Index Terms

Fault Injection for Test-Driven Development of Robust SoC Firmware

Recommendations

Fault injection at host-compiled level with static fault set reduction for SoC firmware robustness testing
CODES '16: Proceedings of the Eleventh IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis

Decreasing hardware reliability makes robust firmware imperative for safety-critical applications. Hence, ensuring correct handling of errors in peripherals is a key objective during firmware design. To adequately support robustness considerations of ...
Software-Implemented Fault Injection at Firmware Level
DEPEND '10: Proceedings of the 2010 Third International Conference on Dependability

Software-implemented fault injection is an established method to emulate hardware faults in computer systems. Existing approaches typically extend the operating system by special drivers or change the application under test. We propose a novel approach ...
Fault Injection and Dependability Evaluation of Fault-Tolerant Systems

The authors describe a dependability evaluation method based on fault injection that establishes the link between the experimental evaluation of the fault tolerance process and the fault occurrence process. The main characteristics of a fault injection ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Embedded Computing Systems

ACM Transactions on Embedded Computing Systems Volume 17, Issue 1

Special Issue on Autonomous Battery-Free Sensing and Communication, Special Issue on ESWEEK 2016 and Regular Papers

January 2018

630 pages

ISSN:1539-9087

EISSN:1558-3465

DOI:10.1145/3136518

Editor:
Sandeep K. Shukla
Indian Institute of Technology, India

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 06 December 2017

Accepted: 01 May 2017

Revised: 01 May 2017

Received: 01 December 2016

Published in TECS Volume 17, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

German Federal Ministry of Education and Research (BMBF) in the project EffektiV

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
299
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gao XZhang CLiu JZhang JYan Y(2024)Intelligent Simulation Platform for Fault Injection to Configuration Bitstream based on Multiple Cores Architecture Processor2024 20th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD)10.1109/ICNC-FSKD64080.2024.10702211(1-6)Online publication date: 27-Jul-2024
https://doi.org/10.1109/ICNC-FSKD64080.2024.10702211
Hsieh CWang Z(2021)A Two-stage Method of Synchronization Prediction Framework in TDDArabian Journal for Science and Engineering10.1007/s13369-021-06207-0Online publication date: 30-Sep-2021
https://doi.org/10.1007/s13369-021-06207-0
Werner MDevarajegowda KChaari MEcker W(2019)Increasing Soft Error Resilience by Software TransformationProceedings of the 56th Annual Design Automation Conference 201910.1145/3316781.3323479(1-4)Online publication date: 2-Jun-2019
https://dl.acm.org/doi/10.1145/3316781.3323479
Kleeberger PRivera JMueller-Gritschneder DSchlichtmann UShibuya T(2019)SeRoHALProceedings of the 24th Asia and South Pacific Design Automation Conference10.1145/3287624.3287652(33-38)Online publication date: 21-Jan-2019
https://dl.acm.org/doi/10.1145/3287624.3287652

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents