research-article

BOSMI: a framework for non-intrusive monitoring and testing of embedded multithreaded software on the logical level

Authors:

Matthias Jung, and

Norbert WehnAuthors Info & Claims

SAMOS '18: Proceedings of the 18th International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation

July 2018

Pages 131 - 138

https://doi.org/10.1145/3229631.3229641

Published: 15 July 2018 Publication History

Abstract

Traditional software testing methods are inefficient in cases where data inputs alone do not determine the outcome of a program's execution. In order to verify such software, testing is often complemented by analysis of the execution trace. For monitoring the execution trace, most approaches today insert additional instructions at the binary level, making the monitoring intrusive. Binary instrumentation operate on a low level, making it difficult to properly modify a program's states and to quantify its code coverage.

In this paper, we present a framework for testing complex embedded multithreaded software on the logical level. Testing software on this level avoids dependency on concrete compilers and relates the execution to the source code, thus enabling coverage. Our non-intrusive execution monitoring and control is implemented using the LLVM interpreter compiler infrastructure. Instead of forcing thread interleaving, we suggest simulating interleaving effects through non-intrusive changes of shared variables. This makes it possible to test a single thread without executing the full software stack, which is especially useful in situations where the full software stack is not available (e.g., pre-integration testing). We complement existing approaches with new features such as dynamic configuration of monitoring and execution rollback to the checkpoints. Our approach introduces acceptable overhead without any complex setup.

References

[1]

Apache-commons. 2017. The Byte Code Engineering Library (Apache Commons BCEL). (2017). http://jakarta.apache.org/bcel/

[2]

Matthias Blasing. 2017. Java Native Access (JNA). (2017). Retrieved November 05, 2017 from https://github.com/java-native-access/jna

[3]

B. Alpern et al. 2000. The Jalapeño virtual machine. IBM Systems Journal 39, 1 (2000), 211--238.

Digital Library

[4]

C.Luk et al. 2005. Pin: building customized program analysis tools with dynamic instrumentation. In Conference on Programming language design and implementation (PLDI). ACM, Chicago, IL, USA.

Digital Library

[5]

C. Wang et al. 2011. Coverage guided systematic concurrency testing. In International Conference on Software Engineering (ICSE '11). ACM, Waikiki, Honolulu, HI, USA, 221--230. https://dl.acm.org/citation.cfm?id=1985824

Digital Library

[6]

G. C. Necula et al. 2002. CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs. In Compiler Construction (CC '02). Springer-Verlag, London, UK, 213--228.

Digital Library

[7]

J. Jahic et al. 2014. Automatic Test Coverage Measurements to support Design Space Exploration. In Design Space Exploration of Cyber-Physical Systems (IDEAL). Springer-Verlag, Berlin, Germany.

[8]

J. Jahic et al. 2017. Supervised Testing of Concurrent Software in Embedded Systems. In International Conference on Embedded Computer Systems: Architectures, Modeling and Simulation (SAMOS XVII). IEEE, Samos Island, Greece.

[9]

R. Agarwal et al. 2005. Optimized run-time race detection and atomicity checking using partial discovered types. In Automated software engineering (ASE '05). ACM, Long Beach, CA, USA, 233--242.

Digital Library

[10]

R. Vallée-Rai et al. 1999. Soot - a Java bytecode optimization framework. In Centre for Advanced Studies on Collaborative research (CASCON '99). IBM Press, Mississauga, Ontario, Canada, 13.

Digital Library

[11]

S. Hong et al. 2012. Testing concurrent programs to achieve high synchronization coverage. In Software Testing and Analysis (ISSTA 2012). ACM, Minneapolis, MN, USA, 210--220.

Digital Library

[12]

S. Savage et al. 1997. Eraser: a dynamic data race detector for multithreaded programs. ACM Transactions on Computer Systems (TOCS) 15, 4 (July 1997), 391--411.

Digital Library

[13]

T. Kuhn et al. 2013. FERAL - Framework for simulator coupling on requirements and architecture level. In Formal Methods and Models for Codesign (MEMOCODE). IEEE, Portland, OR, USA.

[14]

Z. Zhao et al. 2017. Network/System Co-Simulation for Design Space Exploration of IoT Applications. In International Conference on Embedded Computer Systems: Architectures, Modeling and Simulation (SAMOS XVII). IEEE, Samos Island, Greece.

[15]

A. Goldberg and K. Havelund. 2003. Instrumentation of Java Bytecode for Runtime Analysis. Technical Reports from ETH Zurich 408. ETH Zurich, Zurich, Switzerland.

[16]

Helgrind. 2012. Valgrind-project., Helgrind: a data-race detector. (2012). Retrieved 2007 from http://valgrind.org/docs/manual/hgmanual.html

[17]

Hayhurst K. J. 2001. A Practical Tutorial on Modified Condition/Decision Coverage. Technical Report. NASA, NASA Langley Technical Report Server.

Digital Library

[18]

J. Jahic and T. Kuhn. 2014. Analysis of Functional Software Dependencies through Supervised Execution. In Software Reliability Engineering Workshops (ISSREW). IEEE, Naples, Italy.

Digital Library

[19]

L. Lamport. 1978. Time, clocks, and the ordering of events in a distributed system. Commun. ACM 21, 7 (July 1978), 558--565.

Digital Library

[20]

C. Lattner and V. Adve. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In International symposium on Code generation and optimization: feedback-directed and runtime optimization (CGO '04). IEEE, San Jose, CA, USA, 75.

Digital Library

[21]

W. Kunz M. Ammar Ben Khadra, D. Stoffel. 2016. Speculative disassembly of binary code. In Compilers, Architectures and Synthesis for Embedded Systems (CASES). ACM, Pittsburgh, Pennsylvania.

Digital Library

[22]

Xiaozhu Meng and Barton P. Miller. 2016. Binary Code is Not Easy. In Proceedings of the 25th Symposium on Software Testing and Analysis (ISSTA). ACM, New York, NY, USA, 24--35.

Digital Library

[23]

N. Nethercote and J. Seward. 2007. Valgrind: a framework for heavyweight dynamic binary instrumentation. In Conference on Programming Language Design and Implementation (PLDI). ACM, San Diego, California, USA.

Digital Library

[24]

Y. W. Song and Y.-H. Lee. 2014. On the existence of probe effect in multi-threaded embedded programs. In International Conference on Embedded Software (EMSOFT). IEEE, Jaypee Greens, India.

Digital Library

[25]

S. C. Woo. 1995. The SPLASH-2 programs: characterization and methodological considerations. In Computer architecture (ISCA '95). ACM SIGARCH Computer Architecture News, S. Margherita Ligure, Italy, 24--36.

Digital Library

Cited By

Jahić JDoganci VGehring H(2022)CASA: An Approach for Exposing and Documenting Concurrency-Related Software PropertiesEmbedded Computer Systems: Architectures, Modeling, and Simulation10.1007/978-3-031-15074-6_9(139-154)Online publication date: 14-Aug-2022
https://doi.org/10.1007/978-3-031-15074-6_9
Cornaglia AViehl ABringmann O(2022)Accurate LLVM IR to Binary CFGs Mapping for Simulation of Optimized Embedded SoftwareEmbedded Computer Systems: Architectures, Modeling, and Simulation10.1007/978-3-031-04580-6_1(3-15)Online publication date: 27-Apr-2022
https://doi.org/10.1007/978-3-031-04580-6_1
Jahić JBauer TKuhn TWehn NAntonino P(2020)FERA: A Framework for Critical Assessment of Execution Monitoring Based Approaches for Finding Concurrency BugsIntelligent Computing10.1007/978-3-030-52249-0_5(54-74)Online publication date: 4-Jul-2020
https://doi.org/10.1007/978-3-030-52249-0_5

Index Terms

BOSMI: a framework for non-intrusive monitoring and testing of embedded multithreaded software on the logical level
1. Computer systems organization
  1. Architectures
    1. Parallel architectures
      1. Multicore architectures
  2. Embedded and cyber-physical systems
    1. Embedded systems
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software organization and properties
    1. Contextual software domains
      1. Software infrastructure
        Interpreters

Recommendations

Maple: a coverage-driven testing tool for multithreaded programs
OOPSLA '12

Testing multithreaded programs is a hard problem, because it is challenging to expose those rare interleavings that can trigger a concurrency bug. We propose a new thread interleaving coverage-driven testing tool called Maple that seeks to expose ...
Read More
Maple: a coverage-driven testing tool for multithreaded programs
OOPSLA '12: Proceedings of the ACM international conference on Object oriented programming systems languages and applications

Testing multithreaded programs is a hard problem, because it is challenging to expose those rare interleavings that can trigger a concurrency bug. We propose a new thread interleaving coverage-driven testing tool called Maple that seeks to expose ...
Read More
Efficient detection of thread safety violations via coverage-guided generation of concurrent tests
ICSE '17: Proceedings of the 39th International Conference on Software Engineering

As writing concurrent programs is challenging, developers often rely on thread-safe classes, which encapsulate most synchronization issues. Testing such classes is crucial to ensure the correctness of concurrent programs. An effective approach to ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

SAMOS '18: Proceedings of the 18th International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation

July 2018

263 pages

ISBN:9781450364942

DOI:10.1145/3229631

General Chair:
Trevor Mudge
University of Michigan - Ann Arbor
,
Program Chair:
Dionisios N. Pnevmatikatos
Technical University of Crete and ICS - FORTH, Greece

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 July 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

German Federal Ministry of Education and Research (BMBF)

Conference

SAMOS XVIII

SAMOS XVIII: Architectures, Modeling, and Simulation

July 15 - 19, 2018

Pythagorion, Greece

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
94
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Jahić JDoganci VGehring H(2022)CASA: An Approach for Exposing and Documenting Concurrency-Related Software PropertiesEmbedded Computer Systems: Architectures, Modeling, and Simulation10.1007/978-3-031-15074-6_9(139-154)Online publication date: 14-Aug-2022
https://doi.org/10.1007/978-3-031-15074-6_9
Cornaglia AViehl ABringmann O(2022)Accurate LLVM IR to Binary CFGs Mapping for Simulation of Optimized Embedded SoftwareEmbedded Computer Systems: Architectures, Modeling, and Simulation10.1007/978-3-031-04580-6_1(3-15)Online publication date: 27-Apr-2022
https://doi.org/10.1007/978-3-031-04580-6_1
Jahić JBauer TKuhn TWehn NAntonino P(2020)FERA: A Framework for Critical Assessment of Execution Monitoring Based Approaches for Finding Concurrency BugsIntelligent Computing10.1007/978-3-030-52249-0_5(54-74)Online publication date: 4-Jul-2020
https://doi.org/10.1007/978-3-030-52249-0_5

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents