research-article

NQC²: A Non-Intrusive QEMU Code Coverage Plugin

Authors:

Alwalid Salama,

Niko Zurstraßen,

Rainer LeupersAuthors Info & Claims

RAPIDO '24: Proceedings of the 16th Workshop on Rapid Simulation and Performance Evaluation for Design

Pages 16 - 21

https://doi.org/10.1145/3642921.3642924

Published: 06 March 2024 Publication History

Abstract

Code coverage analysis has become a standard approach in software development, facilitating the assessment of test suite effectiveness, the identification of under-tested code segments, and the discovery of performance bottlenecks. When code coverage of software for embedded systems needs to be measured, conventional approaches quickly meet their limits. A commonly used approach involves instrumenting the source files with added code that collects and dumps coverage information during runtime. This inserted code usually relies on the existence of an operating and a file system to dump the collected data. These features are not available for bare-metal programs that are executed on embedded systems.

To overcome this issue, we present NQC², a plugin for QEMU. NQC² extracts coverage information from QEMU during runtime and stores them into a file on the host machine. This approach is even compatible with modified QEMU versions and does not require target-software instrumentation. NQC² outperforms a comparable approach from Xilinx by up to 8.5 x.

References

[1]

Fabrice Bellard. 2005. QEMU, a fast and portable dynamic translator. In USENIX annual technical conference, FREENIX Track, Vol. 41. Califor-nia, USA, 10–5555.

[2]

M. Ammar Ben Khadra, Dominik Stoffel, and Wolfgang Kunz. 2020. Efficient binary-level coverage analysis. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering(ESEC/FSE 2020). https://doi.org/10.1145/3368089.3409694

Digital Library

[3]

Holger Blasum, Frank Görgen, and Jürgen Urban. 2007. Gcov on an embedded system. (2007).

[4]

Nils Bosbach, Lukas Jünger, Rebecca Pelke, Niko Zurstraßen, and Rainer Leupers. 2023. Entropy-Based Analysis of Benchmarks for Instruction Set Simulators. In Proceedings of the DroneSE and RAPIDO: System Engineering for constrained embedded systems(RAPIDO ’23). https://doi.org/10.1145/3579170.3579267

Digital Library

[5]

Emilio G. Cota and Luca P. Carloni. 2019. Cross-ISA machine instrumentation using fast and scalable dynamic binary translation. In Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments.

[6]

Free Software Foundation, Inc.2023. Gcov (Using the GNU Compiler Collection (GCC)). https://gcc.gnu.org/onlinedocs/gcc/Gcov.html

[7]

Shay Gal-On and Markus Levy. 2012. Exploring coremark a benchmark maximizing simplicity and efficacy. The Embedded Microprocessor Benchmark Consortium (2012).

[8]

Edgar E. Iglesias. 2023. edgarigl/qemu-etrace. https://github.com/edgarigl/qemu-etrace original-date: 2016-02-15T09:42:55Z.

[9]

Marko Ivanković, Goran Petrović, René Just, and Gordon Fraser. 2019. Code coverage at Google. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering(ESEC/FSE 2019). https://doi.org/10.1145/3338906.3340459

Digital Library

[10]

Linux Test Project. 2023. LTP GCOV extension (LCOV). https://github.com/linux-test-project/lcov original-date: 2014-05-27T14:38:58Z.

[11]

LLVM Project. 2023. llvm-cov - emit coverage information — LLVM 18.0.0git documentation. https://llvm.org/docs/CommandGuide/llvm-cov.html

[12]

John D. McCalpin. 1995. Memory Bandwidth and Machine Balance in Current High Performance Computers. IEEE Computer Society Technical Committee on Computer Architecture (TCCA) Newsletter (Dec. 1995), 19–25.

[13]

Joan C. Miller and Clifford J. Maloney. 1963. Systematic mistake analysis of digital computer programs. Commun. ACM 6, 2 (Feb. 1963).

Digital Library

[14]

Stefan Nagy and Matthew Hicks. 2019. Full-Speed Fuzzing: Reducing Fuzzing Overhead through Coverage-Guided Tracing. In 2019 IEEE Symposium on Security and Privacy (SP). 787–802. https://doi.org/10.1109/SP.2019.00069 ISSN: 2375-1207.

[15]

NASA Jet Propulsion Laboratory. 2023. nasa-jpl/embedded-gcov. https://github.com/nasa-jpl/embedded-gcov original-date: 2022-02-02T19:25:26Z.

[16]

P. Piwowarski, M. Ohba, and J. Caruso. 1993. Coverage measurement experience during function test. In Proceedings of 1993 15th International Conference on Software Engineering. 287–301. https://doi.org/10.1109/ICSE.1993.346035

[17]

QEMU. 2023. QEMU. https://github.com/qemu/qemu

[18]

QEMU. 2023. QEMU TCG Plugins — QEMU documentation. https://www.qemu.org/docs/master/devel/tcg-plugins.html#api

[19]

Reinhold P. Weicker. 1984. Dhrystone: a synthetic systems programming benchmark. Commun. ACM 27, 10 (Oct. 1984). https://doi.org/10.1145/358274.358283

Digital Library

[20]

Brian A Wichmann. 1970. Some statistics from ALGOL programs. Central Computer Unit, National Physical Laboratory.

[21]

Xilinx. 2023. Xilinx’s fork of QEMU. https://github.com/Xilinx/qemu

Cited By

Reimann LWiesner JJaszczyk KGhinami CGermek DMerchant FLeupers R(2024)Enhancing HW-SW Confidentiality Verification for Embedded Processors with SoftFlow’s Advanced Memory Range FeatureVLSI-SoC 2023: Innovations for Trustworthy Artificial Intelligence10.1007/978-3-031-70947-0_13(251-272)Online publication date: 29-Dec-2024
https://doi.org/10.1007/978-3-031-70947-0_13

Index Terms

NQC²: A Non-Intrusive QEMU Code Coverage Plugin
1. Hardware
  1. Hardware validation
    1. Functional verification
      1. Simulation and emulation
2. Software and its engineering
  1. Software notations and tools
    1. Software maintenance tools

Recommendations

QEMU-based framework for non-intrusive virtual machine instrumentation and introspection
ESEC/FSE 2017: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering

This paper presents the framework based on the emulator QEMU. Our framework provides set of multi-platform analysis tools for the virtual machines and mechanism for creating instrumentation and analysis tools. Our framework is based on a lightweight ...
Efficient instrumentation for code coverage testing

Evaluation of Code Coverage is the problem of identifying the parts of a program that did not execute in one or more runs of a program. The traditional approach for code coverage tools is to use static code instrumentation. In this paper we present a ...
Improving Live Migration Efficiency in QEMU: A Paravirtualized Approach
High Performance Computing
Abstract
Virtual Machines are the key technology in cloud computing. In order to upgrade, repair or service the physical machine where a Virtual Machine is hosted, a common practice is to live-migrate the Virtual Machine to a different server. This ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

RAPIDO '24: Proceedings of the 16th Workshop on Rapid Simulation and Performance Evaluation for Design

January 2024

54 pages

ISBN:9798400717918

DOI:10.1145/3642921

Editors:
Lilia Zaourar
CEA, France
,
Daniel Chillet
University of Rennes 1
,
Daniel Gracia Pérez
Thales Research & Technology, France
,
Morteza Biglari Abhari
The University of Auckland, New Zealand
,
Reda Nouacer
CEA, France
,
Matthias Jung
Fraunhofer
,
Gianluca Palermo
Politecnico di Milano, Italy

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 March 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

RAPIDO '24

RAPIDO '24: Rapid Simulation and Performance Evaluation for Design

January 18, 2024

Munich, Germany

Acceptance Rates

Overall Acceptance Rate 14 of 28 submissions, 50%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
37
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)8

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Reimann LWiesner JJaszczyk KGhinami CGermek DMerchant FLeupers R(2024)Enhancing HW-SW Confidentiality Verification for Embedded Processors with SoftFlow’s Advanced Memory Range FeatureVLSI-SoC 2023: Innovations for Trustworthy Artificial Intelligence10.1007/978-3-031-70947-0_13(251-272)Online publication date: 29-Dec-2024
https://doi.org/10.1007/978-3-031-70947-0_13

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents