research-article

A probabilistic analysis of coverage methods

Authors:

Laurent Fournier,

Ekaterina Kutsy,

Ofer StrichmanAuthors Info & Claims

ACM Transactions on Design Automation of Electronic Systems (TODAES), Volume 16, Issue 4

Article No.: 38, Pages 1 - 20

https://doi.org/10.1145/2003695.2003698

Published: 27 October 2011 Publication History

Abstract

Coverage is an important measure for the quality and completeness of the functional verification of hardware logic designs. Verification teams spend a significant amount of time looking for bugs in the design and in providing high-quality coverage. This process is performed through the use of various sampling strategies for selecting test inputs. The selection of sampling strategies to achieve the verification goals is typically carried out in an intuitive manner.

We studied several commonly used sampling strategies and provide a probabilistic framework for assessing and comparing their relative values. For this analysis, we derived results for two measures of interest: first, the probability of finding a bug within a given number of samplings; and second, the expected number of samplings until a bug is detected. These results are given for both recurring sampling schemes, in which the same inputs might be selected repeatedly, and for nonrecurring sampling schemes, in which already sampled inputs are never selected again.

By considering results from the theory of search, and more specifically, from the well-known multiarmed bandit problem, we demonstrate the optimality of a greedy sampling strategy within our defined framework.

Supplementary Material

fournier (fournier.zip)

Supplemental movie, appendix, image and software files for, A probabilistic analysis of coverage methods

Download
66.89 KB

References

[1]

Agrawal, V. 1981. Sampling techniques for determining fault coverage in LSI circuits. J. Digital Syst. 5, 3, 189--202.

[2]

Agrawal, V. 1990. Fault sampling revisited. IEEE Des. Test Comput. 7, 4, 32--35.

Digital Library

[3]

Aharoni, M., Asaf, S., Fournier, L., Koyfman, A., and Nagel, R. 2003. FPgen—a test-generation framework for datapath floating-point verification. In Proceedings of the IEEE International High-Level Design Validation and Test (HLDVT). IEEE, Los Alamitos, CA, 17--22.

Digital Library

[4]

Campenhout, D. V., Mudge, T., and Hayes, J. P. 1999. High-level test generation for design verification of pipelined microprocessors. In Proceedings of the 36th Design Automation Conference. 185--188.

Digital Library

[5]

Clarke, E. M., Grumberg, O., and Peled, D. A. 1999. Model Checking. MIT Press, Cambridge, MA.

Digital Library

[6]

Cui, H., Seth, S. C., and Mehta, S. K. 2003. Modeling fault coverage of random test patterns. J. Electron. Test: Theory Appl. 19, 271--284.

Digital Library

[7]

Dechter, R. 2003. Constraints Processing. Morgan Kaufmann.

Digital Library

[8]

Fine, S. and Ziv, A. 2003. Coverage directed test generation for functional verification using Bayesian networks. In Proceedings of the 40th Design Automation Conference. 286--291.

Digital Library

[9]

Gittins, J. 1989. Multi-Armed Bandit Allocation Indexes. Prentice-Hall, Englewood Cliffs, NJ.

[10]

Goel, P. 1980. Test generation cost analysis and projections. In Proceedings of the 17th Design Automation Conference. 77--84.

Digital Library

[11]

Gogate, V. and Dechter, R. 2006. A new algorithm for sampling CSP solutions uniformly at random. In Proceedings of the 12th International Conference on Principles and Practice of Constraint Programming (CP). Lecture Notes in Computer Science, Vol. 4204, Springer, Berlin, 711--715.

Digital Library

[12]

IEEE 2008. IEEE Standard for binary floating-point arithmetic. IEEE Std. 754-2008.

[13]

Iwashita, H., Kowatari, S., Nakata, T., and Hirose, F. 1994. Automatic test program generation for pipelined processors. In Proceedings of the International Conference on Computer-Aided Design. 580--583.

Digital Library

[14]

Jain, S. K. and Agrawal, V. D. 1985. Stafan: An alternative to fault simulation. IEEE Des. Test Comput. 2, 38--44.

Digital Library

[15]

Piziali, A. 2004. Functional Verification Coverage Measurement and Analysis. Springer, Berlin.

Digital Library

[16]

Ur, S. and Yadin, Y. 1999. Micro-architecture coverage directed generation of test programs. In Proceedings of the 36th Design Automation Conference. 175--180.

Digital Library

[17]

Wagner, I., Bertacco, V., and Austin, T. 2007. Microprocessor verification via feedback-adjusted Markov models. IEEE Trans. Comput.-Aided Des. Integrat. Circuits Syst. 26, 6, 1126--1138.

Digital Library

[18]

Weber, R. 1992. On the Gittins index for multiarmed bandits. Ann. Appl. Probab. 2, 4, 1024--1033.

[19]

Wile, B., Goss, J. C., and Roesner, W. 2005. Comprehensive Functional Verification—The Complete Industry Cycle. Elsevier, Amsterdam.

Digital Library

[20]

Wunderlich, H.-J. 1985. Protest: A tool for probabilistic testability analysis. In Proceedings of the 22nd Design Automation Conference. 204--211.

Digital Library

Cited By

Dossis M(2023)Synthesis-Embedded Verification2023 8th South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference (SEEDA-CECNSM)10.1109/SEEDA-CECNSM61561.2023.10470600(1-7)Online publication date: 10-Nov-2023
https://doi.org/10.1109/SEEDA-CECNSM61561.2023.10470600
Nie JGeng SPeng XCao WLi PLi X(2022)A Location Analysis for Dynamic Verification2022 3rd International Conference on Computer Vision, Image and Deep Learning & International Conference on Computer Engineering and Applications (CVIDL & ICCEA)10.1109/CVIDLICCEA56201.2022.9824849(524-527)Online publication date: 20-May-2022
https://doi.org/10.1109/CVIDLICCEA56201.2022.9824849
Zhang MGeng SWang WPeng XChu MZhou SZhang ZLu HLi PZhu R(2021)Probabilistic Analysis for Sequential Circuits Verification Using Markov ChainsIEEE Transactions on Circuits and Systems II: Express Briefs10.1109/TCSII.2020.300570568:1(481-485)Online publication date: Jan-2021
https://doi.org/10.1109/TCSII.2020.3005705
Show More Cited By

Index Terms

A probabilistic analysis of coverage methods
1. Hardware
  1. Hardware validation
    1. Functional verification

Recommendations

Comparing two testbench methods for hierarchical functional verification of a bluetooth baseband adaptor
CODES+ISSS '05: Proceedings of the 3rd IEEE/ACM/IFIP international conference on Hardware/software codesign and system synthesis

The continuous improvement on the design methodologies and processes has made possible the creation of huge and very complex digital systems. Design verification is one of the main tasks in the design flow, aiming to certify the system functionality has ...
A Coverage-Driven Constraint Random-Based Functional Verification Method of Pipeline Unit
ICIS '09: Proceedings of the 2009 Eigth IEEE/ACIS International Conference on Computer and Information Science

This paper presents a coverage-driven constraint random-based functional verification (CCRFV) method of pipeline unit in a microprocessor. The environment of verification, which is created by means of verification methodology manual (VMM) for ...
Probabilistic regression suites for functional verification
DAC '04: Proceedings of the 41st annual Design Automation Conference

Random test generators are often used to create regression suites on-the-fly. Regression suites are commonly generated by choosing several specifications and generating a number of tests from each one, without reasoning which specification should be ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Design Automation of Electronic Systems

ACM Transactions on Design Automation of Electronic Systems Volume 16, Issue 4

October 2011

326 pages

ISSN:1084-4309

EISSN:1557-7309

DOI:10.1145/2003695

Issue’s Table of Contents

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 27 October 2011

Accepted: 01 March 2011

Revised: 01 November 2010

Received: 01 June 2010

Published in TODAES Volume 16, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
225
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dossis M(2023)Synthesis-Embedded Verification2023 8th South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference (SEEDA-CECNSM)10.1109/SEEDA-CECNSM61561.2023.10470600(1-7)Online publication date: 10-Nov-2023
https://doi.org/10.1109/SEEDA-CECNSM61561.2023.10470600
Nie JGeng SPeng XCao WLi PLi X(2022)A Location Analysis for Dynamic Verification2022 3rd International Conference on Computer Vision, Image and Deep Learning & International Conference on Computer Engineering and Applications (CVIDL & ICCEA)10.1109/CVIDLICCEA56201.2022.9824849(524-527)Online publication date: 20-May-2022
https://doi.org/10.1109/CVIDLICCEA56201.2022.9824849
Zhang MGeng SWang WPeng XChu MZhou SZhang ZLu HLi PZhu R(2021)Probabilistic Analysis for Sequential Circuits Verification Using Markov ChainsIEEE Transactions on Circuits and Systems II: Express Briefs10.1109/TCSII.2020.300570568:1(481-485)Online publication date: Jan-2021
https://doi.org/10.1109/TCSII.2020.3005705
Zhou MN. N. Hung WSong XGu MSun J(2018)Temporal Coverage Analysis for Dynamic VerificationIEEE Transactions on Circuits and Systems II: Express Briefs10.1109/TCSII.2017.274674465:1(66-70)Online publication date: Jan-2018
https://doi.org/10.1109/TCSII.2017.2746744
Zhou JLan CHung WGuo XSong X(2015)A Quantitative Characterization of Cross CoverageIEEE Transactions on Computers10.1109/TC.2014.236054064:8(2408-2414)Online publication date: 1-Aug-2015
https://doi.org/10.1109/TC.2014.2360540
Quispe JStrum MChau W(2013)PrOCov: Probabilistic output coverage model2013 14th Latin American Test Workshop - LATW10.1109/LATW.2013.6562664(1-6)Online publication date: Apr-2013
https://doi.org/10.1109/LATW.2013.6562664

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents