research-article

Fast and sound random generation for automated testing and benchmarking in objective Caml

Authors:

Benjamin Canou,

Alexis DarrasseAuthors Info & Claims

ML '09: Proceedings of the 2009 ACM SIGPLAN workshop on ML

Pages 61 - 70

https://doi.org/10.1145/1596627.1596637

Published: 30 August 2009 Publication History

Abstract

Numerous software testing methods involve random generation of data structures. However, random sampling methods currently in use by testing frameworks are not satisfactory: often manually written by the programmer or at best extracted in an ad-hoc way relying on no theoretical background. On the other end, random sampling methods with good theoretical properties exist but have a too high cost to be used in testing, in particular when large inputs are needed.

In this paper we describe how we applied the recently developed Boltzmann model of random generation to algebraic data types. We obtain a fully automatic way to derive random generators from Objective Caml type definitions. These generators have linear complexity and, the generation method being uniform, can also be used as a sound sampling back-end for benchmarking tools.

As a result, we provide testing and benchmarking frameworks with a sound and fast generation basis. We also provide a testing and benchmarking library, available for download, showing the viability of this experiment.

References

[1]

M. Abbott, T. Altenkirch, N. Ghani, and C. McBride. Derivatives of containers. Lecture notes in computer science, pages 16--30, 2003.

Digital Library

[2]

F. Bergeron, G. Labelle, and P. Leroux. Combinatorial species and tree-like structures. Cambridge University Press, 1998.

[3]

Jacob Burnim, Sudeep Juvekar, and Koushik Sen. Wise: Automated test generation for worst-case complexity. In ICSE, pages 463--473, 2009.

Digital Library

[4]

Koen Claessen and John Hughes. Quickcheck: a lightweight tool for random testing of haskell programs. In ICFP '00: Proceedings of the fifth ACM SIGPLAN international conference on Functional programming, pages 268--279, New York, NY, USA, 2000. ACM. ISBN 1-58113-202-6. 10.1145/351240.351266.

Digital Library

[5]

A. Darrasse. Random XML sampling the Boltzmann way. ArXiv e-prints, 2008. URL http://arxiv.org/abs/0807.0992.

[6]

Philippe Duchon, Philippe Flajolet, Guy Louchard, and Gilles Schaeffer. Boltzmann samplers for the random generation of combinatorial structures. Combinatorics, Probability and Computing, 13: 577--625, 2004. 10.1017/S0963548304006315.

Digital Library

[7]

Sebastian Fischer and Herbert Kuchen. Data-flow testing of declarative programs. SIGPLAN Not., 43 (9): 201-212, 2008. ISSN 0362-1340. http://doi.acm.org/10.1145/1411203.1411233.

Digital Library

[8]

P. Flajolet, P. Zimmerman, and B. Van Cutsem. A calculus for the random generation of labelled combinatorial structures. Theoretical Computer Science, 132 (1-2): 1--35, 1994.

Digital Library

[9]

Philippe Flajolet and Robert Sedgewick. Analytic Combinatorics. Cambridge University Press, 2009.

Digital Library

[10]

t al.(2007)Henry, Mauny, and Chailloux}2007arXiv0705.1452HG. Henry, M. Mauny, and E. Chailloux. Typer la dé-sérialisation sans sérialiser les types. ArXiv e-prints, May 2007. URL http://arxiv.org/abs/0705.1452.

[11]

G. Huet. The zipper. phJournal of Functional Programming, 7 (05): 549--554, 1997.

Digital Library

[12]

Xavier Leroy, Didier Rémy with Damien Doligez, Jacques Garrigue, and Jérôme Vouillon. The objective caml system release 3.11 documentation and user's manual. Technical report, Inria, november 2008.

[13]

François Maurel. Ocaml-templates, méta-programmation à partir des types. In Actes des journées JFLA Journées francophones des langages applicatifs, pages 21--36, Sainte-Marie-de-Ré, France, January 2004. INRIA. URL http://hal.archives-ouvertes.fr/hal-00153820/en/.

[14]

Alix Mougenot, Alexis Darrasse, Xavier Blanc, and Michèle Soria. Uniform random generation of huge metamodel instances. In Fifth European Conference on Model-Driven Architecture Foundations and Applications, 2009.

Digital Library

[15]

Carine Pivoteau, Bruno Salvy, and Michèle Soria. Boltzmann oracle for combinatorial systems. In Fifth Colloquium on Mathematics and Computer Science Algorithms, Trees, Combinatorics and Probabilities, DMTCS Proceedings, pages 475--488, 2008.

[16]

C. Runciman, M. Naylor, and F. Lindblad. Smallcheck and lazy smallcheck: automatic exhaustive testing for small values. In Proceedings of the first ACM SIGPLAN symposium on Haskell, pages 37--48. ACM New York, NY, USA, 2008.

Digital Library

[17]

Koushik Sen, Darko Marinov, and Gul Agha. Cute: a concolic unit testing engine for c. In ESEC/FSE-13: Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, pages 263--272, New York, NY, USA, 2005. ACM. ISBN 1-59593-014-0. http://doi.acm.org/10.1145/1081706.1081750.

Digital Library

[18]

Walid Taha. A gentle introduction to multi-stage programming, 2003. Available from http://www.cs.rice.edu/taha/publications/journal/dspg04a.pdf.

Cited By

Dien MPépin M(2023)Uniform SAmplINg with BOLTZmann2023 25th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC)10.1109/SYNASC61333.2023.00020(94-97)Online publication date: 11-Sep-2023
https://doi.org/10.1109/SYNASC61333.2023.00020
BANDERIER CDRMOTA M(2014)Formulae and Asymptotics for Coefficients of Algebraic FunctionsCombinatorics, Probability and Computing10.1017/S096354831400072824:1(1-53)Online publication date: 11-Dec-2014
https://doi.org/10.1017/S0963548314000728
Feldt RPoulding S(2013)Finding test data with specific properties via metaheuristic search2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE.2013.6698888(350-359)Online publication date: Nov-2013
https://doi.org/10.1109/ISSRE.2013.6698888
Show More Cited By

Index Terms

Fast and sound random generation for automated testing and benchmarking in objective Caml
1. Mathematics of computing
  1. Discrete mathematics
    1. Combinatorics
2. Software and its engineering
  1. Software notations and tools
    1. Compilers

Recommendations

Reflecting on Random Generation

Expert users of property-based testing often labor to craft random generators that encode detailed knowledge about what it means for a test input to be valid and interesting. Fortunately, the fruits of this labor can also be put to other uses. In the ...
On the Use of Uniform Random Generation of Automata for Testing

Developing efficient and automatic testing techniques is one of the major challenges facing software validation community. In this paper, we show how a uniform random generation process of finite automata, developed in a recent work by Bassino and ...
Linear-time generation of inhomogeneous random directed walks
Proceedings of the Meeting on Analytic Algorithmics and Combinatorics

Directed random walks in dimension two describe the diffusion dynamics of particles in a line. Through a well-known bijection, excursions, i.e. walks in the half-plane, describe families of "simply-generated" Galton--Watson trees. These random objects ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ML '09: Proceedings of the 2009 ACM SIGPLAN workshop on ML

August 2009

76 pages

ISBN:9781605585093

DOI:10.1145/1596627

Program Chair:
Andreas Rossberg
Max Planck Institute for Software Systems

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 August 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICFP '09

Sponsor:

ICFP '09: ACM SIGPLAN International Conference on Functional Programming

August 30, 2009

Edinburgh, Scotland

Acceptance Rates

ML '09 Paper Acceptance Rate 6 of 11 submissions, 55%;

Overall Acceptance Rate 6 of 11 submissions, 55%

Upcoming Conference

ICFP '24

Sponsor:
sigplan

29th ACM SIGPLAN International Conference on Functional Programming

September 2 - 7, 2024

Milan , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
191
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)3

Reflects downloads up to 11 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dien MPépin M(2023)Uniform SAmplINg with BOLTZmann2023 25th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC)10.1109/SYNASC61333.2023.00020(94-97)Online publication date: 11-Sep-2023
https://doi.org/10.1109/SYNASC61333.2023.00020
BANDERIER CDRMOTA M(2014)Formulae and Asymptotics for Coefficients of Algebraic FunctionsCombinatorics, Probability and Computing10.1017/S096354831400072824:1(1-53)Online publication date: 11-Dec-2014
https://doi.org/10.1017/S0963548314000728
Feldt RPoulding S(2013)Finding test data with specific properties via metaheuristic search2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE.2013.6698888(350-359)Online publication date: Nov-2013
https://doi.org/10.1109/ISSRE.2013.6698888
Vasudevan NTratt L(2013)Detecting Ambiguity in Programming Language GrammarsSoftware Language Engineering10.1007/978-3-319-02654-1_9(157-176)Online publication date: 2013
https://doi.org/10.1007/978-3-319-02654-1_9
Yorgey B(2010)Species and functors and types, oh my!ACM SIGPLAN Notices10.1145/2088456.186354245:11(147-158)Online publication date: 30-Sep-2010
https://dl.acm.org/doi/10.1145/2088456.1863542
Yorgey BGibbons J(2010)Species and functors and types, oh my!Proceedings of the third ACM Haskell symposium on Haskell10.1145/1863523.1863542(147-158)Online publication date: 30-Sep-2010
https://dl.acm.org/doi/10.1145/1863523.1863542

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents