research-article

Relatively complete counterexamples for higher-order programs

Authors:

Phúc C. Nguyễn,

David Van HornAuthors Info & Claims

PLDI '15: Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation

Pages 446 - 456

https://doi.org/10.1145/2737924.2737971

Published: 03 June 2015 Publication History

Abstract

In this paper, we study the problem of generating inputs to a higher-order program causing it to error. We first approach the problem in the setting of PCF, a typed, core functional language and contribute the first relatively complete method for constructing counterexamples for PCF programs. The method is relatively complete with respect to a first-order solver over the base types of PCF. In practice, this means an SMT solver can be used for the effective, automated generation of higher-order counterexamples for a large class of programs. We achieve this result by employing a novel form of symbolic execution for higher-order programs. The remarkable aspect of this symbolic execution is that even though symbolic higher-order inputs and values are considered, the path condition remains a first-order formula. Our handling of symbolic function application enables the reconstruction of higher-order counterexamples from this first-order formula. After establishing our main theoretical results, we sketch how to apply the approach to untyped, higher-order, stateful languages with first-class contracts and show how counterexample generation can be used to detect contract violations in this setting. To validate our approach, we implement a tool generating counterexamples for erroneous modules written in Racket.

References

[1]

C. Cadar, V. Ganesh, P. M. Pawlowski, D. L. Dill, and D. R. Engler. EXE: automatically generating inputs of death. In Proceedings of the 13th ACM Conference on Computer and Communications Security. ACM, 2006.

Digital Library

[2]

C. Cadar, D. Dunbar, and D. Engler. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation. USENIX, 2008.

Digital Library

[3]

K. Claessen and J. Hughes. QuickCheck: A lightweight tool for random testing of Haskell programs. In Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming. ACM, 2000.

Digital Library

[4]

S. A. Cook. Soundness and completeness of an axiom system for program verification. In SIAM Journal of Computing, 1978.

[5]

L. De Moura and N. Bjørner. Z3: an efficient SMT solver. In Proceedings of the Theory and practice of software, 14th international conference on Tools and algorithms for the construction and analysis of systems. Springer-Verlag, 2008.

Digital Library

[6]

C. Dimoulas, R. B. Findler, C. Flanagan, and M. Felleisen. Correct blame for contracts: no more scapegoating. In Proceedings of the 38th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2011.

Digital Library

[7]

C. Dimoulas, S. Tobin-Hochstadt, and M. Felleisen. Complete monitors for behavioral contracts. In 21st European Symposium on Programming. Springer Berlin Heidelberg, 2012.

Digital Library

[8]

R. B. Findler and M. Felleisen. Contracts for higher-order functions. In Proceedings of the seventh ACM SIGPLAN International Conference on Functional Programming. ACM, 2002.

Digital Library

[9]

C. Flanagan. Hybrid type checking. In Conference record of the 33rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2006.

Digital Library

[10]

M. Flatt and PLT. Reference: Racket. Technical report, PLT Inc., 2010.

[11]

J. S. Foster, T. Terauchi, and A. Aiken. Flow-sensitive type qualifiers. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 2002.

Digital Library

[12]

P. Godefroid, N. Klarlund, and K. Sen. DART: Directed automated random testing. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 2005.

Digital Library

[13]

M. Greenberg, B. C. Pierce, and S. Weirich. Contracts made manifest. In Proceedings of the 37th annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2010.

Digital Library

[14]

P. Heidegger and P. Thiemann. Contract-Driven testing of JavaScript code. In Objects, Models, Components, Patterns. Springer Berlin Heidelberg, 2010.

Digital Library

[15]

R. Hinze, J. Jeuring, and A. Löh. Typed contracts for functional programming. In Functional and Logic Programming. Springer, 2006.

Digital Library

[16]

J. Hughes. Personal communication, 2015.

[17]

M. Kawaguchi, P. Rondon, and R. Jhala. Dsolve: Safety verification via liquid types. In Computer Aided Verification. Springer Berlin Heidelberg, 2010.

Digital Library

[18]

C. Klein, M. Flatt, and R. B. Findler. Random testing for higherorder, stateful programs. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications. ACM, 2010.

Digital Library

[19]

K. Knowles and C. Flanagan. Hybrid type checking. ACM Trans. Program. Lang. Syst., 2010.

Digital Library

[20]

N. Kobayashi. Model checking Higher-Order programs. J. ACM, 2013.

Digital Library

[21]

N. Kobayashi, R. Sato, and H. Unno. Predicate abstraction and CEGAR for higher-order model checking. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 2011.

Digital Library

[22]

P. C. Nguyễn and D. Van Horn. Relatively complete counterexamples for Higher-Order programs. CoRR, abs/1411.3967, 2015.

Digital Library

[23]

P. C. Nguyễn, S. Tobin-Hochstadt, and D. Van Horn. Soft contract verification. In Proceedings of the 19th ACM SIGPLAN International Conference on Functional Programming. ACM, 2014.

Digital Library

[24]

P. M. Rondon, M. Kawaguci, and R. Jhala. Liquid types. In Proceedings of the 2008 ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 2008.

Digital Library

[25]

D. S. Scott. A type-theoretical alternative to ISWIM, CUCH, OWHY. Theoretical Computer Science, 1993.

Digital Library

[26]

E. L. Seidel, N. Vazou, and R. Jhala. Type targeted testing. In 21st European Symposium on Programming. Springer Berlin Heidelberg, 2015.

Digital Library

[27]

V. St-Amour, S. Tobin-Hochstadt, M. Flatt, and M. Felleisen. Typing the numeric tower. In Practical Aspects of Declarative Languages. Springer Berlin Heidelberg, 2012.

Digital Library

[28]

T. Terauchi. Dependent types from counterexamples. In Proceedings of the 37th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2010.

Digital Library

[29]

S. Tobin-Hochstadt and M. Felleisen. Logical types for untyped languages. In Proceedings of the ACM International Conference on Functional Programming. ACM, 2010.

Digital Library

[30]

S. Tobin-Hochstadt and D. Van Horn. Higher-order symbolic execution via contracts. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications. ACM, 2012.

Digital Library

[31]

D. Van Horn and M. Might. Abstracting abstract machines. In Proceedings of the 15th ACM SIGPLAN International Conference on Functional Programming. ACM, 2010.

Digital Library

[32]

D. Vytiniotis, S. Peyton Jones, K. Claessen, and D. Rosén. HALO: Haskell to logic through denotational semantics. In Proceedings of the 40th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages. ACM, 2013.

Digital Library

[33]

Y. Xie and A. Aiken. Scalable error detection using boolean satisfiability. In Proceedings of the 32nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2005.

Digital Library

[34]

D. N. Xu. Hybrid contract checking via symbolic simplification. In Proceedings of the ACM SIGPLAN 2012 Workshop on Partial Evaluation and Program Manipulation. ACM, 2012.

Digital Library

[35]

D. N. Xu, S. Peyton Jones, and S. Claessen. Static contract checking for Haskell. In Proceedings of the 36th Annual ACM SIGPLANSIGACT Symposium on Principles of Programming Languages. ACM, 2009.

Digital Library

[36]

J. Yang, P. Twohey, D. Engler, and M. Musuvathi. Using model checking to find serious file system errors. In Sixth Symposium on Operating Systems Design and Implementation. USENIX, 2004.

Digital Library

[37]

H. Zhu and S. Jagannathan. Compositional and lightweight dependent type inference for ML. In Conference on Verification, Model-Checking and Abstract Interpretation, 2013.

Digital Library

Cited By

Qian KSmith SStride BWeng SWu K(2024)Semantic-Type-Guided Bug FindingProceedings of the ACM on Programming Languages10.1145/36897888:OOPSLA2(2183-2210)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689788
Moy CNguyễn PTobin-Hochstadt SVan Horn D(2021)Corpse reviver: sound and efficient gradual typing via contract verificationProceedings of the ACM on Programming Languages10.1145/34343345:POPL(1-28)Online publication date: 4-Jan-2021
https://dl.acm.org/doi/10.1145/3434334
Song DLee MOh H(2019)Automatic and scalable detection of logical errors in functional programming assignmentsProceedings of the ACM on Programming Languages10.1145/33606143:OOPSLA(1-30)Online publication date: 10-Oct-2019
https://dl.acm.org/doi/10.1145/3360614
Show More Cited By

Index Terms

Relatively complete counterexamples for higher-order programs

Recommendations

Relatively complete counterexamples for higher-order programs
PLDI '15

In this paper, we study the problem of generating inputs to a higher-order program causing it to error. We first approach the problem in the setting of PCF, a typed, core functional language and contribute the first relatively complete method for ...
Automating relatively complete verification of higher-order functional programs
POPL '13

We present an automated approach to relatively completely verifying safety (i.e., reachability) property of higher-order functional programs. Our contribution is two-fold. First, we extend the refinement type system framework employed in the recent work ...
Automating relatively complete verification of higher-order functional programs
POPL '13: Proceedings of the 40th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages

We present an automated approach to relatively completely verifying safety (i.e., reachability) property of higher-order functional programs. Our contribution is two-fold. First, we extend the refinement type system framework employed in the recent work ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PLDI '15: Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation

June 2015

630 pages

ISBN:9781450334686

DOI:10.1145/2737924

General Chair:
David Grove
IBM Research, USA
,
Program Chair:
Steve Blackburn
Australian National University, Australia

ACM SIGPLAN Notices Volume 50, Issue 6
PLDI '15
June 2015
630 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/2813885
Editor:
Andy Gill
University of Kansas, Lawrence, KS
Issue’s Table of Contents

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 June 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

PLDI '15

Sponsor:

SIGPLAN

PLDI '15: ACM SIGPLAN Conference on Programming Language Design and Implementation

June 13 - 17, 2015

OR, Portland, USA

Acceptance Rates

Overall Acceptance Rate 406 of 2,067 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
406
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)4

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Qian KSmith SStride BWeng SWu K(2024)Semantic-Type-Guided Bug FindingProceedings of the ACM on Programming Languages10.1145/36897888:OOPSLA2(2183-2210)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3689788
Moy CNguyễn PTobin-Hochstadt SVan Horn D(2021)Corpse reviver: sound and efficient gradual typing via contract verificationProceedings of the ACM on Programming Languages10.1145/34343345:POPL(1-28)Online publication date: 4-Jan-2021
https://dl.acm.org/doi/10.1145/3434334
Song DLee MOh H(2019)Automatic and scalable detection of logical errors in functional programming assignmentsProceedings of the ACM on Programming Languages10.1145/33606143:OOPSLA(1-30)Online publication date: 10-Oct-2019
https://dl.acm.org/doi/10.1145/3360614
Schuster CFlanagan C(2019)IDVECompanion Proceedings of the 3rd International Conference on the Art, Science, and Engineering of Programming10.1145/3328433.3328453(1-16)Online publication date: 1-Apr-2019
https://dl.acm.org/doi/10.1145/3328433.3328453
Lin YTzevelekos N(2019)A Bounded Model Checking Technique for Higher-Order ProgramsDependable Software Engineering. Theories, Tools, and Applications10.1007/978-3-030-35540-1_1(1-18)Online publication date: 18-Nov-2019
https://doi.org/10.1007/978-3-030-35540-1_1
Selakovic MPradel MKarim RTip F(2018)Test generation for higher-order functions in dynamic languagesProceedings of the ACM on Programming Languages10.1145/32765312:OOPSLA(1-27)Online publication date: 24-Oct-2018
https://dl.acm.org/doi/10.1145/3276531
Vazou NTanter ÉVan Horn D(2018)Gradual liquid type inferenceProceedings of the ACM on Programming Languages10.1145/32765022:OOPSLA(1-25)Online publication date: 24-Oct-2018
https://dl.acm.org/doi/10.1145/3276502
LOPEZ-GARCIA PDARMAWAN LKLEMEN MLIQAT UBUENO FHERMENEGILDO M(2018)Interval-based resource usage verification by translation into Horn clauses and an application to energy consumptionTheory and Practice of Logic Programming10.1017/S147106841800004218:02(167-223)Online publication date: 11-May-2018
https://doi.org/10.1017/S1471068418000042
Nguyễn PGilray TTobin-Hochstadt SVan Horn D(2017)Soft contract verification for higher-order stateful programsProceedings of the ACM on Programming Languages10.1145/31581392:POPL(1-30)Online publication date: 27-Dec-2017
https://dl.acm.org/doi/10.1145/3158139
Darais DLabich NNguyen PVan Horn D(2017)Abstracting definitional interpreters (functional pearl)Proceedings of the ACM on Programming Languages10.1145/31102561:ICFP(1-25)Online publication date: 29-Aug-2017
https://dl.acm.org/doi/10.1145/3110256
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents