research-article

Testing noninterference, quickly

Authors:

Catalin Hritcu,

Benjamin C. Pierce,

Antal Spector-Zabusky,

Dimitrios Vytiniotis,

Arthur Azevedo de Amorim,

Leonidas LampropoulosAuthors Info & Claims

ICFP '13: Proceedings of the 18th ACM SIGPLAN international conference on Functional programming

Pages 455 - 468

https://doi.org/10.1145/2500365.2500574

Published: 25 September 2013 Publication History

Abstract

Information-flow control mechanisms are difficult to design and labor intensive to prove correct. To reduce the time wasted on proof attempts doomed to fail due to broken definitions, we advocate modern random testing techniques for finding counterexamples during the design process. We show how to use QuickCheck, a property-based random-testing tool, to guide the design of a simple information-flow abstract machine. We find that both sophisticated strategies for generating well-distributed random programs and readily falsifiable formulations of noninterference properties are critically important. We propose several approaches and evaluate their effectiveness on a collection of injected bugs of varying subtlety. We also present an effective technique for shrinking large counterexamples to minimal, easily comprehensible ones. Taken together, our best methods enable us to quickly and automatically generate simple counterexamples for all these bugs.

References

[1]

T. H. Austin and C. Flanagan. Efficient purely-dynamic information flow analysis. In Workshop on Programming Languages and Analysis for Security (PLAS), PLAS. 2009.

Digital Library

[2]

A. Azevedo de Amorim, N. Collins, A. DeHon, D. Demange, C. Hritcu, D. Pichardie, B. C. Pierce, R. Pollack, and A. Tolmach. A verified information-flow architecture. Under submission, July 2013.

[3]

S. Berghofer and T. Nipkow. Random testing in Isabelle/HOL. In 2nd International Conference on Software Engineering and Formal Methods (SEFM). 2004.

Digital Library

[4]

A. Birgisson, D. Hedin, and A. Sabelfeld. Boosting the permissiveness of dynamic information-flow tracking by testing. In 17th European Symposium on Research in Computer Security, ESORICS. 2012.

[5]

L. Bulwahn. The new Quickcheck for Isabelle - random, exhaustive and symbolic testing under one roof. In 2nd International Conference on Certified Programs and Proofs (CPP), volume 7679 of Lecture Notes in Computer Science. 2012.

Digital Library

[6]

L. Bulwahn. Smart testing of functional programs in Isabelle. In 18th International Conference on Logic for Programming, Artificial Intelligence, and Reasoning (LPAR), volume 7180 of Lecture Notes in Computer Science. 2012.

Digital Library

[7]

C. Cadar, D. Dunbar, and D. Engler. KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In 8th USENIX conference on Operating systems design and implementation, OSDI. 2008.

Digital Library

[8]

C. Cadar, P. Godefroid, S. Khurshid, C. S. Pasareanu, K. Sen, N. Tillmann, and W. Visser. Symbolic execution for software testing in practice: preliminary assessment. In 33rd International Conference on Software Engineering, ICSE '11. 2011.

Digital Library

[9]

K. Claessen and J. Hughes. QuickCheck: a lightweight tool for random testing of Haskell programs. In 5th ACM SIGPLAN International Conference on Functional Programming, ICFP. 2000.

Digital Library

[10]

P. Dybjer, Q. Haiyan, and M. Takeyama. Combining testing and proving in dependent type theory. In 16th International Conference on Theorem Proving in Higher Order Logics (TPHOLs), volume 2758 of Lecture Notes in Computer Science. 2003.

[11]

C. Eastlund. Doublecheck your theorems. In ACL2, 2009.

Digital Library

[12]

J. S. Fenton. Memoryless subsystems. The Computer Journal, 17(2):143--147, 1974.

[13]

P. Godefroid, N. Klarlund, and K. Sen. DART: directed automated random testing. In ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI. 2005.

Digital Library

[14]

J. A. Goguen and J. Meseguer. Unwinding and inference control. In IEEE Symposium on Security and Privacy, 1984.

[15]

D. Hedin and A. Sabelfeld. Information-flow security for a core of JavaScript. In 25th IEEE Computer Security Foundations Symposium (CSF), CSF. 2012.

Digital Library

[16]

C. Hrit¸cu, M. Greenberg, B. Karel, B. C. Pierce, and G. Morrisett. All your IFCException are belong to us. In 34th IEEE Symposium on Security and Privacy. May 2013.

Digital Library

[17]

J. Hughes. QuickCheck testing for fun and profit. In 9th International Symposium on Practical Aspects of Declarative Languages (PADL), volume 4354 of Lecture Notes in Computer Science. 2007.

Digital Library

[18]

C. Klein, J. Clements, C. Dimoulas, C. Eastlund, M. Felleisen, M. Flatt, J. A. McCarthy, J. Rafkind, S. Tobin-Hochstadt, and R. B. Findler. Run your research: On the effectiveness of lightweight mechanization. In Principles of Programming Languages (POPL), 2012.

Digital Library

[19]

C. Pacheco and M. D. Ernst. Randoop: feedback-directed random testing for Java. In 22nd ACM SIGPLAN Conference on Object-Oriented Programming Systems And Applications, OOPSLA. 2007.

Digital Library

[20]

H. Raju Chamarthi, P. Dillinger, M. Kaufmann, and P. Manolios. Integrating testing and interactive theorem proving. In ACL2, 2011.

[21]

J. Regehr, Y. Chen, P. Cuoq, E. Eide, C. Ellison, and X. Yang. Testcase reduction for C compiler bugs. In 33rd ACM SIGPLAN conference on Programming Language Design and Implementation. ACM, 2012.

Digital Library

[22]

A. Russo and A. Sabelfeld. Dynamic vs. static flow-sensitive security analysis. In 23rd Computer Security Foundations Symposium (CSF), CSF. 2010.

Digital Library

[23]

A. Sabelfeld and A. Myers. Language-based information-flow security. IEEE Journal on Selected Areas in Communications, 21(1):5--19, January 2003.

Digital Library

[24]

A. Sabelfeld and A. Russo. From dynamic to static and back: Riding the roller coaster of information-flow control research. In Ershov Memorial Conference. 2009.

Digital Library

[25]

D. Stefan, A. Russo, J. C. Mitchell, and D. Mazieres. Flexible dynamic information flow control in Haskell. In 4th Symposium on Haskell. 2011.

Digital Library

[26]

N. Williams, B. Marre, and P. Mouy. On-the-fly generation of Kpath tests for C functions. In 19th IEEE International Conference on Automated Software Engineering, ASE. 2004.

Digital Library

[27]

X. Yang, Y. Chen, E. Eide, and J. Regehr. Finding and understanding bugs in C compilers. In ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI. 2011.

Digital Library

[28]

S. A. Zdancewic. Programming Languages for Information Security. PhD thesis, Cornell University, August 2002.

Digital Library

[29]

A. Zeller and R. Hildebrandt. Simplifying and isolating failure-inducing input. IEEE Transactions on Software Engineering, 28(2):183--200, 2002.

Digital Library

Cited By

MILLER DMOMIGLIANO A(2024)Property-Based Testing by Elaborating Proof OutlinesTheory and Practice of Logic Programming10.1017/S1471068424000176(1-40)Online publication date: 21-Nov-2024
https://doi.org/10.1017/S1471068424000176
Anderson SBlanco RLampropoulos LPierce BTolmach A(2023)Formalizing Stack Safety as a Security Property2023 IEEE 36th Computer Security Foundations Symposium (CSF)10.1109/CSF57540.2023.00037(356-371)Online publication date: Jul-2023
https://doi.org/10.1109/CSF57540.2023.00037
Cardoso EPereira DDe Paula RReis LRibeiro R(2022)A Type-Directed Algorithm to Generate Random Well-Formed Parsing Expression GrammarsProceedings of the XXVI Brazilian Symposium on Programming Languages10.1145/3561320.3561326(8-14)Online publication date: 6-Oct-2022
https://dl.acm.org/doi/10.1145/3561320.3561326
Show More Cited By

Index Terms

Testing noninterference, quickly
1. Security and privacy
  1. Systems security
    1. Information flow control
    2. Operating systems security
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Testing noninterference, quickly
ICFP '13

Information-flow control mechanisms are difficult to design and labor intensive to prove correct. To reduce the time wasted on proof attempts doomed to fail due to broken definitions, we advocate modern random testing techniques for finding ...
Noninterference via symbolic execution
FMOODS'12/FORTE'12: Proceedings of the 14th joint IFIP WG 6.1 international conference and Proceedings of the 32nd IFIP WG 6.1 international conference on Formal Techniques for Distributed Systems

Noninterference is a high-level security property that guarantees the absence of illicit information flow at runtime. Noninterference can be enforced statically using information flow type systems; however, these are criticized for being overly ...
Intransitive noninterference in nondeterministic systems
CCS '12: Proceedings of the 2012 ACM conference on Computer and communications security

This paper addresses the question of how TA-security, a semantics for intransitive information-flow policies in deterministic systems, can be generalized to nondeterministic systems. Various definitions are proposed, including definitions that state ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICFP '13: Proceedings of the 18th ACM SIGPLAN international conference on Functional programming

September 2013

484 pages

ISBN:9781450323260

DOI:10.1145/2500365

General Chair:
Greg Morrisett
Harvard University, USA
,
Program Chair:
Tarmo Uustalu
Institute of Cybernetics at TUT, Estonia

ACM SIGPLAN Notices Volume 48, Issue 9
ICFP '13
September 2013
457 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/2544174
Editor:
Mark W. Bailey
Hamilton College, Clinton, NY
Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages
Northeastern University

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 September 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICFP'13

Sponsor:

SIGPLAN

ICFP'13: ACM SIGPLAN International Conference on Functional Programming

September 25 - 27, 2013

Massachusetts, Boston, USA

Acceptance Rates

ICFP '13 Paper Acceptance Rate 40 of 133 submissions, 30%;

Overall Acceptance Rate 333 of 1,064 submissions, 31%

Upcoming Conference

ICFP '25

Sponsor:
sigplan

ACM SIGPLAN International Conference on Functional Programming

October 12 - 18, 2025

Singapore , Singapore

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
361
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

MILLER DMOMIGLIANO A(2024)Property-Based Testing by Elaborating Proof OutlinesTheory and Practice of Logic Programming10.1017/S1471068424000176(1-40)Online publication date: 21-Nov-2024
https://doi.org/10.1017/S1471068424000176
Anderson SBlanco RLampropoulos LPierce BTolmach A(2023)Formalizing Stack Safety as a Security Property2023 IEEE 36th Computer Security Foundations Symposium (CSF)10.1109/CSF57540.2023.00037(356-371)Online publication date: Jul-2023
https://doi.org/10.1109/CSF57540.2023.00037
Cardoso EPereira DDe Paula RReis LRibeiro R(2022)A Type-Directed Algorithm to Generate Random Well-Formed Parsing Expression GrammarsProceedings of the XXVI Brazilian Symposium on Programming Languages10.1145/3561320.3561326(8-14)Online publication date: 6-Oct-2022
https://dl.acm.org/doi/10.1145/3561320.3561326
O'Connor LWickström OJhala RDillig I(2022)Quickstrom: property-based acceptance testing with LTL specificationsProceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation10.1145/3519939.3523728(1025-1038)Online publication date: 9-Jun-2022
https://dl.acm.org/doi/10.1145/3519939.3523728
Paraskevopoulou ZEline ALampropoulos LJhala RDillig I(2022)Computing correctly with inductive relationsProceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation10.1145/3519939.3523707(966-980)Online publication date: 9-Jun-2022
https://dl.acm.org/doi/10.1145/3519939.3523707
Mantovani MMomigliano A(2021)Towards Substructural Property-Based TestingLogic-Based Program Synthesis and Transformation10.1007/978-3-030-98869-2_6(92-112)Online publication date: 7-Sep-2021
https://dl.acm.org/doi/10.1007/978-3-030-98869-2_6
Lampropoulos LHicks MPierce B(2019)Coverage guided, property based testingProceedings of the ACM on Programming Languages10.1145/33606073:OOPSLA(1-29)Online publication date: 10-Oct-2019
https://dl.acm.org/doi/10.1145/3360607
Farina GChong SGaboardi MKomendantskaya E(2019)Relational Symbolic ExecutionProceedings of the 21st International Symposium on Principles and Practice of Declarative Programming10.1145/3354166.3354175(1-14)Online publication date: 7-Oct-2019
https://dl.acm.org/doi/10.1145/3354166.3354175
Blanco RMiller DMomigliano AKomendantskaya E(2019)Property-Based Testing via Proof ReconstructionProceedings of the 21st International Symposium on Principles and Practice of Declarative Programming10.1145/3354166.3354170(1-13)Online publication date: 7-Oct-2019
https://dl.acm.org/doi/10.1145/3354166.3354170
Zhou ZQian ZReiter MZhang Y(2018)Static Evaluation of Noninterference Using Approximate Model Counting2018 IEEE Symposium on Security and Privacy (SP)10.1109/SP.2018.00052(514-528)Online publication date: May-2018
https://doi.org/10.1109/SP.2018.00052
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten