research-article

ARDiff: scaling program equivalence checking via iterative abstraction and refinement of common code

Authors:

Faridah Akinotcho,

Julia RubinAuthors Info & Claims

ESEC/FSE 2020: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Pages 13 - 24

https://doi.org/10.1145/3368089.3409757

Published: 08 November 2020 Publication History

Abstract

Equivalence checking techniques help establish whether two versions of a program exhibit the same behavior. The majority of popular techniques for formally proving/refuting equivalence relies on symbolic execution – a static analysis approach that reasons about program behaviors in terms of symbolic input variables. Yet, symbolic execution is difficult to scale in practice due to complex programming constructs, such as loops and non-linear arithmetic.

This paper proposes an approach, named ARDiff, for improving the scalability of symbolic-execution-based equivalence checking techniques when comparing syntactically-similar versions of a program, e.g., for verifying the correctness of code upgrades and refactoring. Our approach relies on a set of novel heuristics to determine which parts of the versions’ common code can be effectively pruned during the analysis, reducing the analysis complexity without sacrificing its effectiveness. Furthermore, we devise a new equivalence checking benchmark, extending existing benchmarks with a set of real-life methods containing complex math functions and loops. We evaluate the effectiveness and efficiency of ARDiff on this benchmark and show that it outperforms existing method-level equivalence checking techniques by solving 86% of all equivalent and 55% of non-equivalent cases, compared with 47% to 69% for equivalent and 38% to 52% for non-equivalent cases in related work.

References

[1]

[n.d.]. Github. https://github.com.

[2]

[n.d.]. Refactoring GURU. https://refactoring.guru/remove-assignments-toparameters.

[3]

2019. ASM-DefUse. Software Analysis Experimentation Group ( 2019 ). https: //github.com/saeg/asm-defuse.

[4]

2020. ARDif. https:// www.doi.org/ 10.17605/ OSF.IO/ CHM2K.

[5]

2020. Supplementary Materials. https:// resess.github.io/ PaperAppendices/ ARDif/.

[6]

John Backes, Suzette Person, Neha Rungta, and Oksana Tkachuk. 2013. Regression Verification Using Impact Summaries. In Proc. of SPIN Workshop on Model Checking of Software.

[7]

Eric Bruneton, Romain Lenglet, and Thierry Coupaye. 2002. ASM: a Code Manipulation Tool to Implement Adaptable Systems. Adaptable and Extensible Component Systems 30, 19 ( 2002 ).

[8]

Jerry R Burch and David L Dill. 1994. Automatic Verification of Pipelined Microprocessor Control. In Proc. of the International Conference on Computer Aided Verification (CAV). 68-80.

[9]

Cristian Cadar, Daniel Dunbar, Dawson R Engler, et al. 2008. KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs. In Proc. of the Symposium on Operating Systems Design and Implementation (OSDI). 209-224.

[10]

Cristian Cadar and Hristina Palikareva. 2014. Shadow Symbolic Execution for Better Testing of Evolving Software. In Proc. of the International Conference on Software Engineering (ICSE). 432-435.

Digital Library

[11]

Sagar Chaki, Edmund Clarke, Natasha Sharygina, and Nishant Sinha. 2008. Veriifcation of Evolving Software via Component Substitutability Analysis. In Formal Methods in System Design. 235-266.

Digital Library

[12]

Berkeley Churchill, Oded Padon, Rahul Sharma, and Alex Aiken. 2019. Semantic Program Alignment for Equivalence Checking. In Proc. of Programming Language Design and Implementation (PLDI).

Digital Library

[13]

Edmund Clarke, Orna Grumberg, Somesh Jha, Yuan Lu, and Helmut Veith. 2000. Counterexample-guided Abstraction Refinement. In Proc. of the International Conference on Computer Aided Verification (CAV). 154-169.

[14]

Stephen A Cook. 1971. The Complexity of Theorem-proving Procedures. In Proc. of the Symposium on Theory of Computing (STOC).

Digital Library

[15]

Martin Davis. 1973. Hilbert's Tenth Problem is Unsolvable. The American Mathematical Monthly 80, 3 ( 1973 ), 233-269.

[16]

Martin Davis. 1973. Hilbert's Tenth Problem is Unsolvable. The American Mathematical Monthly 80, 3 ( 1973 ), 233-269.

[17]

Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An Eficient SMT Solver. In Proc. of the International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS). 337-340.

[18]

Leonardo De Moura and Nikolaj Bjørner. 2011. Satisfiability Modulo Theories: Introduction and Applications. Commun. ACM 54, 9 ( 2011 ), 69-77.

[19]

Peter Dinges and Gul Agha. 2014. Solving Complex Path Conditions Through Heuristic Search on Induced Polytopes. In Proc. of the International Symposium on Foundations of Software Engineering (FSE). 425-436.

Digital Library

[20]

Jean-Rémy Falleri, Floréal Morandat, Xavier Blanc, Matias Martinez, and Martin Monperrus. 2014. Fine-grained and Accurate Source Code Diferencing. In Proc. of the International Conference on Automated Software Engineering (ASE). 313-324.

[21]

Grigory Fedyukovich, Arie Gurfinkel, and Nastasha Sharygina. 2016. PropertyDirected Equivalence via Abstract Simulation. In Proc. of the International Conference on Computer Aided Verification. 433-453.

[22]

Dennis Felsing, Sarah Grebing, Vladimir Klebanov, Philipp Rümmer, and Mattias Ulbrich. 2014. Automating Regression Verification. In Proc. of the International Conference on Automated Software Engineering (ASE). 349-360.

Digital Library

[23]

Jeanne Ferrante, Karl J Ottenstein, and Joe D Warren. 1987. The Program Dependence Graph and Its Use in Optimization. Transactions on Programming Languages and Systems (TOPLAS) 9, 3 ( 1987 ), 319-349.

[24]

M. Fowler. [n.d.]. Refactoring home page. https://refactoring.com/catalog/.

[25]

Benny Godlin and Ofer Strichman. 2008. Inference Rules for Proving the Equivalence of Recursive Procedures. Acta Informatica 45, 6 ( 2008 ), 403-439.

Digital Library

[26]

Benny Godlin and Ofer Strichman. 2013. Regression verification: proving the equivalence of similar programs. Software Testing, Verification and Reliability 23, 3 ( 2013 ), 241-258.

[27]

Joseph Y Halpern. 1991. Presburger Arithmetic with Unary Predicates is Π 1-1 Complete. The Journal of Symbolic Logic 56, 2 ( 1991 ), 637-642.

Digital Library

[28]

James C King. 1976. Symbolic Execution and Program Testing. Commun. ACM 19, 7 ( 1976 ), 385-394.

[29]

Daniel Kroening and Ofer Strichman. 2008. Equality Logic and Uninterpreted Functions. In Decision Procedures. 59-80.

[30]

Andreas Kuehlmann and Florian Krohm. 1997. Equivalence Checking Using Cuts and Heaps. In Proc. of the Design Automation Conference (DAC). 263-268.

[31]

Shuvendu K Lahiri, Chris Hawblitzel, Ming Kawaguchi, and Henrique Rebêlo. 2012. Symdif: A Language-agnostic Semantic Dif Tool for Imperative Programs. In Proc. of the International Conference on Computer Aided Verification (CAD). 712-717.

[32]

Xin Li, Yongjuan Liang, Hong Qian, Yi-Qi Hu, Lei Bu, Yang Yu, Xin Chen, and Xuandong Li. 2016. Symbolic Execution of Complex Program Driven by Machine Learning-Based Constraint Solving. In Proc. of the International Conference on Automated Software Engineering (ASE). 554-559.

Digital Library

[33]

Yu-Seung Ma and Jef Ofutt. [n.d.]. Description of muJava's Method-level Mutation Operators. ([n. d.]).

[34]

Federico Mora, Yi Li, Julia Rubin, and Marsha Chechik. 2018. Client-specific Equivalence Checking. In Proc. of the International Conference on Automated Software Engineering (ASE). 441-451.

Digital Library

[35]

Dmitry Mordvinov and Grigory Fedyukovich. 2019. Property Directed Inference of Relational Invariants. In Proc. of Formal Methods in Computer Aided Design (FMCAD). 152-160.

[36]

Hristina Palikareva, Tomasz Kuchta, and Cristian Cadar. 2016. Shadow of a Doubt: Testing for Divergences Between Software Versions. In Proc. of the International Conference on Software Engineering (ICSE). 1181-1192.

Digital Library

[37]

Corina S Păsăreanu and Neha Rungta. 2010. Symbolic PathFinder: Symbolic Execution of Java bytecode. In Proc. of the International Conference on Automated Software Engineering (ASE). 179-180.

Digital Library

[38]

Suzette Person, Matthew B Dwyer, Sebastian Elbaum, and Corina S Păsăreanu. 2008. Diferential Symbolic Execution. In Proc. of the International Symposium on Foundations of Software Engineering (FSE).

[39]

Lauren Pick, Grigory Fedyukovich, and Aarti Gupta. 2018. Exploiting Synchrony and Symmetry in Relational Verification. In Proc. of International Conference on Computer Aided Verification (CAV). 164-182.

[40]

William H Press, Saul A Teukolsky, William T Vetterling, and Brian P Flannery. 2007. Numerical Recipes 3rd Edition: The Art of Scientific Computing.

Digital Library

[41]

David A Ramos and Dawson R Engler. 2011. Practical, Low-efort Equivalence Verification of Real Code. In Proc. of the International Conference on Computer Aided Verification (CAV). 669-685.

[42]

Ondrej Sery, Girogory Fedyukovich, and Natasha Sharygina. 2012. Incremental Upgrade Checking by Means of Interpolation-Based Function Summaries. In Proc. of the Formal Methods in Computer-Aided Design Conference. 114-427.

[43]

Ron Shemer, Arie Gurfinkel, Sharon Shoham, and Yakir Vizel. 2019. Property Directed Self Composition. In Proc. of International Conference on Computer Aided Verification (CAV). 161-179.

[44]

Matheus Souza, Mateus Borges, Marcelo d'Amorim, and Corina S Păsăreanu. 2011. CORAL: Solving Complex Constraints for Symbolic Pathfinder. In Proc. of NASA Formal Methods Symposium. 359-374.

[45]

Anna Trostanetski, Orna Grumberg, and Daniel Kroening. 2017. Modular Demand-driven Analysis of Semantic Diference for Program Versions. In Proc. of the International Static Analysis Symposium (SAS). 405-427.

[46]

Christopher A Welty. 1997. Augmenting Abstract Syntax Trees for Program Understanding. In Proc. of the International Conference Automated Software Engineering (ICSE). 126-133.

[47]

Xusheng Xiao, Tao Xie, Nikolai Tillmann, and Jonathan De Halleux. 2011. Precise Identification of Problems for Structural Test Generation. In Proc. of the International Conference on Software Engineering (ICSE). 611-620.

Digital Library

Cited By

Sarker LBultan TFilkov VRay BZhou M(2024)Quantitative Symbolic Non-Equivalence AnalysisProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695324(2452-2453)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695324
Postolski IBraberman VGarbervetsky DUchitel Sd'Amorim M(2024)Verification of Programs with Common FragmentsCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663783(487-491)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663783
Fan ZRuan HMechtaev SRoychoudhury AChristakis MPradel M(2024)Oracle-Guided Program Selection from Large Language ModelsProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680308(628-640)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680308
Show More Cited By

Index Terms

ARDiff: scaling program equivalence checking via iterative abstraction and refinement of common code
1. Software and its engineering
  1. Software creation and management
    1. Software post-development issues
      1. Software evolution

Recommendations

Semantic program alignment for equivalence checking
PLDI 2019: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation

We introduce a robust semantics-driven technique for program equivalence checking. Given two functions we find a trace alignment over a set of concrete executions of both programs and construct a product program particularly amenable to checking ...
Client-specific equivalence checking
ASE '18: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering

Software is often built by integrating components created by different teams or even different organizations. With little understanding of changes in dependent components, it is challenging to maintain correctness and robustness of the entire system. In ...
Verifying Parallel Code After Refactoring Using Equivalence Checking

To take advantage of multi-core systems, programmers are replacing sequential software with parallel software. Software engineers often avoid writing their parallel software from scratch and prefer refactoring their legacy application, either manually ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEC/FSE 2020: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

November 2020

1703 pages

ISBN:9781450370431

DOI:10.1145/3368089

General Chair:
Prem Devanbu
University of California at Davis, USA
,
Program Chairs:
Myra Cohen
Iowa State University, USA
,
Thomas Zimmermann
Microsoft Research, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 November 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tags

Qualifiers

Research-article

Conference

ESEC/FSE '20

Sponsor:

SIGSOFT

ESEC/FSE '20: 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

November 8 - 13, 2020

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
435
Total Downloads

Downloads (Last 12 months)98
Downloads (Last 6 weeks)18

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sarker LBultan TFilkov VRay BZhou M(2024)Quantitative Symbolic Non-Equivalence AnalysisProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695324(2452-2453)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695324
Postolski IBraberman VGarbervetsky DUchitel Sd'Amorim M(2024)Verification of Programs with Common FragmentsCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663783(487-491)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663783
Fan ZRuan HMechtaev SRoychoudhury AChristakis MPradel M(2024)Oracle-Guided Program Selection from Large Language ModelsProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680308(628-640)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680308
Zheng MShi QLiu XXu XYu LLiu CWei GZhang X(2024)ParDiff: Practical Static Differential Analysis of Network Protocol ParsersProceedings of the ACM on Programming Languages10.1145/36498548:OOPSLA1(1208-1234)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3649854
Chen QYu CLiu RZhang CWang YWang KSu TWang L(2024)Evaluating the Effectiveness of Deep Learning Models for Foundational Program Analysis TasksProceedings of the ACM on Programming Languages10.1145/36498298:OOPSLA1(500-528)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3649829
Zang ZThimmaiah AGligoric MRoychoudhury APaiva AAbreu RStorey M(2024)JOG: Java JIT Peephole Optimizations and Tests from PatternsProceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings10.1145/3639478.3640040(11-15)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639478.3640040
Glock JRoychoudhury APaiva AAbreu RStorey M(2024)Aiding Developer Understanding of Software Changes via Symbolic Execution-based Semantic DifferencingProceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings10.1145/3639478.3639783(142-144)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639478.3639783
Luo CMeng WWang SRoychoudhury APaiva AAbreu RStorey M(2024)Strengthening Supply Chain Security with Fine-grained Safe Patch IdentificationProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639104(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639104
Glock JPichler JPinzger M(2024)PASDAJournal of Systems and Software10.1016/j.jss.2024.112037213:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.jss.2024.112037
Semujju SLiu FHuang HXiang YYan XHao Z(2024)Knowledge transfer based many-objective approach for finding bugs in multi-path loopsComplex & Intelligent Systems10.1007/s40747-023-01323-wOnline publication date: 24-Jan-2024
https://doi.org/10.1007/s40747-023-01323-w
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents