Article

Shadow symbolic execution for better testing of evolving software

Authors:

Cristian Cadar,

Hristina PalikarevaAuthors Info & Claims

ICSE Companion 2014: Companion Proceedings of the 36th International Conference on Software Engineering

Pages 432 - 435

https://doi.org/10.1145/2591062.2591104

Published: 31 May 2014 Publication History

Abstract

In this idea paper, we propose a novel way for improving the testing of program changes via symbolic execution. At a high-level, our technique runs two different program versions in the same symbolic execution instance, with the old version effectively shadowing the new one. In this way, the technique can exploit precise dynamic value information to effectively drive execution toward the behaviour that has changed from one version to the next. We discuss the main challenges and opportunities of this approach in terms of pruning and prioritising path exploration, mapping elements across versions, and sharing common symbolic state between versions.

References

[1]

D. Babi´ c, L. Martignoni, S. McCamant, and D. Song. Statically-directed dynamic automated test generation. In ISSTA’11.

Digital Library

[2]

M. Böhme, B. C. d. S. Oliveira, and A. Roychoudhury. Partition-based regression verification. In ICSE’13.

[3]

M. Böhme, B. C. d. S. Oliveira, and A. Roychoudhury. Regression tests to expose change interaction errors. In ESEC/FSE’13.

[4]

C. Cadar and K. Sen. Symbolic execution for software testing: three decades later. CACM, 56(2):82–90, 2013.

Digital Library

[5]

E. Clarke, D. Kroening, and F. Lerda. A tool for checking ANSI-C programs. In TACAS’04.

[6]

Coverity software. http://www.coverity.com.

[7]

M. d’Amorim, S. Lauterburg, and D. Marinov. Delta execution for eﬃcient state-space exploration of object-oriented programs. In ISSTA’07.

Digital Library

[8]

Fortify software. http://www.fortify.com.

[9]

B. Godlin and O. Strichman. Regression verification. In DAC’09.

Digital Library

[10]

Z. Gu, E. T. Barr, D. J. Hamilton, and Z. Su. Has the bug really been fixed? In ICSE’10.

Digital Library

[11]

P. Hosek and C. Cadar. Safe software updates via multi-version execution. In ICSE’13.

Digital Library

[12]

C. H. P. Kim, S. Khurshid, and D. Batory. Shared execution for eﬃciently testing product lines. In ISSRE’12.

[13]

M. Kim and D. Notkin. Program element matching for multi-version program analyses. In MSR’06.

Digital Library

[14]

J. C. King. Symbolic execution and program testing. CACM, 19(7):385–394, 1976.

Digital Library

[15]

S. K. Lahiri, K. L. McMillan, R. Sharma, and C. Hawblitzel. Differential assertion checking. In ESEC/FSE’13.

Digital Library

[16]

K.-K. Ma, Y. P. Khoo, J. S. Foster, and M. Hicks. Directed symbolic execution. In SAS’11.

Digital Library

[17]

P. D. Marinescu and C. Cadar. High-coverage symbolic patch testing. In SPIN’12.

Digital Library

[18]

P. D. Marinescu and C. Cadar. KATCH: High-coverage testing of software patches. In ESEC/FSE’13.

Digital Library

[19]

M. Maurer and D. Brumley. TACHYON: Tandem execution for eﬃcient live patch testing. In USENIX Security’12.

Digital Library

[20]

D. Notkin. Longitudinal program analysis. In PASTE’02.

Digital Library

[21]

S. Person, M. B. Dwyer, S. Elbaum, and C. S. Pˇ asˇ areanu. Differential symbolic execution. In FSE’08.

Digital Library

[22]

S. Person, G. Yang, N. Rungta, and S. Khurshid. Directed incremental symbolic execution. In PLDI’11.

Digital Library

[23]

R. Santelices, P. K. Chittimalli, T. Apiwattanapong, A. Orso, and M. J. Harrold. Test-suite augmentation for evolving software. In ASE’08.

Digital Library

[24]

K. Taneja, T. Xie, N. Tillmann, and J. de Halleux. eXpress: guided path exploration for eﬃcient regression test generation. In ISSTA’11.

Digital Library

[25]

J. Tucek, W. Xiong, and Y. Zhou. Eﬃcient online validation with delta execution. In ASPLOS’09.

Digital Library

[26]

W. Visser, C. S. Pasareanu, and S. Khurshid. Test input generation with Java PathFinder. In ISSTA’04.

Digital Library

[27]

Z. Xu and G. Rothermel. Directed test suite augmentation. In ASPEC’09.

Digital Library

[28]

Z. Yin, D. Yuan, Y. Zhou, S. Pasupathy, and L. Bairavasundaram. How do fixes become bugs? In ESEC/FSE’11.

Digital Library

Cited By

Zheng MShi QLiu XXu XYu LLiu CWei GZhang X(2024)ParDiff: Practical Static Differential Analysis of Network Protocol ParsersProceedings of the ACM on Programming Languages10.1145/36498548:OOPSLA1(1208-1234)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3649854
Wang AFeng NChechik M(2023)Code-Level Functional Equivalence Checking of Annotative Software Product LinesProceedings of the 27th ACM International Systems and Software Product Line Conference - Volume A10.1145/3579027.3608978(64-75)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1145/3579027.3608978
Daniel LBardin SRezk T(2023)Binsec/Rel: Symbolic Binary Analyzer for Security with Applications to Constant-Time and Secret-ErasureACM Transactions on Privacy and Security10.1145/356303726:2(1-42)Online publication date: 14-Apr-2023
https://dl.acm.org/doi/10.1145/3563037
Show More Cited By

Index Terms

Shadow symbolic execution for better testing of evolving software

Recommendations

Shadow Symbolic Execution for Testing Software Patches

While developers are aware of the importance of comprehensively testing patches, the large effort involved in coming up with relevant test cases means that such testing rarely happens in practice. Furthermore, even when test cases are written to cover ...
Shadow Symbolic Execution with Java PathFinder

Regression testing ensures that a software system when it evolves still performs correctly and that the changes introduce no unintended side-effects. However, the creation of regression test cases that show divergent behavior needs a lot of effort. A ...
Symbolic execution and program testing

This paper describes the symbolic execution of programs. Instead of supplying the normal inputs to a program (e.g. numbers) one supplies symbols representing arbitrary values. The execution proceeds as in a normal execution except that values may be ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE Companion 2014: Companion Proceedings of the 36th International Conference on Software Engineering

May 2014

741 pages

ISBN:9781450327688

DOI:10.1145/2591062

General Chair:
Pankaj Jalote
IIIT-Delhi, India
,
Program Chairs:
Lionel Briand
University of Luxembourg, Luxembourg
,
André van der Hoek
University of California, Irvine, USA

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

TCSE: IEEE Computer Society's Tech. Council on Software Engin.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 May 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICSE '14

Sponsor:

SIGSOFT

ICSE '14: 36th International Conference on Software Engineering

May 31 - June 7, 2014

Hyderabad, India

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
243
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zheng MShi QLiu XXu XYu LLiu CWei GZhang X(2024)ParDiff: Practical Static Differential Analysis of Network Protocol ParsersProceedings of the ACM on Programming Languages10.1145/36498548:OOPSLA1(1208-1234)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3649854
Wang AFeng NChechik M(2023)Code-Level Functional Equivalence Checking of Annotative Software Product LinesProceedings of the 27th ACM International Systems and Software Product Line Conference - Volume A10.1145/3579027.3608978(64-75)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1145/3579027.3608978
Daniel LBardin SRezk T(2023)Binsec/Rel: Symbolic Binary Analyzer for Security with Applications to Constant-Time and Secret-ErasureACM Transactions on Privacy and Security10.1145/356303726:2(1-42)Online publication date: 14-Apr-2023
https://dl.acm.org/doi/10.1145/3563037
Badihi SAkinotcho FLi YRubin JDevanbu PCohen MZimmermann T(2020)ARDiff: scaling program equivalence checking via iterative abstraction and refinement of common codeProceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3368089.3409757(13-24)Online publication date: 8-Nov-2020
https://dl.acm.org/doi/10.1145/3368089.3409757
Feng NMora FHui VChechik MGrundy JLe Goues CLo D(2020)Scaling client-specific equivalence checking via impact boundary searchProceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering10.1145/3324884.3416634(734-745)Online publication date: 21-Dec-2020
https://dl.acm.org/doi/10.1145/3324884.3416634
Daniel LBardin SRezk T(2020)Binsec/Rel: Efficient Relational Symbolic Execution for Constant-Time at Binary-Level2020 IEEE Symposium on Security and Privacy (SP)10.1109/SP40000.2020.00074(1021-1038)Online publication date: May-2020
https://doi.org/10.1109/SP40000.2020.00074
Yang GFilieri ABorges MClun DWen J(2019)Advances in Symbolic Execution10.1016/bs.adcom.2018.10.002(225-287)Online publication date: 2019
https://doi.org/10.1016/bs.adcom.2018.10.002
Mora FLi YRubin JChechik MHuchard MKästner CFraser G(2018)Client-specific equivalence checkingProceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering10.1145/3238147.3238178(441-451)Online publication date: 3-Sep-2018
https://dl.acm.org/doi/10.1145/3238147.3238178
Kuchta TPalikareva HCadar C(2018)Shadow Symbolic Execution for Testing Software PatchesACM Transactions on Software Engineering and Methodology10.1145/320895227:3(1-32)Online publication date: 25-Sep-2018
https://dl.acm.org/doi/10.1145/3208952
Xin QReiss SBultan TSen K(2017)Identifying test-suite-overfitted patches through test case generationProceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3092703.3092718(226-236)Online publication date: 10-Jul-2017
https://dl.acm.org/doi/10.1145/3092703.3092718
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten