research-article

Speedup Automatic Program Repair Using Dynamic Software Updating: An Empirical Study

Authors:

Xiaoxing MaAuthors Info & Claims

Internetware '19: Proceedings of the 11th Asia-Pacific Symposium on Internetware

Article No.: 14, Pages 1 - 10

https://doi.org/10.1145/3361242.3361245

Published: 28 October 2019 Publication History

Abstract

A typical generate-and-validate automatic program repair (APR) tool needs to repeatedly run the same test suite to validate each generated patch. This procedure is expensive when the number of patches is huge. Additionally, to scale to large programs, a program repair tool has to consider a small patch space in practice and thus may sacrifice the capability to find potential correct repairs. In this work, we propose to speed up automatic program repair to mitigate the above issues. One the one hand, we found that restarting processes to load patched code consumes the majority of total validation time. This problem is even severe when the program is running in a managed runtime such as Java virtual machine (JVM). On the other hand, dynamic software updating (DSU) can load and execute new code without restarting. To this end, we propose to use DSU techniques to speed up automatic program repair and present an empirical study in this paper. Within our study, DSU can bring up to 66.3 times speedup in comparison with the traditional restart approach. However, DSU may not be able to handle all patches and can also incur unknown side effects that lead to inconsistent validation results. We then further study the feasibility and consistency of applying DSU to speed up APR. Our results show that 1) less than 1% patches cannot be dynamically updated using the builtin DSU ability of JVM, and 2) DSU based validation leads to potentially harmful inconsistency in only 16 of 1,897,518 patches.

References

[1]

Vidroha Debroy and W. Eric Wong. 2010. Using Mutation to Automatically Suggest Fixes for Faulty Programs. In Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation. 65--74.

[2]

Mikhail Dmitriev. 2001. Towards Flexible and Safe Technology for Runtime Evolution of Java Language Applications. In Proceedings of the Workshop on Engineering Complex Object-Oriented Systems for Evolution.

[3]

Tianxiao Gu, Chun Cao, Chang Xu, Xiaoxing Ma, Linghao Zhang, and Jian Lu. 2012. Javelus: A Low Disruptive Approach to Dynamic Software Updates. In Proceedings of 19th the Asia-Pacific Software Engineering Conference. 527--536.

Digital Library

[4]

Tianxiao Gu, Chun Cao, Chang Xu, Xiaoxing Ma, Linghao Zhang, and Jian Lü. 2014. Low-disruptive dynamic updating of Java applications. Information and Software Technology 56, 9 (2014), 1086--1098.

[5]

Tianxiao Gu, Chengnian Sun, Xiaoxing Ma, Jian Lu, and Zhendong Su. 2016. Automatic Runtime Recovery via Error Handler Synthesis. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering. 684--695.

Digital Library

[6]

Jinru Hua, Mengshi Zhang, Kaiyuan Wang, and Sarfraz Khurshid. 2018. Towards Practical Program Repair with On-demand Candidate Generation. In Proceedings of the 40th International Conference on Software Engineering. 12--23.

Digital Library

[7]

Oracle Inc. 2019. Interface Instrumentation. https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html.

[8]

Jiajun Jiang, Yingfei Xiong, Hongyu Zhang, Qing Gao, and Xiangqun Chen. 2018. Shaping Program Repair Space with Existing Patches and Similar Code. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis. 298--309.

Digital Library

[9]

René Just, Darioush Jalali, and Michael D. Ernst. 2014. Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs. In Proceedings of the 2014 International Symposium on Software Testing and Analysis. 437--440.

[10]

Xuan-Bach D. Le, Duc-Hiep Chu, David Lo, Claire Le Goues, and Willem Visser. 2017. S3: Syntax- and Semantic-guided Repair Synthesis via Programming by Examples. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. 593--604.

Digital Library

[11]

Kui Liu, Anil Koyuncu, Dongsun Kim, and Tegawendé F. Bissyandé. 2018. AVATAR: Fixing Semantic Bugs with Fix Patterns of Static Analysis Violations. In Proceedings 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER). 1--12.

[12]

Fan Long and Martin Rinard. 2015. Staged Program Repair with Condition Synthesis. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. 166--178.

Digital Library

[13]

Fan Long, Stelios Sidiroglou-Douskos, and Martin Rinard. 2014. Automatic Runtime Error Repair and Containment via Recovery Shepherding. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation. 227--238.

Digital Library

[14]

Matias Martinez and Martin Monperrus. 2016. ASTOR: A Program Repair Library for Java (Demo). In Proceedings of the 25th International Symposium on Software Testing and Analysis. 441--444.

Digital Library

[15]

Thanh-Toan Nguyen, Quang-Trung Ta, and Wei-Ngan Chin. 2019. Automatic Program Repair Using Formal Verification and Expression Templates. In Verification, Model Checking, and Abstract Interpretation. 70--91.

[16]

A. Jefferson Offutt and W. Michael Craft. 1994. Using compiler optimization techniques to detect equivalent mutants. Software Testing, Verification and Reliability 4, 3 (1994), 131--154.

[17]

Mike Papadakis, Yue Jia, Mark Harman, and Yves Le Traon. 2015. Trivial Compiler Equivalence: A Large Scale Empirical Study of a Simple, Fast and Effective Equivalent Mutant Detection Technique. In Proceedings of the 37th International Conference on Software Engineering - Volume 1. 936--946.

Digital Library

[18]

pitest.org. 2019. Pitest: Real world mutation testing. http://pitest.org.

[19]

Yuhua Qi, Xiaoguang Mao, and Yan Lei. 2013. Efficient automated program repair through fault-recorded testing prioritization. In 2013 IEEE International Conference on Software Maintenance. IEEE, 180--189.

Digital Library

[20]

Yuhua Qi, Xiaoguang Mao, Yan Lei, Ziying Dai, and Chengsong Wang. 2014. The Strength of Random Search on Automated Program Repair. In Proceedings of the 36th International Conference on Software Engineering. 254--265.

Digital Library

[21]

Zichao Qi, Fan Long, Sara Achour, and Martin Rinard. 2015. An Analysis of Patch Plausibility and Correctness for Generate-and-validate Patch Generation Systems. In Proceedings of the 2015 International Symposium on Software Testing and Analysis. 24--36.

Digital Library

[22]

Edward K. Smith, Earl T. Barr, Claire Le Goues, and Yuriy Brun. 2015. Is the Cure Worse Than the Disease? Overfitting in Automated Program Repair. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. 532--543.

Digital Library

[23]

Suriya Subramanian, Michael Hicks, and Kathryn S. McKinley. 2009. Dynamic software updates: A VM-centric approach. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation. 1--12.

[24]

Susumu Tokumoto, Hiroaki Yoshida, Kazunori Sakamoto, and Shinichi Honiden. 2016. MuVM: Higher Order Mutation Analysis Virtual Machine for C. In Proceedings of the 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST). 320--329.

[25]

Roland H. Untch, A. Jefferson Offutt, and Mary Jean Harrold. 1993. Mutation Analysis Using Mutant Schemata. In Proceedings of the 1993 ACM SIGSOFT International Symposium on Software Testing and Analysis. 139--148.

[26]

Bo Wang, Yingfei Xiong, Yangqingwei Shi, Lu Zhang, and Dan Hao. 2017. Faster Mutation Analysis via Equivalence Modulo States. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. 295--306.

Digital Library

[27]

Westley Weimer, ThanhVu Nguyen, Claire Le Goues, and Stephanie Forrest. 2009. Automatically Finding Patches Using Genetic Programming. In Proceedings of the 31st International Conference on Software Engineering. 364--374.

Digital Library

[28]

Ming Wen, Junjie Chen, Rongxin Wu, Dan Hao, and Shing-Chi Cheung. 2018. Context-aware Patch Generation for Better Automated Program Repair. In Proceedings of the 40th International Conference on Software Engineering. 1--11.

Digital Library

[29]

Thomas Würthinger, Christian Wimmer, and Lukas Stadler. 2010. Dynamic code evolution for Java. In Proceedings of the International Conference on the Principles and Practice of Programming in Java. 10--19.

Digital Library

[30]

Thomas Würthinger, Christian Wimmer, and Lukas Stadler. 2013. Unrestricted and Safe Dynamic Code Evolution for Java. Science of Computer Programming 78, 5 (2013), 481--498.

Digital Library

[31]

Qi Xin and Steven P. Reiss. 2017. Identifying Test-suite-overfitted Patches Through Test Case Generation. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. 226--236.

[32]

Q. Xin and S. P. Reiss. 2017. Leveraging syntax-related code for automated program repair. In Proceedings of the 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). 660--670.

[33]

Yingfei Xiong, Xinyuan Liu, Muhan Zeng, Lu Zhang, and Gang Huang. 2018. Identifying Patch Correctness in Test-based Program Repair. In Proceedings of the 40th International Conference on Software Engineering. 789--799.

Digital Library

[34]

Y. Xiong, J. Wang, R. Yan, J. Zhang, S. Han, G. Huang, and L. Zhang. 2017. Precise Condition Synthesis for Program Repair. In Proceedings of the 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). 416--426.

[35]

J. Xuan, M. Martinez, F. DeMarco, M. ClÃľment, S. L. Marcote, T. Durieux, D. Le Berre, and M. Monperrus. 2017. Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs. IEEE Transactions on Software Engineering 43, 1 (Jan 2017), 34--55.

Digital Library

[36]

Jinqiu Yang, Alexey Zhikhartsev, Yuefei Liu, and Lin Tan. 2017. Better Test Cases for Better Automated Program Repair. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. 831--841.

Digital Library

Cited By

Xiao YYang CWang BXiong Y(2024)Accelerating Patch Validation for Program Repair With Interception-Based Execution SchedulingIEEE Transactions on Software Engineering10.1109/TSE.2024.335996950:3(618-635)Online publication date: Mar-2024
https://doi.org/10.1109/TSE.2024.3359969
Xiao YYang CWang BXiong Y(2023)ExpressAPR: Efficient Patch Validation for Java Automated Program Repair Systems2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)10.1109/ASE56229.2023.00012(2038-2041)Online publication date: 11-Sep-2023
https://doi.org/10.1109/ASE56229.2023.00012
Cao HMeng YShi JLi LLiao TZhao C(2020)A Survey on Automatic Bug Fixing2020 6th International Symposium on System and Software Reliability (ISSSR)10.1109/ISSSR51244.2020.00029(122-131)Online publication date: Oct-2020
https://doi.org/10.1109/ISSSR51244.2020.00029
Show More Cited By

Index Terms

Speedup Automatic Program Repair Using Dynamic Software Updating: An Empirical Study
1. Software and its engineering
  1. Software creation and management

Recommendations

Whole-program dynamic software updating
Automating object transformations for dynamic software updating
OOPSLA '12: Proceedings of the ACM international conference on Object oriented programming systems languages and applications

Dynamic software updating (DSU) systems eliminate costly downtime by dynamically fixing bugs and adding features to executing programs. Given a static code patch, most DSU systems construct runtime code changes automatically. However, a dynamic update ...
Automatic Program Repair with Improved Transformer
FAIML '23: Proceedings of the 2023 International Conference on Frontiers of Artificial Intelligence and Machine Learning

The rise of Deep Learning (DL) has led researchers to explore techniques for Automatic Program Repair (APR) using DL. However, existing methods for APR suffer from issues such as small datasets, low accuracy, and long execution times. In this paper, we ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

Internetware '19: Proceedings of the 11th Asia-Pacific Symposium on Internetware

October 2019

179 pages

ISBN:9781450377010

DOI:10.1145/3361242

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 October 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

Internetware '19

Internetware '19: The 11th Asia-Pacific Symposium on Internetware

October 28 - 29, 2019

Fukuoka, Japan

Acceptance Rates

Internetware '19 Paper Acceptance Rate 20 of 35 submissions, 57%;

Overall Acceptance Rate 55 of 111 submissions, 50%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
168
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xiao YYang CWang BXiong Y(2024)Accelerating Patch Validation for Program Repair With Interception-Based Execution SchedulingIEEE Transactions on Software Engineering10.1109/TSE.2024.335996950:3(618-635)Online publication date: Mar-2024
https://doi.org/10.1109/TSE.2024.3359969
Xiao YYang CWang BXiong Y(2023)ExpressAPR: Efficient Patch Validation for Java Automated Program Repair Systems2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)10.1109/ASE56229.2023.00012(2038-2041)Online publication date: 11-Sep-2023
https://doi.org/10.1109/ASE56229.2023.00012
Cao HMeng YShi JLi LLiao TZhao C(2020)A Survey on Automatic Bug Fixing2020 6th International Symposium on System and Software Reliability (ISSSR)10.1109/ISSSR51244.2020.00029(122-131)Online publication date: Oct-2020
https://doi.org/10.1109/ISSSR51244.2020.00029
Ahmed BLee SSu MZakari A(2020)Dynamic software updating: a systematic mapping studyIET Software10.1049/iet-sen.2019.020114:5(468-481)Online publication date: 2-Sep-2020
https://doi.org/10.1049/iet-sen.2019.0201

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents