article

Automatic fault localization for client-side JavaScript

Authors:

Frolin S. Ocariza,

Karthik Pattabiraman,

Ali MesbahAuthors Info & Claims

Software Testing, Verification & Reliability, Volume 26, Issue 1

Pages 69 - 88

https://doi.org/10.1002/stvr.1576

Published: 01 January 2016 Publication History

Abstract

JAVASCRIPT is a scripting language that plays a prominent role in web applications today. It is dynamic, loosely typed and asynchronous and is extensively used to interact with the Document Object Model DOM at runtime. All these characteristics make JAVASCRIPT code error-prone; unfortunately, JAVASCRIPT fault localization remains a tedious and mainly manual task. Despite these challenges, the problem has received very limited research attention. This paper proposes an automated technique to localize JAVASCRIPT faults based on dynamic analysis, tracing and backward slicing of JAVASCRIPT code. This technique is capable of handling features of JAVASCRIPT code that have traditionally been difficult to analyse, including eval, anonymous functions and minified code. The approach is implemented in an open source tool called AUTOFLOX, and evaluation results indicate that it is capable of 1 automatically localizing DOM-related JAVASCRIPT faults with high accuracy over 96% and no false-positives and 2 isolating JAVASCRIPT faults in production websites and actual bugs from real-world web applications. Copyright © 2015John Wiley & Sons, Ltd.

References

[1]

Marchetto A, Tonella P, Ricca F. State-based testing of AJAX web applications. In Proceedings of the International Conference on Software Testing, Verification and Validation ICST, IEEE Computer Society: Lillehammer, Norway, 2008; pp.121-130.

[2]

Mesbah A, <familyNamePrefix>van</familyNamePrefix>Deursen A, Roest D. Invariant-based automatic testing of modern web applications. IEEE Transactions on Software Engineering TSE 2012; Volume 38 Issue 1: pp.35-53.

[3]

Pattabiraman K, Zorn B. DoDOM: leveraging DOM invariants for web 2.0 application robustness testing. In Proceedings of the International Symposium on Software Reliability Engineering ISSRE, IEEE Computer Society: San Jose, CA, USA, 2010; pp.191-200.

[4]

Artzi S, Dolby J, Jensen SH, Møller A, Tip F. A framework for automated testing of JavaScript web applications. In Proceedings of the International Conference on Software Engineering ICSE, ACM: Honolulu, HI, USA, 2011; pp.571-580.

[5]

Vessey I. Expertise in debugging computer programs: a process analysis. International Journal of Man-Machine Studies 1985; Volume 23 Issue 5: pp.459-494.

[6]

Jones J, Harrold M. Empirical evaluation of the tarantula automatic fault-localization technique. In Proceedings of the International Conference on Automated Software Engineering ASE, ACM: Long Beach, CA, USA, 2005; pp.273-282.

[7]

Ocariza F, Pattabiraman K, Zorn BG. JavaScript errors in the wild: an empirical study. In Proceedings of the International Symposium on Software Reliability Engineering ISSRE, IEEE Computer Society: Hiroshima, Japan, 2011; pp.100-109.

[8]

Ocariza F, Bajaj K, Pattabiraman K, Mesbah A. An empirical study of client-side JavaScript bugs. In Proceedings of the International Symposium on Empirical Software Engineering and Measurement ESEM, IEEE Computer Society: Baltimore, MD, USA, 2013; pp.55-64.

[9]

Abreu R, Zoeteweij P, Gemund AJC. Spectrum-based multiple fault localization. In Proceedings of the International Conference on Automated Software Engineering ASE, IEEE Computer Society: Auckland, New Zealand, 2009; pp.88-99.

[10]

Agrawal H, Horgan JR, London S, Wong WE. Fault localization using execution slices and dataflow tests. In Proceedings of the International Symposium on Software Reliability Engineering ISSRE, IEEE: Toulouse, France, 1995; pp.143-151.

[11]

Cleve H, Zeller A. Locating causes of program failures. In Proceedings of the International Conference on Software Engineering ICSE, ACM: St. Louis, MO, USA, 2005; pp.342-351.

[12]

Ocariza F, Pattabiraman K, Mesbah A. AutoFLox: an automatic fault localizer for client-side JavaScript. In Proceedings of the International Conference on Software Testing, Verification and Validation ICST, IEEE Computer Society: Montreal, QC, Canada, 2012; pp.31-40.

[13]

Richards G, Hammer C, Burg B, Vitek J. The eval that men do: a large-scale study of the use of eval in JavaScript applications. In Proceedings of the European Conference on Object-Oriented Programming ECOOP, Lancaster, UK, 2011; pp.52-78.

[14]

Rhino. Available from: "https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Rhino" {last accessed 28 October 2011}.

[15]

JSBeautifier. Available from: "http://www.jsbeautifier.org/" {last accessed 1 November 2014}.

[16]

Mesbah A, <familyNamePrefix>van</familyNamePrefix>Deursen A, Lenselink S. Crawling Ajax-based web applications through dynamic analysis of user interface state changes. ACM Transactions on the Web TWEB 2012; Volume 6 Issue 1: pp.53-82.

[17]

Andrica S, Candea G. WaRR: high fidelity web application recording and replaying. In Proceedings of the International Conference on Dependable Systems and Networks DSN, Hong Kong, China, 2011; pp.403-410.

[18]

Mickens J, Elson J, Howell J. Mugshot: deterministic capture and replay for JavaScript applications. In Proceedings of the USENIX Conference on Networked Systems Design and Implementation NSDI, San Jose, CA, USA, 2010; pp.159-174.

[19]

Selenium. Available from: "http://seleniumhq.org" {last accessed 28 October 2011}.

[20]

Eclipse IDE. Available from: "http://www.eclipse.org/" {last accessed 1 November 2014}.

[21]

Groeneveld F, Mesbah A, <familyNamePrefix>van</familyNamePrefix>Deursen A. Automatic invariant detection in dynamic web applications. Technical Report TUD-SERG-2010-037, Delft University of Technology, 2010.

[22]

Guarnieri S, Livshits B. Gatekeeper: mostly static enforcement of security and reliability policies for JavaScript code. In Proceedings of the USENIX Security Symposium SSYM, ACM: Montreal, QC, Canada, 2009; pp.151-168.

[23]

Guha A, Krishnamurthi S, Jim T. Using static analysis for AJAX intrusion detection. In Proceedings of the International Conference on the World Wide Web WWW, Madrid, Spain, 2009; pp.561-570.

Digital Library

[24]

Zheng Y, Bao T, Zhang X. Statically locating web application bugs caused by asynchronous calls. In Proceedings of the International Conference on the World Wide Web WWW, ACM: Hyderabad, India, 2011; pp.805-814.

[25]

Bae S, Cho H, Lim I, Ryu S. SAFEWAPI: web API misuse detector for web applications. Proceedings of the International Symposium on Foundations of Software Engineering FSE, ACM: Hong Kong, China, 2014; pp.507-517.

[26]

Jensen SH, Madsen M, Møller A. Modeling the HTML DOM and browser API in static analysis of JavaScript web applications. In Proceedings of the European Software Engineering Conference and Symposium on the Foundations of Software Engineering ESEC/FSE, ACM: Szeged, Hungary, 2011; pp.59-69.

[27]

Mirshokraie S, Mesbah A, Pattabiraman K. JSeft: automated JavaScript unit test generation. In Proceedings of the International Conference on Software Testing, Verification and Validation ICST, IEEE Computer Society: Graz, Austria, 2015.

[28]

Mirshokraie S, Mesbah A. JSART: JavaScript assertion-based regression testing. In Proceedings of the International Conference on Web Engineering ICWE, Springer: Berlin, Germany, 2012; pp.238-252.

Digital Library

[29]

Mirshokraie S, Mesbah A, Pattabiraman K. Efficient JavaScript mutation testing. In Proceedings of the International Conference on Software Testing, Verification and Validation ICST, IEEE Computer Society: Luxembourg, Luxembourg, 2013; pp.74-83.

Digital Library

[30]

Alimadadi S, Sequeira S, Mesbah A, Pattabiraman K. Understanding JavaScript event-based interactions. In Proceedings of the International Conference on Software Engineering ICSE, Hyderabad, India, 2014; pp.367-377.

Digital Library

[31]

Sen K, Kalasapur S, Brutch T, Gibbs S. Jalangi: a selective record-replay and dynamic analysis framework for JavaScript. In Proceedings of the International Symposium on Foundations of Software Engineering FSE, ACM: Saint Petersburg, Russia, 2013; pp.488-498.

[32]

Burg B, Bailey R, Ko AJ, Ernst MD. Interactive record/replay for web application debugging. In Proceedings of the ACM Symposium on User Interface Software and Technology UIST, ACM: St. Andrews, UK, 2013; pp.473-484.

Digital Library

[33]

Yildiz A, Aktemur B, Sozer H. Rumadai: a plug-in to record and replay client-side events of web sites with dynamic content. In Proceedings of the Workshop on Developing Tools as Plug-ins TOPI, IEEE: Zurich, Switzerland, 2012; pp.88-89.

[34]

Firebug. Available from: "http://getfirebug.com" {last accessed 28 October 2011}.

[35]

Bandyopadhyay A, Ghosh S. Tester feedback driven fault localization. In Proceedings of the International Conference on Software Testing, Verification and Validation ICST, IEEE: Montreal, QC, Canada, 2012; pp.41-50.

[36]

Zhou J, Zhang H, Lo D. Where should the bugs be fixed? More accurate information retrieval-based bug localization based on bug reports. In Proceedings of the International Conference on Software Engineering ICSE, IEEE: Zurich, Switzerland, 2012; pp.14-24.

[37]

Shu G, Sun B, Podgurski A, Cao F. Mfl: method-level fault localization with causal inference. In Proceedings of the International Conference on Software Testing, Verification and Validation ICST, IEEE: Luxembourg, Luxembourg, 2013; pp.124-133.

Digital Library

[38]

Chen MY, Kiciman E, Fratkin E, Fox A, Brewer E. Pinpoint: problem determination in large, dynamic internet services. In Proceedings of the International Conference on Dependable Systems and Networks DSN, IEEE Computer Society: Bethesda, MD, USA, 2002; pp.595-604.

[39]

Renieris M, Reiss SP. Fault localization with nearest neighbor queries. In Proceedings of the International Conference on Automated Software Engineering ASE, IEEE Computer Society: Montreal, QC, Canada, 2003; pp.30-39.

[40]

Moon S, Kim Y, Kim M, Yoo S. Ask the mutants: mutating faulty programs for fault localization. In Proceedings of the International Conference on Software Testing, Verification and Validation ICST, IEEE: Cleveland, OH, USA, 2014; pp.153-162.

[41]

Zhang L, Zhang L, Khurshid S. Injecting mechanical faults to localize developer faults for evolving software. In Proceedings of the International Conference on Object-Oriented Programming, Systems, Languages and Applications OOPSLA, ACM: Indianapolis, IN, USA, 2013; pp.765-784.

[42]

Dobolyi K, Weimer W. Modeling consumer-perceived web application fault severities for testing. In Proceedings of the International Symposium on Software Testing and Analysis ISSTA, ISSTA'10, ACM: Trento, Italy, 2010; pp.97-106.

[43]

Zhang X, He H, Gupta N, Gupta R. Experimental evaluation of using dynamic slices for fault location. In Proceedings of the International Symposium on Automated Analysis-Driven Debuggin AADEBUG, ACM: Monterey, CA, USA, 2005; pp.33-42.

[44]

Ocariza F, Pattabiraman K, Mesbah A. Vejovis: suggesting fixes for JavaScript faults. In Proceedings of the International Conference on Software Engineering ICSE, ACM: Hyderabad, India, 2014; pp.837-847.

[45]

Artzi S, Dolby J, Tip F, Pistoia M. Practical fault localization for dynamic web applications. In Proceedings of the International Conference on Software Engineering ICSE, ACM: Cape Town, South Africa, 2010; pp.265-274.

[46]

Samimi H, Schäfer M, Artzi S, Millstein T, Tip F, Hendren L. Automated repair of HTML generation errors in PHP applications using string constraint solving. In Proceedings of the International Conference on Software Engineering ICSE, IEEE: Zurich, Switzerland, 2012; pp.277-287.

Cited By

Wang BKolluri ANikolić IBaluta TSaxena P(2023)User-Customizable Transpilation of Scripting LanguagesProceedings of the ACM on Programming Languages10.1145/35860347:OOPSLA1(201-229)Online publication date: 6-Apr-2023
https://dl.acm.org/doi/10.1145/3586034
Kim IWang WKwon YZhang XGrundy JPollock LPenta M(2023)BFTDETECTOR: Automatic Detection of Business Flow Tampering for Digital Content ServiceProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00048(448-459)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00048
Li FWang MHao D(2022)Bridging the Gap between Different Programming Paradigms in Coverage-based Fault LocalizationProceedings of the 13th Asia-Pacific Symposium on Internetware10.1145/3545258.3545272(75-84)Online publication date: 11-Jun-2022
https://dl.acm.org/doi/10.1145/3545258.3545272
Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Software Testing, Verification & Reliability

Software Testing, Verification & Reliability Volume 26, Issue 1

January 2016

88 pages

ISSN:0960-0833

EISSN:1099-1689

Issue’s Table of Contents

Publisher

John Wiley and Sons Ltd.

United Kingdom

Publication History

Published: 01 January 2016

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang BKolluri ANikolić IBaluta TSaxena P(2023)User-Customizable Transpilation of Scripting LanguagesProceedings of the ACM on Programming Languages10.1145/35860347:OOPSLA1(201-229)Online publication date: 6-Apr-2023
https://dl.acm.org/doi/10.1145/3586034
Kim IWang WKwon YZhang XGrundy JPollock LPenta M(2023)BFTDETECTOR: Automatic Detection of Business Flow Tampering for Digital Content ServiceProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00048(448-459)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00048
Li FWang MHao D(2022)Bridging the Gap between Different Programming Paradigms in Coverage-based Fault LocalizationProceedings of the 13th Asia-Pacific Symposium on Internetware10.1145/3545258.3545272(75-84)Online publication date: 11-Jun-2022
https://dl.acm.org/doi/10.1145/3545258.3545272
Lou YZhu QDong JLi XSun ZHao DZhang LZhang LSpinellis DGousios GChechik MDi Penta M(2021)Boosting coverage-based fault localization via graph-based representation learningProceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3468264.3468580(664-676)Online publication date: 20-Aug-2021
https://dl.acm.org/doi/10.1145/3468264.3468580
Netravali RMickens J(2019)ReverbProceedings of the ACM Symposium on Cloud Computing10.1145/3357223.3362733(428-440)Online publication date: 20-Nov-2019
https://dl.acm.org/doi/10.1145/3357223.3362733
Xu ZMa SZhang XZhu SXu BChaudron MCrnkovic IChechik MHarman M(2018)Debugging with intelligence via probabilistic inferenceProceedings of the 40th International Conference on Software Engineering10.1145/3180155.3180237(1171-1181)Online publication date: 27-May-2018
https://dl.acm.org/doi/10.1145/3180155.3180237
Ocariza FBajaj KPattabiraman KMesbah A(2017)A Study of Causes and Consequences of Client-Side JavaScript BugsIEEE Transactions on Software Engineering10.1109/TSE.2016.258606643:2(128-144)Online publication date: 1-Feb-2017
https://dl.acm.org/doi/10.1109/TSE.2016.2586066
Quintela-Pumares MCabral BFernandez-Lanvin DFernandez-Alvarez ADillon LVisser WWilliams L(2016)Integrating automatic backward error recovery in asynchronous rich clientsProceedings of the 38th International Conference on Software Engineering Companion10.1145/2889160.2889241(192-201)Online publication date: 14-May-2016
https://dl.acm.org/doi/10.1145/2889160.2889241
Madsen MTip FAndreasen ESen KMøller ADillon LVisser WWilliams L(2016)Feedback-directed instrumentation for deployed JavaScript applicationsProceedings of the 38th International Conference on Software Engineering10.1145/2884781.2884846(899-910)Online publication date: 14-May-2016
https://dl.acm.org/doi/10.1145/2884781.2884846

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents