research-article

Open access

Effective interactive resolution of static analysis alarms

Authors:

Mayur NaikAuthors Info & Claims

Proceedings of the ACM on Programming Languages, Volume 1, Issue OOPSLA

Article No.: 57, Pages 1 - 30

https://doi.org/10.1145/3133881

Published: 12 October 2017 Publication History

Abstract

We propose an interactive approach to resolve static analysis alarms. Our approach synergistically combines a sound but imprecise analysis with precise but unsound heuristics, through user interaction. In each iteration, it solves an optimization problem to find a set of questions for the user such that the expected payoff is maximized. We have implemented our approach in a tool, Ursa, that enables interactive alarm resolution for any analysis specified in the declarative logic programming language Datalog. We demonstrate the effectiveness of Ursa on a state-of-the-art static datarace analysis using a suite of 8 Java programs comprising 41-194 KLOC each. Ursa is able to eliminate 74% of the false alarms per benchmark with an average payoff of 12× per question. Moreover, Ursa prioritizes user effort effectively by posing questions that yield high payoffs earlier.

References

[1]

2015. UpWork. http://www.upwork.com . (2015). Accessed: 2015-11-19.

[2]

Nathaniel Ayewah, David Hovemeyer, J. David Morgenthaler, John Penix, and William Pugh. 2008. Using static analysis to find bugs. IEEE Software (2008).

Digital Library

[3]

Thomas Ball, Mayur Naik, and Sriram K. Rajamani. 2003. From symptom to cause: localizing errors in counterexample traces. In POPL.

Digital Library

[4]

Osbert Bastani, Saswat Anand, and Alex Aiken. 2015. Specification inference using context-free language reachability. In POPL .

Digital Library

[5]

Al Bessey, Ken Block, Benjamin Chelf, Andy Chou, Bryan Fulton, Seth Hallem, Charles-Henri Gros, Asya Kamsky, Scott McPeak, and Dawson R. Engler. 2010. A few billion lines of code later: using static analysis to find bugs in the real world. Commun. ACM (2010).

[6]

Sam Blackshear and Shuvendu Lahiri. 2013. Almost-correct specifications: a modular semantic framework for assigning confidence to warnings. In PLDI.

Digital Library

[7]

Tom Copeland. 2005. PMD applied. (2005).

[8]

Isil Dillig, Thomas Dillig, and Alex Aiken. 2012. Automated error diagnosis using abductive inference. In PLDI.

Digital Library

[9]

Lisa Nguyen Quang Do, Karim Ali, Benjamin Livshits, Eric Bodden, Justin Smith, and Emerson Murphy-Hill. 2017. Just-intime static analysis. In ISSTA.

[10]

Michael D. Ernst, Jake Cockrell, William G. Griswold, and David Notkin. 2001. Dynamically discovering likely program invariants to support program evolution. IEEE Trans. Software Eng. (2001).

Digital Library

[11]

Gurobi Optimization, Inc. 2016. Gurobi optimizer reference manual. http://www.gurobi.com . (2016).

[12]

Seth Hallem, Benjamin Chelf, Yichen Xie, and Dawson R. Engler. 2002. A system and language for building system-specific, static analyses. In PLDI.

[13]

James A. Jones and Mary Jean Harrold. 2005. Empirical evaluation of the tarantula automatic fault-localization technique. In ASE.

Digital Library

[14]

James A. Jones, Mary Jean Harrold, and John T. Stasko. 2002. Visualization of test information to assist fault localization. In ICSE .

[15]

Herbert Jordan, Bernhard Scholz, and Pavle Subotic. 2016. Soufflé: on synthesis of program analyzers. In CAV.

[16]

Yungbum Jung, Jaehwang Kim, Jaeho Shin, and Kwangkeun Yi. 2005. Taming false alarms from a domain-unaware C analyzer by a bayesian statistical post analysis. In SAS.

[17]

Ted Kremenek, Ken Ashcraft, Junfeng Yang, and Dawson Engler. 2004. Correlation exploitation in error ranking. In FSE.

Digital Library

[18]

Ted Kremenek and Dawson Engler. 2003. Z-Ranking: using statistical analysis to counter the impact of static analysis approximations. In SAS.

[19]

Wei Le and Mary Lou Soffa. 2010. Path-based Fault Correlations. In FSE.

Digital Library

[20]

Woosuk Lee, Wonchan Lee, and Kwangkeun Yi. 2012. Sound non-statistical clustering of static analysis alarms. In VMCAI.

Digital Library

[21]

Ondrej Lhoták. 2002. Spark: A flexible points-to analysis framework for Java. (2002).

[22]

Ben Liblit, Mayur Naik, Alice X. Zheng, Alexander Aiken, and Michael I. Jordan. 2005. Scalable statistical bug isolation. In PLDI .

Digital Library

[23]

Benjamin Livshits, Manu Sridharan, Yannis Smaragdakis, Ondrej Lhoták, José Nelson Amaral, Bor-Yuh Evan Chang, Samuel Z. Guyer, Uday P. Khedker, Anders Møller, and Dimitrios Vardoulakis. 2015. In defense of soundiness: a manifesto. CACM (2015).

[24]

Magnus Madsen, Ming-Ho Yee, and Ondrej Lhoták. 2016. From Datalog to Flix: a declarative language for fixed points on lattices. In PLDI.

Digital Library

[25]

Ravi Mangal, Xin Zhang, Aditya V. Nori, and Mayur Naik. 2015. A user-guided approach to program analysis. In FSE.

Digital Library

[26]

Mayur Naik. 2006. Chord: A Program Analysis Platform for Java. http://jchord.googlecode.com/ . (2006).

[27]

Mayur Naik, Alex Aiken, and John Whaley. 2006. Effective static race detection for Java. In PLDI.

Digital Library

[28]

Mayur Naik, Hongseok Yang, Ghila Castelnuovo, and Mooly Sagiv. 2012. Abstractions from tests. In POPL.

Digital Library

[29]

Greg Nelson and Derek C. Oppen. 1979. Simplification by cooperating decision procedures. ACM TOPLAS (1979).

[30]

Hakjoo Oh, Wonchan Lee, Kihong Heo, Hongseok Yang, and Kwangkeun Yi. 2016. Selective X-sensitive analysis guided by impact pre-analysis. ACM TOPLAS (2016).

[31]

Oded Padon, Kenneth McMillan, Aurojit Panda, Mooly Sagiv, and Sharon Shoham. 2016. Ivy: safety verification by interactive generalization. In PLDI.

Digital Library

[32]

Christos H. Papadimitriou. 1981. On the complexity of integer programming. J. ACM (1981).

[33]

J. Ross Quinlan. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann.

Digital Library

[34]

Manos Renieris and Steven P. Reiss. 2003. Fault localization with nearest neighbor queries. In ASE.

Digital Library

[35]

Henry Gordon Rice. 1953. Classes of recursively enumerable sets and their decision problems. Trans. Amer. Math. Soc. (1953).

[36]

Yannis Smaragdakis and Martin Bravenboer. 2010. Using Datalog for fast and easy program analysis. In Datalog 2.0 Workshop .

[37]

Yannis Smaragdakis, George Kastrinis, and George Balatsouras. 2014. Introspective analysis: context-sensitivity, across the board. In PLDI.

Digital Library

[38]

Daniel von Dincklage and Amer Diwan. 2009. Optimizing programs with intended semantics. In OOPSLA.

Digital Library

[39]

Daniel von Dincklage and Amer Diwan. 2011. Integrating program analyses with programmer productivity tools. Softw., Pract. Exper. (2011).

[40]

Shiyi Wei, Omer Tripp, Barbara G. Ryder, and Julian Dolby. 2016. Revamping JavaScript static analysis via localization and remediation of root causes of imprecision. In FSE.

Digital Library

[41]

John Whaley, Dzintars Avots, Michael Carbin, and Monica S. Lam. 2005. Using Datalog with binary decision diagrams for program analysis. In APLAS.

Digital Library

[42]

Xin Zhang, Ravi Mangal, Radu Grigore, Mayur Naik, and Hongseok Yang. 2014. On abstraction refinement for program analyses in Datalog. In PLDI.

Digital Library

[43]

Haiyan Zhu, Thomas Dillig, and Isil Dillig. 2013. Automated inference of library specifications for source-sink property verification. In APLAS 2013.

Digital Library

Cited By

Wen CCai YZhang BSu JXu ZLiu DQin SMing ZCong T(2024)Automatically Inspecting Thousands of Static Bug Warnings with Large Language Model: How Far Are We?ACM Transactions on Knowledge Discovery from Data10.1145/365371818:7(1-34)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3653718
Zhang YShi YZhang X(2024)Learning Abstraction Selection for Bayesian Program AnalysisProceedings of the ACM on Programming Languages10.1145/36498458:OOPSLA1(954-982)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3649845
Muske TSerebrenik A(2023)Survey of Approaches for Postprocessing of Static Analysis AlarmsACM Computing Surveys10.1145/349452155:3(1-39)Online publication date: 30-Apr-2023
https://doi.org/10.1145/3494521
Show More Cited By

Index Terms

Effective interactive resolution of static analysis alarms
1. Theory of computation
  1. Semantics and reasoning
    1. Program reasoning
      1. Program analysis
      2. Program verification

Recommendations

Interprocedural pointer alias analysis

We present practical approximation methods for computing and representing interprocedural aliases for a program written in a language that includes pointers, reference parameters, and recursion. We present the following contributions: (1) a framework ...
PMAF: an algebraic framework for static analysis of probabilistic programs
PLDI '18

Automatically establishing that a probabilistic program satisfies some property ϕ is a challenging problem. While a sampling-based approach—which involves running the program repeatedly—can suggest that ϕ holds, to establish that the program satisfies ϕ,...
PMAF: an algebraic framework for static analysis of probabilistic programs
PLDI 2018: Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation

Automatically establishing that a probabilistic program satisfies some property ϕ is a challenging problem. While a sampling-based approach—which involves running the program repeatedly—can suggest that ϕ holds, to establish that the program satisfies ϕ,...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Programming Languages

Proceedings of the ACM on Programming Languages Volume 1, Issue OOPSLA

October 2017

1786 pages

EISSN:2475-1421

DOI:10.1145/3152284

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2017

Published in PACMPL Volume 1, Issue OOPSLA

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Artifacts Evaluated & Functional

Author Tags

Qualifiers

Research-article

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
626
Total Downloads

Downloads (Last 12 months)103
Downloads (Last 6 weeks)11

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wen CCai YZhang BSu JXu ZLiu DQin SMing ZCong T(2024)Automatically Inspecting Thousands of Static Bug Warnings with Large Language Model: How Far Are We?ACM Transactions on Knowledge Discovery from Data10.1145/365371818:7(1-34)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3653718
Zhang YShi YZhang X(2024)Learning Abstraction Selection for Bayesian Program AnalysisProceedings of the ACM on Programming Languages10.1145/36498458:OOPSLA1(954-982)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3649845
Muske TSerebrenik A(2023)Survey of Approaches for Postprocessing of Static Analysis AlarmsACM Computing Surveys10.1145/349452155:3(1-39)Online publication date: 30-Apr-2023
https://doi.org/10.1145/3494521
Liu PLu YYang WPan M(2023)VALAR: Streamlining Alarm Ranking in Static Analysis with Value-Flow Assisted Active Learning2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)10.1109/ASE56229.2023.00098(1940-1951)Online publication date: 11-Sep-2023
https://doi.org/10.1109/ASE56229.2023.00098
Piskachev GBecker MBodden E(2023)Can the configuration of static analyses make resolving security vulnerabilities more effective? - A user studyEmpirical Software Engineering10.1007/s10664-023-10354-328:5Online publication date: 12-Sep-2023
https://dl.acm.org/doi/10.1007/s10664-023-10354-3
Apinis KVojdani V(2023)Context-Sensitive Meta-Constraint Systems for Explainable Program AnalysisTools and Algorithms for the Construction and Analysis of Systems10.1007/978-3-031-30820-8_27(453-472)Online publication date: 20-Apr-2023
https://doi.org/10.1007/978-3-031-30820-8_27
Kim HRaghothaman MHeo KDwyer MDamian DZeller A(2022)Learning probabilistic models for static analysis alarmsProceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510098(1282-1293)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510003.3510098
Zhang PLiu BLu TDing XGu HGu N(2022)Jointly Predicting Future Content in Multiple Social Media Sites Based on Multi-task LearningACM Transactions on Information Systems10.1145/349553040:4(1-28)Online publication date: 11-Jan-2022
https://dl.acm.org/doi/10.1145/3495530
Yang LZeynali AHajiesmaili MSitaraman RTowsley D(2021)Competitive Algorithms for Online Multidimensional Knapsack ProblemsProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/34910425:3(1-30)Online publication date: 15-Dec-2021
https://dl.acm.org/doi/10.1145/3491042
Naik AMendelson JSands NWang YNaik MRaghothaman M(2021)Sporq: An Interactive Environment for Exploring Code using Query-by-ExampleThe 34th Annual ACM Symposium on User Interface Software and Technology10.1145/3472749.3474737(84-99)Online publication date: 10-Oct-2021
https://dl.acm.org/doi/10.1145/3472749.3474737
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents