research-article

Systematic evaluation of the unsoundness of call graph construction algorithms for Java

Authors:

Florian Kübler,

Michael Eichberg,

Mira MeziniAuthors Info & Claims

ISSTA '18: Companion Proceedings for the ISSTA/ECOOP 2018 Workshops

Pages 107 - 112

https://doi.org/10.1145/3236454.3236503

Published: 16 July 2018 Publication History

Abstract

Call graphs are at the core of many static analyses ranging from the detection of unused methods to advanced control-and data-flow analyses. Therefore, a comprehensive understanding of the precision and recall of the respective graphs is crucial to enable an assessment which call-graph construction algorithms are suited in which analysis scenario. For example, malware is often obfuscated and tries to hide its intent by using Reflection. Call graphs that do not represent reflective method calls are, therefore, of limited use when analyzing such apps.

In general, the precision is well understood, but the recall is not, i.e., in which cases a call graph will not contain any call edges. In this paper, we discuss the design of a comprehensive test suite that enables us to compute a fingerprint of the unsoundness of the respective call-graph construction algorithms. This suite also enables us to make a comparative evaluation of static analysis frameworks. Comparing Soot and WALA shows that WALA currently has better support for new Java 8 features and also for Java Reflection. However, in some cases both fail to include expected edges.

References

[1]

Karim Ali and Ondřej Lhoták. 2013. Averroes: Whole-program analysis without the whole program. In European Conference on Object-Oriented Programming. Springer, 378--400.

Digital Library

[2]

Nicholas Allen, Padmanabhan Krishnan, and Bernhard Scholz. 2015. Combining type-analysis with points-to analysis for analyzing java library source-code. In Proceedings of the 4th ACM SIGPLAN International Workshop on State Of the Art in Program Analysis. ACM, 13--18.

Digital Library

[3]

Steven Arzt, Siegfried Rasthofer, Christian Fritz, Eric Bodden, Alexandre Bartel, Jacques Klein, Yves Le Traon, Damien Octeau, and Patrick McDaniel. 2014. Flowdroid: Precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for android apps. Acm Sigplan Notices 49, 6 (2014), 259--269.

Digital Library

[4]

David F Bacon and Peter F Sweeney. 1996. Fast static analysis of C++ virtual function calls. ACM Sigplan Notices 31, 10 (1996), 324--341.

Digital Library

[5]

Stephen M Blackburn, Robin Garner, Chris Hoffmann, Asjad M Khang, Kathryn S McKinley, Rotem Bentzur, Amer Diwan, Daniel Feinberg, Daniel Frampton, Samuel Z Guyer, et al. 2006. The DaCapo benchmarks: Java benchmarking development and analysis. In ACM Sigplan Notices, Vol. 41. ACM, 169--190.

Digital Library

[6]

Eric Bodden, Andreas Sewe, Jan Sinschek, Hela Oueslati, and Mira Mezin. 2011. Taming reflection. In Proceeding of the 33rd international conference on Software engineering - ICSE '11. ACM Press, New York, New York, USA, 241.

Digital Library

[7]

Jeffrey Dean, David Grove, and Craig Chambers. 1995. Optimization of object-oriented programs using static class hierarchy analysis. In European Conference on Object-Oriented Programming. Springer, 77--101.

Digital Library

[8]

JB Dietrich, Henrik Schole, Li Sui, and Ewan Tempero. 2017. XCorpus- An executable Corpus of Java Programs. (2017).

[9]

Michael Eichberg, Ben Hermann, Mira Mezini, and Leonid Glanz. 2015. Hidden Truths in Dead Software Paths. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015). ACM, New York, NY, USA, 474--484. 2786865

Digital Library

[10]

Michael Grace, Yajin Zhou, Qiang Zhang, Shihong Zou, and Xuxian Jiang. 2012. Riskranker: scalable and accurate zero-day android malware detection. In Proceedings of the 10th international conference on Mobile systems, applications, and services. ACM, 281--294.

Digital Library

[11]

IBM. {n. d.}. WALA - Static Analysis Framework for Java. http://wala.sourceforge.net/. ({n. d.}). {Online; accessed 19-APRIL-2018}.

[12]

Ondvrej Lhoták. 2007. Comparing call graphs. In Proceedings of the 7th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering - PASTE '07. ACM Press, New York, New York, USA, 37--42.

Digital Library

[13]

Siliang Li and Gang Tan. 2009. Finding bugs in exceptional situations of JNI programs. In Proceedings of the 16th ACM conference on Computer and communications security. ACM, 442--452.

Digital Library

[14]

Benjamin Livshits, Manu Sridharan, Yannis Smaragdakis, Ondřej Lhoták, J Nelson Amaral, Bor-Yuh Evan Chang, Samuel Z Guyer, Uday P Khedker, Anders Møller, and Dimitrios Vardoulakis. 2015. In defense of soundiness: a manifesto. Commun. ACM 58, 2 (2015), 44--46.

Digital Library

[15]

Benjamin Livshits, John Whaley, and Monica S Lam. 2005. Reflection analysis for Java. In Asian Symposium on Programming Languages and Systems. Springer, 139--160.

Digital Library

[16]

Luis Mastrangelo, Luca Ponzanelli, Andrea Mocci, Michele Lanza, Matthias Hauswirth, and Nathaniel Nystrom. 2015. Use at Your Own Risk: The Java Unsafe API in the Wild. In Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA 2015). ACM, New York, NY, USA, 695--710.

Digital Library

[17]

Gail C Murphy, David Notkin, William G Griswold, and Erica S Lan. 1998. An empirical study of static call graph extractors. ACM Transactions on Software Engineering and Methodology (TOSEM) 7, 2 (1998), 158--191.

Digital Library

[18]

Michael Reif, Michael Eichberg, Ben Hermann, Johannes Lerch, and Mira Mezini. 2016. Call graph construction for java libraries. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 474--486.

Digital Library

[19]

Michael Reif, Michael Eichberg, Ben Hermann, and Mira Mezini. 2017. Hermes: assessment and creation of effective test corpora. In Proceedings of the 6th ACM SIGPLAN International Workshop on State Of the Art in Program Analysis. ACM, 43--48.

Digital Library

[20]

Gang Tan, Andrew W Appel, Srimat Chakradhar, Anand Raghunathan, Srivaths Ravi, and Daniel Wang. 2006. Safe Java native interface. In Proceedings of IEEE International Symposium on Secure Software Engineering, Vol. 97. 106.

[21]

Ewan Tempero, Craig Anslow, Jens Dietrich, Ted Han, Jing Li, Markus Lumpe, Hayden Melton, and James Noble. 2010. The Qualitas Corpus: A curated collection of Java code for empirical studies. In Software Engineering Conference (APSEC), 2010 17th Asia Pacific. IEEE, 336--345.

Digital Library

[22]

Frank Tip and Jens Palsberg. 2000. Scalable propagation-based call graph construction algorithms. Vol. 35. ACM.

Digital Library

[23]

Raja Vallée-Rai, Phong Co, Etienne Gagnon, Laurie Hendren, Patrick Lam, and Vijay Sundaresan. 2010. Soot: A Java bytecode optimization framework. In CASCON First Decade High Impact Papers. IBM Corp., 214--224.

Digital Library

Cited By

Zhang HPei YLiang SXing ZTan SChristakis MPradel M(2024)Characterizing and Detecting Program Representation Faults of Static Analysis FrameworksProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680398(1772-1784)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680398
Helm DKeidel SKampkötter ADüsing JRoth THermann BMezini MChristakis MPradel M(2024)Total Recall? How Good Are Static Call Graphs Really?Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3652114(112-123)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3652114
Helm DRoth TKeidel SReif MMezini MChristakis MPradel M(2024)Unimocg: Modular Call-Graph Algorithms for Consistent Handling of Language FeaturesProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3652109(51-62)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3652109
Show More Cited By

Recommendations

On the unsoundness of static analysis for Android GUIs
SOAP 2016: Proceedings of the 5th ACM SIGPLAN International Workshop on State Of the Art in Program Analysis

Android software presents exciting new challenges for the static analysis community. However, static analyses for Android are typically unsound. This is due to the lack of specification of the Android framework, the continuous evolution of framework ...
Parameterized object sensitivity for points-to analysis for Java

The goal of points-to analysis for Java is to determine the set of objects pointed to by a reference variable or a reference object field. We present object sensitivity, a new form of context sensitivity for flow-insensitive points-to analysis for Java. ...
Type-Based Call Graph Construction Algorithms for Scala

Call graphs have many applications in software engineering. For example, they serve as the basis for code navigation features in integrated development environments and are at the foundation of static analyses performed in verification tools. While many ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA '18: Companion Proceedings for the ISSTA/ECOOP 2018 Workshops

July 2018

143 pages

ISBN:9781450359399

DOI:10.1145/3236454

Conference Chairs:
Julian Dolby
IBM Thomas J. Watson Research Center
,
William G. J. Halfond
University of Southern California
,
Ashish Mishra
Northeastern University, United States.

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 July 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ISSTA '18

Sponsor:

ISSTA '18: International Symposium on Software Testing and Analysis

July 16 - 21, 2018

Amsterdam, Netherlands

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
407
Total Downloads

Downloads (Last 12 months)76
Downloads (Last 6 weeks)3

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang HPei YLiang SXing ZTan SChristakis MPradel M(2024)Characterizing and Detecting Program Representation Faults of Static Analysis FrameworksProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3680398(1772-1784)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3680398
Helm DKeidel SKampkötter ADüsing JRoth THermann BMezini MChristakis MPradel M(2024)Total Recall? How Good Are Static Call Graphs Really?Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3652114(112-123)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3652114
Helm DRoth TKeidel SReif MMezini MChristakis MPradel M(2024)Unimocg: Modular Call-Graph Algorithms for Consistent Handling of Language FeaturesProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3652109(51-62)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3652109
Santos JMirakhorli MShokri A(2024)Seneca: Taint-Based Call Graph Construction for Java Object DeserializationProceedings of the ACM on Programming Languages10.1145/36498518:OOPSLA1(1125-1153)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3649851
Bose PDas DVasan SMariani SGrishchenko IContinella ABianchi AKruegel CVigna GGrundy JPollock LPenta M(2023)Columbus: Android App Testing through Systematic Callback ExplorationProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00121(1381-1392)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00121
Liu JLiu JDi PLiu AZhong ZHarman MMiller H(2022)Record and replay of online traffic for microservices with automatic mocking point identificationProceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice10.1145/3510457.3513029(221-230)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510457.3513029
Liu JLiu JDi PLiu AZhong Z(2022)Record and Replay of Online Traffic for Microservices with Automatic Mocking Point Identification2022 IEEE/ACM 44th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)10.1109/ICSE-SEIP55303.2022.9793867(221-230)Online publication date: May-2022
https://doi.org/10.1109/ICSE-SEIP55303.2022.9793867
Soto-Valero CDurieux TBaudry BSpinellis DGousios GChechik MDi Penta M(2021)A longitudinal analysis of bloated Java dependenciesProceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3468264.3468589(1021-1031)Online publication date: 20-Aug-2021
https://dl.acm.org/doi/10.1145/3468264.3468589
Santos JJones RAshiogwu CMirakhorli MDo LUrban C(2021)Serialization-aware call graph constructionProceedings of the 10th ACM SIGPLAN International Workshop on the State Of the Art in Program Analysis10.1145/3460946.3464319(37-42)Online publication date: 22-Jun-2021
https://dl.acm.org/doi/10.1145/3460946.3464319
Ali KLai XLuo ZLhotak ODolby JTip F(2021)A Study of Call Graph Construction for JVM-Hosted LanguagesIEEE Transactions on Software Engineering10.1109/TSE.2019.295692547:12(2644-2666)Online publication date: 1-Dec-2021
https://doi.org/10.1109/TSE.2019.2956925
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten