research-article

Public Access

APEx: automated inference of error specifications for C APIs

Authors:

Suman JanaAuthors Info & Claims

ASE '16: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering

Pages 472 - 482

https://doi.org/10.1145/2970276.2970354

Published: 25 August 2016 Publication History

Abstract

Although correct error handling is crucial to software robustness and security, developers often inadvertently introduce bugs in error handling code. Moreover, such bugs are hard to detect using existing bug-finding tools without correct error specifications. Creating error specifications manually is tedious and error-prone. In this paper, we present a new technique that automatically infers error specifications of API functions based on their usage patterns in C programs. Our key insight is that error-handling code tend to have fewer branching points and program statements than the code implementing regular functionality. Our scheme leverages this property to automatically identify error handling code at API call sites and infer the corresponding error constraints. We then use the error constraints from multiple call sites for robust inference of API error specifications. We evaluated our technique on 217 API functions from 6 different libraries across 28 projects written in C and found that it can identify error-handling paths with an average precision of 94% and recall of 66%. We also found that our technique can infer correct API error specifications with an average precision of 77% and recall of 47%. To further demonstrate the usefulness of the inferred error specifications, we used them to find 118 previously unknown potential bugs (including several security flaws that are currently being fixed by the corresponding developers) in the 28 tested projects.

References

[1]

Checker developer manual. http://clang-analyzer.llvm.org/checker_ dev_manual.html.

[2]

CVE-2014-0092. https://cve.mitre.org/cgi-bin/cvename.cgi?name= CVE-2014-0092, 2014.

[3]

CVE-2015-0208. https://cve.mitre.org/cgi-bin/cvename.cgi?name= CVE-2015-0208, 2015.

[4]

CVE-2015-0285. https://cve.mitre.org/cgi-bin/cvename.cgi?name= CVE-2015-0285, 2015.

[5]

CVE-2015-0288. https://cve.mitre.org/cgi-bin/cvename.cgi?name= CVE-2015-0288, 2015.

[6]

CVE-2015-0292. https://cve.mitre.org/cgi-bin/cvename.cgi?name= CVE-2015-0292, 2015.

[7]

M. Acharya and T. Xie. Mining API Error-Handling Specifications from Source Code. In International Conference on Fundamental Approaches to Software Engineering (FASE), 2009.

Digital Library

[8]

M. Acharya, T. Xie, J. Pei, and J. Xu. Mining API Patterns as Partial Orders from Source Code: From Usage Scenarios to Specifications. In ACM SIGSOFT symposium on The foundations of software engineering (FSE), 2007.

Digital Library

[9]

N. Ayewah, D. Hovemeyer, J. D. Morgenthaler, J. Penix, and W. Pugh. Using static analysis to find bugs. IEEE software, 25(5):22–29, 2008.

Digital Library

[10]

A. Bessey, K. Block, B. Chelf, A. Chou, B. Fulton, S. Hallem, C. Henri-Gros, A. Kamsky, S. McPeak, and D. Engler. A few billion lines of code later: using static analysis to find bugs in the real world. Communications of the ACM, 53(2):66–75, 2010.

Digital Library

[11]

P. Broadwell, N. Sastry, and J. Traupman. FIG: a prototype tool for online verification of recovery mechanisms. In Workshop on Self-Healing, Adaptive and Self-Managed Systems, 2002.

[12]

R. Buse and W. Weimer. Automatic documentation inference for exceptions. In International Symposium on Software Testing and Analysis (ISSTA), 2008.

Digital Library

[13]

H. Chen and D. Wagner. MOPS: an infrastructure for examining security properties of software. In ACM Conference on Computer and Communications Security (CCS), 2002.

Digital Library

[14]

B. Chess and J. West. Secure programming with static analysis. Pearson Education, 2007.

Digital Library

[15]

D. Engler, D. Chen, S. Hallem, A. Chou, and B. Chelf. Bugs as deviant behavior: A general approach to inferring errors in systems code. In the ACM Symposium on Operating Systems Principles (SOSP), 2001.

Digital Library

[16]

D. Engler and D. Dunbar. Under-constrained execution: making automatic code destruction easy and scalable. In International symposium on Software testing and analysis (ISSTA), pages 1–4. ACM, 2007.

Digital Library

[17]

H. Gunawi, C. Rubio-González, A. Arpaci-Dusseau, R. Arpaci-Dusseau, and B. Liblit. EIO: Error handling is occasionally correct. In USENIX Conference on File and Storage Technologies (FAST), 2008.

Digital Library

[18]

D. Hovemeyer and W. Pugh. Finding bugs is easy. ACM Sigplan Notices, 39(12):92–106, 2004.

Digital Library

[19]

S. Jana, Y. Kang, S. Roth, and B. Ray. Automatically Detecting Error Handling Bugs using Error Specifications. In USENIX Security Symposium (USENIX Security), August 2016.

[20]

J. Lawall, B. Laurie, R. Hansen, N. Palix, and G. Muller. Finding error handling bugs in OpenSSL using Coccinelle. In European Dependable Computing Conference (EDCC), 2010.

Digital Library

[21]

B. H. Liskov and A. Snyder. Exception handling in CLU. IEEE Transactions on Software Engineering, (6):546–558, 1979.

Digital Library

[22]

P. Marinescu, R. Banabic, and G. Candea. An extensible technique for high-precision testing of recovery code. In USENIX Annual Technical Conference, 2010.

Digital Library

[23]

P. Marinescu and G. Candea. Efficient testing of recovery code using fault injection. ACM Transactions on Computer Systems (TOCS), 29(4), 2011.

Digital Library

[24]

P. D. Marinescu and G. Candea. LFI: A practical and general librarylevel fault injector. In IEEE/IFIP International Conference on Dependable Systems & Networks (DSN), pages 379–388. IEEE, 2009.

[25]

M. Nagappan, R. Robbes, Y. Kamei, É. Tanter, S. McIntosh, A. Mockus, and A. E. Hassan. An empirical study of goto in C code from GitHub repositories. In 10th Joint Meeting on Foundations of Software Engineering (FSE), pages 404–414. ACM, 2015.

Digital Library

[26]

B. A. Nejmeh. NPATH: a measure of execution path complexity and its applications. Communications of the ACM, 31(2):188–200, 1988.

Digital Library

[27]

H. A. Nguyen, R. Dyer, T. N. Nguyen, and H. Rajan. Mining preconditions of APIs in large-scale code corpus. In ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE), pages 166–177. ACM, 2014.

Digital Library

[28]

OWASP top 10. https://www.owasp.org/images/e/e8/OWASP_Top_ 10_2007.pdf.

[29]

S. Person, G. Yang, N. Rungta, and S. Khurshid. Directed incremental symbolic execution. In ACM SIGPLAN Notices, volume 46, pages 504–515. ACM, 2011.

Digital Library

[30]

D. Ramos and D. Engler. Under-constrained symbolic execution: correctness checking for real code. In USENIX Security Symposium, 2015.

Digital Library

[31]

M. Robillard and G. Murphy. Analyzing exception flow in Java programs. In ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE), 1999.

Digital Library

[32]

M. P. Robillard and G. C. Murphy. Designing robust Java programs with exceptions. In ACM SIGSOFT Software Engineering Notes, volume 25, pages 2–10. ACM, 2000.

Digital Library

[33]

C. Rubio-González, H. Gunawi, B. Liblit, R. Arpaci-Dusseau, and A. Arpaci-Dusseau. Error propagation analysis for file systems. In ACM SIGPLAN conference on Programming Language Design and Implementation (PLDI), 2009.

Digital Library

[34]

C. Rubio-González and B. Liblit. Expect the unexpected: error code mismatches between documentation and the real world. In PASTE, 2010.

Digital Library

[35]

M. Süßkraut and C. Fetzer. Automatically finding and patching bad error handling. In Sixth European Dependable Computing Conference (EDCC), pages 13–22. IEEE, 2006.

Digital Library

[36]

L. Tan, D. Yuan, G. Krishna, and Y. Zhou. /* icomment: Bugs or bad comments?*/. In ACM SIGOPS Operating Systems Review, volume 41, pages 145–158, 2007.

Digital Library

[37]

W. Weimer and G. Necula. Finding and preventing run-time error handling mistakes. In Annual ACM Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), 2004.

Digital Library

[38]

W. Weimer and G. Necula. Mining Temporal Specifications for Error Detection. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), 2005.

Digital Library

[39]

W. Weimer and G. Necula. Exceptional situations and program reliability. ACM Transactions on Programming Languages and Systems (TOPLAS), 2008.

Digital Library

[40]

D. A. Wheeler. Sloccount. Available at http://www.dwheeler.com/ sloccount/, 2015.

[41]

H. Zhong and Z. Su. Detecting API documentation errors. In ACM SIGPLAN Notices, volume 48, pages 803–816. ACM, 2013.

Digital Library

[42]

H. Zhong, L. Zhang, T. Xie, and H. Mei. Inferring resource specifications from natural language API documentation. In International Conference on Automated Software Engineering (ASE), pages 307– 318, 2009.

Digital Library

Cited By

Yin ZSong YZong G(2024)Discovering API usage specifications for security detection using two-stage code miningCybersecurity10.1186/s42400-024-00224-w7:1Online publication date: 3-Oct-2024
https://doi.org/10.1186/s42400-024-00224-w
Liu HJia ZLi SLei YYu YJiang YMao XLiao X(2024)Cut to the Chase: An Error-Oriented Approach to Detect Error-Handling BugsProceedings of the ACM on Software Engineering10.1145/36607871:FSE(1796-1818)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660787
Chapman PRubio-González CThakur AMonat RRubio-González C(2024)Interleaving Static Analysis and LLM PromptingProceedings of the 13th ACM SIGPLAN International Workshop on the State Of the Art in Program Analysis10.1145/3652588.3663317(9-17)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3652588.3663317
Show More Cited By

Index Terms

APEx: automated inference of error specifications for C APIs
1. Software and its engineering
  1. Software notations and tools
    1. Software libraries and repositories

Recommendations

Automatically diagnosing and repairing error handling bugs in C
ESEC/FSE 2017: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering

Correct error handling is essential for building reliable and secure systems. Unfortunately, low-level languages like C often do not support any error handling primitives and leave it up to the developers to create their own mechanisms for error ...
Mining preconditions of APIs in large-scale code corpus
FSE 2014: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering

Modern software relies on existing application programming interfaces (APIs) from libraries. Formal specifications for the APIs enable many software engineering tasks as well as help developers correctly use them. In this work, we mine large-scale ...
Automatically detecting error handling bugs using error specifications
SEC'16: Proceedings of the 25th USENIX Conference on Security Symposium

Incorrect error handling in security-sensitive code often leads to severe security vulnerabilities. Implementing correct error handling is repetitive and tedious especially in languages like C that do not support any exception handling primitives. This ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASE '16: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering

August 2016

899 pages

ISBN:9781450338455

DOI:10.1145/2970276

General Chair:
David Lo
Singapore Management University, Singapore
,
Program Chairs:
Sven Apel
University of Passau, Germany
,
Sarfraz Khurshid
University of Texas at Austin, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence
SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 August 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Air Force Office of Scientific Research

Conference

ASE'16

Sponsor:

SIGAI
SIGSOFT
IEEE-CS

ASE'16: ACM/IEEE International Conference on Automated Software Engineering

September 3 - 7, 2016

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 82 of 337 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
702
Total Downloads

Downloads (Last 12 months)141
Downloads (Last 6 weeks)13

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yin ZSong YZong G(2024)Discovering API usage specifications for security detection using two-stage code miningCybersecurity10.1186/s42400-024-00224-w7:1Online publication date: 3-Oct-2024
https://doi.org/10.1186/s42400-024-00224-w
Liu HJia ZLi SLei YYu YJiang YMao XLiao X(2024)Cut to the Chase: An Error-Oriented Approach to Detect Error-Handling BugsProceedings of the ACM on Software Engineering10.1145/36607871:FSE(1796-1818)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3660787
Chapman PRubio-González CThakur AMonat RRubio-González C(2024)Interleaving Static Analysis and LLM PromptingProceedings of the 13th ACM SIGPLAN International Workshop on the State Of the Art in Program Analysis10.1145/3652588.3663317(9-17)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3652588.3663317
Ma YTian WGao XSun HLi LChristakis MPradel M(2024)API Misuse Detection via Probabilistic Graphical ModelProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3650212.3652112(88-99)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3650212.3652112
Gong YNie JYou WShi WHuang JLiang BZhang JBaysal OLinares-Vasquez MMoran KSteinmacher I(2024)SICode: Embedding-Based Subgraph Isomorphism Identification for Bug DetectionProceedings of the 32nd IEEE/ACM International Conference on Program Comprehension10.1145/3643916.3646556(304-315)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643916.3646556
Chen KWen MJia HWu RJin H(2024)Towards Effective and Efficient Error Handling Code Fuzzing Based on Software Fault Injection2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00039(320-331)Online publication date: 12-Mar-2024
https://doi.org/10.1109/SANER60148.2024.00039
Chen XHu XHuang YJiang HJi WJiang YJiang YLiu BLiu HLi XLian XMeng GPeng XSun HShi LWang BWang CWang JWang TXuan JXia XYang YYang YZhang LZhou YZhang L(2024)Deep learning-based software engineering: progress, challenges, and opportunitiesScience China Information Sciences10.1007/s11432-023-4127-568:1Online publication date: 24-Dec-2024
https://doi.org/10.1007/s11432-023-4127-5
Zhao BJi SZhang XTian YWang QPu YLyu CBeyah RCalandrino JTroncoso C(2023)UVSCANProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620429(3421-3438)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620429
Hu PLiang RCao YChen KZhang RCalandrino JTroncoso C(2023)AURCProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620317(1415-1432)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620317
Zhu WFeng ZZhang ZChen JOu ZYang MZhang C(2023)Callee: Recovering Call Graphs for Binaries with Transfer and Contrastive Learning2023 IEEE Symposium on Security and Privacy (SP)10.1109/SP46215.2023.10179482(2357-2374)Online publication date: May-2023
https://doi.org/10.1109/SP46215.2023.10179482
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents