research-article

Expect the unexpected: error code mismatches between documentation and the real world

Authors:

Cindy Rubio-González,

Ben LiblitAuthors Info & Claims

PASTE '10: Proceedings of the 9th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering

Pages 73 - 80

https://doi.org/10.1145/1806672.1806687

Published: 06 May 2010 Publication History

Abstract

Inaccurate documentation can mislead programmers and cause software to fail in unexpected ways. We examine mismatches between documented and actual error codes returned by 42 Linux file-related system calls. We use static program analysis to identify the error codes returned by system calls across 52 file systems, including widely-used implementations such as CIFS, ext3, IBM JFS, ReiserFS and XFS. We describe analysis optimizations that dramatically reduce run-time and memory consumption. Comparing analysis results with Linux manual pages reveals over 1,700 undocumented error-code instances affecting all file systems and system calls examined.

References

[1]

E. Berglund and M. Priestley. Open-source documentation: in search of user-driven, just-in-time writing. In SIGDOC, pages 132--141, 2001.

Digital Library

[2]

R. E. Bryant. Binary decision diagrams and beyond: enabling technologies for formal verification. In R. L. Rudell, editor, ICCAD, pages 236--243. IEEE Computer Society, 1995.

Digital Library

[3]

R. P. L. Buse and W. Weimer. Automatic documentation inference for exceptions. In B. G. Ryder and A. Zeller, editors, ISSTA, pages 273--282. ACM, 2008.

Digital Library

[4]

L. H. Etzkorn, W. E. H. Jr., and C. G. Davis. Automated reusability quality analysis of oo legacy software. Information & Software Technology, 43 (5): 295--308, 2001.

[5]

C. Kaner. Liability for defective documentation. In S. B. Jones and D. G. Novick, editors, SIGDOC, pages 192--197. ACM, 2003.

Digital Library

[6]

N. Kidd, T. Reps, and A. Lal. WALi: A C++ library for weighted pushdown systems. http://www.cs.wisc.edu/wpis/wpds/, 2009.

[7]

A. Lal, N. Kidd, T. W. Reps, and T. Touili. Abstract error projection. In H. R. Nielson and G. Filé, editors, SAS, volume 4634 of Lecture Notes in Computer Science, pages 200--217. Springer, 2007.

Digital Library

[8]

A. Lal, T. Touili, N. Kidd, and T. Reps. Interprocedural analysis of concurrent programs under a context bound. Technical Report 1598, University of Wisconsin-Madison, July 2007.

[9]

T. Lethbridge, J. Singer, and A. Forward. How software engineers use documentation: The state of the practice. IEEE Software, 20 (6): 35--39, 2003.

Digital Library

[10]

J. Lind-Nielsen. BuDDy - A Binary Decision Diagram Package. http://sourceforge.net/projects/buddy, 2004.

[11]

S. N. I. Mount, R. M. Newman, R. J. Low, and A. Mycroft. Exstatic: a generic static checker applied to documentation systems. In S. R. Tilley and S. Huang, editors, SIGDOC, pages 52--57. ACM, 2004.

Digital Library

[12]

G. C. Necula, S. McPeak, S. P. Rahul, and W. Weimer. CIL: Intermediate language and tools for analysis and transformation of C programs. In R. N. Horspool, editor, CC, volume 2304 of Lecture Notes in Computer Science, pages 213--228. Springer, 2002.

Digital Library

[13]

T. W. Reps, S. Schwoon, S. Jha, and D. Melski. Weighted pushdown systems and their application to interprocedural dataflow analysis. Sci. Comput. Program., 58 (1-2): 206--263, 2005.

Digital Library

[14]

G. Robles, J. M. González Barahona, and J. L. Prieto Martínez. Assessing and evaluating documentation in libre software projects. In T. Wasserman and M. Pal, editors, Workshop on Evaluation Frameworks for Open Source Software (EFOSS), Como, Italy, June 2006. International Federation for Information Processing.

[15]

C. Rubio-González, H. S. Gunawi, B. Liblit, R. H. Arpaci-Dusseau, and A. C. Arpaci-Dusseau. Error Propagation Analysis for File Systems. In Proceedings of the ACM SIGPLAN 2009 Conference on Programming Language Design and Implementation, Dublin, Ireland, June 15-20 2009.

Digital Library

[16]

P. Sacramento, B. Cabral, and P. Marques. Unchecked exceptions: Can the programmer be trusted to document exceptions? In Second International Conference on Innovative Views of .NET Technologies, Florianópolis, Brazil, Oct. 2006. Microsoft.

[17]

C. Schönberg, F. Weitl, M. Jaksic, and B. Freitag. Logic-based verification of technical documentation. In U. M. Borghoff and B. Chidlovskii, editors, ACM Symposium on Document Engineering, pages 251--252. ACM, 2009.

Digital Library

[18]

D. Schreck, V. Dallmeier, and T. Zimmermann. How documentation evolves over time. In M. D. Penta and M. Lanza, editors, IWPSE, pages 4--10. ACM, 2007.

Digital Library

[19]

J. Singer. Practices of software maintenance. In ICSM, pages 139--145, 1998.

Digital Library

[20]

L. Tan, D. Yuan, G. Krishna, and Y. Zhou. /*icomment: bugs or bad comments?*/. In T. C. Bressoud and M. F. Kaashoek, editors, SOSP, pages 145--158. ACM, 2007.

Digital Library

[21]

G. Venolia. Textual allusions to artifacts in software-related repositories. In S. Diehl, H. Gall, and A. E. Hassan, editors, MSR, pages 151--154. ACM, 2006.

Digital Library

[22]

M. N. Wegman and F. K. Zadeck. Constant propagation with conditional branches. In POPL, pages 291--299, 1985.

Digital Library

[23]

Y. Xie and D. R. Engler. Using redundancies to find errors. IEEE Trans. Software Eng., 29 (10): 915--928, 2003.

Digital Library

Cited By

Chen XHu XHuang YJiang HJi WJiang YJiang YLiu BLiu HLi XLian XMeng GPeng XSun HShi LWang BWang CWang JWang TXuan JXia XYang YYang YZhang LZhou YZhang L(2024)Deep learning-based software engineering: progress, challenges, and opportunitiesScience China Information Sciences10.1007/s11432-023-4127-568:1Online publication date: 24-Dec-2024
https://doi.org/10.1007/s11432-023-4127-5
Hu PLiang RCao YChen KZhang RCalandrino JTroncoso C(2023)AURCProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620317(1415-1432)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620317
Niazi TDas TAhmed GWaqas SKhan SKhan SAbdelatif AWasi S(2023)Investigating Novice Developers’ Code Commenting Trends Using Machine Learning TechniquesAlgorithms10.3390/a1601005316:1(53)Online publication date: 12-Jan-2023
https://doi.org/10.3390/a16010053
Show More Cited By

Index Terms

Recommendations

Error propagation analysis for file systems
PLDI '09

Unchecked errors are especially pernicious in operating system file management code. Transient or permanent hardware failures are inevitable, and error-management bugs at the file system layer can cause silent, unrecoverable data corruption. We propose ...
Error propagation analysis for file systems
PLDI '09: Proceedings of the 30th ACM SIGPLAN Conference on Programming Language Design and Implementation

Unchecked errors are especially pernicious in operating system file management code. Transient or permanent hardware failures are inevitable, and error-management bugs at the file system layer can cause silent, unrecoverable data corruption. We propose ...
Defective error/pointer interactions in the Linux kernel
ISSTA '11: Proceedings of the 2011 International Symposium on Software Testing and Analysis

Linux run-time errors are represented by integer values referred to as error codes. These values propagate across long function-call chains before being handled. As these error codes propagate, they are often temporarily or permanently encoded into ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PASTE '10: Proceedings of the 9th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering

June 2010

96 pages

ISBN:9781450300827

DOI:10.1145/1806672

Program Chairs:
Sorin Lerner
University of California, San Diego, USA
,
Atanas Rountev
Ohio State University, USA

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 May 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

PASTE '10

Sponsor:

PASTE '10: ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering

June 5 - 6, 2010

Ontario, Toronto, Canada

Acceptance Rates

Overall Acceptance Rate 57 of 159 submissions, 36%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
164
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen XHu XHuang YJiang HJi WJiang YJiang YLiu BLiu HLi XLian XMeng GPeng XSun HShi LWang BWang CWang JWang TXuan JXia XYang YYang YZhang LZhou YZhang L(2024)Deep learning-based software engineering: progress, challenges, and opportunitiesScience China Information Sciences10.1007/s11432-023-4127-568:1Online publication date: 24-Dec-2024
https://doi.org/10.1007/s11432-023-4127-5
Hu PLiang RCao YChen KZhang RCalandrino JTroncoso C(2023)AURCProceedings of the 32nd USENIX Conference on Security Symposium10.5555/3620237.3620317(1415-1432)Online publication date: 9-Aug-2023
https://dl.acm.org/doi/10.5555/3620237.3620317
Niazi TDas TAhmed GWaqas SKhan SKhan SAbdelatif AWasi S(2023)Investigating Novice Developers’ Code Commenting Trends Using Machine Learning TechniquesAlgorithms10.3390/a1601005316:1(53)Online publication date: 12-Jan-2023
https://doi.org/10.3390/a16010053
Zhang XZou WChen LLi YZhou Y(2023)Towards the Analysis and Completion of Syntactic Structure Ellipsis for Inline CommentsIEEE Transactions on Software Engineering10.1109/TSE.2022.321627949:4(2285-2302)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TSE.2022.3216279
Steinbeck MKoschke R(2021)Javadoc Violations and Their Evolution in Open-Source Software2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER50967.2021.00031(249-259)Online publication date: Mar-2021
https://doi.org/10.1109/SANER50967.2021.00031
Alghamdi MHayashi SKobayashi TTreude C(2021)Characterising the Knowledge about Primitive Variables in Java Code Comments2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR)10.1109/MSR52588.2021.00058(460-470)Online publication date: May-2021
https://doi.org/10.1109/MSR52588.2021.00058
Tang WHu YFan GYao PWu RBai GWang PZhang CGrundy J(2021)TranscodeProceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE51524.2021.9678823(829-841)Online publication date: 15-Nov-2021
https://dl.acm.org/doi/10.1109/ASE51524.2021.9678823
Zhai JXu XShi YTao GPan MMa SXu LZhang WTan LZhang XRothermel GBae D(2020)CPCProceedings of the ACM/IEEE 42nd International Conference on Software Engineering10.1145/3377811.3380427(1359-1371)Online publication date: 27-Jun-2020
https://dl.acm.org/doi/10.1145/3377811.3380427
Křikava FMiller HVitek J(2019)Scala implicits are everywhere: a large-scale study of the use of Scala implicits in the wildProceedings of the ACM on Programming Languages10.1145/33605893:OOPSLA(1-28)Online publication date: 10-Oct-2019
https://dl.acm.org/doi/10.1145/3360589
Li YWang SNguyen TVan Nguyen S(2019)Improving bug detection via context-based code representation learning and attention-based neural networksProceedings of the ACM on Programming Languages10.1145/33605883:OOPSLA(1-30)Online publication date: 10-Oct-2019
https://dl.acm.org/doi/10.1145/3360588
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents