Article

Characterizing logging practices in open-source software

Authors:

Yuanyuan ZhouAuthors Info & Claims

ICSE '12: Proceedings of the 34th International Conference on Software Engineering

Pages 102 - 112

Published: 02 June 2012 Publication History

Abstract

Software logging is a conventional programming practice. As its efficacy is often important for users and developers to understand what have happened in production run, yet software logging is often done in an arbitrarily manner. So far, there have been little study for understanding logging practices in real world software. This paper makes the first attempt (to the best of our knowledge) to provide quantitative characteristic study of the current log messages within four pieces of large open-source software. First, we quantitatively show that software logging is pervasive. By examining developers’ own modifications to logging code in revision history, we find that they often do not make the log messages right in their first attempts, and thus need to spend significant amount of efforts to modify the log messages as after-thoughts. Our study further provides several interesting findings on where developers spend most of their efforts in modifying the log messages, which can give insights for programmers, tool developers, and language and compiler designers to improve the current logging practice. To demonstrate the benefit of our study, we built a simple checker based on one of our findings and effectively detected 138 new problematic logging code from studied software (24 of them are already confirmed and fixed by developers).

References

[1]

ArcSight Log analysis. http://www.arcsight.com/Logger.

[2]

T. Ball and S. K. Rajamani. The SLAM project: debugging system software via static analysis. In Proceedings of the 29th ACM SIGPLAN-SIGACT symposium on Principles of programming languages (POPL), pages 1-3, 2002.

Digital Library

[3]

L. Baresi and M. Young. Toward translating design constraints to run-time assertions. Electron. Notes Theor. Comput. Sci., 116:73-84, January 2005.

Digital Library

[4]

M. Bruntink, A. v. Deursen, and T. Tourwé. Discovering faults in idiom-based exception handling. In Proceedings of the 28th international conference on Software engineering (ICSE), pages 242-251, 2006.

Digital Library

[5]

Cisco system log management. http://www.cisco.com/en/US/ docs/voice_ip_comm/cucm/service/3_3_2/ccmsrvs/sssyslog. html.

[6]

Dell. Streamlined Troubleshooting with the Dell system E-Support tool. Dell Power Solutions, 2008.

[7]

EMC seen collecting and managing log as key driver for 94 percent of customers. http://www.rsa.com/press_release.aspx? id=7596.

[8]

D. Engler, B. Chelf, and A. Chou. Checking system rules using system-specific, programmer-written compiler extensions. In Proceedings of the 4th conference on Symposium on Operating System Design & Implementation (OSDI), pages 1-16, 2000.

Digital Library

[9]

D. Engler, D. Y. Chen, S. Hallem, A. Chou, and B. Chelf. Bugs as deviant behavior: A general approach to inferring errors in systems code. In Proceedings of the eighteenth ACM symposium on Operating systems principles (SOSP), pages 57-72, 2001.

Digital Library

[10]

M. Gabel, J. Yang, Y. Yu, M. Goldszmidt, and Z. Su. Scalable and systematic detection of buggy inconsistencies in source code. In Proceedings of the ACM international conference on Object oriented programming systems languages and applications (OOPSLA), pages 175-190, 2010.

Digital Library

[11]

M. Hauswirth and T. M. Chilimbi. Low-overhead memory leak detection using adaptive statistical profiling. In Proceedings of the 11th international conference on Architectural support for programming languages and operating systems, ASPLOS-XI, pages 156-164, 2004.

Digital Library

[12]

M.-C. Hsueh, T. K. Tsai, and R. K. Iyer. Fault injection techniques and tools. Computer, 30:75-82, 1997.

Digital Library

[13]

W. Jiang, C. Hu, S. Pasupathy, A. Kanevsky, Z. Li, and Y. Zhou. Understanding customer problem troubleshooting from storage system logs. In Proccedings of the 7th conference on File and storage technologies (FAST), pages 43-56, 2009.

Digital Library

[14]

B. W. Kernighan and R. Pike. The Practice of Programming. Addison-Wesley, 1999.

Digital Library

[15]

Z. Li, S. Lu, S. Myagmar, and Y. Zhou. CP-Miner: A Tool for Finding Copy-paste and Related Bugs in Operating System Code. In Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation (OSDI), pages 176-192, 2004.

Digital Library

[16]

Apache Logging Services - Log4j. http://logging.apache.org/ log4j.

[17]

L. Mariani and F. Pastore. Automated identification of failure causes in system logs. In Proceedings of the 2008 19th International Symposium on Software Reliability Engineering (ISSRE), pages 117-126, 2008.

Digital Library

[18]

L. Mariani, F. Pastore, and M. Pezzè. A toolset for automated failure analysis. In Proceedings of the 31st International Conference on Software Engineering (ICSE), pages 563-566, 2009.

Digital Library

[19]

J. C. Munson and S. G. Elbaum. Code churn: A measure for estimating the impact of code change. In Proceedings of the International Conference on Software Maintenance (ICSM), pages 24-31, 1998.

Digital Library

[20]

N. Nagappan and T. Ball. Use of relative code churn measures to predict system defect density. In Proceedings of the 27th international conference on Software engineering (ICSE), pages 284-292, 2005.

Digital Library

[21]

NetApp. Proactive health management with auto-support. NetApp White Paper, 2007.

[22]

Netcraft report: Apache httpd is the most popular webserver. http://news.netcraft.com/archives/category/webserversurvey/.

[23]

Lessons from the success of ssh. http://www.cs.virginia.edu/ ~drl7x/sshVsTelnetWeb3.pdf.

[24]

M. Pezzè and J. Wuttke. Automatic generation of runtime failure detectors from property templates. In B. H. Cheng, R. Lemos, H. Giese, P. Inverardi, and J. Magee, editors, Software Engineering for Self-Adaptive Systems, pages 223-240. Springer-Verlag, Berlin, Heidelberg, 2009.

Digital Library

[25]

SLOCCount: Source Lines Of Code Count. http://www. dwheeler.com/sloccount/.

[26]

Splunk Log management. http://www.splunk.com/view/logmanagement/ SP-CAAAC6F.

[27]

Squid bug report 3319. http://bugs.squid-cache.org/show_ bug.cgi?id=3319.

[28]

RFC3164 - the BSD Syslog protocol. http://tools.ietf.org/ html/rfc3164.

[29]

L. Tan, D. Yuan, G. Krishna, and Y. Zhou. /* iComment: Bugs or bad comments? */. In Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles (SOSP), pages 145-158, 2007.

Digital Library

[30]

Top 10 Enterprise database systems to consider. http://www.serverwatch.com/trends/article.php/3883441/Top- 10-Enterprise-Database-Systems-to-Consider.htm.

[31]

D. Wessels. Squid: The Definitive Guide. O'Reilly, 2004.

Digital Library

[32]

W. Xu, L. Huang, A. Fox, D. Patterson, and M. I. Jordan. Detecting large-scale system problems by mining console logs. In Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles (SOSP), pages 117-132, 2009.

Digital Library

[33]

A. T. T. Ying, G. C. Murphy, R. Ng, and M. C. Chu-Carroll. Predicting source code changes by mining change history. IEEE Trans. Softw. Eng., 30:574-586, September 2004.

Digital Library

[34]

D. Yuan, H. Mai, W. Xiong, L. Tan, Y. Zhou, and S. Pasupathy. SherLog: Error diagnosis by connecting clues from runtime logs. In Proceedings of the fifteenth edition of ASPLOS on Architectural support for programming languages and operating systems (ASPLOS), pages 143-154, 2010.

Digital Library

[35]

D. Yuan, S. Park, P. Huang, Y. Liu, M. Lee, Y. Zhou, and S. Savage. Did you log the error? Characterizing and improving software error reporting. Technical Report.

[36]

D. Yuan, J. Zheng, S. Park, Y. Zhou, and S. Savage. Improving software diagnosability via log enhancement. In Proceedings of the sixteenth international conference on Architectural support for programming languages and operating systems (ASPLOS), pages 3-14, 2011.

Digital Library

[37]

A. Zeller. Isolating cause-effect chains from computer programs. In Proceedings of the 10th ACM SIGSOFT symposium on Foundations of software engineering, SIGSOFT'02/FSE- 10, pages 1-10, 2002.

Digital Library

Cited By

Li YHuo YJiang ZZhong RHe PSu YBriand LLyu M(2024)Exploring the Effectiveness of LLMs in Automated Logging Statement Generation: An Empirical StudyIEEE Transactions on Software Engineering10.1109/TSE.2024.347537550:12(3188-3207)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3475375
Ding ZTang YCheng XLi HShang W(2023) LoGenText-Plus: Improving Neural Machine Translation Based Logging Texts Generation with Syntactic TemplatesACM Transactions on Software Engineering and Methodology10.1145/362474033:2(1-45)Online publication date: 22-Dec-2023
https://dl.acm.org/doi/10.1145/3624740
Li LLiang YLiu ZYu ZChandra SBlincoe KTonella P(2023)Understanding Solidity Event Logging Practices in the WildProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616342(300-312)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616342
Show More Cited By

Recommendations

Characterizing logging practices in Java-based open source software projects --- a replication study in Apache Software Foundation

Log messages, which are generated by the debug statements that developers insert into the code at runtime, contain rich information about the runtime behavior of software systems. Log messages are used widely for system monitoring, problem diagnoses and ...
A literature review and existing challenges on software logging practices: From the creation to the analysis of software logs
Abstract
Software logging is the practice of recording different events and activities that occur within a software system, which are useful for different activities such as failure prediction and anomaly detection. While previous research focused on ...
Characterizing and detecting duplicate logging code smells
ICSE '19: Proceedings of the 41st International Conference on Software Engineering: Companion Proceedings

Software logs are widely used by developers to assist in various tasks. Despite the importance of logs, prior studies show that there is no industrial standard on how to write logging statements. Recent research on logs often only considers the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '12: Proceedings of the 34th International Conference on Software Engineering

June 2012

1657 pages

ISBN:9781467310673

General Chair:
Martin Glinz,
Program Chairs:
Gail Murphy,
Mauro Pezzè

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

IEEE Press

Publication History

Published: 02 June 2012

Check for updates

Qualifiers

Article

Conference

ICSE '12

Sponsor:

SIGSOFT

ICSE '12: 34th International Conference on Software Engineering

June 2 - 9, 2012

Zurich, Switzerland

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

56
Total Citations
View Citations
660
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)2

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li YHuo YJiang ZZhong RHe PSu YBriand LLyu M(2024)Exploring the Effectiveness of LLMs in Automated Logging Statement Generation: An Empirical StudyIEEE Transactions on Software Engineering10.1109/TSE.2024.347537550:12(3188-3207)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TSE.2024.3475375
Ding ZTang YCheng XLi HShang W(2023) LoGenText-Plus: Improving Neural Machine Translation Based Logging Texts Generation with Syntactic TemplatesACM Transactions on Software Engineering and Methodology10.1145/362474033:2(1-45)Online publication date: 22-Dec-2023
https://dl.acm.org/doi/10.1145/3624740
Li LLiang YLiu ZYu ZChandra SBlincoe KTonella P(2023)Understanding Solidity Event Logging Practices in the WildProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616342(300-312)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616342
Liu XYang CLi DZhou YLi SChen JChen Z(2023)Adonis: Practical and Efficient Control Flow Recovery through OS-level TracesACM Transactions on Software Engineering and Methodology10.1145/360718733:1(1-27)Online publication date: 4-Jul-2023
https://dl.acm.org/doi/10.1145/3607187
Bogatinovski JNedelkoski SAcker ACardoso JKao ORastogi ATufano RBavota GArnaoudova VHaiduc S(2022)QuLogProceedings of the 30th IEEE/ACM International Conference on Program Comprehension10.1145/3524610.3527906(275-286)Online publication date: 16-May-2022
https://dl.acm.org/doi/10.1145/3524610.3527906
Zhao YSu TLiu YZheng WWu XKavuluru RHalfond WYu T(2022)ReCDroid+: Automated End-to-End Crash Reproduction from Bug Reports for Android AppsACM Transactions on Software Engineering and Methodology10.1145/348824431:3(1-33)Online publication date: 7-Mar-2022
https://dl.acm.org/doi/10.1145/3488244
Leroy DLelandais BOudot MCombemale BVisser EKolovos DSöderberg E(2021)Monilogging for executable domain-specific languagesProceedings of the 14th ACM SIGPLAN International Conference on Software Language Engineering10.1145/3486608.3486906(2-15)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3486608.3486906
Alves MPaula H(2021)Identifying Logging Practices in Open Source Python Containerized Application ProjectsProceedings of the XXXV Brazilian Symposium on Software Engineering10.1145/3474624.3474631(16-20)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1145/3474624.3474631
He SHe PChen ZYang TSu YLyu M(2021)A Survey on Automated Log Analysis for Reliability EngineeringACM Computing Surveys10.1145/346034554:6(1-37)Online publication date: 13-Jul-2021
https://dl.acm.org/doi/10.1145/3460345
Chen BJiang Z(2021)A Survey of Software Log InstrumentationACM Computing Surveys10.1145/344897654:4(1-34)Online publication date: 3-May-2021
https://dl.acm.org/doi/10.1145/3448976
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents