Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/2970276.2970342acmconferencesArticle/Chapter ViewAbstractPublication PagesaseConference Proceedingsconference-collections
research-article
Public Access

Towards automatically generating descriptive names for unit tests

Published: 25 August 2016 Publication History

Abstract

During maintenance, developers often need to understand the purpose of a test. One of the most potentially useful sources of information for understanding a test is its name. Ideally, test names are descriptive in that they accurately summarize both the scenario and the expected outcome of the test. Despite the benefits of being descriptive, test names often fall short of this goal. In this paper we present a new approach for automatically generating descriptive names for existing test bodies. Using a combination of natural-language program analysis and text generation, the technique creates names that summarize the test's scenario and the expected outcome. The results of our evaluation show that, (1) compared to alternative approaches, the names generated by our technique are significantly more similar to human-generated names and are nearly always preferred by developers, (2) the names generated by our technique are preferred over or are equivalent to the original test names in 83% of cases, and (3) our technique is several orders of magnitude faster than manually writing test names.

References

[1]
M. Allamanis, E. T. Barr, C. Bird, and C. Sutton. Suggesting accurate method and class names. In Proceedings of the 10th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, pages 38– 49, 2015.
[2]
Y. Benjamini and Y. Hochberg. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society.
[3]
Series B (Methodological), 57(1):289–300, 1995.
[4]
S. Butler, M. Wermelinger, Y. Yu, and H. Sharp. Improving the tokenisation of identifier names. In Proceedings of the 25th European conference on Object-oriented programming, pages 130–154, 2011.
[5]
A. Corazza, S. D. Martino, and V. Maggio. LINSEN: An approach to split identifiers and expand abbreviations with linear complexity. In Proceedings of the 2012 IEEE International Conference on Software Maintenance, pages 233–242, 2012.
[6]
B. Eddy, J. Robinson, N. Kraft, and J. Carver. Evaluating source code summarization techniques: Replication and expansion. In Proceedings of the 2013 IEEE 21st International Conference on Program Comprehension, pages 13–22, 2013.
[7]
E. Enslen, E. Hill, L. Pollock, and K. Vijay-Shanker. Mining source code to automatically split identifiers for software analysis. In Proceedings of the 6th International Working Conference on Mining Software Repositories, pages 71–80, 2009.
[8]
L. Guerrouj, M. Di Penta, G. Antoniol, and Y.-G. Guéhéneuc. TIDIER: An identifier splitting approach using speech recognition techniques. Journal of Software: Evolution and Process, 25:575–599, 2013.
[9]
S. Gupta, S. Malik, L. Pollock, and K. Vijay-Shanker. Part-of-speech tagging of program identifiers for improved text-based software engineering tools. In Proceedings of the 21st IEEE International Conference on Program Comprehension, pages 3–12, 2013.
[10]
S. Haiduc, J. Aponte, L. Moreno, and A. Marcus. On the use of automated text summarization techniques for summarizing source code. In Proceedings of the 17th Working Conference on Reverse Engineering, pages 35–44, 2010.
[11]
E. Hill, Z. P. Fry, H. Boyd, G. Sridhara, Y. Novikova, L. Pollock, and K. Vijay-Shanker. AMAP: Automatically mining abbreviation expansions in programs to enhance software maintenance tools. In Proceedings of the 5th International Working Conference on Mining Software Repositories, pages 79–88, 2008.
[12]
E. Hill, D. Binkley, D. Lawrie, L. Pollock, and K. Vijay-Shanker. An empirical study of identifier splitting techniques. Empirical Software Engineering, 19(6):1754– 1780, 2014.
[13]
E. Høst and B. Østvold. Canonical method names for Java. In Software Language Engineering, volume 6563 of Lecture Notes in Computer Science, pages 226–245. 2011.
[14]
E. W. Høst and B. M. Østvold. The programmer’s lexicon, volume I: The verbs. In SCAM ’07: Proceedings of the 7th IEEE International Working Conference on Source Code Analysis and Manipulation, pages 193–202, 2007.
[15]
E. W. Høst and B. M. Østvold. Debugging method names. In Proceedings of the 23rd European Conference on Object-Oriented Programming, pages 294–317, 2009.
[16]
E. W. Høst and B. M. Østvold. The Java programmer’s phrase book. In Proceedings of the 1st International Conference on Software Language Engineering, pages 322–341, 2009.
[17]
M. J. Howard, S. Gupta, L. Pollock, and K. Vijay-Shanker. Automatically mining software-based, semantically-similar words from comment-code mappings. In Proceedings of the 10th Working Conference on Mining Software Repositories, pages 377–386, 2013.
[18]
M. Kamimura and G. Murphy. Towards generating human-oriented summaries of unit test cases. In Proceedings of the 21st IEEE International Conference on Program Comprehension, pages 215–218, 2013.
[19]
A. J. Ko, B. A. Myers, M. J. Coblenz, and H. H. Aung. An exploratory study of how developers seek, relate, and collect relevant information during software maintenance tasks. IEEE Transactions on Software Engineering, 32 (12):971–987, 2006.
[20]
B. Li, C. Vendome, M. Linares-Vasquez, D. Poshyvanyk, and N. Kraft. Automatically documenting unit test cases. In IEEE International Conference on Software Testing, Verification and Validation, pages 341–352, 2016.
[21]
N. Madani, L. Guerrouj, M. Di Penta, Y. Gueheneuc, and G. Antoniol. Recognizing words from source code identifiers using speech recognition techniques. In Proceedings of the 14th European Conference on Software Maintenance and Reengineering, pages 68–77, 2010.
[22]
P. W. McBurney and C. McMillan. Automatic documentation generation via source code summarization of method context. In Proceedings of the 22nd International Conference on Program Comprehension, pages 279–290, 2014.
[23]
L. Moreno, J. Aponte, G. Sridhara, A. Marcus, L. Pollock, and K. Vijay-Shanker. Automatic generation of natural language summaries for Java classes. In Proceedings of the 21st IEEE International Conference on Program Comprehension, pages 23–32, 2013.
[24]
Y. Oda, H. Fudaba, G. Neubig, H. Hata, S. Sakti, T. Toda, and S. Nakamura. Learning to generate pseudocode from source code using statistical machine translation. In 30th IEEE/ACM International Conference on Automated Software Engineering, pages 574–584, 2015.
[25]
K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 311– 318, 2002.
[26]
G. Sridhara, E. Hill, L. Pollock, and K. Vijay-Shanker. Identifying word relations in software: A comparative study of semantic similarity tools. In Proceedings of the 16th IEEE International Conference on Program Comprehension, pages 123–132, 2008.
[27]
G. Sridhara, E. Hill, D. Muppaneni, L. Pollock, and K. Vijay-Shanker. Towards automatically generating summary comments for Java methods. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering, pages 43–52, 2010.
[28]
G. Sridhara, L. Pollock, and K. Vijay-Shanker. Automatically detecting and describing high level actions within methods. In Proceedings of the 33rd ACM/IEEE International Conference on Software Engineering, pages 101–110, 2011.
[29]
G. Sridhara, L. Pollock, and K. Vijay-Shanker. Generating parameter comments and integrating with method summaries. In Proceedings of the 19th IEEE International Conference on Program Comprehension, pages 71–80, 2011.
[30]
T. Suzuki, K. Sakamoto, F. Ishikawa, and S. Honiden. An approach for evaluating and suggesting method names using n-gram models. In Proceedings of the 22nd International Conference on Program Comprehension, pages 271–274, 2014.
[31]
A. Trenk. Testing on the toilet: Writing descriptive test names. http://googletesting.blogspot.com/2014/ 10/testing-on-toilet-writing-descriptive.html, 2015.
[32]
J. Yang and L. Tan. Inferring semantically related words from software context. In Proceedings of the 9th IEEE Working Conference on Mining Software Repositories (MSR ’12), pages 161–170, 2012.
[33]
A. T. T. Ying and M. P. Robillard. Code fragment summarization. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, pages 655–658, 2013.
[34]
B. Zhang, E. Hill, and J. Clause. Automatically generating test templates from test names. In Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering, pages 506–511, 2015.
[35]
H. Zhong and Z. Su. Detecting API documentation errors. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, pages 803–816, 2013.

Cited By

View all
  • (2024)An Empirical Study on Focal Methods in Deep-Learning-Based Approaches for Assertion GenerationProceedings of the ACM on Software Engineering10.1145/36607851:FSE(1750-1771)Online publication date: 12-Jul-2024
  • (2024)Understandable Test Generation Through Capture/Replay and LLMsProceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings10.1145/3639478.3639789(261-263)Online publication date: 14-Apr-2024
  • (2024)Investigating the readability of test codeEmpirical Software Engineering10.1007/s10664-023-10390-z29:2Online publication date: 26-Feb-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ASE '16: Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering
August 2016
899 pages
ISBN:9781450338455
DOI:10.1145/2970276
  • General Chair:
  • David Lo,
  • Program Chairs:
  • Sven Apel,
  • Sarfraz Khurshid
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 August 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Descriptive names
  2. Maintenance
  3. Unit testing

Qualifiers

  • Research-article

Funding Sources

Conference

ASE'16
Sponsor:

Acceptance Rates

Overall Acceptance Rate 82 of 337 submissions, 24%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)76
  • Downloads (Last 6 weeks)11
Reflects downloads up to 13 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)An Empirical Study on Focal Methods in Deep-Learning-Based Approaches for Assertion GenerationProceedings of the ACM on Software Engineering10.1145/36607851:FSE(1750-1771)Online publication date: 12-Jul-2024
  • (2024)Understandable Test Generation Through Capture/Replay and LLMsProceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings10.1145/3639478.3639789(261-263)Online publication date: 14-Apr-2024
  • (2024)Investigating the readability of test codeEmpirical Software Engineering10.1007/s10664-023-10390-z29:2Online publication date: 26-Feb-2024
  • (2023)Automated Identification of Uniqueness in JUnit TestsACM Transactions on Software Engineering and Methodology10.1145/353331332:1(1-32)Online publication date: 13-Feb-2023
  • (2023)Function Call Graph Context Encoding for Neural Source Code SummarizationIEEE Transactions on Software Engineering10.1109/TSE.2023.327977449:9(4268-4281)Online publication date: Sep-2023
  • (2023)Generating Understandable Unit Tests through End-to-End Test Scenario Carving2023 IEEE 23rd International Working Conference on Source Code Analysis and Manipulation (SCAM)10.1109/SCAM59687.2023.00021(107-118)Online publication date: 2-Oct-2023
  • (2023)An Exploratory Study on the Usage and Readability of Messages Within Assertion Methods of Test Cases2023 IEEE/ACM 2nd International Workshop on Natural Language-Based Software Engineering (NLBSE)10.1109/NLBSE59153.2023.00015(32-39)Online publication date: May-2023
  • (2023)Label Smoothing Improves Neural Source Code Summarization2023 IEEE/ACM 31st International Conference on Program Comprehension (ICPC)10.1109/ICPC58990.2023.00025(101-112)Online publication date: May-2023
  • (2022)How Does This New Developer Test Fit In? A Visualization to Understand Amplified Test Cases2022 Working Conference on Software Visualization (VISSOFT)10.1109/VISSOFT55257.2022.00011(17-28)Online publication date: Oct-2022
  • (2022)An Ensemble Approach for Annotating Source Code Identifiers With Part-of-Speech TagsIEEE Transactions on Software Engineering10.1109/TSE.2021.309824248:9(3506-3522)Online publication date: 1-Sep-2022
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media