research-article

An Extractive-and-Abstractive Framework for Source Code Summarization

Authors:

Zhenyu ChenAuthors Info & Claims

ACM Transactions on Software Engineering and Methodology, Volume 33, Issue 3

Article No.: 75, Pages 1 - 39

https://doi.org/10.1145/3632742

Published: 14 March 2024 Publication History

Abstract

(Source) Code summarization aims to automatically generate summaries/comments for given code snippets in the form of natural language. Such summaries play a key role in helping developers understand and maintain source code. Existing code summarization techniques can be categorized into extractive methods and abstractive methods. The extractive methods extract a subset of important statements and keywords from the code snippet using retrieval techniques and generate a summary that preserves factual details in important statements and keywords. However, such a subset may miss identifier or entity naming, and consequently, the naturalness of the generated summary is usually poor. The abstractive methods can generate human-written-like summaries leveraging encoder-decoder models. However, the generated summaries often miss important factual details.

To generate human-written-like summaries with preserved factual details, we propose a novel extractive-and-abstractive framework. The extractive module in the framework performs the task of extractive code summarization, which takes in the code snippet and predicts important statements containing key factual details. The abstractive module in the framework performs the task of abstractive code summarization, which takes in the code snippet and important statements in parallel and generates a succinct and human-written-like natural language summary. We evaluate the effectiveness of our technique, called EACS, by conducting extensive experiments on three datasets involving six programming languages. Experimental results show that EACS significantly outperforms state-of-the-art techniques for all three widely used metrics, including BLEU, METEOR, and ROUGH-L. In addition, the human evaluation demonstrates that the summaries generated by EACS have higher naturalness and informativeness and are more relevant to given code snippets.

References

[1]

Nahla J. Abid, Jonathan I. Maletic, and Bonita Sharif. 2019. Using developer eye movements to externalize the mental model used in code summarization tasks. In Proceedings of the 11th Symposium on Eye Tracking Research & Applications. ACM, Denver, CO, USA, 13:1–13:9.

Digital Library

[2]

Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2020. A transformer-based approach for source code summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Online, 4998–5007.

[3]

Miltiadis Allamanis, Earl T. Barr, Premkumar T. Devanbu, and Charles Sutton. 2018. A survey of machine learning for big code and naturalness. Comput. Surveys 51, 4 (2018), 81:1–81:37.

[4]

Uri Alon, Omer Levy, and Eran Yahav. 2018. Code2seq: Generating sequences from structured representations of code. CoRR abs/1808.01400 (2018).

[5]

Lorena Arcega, Jaime Font, Øystein Haugen, and Carlos Cetina. 2019. An approach for bug localization in models using two levels: Model and metamodel. Software and Systems Modeling 18, 6 (2019), 3551–3576.

Digital Library

[6]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Representations. OpenReview.net, San Diego, CA, USA, 1–15.

[7]

Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization. Association for Computational Linguistics, Ann Arbor, Michigan, USA, 65–72.

[8]

Aakash Bansal, Sakib Haque, and Collin McMillan. 2021. Project-level encoding for neural source code summarization of subroutines. In Proceedings of the 29th International Conference on Program Comprehension. IEEE, Madrid, Spain, 253–264.

[9]

Antonio Valerio Miceli Barone and Rico Sennrich. 2017. A parallel corpus of Python functions and documentation strings for automated code documentation and code generation. In Proceedings of the 8th International Joint Conference on Natural Language Processing. Asian Federation of Natural Language Processing, Taipei, Taiwan, 314–319.

[10]

Qiuyuan Chen, Xin Xia, Han Hu, David Lo, and Shanping Li. 2021. Why my code summarization model does not work: Code comment improvement with category prediction. ACM Transactions on Software Engineering and Methodology 30, 2 (2021), 25:1–25:29.

Digital Library

[11]

Yen-Chun Chen and Mohit Bansal. 2018. Fast abstractive summarization with reinforce-selected sentence rewriting. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Melbourne, Australia, 675–686.

[12]

Jianpeng Cheng and Mirella Lapata. 2016. Neural summarization by extracting sentences and words. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. The Association for Computer Linguistics, Berlin, Germany, 1–11.

[13]

Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. In Proceedings of the 8th Workshop on Syntax, Semantics and Structure in Statistical Translation. Association for Computational Linguistics, Doha, Qatar, 103–111.

[14]

Sumit Chopra, Michael Auli, and Alexander M. Rush. 2016. Abstractive sentence summarization with attentive recurrent neural networks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. The Association for Computational Linguistics, San Diego California, USA, 93–98.

[15]

Sergio Cozzetti B. de Souza, Nicolas Anquetil, and Káthia Marçal de Oliveira. 2005. A study of the documentation essential to software maintenance. In Proceedings of the 23rd Annual International Conference on Design of Communication: Documenting & Designing for Pervasive Information. ACM, Coventry, UK, 68–75.

Digital Library

[16]

Scott C. Deerwester, Susan T. Dumais, Thomas K. Landauer, George W. Furnas, and Richard A. Harshman. 1990. Indexing by latent semantic analysis. Journal of the American Society for Information Science 41, 6 (1990), 391–407.

[17]

Olive Jean Dunn. 1964. Multiple comparisons using rank sums. Technometrics 6, 3 (1964), 241–252.

[18]

Brian P. Eddy, Jeffrey A. Robinson, Nicholas A. Kraft, and Jeffrey C. Carver. 2013. Evaluating source code summarization techniques: Replication and expansion. In Proceedings of the 21st International Conference on Program Comprehension. IEEE Computer Society, San Francisco, CA, USA, 13–22.

[19]

Wafaa S. El-Kassas, Cherif R. Salama, Ahmed A. Rafea, and Hoda K. Mohamed. 2021. Automatic text summarization: A comprehensive survey. Expert Systems with Applications 165, 1 (2021), 113679–113704.

[20]

Günes Erkan and Dragomir R. Radev. 2004. LexRank: Graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research 22, 1 (2004), 457–479.

[21]

Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. CodeBERT: A pre-trained model for programming and natural languages. In Proceedings of the 25th Conference on Empirical Methods in Natural Language Processing: Findings. Association for Computational Linguistics, Online Event, 1536–1547.

[22]

Patrick Fernandes, Miltiadis Allamanis, and Marc Brockschmidt. 2018. Structured neural summarization. CoRR abs/1811.01824 (2018).

[23]

Catherine O. Fritz, Peter E. Morris, and Jennifer J. Richler. 2012. Effect size estimates: Current use, calculations, and interpretation. Journal of Experimental Psychology: General 141, 1 (2012), 2–18.

[24]

Mahak Gambhir and Vishal Gupta. 2017. Recent automatic text summarization techniques: A survey. Artificial Intelligence Review 47, 1 (2017), 1–66.

Digital Library

[25]

Kavita Ganesan, ChengXiang Zhai, and Jiawei Han. 2010. Opinosis: A graph based approach to abstractive summarization of highly redundant opinions. In Proceedings of the 23rd International Conference on Computational Linguistics. Tsinghua University Press, Beijing, China, 340–348.

[26]

Yuexiu Gao and Chen Lyu. 2022. M2TS: Multi-scale multi-modal approach based on transformer for source code summarization. CoRR abs/2203.09707 (2022).

[27]

Pierre-Etienne Genest and Guy Lapalme. 2012. Fully abstractive approach to guided summarization. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics. The Association for Computer Linguistics, Jeju Island, Korea, 354–358.

Digital Library

[28]

Inc. GitHub. 2008. GitHub. site: https://github.comAccessed: 2022.

[29]

Zi Gong, Cuiyun Gao, Yasheng Wang, Wenchao Gu, Yun Peng, and Zenglin Xu. 2022. Source code summarization with structural relative position guided transformer. CoRR abs/2202.06521 (2022).

[30]

David Gros, Hariharan Sezhiyan, Prem Devanbu, and Zhou Yu. 2020. Code to comment “translation”: Data, metrics, baselining & evaluation. In Proceedings of the 35th International Conference on Automated Software Engineering. IEEE, Melbourne, Australia, 746–757.

[31]

Som Gupta and Sanjai Kumar Gupta. 2019. Abstractive summarization: An overview of the state of the art. Expert Systems with Applications 121, 1 (2019), 49–65.

Digital Library

[32]

Sonia Haiduc, Jairo Aponte, and Andrian Marcus. 2010. Supporting program comprehension with source code summarization. In Proceedings of the 32nd International Conference on Software Engineering. ACM, Cape Town, South Africa, 223–226.

Digital Library

[33]

Sonia Haiduc, Jairo Aponte, Laura Moreno, and Andrian Marcus. 2010. On the use of automated text summarization techniques for summarizing source code. In Proceedings of the 17th Working Conference on Reverse Engineering. IEEE Computer Society, Beverly, MA, USA, 35–44.

Digital Library

[34]

Sakib Haque, Aakash Bansal, Lingfei Wu, and Collin McMillan. 2021. Action word prediction for neural source code summarization. In Proceedings of the 28th International Conference on Software Analysis, Evolution and Reengineering. IEEE, Honolulu, HI, USA, 330–341.

[35]

Sakib Haque, Zachary Eberhart, Aakash Bansal, and Collin McMillan. 2022. Semantic similarity metrics for evaluating source code summarization. In Proceedings of the 30th International Conference on Program Comprehension. ACM, Virtual Event, 36–47.

Digital Library

[36]

Carl S. Hartzman and Charles F. Austin. 1993. Maintenance productivity: Observations based on an experience in a large system environment. In Proceedings of the 3rd Conference of the Centre for Advanced Studies on Collaborative Research. IBM, Toronto, Ontario, Canada, 138–170.

[37]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735–1780.

Digital Library

[38]

Wan Ting Hsu, Chieh-Kai Lin, Ming-Ying Lee, Kerui Min, Jing Tang, and Min Sun. 2018. A unified model for extractive and abstractive summarization using inconsistency loss. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Melbourne, Australia, 132–141.

[39]

Xing Hu, Ge Li, Xin Xia, David Lo, and Zhi Jin. 2018. Deep code comment generation. In Proceedings of the 26th International Conference on Program Comprehension. ACM, Gothenburg, Sweden, 200–210.

Digital Library

[40]

Xing Hu, Ge Li, Xin Xia, David Lo, and Zhi Jin. 2020. Deep code comment generation with hybrid lexical and syntactical information. Empirical Software Engineering 25, 3 (2020), 2179–2217.

Digital Library

[41]

Xing Hu, Ge Li, Xin Xia, David Lo, Shuai Lu, and Zhi Jin. 2018. Summarizing source code with transferred API knowledge. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. IJCAI.org, Stockholm, Sweden, 2269–2275.

Digital Library

[42]

Hamel Husain, Ho-Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Marc Brockschmidt. 2019. CodeSearchNet challenge: Evaluating the state of semantic code search. CoRR abs/1909.09436 (2019).

[43]

Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, and Luke Zettlemoyer. 2016. Summarizing source code using a neural attention model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. The Association for Computer Linguistics, Berlin, Germany, 2073–2083.

[44]

Peddamail Jayavardhan Reddy, Ziyu Yao, Zhen Wang, and Huan Sun. 2018. A comprehensive study of StaQC for deep code summarization. In Proceedings of the 24th International Conference on Knowledge Discovery and Data Mining. ACM, London, UK, 1–8.

[45]

Mira Kajko-Mattsson. 2005. A survey of documentation practice within corrective maintenance. Empirical Software Engineering 10, 1 (2005), 31–55.

Digital Library

[46]

Atif Khan, Naomie Salim, and Yogan Jaya Kumar. 2015. A framework for multi-document abstractive summarization based on semantic role labelling. Applied Soft Computing 30, 1 (2015), 737–747.

Digital Library

[47]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations – Poster. OpenReview.net, San Diego, CA, USA, 1–15.

[48]

Kirill Kireyev. 2008. Using latent semantic analysis for extractive summarization. In Proceedings of the 1st Text Analysis Conference. NIST, Gaithersburg, Maryland, USA, 1–4.

[49]

Jens Krinke. 2006. Effects of context on program slicing. Journal of Systems and Software 79, 9 (2006), 1249–1260.

Digital Library

[50]

Litton J. Kurisinkel, Yue Zhang, and Vasudeva Varma. 2017. Abstractive multi-document summarization by partial tree extraction, recombination and linearization. In Proceedings of the 8th International Joint Conference on Natural Language Processing. Asian Federation of Natural Language Processing, Taipei, Taiwan.

[51]

Alexander LeClair, Sakib Haque, Lingfei Wu, and Collin McMillan. 2020. Improved code summarization via a graph neural network. In Proceedings of the 28th International Conference on Program Comprehension. ACM, Seoul, Republic of Korea, 184–195.

Digital Library

[52]

Alexander LeClair, Siyuan Jiang, and Collin McMillan. 2019. A neural model for generating natural language summaries of program subroutines. In Proceedings of the 41st International Conference on Software Engineering. IEEE / ACM, Montreal, QC, Canada, 795–806.

Digital Library

[53]

Alexander LeClair and Collin McMillan. 2019. Recommendations for datasets for source code summarization. In Proceedings of the 23rd Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Minneapolis, MN, USA, 3931–3937.

[54]

Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Online, 7871–7880.

[55]

Jia Li, Yongmin Li, Ge Li, Xing Hu, Xin Xia, and Zhi Jin. 2021. EditSum: A retrieve-and-edit framework for source code summarization. In Proceedings of the 36th International Conference on Automated Software Engineering. IEEE, Melbourne, Australia, 155–166.

Digital Library

[56]

Chen Lin, Zhichao Ouyang, Junqing Zhuang, Jianqiang Chen, Hui Li, and Rongxin Wu. 2021. Improving code summarization with block-wise abstract syntax tree splitting. In Proceedings of the 29th International Conference on Program Comprehension. IEEE, Madrid, Spain, 184–195.

[57]

Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics – Workshop on Text Summarization Branches Out. Association for Computational Linguistics, Barcelona, Spain, 74–81.

[58]

Shangqing Liu, Yu Chen, Xiaofei Xie, Jing Kai Siow, and Yang Liu. 2020. Automatic code summarization via multi-dimensional semantic fusing in GNN. CoRR abs/2006.05405 (2020).

[59]

Yang Liu. 2019. Fine-tune BERT for extractive summarization. CoRR abs/1903.10318 (2019).

[60]

Yang Liu and Mirella Lapata. 2019. Text summarization with pretrained encoders. In Proceedings of the 23rd Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Hong Kong, China, 3728–3738.

[61]

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. CoRR abs/1907.11692 (2019), 1–13.

[62]

Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin B. Clement, Dawn Drain, Daxin Jiang, Duyu Tang, Ge Li, Lidong Zhou, Linjun Shou, Long Zhou, Michele Tufano, Ming Gong, Ming Zhou, Nan Duan, Neel Sundaresan, Shao Kun Deng, Shengyu Fu, and Shujie Liu. 2021. CodeXGLUE: A machine learning benchmark dataset for code understanding and generation. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks. virtual, 1–14.

[63]

Robert C. Martin. 2009. Clean Code: A Handbook of Agile Software Craftsmanship. Pearson Education.

Digital Library

[64]

Mani Maybury. 1999. Advances in Automatic Text Summarization. MIT Press.

[65]

Paul W. McBurney and Collin McMillan. 2016. Automatic source code summarization of context for Java methods. IEEE Transactions on Software Engineering 42, 2 (2016), 103–119.

Digital Library

[66]

Microsoft. 2021. CodeBERT. site: https://github.com/microsoft/CodeBERT. Accessed August, 2022.

[67]

David M. Mimno, Wei Li, and Andrew McCallum. 2007. Mixtures of hierarchical topics with pachinko allocation. In Proceedings of the 24th International Conference Machine Learning. ACM, Corvallis, Oregon, USA, 633–640.

Digital Library

[68]

Jessica Moore, Ben Gelman, and David Slater. 2019. A convolutional neural network for language-agnostic source code summarization. In Proceedings of the 14th International Conference on Evaluation of Novel Approaches to Software Engineering. SciTePress, Heraklion, Crete, Greece, 15–26.

Digital Library

[69]

Laura Moreno, Jairo Aponte, Giriprasad Sridhara, Andrian Marcus, Lori L. Pollock, and K. Vijay-Shanker. 2013. Automatic generation of natural language summaries for Java classes. In Proceedings of the 21st International Conference on Program Comprehension. IEEE Computer Society, San Francisco, CA, USA, 23–32.

[70]

Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017. SummaRuNNer: A recurrent neural network based sequence model for extractive summarization of documents. In Proceedings of the 31st Conference on Artificial Intelligence. AAAI Press, San Francisco, California, USA, 3075–3081.

[71]

Ramesh Nallapati, Bowen Zhou, and Mingbo Ma. 2016. Classify or select: Neural architectures for extractive document summarization. CoRR abs/1611.04244 (2016).

[72]

OpenAI. 2022. ChatGPT. site: https://chat-gpt.org/Accessed March, 2023.

[73]

Tatsuro Oya, Yashar Mehdad, Giuseppe Carenini, and Raymond T. Ng. 2014. A template-based abstractive meeting summarization: Leveraging summary and source text relationships. In Proceedings of the 8th International Natural Language Generation Conference. The Association for Computer Linguistics, Philadelphia, PA, USA, 45–53.

[74]

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. ACL, Philadelphia, PA, USA, 311–318.

Digital Library

[75]

Long N. Phan, Hieu Tran, Daniel Le, Hieu Nguyen, James T. Anibal, Alec Peltekian, and Yanfang Ye. 2021. CoTexT: Multi-task learning with code-text transformer. CoRR abs/2105.08645 (2021).

[76]

Jonathan Pilault, Raymond Li, Sandeep Subramanian, and Chris Pal. 2020. On extractive and abstractive neural document summarization with transformer language models. In Proceedings of the 25th Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Online, 9308–9319.

[77]

Maja Popovic. 2015. chrF: Character n-gram F-score for automatic MT evaluation. In Proceedings of the 10th Workshop on Statistical Machine Translation. The Association for Computer Linguistics, Lisbon, Portugal, 392–395.

[78]

Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 21, 140 (2020), 1–67.

[79]

Sarah Rastkar. 2010. Summarizing software concerns. In Proceedings of the 32nd International Conference on Software Engineering. ACM, Cape Town, South Africa, 527–528.

Digital Library

[80]

Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Association for Computational Linguistics, Hong Kong, China, 3980–3990.

[81]

Paige Rodeghero, Collin McMillan, Paul W. McBurney, Nigel Bosch, and Sidney K. D’Mello. 2014. Improving automated source code summarization via an eye-tracking study of programmers. In Proceedings of the 36th International Conference on Software Engineering. ACM, Hyderabad, India, 390–401.

Digital Library

[82]

Devjeet Roy, Sarah Fakhoury, and Venera Arnaoudova. 2021. Reassessing automatic evaluation metrics for code summarization tasks. In Proceedings of the 29th Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, Athens, Greece, 1105–1116.

Digital Library

[83]

Salesforce. 2021. CodeT5. site: https://github.com/salesforce/CodeT5Accessed August, 2022.

[84]

Gerard Salton, Anita Wong, and Chung-Shu Yang. 1975. A vector space model for automatic indexing. Commun. ACM 18, 11 (1975), 613–620.

Digital Library

[85]

Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Vancouver, Canada, 1073–1083.

[86]

Ramin Shahbazi, Rishab Sharma, and Fatemeh H. Fard. 2021. API2Com: On the improvement of automatically generated code comments using API Documentations. In Proceedings of the 29th International Conference on Program Comprehension. IEEE, Madrid, Spain, 411–421.

[87]

Ensheng Shi, Yanlin Wang, Lun Du, Junjie Chen, Shi Han, Hongyu Zhang, Dongmei Zhang, and Hongbin Sun. 2021. Neural code summarization: How far are we?. In CoRR, Vol. abs/2107.07112.

[88]

Ensheng Shi, Yanlin Wang, Lun Du, Junjie Chen, Shi Han, Hongyu Zhang, Dongmei Zhang, and Hongbin Sun. 2022. On the evaluation of neural code summarization. In Proceedings of the 44th International Conference on Software Engineering. IEEE, Pittsburgh, PA, USA, 1597–1608.

Digital Library

[89]

Ensheng Shi, Yanlin Wang, Lun Du, Hongyu Zhang, Shi Han, Dongmei Zhang, and Hongbin Sun. 2021. CAST: Enhancing code summarization with hierarchical splitting and reconstruction of abstract syntax trees. In Proceedings of the 26th Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Virtual Event / Punta Cana, Dominican Republic, 4053–4062.

[90]

Tian Shi, Yaser Keneshloo, Naren Ramakrishnan, and Chandan K. Reddy. 2021. Neural abstractive text summarization with sequence-to-sequence models. Transactions on Data Science 2, 1 (2021), 1:1–1:37.

[91]

Yusuke Shido, Yasuaki Kobayashi, Akihiro Yamamoto, Atsushi Miyamoto, and Tadayuki Matsumura. 2019. Automatic source code summarization with extended tree-LSTM. In Proceedings of the 18th International Joint Conference on Neural Networks. IEEE, Hungary, 1–8.

[92]

GraphPad Software. 1995. Friedman’s test. site: https://www.graphpad.com/guides/prism/latest/statistics/how_the_friedman_test_works.htm Accessed March, 2023.

[93]

GraphPad Software. 1995. Which multiple comparisons tests does Prism offer? https://www.graphpad.com/guides/prism/latest/statistics/stat_summary_of_multiple_comparison.htm. Accessed March, 2023

[94]

Giriprasad Sridhara, Emily Hill, Divya Muppaneni, Lori L. Pollock, and K. Vijay-Shanker. 2010. Towards automatically generating summary comments for Java methods. In Proceedings of the 25th International Conference on Automated Software Engineering. ACM, Antwerp, Belgium, 43–52.

Digital Library

[95]

Sean Stapleton, Yashmeet Gambhir, Alexander LeClair, Zachary Eberhart, Westley Weimer, Kevin Leach, and Yu Huang. 2020. A human study of comprehension and code summarization. In Proceedings of the 28th International Conference on Program Comprehension. ACM, Seoul, Republic of Korea, 2–13.

Digital Library

[96]

Josef Steinberger and Karel Jezek. 2009. Update summarization based on latent semantic analysis. In Proceedings of the 12th International Conference on Text, Speech and Dialogue. Springer, Pilsen, Czech Republic, 77–84.

Digital Library

[97]

Josef Steinberger and Karel Jezek. 2004. Using latent semantic analysis in text summarization and summary evaluation. In Proceedings of the 7th International Conference ISIM. 1–8.

[98]

Weisong Sun, Chunrong Fang, Yuchen Chen, Quanjun Zhang, Guanhong Tao, Yudu You, Tingxu Han, Yifei Ge, Yuling Hu, Bin Luo, and Zhenyu Chen. 2022. EACS. https://github.com/wssun/EACS

[99]

Weisong Sun, Chunrong Fang, Yuchen Chen, Guanhong Tao, Tingxu Han, and Quanjun Zhang. 2022. Code search based on context-aware code translation. In Proceedings of the 44th International Conference on Software Engineering. ACM, Pittsburgh, PA, USA, 388–400.

Digital Library

[100]

Ted Tenny. 1988. Program readability: Procedures versus comments. IEEE Transactions on Software Engineering 14, 9 (1988), 1271–1279.

Digital Library

[101]

Maciej Tomczak and Ewa Tomczak. 2014. The need to report effect size estimates revisited. An overview of some recommended measures of effect size. Trends in Sport Sciences 21, 1 (2014), 19–25.

[102]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems. Curran Associates Inc., Long Beach, CA, USA, 5998–6008.

[103]

Yao Wan, Zhou Zhao, Min Yang, Guandong Xu, Haochao Ying, Jian Wu, and Philip S. Yu. 2018. Improving automatic source code summarization via deep reinforcement learning. In Proceedings of the 33rd International Conference on Automated Software Engineering. ACM/IEEE, Montpellier, France, 397–407.

Digital Library

[104]

Shuai Wang, Xiang Zhao, Bo Li, Bin Ge, and Daquan Tang. 2017. Integrating extractive and abstractive models for long text summarization. In Proceedings of the 6th International Congress on Big Data, BigData Congress. IEEE Computer Society, Honolulu, HI, USA, 305–312.

[105]

Wenhua Wang, Yuqun Zhang, Yulei Sui, Yao Wan, Zhou Zhao, Jian Wu, Philip Yu, and Guandong Xu. 2020. Reinforcement-learning-guided source code summarization using hierarchical attention. IEEE Transactions on Software Engineering (Early Access) (2020), 1–19.

[106]

Yanlin Wang, Ensheng Shi, Lun Du, Xiaodi Yang, Yuxuan Hu, Shi Han, Hongyu Zhang, and Dongmei Zhang. 2021. CoCoSum: Contextual code summarization with multi-relational graph neural network. CoRR abs/2107.01933 (2021), 1–24.

[107]

Yue Wang, Weishi Wang, Shafiq R. Joty, and Steven C. H. Hoi. 2021. CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. In Proceedings of the 26th Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Virtual Event / Punta Cana, Dominican Republic, 8696–8708.

[108]

Bolin Wei, Ge Li, Xin Xia, Zhiyi Fu, and Zhi Jin. 2019. Code generation as a dual task of code summarization. In Proceedings of the 33rd Annual Conference on Neural Information Processing Systems (NeurIPS 2019). Vancouver, BC, Canada, 6559–6569.

[109]

Bolin Wei, Yongmin Li, Ge Li, Xin Xia, and Zhi Jin. 2020. Retrieve and refine: Exemplar-based neural comment generation. In Proceedings of the 35th International Conference on Automated Software Engineering. IEEE, Melbourne, Australia, 349–360.

Digital Library

[110]

Frank Wilcoxon, S. K. Katti, and Roberta A. Wilcox. 1963. Critical Values and Probability Levels for the Wilcoxon Rank Sum Test and the Wilcoxon Signed Rank Test. American Cyanamid Company.

[111]

Claes Wohlin, Per Runeson, Martin Höst, Magnus C. Ohlsson, and Björn Regnell. 2012. Experimentation in Software Engineering. Springer.

[112]

Edmund Wong, Taiyue Liu, and Lin Tan. 2015. CloCom: Mining existing source code for automatic comment generation. In Proceedings of the 22nd International Conference on Software Analysis, Evolution, and Reengineering. IEEE Computer Society, Montreal, QC, Canada, 380–389.

[113]

Scott N. Woodfield, Hubert E. Dunsmore, and Vincent Yun Shen. 1981. The effect of modularization and comments on program comprehension. In Proceedings of the 5th International Conference on Software Engineering. IEEE Computer Society, San Diego, California, USA, 215–223.

Digital Library

[114]

Hongqiu Wu, Hai Zhao, and Min Zhang. 2021. Code summarization with structure-induced transformer. In Proceedings of the Findings of the 59th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Online Event, 1078–1090.

[115]

Yuxiang Wu and Baotian Hu. 2018. Learning to extract coherent summary via deep reinforcement learning. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. AAAI Press, New Orleans, Louisiana, USA, 5602–5609.

[116]

Xin Xia, Lingfeng Bao, David Lo, Zhenchang Xing, Ahmed E. Hassan, and Shanping Li. 2018. Measuring program comprehension: A large-scale field study with professionals. IEEE Transactions on Software Engineering 44, 10 (2018), 951–976.

Digital Library

[117]

Zhen Yang, Jacky Keung, Xiao Yu, Xiaodong Gu, Zhengyuan Wei, Xiaoxue Ma, and Miao Zhang. 2021. A multi-modal transformer-based code summarization approach for smart contracts. In Proceedings of the 29th International Conference on Program Comprehension. IEEE, Madrid, Spain, 1–12.

[118]

Wenyuan Zeng, Wenjie Luo, Sanja Fidler, and Raquel Urtasun. 2016. Efficient summarization with read-again and copy mechanism. CoRR abs/1611.03382 (2016), 1–11.

[119]

Juan Zhai, Xiangzhe Xu, Yu Shi, Guanhong Tao, Minxue Pan, Shiqing Ma, Lei Xu, Weifeng Zhang, Lin Tan, and Xiangyu Zhang. 2020. CPC: Automatically classifying and propagating natural language comments via program analysis. In Proceedings of the 42nd International Conference on Software Engineering. ACM, Seoul, South Korea, 1359–1371.

Digital Library

[120]

Jiyang Zhang, Sheena Panthaplackel, Pengyu Nie, Raymond J. Mooney, Junyi Jessy Li, and Milos Gligoric. 2021. Learning to generate code comments from class hierarchies. CoRR abs/2103.13426 (2021).

[121]

Jian Zhang, Xu Wang, Hongyu Zhang, Hailong Sun, and Xudong Liu. 2020. Retrieval-based neural source code summarization. In Proceedings of the 42nd International Conference on Software Engineering. ACM, Seoul, South Korea, 1385–1397.

Digital Library

[122]

Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. 2020. PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization. In Proceedings of the 37th International Conference on Machine Learning. PMLR, Virtual Event, 11328–11339.

[123]

Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, and Tiejun Zhao. 2018. Neural document summarization by jointly learning to score and select sentences. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Melbourne, Australia, 654–663.

[124]

Yu Zhou, Xiaoqing Zhang, Juanjuan Shen, Tingting Han, Taolue Chen, and Harald C. Gall. 2021. Adversarial robustness of deep code comment generation. ACM Transactions on Software Engineering and Methodology 1, 1 (2021), 1–30.

[125]

Qihao Zhu, Zeyu Sun, Xiran Liang, Yingfei Xiong, and Lu Zhang. 2020. OCoR: An overlapping-aware code retriever. In Proceedings of the 35th International Conference on Automated Software Engineering. IEEE, Melbourne, Australia, 883–894.

Digital Library

[126]

Yuxiang Zhu and Minxue Pan. 2019. Automatic code summarization: A systematic literature review. CoRR abs/1909.04352 (2019).

Cited By

Song YYang LLuo WXiao XTang Z(2025)Boosting multi-document summarization with hierarchical graph convolutional networksNeurocomputing10.1016/j.neucom.2024.128753614(128753)Online publication date: Jan-2025
https://doi.org/10.1016/j.neucom.2024.128753
Zhao JSun XFeng C(2025)Introducing bidirectional attention for autoregressive models in abstractive summarizationInformation Sciences10.1016/j.ins.2024.121497689(121497)Online publication date: Jan-2025
https://doi.org/10.1016/j.ins.2024.121497
Luo JShi HZhang YWang RShen YChen YShi XLiu RHu CJiang Yd'Amorim M(2024)CVECenter: Industry Practice of Automated Vulnerability Management for Linux Distribution CommunityCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663852(329-339)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663852
Show More Cited By

Index Terms

An Extractive-and-Abstractive Framework for Source Code Summarization
1. Software and its engineering
  1. Software notations and tools
    1. Software maintenance tools

Recommendations

Extractive vs. NLG-based abstractive summarization of evaluative text: the effect of corpus controversiality
INLG '08: Proceedings of the Fifth International Natural Language Generation Conference

Extractive summarization is the strategy of concatenating extracts taken from a corpus into a summary, while abstractive summarization involves paraphrasing the corpus using novel sentences. We define a novel measure of corpus controversiality of ...
Why My Code Summarization Model Does Not Work: Code Comment Improvement with Category Prediction
Continuous Special Section: AI and SE

Code summarization aims at generating a code comment given a block of source code and it is normally performed by training machine learning algorithms on existing code block-comment pairs. Code comments in practice have different intentions. For example,...
A Neural-Network based Code Summarization Approach by Using Source Code and its Call Dependencies
Internetware '19: Proceedings of the 11th Asia-Pacific Symposium on Internetware

Code summarization aims at generating natural language abstraction for source code, and it can be of great help for program comprehension and software maintenance. The current code summarization approaches have made progress with neural-network. However,...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Software Engineering and Methodology

ACM Transactions on Software Engineering and Methodology Volume 33, Issue 3

March 2024

943 pages

EISSN:1557-7392

DOI:10.1145/3613618

Editor:
Mauro Pezzé
USI Universitá della Svizzera italiana and SIT Schaffhausen Institute of Technology, Switzerland

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 March 2024

Online AM: 13 November 2023

Accepted: 03 November 2023

Revised: 04 April 2023

Received: 11 August 2022

Published in TOSEM Volume 33, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
Program B for Outstanding PhD Candidate of Nanjing University

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
729
Total Downloads

Downloads (Last 12 months)638
Downloads (Last 6 weeks)66

Reflects downloads up to 23 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Song YYang LLuo WXiao XTang Z(2025)Boosting multi-document summarization with hierarchical graph convolutional networksNeurocomputing10.1016/j.neucom.2024.128753614(128753)Online publication date: Jan-2025
https://doi.org/10.1016/j.neucom.2024.128753
Zhao JSun XFeng C(2025)Introducing bidirectional attention for autoregressive models in abstractive summarizationInformation Sciences10.1016/j.ins.2024.121497689(121497)Online publication date: Jan-2025
https://doi.org/10.1016/j.ins.2024.121497
Luo JShi HZhang YWang RShen YChen YShi XLiu RHu CJiang Yd'Amorim M(2024)CVECenter: Industry Practice of Automated Vulnerability Management for Linux Distribution CommunityCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering10.1145/3663529.3663852(329-339)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3663529.3663852
Fang CSun WChen YChen XWei ZZhang QYou YLuo BLiu YChen Z(2024)Esale: Enhancing Code-Summary Alignment Learning for Source Code SummarizationIEEE Transactions on Software Engineering10.1109/TSE.2024.342227450:8(2077-2095)Online publication date: Aug-2024
https://doi.org/10.1109/TSE.2024.3422274
Huang QLiao DXing ZYuan ZLu QXu XLu J(2024)SE Factual Knowledge in Frozen Giant Code Model: A Study on FQN and Its RetrievalIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.343688336:12(9220-9234)Online publication date: Dec-2024
https://doi.org/10.1109/TKDE.2024.3436883
Zhu JMiao YXu TZhu JSun X(2024)On the Effectiveness of Large Language Models in Statement-level Code Summarization2024 IEEE 24th International Conference on Software Quality, Reliability and Security (QRS)10.1109/QRS62785.2024.00030(216-227)Online publication date: 1-Jul-2024
https://doi.org/10.1109/QRS62785.2024.00030
Zhang FFan JLi WKhoo S(2024)Code Parameter Summarization Based on Transformer and Fusion StrategyIET Software10.1049/sfw2/37066732024:1Online publication date: 31-Dec-2024
https://doi.org/10.1049/sfw2/3706673
Zhang XHou XQiao XSong W(2024)A review of automatic source code summarizationEmpirical Software Engineering10.1007/s10664-024-10553-629:6Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1007/s10664-024-10553-6
Chen PWu SChen ZZhang JZhang XFeng Z(2023)LenANet: A Length-Controllable Attention Network for Source Code SummarizationNeural Information Processing10.1007/978-981-99-8145-8_43(567-578)Online publication date: 27-Nov-2023
https://doi.org/10.1007/978-981-99-8145-8_43

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents