research-article

Open access

Mining idioms in the wild

Authors:

Aishwarya Sivaraman,

Tobi Akomolede,

Satish ChandraAuthors Info & Claims

ICSE-SEIP '22: Proceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice

Pages 187 - 196

https://doi.org/10.1145/3510457.3513046

Published: 17 October 2022 Publication History

Abstract

Existing code repositories contain numerous instances of code patterns that are idiomatic ways of accomplishing a particular programming task. Sometimes, the programming language in use supports specific operators or APIs that can express the same idiomatic imperative code much more succinctly. However, those code patterns linger in repositories because the developers may be unaware of the new APIs or have not gotten around to them. Detection of idiomatic code can also point to the need for new APIs.

We share our experiences in mining imperative idiomatic patterns from the Hack repo at Facebook. We found that existing techniques either cannot identify meaningful patterns from syntax trees or require test-suite-based dynamic analysis to incorporate semantic properties to mine useful patterns. The key insight of the approach proposed in this paper --- Jezero --- is that semantic idioms from a large codebase can be learned from canonicalized dataflow trees. We propose a scalable, lightweight static analysis-based approach to construct such a tree that is well suited to mine semantic idioms using nonparametric Bayesian methods.

Our experiments with Jezero on Hack code show a clear advantage of adding canonicalized dataflow information to ASTs: Jezero was significantly more effective in finding new refactoring opportunities from unannotated legacy code than a baseline that did not have the dataflow augmentation.

References

[1]

Miltiadis Allamanis, Earl T Barr, Christian Bird, Premkumar Devanbu, Mark Mar-ron, and Charles Sutton. 2018. Mining semantic loop idioms. IEEE Transactions on Software Engineering 44, 7 (2018), 651--668.

[2]

Miltiadis Allamanis and Charles Sutton. 2014. Mining idioms from source code. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. 472--483.

Digital Library

[3]

Johannes Bader, Andrew Scott, Michael Pradel, and Satish Chandra. 2019. Getafix: Learning to fix bugs automatically. Proceedings of the ACM on Programming Languages 3, OOPSLA (2019), 1--27.

Digital Library

[4]

Ira D Baxter, Andrew Yahin, Leonardo Moura, Marcelo Sant'Anna, and Lorraine Bier. 1998. Clone detection using abstract syntax trees. In Proceedings. International Conference on Software Maintenance (Cat. No. 98CB36272). IEEE, 368--377.

[5]

Trevor Cohn, Phil Blunsom, and Sharon Goldwater. 2010. Inducing tree-substitution grammars. The Journal of Machine Learning Research 11 (2010).

[6]

Yaniv David and Eran Yahav. 2014. Tracelet-based code search in executables. Acm Sigplan Notices 49, 6 (2014), 349--360.

Digital Library

[7]

Jean-Rémy Falleri, Floréal Morandat, Xavier Blanc, Matias Martinez, and Martin Monperrus. 2014. Fine-grained and Accurate Source Code Differencing. In Proceedings of the International Conference on Automated Software Engineering.

Digital Library

[8]

Xiang Gao, Shraddha Barke, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani, Alan Leung, Nachiappan Nagappan, and Ashish Tiwari. 2020. Feedback-driven semi-supervised synthesis of program transformations. Proceedings of the ACM on Programming Languages 4, OOPSLA (2020), 1--30.

Digital Library

[9]

Andrew Gelman, John B Carlin, Hal S Stern, David B Dunson, Aki Vehtari, and Donald B Rubin. 2013. Bayesian data analysis. CRC press.

[10]

Lingxiao Jiang, Ghassan Misherghi, Zhendong Su, and Stephane Glondu. 2007. DECKARD: Scalable and Accurate Tree-Based Detection of Code Clones. In Proceedings of the 29th International Conference on Software Engineering (ICSE '07). IEEE Computer Society, Washington, DC, USA, 96--105.

Digital Library

[11]

Rainer Koschke, Raimar Falke, and Pierre Frenzel. 2006. Clone detection using abstract syntax suffix trees. In Working Conference on Reverse Engineering. IEEE.

Digital Library

[12]

Zhenmin Li, Shan Lu, Suvda Myagmar, and Yuanyuan Zhou. 2004. CP-Miner: A Tool for Finding Copy-paste and Related Bugs in Operating System Code. In OSdi, Vol. 4. 289--302.

Digital Library

[13]

Percy Liang, Michael I Jordan, and Dan Klein. 2010. Type-based MCMC. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. 573--581.

[14]

David Mandelin, Lin Xu, Rastislav Bodík, and Doug Kimelman. 2005. Jungloid mining: helping to navigate the API jungle. ACM Sigplan Notices 40, 6 (2005).

Digital Library

[15]

Na Meng, Lisa Hua, Miryung Kim, and Kathryn S McKinley. 2015. Does automated refactoring obviate systematic editing?. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol. 1. IEEE, 392--402.

[16]

Na Meng, Miryung Kim, and Kathryn S McKinley. 2011. Systematic editing: generating program transformations from an example. ACM SIGPLAN Notices 46, 6 (2011), 329--342.

Digital Library

[17]

Na Meng, Miryung Kim, and Kathryn S McKinley. 2013. LASE: locating and applying systematic edits by learning from examples. In 2013 35th International Conference on Software Engineering (ICSE). IEEE, 502--511.

[18]

Kevin P Murphy. 2012. Machine learning: a probabilistic perspective. MIT press.

Digital Library

[19]

Varot Premtoon, James Koppel, and Armando Solar-Lezama. 2020. Semantic code search via equational reasoning. In PLDI. 1066--1082.

[20]

Dhavleesh Rattan, Rajesh Bhatia, and Maninder Singh. 2013. Software clone detection: A systematic review. Information and Software Technology 55, 7 (2013).

[21]

Steven P Reiss. 2009. Semantics-based code search. In 2009 IEEE 31st International Conference on Software Engineering. IEEE, 243--253.

Digital Library

[22]

Charles Rich and Richard C. Waters. 1988. The programmer's apprentice: A research overview. Computer 21, 11 (1988), 10--25.

Digital Library

[23]

Reudismam Rolim, Gustavo Soares, Loris D'Antoni, Oleksandr Polozov, Sumit Gulwani, Rohit Gheyi, Ryo Suzuki, and Björn Hartmann. 2017. Learning syntactic program transformations from examples. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). IEEE, 404--415.

Digital Library

[24]

Barry K Rosen, Mark N Wegman, and F Kenneth Zadeck. 1988. Global value numbers and redundant computations. In Proceedings of the 15th ACM SIGPLAN-SIGACT symposium on Principles of programming languages. 12--27.

Digital Library

[25]

Hitesh Sajnani, Vaibhav Saini, Jeffrey Svajlenko, Chanchal K Roy, and Cristina V Lopes. 2016. SourcererCC: Scaling code clone detection to big-code. In Software Engineering (ICSE), 2016 IEEE. IEEE, 1157--1168.

Digital Library

[26]

Aishwarya Sivaraman, Tianyi Zhang, Guy Van den Broeck, and Miryung Kim. 2019. Active inductive logic programming for code search. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 292--303.

Digital Library

[27]

Jue Wang, Yingnong Dang, Hongyu Zhang, Kai Chen, Tao Xie, and Dongmei Zhang. 2013. Mining succinct and high-coverage API usage patterns from source code. In Conference on Mining Software Repositories (MSR). IEEE, 319--328.

[28]

Wenhan Wang, Ge Li, Bo Ma, Xin Xia, and Zhi Jin. 2020. Detecting code clones with graph neural network and flow-augmented abstract syntax tree. In 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 261--271.

[29]

Xiaoyin Wang, David Lo, Jiefeng Cheng, Lu Zhang, Hong Mei, and Jeffrey Xu Yu. 2010. Matching dependence-related queries in the system dependence graph. In Proceedings of the IEEE/ACM international conference on Automated software engineering. 457--466.

Digital Library

[30]

Hao Zhong, Tao Xie, Lu Zhang, Jian Pei, and Hong Mei. 2009. MAPO: Mining and recommending API usage patterns. In European Conference on Object-Oriented Programming. Springer, 318--343.

Digital Library

Cited By

OBrien DDyer RNguyen TRajan HRoychoudhury APaiva AAbreu RStorey M(2024)Data-Driven Evidence-Based Syntactic Sugar DesignProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639580(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639580
Yang YHu XXia XLo DYang XRoychoudhury APaiva AAbreu RStorey M(2024)Streamlining Java Programming: Uncovering Well-Formed Idioms with IdioMineProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639135(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639135
Büyük ONizam A(2023)Deep learning with class-level abstract syntax tree and code histories for detecting code modification requirementsJournal of Systems and Software10.1016/j.jss.2023.111851206:COnline publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1016/j.jss.2023.111851
Show More Cited By

Recommendations

Making Python code idiomatic by automatic refactoring non-idiomatic Python code with pythonic idioms
ESEC/FSE 2022: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Compared to other programming languages (e.g., Java), Python has more idioms to make Python code concise and efficient. Although pythonic idioms are well accepted in the Python community, Python programmers are often faced with many challenges in ...
Mining idioms from source code
FSE 2014: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering

We present the first method for automatically mining code idioms from a corpus of previously written, idiomatic software projects. We take the view that a code idiom is a syntactic fragment that recurs across projects and has a single semantic purpose. ...
Fixing idioms: a recursion primitive for applicative DSLs
PEPM '13: Proceedings of the ACM SIGPLAN 2013 workshop on Partial evaluation and program manipulation

In a lazy functional language, the standard encoding of recursion in DSLs uses the host language's recursion, so that DSL algorithms automatically use the host language's least fixpoints, even though many domains require algorithms to produce different ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE-SEIP '22: Proceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice

May 2022

371 pages

ISBN:9781450392266

DOI:10.1145/3510457

Conference Chairs:
Mark Harman
Facebook, Inc & University College London
,
Heather Miller
Carnegie Mellon University

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution-NoDerivatives International 4.0 License.

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

IEEE CS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2022

Check for updates

Qualifiers

Research-article

Conference

ICSE '22

Sponsor:

SIGSOFT

ICSE '22: 44th International Conference on Software Engineering

May 21 - 29, 2022

Pennsylvania, Pittsburgh

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
226
Total Downloads

Downloads (Last 12 months)110
Downloads (Last 6 weeks)21

Reflects downloads up to 11 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

OBrien DDyer RNguyen TRajan HRoychoudhury APaiva AAbreu RStorey M(2024)Data-Driven Evidence-Based Syntactic Sugar DesignProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639580(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639580
Yang YHu XXia XLo DYang XRoychoudhury APaiva AAbreu RStorey M(2024)Streamlining Java Programming: Uncovering Well-Formed Idioms with IdioMineProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639135(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639135
Büyük ONizam A(2023)Deep learning with class-level abstract syntax tree and code histories for detecting code modification requirementsJournal of Systems and Software10.1016/j.jss.2023.111851206:COnline publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1016/j.jss.2023.111851
Zhang ZXing ZXia XXu XZhu LRoychoudhury ACadar CKim M(2022)Making Python code idiomatic by automatic refactoring non-idiomatic Python code with pythonic idiomsProceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3540250.3549143(696-708)Online publication date: 7-Nov-2022
https://dl.acm.org/doi/10.1145/3540250.3549143

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents