Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

EASE: An Effort-aware Extension of Unsupervised Key Class Identification Approaches

Published: 21 April 2024 Publication History

Abstract

Key class identification approaches aim at identifying the most important classes to help developers, especially newcomers, start the software comprehension process. So far, many supervised and unsupervised approaches have been proposed; however, they have not considered the effort to comprehend classes. In this article, we identify the challenge of “effort-aware key class identification”; to partially tackle it, we propose an approach, EASE, which is implemented through a modification to existing unsupervised key class identification approaches to take into consideration the effort to comprehend classes. First, EASE chooses a set of network metrics that has a wide range of applications in the existing unsupervised approaches and also possesses good discriminatory power. Second, EASE normalizes the network metric values of classes to quantify the probability of any class to be a key class and utilizes Cognitive Complexity to estimate the effort required to comprehend classes. Third, EASE proposes a metric, RKCP, to measure the relative key-class proneness of classes and further uses it to sort classes in descending order. Finally, an effort threshold is utilized, and the top-ranked classes within the threshold are identified as the cost-effective key classes. Empirical results on a set of 18 software systems show that (i) the proposed effort-aware variants perform significantly better in almost all (≈98.33%) the cases, (ii) they are superior to most of the baseline approaches with only several exceptions, and (iii) they are scalable to large-scale software systems. Based on these findings, we suggest that (i) we should resort to effort-aware key class identification techniques in budget-limited scenarios; and (ii) when using different techniques, we should carefully choose the weighting mechanism to obtain the best performance.

References

[1]
Amritanshu Agrawal and Tim Menzies. 2018. Is “better data” better than “better data miners”?: On the benefits of tuning SMOTE for defect prediction. In Proceedings of the 40th International Conference on Software Engineering (ICSE’18), Michel Chaudron, Ivica Crnkovic, Marsha Chechik, and Mark Harman (Eds.). ACM, 1050–1061.
[2]
Sara Alspaugh, Kristen R. Walcott, Michael Belanich, Gregory M. Kapfhammer, and Mary Lou Soffa. 2007. Efficient time-aware prioritization with knapsack solvers. In Proceedings of the 1st ACM International Workshop on Empirical Assessment of Software Engineering Languages and Technologies: Held in Conjunction with the 22nd IEEE/ACM International Conference on Automated Software Engineering (ASE’07). 13–18.
[3]
Richard A. Armstrong. 2014. When to use the Bonferroni correction. Ophthalm. Physiol. Optics 34, 5 (2014), 502–508.
[4]
Marvin Muñoz Barón, Marvin Wyrich, and Stefan Wagner. 2020. An empirical validation of cognitive complexity as a measure of source code understandability. In Proceedings of the ACM/IEEEInternational Symposium on Empirical Software Engineering and Measurement (ESEM’20), Maria Teresa Baldassarre, Filippo Lanubile, Marcos Kalinowski, and Federica Sarro (Eds.). ACM, 5:1–5:12.
[5]
Markus Bauer. 1999. Analysing software systems by using combinations of metrics. In Object-oriented Technology, ECOOP’99 Workshop Reader, ECOOP’99 Workshops, Panels, and Posters, Lisbon, Portugal, June 14–18, 1999, Proceedings (Lecture Notes in Computer Science), Vol. 1743. Springer, 170–171.
[6]
Lionel C. Briand, John W. Daly, and Jürgen Wüst. 1999. A unified framework for coupling measurement in object-oriented systems. IEEE Trans. Softw. Eng. 25, 1 (1999), 91–121.
[7]
Lionel C. Briand, Premkumar T. Devanbu, and Walcélio L. Melo. 1997. An investigation into coupling measures for C++. In Proceedings of the 19th International Conference on Software Engineering. ACM, 412–421.
[8]
G. Ann Campbell. 2018. Cognitive complexity: An overview and evaluation. In Proceedings of the International Conference on Technical Debt (TechDebt@ICSE’18), Robert L. Nord, Frank Buschmann, and Philippe Kruchten (Eds.). ACM, 57–58.
[9]
Chun Yong Chong and Sai Peck Lee. 2017. Automatic clustering constraints derivation from object-oriented software using weighted complex network with graph theory analysis. J. Syst. Softw. 133 (2017), 28–53.
[10]
Norman Cliff. 2014. Ordinal Methods for Behavioral Data Analysis. Psychology Press.
[11]
Giulio Concas, Michele Marchesi, Sandro Pinna, and Nicola Serra. 2007. Power-laws in a large object-oriented software system. IEEE Trans. Softw. Eng. 33, 10 (2007), 687–708.
[12]
Bas Cornelissen, Andy Zaidman, Arie van Deursen, Leon Moonen, and Rainer Koschke. 2009. A systematic survey of program comprehension through dynamic analysis. IEEE Trans. Softw. Eng. 35, 5 (2009), 684–702.
[13]
Cynthia L. Corritore and Susan Wiedenbeck. 2001. An exploratory study of program comprehension strategies of procedural and object-oriented programmers. Int. J. Hum. Comput. Stud. 54, 1 (2001), 1–23.
[14]
Uri Dekel and James D. Herbsleb. 2009. Reading the documentation of invoked API functions in program comprehension. In Proceedings of the 17th IEEE International Conference on Program Comprehension (ICPC’09). IEEE Computer Society, 168–177.
[15]
Hyunsook Do, Siavash Mirarab, Ladan Tahvildari, and Gregg Rothermel. 2010. The effects of time constraints on test case prioritization: A series of controlled experiments. IEEE Trans. Softw. Eng. 36, 5 (2010), 593–617.
[16]
Liliane do Nascimento Vale and Marcelo de Almeida Maia. 2019. Key classes in object-oriented systems: Detection and assessment. Int. J. Softw. Eng. Knowl. Eng. 29, 10 (2019), 1439–1463.
[17]
Fernando Brito e Abreu and Miguel Goulão. 2001. Coupling and cohesion as modularization drivers: Are we being over-persuaded? In Proceedings of the 5th Conference on Software Maintenance and Reengineering (CSMR’01), Pedro Sousa and Jürgen Ebert (Eds.). IEEE Computer Society, 47–57.
[18]
Fernando Brito e Abreu, Gonçalo Pereira, and Pedro Manuel Antunes Sousa. 2000. A coupling-guided cluster analysis approach to reengineer the modularity of object-oriented systems. In Proceedings of the 4th European Conference on Software Maintenance and Reengineering (CSMR’00). IEEE Computer Society, 13–22.
[19]
Janet Feigenspan, Sven Apel, Jörg Liebig, and Christian Kästner. 2011. Exploring software measures to assess program comprehension. In Proceedings of the 5th International Symposium on Empirical Software Engineering and Measurement (ESEM’11). IEEE Computer Society, 127–136.
[20]
Kecia Aline M. Ferreira, Mariza Andrade da Silva Bigonha, Roberto da Silva Bigonha, Luiz F. O. Mendes, and Heitor C. Almeida. 2012. Identifying thresholds for object-oriented software metrics. J. Syst. Softw. 85, 2 (2012), 244–257.
[21]
Lee Friedman and Oleg V. Komogortsev. 2019. Assessment of the effectiveness of seven biometric feature normalization techniques. IEEE Trans. Inf. Forens. Secur. 14, 10 (2019), 2528–2536.
[22]
Thomas Fritz, Gail C. Murphy, Emerson R. Murphy-Hill, Jingwen Ou, and Emily Hill. 2014. Degree-of-knowledge: Modeling a developer’s knowledge of code. ACM Trans. Softw. Eng. Methodol. 23, 2 (2014), 14:1–14:42.
[23]
Salvador García, Alberto Fernández, Julián Luengo, and Francisco Herrera. 2010. Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power. Inf. Sci. 180, 10 (2010), 2044–2064.
[24]
Salvador García and Francisco Herrera. 2008. An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J. Mach. Learn. Res. 9 (2008), 2677–2694.
[25]
Les Hatton. 2009. Power-law distributions of component size in general software systems. IEEE Trans. Softw Eng. 35, 4 (2009), 566–572.
[26]
Jorge E. Hirsch. 2005. An index to quantify an individual’s scientific research output. Proc. Natl. Acad. Sci. USA 102, 46 (2005), 16569–16572.
[27]
Ioana Sora. 2015. A PageRank based recommender system for identifying key classes in software systems. In Proceedings of the 10th IEEE Jubilee International Symposium on Applied Computational Intelligence and Informatics (SACI’15). IEEE, 495–500.
[28]
Bihui Jin, Liming Liang, Rousseau Ronald, and Egghe Leo. 2007. The R- and AR-indices: Complementing the h-index. Chinese Sci. Bull. 52, 6 (2007), 855–863.
[29]
Yasutaka Kamei, Shinsuke Matsumoto, Akito Monden, Ken-ichi Matsumoto, Bram Adams, and Ahmed E. Hassan. 2010. Revisiting common bug prediction findings using effort-aware models. In Proceedings of the IEEE International Conference on Software Maintenance. IEEE, 1–10.
[30]
Yasutaka Kamei, Emad Shihab, Bram Adams, Ahmed E. Hassan, Audris Mockus, Anand Sinha, and Naoyasu Ubayashi. 2012. A large-scale empirical study of just-in-time quality assurance. IEEE Trans. Softw. Eng. 39, 6 (2012), 757–773.
[31]
Dazhou Kang, Baowen Xu, Jianjiang Lu, and William C. Chu. 2004. A complexity measure for ontology based on UML. In Proceedings of the 10th IEEE International Workshop on Future Trends of Distributed Computing Systems (FTDCS’04). IEEE Computer Society, 222–228.
[32]
Stefan Lessmann, Bart Baesens, Christophe Mues, and Swantje Pietsch. 2008. Benchmarking classification models for software defect prediction: A proposed framework and novel findings. IEEE Trans. Softw. Eng. 34, 4 (2008), 485–496.
[33]
Timothy Lethbridge, Janice Singer, and Andrew Forward. 2003. How software engineers use documentation: The state of the practice. IEEE Softw. 20, 6 (2003), 35–39.
[34]
Charles X. Ling, Victor S. Sheng, and Qiang Yang. 2006. Test strategies for cost-sensitive decision trees. IEEE Trans. Knowl. Data Eng. 18, 8 (2006), 1055–1067.
[35]
Huihui Liu, Yijun Yu, Bixin Li, Yibiao Yang, and Ru Jia. 2018. Are smell-based metrics actually useful in effort-aware structural change-proneness prediction? An empirical study. In Proceedings of the 25th Asia-Pacific Software Engineering Conference (APSEC’18). IEEE, 315–324.
[36]
Panagiotis Louridas, Diomidis Spinellis, and Vasileios Vlachos. 2008. Power laws in software. ACM Trans. Softw. Eng. Methodol. 18, 1 (2008), 2:1–2:26.
[37]
Yutao Ma, Keqing He, Bing Li, Jing Liu, and Xiaoyan Zhou. 2010. A hybrid set of complexity metrics for large-scale object-oriented software systems. J. Comput. Sci. Technol. 25, 6 (2010), 1184–1201.
[38]
Jonathan I. Maletic and Andrian Marcus. 2001. Supporting program comprehension using semantic and structural information. In Proceedings of the 23rd International Conference on Software Engineering (ICSE’01). IEEE Computer Society, 103–112.
[39]
Andrian Marcus, Denys Poshyvanyk, and Rudolf Ferenc. 2008. Using the conceptual cohesion of classes for fault prediction in object-oriented systems. IEEE Trans. Softw. Eng. 34, 2 (2008), 287–300.
[40]
Paul W. McBurney, Siyuan Jiang, Marouane Kessentini, Nicholas A. Kraft, Ameer Armaly, Mohamed Wiem Mkaouer, and Collin McMillan. 2018. Towards prioritizing documentation effort. IEEE Trans. Softw. Eng. 44, 9 (2018), 897–913.
[41]
Thomas J. McCabe. 1976. A complexity measure. IEEE Trans. Softw. Eng. 2, 4 (1976), 308–320.
[42]
Thilo Mende and Rainer Koschke. 2010. Effort-aware defect prediction models. In Proceedings of the 14th European Conference on Software Maintenance and Reengineering. IEEE, 107–116.
[43]
P. Meyer, Harvey P. Siy, and Sanjukta Bhowmick. 2014. Identifying important classes of large software systems through k-Core decomposition. Adv. Complex Syst. 17, 7-8 (2014).
[44]
Roberto Minelli, Andrea Mocci, and Michele Lanza. 2015. I know what you did last summer: An investigation of how developers spend their time. In Proceedings of the IEEE 23rd International Conference on Program Comprehension (ICPC’15). IEEE Computer Society, 25–35.
[45]
Jaechang Nam, Sinno Jialin Pan, and Sunghun Kim. 2013. Transfer defect learning. In Proceedings of the 35th International Conference on Software Engineering (ICSE’13). 382–391.
[46]
Michael P. O’Brien, Jim Buckley, and Teresa M. Shaft. 2004. Expectation-based, inference-based, and bottom-up software comprehension. J. Softw. Mainten. Res. Pract. 16, 6 (2004), 427–447.
[47]
Mohd Hafeez Osman, Michel R. V. Chaudron, and Peter van der Putten. 2013. An analysis of machine learning algorithms for condensing reverse engineered class diagrams. In Proceedings of the IEEE International Conference on Software Maintenance. IEEE Computer Society, 140–149.
[48]
Weifeng Pan, Xin Du, Hua Ming, Dae-Kyoo Kim, and Zijiang Yang. 2023. Identifying key classes for initial software comprehension: Can we do it better? In Proceedings of the IEEE/ACM 45th International Conference on Software Engineering (ICSE’23). 1878–1889.
[49]
Weifeng Pan, Hua Ming, Carl K. Chang, Zijiang Yang, and Dae-Kyoo Kim. 2021. ElementRank: Ranking Java software classes and packages using a multilayer complex network-based approach. IEEE Trans. Softw. Eng. 47, 10 (2021), 2272–2295.
[50]
Weifeng Pan, Hua Ming, Dae-Kyoo Kim, and Zijiang Yang. 2023. Pride: Prioritizing documentation effort based on a PageRank-like algorithm and simple filtering rules. IEEE Trans. Softw. Eng. 49, 3 (2023), 1118–1151.
[51]
Weifeng Pan, Hua Ming, Zijiang Yang, and Tian Wang. 2022. Comments on “Using k-core Decomposition on class dependency networks to improve bug prediction model’s practical performance.” IEEE Trans. Softw. Eng. 48, 12 (2022), 5176–5187.
[52]
Weifeng Pan, Beibei Song, Kangshun Li, and Kejun Zhang. 2018. Identifying key classes in object-oriented software using generalized k-core decomposition. Fut. Gen. Comput. Syst. 81 (2018), 188–202.
[53]
Fabrizio Perin, Lukas Renggli, and Jorge Ressia. 2010. Ranking software artifacts. In Proceedings of the 4th Workshop on FAMIX and Moose in Reengineering (FAMOOSr’10), Vol. 120. Citeseer.
[54]
Alex Potanin, James Noble, Marcus R. Frean, and Robert Biddle. 2005. Scale-free geometry in OO programs. Commun. ACM 48, 5 (2005), 99–103.
[55]
Amarjeet Prajapati and Jitender Kumar Chhabra. 2017. Improving modular structure of software system using structural and lexical dependency. Inf. Softw. Technol. 82 (2017), 96–120.
[56]
Yu Qu, Qinghua Zheng, Jianlei Chi, Yangxu Jin, Ancheng He, Di Cui, Hengshan Zhang, and Ting Liu. 2021. Using K-core decomposition on class dependency networks to improve bug prediction model’s practical performance. IEEE Trans. Softw. Eng. 47, 2 (2021), 348–366.
[57]
Václav Rajlich and Norman Wilde. 2002. The role of concepts in program comprehension. In Proceedings of the 10th International Workshop on Program Comprehension (IWPC’02). IEEE Computer Society, 271–278.
[58]
Tobias Roehm, Rebecca Tiarks, Rainer Koschke, and Walid Maalej. 2012. How do professional developers comprehend software? In Proceedings of the 34th International Conference on Software Engineering (ICSE’12), Martin Glinz, Gail C. Murphy, and Mauro Pezzè (Eds.). IEEE Computer Society, 255–265.
[59]
Simone Scalabrino, Gabriele Bavota, Christopher Vendome, Mario Linares-Vásquez, Denys Poshyvanyk, and Rocco Oliveto. 2021. Automatically assessing code understandability. IEEE Trans. Softw. Eng. 47, 3 (2021), 595–613.
[60]
Simone Scalabrino, Gabriele Bavota, Christopher Vendome, Mario Linares Vásquez, Denys Poshyvanyk, and Rocco Oliveto. 2017. Automatically assessing code understandability: How far are we? In Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE’17). IEEE Computer Society, 417–427.
[61]
Simone Scalabrino, Mario Linares Vásquez, Denys Poshyvanyk, and Rocco Oliveto. 2016. Improving code readability models with textual features. In Proceedings of the 24th IEEE International Conference on Program Comprehension (ICPC’16). IEEE Computer Society, 1–10.
[62]
Elliot Soloway and Kate Ehrlich. 1984. Empirical studies of programming knowledge. IEEE Trans. Softw. Eng. 10, 5 (1984), 595–609.
[63]
Ioana Sora. 2015. Finding the right needles in hay—Helping program comprehension of large software systems. In Proceedings of the 10th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE’15), Joaquim Filipe and Leszek A. Maciaszek (Eds.). SciTePress, 129–140.
[64]
Ioana Sora and Ciprian-Bogdan Chirila. 2019. Finding key classes in object-oriented software systems by techniques based on static analysis. Inf. Softw. Technol. 116 (2019).
[65]
Daniela Steidl, Benjamin Hummel, and Elmar Jürgens. 2012. Using network analysis for recommendation of central software classes. In Proceedings of the 19th Working Conference on Reverse Engineering (WCRE’12). IEEE Computer Society, 93–102.
[66]
Ewan D. Tempero, Craig Anslow, Jens Dietrich, Ted Han, Jing Li, Markus Lumpe, Hayden Melton, and James Noble. 2010. The Qualitas Corpus: A curated collection of Java code for empirical studies. In Proceedings of the 17th Asia Pacific Software Engineering Conference (APSEC’10), Jun Han and Tran Dan Thu (Eds.). IEEE Computer Society, 336–345.
[67]
Ferdian Thung, David Lo, Mohd Hafeez Osman, and Michel R. V. Chaudron. 2014. Condensing class diagrams by analyzing design and network metrics using optimistic classification. In Proceedings of the 22nd International Conference on Program Comprehension (ICPC’14). ACM, 110–121.
[68]
Peter D. Turney. 2000. Types of cost in inductive concept learning. In Proceedings of the Workshop on Cost-sensitive Learning at the 17th International Conference on Machine Learning (ICML’00). IEEE Computer Society, 15–21.
[69]
Kristen R. Walcott, Mary Lou Soffa, Gregory M. Kapfhammer, and Robert S. Roos. 2006. TimeAware test suite prioritization. In Proceedings of the International Symposium on Software Testing and Analysis. 1–12.
[70]
M. S. Wang, H. M. Lu, Y. M. Zhou, and B. W. Xu. 2011. Identifying key classes using h-index and its variants. J. Front. Comput. Sci. Technol. 5, 10 (2011), 891–903.
[71]
Shuo Wang and Xin Yao. 2013. Using class imbalance learning for software defect prediction. IEEE Trans. Reliab. 62, 2 (2013), 434–443.
[72]
Ian H. Witten, Eibe Frank, and Mark A. Hall. 2011. Data Mining: Practical Machine Learning Tools and Techniques, 3rd Edition. Morgan Kaufmann, Elsevier.
[73]
Marvin Wyrich, Andreas Preikschat, Daniel Graziotin, and Stefan Wagner. 2021. The mind is a powerful place: How showing code comprehensibility metrics influences code understanding. In Proceedings of the 43rd IEEE/ACM International Conference on Software Engineering (ICSE’21). IEEE, 512–523.
[74]
Xinli Yang, David Lo, Xin Xia, and Jianling Sun. 2016. Condensing class diagrams with minimal manual labeling cost. In Proceedings of the 40th IEEE Annual Computer Software and Applications Conference (COMPSAC’16). IEEE Computer Society, 22–31.
[75]
Yibiao Yang, Yuming Zhou, Hongmin Lu, Lin Chen, Zhenyu Chen, Baowen Xu, Hareton Leung, and Zhenyu Zhang. 2015. Are slice-based cohesion metrics actually useful in effort-aware post-release fault-proneness prediction? An empirical study. IEEE Trans. Softw. Eng. 41, 4 (2015), 331–357.
[76]
Andy Zaidman, Toon Calders, Serge Demeyer, and Jan Paredaens. 2005. Applying webmining techniques to execution traces to support the program comprehension process. In Proceedings of the 9th European Conference on Software Maintenance and Reengineering (CSMR’05). IEEE Computer Society, 134–142.
[77]
Andy Zaidman and Serge Demeyer. 2008. Automatic identification of key classes in a software system using webmining techniques. J. Softw. Mainten. Res. Pract. 20, 6 (2008), 387–417.
[78]
Lu Zhang, Shan-Shan Hou, Chao Guo, Tao Xie, and Hong Mei. 2009. Time-aware test-case prioritization using integer linear programming. In Proceedings of the 18th International Symposium on Software Testing and Analysis. 213–224.

Cited By

View all

Index Terms

  1. EASE: An Effort-aware Extension of Unsupervised Key Class Identification Approaches

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Software Engineering and Methodology
    ACM Transactions on Software Engineering and Methodology  Volume 33, Issue 4
    May 2024
    940 pages
    EISSN:1557-7392
    DOI:10.1145/3613665
    • Editor:
    • Mauro Pezzè
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 21 April 2024
    Online AM: 02 December 2023
    Accepted: 07 November 2023
    Revised: 27 September 2023
    Received: 13 May 2023
    Published in TOSEM Volume 33, Issue 4

    Check for updates

    Author Tags

    1. Key classes
    2. network metrics
    3. complex networks
    4. static analysis
    5. program comprehension

    Qualifiers

    • Research-article

    Funding Sources

    • National Natural Science Foundation of China
    • Natural Science Foundation of Zhejiang Province
    • Zhejiang Gongshang University “Digital+” Disciplinary Construction Management Project

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 314
      Total Downloads
    • Downloads (Last 12 months)314
    • Downloads (Last 6 weeks)22
    Reflects downloads up to 03 Oct 2024

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media