research-article

On the impact of UML analysis models on source-code comprehensibility and modifiability

Authors:

Giuseppe Scanniello,

Carmine Gravino,

Marcela Genero,

Jose' A. Cruz-Lemus, and

Genoveffa TortoraAuthors Info & Claims

ACM Transactions on Software Engineering and Methodology (TOSEM), Volume 23, Issue 2

Article No.: 13, Pages 1 - 26

https://doi.org/10.1145/2491912

Published: 04 April 2014 Publication History

Abstract

We carried out a family of experiments to investigate whether the use of UML models produced in the requirements analysis process helps in the comprehensibility and modifiability of source code. The family consists of a controlled experiment and 3 external replications carried out with students and professionals from Italy and Spain. 86 participants with different abilities and levels of experience with UML took part. The results of the experiments were integrated through the use of meta-analysis. The results of both the individual experiments and meta-analysis indicate that UML models produced in the requirements analysis process influence neither the comprehensibility of source code nor its modifiability.

References

[1]

Marwen Abbes, Foutse Khomh, Yann-Gael Gueheneuc, and Giuliano Antoniol. 2011. An empirical study of the impact of two antipatterns, blob and spaghetti code, on program comprehension. In Proceedings of the European Conference on Software Maintenance and Reengineering. IEEE Computer Society, Los Alamitos, CA, 181--190.

Digital Library

[2]

Silvia Mara Abrahão, Carmine Gravino, Emilio Insfran Pelozo, Giuseppe Scanniello, and Genoveffa Tortora. 2012. Assessing the effectiveness of sequence diagrams in the comprehension of functional requirements: results from a family of five experiments. IEEE Trans. Softw. Eng.

Digital Library

[3]

Ritu Agarwal and Atish P. Sinha. 2003. Object-oriented modeling with UML: A study of developers' perceptions. Commun. ACM 46, 9, 248--256.

Digital Library

[4]

Bente Anda, Kai Hansen, Ingolf Gullesen, and Hanne Kristin Thorsen. 2006. Experiences from introducing UML-based development in a large safety-critical project. Empirical Softw. Eng. 11, 4, 555--581.

Digital Library

[5]

Erik Arisholm, Lionel C. Briand, Siw Elisabeth Hove, and Yvan Labiche. 2006. The impact of UML documentation on software maintenance: An experimental evaluation. IEEE Trans. Softw. Eng. 32, 365--381.

Digital Library

[6]

Hazeline U. Asuncion, Frédéric François, and Richard N. Taylor. 2007. An end-to-end industrial software traceability tool. In Proceedings of the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. 115--124.

Digital Library

[7]

Ricardo A. Baeza-Yates and Berthier Ribeiro-Neto. 1999. Modern Information Retrieval. Addison-Wesley Longman Publishing Co., Inc., Boston, MA.

Digital Library

[8]

R. Baker. 1995. Modern permutation test software. In Randomization Tests, 3rd Ed., E. Edgington, Ed. Marcel Dekker, Inc., New York, NY.

[9]

Victor Basili, Forrest Shull, and Filippo Lanubile. 1999. Building knowledge through families of experiments. IEEE Trans. Softw. Eng. 25, 4, 456--473.

Digital Library

[10]

Victor R. Basili, Scott Green, Oliver Laitenberger, Filippo Lanubile, Forrest Shull, Lars Sivert SØrumgård, and Marvin V. Zelkowitz. 1996. The empirical investigation of perspective-based reading. Empirical Softw. Eng. 1, 2, 133--164.

[11]

Victor R. Basili and H. Dieter Rombach. 1988. The TAME project: Towards improvement-oriented software environments. IEEE Trans. Softw. Eng. 14, 6, 758--773.

Digital Library

[12]

Biostat. 2006. Comprehensive Meta-Analysis v. 2. In Biostat Manual, Englewood, NJ.

[13]

Grady Booch, James Rumbaugh, and Ivar Jacobson. 2005. Unified Modeling Language User Guide, 2nd Ed. Addison-Wesley Professional.

Digital Library

[14]

B. Bruegge and A. H. Dutoit. 2003. Object-Oriented Software Engineering: Using UML, Patterns and Java, 2nd Ed. Prentice-Hall.

Digital Library

[15]

David Budgen, Andy J. Burn, O. Pearl Brereton, Barbara A. Kitchenham, and Rialette Pretorius. 2011. Empirical evidence about the UML: A systematic literature review. Softw. Pract. Exp. 41, 4, 363--392.

Digital Library

[16]

Jeffrey Carver, Letizia Jaccheri, Sandro Morasca, and Forrest Shull. 2003. Issues in using students in empirical studies in software engineering education. In Proceedings of the International Symposium on Software Metrics. IEEE Computer Society, 239.

Digital Library

[17]

David Cohen, Mikael Lindvall, and Patricia Costa. 2004. An introduction to agile methods. Adv. Comput. 62, 1--66.

[18]

M. Colosimo, A. De Lucia, G. Scanniello, and G. Tortora. 2009. Evaluating legacy system migration technologies through empirical studies. Inf. Softw. Technol. 51, 12, 433--447.

Digital Library

[19]

W. J. Conover. 1998. Practical Nonparametric Statistics. 3rd Ed. Wiley.

[20]

Sergio Cozzetti B. de Souza, Nicolas Anquetil, and Káthia M. de Oliveira. 2005. A study of the documentation essential to software maintenance. In Proceedings of the International Conference on Design of Communication: Documenting & Designing for Pervasive Information. ACM, New York, NY, 68--75.

Digital Library

[21]

J. L. Devore and N. Farnum. 1999. Applied Statistics for Engineers and Scientists. Duxbury.

[22]

B. Dobing and J. Parsons. 2006. How UML is used. Commun. ACM 49, 5, 109--113.

Digital Library

[23]

Wojciech J. Dzidek, Erik Arisholm, and Lionel C. Briand. 2008. A realistic empirical evaluation of the costs and benefits of UML in software maintenance. IEEE Trans. Softw. Eng. 34, 407--432.

Digital Library

[24]

P. Ellis. 2010. The Essential Guide to Effect Sizes: Statistical Power, Meta-Analysis, and the Interpretation of Research Results. Cambridge University Press.

[25]

John Erickson and Keng Siau. 2007. Theoretical and practical complexity of modeling methods. Commun. ACM 50, 8, 46--51.

Digital Library

[26]

A. Fernández-Saez, M. Genero, J. Nelson, G. Poels, and M. Piattini. 2012. A systematic literature review on the quality of UML models. J. Data. Manage. 22, 3, 46--70.

Digital Library

[27]

Carmine Gravino, Michele Risi, Giuseppe Scanniello, and Genoveffa Tortora. 2011. Does the documentation of design pattern instances impact on source code comprehension&quest; Results from two controlled experiments. In Proceeding of the Working Conference on Reverse Engineering. IEEE Computer Society, Los Alamitos, CA, 67--76.

Digital Library

[28]

Carmine Gravino, Michele Risi, Giuseppe Scanniello, and Genoveffa Tortora. 2012. Do professional developers benefit from design pattern documentation&quest; A replication in the context of source code comprehension. In Proceedings of the International Conference on Model Driven Engineering Languages and Systems. Springer-Verlag, Berlin, 185--201.

Digital Library

[29]

Carmine Gravino, Genoveffa Tortora, and Giuseppe Scanniello. 2010. An empirical investigation on the relation between analysis models and source code comprehension. In Proceedings of the ACM Symposium on Applied Computing. ACM, 2365--2366.

Digital Library

[30]

Martin Grossman, Jay E. Aronson, and Richard V. McCarthy. 2005. Does UML make the grade&quest; Insights from the software development community. Inf. Softw. Technol. 47, 6, 383--397.

Digital Library

[31]

L. Hedges and I. Olkin. 1985. Statistical Methods for Meta-Analysis. Academia Press.

[32]

Martin Höst, Björn Regnell, and Claes Wohlin. 2000. Using students as subjects: Comparative study of students and professionals in lead-time impact assessment. Empirical Softw. Eng. 5, 3, 201--214.

Digital Library

[33]

Andreas Jedlitschka, Marcus Ciolkowski, and Dietmar Pfahl. 2008. Reporting experiments in software engineering. In Guide to Advanced Empirical Software Engineering, Forrest Shull, Janice Singer, and Dag I. K. Sjøberg, Eds., Springer, 201--228.

[34]

N. Juristo and A. Moreno. 2001. Basics of Software Engineering Experimentation. Kluwer Academic Publishers, Englewood Cliffs, NJ.

Digital Library

[35]

Vigdis By Kampenes, Tore Dybå, Jo Erskine Hannay, and Dag I. K. Sjøberg. 2007. A systematic review of effect size in software engineering experiments. Inf. Softw. Technol. 49, 11--12, 1073--1086.

Digital Library

[36]

Barbara Kitchenham, Hiyam Al-Khilidar, Muhammed Babar, Mike Berry, Karl Cox, Jacky Keung, Felicia Kurniawati, Mark Staples, He Zhang, and Liming Zhu. 2008. Evaluating guidelines for reporting empirical software engineering studies. Empirical Softw. Eng. 13, 97--121.

Digital Library

[37]

B. Kitchenham, S. Pfleeger, L. Pickard, P. Jones, D. Hoaglin, K. El Emam, and J. Rosenberg. 2002. Preliminary guidelines for empirical research in software engineering. IEEE Trans. Softw. Eng. 28, 8, 721--734.

Digital Library

[38]

Soren Lauesen. 2002. Software Requirements: Styles and Techniques. Addison-Wesley.

Digital Library

[39]

H. Levene. 1960. Robust tests for equality of variances. In Contributions to Probability and Statistics, I. Olkin Ed., Stanford University Press., Palo Alto, CA.

[40]

M. Lindvall and K. Sandahl. 1996. Practical implications of traceability. Softw. Pract. Exp. 26, 10, 1161--1180.

Digital Library

[41]

Mitch Lubars, Colin Potts, and Charlie Richter. 1993. A review of the state of the practice in requirements modeling. In Proceedings of the International Symposium on Requirements Engineering. IEEE Computer Society Press, 2--14.

[42]

J. McDermid. 1991. Software Engineer's Reference Book. Butterworth-Heinemann, Ltd., Oxford, U.K.

Digital Library

[43]

Manoel G. Mendonça, José C. Maldonado, Maria C. F. de Oliveira, Jeffrey Carver, Sandra C. P. F. Fabbri, Forrest Shull, Guilherme H. Travassos, Erika Nina Höhn, and Victor R. Basili. 2008. A framework for software engineering experimental replications. In Proceedings of the International Conference on Engineering of Complex Computer Systems. IEEE Computer Society, Los Alamitos, CA, 203--212.

Digital Library

[44]

OMG. 2010. Unified Modeling Language (TM) URL. Tech. Rep. Object Management Group. http://www.uml.org.

[45]

A. N. Oppenheim. 1992. Questionnaire Design, Interviewing and Attitude Measurement. Pinter, London.

[46]

Lutz Prechelt, Barbara Unger-Lamprecht, Michael Philippsen, and Walter F. Tichy. 2002. Two controlled experiments assessing the usefulness of design pattern documentation in program maintenance. IEEE Trans. Softw. Eng. 28, 6, 595--606.

Digital Library

[47]

Filippo Ricca, Massimiliano Di Penta, Marco Torchiano, Paolo Tonella, and Mariano Ceccato. 2010. How Developers' experience and ability influence web application comprehension tasks supported by UML stereotypes: a series of four experiments. IEEE Trans. Softw. Eng. 36, 1, 96--118.

Digital Library

[48]

Giuseppe Scanniello, Carmine Gravino, and Genny Tortora. 2010. Investigating the role of UML in the software modeling and maintenance—A preliminary industrial survey. In Proceedings of the International Conference on Enterprise Information Systems. SciTePress, 141--148.

[49]

Giuseppe Scanniello, Carmine Gravino, and Genoveffa Tortora. 2012. Does the combined use of class and sequence diagrams improve the source code comprehension&quest;: Results from a controlled experiment. In Proceedings of the International Workshop on Experiences and Empirical Studies in Software Modelling. ACM, New York, NY, 25--30.

Digital Library

[50]

G. Scanniello, F. Ricca, and M. Torchiano. 2011. On the effectiveness of the UML object diagrams: A replicated experiment. In Proceedings of the International Conference on Evaluation and Assessment in Software Engineering. IET Digital Library, 76--85.

[51]

S. Shapiro and M. Wilk. 1965. An analysis of variance test for normality. Biometrika 52, 3--4, 591--611.

[52]

Forrest Shull, Jeffrey C. Carver, Sira Vegas, and Natalia Juristo Juzgado. 2008. The role of replications in empirical software engineering. Empirical Softw. Eng. 13, 2, 211--218.

Digital Library

[53]

Forrest Shull, Manoel G. Mendoncça, Victor Basili, Jeffrey Carver, José C. Maldonado, Sandra Fabbri, Guilherme Horta Travassos, and Maria Cristina Ferreira. 2004. Knowledge-sharing issues in experimental software engineering. Empirical Softw. Eng. 9, 1--2, 111--137.

Digital Library

[54]

D. I. K. Sjøberg, J. E. Hannay, O. Hansen, V. B. Kampenes, A. Karahasanovic, N. Liborg, and A. C. Rekdal. 2005. A survey of controlled experiments in software engineering. IEEE Trans. Softw. Eng. 31, 9, 733--753.

Digital Library

[55]

Scott Tilley and Shihong Huang. 2003. A qualitative assessment of the efficacy of UML diagrams as a form of graphical documentation in aiding program understanding. In Proceedings of the International Conference on Documentation. ACM, New York, NY, 184--191.

Digital Library

[56]

Eirik Tryggeseth. 1997. Report from an experiment: Impact of documentation on maintenance. Empirical Softw. Eng. 2, 2, 201--207.

Digital Library

[57]

S. Vegas, N. Juristo, A. Moreno, M. Solari, and P. Letelier. 2006. Analysis of the influence of communication between researchers on experiment replication. In Proceedings of the International Symposium on Empirical Software Engineering. ACM, New York, NY, 28--37.

Digital Library

[58]

Jan Verelst. 2004. The influence of the level of abstraction on the evolvability of conceptual models of information systems. In Proceedings of the International Symposium on Empirical Software Engineering. IEEE Computer Society, 17--26.

Digital Library

[59]

Klaus Weidenhaupt, Klaus Pohl, Matthias Jarke, and Peter Haumer. 1998. Scenarios in system development: Current practice. IEEE Softw. 15, 34--45.

Digital Library

[60]

C. Wohlin, P. Runeson, M. Höst, M. Ohlsson, B. Regnell, and A. Wesslén. 2000. Experimentation in Software Engineering-An Introduction. Kluwer.

Digital Library

Cited By

Romeo JRaglianti MNagy CLanza MBaysal OLinares-Vasquez MMoran KSteinmacher I(2024)Capturing and Understanding the Drift Between Design, Implementation, and DocumentationProceedings of the 32nd IEEE/ACM International Conference on Program Comprehension10.1145/3643916.3644399(382-386)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643916.3644399
Cai YKazman R(2024)Software design analysis and technical debt management based on design rule theoryInformation and Software Technology10.1016/j.infsof.2023.107322164:COnline publication date: 10-Jan-2024
https://dl.acm.org/doi/10.1016/j.infsof.2023.107322
Jia QCai YÇakmak OLiu AMuccini H(2023)A Model-Based, Quality Attribute-Guided Architecture Re-Design Process at GoogleProceedings of the 45th International Conference on Software Engineering: Software Engineering in Practice10.1109/ICSE-SEIP58684.2023.00011(61-73)Online publication date: 17-May-2023
https://dl.acm.org/doi/10.1109/ICSE-SEIP58684.2023.00011
Show More Cited By

Index Terms

On the impact of UML analysis models on source-code comprehensibility and modifiability
1. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. Software management
        Software maintenance
2. Software and its engineering
  1. Software creation and management
    1. Software post-development issues
  2. Software notations and tools
    1. General programming languages
      1. Language types

Recommendations

Assessing the Effect of Screen Mockups on the Comprehension of Functional Requirements

Over the last few years, the software engineering community has proposed a number of modeling methods to represent functional requirements. Among them, use cases are recognized as an easy to use and intuitive way to capture and define such requirements. ...
Read More
Empirical Validation of Measures for UML Class Diagrams: A Meta-Analysis Study
Models in Software Engineering

The main goal of this paper is to show the findings obtained through a meta-analysis study carried out with the data obtained from a family of five controlled experiments performed in academic environments. This family of experiments was carried out to ...
Read More
Does the level of detail of UML models affect the maintainability of source code?
MODELS'11: Proceedings of the 2011th international conference on Models in Software Engineering

This paper presents an experiment carried out as a pilot study to obtain a first insight into the influence of the quality of UML models on the maintenance of the corresponding source code. The quality of the UML models is assessed by studying the ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Software Engineering and Methodology

ACM Transactions on Software Engineering and Methodology Volume 23, Issue 2

March 2014

319 pages

ISSN:1049-331X

EISSN:1557-7392

DOI:10.1145/2600788

Editor:
David S. Rosenblum

Issue’s Table of Contents

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 April 2014

Accepted: 01 June 2013

Revised: 01 February 2013

Received: 01 July 2012

Published in TOSEM Volume 23, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Ministerio de Economía y Competitividad
GEODAS-BC project (Ministerio de Economa y Competitividad and Fondo Europeo de Desarrollo Regional FEDER, TIN2012-37493-C03-01)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
743
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)5

Other Metrics

View Author Metrics

Citations

Cited By

Romeo JRaglianti MNagy CLanza MBaysal OLinares-Vasquez MMoran KSteinmacher I(2024)Capturing and Understanding the Drift Between Design, Implementation, and DocumentationProceedings of the 32nd IEEE/ACM International Conference on Program Comprehension10.1145/3643916.3644399(382-386)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643916.3644399
Cai YKazman R(2024)Software design analysis and technical debt management based on design rule theoryInformation and Software Technology10.1016/j.infsof.2023.107322164:COnline publication date: 10-Jan-2024
https://dl.acm.org/doi/10.1016/j.infsof.2023.107322
Jia QCai YÇakmak OLiu AMuccini H(2023)A Model-Based, Quality Attribute-Guided Architecture Re-Design Process at GoogleProceedings of the 45th International Conference on Software Engineering: Software Engineering in Practice10.1109/ICSE-SEIP58684.2023.00011(61-73)Online publication date: 17-May-2023
https://dl.acm.org/doi/10.1109/ICSE-SEIP58684.2023.00011
Kitchenham BMadeyski LScanniello GGravino C(2022)The Importance of the Correlation in Crossover ExperimentsIEEE Transactions on Software Engineering10.1109/TSE.2021.307048048:8(2802-2813)Online publication date: 1-Aug-2022
https://doi.org/10.1109/TSE.2021.3070480
Júnior EFarias KSilva B(2021)A Survey on the Use of UML in the Brazilian IndustryProceedings of the XXXV Brazilian Symposium on Software Engineering10.1145/3474624.3474632(275-284)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1145/3474624.3474632
Panach JDieste ÓMarín BEspaña SVegas SPastor ÓJuristo N(2021)Evaluating Model-Driven Development Claims with Respect to Quality: A Family of ExperimentsIEEE Transactions on Software Engineering10.1109/TSE.2018.288470647:1(130-145)Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1109/TSE.2018.2884706
Santos AGomez OJuristo N(2020)Analyzing Families of Experiments in SE: A Systematic Mapping StudyIEEE Transactions on Software Engineering10.1109/TSE.2018.286463346:5(566-583)Online publication date: 1-May-2020
https://doi.org/10.1109/TSE.2018.2864633
Romano SVendome CScanniello GPoshyvanyk D(2020)A Multi-Study Investigation into Dead CodeIEEE Transactions on Software Engineering10.1109/TSE.2018.284278146:1(71-99)Online publication date: 1-Jan-2020
https://doi.org/10.1109/TSE.2018.2842781
Cruz MBernardez BDuran AGalindo JRuiz-Cortes A(2020)Replication of Studies in Empirical Software Engineering: A Systematic Mapping Study, From 2013 to 2018IEEE Access10.1109/ACCESS.2019.29521918(26773-26791)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2019.2952191
Kitchenham BMadeyski LBrereton P(2020)Meta-analysis for families of experiments in software engineering: a systematic review and reproducibility and validity assessmentEmpirical Software Engineering10.1007/s10664-019-09747-025:1(353-401)Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.1007/s10664-019-09747-0
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents