research-article

Validating software measures using action research a method and industrial experiences

Authors:

Miroslaw Staron,

Jörgen HanssonAuthors Info & Claims

EASE '16: Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering

Article No.: 23, Pages 1 - 10

https://doi.org/10.1145/2915970.2916001

Published: 01 June 2016 Publication History

Abstract

Validating software measures for using them in practice is a challenging task. Usually more than one complementary validation methods are applied for rigorously validating software measures: Theoretical methods help with defining the measures with expected properties and empirical methods help with evaluating the predictive power of measures. Despite the variety of these methods there still remain cases when the validation of measures is difficult. Particularly when the response variables of interest are not accurately measurable and the practical context cannot be reduced to an experimental setup the abovementioned methods are not effective. In this paper we present a complementary empirical method for validating measures. The method relies on action research principles and is meant to be used in combination with theoretical validation methods. The industrial experiences documented in this paper show that in many practical cases the method is effective.

References

[1]

ISO/IEC 15939 (2001). "Information technology --- Software engineering --- Software measurement process."

[2]

Antinyan, V., et al. (2014). Identifying risky areas of software code in Agile/Lean software development: An industrial experience report. Software Maintenance, Reengineering and Reverse Engineering (CSMR-WCRE), 2014 Software Evolution Week-IEEE Conference on, IEEE.

[3]

Basili, V. R., et al. (1996). "A validation of object-oriented design metrics as quality indicators." Software Engineering, IEEE Transactions on 22(10): 751--761.

Digital Library

[4]

Baskerville, R. L. and A. T. Wood-Harper (1996). "A critical perspective on action research as a method for information systems research." Journal of Information Technology 11(3): 235--246.

[5]

Boehm, B., et al. (2005). "COCOMO suite methodology and evolution." CrossTalk 18(4): 20--25.

[6]

Boström, P., et al. (2010). An approach to contract-based verification of Simulink models, Tech. Rep. 985, TUCS.

[7]

Briand, L., et al. (1995). "Theoretical and empirical validation of software product measures." International Software Engineering Research Network, Technical Report ISERN-95-03.

[8]

Briand, L., et al. (1996). "On the application of measurement theory in software engineering." Empirical Software Engineering 1(1): 61--88.

[9]

Briand, L. C., et al. (1996). "Property-based software engineering measurement." Software Engineering, IEEE Transactions on 22(1): 68--86.

Digital Library

[10]

Catal, C. (2011). "Software fault prediction: A literature review and current trends." Expert systems with applications 38(4): 4626--4636.

Digital Library

[11]

Costello, R. J. and D.-B. Liu (1995). "Metrics for requirements engineering." Journal of Systems and Software 29(1): 39--63.

Digital Library

[12]

El-Emam, K. (2000). "A methodology for validating software product metrics."

[13]

Elbaum, S. G. and J. C. Munson (1998). Getting a handle on the fault injection process: validation of measurement tools. Software Metrics Symposium, 1998. Metrics 1998. Proceedings. Fifth International, IEEE.

Digital Library

[14]

Fabbrini, F., et al. (2001). An automatic quality evaluation for natural language requirements. Proceedings of the Seventh International Workshop on Requirements Engineering: Foundation for Software Quality REFSQ.

[15]

Gleich, B., et al. (2010). Ambiguity detection: Towards a tool explaining ambiguity sources. Requirements Engineering: Foundation for Software Quality, Springer: 218--232.

[16]

Graylin, J., et al. (2009). "Cyclomatic complexity and lines of code: empirical evidence of a stable linear relationship." Journal of Software Engineering and Applications 2(03): 137.

[17]

Gyimothy, T., et al. (2005). "Empirical validation of object-oriented metrics on open source software for fault prediction." Software Engineering, IEEE Transactions on 31(10): 897--910.

Digital Library

[18]

Hall, T., et al. (2012). "A systematic literature review on fault prediction performance in software engineering." Software {1} Engineering, IEEE Transactions on 38(6): 1276--1304.

Digital Library

[19]

Halstead, M. H. (1977). Elements of Software Science (Operating and programming systems series), Elsevier Science Inc.

Digital Library

[20]

Hyatt, L. E. and L. H. Rosenberg (1997). "Software metrics program for risk assessment." Acta astronautica 40(2): 223--233.

[21]

ISO, I. (2007). "IEC 25020 Software and system engineering--Software product Quality Requirements and Evaluation (SQuaRE)--Measurement reference model and guide." International Organization for Standarization.

[22]

Iversen, J. H., et al. (2004). "Managing risk in software process improvement: an action research approach." Mis Quarterly: 395--433.

Digital Library

[23]

Kaner, C. (2004). Software engineering metrics: What do they measure and how do we know? In METRICS 2004. IEEE CS, Citeseer.

[24]

Kitchenham, B. (2010). "What's up with software metrics?-- A preliminary mapping study." Journal of systems and software 83 (1): 37--51.

Digital Library

[25]

Kitchenham, B., et al. (1995). "Towards a framework for software measurement validation." Software Engineering, IEEE Transactions on 21(12): 929--944.

Digital Library

[26]

Li, W. and S. Henry (1993). "Object-oriented metrics that predict maintainability." Journal of systems and software 23(2): 111--122.

Digital Library

[27]

Mair, C. and M. Shepperd (2011). Human judgement and software metrics: vision for the future. Proceedings of the 2nd international workshop on emerging trends in software metrics, ACM.

Digital Library

[28]

McCabe, T. J. (1976). "A complexity measure." Software Engineering, IEEE Transactions on(4): 308--320.

Digital Library

[29]

McGarry, J. (2002). Practical software measurement: objective information for decision makers, Addison-Wesley Professional.

Digital Library

[30]

Meneely, A., et al. (2012). "Validating software metrics: A spectrum of philosophies." ACM Transactions on Software Engineering and Methodology (TOSEM) 21(4): 24.

Digital Library

[31]

Monden, A., et al. (2002). Software quality analysis by code clones in industrial legacy software. Software Metrics, 2002. Proceedings. Eighth IEEE Symposium on, IEEE.

Digital Library

[32]

Moody, D. L. (1998). Metrics for evaluating the quality of entity relationship models. Conceptual Modeling--ER'98, Springer: 211--225.

Digital Library

[33]

Moser, R., et al. (2008). A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction. Software Engineering, 2008. ICSE'08. ACM/IEEE 30th International Conference on, IEEE.

Digital Library

[34]

Olague, H. M., et al. (2007). "Empirical validation of three software metrics suites to predict fault-proneness of object-oriented classes developed using highly iterative or agile software development processes." Software Engineering, IEEE Transactions on 33(6): 402--419.

Digital Library

[35]

Olszewska, M. (2011). Simulink-specific design quality metrics, Technical report, Turku, Finland.

[36]

Prabhu, J. (2010). "Complexity Analysis of Simulink Models to improve the Quality of Outsourcing in an Automotive Company." Manuscript, August.

[37]

Reicherdt, R. and S. Glesner (2012). Slicing MATLAB simulink models. Software Engineering (ICSE), 2012 34th International Conference on, IEEE.

Digital Library

[38]

Schneidewind, N. F. (1992). "Methodology for validating software metrics." Software Engineering, IEEE Transactions on 18(5): 410--422.

Digital Library

[39]

Sellami, A. and A. Abran (2003). The contribution of metrology concepts to understanding and clarifying a proposed framework for software measurement validation. Proceedings of the 13th International Workshop on Software Measurement (IWSM), Montreal, Canada.

[40]

Shepperd, M. and D. C. Ince (1994). "A critique of three metrics." Journal of systems and software 26(3): 197--210.

[41]

Sjøberg, D. I., et al. (2005). "A survey of controlled experiments in software engineering." Software Engineering, IEEE Transactions on 31(9): 733--753.

Digital Library

[42]

Van Solingen, R. and E. Berghout (1999). The Goal / Question/Metric Method: a practical guide for quality improvement of software development, McGraw-Hill.

[43]

Weyuker, E. J. (1988). "Evaluating software complexity measures." Software Engineering, IEEE Transactions on 14(9): 1357--1365.

Digital Library

[44]

Zhang, H., et al. (2007). Predicting defective software components from code complexity measures. Dependable Computing, 2007. PRDC 2007. 13th Pacific Rim International Symposium on, IEEE.

Digital Library

[45]

Zimmermann, T., et al. (2009). Cross-project defect prediction: a large scale experiment on data vs. domain vs. process. Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering, ACM.

Digital Library

Cited By

Salomón SDuque RBringas Sde Oliveira K(2025)Quality‐in‐Use in PracticeJournal of Software: Evolution and Process10.1002/smr.276437:1Online publication date: 22-Jan-2025
https://dl.acm.org/doi/10.1002/smr.2764
Staron M(2024)Teaching Action ResearchHandbook on Teaching Empirical Software Engineering10.1007/978-3-031-71769-7_14(387-411)Online publication date: 25-Dec-2024
https://doi.org/10.1007/978-3-031-71769-7_14
Silva MFreire APerkusich MAlbuquerque DGorgonio KAlmeida HPerkusich AGuimaraes E(2021)A Comparative Analysis of Agile Teamwork Quality Models2021 International Conference on Software, Telecommunications and Computer Networks (SoftCOM)10.23919/SoftCOM52868.2021.9559062(1-6)Online publication date: 23-Sep-2021
https://doi.org/10.23919/SoftCOM52868.2021.9559062
Show More Cited By

Recommendations

Action research as a model for industry-academia collaboration in the software engineering context
WISE '14: Proceedings of the 2014 international workshop on Long-term industrial collaboration on software engineering

Background: Action research is a well-established research methodology. It is following a post-positivist research philosophy grounded in critical thinking. The methodology is driven by practical problems, emphasis participatory research, and develops ...
Validating software measures

There are two distinct notions of validation for software measures. On the one hand there is the informal notion that a measure is only valid if it is useful and practical. Although this view of validation is important, it is almost impossible to test. ...
Validation of software in measuring instruments

Most measuring instruments of today are based on embedded systems or personal computers. The software embedded into the instruments controls functions that must be possible to verify and validate. Validation of software is certainly performed by the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

EASE '16: Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering

June 2016

310 pages

ISBN:9781450336918

DOI:10.1145/2915970

Conference Chair:
Sarah Beecham,
Program Chairs:
Barbara Kitchenham,
Stephen G. MacDonell

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 June 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

EASE '16

EASE '16: 20th International Conference on Evaluation and Assessment in Software Engineering

June 1 - 3, 2016

Limerick, Ireland

Acceptance Rates

Overall Acceptance Rate 71 of 232 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
253
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Salomón SDuque RBringas Sde Oliveira K(2025)Quality‐in‐Use in PracticeJournal of Software: Evolution and Process10.1002/smr.276437:1Online publication date: 22-Jan-2025
https://dl.acm.org/doi/10.1002/smr.2764
Staron M(2024)Teaching Action ResearchHandbook on Teaching Empirical Software Engineering10.1007/978-3-031-71769-7_14(387-411)Online publication date: 25-Dec-2024
https://doi.org/10.1007/978-3-031-71769-7_14
Silva MFreire APerkusich MAlbuquerque DGorgonio KAlmeida HPerkusich AGuimaraes E(2021)A Comparative Analysis of Agile Teamwork Quality Models2021 International Conference on Software, Telecommunications and Computer Networks (SoftCOM)10.23919/SoftCOM52868.2021.9559062(1-6)Online publication date: 23-Sep-2021
https://doi.org/10.23919/SoftCOM52868.2021.9559062
Staron MMeding WSöder OOchodek M(2021)Improving Quality of Code Review Datasets – Token-Based Feature Extraction MethodSoftware Quality: Future Perspectives on Software Engineering Quality10.1007/978-3-030-65854-0_7(81-93)Online publication date: 6-Jan-2021
https://doi.org/10.1007/978-3-030-65854-0_7
Saraiva RMedeiros APerkusich MValadares DGorgonio KPerkusich AAlmeida H(2020)A Bayesian Networks-Based Method to Analyze the Validity of the Data of Software Measurement ProgramsIEEE Access10.1109/ACCESS.2020.30352178(198801-198821)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3035217
Staron MStaron M(2019)Action TakingAction Research in Software Engineering10.1007/978-3-030-32610-4_5(73-92)Online publication date: 25-Nov-2019
https://doi.org/10.1007/978-3-030-32610-4_5
Staron MStaron M(2019)Action Research as Research Methodology in Software EngineeringAction Research in Software Engineering10.1007/978-3-030-32610-4_2(15-36)Online publication date: 25-Nov-2019
https://doi.org/10.1007/978-3-030-32610-4_2
Staron M(2019)Action Research in Software Engineering: Metrics’ Research Perspective (Invited Talk)SOFSEM 2019: Theory and Practice of Computer Science10.1007/978-3-030-10801-4_4(39-49)Online publication date: 11-Jan-2019
https://doi.org/10.1007/978-3-030-10801-4_4
Wen SNilsson CStaron MSentilles SBoehm BTrubiani CFranch XKoziolek A(2018)Assessing the release readiness of engine control softwareProceedings of the 1st International Workshop on Software Qualities and Their Dependencies10.1145/3194095.3194099(5-12)Online publication date: 28-May-2018
https://dl.acm.org/doi/10.1145/3194095.3194099
Parra OEspana SPanach J(2017)Extending and validating gestUI using technical action research2017 11th International Conference on Research Challenges in Information Science (RCIS)10.1109/RCIS.2017.7956558(341-352)Online publication date: May-2017
https://doi.org/10.1109/RCIS.2017.7956558

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten