Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/1987875.1987880acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
research-article

Optimizing cost and quality by integrating inspection and test processes

Published: 21 May 2011 Publication History

Abstract

Inspections and testing are two of the most commonly performed software quality assurance processes today. Typically, these processes are applied in isolation, which, however, fails to exploit the benefits of systematically combining and integrating them. Expected benefits of such process integration are higher defect detection rates or reduced quality assurance effort. Moreover, when conducting testing without any prior information regarding the system's quality, it is often unclear which parts or which defect types should be prioritized. Existing approaches do not explicitly use information from inspections in a systematical way to focus testing processes. In this article, we present an integrated two-stage approach that routes inspection data to test processes in order to prioritize code classes and defect types. While an initial version of the approach focused on prioritizing code classes, this article focuses on the prioritization of defect types for testing. Results from a case study where the approach was applied on the code level show that those defect types could be prioritized before the testing that afterwards actually showed up most often during the test process. In addition, an overview of related work and an outlook on future research directions are given.

References

[1]
Aurum, A., Petersson, H., Wohlin, C., State-of-the-art: software inspections after 25 years, Software Testing, Verification and Reliability, 12 (3), 133--154, 2002.
[2]
Juristo, N., Moreno, A. M., Vegas, S., Reviewing 25 years of testing technique experiments, Empirical Software Engineering, 9 (1-2), 7--44, 2004.
[3]
Harrold, M. J., Testing: A Roadmap, International Conference on Software Engineering, The Future of Software Engineering, 61--72, 2000.
[4]
Health, Social, and Economic Research, The economic impacts of inadequate infrastructure for software testing, National Institute of Standards and Technology, 2002.
[5]
Basili, V. R., Briand, L. C., Melo, W. L., A validation of object-oriented design metrics as quality indicators, IEEE Transactions on Software Engineering, 751--761, 1996.
[6]
Illes-Seifert, T., Paech, B., Exploring the relationship of a file's history and its fault-proneness: An empirical method and its application to open source programs, Information and Software Technology, 52 (5), 539--558, 2009.
[7]
Orthogonal Defect Classification, IBM, available: http://www.research.ibm.com/softeng/ODC/ODC.HTM.
[8]
Elbaum, S., Rothermel, G., Kanduri, S., and Malishevsky, A. G., Selecting a cost-effective test case prioritization technique, Software Quality Control, 12 (3), 185--210, 2004.
[9]
Strooper, P., Wojcicki, M. A., Selecting V&V technology combinations: how to pick a winner? 12th IEEE International Conference on Engineering Complex Computer Systems, 87--96, 2007.
[10]
FindBugs, http://findbugs.sourceforge.net/.
[11]
Chebaro, O., Kosmatov, N., Giorgetti, A., Julliand., J., Combining static analysis and test generation for C program debugging, International Conference on Tests and Proofs, 94--100, 2010.
[12]
Endres, A., Rombach, D., A Handbook of Software and Systems Engineering, Addison Wesley, 2003.
[13]
Laitenberger, O., Studying the effects of code inspections and structural testing on software quality, 9th International Symposium on Software Reliability Eng., 237--246, 1998.
[14]
Elberzhager, F., Eschbach, R., Muench, J., Using inspection results for prioritizing inspection data, 21st International Symposium on Software Reliability Engineering, Supplemental Proceedings, 263--272, 2010, available: http://inspection.iese.fhg.de/?p=documents.
[15]
Ostrand, T. J., Weyuker, E. J., The distribution of faults in a large industrial software system, International Symposium on Software Testing and Analysis, 55--64, 2002.
[16]
Emam, K. E., Benlarbi, S., Goel, N., Mela, W., Lounis, H., Rai, S. N., The optimal class size for object oriented software, IEEE Trans. Softw. Eng., 28 (5), 494--509, 2002.
[17]
Anderson, C., Runeson, P., A replicated quantitative analysis of fault distributions in complex software systems, IEEE Trans. Softw. Eng., 33 (5), 273--286, 2007.
[18]
Nagappan, N., Ball, T., Zeller, A., Mining metrics to predict component failures, International Conference on Software Engineering, 452--461, 2006.
[19]
Fenton, N. E., Ohlsson, N., Quantitative analysis of faults and failures in a complex software System, IEEE Trans. Softw. Eng., 26 (8), 797--814, 2000.
[20]
Schröter, A., Zimmermann, T., Premraj, R., Zeller, A., If your bug database could talk, 5th International Symposium on Empirical Software Engineering, 18--20, 2006.
[21]
Illes-Seifert, T., Paech, B., Exploring the relationship of a file's history and its fault-proneness: An empirical method and its application to open source programs, Information and Software Technology, 52 (5), 539--558, 2009.
[22]
Nagappan, N., Ball, T., Murphy, B., Using historical in-process and product metrics for early estimation of software failures, 17th International Symposium on Software Reliability Engineering, 62--74, 2006.
[23]
Ostrand, T. J, Weyuker, E. J, Bell, R. M, We're finding most of the bugs, but what are we missing? 3rd International Conference on Software Testing, Verification and Validation, 313--322, 2010.
[24]
Basili, V. R., Selby, R. W., Comparing the effectiveness of software testing strategies, IEEE Transactions on Software Engineering, 13 (12), 1278--1296, 1987.
[25]
Runeson, P., Andrews, A., Detection or isolation of defects? An experimental comparison of unit testing and code inspection, 14th International Symposium on Software Reliability Engineering, 3--13, 2003.
[26]
Elbaum, S., Malishevsky, A. G., Rothermel, G., Test case prioritization: a family of empirical studies, IEEE Transactions on Software Eng., 28 (2), 159--182, 2002.
[27]
Karolak, D. W., Software Engineering Risk Management, IEEE Computer Society Press, Wiley, 1996.
[28]
Liu, S., Tamai, T., Nakajima, S., Integration of formal specification, review, and testing for software component quality assurance. ACM Symposium on Applied Computing, 415--421, 2009.
[29]
Runeson, P., Andersson, C., Thelin, T., Andrews, A., Berling, T., What do we know about defect detection methods? IEEE Software, 23 (3), 82--90, 2006.
[30]
Winkler, D., Biffl, S., Fader, K., Investigating the temporal behavior of defect detection in software inspection and inspection-based testing, 11th International Conference on Product Focused Software Development and Process Improvement, 17--31, 2010.
[31]
Harding, J. T., Using inspection data to forecast test defects, Software Technology Transition, 19--24, 1998.
[32]
Boehm, B., Basili, V. R., Software defect reduction top 10 list, IEEE Computer, 34 (1), 135--137, 2001.
[33]
Burnstein, I., Practical Software Testing, Springer, 2002.
[34]
Wiegers, K. E., Peer Reviews in Software, A.-Wesley, 2002.
[35]
Gilb, T., Graham, D., Software Inspections, A.-Wesley, 1993.
[36]
Barnard, J., Price, A., Managing code inspection information, IEEE Software, 11 (2), 59--69, 1994.
[37]
Mantyla, M. V., Lassenius, C., What types of defects are really discovered in code reviews? IEEE Transactions on Software Engineering, 35 (3), 430--448, 2009.
[38]
Kamsties, E., Lott, C. M., An empirical evaluation of three defect detection techniques, 5th European Software Engineering Conference, 362--383, 1995.
[39]
Chaar, J. K., Halling, M. J., Bhandari, I. S., Chillarege, R., In-process evaluation for software inspection and test, IEEE Transactions on Software Eng., 19 (11), 1055--1070, 1993.
[40]
Ohlsson, N., Helander, M., Wohlin, C., Quality improvement by identification of fault-prone modules using software design metrics, 6th International Conference on Software Engineering, 1--13, 1996.
[41]
Wohlin, C., Runeson, P., Host, M., Ohlsson, M. C., Regnell, B., Wesslen, A., Experimentation in software engineering an introduction, Kluwer, 2000.
[42]
Aggarwal, A., Jalote, P., Integrating static and dynamic analysis for detecting vulnerabilities, 30th Annual International Conference on Computer Software and Applications, 343--350, 2006.
[43]
Wood, M., Roper, M., Brooks, A., Miller, J., Comparing and combining software defect detection techniques - a replicated empirical study, 6th European Software Engineering Conference, 262--277, 1997.
[44]
Gack, G. A., An economic analysis of software defect removal methods, based on Managing the Black Hole: The Executive's Guide to Software Project Risk, Business Expert Publishing, 2010.
[45]
Freimut, B., Denger, C., Ketterer, M., An industrial case study of implementing and validating defect classification for process improvement and quality management 11th IEEE International Software Metrics Symp., 19--28, 2005.
[46]
Arisholm, E., Briand, L. C., Johannesson, E. B., A systematic and comprehensive investigation of methods to build and evaluate fault prediction models, Journal of Systems and Software, 83 (1), 2--17, 2009.
[47]
Schultz, C., Orthogonal defect classification-based test planning and development, International Conference on Applications of Software Measurement, 15--19, 1999.
[48]
Bridge, N., Miller, C., Orthogonal defect classification using defect data to improve software development, Int. Conference on Software Quality, 197--213, 1997.
[49]
McCabe, T. J., A complexity measure, IEEE Transactions on Software Engineering, 2 (4), 308--320, 1976.
[50]
Cohen, J., Best Kept Secrets of Peer Code Review: Code reviews at Cisco Systems, 63--87, 2006.

Cited By

View all
  • (2018)Reducing test effortInformation and Software Technology10.1016/j.infsof.2012.04.00754:10(1092-1106)Online publication date: 29-Dec-2018
  • (2018)A systematic mapping study on the combination of static and dynamic quality assurance techniquesInformation and Software Technology10.1016/j.infsof.2011.06.00354:1(1-15)Online publication date: 29-Dec-2018
  • (2014)Analyzing the relationships between inspections and testing to provide a software testing focusInformation and Software Technology10.1016/j.infsof.2014.02.00756:7(793-806)Online publication date: Jul-2014
  • Show More Cited By

Index Terms

  1. Optimizing cost and quality by integrating inspection and test processes

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICSSP '11: Proceedings of the 2011 International Conference on Software and Systems Process
    May 2011
    256 pages
    ISBN:9781450307307
    DOI:10.1145/1987875
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 21 May 2011

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. case study
    2. defect types
    3. inspection
    4. odc
    5. quality assurance strategy
    6. testing
    7. testing focus
    8. two-stage prioritization

    Qualifiers

    • Research-article

    Conference

    ICSSP '11

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)10
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 04 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2018)Reducing test effortInformation and Software Technology10.1016/j.infsof.2012.04.00754:10(1092-1106)Online publication date: 29-Dec-2018
    • (2018)A systematic mapping study on the combination of static and dynamic quality assurance techniquesInformation and Software Technology10.1016/j.infsof.2011.06.00354:1(1-15)Online publication date: 29-Dec-2018
    • (2014)Analyzing the relationships between inspections and testing to provide a software testing focusInformation and Software Technology10.1016/j.infsof.2014.02.00756:7(793-806)Online publication date: Jul-2014
    • (2014)An Integrated Analysis and Testing Methodology to Support Model-Based Quality AssuranceSoftware Quality. Model-Based Approaches for Advanced Software and Systems Engineering10.1007/978-3-319-03602-1_9(135-154)Online publication date: 2014
    • (2012)Guiding Testing Activities by Predicting Defect-Prone Parts Using Product and Inspection MetricsProceedings of the 2012 38th Euromicro Conference on Software Engineering and Advanced Applications10.1109/SEAA.2012.30(406-413)Online publication date: 5-Sep-2012
    • (2012)From Assumptions to Context-Specific Knowledge in the Area of Combined Static and Dynamic Quality AssuranceProceedings of the 2012 38th Euromicro Conference on Software Engineering and Advanced Applications10.1109/SEAA.2012.29(298-301)Online publication date: 5-Sep-2012
    • (2012)Checking consistency between documents of requirements engineering phase2012 XXXVIII Conferencia Latinoamericana En Informatica (CLEI)10.1109/CLEI.2012.6427206(1-10)Online publication date: Oct-2012
    • (2012)Inspection and Test Process Integration Based on Explicit Test Prioritization StrategiesSoftware Quality. Process Automation in Software Development10.1007/978-3-642-27213-4_12(181-192)Online publication date: 2012
    • (2012)Integrating inspection and test processes based on context-specific assumptionsJournal of Software: Evolution and Process10.1002/smr.156926:4(371-385)Online publication date: 30-Aug-2012

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media