research-article

Search-based optimization for the testing resource allocation problem: research trends and opportunities

Authors:

Roberto Pietrantuono,

Stefano RussoAuthors Info & Claims

SBST '18: Proceedings of the 11th International Workshop on Search-Based Software Testing

Pages 6 - 12

https://doi.org/10.1145/3194718.3194721

Published: 28 May 2018 Publication History

Abstract

This paper explores the usage of search-based techniques for the Testing Resource Allocation Problem (TRAP). We focus on the analysis of the literature, surveying the research proposals where search-based techniques are exploited for different formulations of the TRAP. Three dimensions are considered: the model formulation, solution, and validation. The analysis allows to derive several observations, and finally outline some new research directions towards better (namely, closer to real-world settings) modelling and solutions, highlighting the most promising areas of investigation.

References

[1]

A. Monden et al. 2013. Assessing the Cost Effectiveness of Fault Prediction in Acceptance Testing. IEEE Transactions on Software Engineering 39, 10 (Oct 2013), 1345--1357.

Digital Library

[2]

A.G. Aggarwal, G. Kaur, and P.K. Kapur. 2010. Optimal testing resource allocation for modular software considering imperfect debugging and change point using genetic algorithm. In 2nd International Conference on Reliability, Safety and Hazard - Risk-Based Technologies and Physics-of-Failure Methods (ICRESH). IEEE, 535--541.

[3]

A. Avizienis, J.-C. Laprie, B. Randell, and C. Landwehr. 2004. Basic concepts and taxonomy of dependable and secure computing. IEEE Transactions on Dependable and Secure Computing 1, 1 (Jan 2004), 11--33.

Digital Library

[4]

B.W. Boehm. 1981. Software Engineering Economics (1st ed.). Prentice Hall PTR.

Digital Library

[5]

G. Carrozza, R. Pietrantuono, and S. Russo. 2014. Dynamic test planning: a study in an industrial context. International Journal on Software Tools for Technology Transfer 16, 5 (Oct 2014), 593--607.

Digital Library

[6]

M. Cinque, D. Cotroneo, A. Pecchia, R. Pietrantuono, and S. Russo. 2017. Debugging-workflow-aware software reliability growth analysis. Software Testing, Verification and Reliability 27, 7 (Nov 2017).

[7]

M. Cinque, C. Gaiani, D. De Stradis, A. Pecchia, R. Pietrantuono, and S. Russo. 2014. On the Impact of Debugging on Software Reliability Growth Analysis: A Case Study. In Computational Science and Its Applications - ICCSA 2014 (LNCS), Murgante, B. et alii (Ed.), Vol. 8583. Springer International Publishing, 461--475.

[8]

D. Cotroneo, R. Pietrantuono, and S. Russo. 2013. Combining Operational and Debug Testing for Improving Reliability. IEEE Transactions on Reliability 62, 2 (2013), 408--423.

[9]

Y.S. Dai, M. Xie, K.L. Poh, and B. Yang. 2003. Optimal testing-resource allocation with genetic algorithm for modular software systems. Journal of Systems and Software 66, 1 (Apr 2003), 47--55.

Digital Library

[10]

L. Fiondella and S.S. Gokhale. 2012. Optimal Allocation of Testing Effort Considering Software Architecture. IEEE Transactions on Reliability 61, 2 (June 2012), 580--589.

[11]

R. Gao and S. Xiong. 2015. A genetic local search algorithm for optimal testing resource allocation in module software systems. In Intelligent Computing Theories and Methodologies (LNCS), D.-S. Huang et al. (Ed.), Vol. 9226. Springer International Publishing, 13--23.

[12]

H. Hemmati, M. Nagappan, and A.E. Hassan. 2015. Investigating the effect of "defect co-fix" on quality assurance resource allocation: A search-based approach. Journal of Systems and Software 103 (May 2015), 412--422.

Digital Library

[13]

C.-Y. Huang, S.-Y. Kuo, and M.R. Lyu. 2007. An Assessment of Testing-Effort Dependent Software Reliability Growth Models. IEEE Transactions on Reliability 56, 2 (June 2007), 198--211.

[14]

C.-Y. Huang and J.-H. Lo. 2006. Optimal resource allocation for cost and reliability of modular software systems in the testing phase. Journal of Systems and Software 79, 5 (May 2006), 653--664.

Digital Library

[15]

C.-Y. Huang and M.R. Lyu. 2005. Optimal release time for software systems considering cost, testing-effort, and test efficiency. IEEE Transactions on Reliability 54, 4 (Dec 2005), 583--591.

[16]

P.K. Kapur, A.G. Aggarwal, K. Kapoor, and G. Kaur. 2009. Optimal testing resource allocation for modular software considering cost, testing effort and reliability using genetic algorithm. International Journal of Reliability, Quality and Safety Engineering 16, 06 (Dec 2009), 495--508.

[17]

V. Kumar, P.K. Kapur, N. Taneja, and R. Sahni. 2017. On allocation of resources during testing phase incorporating flexible software reliability growth model with testing effort under dynamic environment. International Journal of Operational Research 30, 4 (2017), 523--539.

[18]

V. Kumar, S.K. Khatri, H. Dua, M. Sharma, and P. Mathur. 2014. An assessment of testing cost with effort-dependent fdp and fcp under learning effect: a genetic algorithm approach. International Journal of Reliability, Quality and Safety Engineering 21, 06 (2014), 1450027.

[19]

W. Kuo and R. Wan. 2007. Recent Advances in Optimal Reliability Allocation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 37, 2 (Mar 2007), 143--156.

Digital Library

[20]

J.-H. Lo and C.-Y. Huang. 2006. An integration of fault detection and correction processes in software reliability analysis. Journal of Systems and Software 79, 9 (2006), 1312--1323.

Digital Library

[21]

Y. Lu, F. Yue, G. Zhang, Z. Su, and Y.-Q. Wang. 2016. Model and solution to testing resource dynamic allocation for series-parallel software systems. 27 (08 2016), 1964--1977.

[22]

M.R. Lyu (Ed.). 1996. Handbook of Software Reliability Engineering. McGraw-Hill, Inc., Hightstown, NJ, USA.

Digital Library

[23]

M. Nasar and P. Johri. 2016. Testing Resource Allocation for Fault Detection Process. In Smart Trends in Information Technology and Computer Communications, A. Unal et al. (Ed.). Springer, Singapore, 683--690.

[24]

Mauro Pezzè and Michal Young. 2007. Software Testing and Analysis: Process, Principles and Techniques. Wiley.

[25]

R. Pietrantuono, P. Potena, A. Pecchia, D. Rodriguez, S. Russo, and L. Fernandez. 2017. Multi-Objective Testing Resource Allocation under Uncertainty. IEEE Transactions on Evolutionary Computation PP, 99 (2017).

[26]

R. Pietrantuono, S. Russo, and K.S. Trivedi. 2010. Software Reliability and Testing Time Allocation: An Architecture-Based Approach. IEEE Transactions on Software Engineering 36, 3 (May 2010), 323--337.

Digital Library

[27]

M. Sangeetha, C. Arumugam, K.M. Senthil Kumar, and S. Hari Shankar. 2015. An Effective Approach to Support Multi-objective Optimization in Software Reliability Allocation for Improving Quality. Procedia Computer Science 47 (2015), 118--127.

[28]

Y. Shuaishuai, F. Dong, and B. Li. 2013. Optimal Testing Resource Allocation for modular software systems based-on multi-objective evolutionary algorithms with effective local search strategy. In IEEE Workshop on Memetic Computing (MC). IEEE, 1--8.

[29]

S. Wang, S. Ali, T. Yue, Ø. Bakkeli, and M. Liaaen. 2016. Enhancing Test Case Prioritization in an Industrial Setting with Resource Awareness and Multi-objective Search. In 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C). 182--191.

Digital Library

[30]

Z. Wang, K. Tang, and X. Yao. 2008. A multi-objective approach to testing resource allocation in modular software systems. In IEEE Congress on Evolutionary Computation. IEEE, 1148--1153.

[31]

Z. Wang, K. Tang, and X. Yao. 2010. Multi-Objective Approaches to Optimal Testing Resource Allocation in Modular Software Systems. IEEE Transactions on Reliability 59, 3 (Sept 2010), 563--575.

[32]

S. Yamada, J. Hishitani, and S. Osaki. 1993. Software-Reliability Growth with a Weibull Test-Effort: A Model & Application. IEEE Transactions on Reliability 42, 1 (Mar 1993), 100--106.

[33]

B. Yang, Y. Hu, and C.-Y. Huang. 2015. An Architecture-Based Multi-Objective Optimization Approach to Testing Resource Allocation. IEEE Transactions on Reliability 64, 1 (Mar 2015), 497--515.

[34]

G. Zhang, Z. Su, M. Li, F. Yue, J. Jiang, and X. Yao. 2017. Constraint Handling in NSGA-II for Solving Optimal Testing Resource Allocation Problems. IEEE Transactions on Reliability 66, 4 (Dec 2017), 1193--1212.

Cited By

Juhászné Bíró TNémeth P(2022)The Importance and Applicability of Metaheuristics in Supply ChainsBlockchain Applications in Cryptocurrency for Technological Evolution10.4018/978-1-6684-6247-8.ch015(249-267)Online publication date: 30-Dec-2022
https://doi.org/10.4018/978-1-6684-6247-8.ch015
Su ZZhang GYue FZhan DLi MLi BYao X(2021)Enhanced Constraint Handling for Reliability-Constrained Multiobjective Testing Resource AllocationIEEE Transactions on Evolutionary Computation10.1109/TEVC.2021.305553825:3(537-551)Online publication date: Jun-2021
https://doi.org/10.1109/TEVC.2021.3055538

Index Terms

Search-based optimization for the testing resource allocation problem: research trends and opportunities
1. Software and its engineering
  1. Software creation and management
  2. Software organization and properties
    1. Extra-functional properties
      1. Software reliability

Recommendations

Efficient allocation of testing resources for software module testing based on the hyper-geometric distribution software reliability growth model
ISSRE '96: Proceedings of the The Seventh International Symposium on Software Reliability Engineering

A considerable amount of testing resources is required during software module testing. In this paper, based on the HGDM (Hyper-Geometric Distribution Model) software reliability growth model, we investigate the following optimal resource allocation ...
Faster mutation testing inspired by test prioritization and reduction
ISSTA 2013: Proceedings of the 2013 International Symposium on Software Testing and Analysis

Mutation testing is a well-known but costly approach for determining test adequacy. The central idea behind the approach is to generate mutants, which are small syntactic transformations of the program under test, and then to measure for a given test ...
On the testing resource allocation problem: Research trends and perspectives
Highlights
- TRAP is a well rooted area with consolidated findings and many future directions.
Abstract
In testing a software application, a primary concern is how to effectively plan the assignment of resources available for testing to the software components so as to achieve a target goal under given constraints. In the literature, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SBST '18: Proceedings of the 11th International Workshop on Search-Based Software Testing

May 2018

84 pages

ISBN:9781450357418

DOI:10.1145/3194718

Program Chairs:
Juan Pablo Galeotti
Universidad de Buenos Aires, Argentina
,
Alessandra Gorla
IMDEA Software, Spain

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS: Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 May 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

MIUR

Conference

ICSE '18

Sponsor:

SIGSOFT
IEEE-CS

ICSE '18: 40th International Conference on Software Engineering

May 28 - 29, 2018

Gothenburg, Sweden

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
85
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Juhászné Bíró TNémeth P(2022)The Importance and Applicability of Metaheuristics in Supply ChainsBlockchain Applications in Cryptocurrency for Technological Evolution10.4018/978-1-6684-6247-8.ch015(249-267)Online publication date: 30-Dec-2022
https://doi.org/10.4018/978-1-6684-6247-8.ch015
Su ZZhang GYue FZhan DLi MLi BYao X(2021)Enhanced Constraint Handling for Reliability-Constrained Multiobjective Testing Resource AllocationIEEE Transactions on Evolutionary Computation10.1109/TEVC.2021.305553825:3(537-551)Online publication date: Jun-2021
https://doi.org/10.1109/TEVC.2021.3055538

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents