Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/2600821.2600840acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
Article

An expert-based cost estimation model for system test execution

Published: 26 May 2014 Publication History

Abstract

To execute system tests, two fundamentally different execution techniques exist: manual and automated execution. For each system test suite, one must decide how to employ those techniques (this strategy is called execution mode). Despite general conditions such as fixed testing strategies or development philosophies, almost all projects permit a wide range of possible execution modes to choose from. In industry, execution techniques are often chosen by experts based on rules of thumb, experience and best practices. Although the results are mostly tolerable, they may be not cost-effective. In retrospect, it is often unclear on what basis those decisions were made, making it difficult to assess whether they are still valid. Finally, it is hard to predict the costs for test execution beforehand. We introduce a cost model to estimate the economic impact of execution modes. Our cost model is based on expert estimations and gives additional input for testing experts in balancing pros and cons of execution modes at hand. Furthermore, it helps documenting and persists decisions during the life time of a test suite. Additionally, we report on a first case study, applying our cost model in industry.

References

[1]
E. Aranha and P. Borba. An estimation model for test execution effort. In ESEM, 2007.
[2]
B. Boehm, C. Abts, and S. Chulani. Software development cost estimation approaches - a survey. Annals of Software Engineering, 10(1-4), 2000.
[3]
B. W. Boehm, Clark, Horowitz, Brown, Reifer, Chulani, R. Madachy, and B. Steece. Software Cost Estimation with Cocomo II. Prentice Hall, 2000.
[4]
F. Calzolari, P. Tonella, and G. Antoniol. Dynamic model for maintenance and testing effort. In ICSM, 1998.
[5]
E. de Almeida, B. de Abreu, and R. Moraes. An alternative approach to test effort estimation based on use cases. In ICST, 2009.
[6]
A. Engel and M. Last. Modeling software testing costs and risks using fuzzy logic paradigm. Journal of Systems and Software, 80(6), 2007.
[7]
M. Fewster and D. Graham. Software test automation: effective use of test execution tools. Addison-W., 1999.
[8]
D. Hoffman. Cost benefits analysis of test automation. In STAR East, 1999.
[9]
M. Jorgensen and M. Shepperd. A systematic review of software development cost estimation studies. Software Engineering, IEEE Trans. on, 33(1), 2007.
[10]
D. S. Kushwaha and A. K. Misra. Software test effort estimation. SIGSOFT Softw. Eng. Notes, 33(3), 2008.
[11]
K. Molokken and M. Jorgensen. A review of software surveys on software effort estimation. In ISESE, 2003.
[12]
H. Pham and X. Zhang. A software cost model with warranty and risk costs. Computers, IEEE Trans. on, 48(1), 1999.
[13]
R. Ramler and K. Wolfmaier. Economic perspectives in test automation: Balancing automated and manual testing with opportunity cost. In AST, 2006.
[14]
C. Schwaber and M. Gilpin. Evaluating automated functional testing tools. Forrester Research, 2005.
[15]
P. R. Srivastava, A. Varshney, P. Nama, and X.-S. Yang. Software test effort estimation: A model based on cuckoo search. Int. J. Bio-Insp. Comp., 4(5), 2012.
[16]
Z. Xiaochun, Z. Bo, W. Fan, Q. Yi, and C. Lu. Estimate test execution effort at an early stage: An empirical study. In CW, 2008.
[17]
X. Zhu, B. Zhou, L. Hou, J. Chen, and L. Chen. An experience-based approach for test execution effort estimation. In ICYCS, 2008.

Cited By

View all
  • (2022)Systematic literature review of mobile application development and testing effort estimationJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2018.11.00234:2(1-15)Online publication date: 1-Feb-2022
  • (2020)Automated System Testing for FPGA-based ADC Reference Design BoardsProceedings of the 6th International Conference on Communication and Information Processing10.1145/3442555.3442583(172-178)Online publication date: 27-Nov-2020

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICSSP '14: Proceedings of the 2014 International Conference on Software and System Process
May 2014
199 pages
ISBN:9781450327541
DOI:10.1145/2600821
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 May 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. System testing
  2. cost model
  3. manual testing
  4. test automation

Qualifiers

  • Article

Conference

ICSSP '14

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 22 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Systematic literature review of mobile application development and testing effort estimationJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2018.11.00234:2(1-15)Online publication date: 1-Feb-2022
  • (2020)Automated System Testing for FPGA-based ADC Reference Design BoardsProceedings of the 6th International Conference on Communication and Information Processing10.1145/3442555.3442583(172-178)Online publication date: 27-Nov-2020

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media