research-article

A systematic mapping study on testing technique experiments: has the situation changed since 2000?

Authors:

Jorge E. González,

Natalia Juristo,

Sira VegasAuthors Info & Claims

ESEM '14: Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

Article No.: 3, Pages 1 - 4

https://doi.org/10.1145/2652524.2652569

Published: 18 September 2014 Publication History

Get Access

Abstract

Context: Juristo et al. [7] published a literature review about testing technique experiments. The goal was to provide a picture of which techniques and aspects of techniques had been studied experimentally, and try to compile a body of knowledge on testing techniques. Goal: In this paper, we extend Juristo et al.'s study to cover the years from 2000 (where it ended) until 2013. Method: We have performed a systematic mapping study. Results: The situation in testing experimentation has not changed since Juristo et al.'s study. Conclusions: The research field has the same shortcomings.

References

[1]

O. Dieste, A. Grimán, N. Juristo. Developing search strategies for detecting relevant experiments. Empirical Software Engineering, vol 14, no. 5, p. 513--539, 2009.

Digital Library

Google Scholar

[2]

E. Engstrom, P. Runeson, M. Skoglund, A systematic review on regression test selection techniques, Information and Software Technology, vol. 52, no. 1, p. 14--30, 2010.

Digital Library

Google Scholar

[3]

E. Engstrom, M. Skoglund, P. Runeson, Empirical evaluations of regression test selection techniques: A systematic review, In Proceedings of the 2^nd ESEM, pp. 22--31, 2008.

Digital Library

Google Scholar

[4]

Y. Jia, M. Harman, An analysis and survey of the development of mutation testing, IEEE Transactions on Software Engineering, vol. 37, no. 5, p. 649--678, 2011.

Digital Library

Google Scholar

[5]

A. Jedlitschka, M. Ciolkowski, D. Pfahl. Reporting Experiments in Software Engineering. Guide to Advanced Empirical Software Engineering. Springer, 2008.

Google Scholar

[6]

N. Juristo, A. M. Moreno. Basics of Software Engineering Experimentation. Springer, 2001.

Digital Library

Google Scholar

[7]

N. Juristo, A. M. Moreno, S. Vegas, Reviewing 25 Years of Testing Technique Experiments. Empirical Software Engineering, vol. 9, no. 1-2, pp. 7--44, 2004.

Digital Library

Google Scholar

[8]

K. Petersen, R. Feldt, S. Mujtaba, M. Mattsson. Systematic mapping studies in software engineering. In Proceedings of the 12^th EASE. pp. 71--80, 2008,

Digital Library

Google Scholar

[9]

Y. Singh, A. Kaur, B. Suri, S. Singhal, Systematic literature review on regression test prioritization techniques, Informatica, vol. 36, no. 4, p. 379--408, 2012.

Google Scholar

[10]

C. Wohlin, P. Runeson, M. Höst. Experimentation in Software Engineering. Springer, 2012.

Digital Library

Google Scholar

Cited By

View all

van Heugten Breurkes JGilson FGalster M(2022)Overlap between Automated Unit and Acceptance Testing – a Systematic Literature ReviewProceedings of the 26th International Conference on Evaluation and Assessment in Software Engineering10.1145/3530019.3530028(80-89)Online publication date: 13-Jun-2022
https://dl.acm.org/doi/10.1145/3530019.3530028
Felizardo KRamos AO. Melo Cde Souza ÉVijaykumar NNakagawa E(2021)Global and Latin American female participation in evidence-based software engineering: a systematic mapping studyJournal of the Brazilian Computer Society10.1186/s13173-021-00109-727:1Online publication date: 19-Mar-2021
https://doi.org/10.1186/s13173-021-00109-7
Paiva SSimão ACavalcante EDantas FBatista T(2020)An Experimental Study for Complete-IOCO TheoryProceedings of the 5th Brazilian Symposium on Systematic and Automated Software Testing10.1145/3425174.3425232(107-116)Online publication date: 20-Oct-2020
https://dl.acm.org/doi/10.1145/3425174.3425232
Show More Cited By

Index Terms

A systematic mapping study on testing technique experiments: has the situation changed since 2000?
1. Software and its engineering

Recommendations

A mapping study on testing non-testable systems

The terms "Oracle Problem" and "Non-testable system" interchangeably refer to programs in which the application of test oracles is infeasible. Test oracles are an integral part of conventional testing techniques; thus, such techniques are inoperable in ...
A systematic mapping on software testing for blockchains
Abstract
The purpose of this study is to identify and classify studies published on software testing techniques applied to blockchain systems. Previously published reviews in related areas have a narrow focus and/or do not follow a systematic review ...
A Survey of Testing Techniques Based on Large Language Models
ICCMT '24: Proceedings of the 2024 International Conference on Computer and Multimedia Technology

With the development of software testing technology, Large Language Model (LLM) driven testing method have gradually become an emerging trend in the field of software testing. This paper presents a comprehensive review of LLM-based testing techniques. ...

Comments

Information & Contributors

Information

Published In

ESEM '14: Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement

September 2014

461 pages

ISBN:9781450327749

DOI:10.1145/2652524

General Chair:
Maurizio Morisio
Politecnico di Torino, Italy
,
Program Chairs:
Tore Dybå
Sintef, Norway
,
Marco Torchiano
Politecnico di Torino, Italy

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 September 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Ministerio de Economía y Competitividad

Conference

ESEM '14

Sponsor:

SIGSOFT

ESEM '14: 2014 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement

September 18 - 19, 2014

Torino, Italy

Acceptance Rates

ESEM '14 Paper Acceptance Rate 23 of 123 submissions, 19%;

Overall Acceptance Rate 130 of 594 submissions, 22%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
316
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

van Heugten Breurkes JGilson FGalster M(2022)Overlap between Automated Unit and Acceptance Testing – a Systematic Literature ReviewProceedings of the 26th International Conference on Evaluation and Assessment in Software Engineering10.1145/3530019.3530028(80-89)Online publication date: 13-Jun-2022
https://dl.acm.org/doi/10.1145/3530019.3530028
Felizardo KRamos AO. Melo Cde Souza ÉVijaykumar NNakagawa E(2021)Global and Latin American female participation in evidence-based software engineering: a systematic mapping studyJournal of the Brazilian Computer Society10.1186/s13173-021-00109-727:1Online publication date: 19-Mar-2021
https://doi.org/10.1186/s13173-021-00109-7
Paiva SSimão ACavalcante EDantas FBatista T(2020)An Experimental Study for Complete-IOCO TheoryProceedings of the 5th Brazilian Symposium on Systematic and Automated Software Testing10.1145/3425174.3425232(107-116)Online publication date: 20-Oct-2020
https://dl.acm.org/doi/10.1145/3425174.3425232
Kazmi RJawawi DMohamad RGhani I(2017)Effective Regression Test Case SelectionACM Computing Surveys10.1145/305726950:2(1-32)Online publication date: 25-May-2017
https://dl.acm.org/doi/10.1145/3057269

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A mapping study on testing non-testable systems

A systematic mapping on software testing for blockchains

A Survey of Testing Techniques Based on Large Language Models