research-article

Towards developer support for merging forked test cases

Authors:

Sandro Schulze,

Johannes WünscheAuthors Info & Claims

SPLC '22: Proceedings of the 26th ACM International Systems and Software Product Line Conference - Volume A

Pages 131 - 141

https://doi.org/10.1145/3546932.3547002

Published: 12 September 2022 Publication History

Abstract

Developers rely on branching and forking mechanisms of modern versioning systems to evolve and maintain their software systems. As a result, systems often exist in the form of various short-living or even long-living (i.e., clone & own development) variants. Such variants may have to be merged with the main system or other variants, for instance, to propagate features or bug fixes. Within such merging processes, test cases are highly interesting, since they allow to improve the test coverage and hopefully the reliability of the system (e.g., by merging missing tests and bug fixes in test code). However, as all source code, test cases may evolve independently between two or more variants, which makes it non-trivial to decide what changes of the test cases are relevant for the merging. For instance, some test cases in one variant may be irrelevant in another variant (e.g., because the feature shall not be propagated) or may subsume existing test cases. In this paper, we propose a technique that allows for a fine-grained comparison of test cases to support developers in deciding whether and how to merge these. Precisely, inspired by code-clone detection, we use abstract syntax trees to decide on the relations between test cases of different variants. We evaluate the applicability of our technique qualitatively on five open-source systems written in Java (e.g., JUnit 5, Guava). Our insights into the merge potential of 50 pull requests with test cases from these systems indicate that our technique can support the comprehension of differences in variants' test cases, and also highlight future research opportunities.

References

[1]

Qurat U. Ain, Wasi H. Butt, Muhammad W. Anwar, Farooque Azam, and Bilal Maqbool. 2019. A Systematic Review on Code Clone Detection. IEEE Access 7 (2019), 86121--86144.

[2]

Sofia Ananieva, Sandra Greiner, Timo Kehrer, Jacob Krüger, Thomas Kühn, Lukas Linsbauer, Sten Grüner, Anne Koziolek, Henrik Lönn, S. Ramesh, and Ralf Reussner. 2022. A Conceptual Model for Unifying Variability in Space and Time: Rationale, Validation, and Illustrative Applications. Empirical Software Engineering 27, 101 (2022), 1--53.

Digital Library

[3]

Sofia Ananieva, Sandra Greiner, Thomas Kühn, Jacob Krüger, Lukas Linsbauer, Sten Grüner, Timo Kehrer, Heiko Klare, Anne Koziolek, Henrik Lönn, Sebastian Krieter, Christoph Seidl, S. Ramesh, Ralf Reussner, and Bernhard Westfechtel. 2020. A Conceptual Model for Unifying Variability in Space and Time. In International Systems and Software Product Line Conference (SPLC). ACM, 15:1--12.

Digital Library

[4]

Sven Apel, Don Batory, Christian Kästner, and Gunter Saake. 2013. Feature-Oriented Software Product Lines. Springer.

[5]

Sven Apel, Jörg Liebig, Benjamin Brandl, Christian Lengauer, and Christian Kästner. 2011. Semistructured Merge: Rethinking Merge in Revision Control Systems. In Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE). ACM, 190--200.

Digital Library

[6]

Stefan Bellon, Rainer Koschke, Giuliano Antoniol, Jens Krinke, and Ettore Merlo. 2007. Comparison and Evaluation of Clone Detection Tools. IEEE Transactions on Software Engineering 33, 9 (2007), 577--591.

Digital Library

[7]

Jan Bosch (Ed.). 2014. Continuous Software Engineering. Springer.

[8]

Emanuela G. Cartaxo, Patrícia D. L. Machado, and Francisco G. O. Neto. 2009. On the Use of a Similarity Function for Test Case Selection in the Context of Model-Based Testing. Software Testing, Verification and Reliability 21, 2 (2009), 75--100.

Digital Library

[9]

Peng-Hua Chu, Nien-Lin Hsueh, Hong-Hsiang Chen, and Chien-Hung Liu. 2011. A Test Case Refactoring Approach for Pattern-Based Software Development. Software Quality Journal 20, 1 (2011), 43--75.

Digital Library

[10]

Harald Cichos and Thomas S. Heinze. 2011. Efficient Test Suite Reduction by Merging Pairs of Suitable Test Cases. In International Conference on Model Driven Engineering Languages and Systems (MoDELS). Springer, 244--258.

[11]

Emilio Cruciani, Breno Miranda, Roberto Verdecchia, and Antonia Bertolino. 2019. Scalable Approaches for Test Suite Reduction. In International Conference on Software Engineering (ICSE). IEEE, 419--429.

Digital Library

[12]

Laura Dabbish, Colleen Stuart, Jason Tsay, and Jim Herbsleb. 2012. Social Coding in GitHub: Transparency and Collaboration in an Open Software Repository. In Conference on Computer Supported Cooperative Work (CSCW). ACM, 1277--1286.

Digital Library

[13]

Yael Dubinsky, Julia Rubin, Thorsten Berger, Slawomir Duszynski, Martin Becker, and Krzysztof Czarnecki. 2013. An Exploratory Study of Cloning in Industrial Software Product Lines. In European Conference on Software Maintenance and Reengineering (CSMR). IEEE, 25--34.

Digital Library

[14]

Georgios Gousios, Andy Zaidman, Margaret-Anne Storey, and Arie van Deursen. 2015. Work Practices and Challenges in Pull-Based Development: The Integrator's Perspective. In International Conference on Software Engineering (ICSE). IEEE, 358--368.

[15]

Eduardo M. Guerra and Clovis T. Fernandes. 2007. Refactoring Test Code Safely. In International Conference on Software Engineering Advances (ICSEA). IEEE, 44:1--6.

Digital Library

[16]

Dan Hao, Lu Zhang, Xingxia Wu, Hong Mei, and Gregg Rothermel. 2012. On-Demand Test Suite Reduction. In International Conference on Software Engineering (ICSE). IEEE, 738--748.

[17]

Jacob Krüger, Mustafa Al-Hajjaji, Sandro Schulze, Gunter Saake, and Thomas Leich. 2018. Towards Automated Test Refactoring for Software Product Lines. In International Systems and Software Product Line Conference (SPLC). ACM, 143--148.

Digital Library

[18]

Jacob Krüger and Thorsten Berger. 2020. An Empirical Analysis of the Costs of Clone- and Platform-Oriented Software Reuse. In Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE). ACM, 432--444.

Digital Library

[19]

Jacob Krüger, Wardah Mahmood, and Thorsten Berger. 2020. Promote-pl: A Round-Trip Engineering Process Model for Adopting and Evolving Product Lines. In International Systems and Software Product Line Conference (SPLC). ACM, 2:1--12.

Digital Library

[20]

Jacob Krüger, Mukelabai Mukelabai, Wanzi Gu, Hui Shen, Regina Hebig, and Thorsten Berger. 2019. Where is My Feature and What is it About? A Case Study on Recovering Feature Facets. Journal of Systems and Software 152 (2019), 239--253.

Digital Library

[21]

Elias Kuiter, Jacob Krüger, Sebastian Krieter, Thomas Leich, and Gunter Saake. 2018. Getting Rid of Clone-And-Own: Moving to a Software Product Line for Temperature Monitoring. In International Systems and Software Product Line Conference (SPLC). ACM, 189--189.

Digital Library

[22]

Remo Lachmann, Sascha Lity, Mustafa Al-Hajjaji, Franz Fürchtegott, and Ina Schaefer. 2016. Fine-Grained Test Case Prioritization for Integration Testing of Delta-Oriented Software Product Lines. In International Workshop on Feature-Oriented Software Development (FOSD). ACM, 1--10.

Digital Library

[23]

Yuejian Li and Nancy J. Wahl. 1999. An Overview of Regression Testing. ACM SIGSOFT Software Engineering Notes 24, 1 (1999), 69--73.

Digital Library

[24]

Panagiotis Louridas. 2006. Version Control. IEEE Software 23, 1 (2006), 104--107.

Digital Library

[25]

Wardah Mahmood, Moses Chagama, Thorsten Berger, and Regina Hebig. 2020. Causes of Merge Conflicts: A Case Study of ElasticSearch. In International Working Conference on Variability Modelling of Software-Intensive Systems (VaMoS). ACM, 9:1--9.

Digital Library

[26]

Hung V. Nguyen, My H. Nguyen, Son C. Dang, Christian Kästner, and Tien N. Nguyen. 2015. Detecting Semantic Merge Conflicts with Variability-Aware Execution. In Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE). ACM, 926--929.

Digital Library

[27]

Harrie Passier, Lex Bijlsma, and Christoph Bockisch. 2016. Maintaining Unit Tests During Refactoring. In International Conference on Principles and Practices of Programming on the Java Platform (PPPJ). ACM, 1--6.

Digital Library

[28]

Gregg Rothermel, Mary J. Harrold, Jeffery von Ronne, and Christie Hong. 2002. Empirical Studies of Test-Suite Reduction. Software Testing, Verification and Reliability 12, 4 (2002), 219--249.

[29]

Chanchal K. Roy, James R. Cordy, and Rainer Koschke. 2009. Comparison and Evaluation of Code Clone Detection Techniques and Tools: A Qualitative Approach. Science of Computer Programming 74, 7 (2009), 470--495.

Digital Library

[30]

August Shi, Alex Gyori, Suleman Mahmood, Peiyuan Zhao, and Darko Marinov. 2018. Evaluating Test-Suite Reduction in Real Software Evolution. In International Symposium on Software Testing and Analysis (ISSTA). ACM, 84--94.

Digital Library

[31]

G. Shobha, Ajay Rana, Vineet Kansal, and Sarvesh Tanwar. 2021. Code Clone Detection---A Systematic Review. In International Conference on Emerging Technologies in Data Mining and Information Security (IEMIS). Springer, 645--655.

[32]

Leuson D. Silva, Paulo Borba, Wardah Mahmood, Thorsten Berger, and João Moisakis. 2020. Detecting Semantic Conflicts via Automated Behavior Change Detection. In International Conference on Software Maintenance and Evolution (ICSME). IEEE, 174--184.

[33]

Diomidis Spinellis. 2005. Version Control Systems. IEEE Software 22, 5 (2005), 108--109.

Digital Library

[34]

Ştefan Stąnciulescu, Sandro Schulze, and Andrzej Wąsowski. 2015. Forked and Integrated Variants in an Open-Source Firmware Project. In International Conference on Software Maintenance and Evolution (ICSME). IEEE, 151--160.

Digital Library

[35]

Daniel Strüber, Mukelabai Mukelabai, Jacob Krüger, Stefan Fischer, Lukas Linsbauer, Jabier Martinez, and Thorsten Berger. 2019. Facing the Truth: Benchmarking the Techniques for the Evolution of Variant-Rich Systems. In International Systems and Software Product Line Conference (SPLC). ACM, 177--188.

Digital Library

[36]

Arie van Deursen, Leon M. F. Moonen, Alex van den Bergh, and Gerard Kok. 2001. Refactoring Test Code. Technical Report SEN-R0119. CWI.

[37]

Shurui Zhou, Ştefan Stąnciulescu, Olaf Leßenich, Yingfei Xiong, Andrzej Wąsowski, and Christian Kästner. 2018. Identifying Features in Forks. In International Conference on Software Engineering (ICSE). ACM, 106--116.

Digital Library

Cited By

Chen SCleophas LSchulze SKrüger J(2024)Use the Forks, Look! Visualizations for Exploring Fork Ecosystems2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00107(993-1004)Online publication date: 12-Mar-2024
https://doi.org/10.1109/SANER60148.2024.00107
Krüger JMikulinski ASchulze SLeich TSaake G(2023)DSDGenProceedings of the 27th ACM International Systems and Software Product Line Conference - Volume B10.1145/3579028.3609015(47-56)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1145/3579028.3609015
Krieter SKrüger JLeich TSaake G(2023)VariantIncProceedings of the 27th ACM International Systems and Software Product Line Conference - Volume A10.1145/3579027.3608984(129-140)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1145/3579027.3608984
Show More Cited By

Index Terms

Towards developer support for merging forked test cases
1. Software and its engineering
  1. Software creation and management

Recommendations

A Static Approach to Prioritizing JUnit Test Cases

Test case prioritization is used in regression testing to schedule the execution order of test cases so as to expose faults earlier in testing. Over the past few years, many test case prioritization techniques have been proposed in the literature. Most ...
An Eclipse Plug-in Tool for Generating Test Cases from Source Codes
APIT '19: Proceedings of the 2019 Asia Pacific Information Technology Conference

Most software development will use a standard software development life cycle (SDLC). In SDLC, four phases are involved during development of any system. They are analysis, design, implement and testing. During testing phase, test cases are used in ...
Merging Test Models
ICECCS '13: Proceedings of the 2013 18th International Conference on Engineering of Complex Computer Systems

Over the last decade several model merging approaches have been proposed. Most of these approaches focused on version control systems. We are interested in merging test models. We present a merging approach to reconcile unit test models and generate ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SPLC '22: Proceedings of the 26th ACM International Systems and Software Product Line Conference - Volume A

September 2022

266 pages

ISBN:9781450394437

DOI:10.1145/3546932

General Chairs:
Alexander Felfernig
Graz University of Technology, Austria
,
Lidia Fuentes
University of Málaga, Spain

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 September 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Author Tags

Qualifiers

Research-article

Conference

SPLC '22

Sponsor:

SIGSOFT

SPLC '22: 26th ACM International Systems and Software Product Line Conference

September 12 - 16, 2022

Graz, Austria

Acceptance Rates

SPLC '22 Paper Acceptance Rate 14 of 41 submissions, 34%;

Overall Acceptance Rate 167 of 463 submissions, 36%

Upcoming Conference

SPLC '24

Sponsor:
sigsoft

28th ACM International Systems and Software Product Line Conference

September 2 - 6, 2024

Dommeldange , Luxembourg

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
82
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)1

Reflects downloads up to 09 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen SCleophas LSchulze SKrüger J(2024)Use the Forks, Look! Visualizations for Exploring Fork Ecosystems2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER60148.2024.00107(993-1004)Online publication date: 12-Mar-2024
https://doi.org/10.1109/SANER60148.2024.00107
Krüger JMikulinski ASchulze SLeich TSaake G(2023)DSDGenProceedings of the 27th ACM International Systems and Software Product Line Conference - Volume B10.1145/3579028.3609015(47-56)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1145/3579028.3609015
Krieter SKrüger JLeich TSaake G(2023)VariantIncProceedings of the 27th ACM International Systems and Software Product Line Conference - Volume A10.1145/3579027.3608984(129-140)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1145/3579027.3608984
Mukelabai MDerks CKrüger JBerger T(2023)To Share, or Not to Share: Exploring Test-Case Reusability in Fork Ecosystems2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)10.1109/ASE56229.2023.00191(837-849)Online publication date: 11-Sep-2023
https://doi.org/10.1109/ASE56229.2023.00191

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents