research-article

Can We Increase the Test-coverage in Libraries using Dependent Projects’ Test-suites?

Authors:

Igor Schittekat,

Mehrdad Abdi,

Serge DemeyerAuthors Info & Claims

EASE '22: Proceedings of the 26th International Conference on Evaluation and Assessment in Software Engineering

Pages 294 - 298

https://doi.org/10.1145/3530019.3535309

Published: 13 June 2022 Publication History

Get Access

Abstract

Modern software systems increasingly depend on packages released on code sharing platforms such as GitHub, Bitbucket, and GitLab. To minimize the risk of lurking defects in such packages, strong test suites covering the normal as well as the exceptional paths are needed. In this paper we explore the potential of using tests from dependent projects to increase the code coverage of base packages. We extracted 4 popular Python packages from GitHub together with 14 dependent projects and analyzed the code coverage of the available tests. We observed that adopting the tests of the dependent projects in the test suite of the base library, would increase the line coverage in 9 out of 14 (64%) of the cases and the mutation coverage in all of them (100%). Our results suggest that a tool which would generate tests for the base package based on the tests in the dependent projects, would help to strengthen the test suite.

References

[1]

Robert V. Binder. 2000. Testing Object-Oriented Systems: Models, Patterns, and Tools. AddissonWesley.

Google Scholar

[2]

Benjamin Danglot, Oscar Vera-Perez, Zhongxing Yu, Andy Zaidman, Martin Monperrus, and Benoit Baudry. 2019. A snowballing literature study on test amplification. Journal of Systems and Software 157 (2019), 110398.

Crossref

Google Scholar

[3]

Alexandre Decan, Tom Mens, and Maëlick Claes. 2017. An empirical comparison of dependency issues in OSS packaging ecosystems. In 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER). 2 – 12. https://doi.org/10.1109/SANER.2017.7884604

Crossref

Google Scholar

[4]

Joseph Hejderup and Georgios Gousios. 2022. Can we trust tests to automate dependency updates? A case study of Java Projects. Journal of Systems and Software 183 (2022), 111097. https://doi.org/10.1016/j.jss.2021.111097

Crossref

Google Scholar

[5]

Jetbrains. 2021. The State of Developer Ecosystem 2021. https://www.jetbrains.com/lp/devecosystem-2021/ Last accessed: March 2022.

Google Scholar

[6]

Yue Jia and Mark Harman. 2011. An Analysis and Survey of the Development of Mutation Testing. IEEE Transactions on Software Engineering 37, 5 (2011), 649–678. https://doi.org/10.1109/TSE.2010.62

Digital Library

Google Scholar

[7]

Filip Křikava and Jan Vitek. 2018. Tests from Traces: Automated Unit Test Extraction for R. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis (Amsterdam, Netherlands) (ISSTA 2018). Association for Computing Machinery, New York, NY, USA, 232–241. https://doi.org/10.1145/3213846.3213863

Digital Library

Google Scholar

[8]

Konstantinos Manikas and Klaus Hansen. 2013. Software ecosystems – A systematic literature review. Journal of Systems and Software 86 (05 2013), 1294–1306. https://doi.org/10.1016/j.jss.2012.12.026

Digital Library

Google Scholar

[9]

David Saff, Shay Artzi, Jeff H. Perkins, and Michael D. Ernst. 2005. Automatic Test Factoring for Java. In Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering (Long Beach, CA, USA) (ASE ’05). Association for Computing Machinery, New York, NY, USA, 114–123. https://doi.org/10.1145/1101908.1101927

Digital Library

Google Scholar

[10]

Ebert Schoofs, Mehrdad Abdi, and Serge Demeyer. 2021. AmPyfier: Test Amplification in Python. arxiv:2112.11155 [cs.SE]

Google Scholar

[11]

Hongyu Zhai, Casey Casalnuovo, and Prem Devanbu. 2019. Test Coverage in Python Programs. In 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR). 116–120. https://doi.org/10.1109/MSR.2019.00027

Digital Library

Google Scholar

[12]

Sai Zhang, David Saff, Yingyi Bu, and Michael D. Ernst. 2011. Combined Static and Dynamic Automated Test Generation. In Proceedings of the 2011 International Symposium on Software Testing and Analysis (Toronto, Ontario, Canada) (ISSTA ’11). Association for Computing Machinery, New York, NY, USA, 353–363. https://doi.org/10.1145/2001420.2001463

Digital Library

Google Scholar

Cited By

View all

Morisaki KShimari KIshio TMatsumoto K(2024)Test Case Generation for Python Libraries Using Dependent Projects' Test-Suites2024 IEEE International Conference on Software Analysis, Evolution and Reengineering - Companion (SANER-C)10.1109/SANER-C62648.2024.00029(167-174)Online publication date: 12-Mar-2024
https://doi.org/10.1109/SANER-C62648.2024.00029
Businge JAbdi MDemeyer S(2023)Analyzing Variant Forks of Software Repositories from Social Coding PlatformsSoftware Ecosystems10.1007/978-3-031-36060-2_6(131-152)Online publication date: 26-May-2023
https://doi.org/10.1007/978-3-031-36060-2_6
Abdi MDemeyer S(2022)Test Transplantation through Dynamic Test Slicing2022 IEEE 22nd International Working Conference on Source Code Analysis and Manipulation (SCAM)10.1109/SCAM55253.2022.00009(35-39)Online publication date: Oct-2022
https://doi.org/10.1109/SCAM55253.2022.00009

Recommendations

Checked Coverage and Object Branch Coverage: New Alternatives for Assessing Student-Written Tests
SIGCSE '15: Proceedings of the 46th ACM Technical Symposium on Computer Science Education

Many educators currently use code coverage metrics to assess student-written software tests. While test adequacy criteria such as statement or branch coverage can also be used to measure the thoroughness of a test suite, they have limitations. Coverage ...
Achieving scalable mutation-based generation of whole test suites

Without complete formal specification, automatically generated software tests need to be manually checked in order to detect faults. This makes it desirable to produce the strongest possible test set while keeping the number of tests as small as ...
Comparing test quality measures for assessing student-written tests
ICSE Companion 2014: Companion Proceedings of the 36th International Conference on Software Engineering

Many educators now include software testing activities in programming assignments, so there is a growing demand for appropriate methods of assessing the quality of student-written software tests. While tests can be hand-graded, some educators also use ...

Comments

Information & Contributors

Information

Published In

EASE '22: Proceedings of the 26th International Conference on Evaluation and Assessment in Software Engineering

June 2022

466 pages

ISBN:9781450396134

DOI:10.1145/3530019

Editors:
Miroslaw Staron
Chalmers | University of Gothenburg, Sweden
,
Christian Berger
Chalmers | University of Gothenburg, Sweden
,
Jocelyn Simmonds
University of Chile, Chile
,
Rafael Prikladnicki
School of Technology at PUCRS University, Brazil

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 June 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

FWO, FNRS

Conference

EASE 2022

EASE 2022: The International Conference on Evaluation and Assessment in Software Engineering 2022

June 13 - 15, 2022

Gothenburg, Sweden

Acceptance Rates

Overall Acceptance Rate 71 of 232 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
94
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)2

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Morisaki KShimari KIshio TMatsumoto K(2024)Test Case Generation for Python Libraries Using Dependent Projects' Test-Suites2024 IEEE International Conference on Software Analysis, Evolution and Reengineering - Companion (SANER-C)10.1109/SANER-C62648.2024.00029(167-174)Online publication date: 12-Mar-2024
https://doi.org/10.1109/SANER-C62648.2024.00029
Businge JAbdi MDemeyer S(2023)Analyzing Variant Forks of Software Repositories from Social Coding PlatformsSoftware Ecosystems10.1007/978-3-031-36060-2_6(131-152)Online publication date: 26-May-2023
https://doi.org/10.1007/978-3-031-36060-2_6
Abdi MDemeyer S(2022)Test Transplantation through Dynamic Test Slicing2022 IEEE 22nd International Working Conference on Source Code Analysis and Manipulation (SCAM)10.1109/SCAM55253.2022.00009(35-39)Online publication date: Oct-2022
https://doi.org/10.1109/SCAM55253.2022.00009

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Abstract

References

Cited By

Recommendations

Checked Coverage and Object Branch Coverage: New Alternatives for Assessing Student-Written Tests

Achieving scalable mutation-based generation of whole test suites

Comparing test quality measures for assessing student-written tests

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

HTML Format

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations