Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3092703.3098222acmconferencesArticle/Chapter ViewAbstractPublication PagesisstaConference Proceedingsconference-collections
short-paper

CUT: automatic unit testing in the cloud

Published: 10 July 2017 Publication History

Abstract

Unit tests can be significantly sped up by running them in parallel over distributed execution environments, such as the cloud. However, manually setting up such environments and configuring the testing frameworks to effectively use them is cumbersome and requires specialized expertise that developers might lack.
We present Cloud Unit Testing (CUT), a tool for automatically executing unit tests in distributed execution environments. Given a set of unit tests, CUT allocates appropriate computational resources, i.e., virtual machines or containers, and schedules the execution of tests over them. Developers do not need to change existing unit test code, and can easily control relevant aspects of test execution, including resource allocation and test scheduling. Additionally, during the execution CUT monitors and publishes events about the running tests which enables stream analytics.
CUT and videos showcasing its main features are freely available at: https://www.st.cs.uni-saarland.de/testing/cut/

References

[1]
Apache. 2016. Apache ActiveMQ homepage. http://activemq.apache.org/. (May 2016).
[2]
Apache. 2016. Maven Homepage. http://maven.apache.org/. (Jan 2016).
[3]
Linda Di Geronimo, Filomena Ferrucci, Alfonso Murolo, and Federica Sarro. 2012. A Parallel Genetic Algorithm Based on Hadoop MapReduce for the Automatic Generation of JUnit Test Suites. In Proceedings of the International Conference on Software Testing, Verification and Validation (ICST ’12). 785–793.
[4]
Milos Gligoric, Lamyaa Eloussi, and Darko Marinov. 2015. Practical Regression Test Selection with Dynamic File Dependencies. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA 15). 211–222.
[5]
Georgios Gousios, Dominik Safaric, and Joost Visser. 2016. Streaming Software Analytics. In Proceedings of the 2Nd International Workshop on BIG Data Software Engineering (BIGDSE ’16). 8–11.
[6]
Alex Gyori, August Shi, Farah Hariri, and Darko Marinov. 2015. Reliable Testing: Detecting State-polluting Tests to Prevent Test Dependency. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA 15). 223–233.
[7]
Docker Inc. 2017. Docker Homepage. http://www.docker.com/. (Feb 2017).
[8]
RightScale Inc. 2017. Dynamic scaling Jenkins in the cloud. http://www.rightscale. com/blog/cloud-management-best-practices/dynamic-scaling-jenkins-cloud. (May 2017).
[9]
K. Incki, I. Ari, and H. Sozer. 2012. A Survey of Software Testing in the Cloud. In International Conference on Software Security and Reliability Companion (SERE-C 12). 18–23.
[10]
JUnit. 2016. JUnit homepage. http://www.junit.org. (May 2016).
[11]
Sebastian Kappler. 2016. Finding and Breaking Test Dependencies to Speed Up Test Execution. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE 2016). 1136–1138.
[12]
Shadi Abdul Khalek and Sarfraz Khurshid. 2011. Efficiently Running Test Suites Using Abstract Undo Operations. In Proceedings of the International Symposium on Software Reliability Engineering (ISSRE 11). 110–119.
[13]
Wing Lam, Sai Zhang, and Michael D. Ernst. 2015. When tests collide: Evaluating and coping with the impact of test dependence. Technical Report UW-CSE-15-03-01. University of Washington.
[14]
P. Leitner, Z. Rostyslav, A. Gambi, and S. Dustdar. 2013. A Framework and Middleware for Application-Level Autonomic Cloud Bursting on Top of Infrastructureas-a-Service Clouds. In Proceedings of the International Conference on Utility and Cloud Computing (UCC 13).
[15]
Manuel Oriol and Faheem Ullah. 2010. YETI on the Cloud. In Proceedings of the 2010 Third International Conference on Software Testing, Verification, and Validation Workshops (ICSTW ’10). 434–437.
[16]
Tauhida Parveen and Scott Tilley. 2010. When to Migrate Software Testing to the Cloud?. In Proceedings of the International Conference on Software Testing, Verification, and Validation Workshops (ICSTW ’10). 424–427.
[17]
August Shi, Alex Gyori, Milos Gligoric, Andrey Zaytsev, and Darko Marinov. 2014. Balancing Trade-offs in Test-suite Reduction. In Proceedings of the International Symposium on Foundations of Software Engineering (FSE 14). 246–256.
[18]
Scott Tilley and Tauhida Parveen. 2012. HadoopUnit: Test Execution in the Cloud. In Software Testing in the Cloud: Migration and Execution. Springer Publishing Company, Incorporated, Chapter 3.
[19]
Rostyslav Zabolotnyi, Philipp Leitner, Waldemar Hummer, and Schahram Dustdar. 2015. JCloudScale: Closing the Gap Between IaaS and PaaS. ACM Transactions on Internet Technology 15, 3 (July 2015), 10:1–10:20.
[20]
Sai Zhang, Darioush Jalali, Jochen Wuttke, Kivanc Muslu, Wing Lam, Michael D. Ernst, and David Notkin. 2014. Empirically Revisiting the Test Independence Assumption. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA 14). 385–396. Abstract 1 The Need for Speed 2 CUT Overview 2.1 CUT Components and Workflow 2.2 CUT Implementation 3 Experimental Evaluation 3.1 Parallelization and Distribution 3.2 Test Clustering and Scheduling 4 Related Work 5 Conclusions and Future Work References

Cited By

View all
  • (2023)Reducing branch divergence to speed up parallel execution of unit testing on GPUsThe Journal of Supercomputing10.1007/s11227-023-05375-079:16(18340-18374)Online publication date: 13-May-2023
  • (2022)A Framework for Automated API Fuzzing at Enterprise Scale2022 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST53961.2022.00018(377-388)Online publication date: Apr-2022
  • (2022)Analyzing the interactions among factors affecting cloud adoption for software testing: a two-stage ISM-ANN approachSoft Computing10.1007/s00500-022-07062-326:16(8047-8075)Online publication date: 25-Apr-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ISSTA 2017: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis
July 2017
447 pages
ISBN:9781450350761
DOI:10.1145/3092703
  • General Chair:
  • Tevfik Bultan,
  • Program Chair:
  • Koushik Sen
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 July 2017

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. automated unit testing
  2. test dependency
  3. test parallelization
  4. test scheduling
  5. testing in the cloud

Qualifiers

  • Short-paper

Conference

ISSTA '17
Sponsor:

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)16
  • Downloads (Last 6 weeks)3
Reflects downloads up to 03 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Reducing branch divergence to speed up parallel execution of unit testing on GPUsThe Journal of Supercomputing10.1007/s11227-023-05375-079:16(18340-18374)Online publication date: 13-May-2023
  • (2022)A Framework for Automated API Fuzzing at Enterprise Scale2022 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST53961.2022.00018(377-388)Online publication date: Apr-2022
  • (2022)Analyzing the interactions among factors affecting cloud adoption for software testing: a two-stage ISM-ANN approachSoft Computing10.1007/s00500-022-07062-326:16(8047-8075)Online publication date: 25-Apr-2022
  • (2021)When life gives you oranges: detecting and diagnosing intermittent job failures at MozillaProceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3468264.3473931(1381-1392)Online publication date: 20-Aug-2021
  • (2021)An Empirical Study of Parallelizing Test Execution Using CUDA Unified Memory and OpenMP GPU Offloading2021 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)10.1109/ICSTW52544.2021.00052(271-278)Online publication date: Apr-2021
  • (2021)Soundy Automated Parallelization of Test Execution2021 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME52107.2021.00034(309-319)Online publication date: Sep-2021
  • (2019)Assessing the state and improving the art of parallel testing for CProceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3293882.3330573(123-133)Online publication date: 10-Jul-2019
  • (2018)Cloud-Based Test Tools: A Brief Comparative ViewCybernetics and Information Technologies10.2478/cait-2018-004418:4(3-14)Online publication date: 14-Dec-2018
  • (2018)piCoq: parallel regression proving for large-scale verification projectsProceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3213846.3213877(344-355)Online publication date: 12-Jul-2018
  • (2018)Fine-grained test minimizationProceedings of the 40th International Conference on Software Engineering10.1145/3180155.3180203(210-221)Online publication date: 27-May-2018
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media