research-article

Teaching and training developer-testing techniques and tool support

Authors:

Jonathan de Halleux,

Nikolai Tillmann,

Wolfram SchulteAuthors Info & Claims

OOPSLA '10: Proceedings of the ACM international conference companion on Object oriented programming systems languages and applications companion

Pages 175 - 182

https://doi.org/10.1145/1869542.1869570

Published: 17 October 2010 Publication History

Abstract

Developer testing is a type of testing where developers test their code as they write it, as opposed to testing done by a separate quality assurance organization. Developer testing has been widely recognized as an important and valuable means of improving software reliability, as it exposes faults early in the software development life cycle. Effectively conducting developer testing requires both effective tool support by tools and developer-testing skills by developers. In this paper, we describe our experiences and lessons learned in teaching and training developer-testing techniques and tool support in both university and industrial settings. We highlight differences in teaching and training in these two settings, and observations from interacting with practitioners in our process of teaching and training.

References

[1]

}}JUnit. http://www.junit.org.

[2]

}}Microsoft Research Code Contracts. http://research.microsoft.com/en-us/projects/contracts.

[3]

}}Microsoft Research Pex. http://research.microsoft.com/Pex.

[4]

}}NUnit. http://www.nunit.org/.

[5]

}}Pex for fun. http://www.pexforfun.com/.

[6]

}}P. Ammann and J. Offutt. Introduction to Software Testing. Cambridge University Press, 2008. http://www.introsoftwaretesting.com/.

Digital Library

[7]

}}K. Beck. Test Driven Development: By Example. Addison-Wesley, 2003.

Digital Library

[8]

}}J. de Halleux and N. Tillmann. Moles: tool-assisted environment isolation with closures. In Proc. TOOLS, pages 253--270, 2010.

Digital Library

[9]

}}L. M. de Moura and N. Bjørner. Z3: An efficient SMT solver. In Proc. TACAS, pages 337--340, 2008.

Digital Library

[10]

}}M. Fowler. Refactoring: Improving the Design of Existing Code. Addison Wesley, 1999.

Digital Library

[11]

}}P. Godefroid, N. Klarlund, and K. Sen. DART: Directed automated random testing. In Proc. PLDI, pages 213--223, 2005.

Digital Library

[12]

}}J. V. Guttag and J. J. Horning. The algebraic specification of abstract data types. Acta Informatica, 10:27--52, 1978.

Digital Library

[13]

}}C. A. R. Hoare. An axiomatic basis for computer programming. Commun. ACM, 12(10):576--580, 1969.

Digital Library

[14]

}}J. Jacky, M. Veanes, C. Campbell, and W. Schulte. Modelbased Software Testing and Analysis with C#. Cambridge University Press, 2008.

Digital Library

[15]

}}J. C. King. Symbolic execution and program testing. Commun. ACM, 19(7):385--394, 1976.

Digital Library

[16]

}}M. R. Marri, S. Thummalapenta, T. Xie, N. Tillmann, and J. de Halleux. Retrofitting unit tests for parameterized unit testing. Technical Report TR-2010-9, North Carolina State University Department of Computer Science, Raleigh, NC, March 2010.

[17]

}}M. R. Marri, T. Xie, N. Tillmann, J. de Halleux, and W. Schulte. An empirical study of testing file-systemdependent software with mock objects. In Proc. AST, Business and Industry Case Studies, pages 149--153, 2009.

[18]

}}B. Meyer. Object-Oriented Software Construction. Prentice Hall, 1988.

Digital Library

[19]

}}R. Pandita, T. Xie, N. Tillmann, and J. de Halleux. Guided test generation for coverage criteria. In Proc. ICSM, 2010.

Digital Library

[20]

}}K. Sen, D. Marinov, and G. Agha. CUTE: A concolic unit testing engine for C. In Proc. ESEC/FSE, pages 263--272, 2005.

Digital Library

[21]

}}N. Tillmann and J. de Halleux. Pex - white box test generation for .NET. In Proc. TAP, pages 134--153, 2008.

Digital Library

[22]

}}N. Tillmann andW. Schulte. Parameterized unit tests. In Proc. ESEC/FSE, pages 253--262, 2005.

Digital Library

[23]

}}N. Tillmann and W. Schulte. Mock-object generation with behavior. In Proc. ASE, pages 365--368, 2006.

Digital Library

[24]

}}G. Venolia, R. DeLine, and T. LaToza. Software development at microsoft observed. Technical Report MSR-TR-2005-140, Microsoft Research, Redmond, WA, October 2005.

[25]

}}X. Xiao, T. Xie, N. Tillmann, and J. de Halleux. Issue analysis for residual structural coverage in dynamic symbolic execution. Technical Report TR-2010-7, North Carolina State University Department of Computer Science, Raleigh, NC,March 2010.

[26]

}}L. Zhang, T. Xie, L. Zhang, N. Tillmann, J. de Halleux, and H. Mei. Test generation via dynamic symbolic execution for mutation testing. In Proc. ICSM, 2010.

Digital Library

Cited By

Ivanov DBabushkin AGrigoryev SIatchenii PKalugin VKichin EKulikov EMisonizhnik AMordvinov DMorozov SNaumenko OPleshakov APonomarev PShmidt SUtkin AVolodin VVolynets AGrundy J(2023)UnitTestBot: Automated Unit Test Generation for C Code in Integrated Development EnvironmentsProceedings of the 45th International Conference on Software Engineering: Companion Proceedings10.1109/ICSE-Companion58688.2023.00107(380-384)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE-Companion58688.2023.00107
Swacha J(2017)Scripting Environments of Gamified Learning Management Systems for Programming EducationGamification-Based E-Learning Strategies for Computer Programming Education10.4018/978-1-5225-1034-5.ch013(278-294)Online publication date: 2017
https://doi.org/10.4018/978-1-5225-1034-5.ch013
Xie TTillmann NLakshman PDillon LVisser WWilliams L(2016)Advances in unit testingProceedings of the 38th International Conference on Software Engineering Companion10.1145/2889160.2891056(904-905)Online publication date: 14-May-2016
https://dl.acm.org/doi/10.1145/2889160.2891056
Show More Cited By

Index Terms

Teaching and training developer-testing techniques and tool support

Recommendations

Parameterized unit testing: theory and practice
ICSE '10: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2

Unit testing has been widely recognized as an important and valuable means of improving software reliability, as it exposes bugs early in the software development life cycle. However, manual unit testing is often tedious and insufficient. Testing tools ...
Guest Editors' Introduction: Software Testing Practices in Industry

Four papers and a roundtable discussion shed light on the current state of software testing practices. Case studies from industry experience address topics including unit-testing practices, agile testing, and automating software testing. Although many ...
Tool Support for Change-Centric Test Development

Current development tools for unit testing help automate test execution, report results, and generate test stubs, but they offer no aid in designing tests aimed specifically at the effects of changes to a program. This article describes a method for ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

OOPSLA '10: Proceedings of the ACM international conference companion on Object oriented programming systems languages and applications companion

October 2010

352 pages

ISBN:9781450302401

DOI:10.1145/1869542

General Chairs:
William R. Cook
University of Texas at Austin
,
Siobhán Clarke
Trinity College Dublin
,
Program Chair:
Martin Rinard
MIT

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SPLASH '10

Sponsor:

SIGPLAN

SPLASH '10: Systems Programming Languages and Applications: Software for Humanity

October 17 - 21, 2010

Nevada, Reno/Tahoe, USA

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
242
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)1

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ivanov DBabushkin AGrigoryev SIatchenii PKalugin VKichin EKulikov EMisonizhnik AMordvinov DMorozov SNaumenko OPleshakov APonomarev PShmidt SUtkin AVolodin VVolynets AGrundy J(2023)UnitTestBot: Automated Unit Test Generation for C Code in Integrated Development EnvironmentsProceedings of the 45th International Conference on Software Engineering: Companion Proceedings10.1109/ICSE-Companion58688.2023.00107(380-384)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE-Companion58688.2023.00107
Swacha J(2017)Scripting Environments of Gamified Learning Management Systems for Programming EducationGamification-Based E-Learning Strategies for Computer Programming Education10.4018/978-1-5225-1034-5.ch013(278-294)Online publication date: 2017
https://doi.org/10.4018/978-1-5225-1034-5.ch013
Xie TTillmann NLakshman PDillon LVisser WWilliams L(2016)Advances in unit testingProceedings of the 38th International Conference on Software Engineering Companion10.1145/2889160.2891056(904-905)Online publication date: 14-May-2016
https://dl.acm.org/doi/10.1145/2889160.2891056
Beller MGousios GZaidman ABertolino ACanfora GElbaum S(2015)How (much) do developers test?Proceedings of the 37th International Conference on Software Engineering - Volume 210.5555/2819009.2819101(559-562)Online publication date: 16-May-2015
https://dl.acm.org/doi/10.5555/2819009.2819101
Beller MGousios GZaidman A(2015)How (Much) Do Developers Test?2015 IEEE/ACM 37th IEEE International Conference on Software Engineering10.1109/ICSE.2015.193(559-562)Online publication date: May-2015
https://doi.org/10.1109/ICSE.2015.193
Tillmann Nde Halleux JXie TCrnkovic IChechik MGrünbacher P(2014)Transferring an automated test generation tool to practiceProceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering10.1145/2642937.2642941(385-396)Online publication date: 15-Sep-2014
https://dl.acm.org/doi/10.1145/2642937.2642941
Tillmann Nde Halleux JXie TBishop JSahami MFox AHearst MChi M(2014)Code huntProceedings of the first ACM conference on Learning @ scale conference10.1145/2556325.2567870(221-222)Online publication date: 4-Mar-2014
https://dl.acm.org/doi/10.1145/2556325.2567870
Xie TTillmann Nde Halleux JCooper KScacchi WWang A(2013)Educational software engineeringProceedings of the 3rd International Workshop on Games and Software Engineering: Engineering Computer Games to Enable Positive, Progressive Change10.5555/2662593.2662600(36-39)Online publication date: 18-May-2013
https://dl.acm.org/doi/10.5555/2662593.2662600
Tillmann NDe Halleux JXie TGulwani SBishop JNotkin DCheng BPohl K(2013)Teaching and learning programming and software engineering via interactive gamingProceedings of the 2013 International Conference on Software Engineering10.5555/2486788.2486941(1117-1126)Online publication date: 18-May-2013
https://dl.acm.org/doi/10.5555/2486788.2486941
Tillmann Nde Halleux JXie TGulwani SBishop J(2013)Teaching and learning programming and software engineering via interactive gaming2013 35th International Conference on Software Engineering (ICSE)10.1109/ICSE.2013.6606662(1117-1126)Online publication date: May-2013
https://doi.org/10.1109/ICSE.2013.6606662
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents