research-article

Integrating Testing into Software Engineering Courses Supported by a Collaborative Learning Environment

Authors:

Peter J. Clarke,

Edward L. JonesAuthors Info & Claims

ACM Transactions on Computing Education (TOCE), Volume 14, Issue 3

Article No.: 18, Pages 1 - 33

https://doi.org/10.1145/2648787

Published: 27 October 2014 Publication History

Abstract

As software becomes more ubiquitous and complex, the cost of software bugs continues to grow at a staggering rate. To remedy this situation, there needs to be major improvement in the knowledge and application of software validation techniques. Although there are several software validation techniques, software testing continues to be one of the most widely used in industry. The high demand for software engineers in the next decade has resulted in more software engineering (SE) courses being offered in academic institutions. However, due to the number of topics to be covered in SE courses, little or no attention is given to software testing, resulting in students entering industry with little or no testing experience.

We propose a minimally disruptive approach of integrating software testing into SE courses by providing students access to a collaborative learning environment containing learning materials on testing techniques and testing tools. In this article, we describe the learning environment and the studies conducted to measure the benefits accrued by students using the learning environment in the SE courses.

References

[1]

ACM/IEEE-CS Interim Review Task Force. 2008. Computer Science Curriculum 2008: An Interim Revision of CS 2001. Retrieved August 7, 2014, from http://www.acm.org/education/curricula/ComputerScience 2008.pdf.

[2]

Paul Ammann and Jeff Offutt. 2008. Introduction to Software Testing. Cambridge University Press, New York, NY.

Digital Library

[3]

Tara Astigarraga, Eli M. Dow, Christina Lara, Richard Prewitt, and Maria R. Ward. 2010. The emerging role of software testing in curricula. In Transforming Engineering Education: Creating Interdisciplinary Skills for Complex Global Environments, 2010 IEEE. IEEE, Los Alamitos, CA, 1--26.

[4]

Ellen Francine Barbosa, José Carlos Maldonado, Richard LeBlanc, and Mark Guzdial. 2003. Introducing testing practices into objects and design course. In Proceedings of the 16th Conference on Software Engineering Education and Training. IEEE, Los Alamitos, CA, 279--286.

Digital Library

[5]

Victor R. Basili, Richard W. Selby, and David H. Hutchens. 1986. Experimentation in software engineering. IEEE Transactions on Software Engineering SE-12, 7, 733--743.

Digital Library

[6]

Sebastian Bergmann. 2013. PHPUnit. Retrieved August 7, 2014, from https://github.com/sebastianbergmann/phpunit/.

[7]

Robert V. Binder. 1999. Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison Wesley Longman, Boston, MA.

Digital Library

[8]

Pierre Bourque and Robert Dupuis. 2004. Guide to the Software Engineering Body of Knowledge 2004 Version. IEEE, Los Alamitos, CA.

[9]

Bernd Bruegge and Allen H. Dutoit. 2009. Object-Oriented Software Engineering Using UML, Patterns, and Java (3rd ed.). Prentice Hall, Upper Saddle River, NJ.

Digital Library

[10]

Lilian Cassel, Lois Delcambre, Edward Fox, and Richard Furuta. 2012. Ensemble: Computing Portal Connecting Computing Educators. Retrieved August 7, 2014, from http://www.computingportal.org/.

[11]

Yaofei Chen, Rose Dios, Ali Mili, Lan Wu, and Kefei Wang. 2005. An empirical study of programming language trends. IEEE Software 22, 3, 72--78.

Digital Library

[12]

CITIDEL Team. 2010. CITIDEL: Computing and Information Technology Interactive Digital Educational Library. Retrieved August 7, 2014, from http://www.citidel.org/.

[13]

Mike Clark. 2012. JDepend. Retrieved August 7, 2014, from http://clarkware.com/software/JDepend.html.

[14]

Peter J. Clarke, Andrew A. Allen, Tariq M. King, Edward L. Jones, and Prathiba Natesan. 2010. Using a Web-based repository to integrate testing tools into programming courses. In Proceedings of the ACM OOPSLA 2010 Companion (SPLASH’10). ACM, New York, NY, 193--200.

Digital Library

[15]

Peter J. Clarke, Jairo Pava, Debra Davis, and Tariq M. King. 2012. Using WReSTT in SE courses: An empirical study. In Proceedings of the 43rd SIGCSE Conference. ACM, New York, NY, 307--312.

Digital Library

[16]

Peter J. Clarke, Jairo Pava, Yali Wu, and Tariq M. King. 2011. Collaborative Web-based learning of testing tools in SE courses. In Proceedings of the 42nd SIGCSE Conference. ACM, New York, NY, 147--152.

Digital Library

[17]

CNN. 2012. Is Knight’s &dollar;440 million glitch the costliest computer bug ever&quest; (2012). CNN Money, August 9. Retrieved August 7, 2014, from http://money.cnn.com/2012/08/09/technology/knight-expensive- computer-bug/index.html.

[18]

CNSS. 2005. Software 2015: A National Software Strategy to Ensure U.S. Security and Competitiveness. Technical Report. Center for National Software Studies, Upper Marlboro, MD.

[19]

Cobertura Team. 2012. Cobertura. Retrieved August 7, 2014, from http://cobertura.sourceforge.net/.

[20]

CoverStory Team. 2013. CoverStory. Retrieved August 7, 2014, from http://code.google.com/p/coverstory/.

[21]

Tony Cowling. 2012. Stages in teaching software testing. In Proceedings of the 2012 International Conference on Software Engineering (ICSE’12). IEEE, Los Alamitos, CA, 1185--1194. Available at http://dl.acm.org/citation.cfm&quest;id=2337223.2337379.

Digital Library

[22]

David C. Crowther and Peter J. Clarke. 2005. Examining software testing tools. Dr. Dobbs Journal 373, 1, 26--33.

[23]

Chetan Desai, David S. Janzen, and John Clements. 2009. Implications of integrating test-driven development into CS1/CS2 curricula. ACM SIGCSE Bulletin 41, 1, 148--152.

Digital Library

[24]

Adrián Domínguez, Joseba Saenz de Navarrete, Luis de Marcos, Luis Fernández-Sanz, Carmen Pagés, and José-Javier Martínez-Herráiz. 2013. Gamifying learning experiences: Practical implications and outcomes. Computers and Education 63, 380--392.

Digital Library

[25]

Drupal Community. 2012. Drupal. Retrieved August 7, 2014, from http://drupal.org/.

[26]

Thomas Dvornik, David S. Janzen, John Clements, and Olga Dekhtyar. 2011. Supporting introductory test-driven labs with WebIDE. In Proceedings of the 24th IEEE-CS Software Engineering Education and Training Conference (CSEET’11). IEEE, Los Alamitos, CA, 51--60.

Digital Library

[27]

Stephen H. Edwards. 2003. Rethinking computer science education from a test-first perspective. In Companion of the 18th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA’03). ACM, New York, NY, 148--155.

Digital Library

[28]

Facebook Team. 2012. Facebook. Retrieved August 7, 2014, from http://www.facebook.com/.

[29]

Michael Feathers. 2012. CppUnit. Retrieved August 7, 2014, from http://sourceforge.net/projects/cppunit/.

[30]

Stephen Frezza. 2002. Integrating testing and design methods for undergraduates: Teaching software testing in the context of software design. In Proceedings of the 32nd Annual Frontiers in Education, 2002 (FIE’02), Vol 3. IEEE, Los Alamitos, CA, SIG-1--SIG-4.

[31]

Michael P. Gallaher and Brian M. Kropp. 2011. The Economic Impacts of Inadequate Infrastructure for Software Testing. Retrieved August 7, 2014, from http://www.rti.org/pubs/software_testing.pdf.

[32]

Erich Gamma and Kent Beck. 2012. JUnit. Retrieved August 7, 2014, from http://www.junit.org/.

[33]

Vahid Garousi. 2010. An open modern software testing laboratory courseware—an experience report. In Proceedings of the 23rd IEEE-CS Software Engineering Education and Training Conference (CSEET’10). IEEE, Los Alamitos, CA, 177--184.

Digital Library

[34]

Michael H. Goldwasser. 2002. A gimmick to integrate software testing throughout the curriculum. In Proceedings of the 33rd SIGCSE Conference. ACM, New York, NY, 271--275.

Digital Library

[35]

Starr R. Hiltz. 1998. Collaborative learning in asynchronous learning networks: Building learning communities. (1998). In Proceedings of WEB98.

[36]

Marc R. Hoffmann. 2012. EclEmma. Retrieved August 8, 2014, from http://www.eclemma.org/.

[37]

IBM. 2012. Rational Functional Tester. Retrieved August 7, 2014, from http://www-01.ibm.com/software/awdtools/tester/functional/.

[38]

David S. Janzen and Hossein Saiedian. 2006. Test-driven learning: Intrinsic integration of testing into the CS/SE curriculum. ACM SIGCSE Bulletin 38, 1, 254--258.

Digital Library

[39]

Edward L. Jones. 2000. Software testing in the computer science curriculum—a holistic approach. In Proceedings of the Australasian Conference on Computing Education (ACSE’00). ACM, New York, NY, 153--157.

Digital Library

[40]

Cem Kaner, James Bach, and Bret Pettichord. 2001. Lessons Learned in Software Testing. John Wiley & Sons, New York, NY.

Digital Library

[41]

Timothy C. Lethbridge, Jorge Diaz-Herrera, Richard J. LeBlanc Jr., and J. Barrie Thompson. 2007. Improving software practice through education: Challenges and future trends. In Proceedings of the 2007 Future of Software Engineering (FOSE’07). IEEE, Los Alamitos, CA, 12--28.

Digital Library

[42]

Cen Li, Zhijiang Dong, Roland H. Untch, and Michael Chasteen. 2013. Engaging computer science students through gamification in an online social network based collaborative learning environment. International Journal of Information and Education Technology 3, 1, 72--77.

[43]

Qing Li, Rynson W. H. Lau, Timothy K. Shih, and Frederick W. B. Li. 2008. Technology supports for distributed and collaborative learning over the Internet. Transactions on Internet Technology 8, 2, Article No. 5.

Digital Library

[44]

Michael J. Lutz, W. Michael McCracken, Susan Mengel, Mark Sebern, Greg W. Hislop, and Thomas B. Hilburn. 2010. SWENET—SEEK Category: Software Verification and Validation (VAV). Retrieved August 7, 2014, from http://www.swenet.org/browseModules.aspx&quest;categoryID=11.

[45]

Thomas W. Malone. 1980. What makes things fun to learn&quest; Heuristics for designing instructional computer games. In Proceedings of the 3rd ACM SIGSMALL Symposium and the 1st SIGPC Symposium on Small Systems (SIGSMALL’80). ACM, New York, NY, 162--169.

Digital Library

[46]

Aditya P. Mathur. 2008. Foundations of Software Testing. Pearson Education, Delhi, India.

Digital Library

[47]

Microsoft Corporation. 2013. Using Testing Tools in Visual Studio Professional Edition. Retrieved August 7, 2014, from http://msdn.microsoft.com/en-us/library/bb385902(v=VS.90).aspx.

[48]

Glenford J. Myers. 2004. Art of Software Testing (2nd ed.). John Wiley & Sons, New York, NY.

Digital Library

[49]

John Neter, William Wasserman, and Michael H. Kutner. 1990. Applied Linear Statistical Models. Irwin Press, Boston, MA.

[50]

William Perry. 2006. Effective Methods for Software Testing (3rd ed.). John Wiley & Sons, New York, NY.

Digital Library

[51]

Shari Lawrence Pfleeger and Joanne M. Atlee. 2009. Software Engineering: Theory and Practice (4th ed.). Pearson Education, Cranbury Township, NJ.

[52]

Alex Ramírez and Geoffrey C. Fox. 2011. A Report of the National Science Foundation Advisory Committee for Cyberinfrastructure Task Force on Cyberlearning and Workforce Development. Retrieved August 7, 2014, from http://www.nsf.gov/cise/aci/taskforces/TaskForceReport_Learning.pdf.

[53]

Sen&colon;te. 2013. Moving OCUnit tests between Logic and Application tests. Retrieved August 7, 2014, from http://www.sente.ch/&quest;p=276&lang=en.

[54]

Terry Shepard, Margaret Lamb, and Diane Kelly. 2001. More testing should be taught. Communications of the ACM 44, 6, 103--108.

Digital Library

[55]

Charlie Y. Shim, Mina Choi, and Jung Yeop Kim. 2009. Promoting collaborative learning in software engineering by adapting the PBL strategy. In Proceedings of the WASET International Conference on Computer and Information Technology (ICCIT ’09). IEEELos Alamitos, CA, 1167--1170.

[56]

Sidney Siegel and N. John Castellan. 1988. Nonparametric Statistics for the Behavioral Sciences (2nd ed.). McGraw-Hill, New York, NY.

[57]

Barbara Leigh Smith and Jean T. MacGregor. 1992. What is collaborative learning&quest; In Collaborative Learning: A Sourcebook for Higher Education, A. S. Goodsell, M. R. Maher, and V. Tinto (Eds.). National Center on Postsecondary Teaching, Learning, and Assessment, University Park, PA.

[58]

Software Quality Engineering Research Group (SoftQual). 2012. SoftQual Repository of Software Testing Laboratory Courseware. Retrieved August 7, 2014, from http://www.softqual.ucalgary.ca/projects/testing_labs/.

[59]

Ian Sommerville. 2004. Software Engineering (7th ed.). Pearson Addison Wesley, Boston, MA.

Digital Library

[60]

Bénédicte Talon, Dominique Leclet, Arnaud Lewandowski, and Grégory Bourguin. 2009. Learning software testing using a collaborative activities oriented platform. In Proceedings of the 2009 9th IEEE International Conference on Advanced Learning Technologies (ICALT’09). IEEE, Los Alamitos, CA, 443--445.

Digital Library

[61]

William Trochim. 2001. The Research Methods Knowledge Base (2nd ed.). Atomic Dog Publishing, Cincinnati, OH.

[62]

Tom Tullis and Bill Albert. 2008. Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Elsevier: Morgan Kaufmann, San Francisco, CA.

Digital Library

[63]

Ultimate Software. 2012. Simple Web Automation Toolkit. Retrieved August 7, 2014, from http://sourceforge.net/projects/ulti-swat/.

[64]

Wikipedia. 2012. List of Software Bugs. Retrieved August 7, 2014, from http://en.wikipedia.org/wiki/List_of_software_bugs.

[65]

Laurie Williams. 2010. Software Engineering: Testing. OpenSeminar. Retrieved August 7, 2014, from http://openseminar.org/se/modules/7/index/screen.do.

[66]

Claes Wohlin, Per Runeson, Martin Höst, Magnus C. Ohlsson, Bjöorn Regnell, and Anders Wesslén. 2000. Experimentation in Software Engineering: An Introduction. Kluwer Academic Publishers, Norwell, MA.

Digital Library

[67]

WReSTT Team. 2012. WReSTT: Web-based Repository of Software Testing Tools. Retrieved August 7, 2014, from http://wrestt.cis.fiu.edu/.

Cited By

Cammaerts FTramontana PPaiva AFlores NPastor Ricós FSnoeck M(2024)Exploring students' opinion on software testing coursesProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering10.1145/3661167.3661276(570-579)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3661167.3661276
Pawagi MKumar V(2024)Probeable Problems for Beginner-level Programming-with-AI ContestsProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671108(166-176)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3632620.3671108
Garcia RTreude CValentine AStephenson BStone JBattestilli LRebelsky SShoop L(2024)Application of Collaborative Learning Paradigms within Software Engineering Education: A Systematic Mapping StudyProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630780(366-372)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630780
Show More Cited By

Index Terms

Integrating Testing into Software Engineering Courses Supported by a Collaborative Learning Environment
1. Applied computing
  1. Education
    1. Collaborative learning

Recommendations

Impact of Using Tools in an Undergraduate Software Testing Course Supported by WReSTT

Software continues to affect a major part of our daily lives, including the way we use our phones, home appliances, medical devices, and cars. The pervasiveness of software has led to a growing demand for software developers over the next decade. To ...
Collaborative web-based learning of testing tools in SE courses
SIGCSE '11: Proceedings of the 42nd ACM technical symposium on Computer science education

One of the main concerns in the software industry continues to be the development of high quality software. This concern will be exacerbated as software systems become more complex. The training of software developers continues to grow in academia since ...
Teaching Testing with Modern Technology Stacks in Undergraduate Software Engineering Courses
ITiCSE '21: Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1

Students' experience with software testing in undergraduate computing courses is often relatively shallow, as compared to the importance of the topic. This experience report describes introducing industrial-strength testing into CMPSC 156, an upper ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computing Education

ACM Transactions on Computing Education Volume 14, Issue 3

November 2014

129 pages

EISSN:1946-6226

DOI:10.1145/2668970

Editors:
Robert McCartney
University of Connecticut, USA
,
Josh Tenenberg
University of Washington-Tacoma, USA

Issue’s Table of Contents

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2014

Accepted: 01 April 2014

Revised: 01 April 2014

Received: 01 December 2012

Published in TOCE Volume 14, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Division of Undergraduate Education

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
659
Total Downloads

Downloads (Last 12 months)60
Downloads (Last 6 weeks)8

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cammaerts FTramontana PPaiva AFlores NPastor Ricós FSnoeck M(2024)Exploring students' opinion on software testing coursesProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering10.1145/3661167.3661276(570-579)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3661167.3661276
Pawagi MKumar V(2024)Probeable Problems for Beginner-level Programming-with-AI ContestsProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671108(166-176)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3632620.3671108
Garcia RTreude CValentine AStephenson BStone JBattestilli LRebelsky SShoop L(2024)Application of Collaborative Learning Paradigms within Software Engineering Education: A Systematic Mapping StudyProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630780(366-372)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630780
Shin AKazerouni A(2023)A Model of How Students Engineer Test Cases With FeedbackACM Transactions on Computing Education10.1145/362860424:1(1-31)Online publication date: 20-Oct-2023
https://dl.acm.org/doi/10.1145/3628604
Fulcini TCoppola RArdito LTorchiano M(2023)A Review on Tools, Mechanics, Benefits, and Challenges of Gamified Software TestingACM Computing Surveys10.1145/358227355:14s(1-37)Online publication date: 10-Feb-2023
https://dl.acm.org/doi/10.1145/3582273
(2023)FLOD: Full-Lifecycle Online Judge For Accompanying Programming Teaching2023 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE)10.1109/TALE56641.2023.10398364(1-8)Online publication date: 28-Nov-2023
https://doi.org/10.1109/TALE56641.2023.10398364
Zivkovic TDraskovic DNikolic B(2023)Learning environments in software testing education: An overviewComputer Applications in Engineering Education10.1002/cae.2265731:6(1497-1521)Online publication date: 27-Jun-2023
https://doi.org/10.1002/cae.22657
Farooq MTehseen ROmer URiaz STahir S(2021)Software Testing Education: A Systematic Literature ReviewVFAST Transactions on Software Engineering10.21015/vtse.v9i4.9539:4(109-125)Online publication date: 31-Dec-2021
https://doi.org/10.21015/vtse.v9i4.953
Clarke PDavis DBuckley IPotvin GThirunarayanan MJones E(2021)Combining Learning and Engagement Strategies in a Software Testing Learning EnvironmentACM Transactions on Computing Education10.1145/346913122:2(1-25)Online publication date: 1-Nov-2021
https://dl.acm.org/doi/10.1145/3469131
Zivkovic TZivkovic M(2021)Survey of Learning Environments for Software Testing Education7th Conference on the Engineering of Computer Based Systems10.1145/3459960.3459971(1-9)Online publication date: 26-May-2021
https://dl.acm.org/doi/10.1145/3459960.3459971
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents