research-article

Open access

Gamification in Test-Driven Development Practice

Author:

Wei RenAuthors Info & Claims

Gamify 2023: Proceedings of the 2nd International Workshop on Gamification in Software Development, Verification, and Validation

Pages 38 - 46

https://doi.org/10.1145/3617553.3617889

Published: 04 December 2023 Publication History

Abstract

The challenge of effectively developing and sustaining high-performance professional development practices in software engineering education is one that must be addressed. Test-driven development (TDD), an example of a key professional practical activity, is strongly linked to these high-performance practices. To examine the effects of gamification - the use of game design elements in a non-game context - on motivating students to develop and sustain TDD practice, an experiment was conducted and utilized ordinary least squares (OLS) regression to analyze the data. This experiment showed that gamification motivates students to do high-performing TDD practice. More specifically, gamification changes the individual's TDD behavior, increases engagement in the development activity, and the effect continues for a longer period even after gamification has ceased. Furthermore, a positive association between gamification and the maintainability of the team codebase was supported by the data.

References

[1]

Karim Al-Yafi and Mazen El-Masri. 2016. Gamification of e-government services: A discussion of potential transformation.

[2]

Manal M Alhammad and Ana M Moreno. 2018. Gamification in software engineering education: A systematic mapping. Journal of Systems and Software, 141 (2018), 131–150.

[3]

Luca Ardito, Riccardo Coppola, Luca Barbato, and Diego Verga. 2020. A Tool-Based Perspective on Software Code Maintainability Metrics: A Systematic Literature Review. Scientific Programming, 2020 (2020).

[4]

Sebastian Baltes, Jascha Knack, Daniel Anastasiou, Ralf Tymann, and Stephan Diehl. 2018. (No) influence of continuous integration on the commit activity in GitHub projects. In Proceedings of the 4th ACM SIGSOFT International Workshop on Software Analytics. 1–7.

Digital Library

[5]

Carlos Futino Barreto and César França. 2021. Gamification in Software Engineering: A literature Review. In 2021 IEEE/ACM 13th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE). 105–108.

[6]

Kent Beck. 2003. Test-driven development: by example. Addison-Wesley Professional.

Digital Library

[7]

Thirumalesh Bhat and Nachiappan Nagappan. 2006. Evaluating the efficacy of test-driven development: industrial case studies. In Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering. 356–363.

Digital Library

[8]

Wilson Bissi, Adolfo Gustavo Serra Seca Neto, and Maria Claudia Figueiredo Pereira Emer. 2016. The effects of test driven development on internal quality, external quality and productivity: A systematic review. Information and Software Technology, 74 (2016), 45–54.

Digital Library

[9]

Bruce E Blaine. 2018. Winsorizing. The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation, 1817.

[10]

John R Busenbark, Hyunjung Yoon, Daniel L Gamache, and Michael C Withers. 2021. Omitted Variable Bias: Examining Management Research With the Impact Threshold of a Confounding Variable (ITCV). Journal of Management, 01492063211006458.

[11]

Rachel C Callan, Kristina N Bauer, and Richard N Landers. 2015. How to avoid the dark side of gamification: Ten business scenarios and their unintended consequences. In Gamification in education and business. Springer, 553–568.

[12]

Celia Chen, Reem Alfayez, Kamonphop Srisopha, Barry Boehm, and Lin Shi. 2017. Why is it important to measure maintainability and what are the best ways to do it? In 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C). 377–378.

Digital Library

[13]

Scott H Costello. 1984. Software engineering under deadline pressure. ACM SIGSOFT Software Engineering Notes, 9, 5 (1984), 15–19.

Digital Library

[14]

Rodrigo Cursino, Daniel Ferreira, Maria Lencastre, Roberta Fagundes, and João Pimentel. 2018. Gamification in requirements engineering: a systematic review. In 2018 11th International Conference on the Quality of Information and Communications Technology (QUATIC). 119–125.

[15]

Daniel de Paula Porto, Gabriela Martins de Jesus, Fabiano Cutigi Ferrari, and Sandra Camargo Pinto Ferraz Fabbri. 2021. Initiatives and challenges of using gamification in software engineering: A Systematic Mapping. Journal of Systems and Software, 173 (2021), 110870.

[16]

Sebastian Deterding, Miguel Sicart, Lennart Nacke, Kenton O’Hara, and Dan Dixon. 2011. Gamification. using game-design elements in non-gaming contexts. In CHI’11 extended abstracts on human factors in computing systems. 2425–2428.

[17]

Daniel J Dubois and Giordano Tamburrelli. 2013. Understanding gamification mechanisms for software development. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. 659–662.

Digital Library

[18]

John Estdale and Elli Georgiadou. 2018. Applying the ISO/IEC 25010 quality models to software product. In European Conference on Software Process Improvement. 492–503.

[19]

Jon Eyolfson, Lin Tan, and Patrick Lam. 2011. Do time of day and developer experience affect commit bugginess? In Proceedings of the 8th Working Conference on Mining Software Repositories. 153–162.

Digital Library

[20]

Donald E Farrar and Robert R Glauber. 1967. Multicollinearity in regression analysis: the problem revisited. The Review of Economic and Statistics, 92–107.

[21]

Davide Fucci, Hakan Erdogmus, Burak Turhan, Markku Oivo, and Natalia Juristo. 2016. A dissection of the test-driven development process: Does it really matter to test-first or to test-last? IEEE Transactions on Software Engineering, 43, 7 (2016), 597–614.

Digital Library

[22]

Tommaso Fulcini, Riccardo Coppola, Luca Ardito, and Marco Torchiano. 2023. A Review on Tools, Mechanics, Benefits, and Challenges of Gamified Software Testing. Comput. Surveys.

[23]

Félix García, Oscar Pedreira, Mario Piattini, Ana Cerdeira-Pena, and Miguel Penabad. 2017. A framework for gamification in software engineering. Journal of Systems and Software, 132 (2017), 21–40.

Digital Library

[24]

Hassan Hajjdiab and Al Shaima Taleb. 2011. Adopting agile software development: issues and challenges. International Journal of Managing Value and Supply Chains (IJMVSC), 2, 3 (2011), 1–10.

[25]

Juho Hamari. 2013. Transforming homo economicus into homo ludens: A field experiment on gamification in a utilitarian peer-to-peer trading service. Electronic commerce research and applications, 12, 4 (2013), 236–245.

[26]

Juho Hamari, Jonna Koivisto, and Harri Sarsa. 2014. Does gamification work?–a literature review of empirical studies on gamification. In 2014 47th Hawaii international conference on system sciences. 3025–3034.

Digital Library

[27]

Larry V Hedges. 1994. Fixed effects models. The handbook of research synthesis, 285 (1994), 299.

[28]

J Dave Jobson. 2012. Applied multivariate data analysis: regression and experimental design. Springer Science & Business Media.

[29]

Jonna Koivisto and Juho Hamari. 2019. The rise of motivational information systems: A review of gamification research. International Journal of Information Management, 45 (2019), 191–210.

Digital Library

[30]

Kalimuthu Krishnamoorthy. 2016. Handbook of statistical distributions with applications. CRC Press.

[31]

Davy Landman, Alexander Serebrenik, Eric Bouwers, and Jurgen J Vinju. 2016. Empirical analysis of the relationship between CC and SLOC in a large corpus of Java methods and C functions. Journal of Software: Evolution and Process, 28, 7 (2016), 589–618.

Digital Library

[32]

Erin S Lane and Sara E Harris. 2015. A new tool for measuring student behavioral engagement in large university classes. Journal of College Science Teaching, 44, 6 (2015), 83–91.

[33]

Nikoletta-Zampeta Legaki, Nannan Xi, Juho Hamari, Kostas Karpouzis, and Vassilios Assimakopoulos. 2020. The effect of challenge-based gamification on learning: An experiment in the context of statistics education. International journal of human-computer studies, 144 (2020), 102496.

[34]

Ming Liu, Rafael A Calvo, Abelardo Pardo, and Andrew Martin. 2014. Measuring and visualizing students’ behavioral engagement in writing activities. IEEE Transactions on learning technologies, 8, 2 (2014), 215–224.

Digital Library

[35]

Jenni Majuri, Jonna Koivisto, and Juho Hamari. 2018. Gamification of education and learning: A review of empirical literature. In Proceedings of the 2nd international GamiFIN conference, GamiFIN 2018.

[36]

Richard VR Mariano, Geanderson E dos Santos, Markos V de Almeida, and Wladmir C Brandão. 2019. Feature changes in source code for commit classification into maintenance activities. In 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA). 515–518.

[37]

Rodrigo Henrique Barbosa Monteiro, Maurício Ronny de Almeida Souza, Sandro Ronaldo Bezerra Oliveira, Carlos dos Santos Portela, and Cesar Elias de Cristo Lobato. 2021. The Diversity of Gamification Evaluation in the Software Engineering Education and Industry: Trends, Comparisons and Gaps. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET). 154–164.

[38]

Paul Oman and Jack Hagemeister. 1992. Metrics for assessing a software system’s maintainability. In Proceedings Conference on Software Maintenance 1992. 337–338.

[39]

Fidelia A Orji, Julita Vassileva, and Jim Greer. 2021. Evaluating a Persuasive Intervention for Engagement in a Large University Class. International Journal of Artificial Intelligence in Education, 31, 4 (2021), 700–725.

[40]

Jan-Peter Ostberg and Stefan Wagner. 2014. On automatically collectable metrics for software maintainability evaluation. In 2014 Joint Conference of the International Workshop on Software Measurement and the International Conference on Software Process and Product Measurement. 32–37.

Digital Library

[41]

Anni Pakarinen, Heidi Parisod, Iiro Linden, Minna Elisabeth Aromaa, Jouni Smed, Ville Leppänen, and Sanna Salanterä. 2017. Usability of a gamified application to promote family wellbeing in child health clinics. In GamiFIN. 141–149.

[42]

Mataž Pancur, Mojca Ciglaric, M Trampus, and Tone Vidmar. 2003. Towards empirical evaluation of test-driven development in a university environment. In The IEEE Region 8 EUROCON 2003. Computer as a Tool. 2, 83–86.

[43]

Christian R Prause and Matthias Jarke. 2015. Gamification for enforcing coding conventions. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering. 649–660.

Digital Library

[44]

Christian R Prause, Jan Nonnen, and Mark Vinkovits. 2012. A Field Experiment on Gamification of Code Quality in Agile Development. In PPIG. 17.

[45]

Hamed Qahri-Saremi and Ofir Turel. 2016. School engagement, information technology use, and educational development: An empirical investigation of adolescents. Computers & Education, 102 (2016), 65–78.

Digital Library

[46]

Huilian Sophie Qiu, Alexander Nolte, Anita Brown, Alexander Serebrenik, and Bogdan Vasilescu. 2019. Going farther together: The impact of social capital on sustained participation in open source. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). 688–699.

Digital Library

[47]

B Ramachandra Reddy and Aparajita Ojha. 2019. Performance of Maintainability Index prediction models: a feature selection based study. Evolving Systems, 10, 2 (2019), 179–204.

[48]

Wei Ren and Stephen Barrett. 2023. Test-driven development, engagement in activity, and maintainability: An empirical study. IET Software, 17, 4 (2023), 509–525.

Digital Library

[49]

Wei Ren, Stephen Barrett, and Sourojit Das. 2020. Toward Gamification to Software Engineering and Contribution of Software Engineer. In Proceedings of the 2020 4th International Conference on Management Engineering, Software Engineering and Service Sciences. 1–5.

Digital Library

[50]

Valentin Riemer and Claudia Schrader. 2016. Impacts of behavioral engagement and self-monitoring on the development of mental models through serious games: Inferences from in-game measures. Computers in Human Behavior, 64 (2016), 264–273.

Digital Library

[51]

Adrian Santos, Sira Vegas, Oscar Dieste, Fernando Uyaguari, Ayşe Tosun, Davide Fucci, Burak Turhan, Giuseppe Scanniello, Simone Romano, and Itir Karac. 2021. A family of experiments on test-driven development. Empirical Software Engineering, 26, 3 (2021), 1–53.

Digital Library

[52]

Muhammad Imran Sarwar, Wasif Tanveer, Imran Sarwar, and Waqar Mahmood. 2008. A comparative study of mi tools: defining the roadmap to mi tools standardization. In 2008 IEEE international multitopic conference. 379–385.

[53]

Andrew Stellman and Jennifer Greene. 2005. Applied software project management. " O’Reilly Media, Inc.".

[54]

James H Stock and Mark W Watson. 2008. Heteroskedasticity-robust standard errors for fixed effects panel data regression. Econometrica, 76, 1 (2008), 155–174.

[55]

Ayse Tosun, Oscar Dieste, Davide Fucci, Sira Vegas, Burak Turhan, Hakan Erdogmus, Adrian Santos, Markku Oivo, Kimmo Toro, and Janne Jarvinen. 2017. An industry experiment on the effects of test-driven development on external quality and productivity. Empirical Software Engineering, 22, 6 (2017), 2763–2805.

Digital Library

[56]

SRP van Hal, Mathieu Post, and Kasper Wendel. 2019. Generating Commit Messages from Git Diffs. arXiv preprint arXiv:1911.11690.

[57]

Ming Wen, Rongxin Wu, and Shing-Chi Cheung. 2016. Locus: Locating bugs from software changes. In 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE). 262–273.

Digital Library

[58]

Claes Wohlin, Per Runeson, Martin Höst, Magnus C Ohlsson, Björn Regnell, and Anders Wesslén. 2012. Experimentation in software engineering. Springer Science & Business Media.

[59]

Lei Zhu, Ming Shan, and Bon-Gang Hwang. 2018. Overview of design for maintainability in building and construction research. Journal of Performance of Constructed Facilities, 32, 1 (2018), 04017116.

Cited By

Marcaida JTayag DAcebes RPerez CDizon RAmaro RViray RCabrera S(2024)Level up Learning: A Systematic Review of Gamification in Physical EducationJournal of Interdisciplinary Perspectives10.69569/jip.2024.04802:11Online publication date: 2024
https://doi.org/10.69569/jip.2024.0480
Laudadio LCoppola RTorchiano MTomic SCoppola RArdito LFraser GLeotta M(2024)ScoutDroid: A Tool For Mobile Augmented Testing with Live FeedbackProceedings of the 3rd ACM International Workshop on Gamification in Software Development, Verification, and Validation10.1145/3678869.3685688(34-37)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678869.3685688
Cerioli MLeotta MRibaudo MCoppola RArdito LFraser GLeotta M(2024)Enlisting Students in Gamifying Software Testing Education: Results and Key InsightsProceedings of the 3rd ACM International Workshop on Gamification in Software Development, Verification, and Validation10.1145/3678869.3685685(18-25)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678869.3685685
Show More Cited By

Index Terms

Gamification in Test-Driven Development Practice
1. Software and its engineering
  1. Software creation and management
    1. Software development process management
      1. Software development methods
        Agile software development

Recommendations

The effectiveness of test-driven development: an industrial case study

Test-driven development (TDD) is a software development practice, where test cases are incrementally written before implementing the production code. This paper presents the results of a multi-case study investigating the effectiveness of TDD within an ...
Test-Driven Development in scientific software: a survey

Scientific software developers are increasingly employing various software engineering practices. Specifically, scientists are beginning to use Test-Driven Development (TDD). Even with this increasing use of TDD, the effect of TDD on scientific software ...
Test-Driven Development: Concepts, Taxonomy, and Future Direction

Test-driven development creates software in very short iterations with minimal upfront design. Poised for widespread adoption TDD has become the focus of an increasing number of researchers and developers.

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

Gamify 2023: Proceedings of the 2nd International Workshop on Gamification in Software Development, Verification, and Validation

December 2023

51 pages

ISBN:9798400703737

DOI:10.1145/3617553

General Chairs:
Riccardo Coppola
Politecnico di Torino, Italy
,
Luca Ardito
Politecnico di Torino, Italy
,
Gordon Fraser
University of Passau, Germany
,
Maurizio Leotta
University of Genova, Italy

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution 4.0 International License.

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 December 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

Gamify '23

Sponsor:

SIGSOFT

Gamify '23: 2nd International Workshop on Gamification in Software Development, Verification, and Validation

December 4, 2023

CA, San Francisco, USA

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
306
Total Downloads

Downloads (Last 12 months)306
Downloads (Last 6 weeks)55

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Marcaida JTayag DAcebes RPerez CDizon RAmaro RViray RCabrera S(2024)Level up Learning: A Systematic Review of Gamification in Physical EducationJournal of Interdisciplinary Perspectives10.69569/jip.2024.04802:11Online publication date: 2024
https://doi.org/10.69569/jip.2024.0480
Laudadio LCoppola RTorchiano MTomic SCoppola RArdito LFraser GLeotta M(2024)ScoutDroid: A Tool For Mobile Augmented Testing with Live FeedbackProceedings of the 3rd ACM International Workshop on Gamification in Software Development, Verification, and Validation10.1145/3678869.3685688(34-37)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678869.3685688
Cerioli MLeotta MRibaudo MCoppola RArdito LFraser GLeotta M(2024)Enlisting Students in Gamifying Software Testing Education: Results and Key InsightsProceedings of the 3rd ACM International Workshop on Gamification in Software Development, Verification, and Validation10.1145/3678869.3685685(18-25)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678869.3685685
Clerissi DLeotta MRicca FCoppola RArdito LFraser GLeotta M(2024)A Gaming Quest to Improve Web Locators RobustnessProceedings of the 3rd ACM International Workshop on Gamification in Software Development, Verification, and Validation10.1145/3678869.3685684(10-17)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3678869.3685684
Coppola RArdito LLeotta M(2024)Gamify: Gamification in Software Development, Verification,and ValidationACM SIGSOFT Software Engineering Notes10.1145/3650142.365015149:2(27-30)Online publication date: 3-Apr-2024
https://dl.acm.org/doi/10.1145/3650142.3650151
Aldalur I(2024)Gamifying software engineering subject to enhance the quality of knowledgeSoftware: Practice and Experience10.1002/spe.333954:12(2299-2315)Online publication date: 16-May-2024
https://doi.org/10.1002/spe.3339

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents