Investigating the Effects of Tailored Gamification on Learners’ Engagement over Time in a Learning Environment
Article No.: 384, Pages 264 - 288
Abstract
Gamification has been widely used to increase learners’ motivation and engagement in digital learning environments. Various studies have highlighted the need to tailor gamification according to users’ characteristics. However, little is known about how tailoring gamification affects learners’ engagement when interacting with the environment. In this paper, we analyse learners’ behaviours in a large-scale field study in real-world classroom conditions over a six-week period. We identify three behavioural patterns and show at a global level that two of these patterns are influenced by adaptation. When we look at how learners’ engagement evolves over time, we see more differences in the adapted condition, specifically in the final lessons of the experiment. Globally learners’ engaged behaviours gradually decreased over time but tailoring the game elements to learners seemed to reduce this decrease or make it more stable, depending on the behavioural patterns.
References
[1]
Ashraf Abdul, Jo Vermeulen, Danding Wang, Brian Y. Lim, and Mohan Kankanhalli. 2018. Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda. Association for Computing Machinery, New York, NY, USA. 1–18. isbn:9781450356206
[2]
Maximilian Altmeyer, Marc Schubhan, Antonio Krüger, and Pascal Lessel. 2021. A Long-Term Investigation on the Effects of (Personalized) Gamification on Course Participation in a Gym. arxiv:2107.12597. arxiv:2107.12597
[3]
Gabriel Barata, Sandra Gama, Joaquim Jorge, and Daniel Gonçalves. 2017. Studying student differentiation in gamified education: A long-term study. Computers in Human Behavior, 71 (2017), 550–585.
[4]
Amy S Beavers, John W Lounsbury, Jennifer K Richards, Schuyler W Huck, Gary J Skolits, and Shelley L Esquivel. 2013. Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research, and Evaluation, 18, 1 (2013), 6.
[5]
Souha Bennani, Ahmed Maalel, and Henda Ben Ghezala. 2021. Adaptive gamification in E-learning: A literature review and future challenges. Computer Applications in Engineering Education, 30 (2021), Dec., https://doi.org/10.1002/cae.22477
[6]
Martin Böckle, Jasminko Novak, and Markus Bick. 2017. Towards adaptive gamification: a synthesis of current developments.
[7]
Michelle Briffa, Nevena Jaftha, Giuseppe Loreto, Flavia Cristina Morone Pinto, Tatjana Chircop, and Corradino Hill. 2020. Improved students’ performance within gamified learning environment: A meta-analysis study. International Journal of Education and Research, 8, 1 (2020), 223–244.
[8]
Charles Butler. 2014. A framework for evaluating the effectiveness of gamification techniques by personality type. In HCI in Business: First International Conference, HCIB 2014, Held as Part of HCI International 2014, Heraklion, Crete, Greece, June 22-27, 2014. Proceedings 1. 381–389.
[9]
David Codish and Gilad Ravid. 2015. Detecting playfulness in educational gamification through behavior patterns. IBM Journal of Research and Development, 59, 6 (2015), 6–1.
[10]
Anna B Costello and Jason Osborne. 2005. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical assessment, research, and evaluation, 10, 1 (2005), 7.
[11]
Luma da Rocha Seixas, Alex Sandro Gomes, and Ivanildo José de Melo Filho. 2016. Effectiveness of Gamification in the Engagement of Students. Computers in Human Behavior, 58 (2016), 48–63. issn:07475632
[12]
Edward Deci and Richard Ryan. 2000. The "what" and "why" of goal pursuits: Human needs and the self-determination of behavior. Psychological inquiry, 11, 4 (2000), 227–268.
[13]
Edward L. Deci and Richard M. Ryan. 1985. The General Causality Orientations Scale: Self-Determination in Personality. Journal of research in personality, 19, 2 (1985), 109–134.
[14]
Edward L. Deci and Richard M. Ryan. 2000. The "What" and "Why" of Goal Pursuits: Human Needs and the Self-Determination of Behavior. 11, 4 (2000), 227–268. issn:1047-840X https://doi.org/10.1207/S15327965PLI1104_01
[15]
Paul Denny, Fiona McDonald, Ruth Empson, Philip Kelly, and Andrew Petersen. 2018. Empirical Support for a Causal Relationship Between Gamification and Learning Outcomes. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, Montreal, QC, Canada, April 21-26, 2018, Regan L. Mandryk, Mark Hancock, Mark Perry, and Anna L. Cox (Eds.). ACM, 311. https://doi.org/10.1145/3173574.3173885
[16]
Sebastian Deterding. 2011. Situated Motivational Affordances of Game Elements: A Conceptual Model. In Gamification: Using Game Design Elements in Non-Gaming Contexts, a Workshop at CHI. 10.
[17]
Sebastian Deterding. 2015. The Lens of Intrinsic Skill Atoms: A Method for Gameful Design. 30, 3-4 (2015), 294–335.
[18]
Sebastian Deterding, Dan Dixon, Rilla Khaled, and Lennart Nacke. 2011. From Game Design Elements to Gamefulness: Defining Gamification. In Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments. ACM, 9–15. http://dl.acm.org/citation.cfm?id=2181040
[19]
Darina Dicheva, Christo Dichev, Gennady Agre, and Galia Angelova. 2015. Gamification in Education: A Systematic Mapping Study. Educational Technology & Society, 18 (2015), July, 75–88.
[20]
Lu Ding, Erkan Er, and Michael Orey. 2018. An Exploratory Study of Student Engagement in Gamified Online Discussions. Computers & Education, 120 (2018), 213–226. issn:0360-1315
[21]
Lu Ding, ChanMin Kim, and Michael Orey. 2017. Studies of Student Engagement in Gamified Online Discussions. Computers & Education, 115 (2017), 126–142. issn:0360-1315
[22]
Leandre R Fabrigar, Duane T Wegener, Robert C MacCallum, and Erin J Strahan. 1999. Evaluating the use of exploratory factor analysis in psychological research. Psychological methods, 4, 3 (1999), 272.
[23]
Andy Field, Jeremy Miles, and Zoë Field. 2012. Discovering statistics using R. Sage publications.
[24]
Ed Fincham, Alexander Whitelock-Wainwright, Vitomir Kovanović, Srećko Joksimović, Jan-Paul van Staalduinen, and Dragan Gašević. 2019. Counting Clicks Is Not Enough: Validating a Theorized Model of Engagement in Learning Analytics. In Proceedings of the 9th International Conference on Learning Analytics & Knowledge. 501–510.
[25]
Jennifer A Fredricks, Phyllis C Blumenfeld, and Alison H Paris. 2004. School Engagement: Potential of the Concept, State of the Evidence. Review of Educational Research, 74, 1 (2004), 59–109. issn:0034-6543
[26]
Stuart Hallifax, Elise Lavoué, and Audrey Serna. 2020. To Tailor or Not to Tailor Gamification? An Analysis of the Impact of Tailored Game Elements on Learners’ Behaviours and Motivation. In Artificial Intelligence in Education, Ig Ibert Bittencourt, Mutlu Cukurova, Kasia Muldner, Rose Luckin, and Eva Millán (Eds.). Springer International Publishing, 216–227. isbn:978-3-030-52237-7
[27]
Stuart Hallifax, Audrey Serna, Jean-Charles Marty, and Elise Lavoué. 2018. A Design Space For Meaningful Structural Gamification. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, LBW073.
[28]
Stuart Hallifax, Audrey Serna, Jean-Charles Marty, Guillaume Lavoué, and Elise Lavoué. 2019. Factors to Consider for Tailored Gamification. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play (CHI PLAY ’19). ACM, 559–572. isbn:978-1-4503-6688-5 https://doi.org/10.1145/3311350.3347167
[29]
Stuart Hallifax, Audrey Serna, Jean-Charles Marty, and Élise Lavoué. 2019. Adaptive Gamification in Education: A Literature Review of Current Trends and Developments. In Transforming Learning with Meaningful Technologies, Maren Scheffel, Julien Broisin, Viktoria Pammer-Schindler, Andri Ioannou, and Jan Schneider (Eds.). Springer International Publishing, 294–307. isbn:978-3-030-29736-7
[30]
Juho Hamari, Jonna Koivisto, and Harri Sarsa. 2014. Does Gamification Work?–A Literature Review of Empirical Studies on Gamification. In System Sciences (HICSS), 2014 47th Hawaii International Conference On. IEEE, 3025–3034. http://ieeexplore.ieee.org/abstract/document/6758978/
[31]
Michael D. Hanus and Jesse Fox. 2015. Assessing the Effects of Gamification in the Classroom: A Longitudinal Study on Intrinsic Motivation, Social Comparison, Satisfaction, Effort, and Academic Performance. Computers & Education, 80 (2015), 152–161. issn:03601315
[32]
Muhammad Awais Hassan, Ume Habiba, Fiaz Majeed, and Muhammad Shoaib. 2019. Adaptive gamification in e-learning based on students’ learning styles. Interactive Learning Environments, 1–21.
[33]
James C Hayton, David G Allen, and Vida Scarpello. 2004. Factor retention decisions in exploratory factor analysis: A tutorial on parallel analysis. Organizational research methods, 7, 2 (2004), 191–205.
[34]
Benjamin Heilbrunn, Philipp Herzig, and Alexander Schill. 2017. Gamification analytics—methods and tools for monitoring and adapting gamification designs. Gamification: Using game elements in serious contexts, 31–47.
[35]
Srećko Joksimović, Oleksandra Poquet, Vitomir Kovanović, Nia Dowell, Caitlin Mills, Dragan Gašević, Shane Dawson, Arthur C. Graesser, and Christopher Brooks. 2018. How Do We Model Learning at Scale? A Systematic Review of Research on MOOCs. 88, 1 (2018), 43–86.
[36]
Ana Carolina Tomé Klock, Isabela Gasparini, Marcelo Soares Pimenta, and Juho Hamari. 2020. Tailored Gamification: A Review of Literature. 144 (2020), 102495. issn:1071-5819 https://doi.org/10.1016/j.ijhcs.2020.102495
[37]
Jonna Koivisto and Juho Hamari. 2019. The Rise of Motivational Information Systems: A Review of Gamification Research. 45 (2019), 191–210. issn:0268-4012 https://doi.org/10.1016/j.ijinfomgt.2018.10.013
[38]
Richard N. Landers, Kristina N. Bauer, and Rachel C. Callan. 2017. Gamification of Task Performance with Leaderboards: A Goal Setting Experiment. Computers in Human Behavior, 71 (2017), 508–515. issn:0747-5632
[39]
Elise Lavoué, Qinjie Ju, Stuart HALLIFAX, and Audrey Serna. 2021. Analyzing the Relationships between Learners’ Motivation and Observable Engaged Behaviors in a Gamified Learning Environment. https://hal.archives-ouvertes.fr/hal-03221038
[40]
Élise Lavoué, Baptiste Monterrat, Michel Desmarais, and Sébastien George. 2019. Adaptive Gamification for Learning Environments. 12, 1 (2019), 16–28. issn:1939-1382 https://doi.org/10.1109/TLT.2018.2823710
[41]
Elizabeth A. Linnenbrink and Paul R. Pintrich. 2003. The Role of Self-Efficacy Beliefs in Student Engagement and Learning in the Classroom. Reading & Writing Quarterly: Overcoming Learning Difficulties, 19, 2 (2003), 119–137. issn:1521-0693(Electronic),1057-3569(Print)
[42]
C. Lopez and C. Tucker. 2018. Towards Personalized Adaptive Gamification: A Machine Learning Model for Predicting Performance. 1–1. issn:2475-1502 https://doi.org/10.1109/TG.2018.2883661
[43]
A. C. Marczewski. 2015. Even Ninja Monkeys like to Play.
[44]
Baptiste Monterrat, Michel Desmarais, Elise Lavoué, and Sébastien George. 2015. A Player Model for Adaptive Gamification in Learning Environments. In International Conference on Artificial Intelligence in Education. Springer, 297–306. http://link.springer.com/chapter/10.1007/978-3-319-19773-9_30
[45]
Baptiste Monterrat, Élise Lavoué, and Sébastien George. 2017. Adaptation of Gaming Features for Motivating Learners. 48, 5 (2017), 625–656. issn:1046-8781 https://doi.org/10.1177/1046878117712632
[46]
A. Mora, G. F. Tondello, L. E. Nacke, and J. Arnedo-Moreno. 2018. Effect of Personalized Gameful Design on Student Engagement. In 2018 IEEE Global Engineering Education Conference (EDUCON). 1925–1933. https://doi.org/10.1109/EDUCON.2018.8363471
[47]
Heather L. O’Brien and Elaine G. Toms. 2008. What Is User Engagement? A Conceptual Framework for Defining User Engagement with Technology. 59, 6 (2008), 938–955. issn:1532-2890 https://doi.org/10.1002/asi.20801
[48]
Wilk Oliveira and Ig Ibert Bittencourt. 2019. Tailored Gamification to Educational Technologies. Springer Singapore. isbn:978-981-329-811-8 https://doi.org/10.1007/978-981-32-9812-5
[49]
Wilk Oliveira, Juho Hamari, Lei Shi, Armando M Toda, Luiz Rodrigues, Paula T Palomino, and Seiji Isotani. 2022. Tailored gamification in education: A literature review and future agenda. Education and Information Technologies, 1–34.
[50]
Wilk Oliveira, Armando Toda, Paula Toledo, Lei Shi, Julita Vassileva, Ig Ibert Bittencourt, and Seiji Isotani. 2020. Does Tailoring Gamified Educational Systems Matter? The Impact on Students’ Flow Experience. In Proceedings of the 53rd Hawaii International Conference on System Sciences.
[51]
Rita Orji, Lennart E. Nacke, and Chrysanne DiMarco. 2017. Towards Personality-Driven Persuasive Health Games and Gamified Systems. In Proceedings of SIGCHI Conference on Human Factors Computing System. https://www.researchgate.net/profile/Rita_Orji/publication/314435151_Towards_Personality-driven_Persuasive_Health_Games_and_Gamified_Systems/links/58c2275aaca272e36dcc8d91/Towards-Personality-driven-Persuasive-Health-Games-and-Gamified-Systems.pdf
[52]
Rita Orji, Gustavo F. Tondello, and Lennart E. Nacke. 2018. Personalizing Persuasive Strategies in Gameful Systems to Gamification User Types. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 61, 62.
[53]
Ranilson Paiva, Ig Ibert Bittencourt, Thyago Tenório, Patricia Jaques, and Seiji Isotani. 2016. What do students do on-line? Modeling students’ interactions to improve their learning experience. Computers in Human Behavior, 64 (2016), 769–781.
[54]
Paul R. Pintrich. 2004. A Conceptual Framework for Assessing Motivation and Self-Regulated Learning in College Students. Educational Psychology Review, 16, 4 (2004), Dec., 385–407. issn:1040-726X, 1573-336X
[55]
Amon Rapp, Frank Hopfgartner, Juho Hamari, Conor Linehan, and Federica Cena. 2019. Strengthening Gamification Studies: Current Trends and Future Opportunities of Gamification Research. 127 (2019), 1–6. issn:1071-5819 https://doi.org/10.1016/j.ijhcs.2018.11.007
[56]
Luiz Rodrigues, Filipe D. Pereira, Armando M. Toda, Paula T. Palomino, Marcela Pessoa, Leandro Silva Galvão Carvalho, David Fernandes, Elaine H. T. Oliveira, Alexandra I. Cristea, and Seiji Isotani. 2022. Gamification suffers from the novelty effect but benefits from the familiarization effect: Findings from a longitudinal study. International Journal of Educational Technology in Higher Education, 19, 1 (2022), Feb., 13. issn:2365-9440 https://doi.org/10.1186/s41239-021-00314-6
[57]
Luiz Rodrigues, Armando M. Toda, Paula T. Palomino, Wilk Oliveira, and Seiji Isotani. 2021. Personalized Gamification: A Literature Review of Outcomes, Experiments, and Approaches. 699–706. isbn:9781450388504 https://doi.org/10.1145/3434780.3436665
[58]
F. Roosta, F. Taghiyareh, and M. Mosharraf. 2016. Personalization of Gamification-Elements in an e-Learning Environment Based on Learners’ Motivation. In 2016 8th International Symposium on Telecommunications (IST). 637–642. https://doi.org/10.1109/ISTEL.2016.7881899
[59]
R. M. Ryan and E. L. Deci. 2000. Self-Determination Theory and the Facilitation of Intrinsic Motivation, Social Development, and Well-Being. 55, 1 (2000), 68–78. issn:0003-066X 11392867.
[60]
Michael Sailer, Jan Ulrich Hense, Sarah Katharina Mayr, and Heinz Mandl. 2017. How Gamification Motivates: An Experimental Study of the Effects of Specific Game Design Elements on Psychological Need Satisfaction. Computers in Human Behavior, 69 (2017), 371–380.
[61]
Michael Sailer and Lisa Homner. 2020. The Gamification of Learning: a Meta-analysis. Educational Psychology Review, 32 (2020), March, 77–112. https://doi.org/10.1007/s10648-019-09498-w
[62]
Diana R Sanchez, Markus Langer, and Rupinder Kaur. 2020. Gamification in the classroom: Examining the impact of gamified quizzes on student learning. Computers & Education, 144 (2020), 103666.
[63]
Katie Seaborn and Deborah I Fels. 2015. Gamification in theory and action: A survey. International Journal of human-computer studies, 74 (2015), 14–31.
[64]
Daniel Strmecki, Andrija Bernik, and Danijel Radosevic. 2015. Gamification in E-Learning: Introducing Gamified Design Elements into E-Learning Systems. J. Comput. Sci., 11, 12 (2015), 1108–1117.
[65]
Sujit Subhash and Elizabeth A. Cudney. 2018. Gamified learning in higher education: A systematic review of the literature. Computers in Human Behavior, 87 (2018), Oct., 192–206. issn:0747-5632 https://doi.org/10.1016/j.chb.2018.05.028
[66]
Necati Taşk∈ and Ebru Kılıç Çakmak. 2022. Effects of gamification on behavioral and cognitive engagement of students in the online learning environment. International Journal of Human–Computer Interaction, 1–12.
[67]
Gustavo F. Tondello, Alberto Mora, Andrzej Marczewski, and Lennart E. Nacke. [n.d.]. Empirical Validation of the Gamification User Types Hexad Scale in English and Spanish. 127 ([n. d.]), 95–111.
[68]
Gustavo F. Tondello and Lennart E. Nacke. 2020. Validation of User Preferences and Effects of Personalized Gamification on Task Performance. 2 (2020), issn:2624-9898 https://doi.org/10.3389/fcomp.2020.00029
[69]
Robert J. Vallerand, Luc G. Pelletier, Marc R. Blais, Nathalie M. Briere, Caroline Senecal, and Evelyne F. Vallieres. 1992. The Academic Motivation Scale: A Measure of Intrinsic, Extrinsic, and Amotivation in Education. 52, 4 (1992), 1003–1017.
[70]
Julita Vassileva. 2012. Motivating Participation in Social Computing Applications: A User Modeling Perspective. 22, 1 (2012), 177–201. http://www.springerlink.com/index/e40qq8928p3318h6.pdf
[71]
Zamzami Zainuddin. 2018. Students’ learning performance and perceived motivation in gamified flipped-class instruction. Computers & Education, 126 (2018), Nov., 75–88. issn:0360-1315
[72]
Zamzami Zainuddin, Samuel Kai Wah Chu, Muhammad Shujahat, and Corinne Jacqueline Perera. 2020. The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educational Research Review, 30 (2020), June, 100326. issn:1747-938X https://doi.org/10.1016/j.edurev.2020.100326
Index Terms
- Investigating the Effects of Tailored Gamification on Learners’ Engagement over Time in a Learning Environment
Recommendations
Analyzing the relationships between learners’ motivation and observable engaged behaviors in a gamified learning environment
Highlights- We conducted an ecological study with 257 students from 5 middle schools
- We ...
AbstractOver the last ten years, gamification has been widely integrated in digital learning environments as a way to increase learners’ motivation. However, little is known about engaged behaviors adopted by learners when using gamified ...
Improving student engagement during in-person classes by using functionalities of a digital learning environment
AbstractThe introduction of digital learning environments in higher education requires teachers to be able to optimize their use to improve student engagement in the learning process during in-person classes. In a quasi-experiment (N = 303), ...
Highlights- Functionalities of a digital environment influences student engagement in learning.
How Gamification Affects Students' Engagement and Language Learning Beliefs in Pre-Class Learning of Flipped EFL Courses: A Theoretical Analysis
WSSE '20: Proceedings of the 2nd World Symposium on Software EngineeringStudents' inadequate engagement in pre-class learning is one of the causal factors that impede effective flipped learning. One possible solution can be the implementation of gamification in pre-class learning activities since motivating effects of game ...
Comments
Information & Contributors
Information
Published In
November 2023
1360 pages
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 October 2023
Published in PACMHCI Volume 7, Issue CHI PLAY
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 203Total Downloads
- Downloads (Last 12 months)159
- Downloads (Last 6 weeks)12
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in