Gamification Methods in Higher Education

Gamification Methods in Higher Education Lila A. Loos(&) and Martha E. Crosby(&) University of Hawaii at Manoa, 1680 East-West Road POST 305D, Honolulu, HI 96822, USA {Lila7194,Crosby}@Hawaii.edu Abstract. Gamification impacts the classroom through the integration of personal and social elements, creating an immersive learning environment. Rooted in the motivational framework of flow experiences, gamification is a contemporary educational approach that leverages the characteristics of gaming culture. This study examined how gamification techniques were operationalized by eight professors and one director of instructional technology. The results of interviews among a purposive sample show that gamification improves learning outcomes. Subjects ranging from computer science and technology to languages and multimedia journalism are taught in classrooms on campus, online, and in hybrid settings. Game design mechanics such as points, challenges, and collaboration rank as the top three elements of gamification. Professors noted increases in student coursework completion and discussion participation; students were exposed to real life situations and had fun learning. This study addresses the gap in educational perspectives through gamification to encourage desirable behavior for the benefit of the learner community. Keywords: Gamification
Flow
Engagement
Interaction
Higher education 1 Introduction The use of gamification to promote student engagement embodies an evolving approach to learning in higher education. The application of digital game playing represents a shift in the information culture to the Internet and is especially relevant to those born after 1990 because this method intensifies engagement and active participation [1]. Gamification fosters critical thinking about real world topics, prepares students to enter a technical workforce [2], and provides solutions to help schools solve difficult motivation and engagement problems [3]. Irrespective of the application of gamification to enhance or resolve educational concerns, skills necessary for gaming, such as anticipating, thinking recursively, and organizing information within time constraints, align with preferred educational abilities and skills [4]. Based on the concept of active learning, gamification techniques are an alternative to the passive classroom and a driver of student engagement. In this study, experts are interviewed to discover the methods and the impact of gamification, revealing an adoption of concept-mastery-based learning. The following sections strengthen educators’ foundational knowledge of gamification: diverse definitions of gamification in the non-game context of education, framework of flow, method, results and discussion, and conclusion. © Springer International Publishing AG 2017 P. Zaphiris and A. Ioannou (Eds.): LCT 2017, Part I, LNCS 10295, pp. 474–486, 2017. DOI: 10.1007/978-3-319-58509-3_37 Gamification Methods in Higher Education 475 2 Diverse Definitions of Gamification in the Non-game Context of Education Gamification is defined as the use of game elements to create an active learning environment by engaging students in the process of knowledge acquisition. Although the term was first used in 2008, gamification’s popularity was established in 2010 and its definition has since come under dispute [5]. Deterding, Dixon, Khaled, and Nacke [5] define gamification as “the use of game design elements in non-game contexts” (p. 12). The power of game-like techniques in the context of learner behavior has been explored and defined by researchers studying subjects in the educational domain. Kapp’s [6] multifaceted integration of game-like elements combines elements such as narrative, challenge, sense of control and mastery, and decision-making in a non-game environment. As educators integrate game elements into the classroom, the gamification dialog evolves and its definition develops. Although learning from game elements is dissimilar from playing a game designed to promote the mastery of a specific concept, educators may include games in their methodological approach to gamification. One example of game integration is Fink, Best, Manz, Popovsky, and Endicott-Popovsky’s [7] presentation of security scenarios to assess learner responses. The observational experiment measures cyber defense situational awareness by scoring teams based on the conditions for each requirement. In this instance, incorporating games to improve readiness for real world cyber defense provides an immersive simulation similar to Reiners et al.’s [8] authentic problem solving environment, in which student “learning should be about fun, play, and passion” (p. 7). Huang and Soman [9] suggest distinguishing games from gamification because the latter utilizes game elements, such as points and leaderboards, in the learning process to accomplish an action that increases skill and knowledge and stimulates engagement and motivation. Based on outcomes, gamification includes games because they activate learning in the classroom, suggesting a more grounded definition of gamification. Gros [10] focuses on the integration of the structural aspects of games, such as icons and imagery, to “promote conceptual learning, problem solving skills, co-operation, and practical participation” (p. 30). The team-based element harnesses student competitiveness to enhance learning. Likewise, Burke’s [11] method of using game mechanics such as challenges, points, badges, and leaderboards to track students’ progress through levels of skill development and autonomy encourages motivated players toward goal achievement. Alternatively, Lee and Hammer [3] emphasize the design of gamification targets to meet the specific needs of schools and achieve effective results, distinguishing gamification from its normal classroom counterparts such as grades, tests, group projects, and extra credit. The authors propose that game elements do not automatically produce student engagement; rather, they represent an intervention for addressing cognitive, emotional, and social problems. Although not defined by a specific pedagogy, the structure of gamification supports techniques for enhancing student learning experiences, supporting Kapp’s [6] application of game elements that activate desirable learning behavior or offer solutions to problematic learning issues. The evolving phenomena are examined through current methods of 476 L.A. Loos and M.E. Crosby gamification and their potential for promoting constructive learning in higher education. 3 Framework of Flow The cultivation of motivation and engagement through gamification is foundational to Csikszentmihalyi’s [12] theoretical framework of flow. Positive psychology recognizes the subjective experience of flow as it is conditioned by goals and the adjustment of responses based on feedback [13]. Empirical evidence suggests that the integration of flow has a positive influence in higher education contexts. Csikszentmihalyi [12] describes flow as a powerful force, a “state of concentration so focused that it amounts to absolute absorption in an activity” (p. 1). Csikszentmihalyi’s [12] studies of interviewees report similar experiences of flow, including the ability to complete tasks, concentrate deeply, have clear goals, receive immediate feedback, achieve effortless involvement, control one’s actions, and have no concern for self or sense of the passage of time during the activity. These common flow characteristics produce states of enjoyment represented by deep concentration on an activity that, based on a student’s skill level, is sufficiently challenging. Csikszentmihalyi [14] argues that these relevant components allow flow to be experienced during “almost any activity” (p. 2), and he advocates flow-augmented creativity in the student and teacher domain. Flow experiences require a combination of “well-presented knowledge, interested students, and stimulating teachers” [15] p. 181. The intrinsic characteristics of flow are argued to be present in gamification, affecting learners by encouraging engaged behavior. Contrary to the standard approach of memorization-based classroom teaching, Khan [16] suggests, “the most effective way to teach would be to emphasize the flow of a subject, the chain of associations that relates one concept to the next and across subjects” (p. 48). The association between flow and the integration of game mechanics motivates student learning. Steele and Fullagar [17] studied engagement in college coursework mediated by flow. A positive association was found between flow and instructor support of autonomy defined by goals and tasks. Feedback from a teacher or from the task itself is a predictor of progress toward goals and a state of flow. Gammon and Lawrence [18] studied the effect of feedback and self-assessment on flow analytics, focusing on test-taking experiences. They found that student feedback and self-assessment experiences presented supplementary opportunities for teachers to enhance learning material, thus supporting learner performance. Similar studies suggest that the flow condition of engagement occurs when the task is at a level appropriate to one’s capabilities. Nakamura and Csikszentmihalyi [13] describe two empirical studies. The first examined recreational activities, emphasizing the intrinsic rewards of play and games, and the second studied surgeons who are rewarded extrinsically. In both groups, the results demonstrate a sense of engagement perceived by the participants while participating in activities at levels appropriate to their individual abilities. Csikszentmihalyi [14] emphasizes the importance of balancing challenges and skills to avoid the distraction of more ambitious tasks and to encourage concentration, interest, and enjoyment, which result in flow. In a subsequent Gamification Methods in Higher Education 477 study, D. Shernoff, Csikszentmihalyi, Shneider, and E. Shernoff [19] encouraged instructors to engage learners through incremental skill-building and immediate feedback to match student ability to skill level. Beylefeld and Struwig [20] argue that the “relationship between flow and motivation is highly relevant in higher education environments” (p. 933). Their study investigates students’ ability to master medical microbiology through game-based learning strategies by documenting the impact of flow on student engagement in skill development. The results showed that students responded positively to the flow-inducing game-based learning used to promote engagement with and mastery of microbiology content, describing it as an “invigorating teaching device” (p. 938). Kiili’s [21] study of flow experiences through content creation in educational games reports student satisfaction with appropriate challenges, problems, and a story line. The author recognizes the benefit of flow in the design of educational games: “the reward of flow is obvious: it has a positive effect on learning” (p. 196). Flow is argued to have a positive effect on learner attentiveness, thereby increasing engagement with bodies of knowledge. This study describes relevant approaches, encompassing both games and game mechanics, to enhancing learner outcomes. 4 Method The purpose of this study is to discover how gamification techniques are used in higher education to engage students and improve learning outcomes (Table 1). Game-based techniques are examined through discussions with eight university professors and an instructional technology designer, all of whom apply immersive methods to improve learning outcomes. The interview questions were designed to capture emergent themes that deepen our understanding of gamification processes and learner impact. The research instrument consisted of nine open-ended questions and three closed-ended questions (Appendix). The questions gave respondents maximum flexibility in structuring their responses [22] and allowed paths of discovery to reveal themselves for comparative induction. The interview focused on the selection and implementation of gamification elements, methodology, and classroom results. The comprehensive data collected through participant perspectives and experiences revealed a variety of design methods and prompted in-depth discussions of the implementation of gamification in higher education. Table 1. Research questions RQ1: What gamification techniques do educators employ as an approach to higher education learning? RQ2: How do educators use gamification-based learning to engage students in higher education? RQ3: What is the impact of gamification on the learning outcomes of higher education students? 478 L.A. Loos and M.E. Crosby The responses were recorded, transcribed and analyzed to discover the emergent themes. Similarities and differences were compared to increase the validity of the findings [23]. Through inductive comparisons, descriptive connections were made by categorizing and delineating the themes used in the coding process [24]; the core interview themes are formulated and presented in the following sections. 5 Results and Discussion A sample of nine professionals utilizing gamification in education participated in the interview process. The data gathered in 2014 was collected from various United States universities and a private organization in England. Pseudonyms are used to protect interviewees’ identities. Table 2 provides a summary of the interviewees. Table 2. Participants Pseudonym Occupational title Professor Academic specialty or industry specialty Computer science Sports education Communications Learning technologies P6 Professor Professor Lecturer, manager Technology director Professor P7 Professor Information systems P8 P9 Professor Professor Journalism Social interactions P1 P2 P3 P4 P5 Instructional technology design Game design University or company name University of Washington University of Idaho University of Hawaii University of Brighton Rosetta Stone Carnegie Mellon University University of Michigan University of Hawaii Syracuse University The courses taught by the respondents range from technology to languages. Four of the respondents with fewer than ten years of teaching experience use gamification in language, social media innovation, multimedia journalism, news writing, introduction to mass communications, and introduction to information courses. Seven of these courses are located on university campuses and one course is offered online. One respondent has ten years of teaching experience in a variety of venues (campus, online, and in hybrid settings) and applies gamification in computer science and information assurance courses. Burke [11] argues that the value of gamification becomes a reality through educators who understand its importance and reports that the most significant barrier to success is the lack of gamification design skills. The participants discussed how they define, drive, measure, design, determine, and personalize game mechanics in their approaches to implementing gamification. Gamification Methods in Higher Education 5.1 479 Defining Gamification Participants consistently described gamification as an interactive and immersive mechanism aimed at challenging students and activating classroom participation. Table 3 lists definitions focused on the design aspect of gamification. Table 3. Gamification definitions Definition “The use of design principles commonly used in games for purposes other than games” “Game-like features such as a point system, competitive ranking, or badging” “An educational process for the purpose of activation of student learning and transferring them from a productive to more creative thinking and approach to the learning process and application of this knowledge to real life” “Ways to bring real work activity into the classroom to help students learn beyond just reading and listening to lectures” “Choose the way they want to learn including the modes of learning and much of the significant attributes of assignments and activities” “Meaningful changes in the system that are equally valuable yet tailored to what the student wants to learn” “Giving people compelling choices in which they can pick their path to learning and success” Participant P6 P5 P2 P1 P8 P8 P8 Participants emphasized structural definitions of gamification. Similarly, Werbach [25] describes gamification as an academic expansion beyond points and badges to aligning learning experiences with game mechanics. Similar to the description of student agency advocated by P8, flow studies [6, 12, 17, 26, 27] suggest that gamification supports autonomy and personal control. All participants described game-like principles in their approach to active education. Table 4. Gamification drivers Driver “Ways to activate a student so they become involved in their learning process so that learning becomes more than just memorization” “As the material becomes more interactive, it becomes more engaging” “Commonly used as a method of giving feedback about performance and for incentivizing performance” “Fundamentally and the most important part of learning is developing a love for learning and if you can develop that love about the topic you’re teaching, then the students will embrace that and gravitate toward that the rest of their lives” “Provided feedback in a complex social system” Participant P1 P3 P4 P8 P9 480 5.2 L.A. Loos and M.E. Crosby Gamification Drivers Identified as motivational experiences, the gamification drivers mentioned by the participants are shown in Table 4. The informants reported increased student involvement, social interaction, and feedback as factors that supported improved learning outcomes. Several studies [9, 12, 17–19, 27, 28] have reported that feedback is a motivational tool that can be used to align student abilities with skills. 5.3 Measuring Student Learning Outcomes with Gamification Measuring learning outcomes using a point system was the most common form of student assessment. Although the majority of respondents gamified specific course content, P8 applied gamification to the entire classroom structure and observed that grades are at the forefront of students’ minds. In a gamified course, grades begin at zero and students gain points through the point system. Alternatively, P5 proposed a virtual interactive game environment in which an avatar drives engagement and allows students to demonstrate their mastery of the learning objectives. P3, like P4, uses gamification as a motivator but uses exams and grades to assess the quality of student work. Table 5 lists participant measures of gamification. Table 5. Measuring student learning outcomes with gamification Measurement by gamification Participant “Can be done inside or outside of a game system” P6 “Learning outcomes cannot be measured using gamification” P5 “Think it is inappropriate to use gamification to measure learning outcomes” P7 “Game is really just there to deliver the lesson” P3 “Allocated points to a rubric for student evaluation” P1 5.4 Gamification Pedagogy De Byl’s [29] study of student enjoyment and engagement in a gamified curriculum suggests that “educators are always seeking a new pedagogy or technology that might engage and immerse their students” (p. 256). Although more than half of the respondents have not applied pedagogy to gamification methods (Table 6), P2 identified pedagogy as necessary for effective implementation: “gamification is not a system by itself, it has to be treated as a pedagogical system.” P1 uses the high-level Kuzmina-Bespalko-Popovsky (KBP) model, which is based on American and Russian pedagogical approaches [30] and modifies content and delivery methods to gamify education. This system-centered process produces creative problem solvers through interconnected and dynamic elements comprising the “student, teacher, goals, content, and didactic processes of the curriculum” (p. 59). Like Csikszentmihalyi’s [12] match between student ability and skill level, Vygotsky [31] suggests a relationship between learning and development in which levels should be matched; an approach identified by P7. Vygotsky’s [31] theory on the zone of proximal development is the “distance Gamification Methods in Higher Education 481 Table 6. Gamification pedagogy Pedagogy Pedagogy contains elements of a method, organization, and remedy Kuzmina-Bespalko-Popovsky (KBP) model Vygotsky’s zone of proximal development and Csikszentmihalyi’s concept of flow Participant P2 P1 P7 between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance or in collaboration with more capable peers” (p. 40). Shernoff, Hamari, and Rowe [32] propose a similar viewpoint in which flow is achieved by increasing the level of skill to the next, more challenging, level. 5.5 Gamification Elements Table 7 provides a list of gamification elements employed in the classroom. Ten mechanisms are arranged in descending order according to the frequency with which they were mentioned by the participants. The use of game design characteristics such as points, the addition of new challenges, collaboration, and games were the four most frequently used mechanisms of gamification. A notable exception is illuminated by P6, who does not believe gamification exists: If it does exist, every classroom is already gamified as students already receive points, grades. The only question worth discussing is where the designs of the systems are good ones or not, whether they are gamified is unimportant and misleading. This participant claims that gamification elements are already present in schools but under different labels. Examples of game elements that parallel existing classroom components are the use of points to determine grades, bonuses taking the place of extra Table 7. Gamification elements Gamification elements Points increase your level Addition of new challenges Collaboration and/or interaction Games in the educational process Achievements earn recognition Leaderboard displays levels in descending order Master one level before moving to the next level Bonuses received as extra reward Flexibility of path selection Countdown for time limits or deadlines Participant P1, P2, P3, P1, P3, P4, P1, P3, P4, P1, P2, P3, P3, P5, P7, P4, P5, P7, P1, P2, P5, P7, P8, P9 P1, P8, P9 P8, P9 P4, P5, P5, P4, P8 P8 P7 P5, P7, P8, P9 P7, P8, P9 P8, P9 P5 482 L.A. Loos and M.E. Crosby credit, countdowns to deadlines, or additional challenges replacing surprise quizzes. Although de Byl [29] recognizes the existing alignment between gamification and the educational system, the author acknowledges differences, including increased engagement and the transparency of points, levels, and status, between a gamified setting and the traditional classroom’s system of weighted assignment and test grades. P5 considers the learning goal prior to selecting the appropriate gamification elements. Although studies have found that feedback is a motivational tool that produces positive results [9, 12, 17–19, 27, 28], it was not mentioned as an element employed by the participants. Feedback was used to define gamification by one respondent, however, and was another respondent’s reason for employing gamification. 5.6 Personalizing Gamification Techniques Five respondents reflected on the benefits of tailoring gamification techniques to students’ personality or cognitive ability, as shown in Table 8. P9 feels that personalization is risky and would not apply it when instructing a classroom of 250 students, arguing instead that teaching options should be aligned with learning goals. P1 uses the KBP pedagogical model to understand the student and adjust course content accordingly. Table 8. Personalizing gamification techniques Personalization “I give them options so the students themselves can kind of adjust their learning based on the topic that they pick” Aligns student physiology with athletic activity “Always get to know the student and then tailor each assignment to that student’s needs and talents” “Adapt techniques from semester to semester based on student feedback and I try to be sensitive to a full range of student needs and motivations” “I don’t personalize techniques based on personality other than the idea that there are different learner styles and I try to match the literature” Participant P1 P2 P8 P7 P9 6 Evaluation of Gamification Participants’ ability to effectively address classroom difficulties necessitates the continuous modification and improvement of their gamification design methods. To maximize learner engagement, educators evaluated the appropriate selection of subject matter delivery methods and experiences, as shown in Table 9. Instructors’ methods emphasized gamification as a positive accompaniment to lectures that accommodates various learner types. Furthermore, student feedback and outcomes improve the implementation of gamification in the classroom. The participants reported no major problems, with the exception of the barrier created by cultural expectations of the lecture format in higher education. Gamification Methods in Higher Education 483 Table 9. Evaluation of gamification Evaluation Applications to real world scenarios Students work harder with games compared to regular assignments Hands-on work Well-designed teacher and student assessments Potential inability of students to get along in their particular groups Flexibility in assignments and higher student ratings for the gamified class Can be disruptive and alienating when rules change Creates a sense of fairness by accommodating highly attentive and mediocre learners Some students prefer to have content delivered via lecture Effective processes defined by pedagogy Well-designed teacher and student assessments Build layers of gamification to understand what works best Participant P1 P1 P3 P3 P3 P9 P4, P9 P9 P1, P1, P3, P5, P8 P2, P8 P5, P7 P7 7 Conclusion The results indicate that gamification’s use of game mechanics activates student involvement, engagement, and motivation. Desiring to enrich student engagement in higher education, the participants leveraged students’ ubiquitous exposure to the intense state of game technologies to create an intensified exploration of their course’s subject matter. Through the practical application of gamification elements characterized by personal and social combinations of classroom activities, including exposure to real life work activities, optimal learner experiences were observed to produce increased student discussion, higher coursework scores, and more enjoyment for learners. Gamification is an incentivizing technique that explains, reshapes, and brings attention to immersive learning methods. Games are an element of gamification that supplement active learning and do not replace teacher instruction. Research indicates [28, 33] that gamification has yet to become a mainstream program of study in higher education. Some studies, however, suggest it has the potential to become a conventional method in less than five years [34]. This study advances gamification’s role as a game-based learning tool for higher education by identifying methods and exploring educators’ successes and challenges. Arnab et al. [35] suggest a gap in the research findings regarding the assimilation potential for education adoption. In response to these obstacles, this research contributes to the future development of more complex gamification design approaches that integrate game elements for more effective implementation. Additionally, future research in pedagogy should systematically develop a gamification discourse and assess the impact of gamification on the learning environment. Ignoring the gamification movement may deprive students of the opportunity to discover information and ideas through active involvement. 484 L.A. Loos and M.E. Crosby Appendix Interview Questions Closed-ended questions Link to research question Table 10 1. What is your title? 2. What is your academic specialty? 3. What classes have you recently taught that contain gamification elements? a. What was the subject of the class? b. Was the class on-campus, online, or hybrid? c. How long have you taught the class? Open-ended questions 1 Link to research question Table 10 1. How do you define gamification? 1, 2, 3 2. What is the purpose of employing gamification in the classroom? 2, 3 3. How do you measure student learning outcomes utilizing 3 gamification? 4. Is there any gamification pedagogy that you particularly identify 2 with? 5. How do you select gamification elements? 1 6. Have you ever personalized gamifying techniques based on 1 student personality or cognitive ability? 7. Could you reflect on the positive results, if any, that you have 3 experienced using gamification in the classroom? Were the results quantified? 8. Have you experienced any problems using gamification in the 3 classroom? Were the results quantified? 9. Is there any general advice or best practices that you would like 2, 3 to elaborate on? Table 10. Research questions 1. What gamification techniques do educators employ in higher education learning? 2. How do educators use gamification-based learning to engage students in higher education? 3. What is the impact of gamification on the learning outcomes of higher education students? Gamification Methods in Higher Education 485 References 1. McGonigal, J.: Reality is Broken: Why Games Make Us Better and How They Can Change the World. Penguin, Jonathan Cape (2011) 2. Sardone, N.B., Devlin-Scherer, R.: Teacher candidate responses to digital games. J. Res. Technol. Educ. 42(4), 409–425 (2010). doi:10.1080/15391523.2010.10782558 3. Lee, J.J., Hammer, J.: Gamification in education: what, how, why bother? Acad. Exch. Q. 15 (2), 146 (2011). https://dialnet.unirioja.es/servlet/articulo?codigo=3714308 4. Pillay, H.: An investigation of cognitive processes engaged in by recreational computer game players: implications for skills of the future. J. Res. Technol. Educ. 34(3), 336–350 (2002). doi:10.1080/15391523.2002.10782354 5. Deterding, S., Khaled, R., Nacke, L., Dixon, D.: Gamification: toward a definition. In: Paper Presented at the Gamification Workshop Proceedings of ACM CHI 2011 Conference on Human Factors in Computing Systems, Vancouver, BC, Canada: ACM (2011). http:// gamification-research.org/wp-content/uploads/2011/04/02-Deterding-Khaled-Nacke-Dixon.pdf 6. Kapp, K.M.: The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education. Wiley, San Francisco (2012) 7. Fink, G., Best, D., Manz, D., Popovsky, V., Endicott-Popovsky, B.: Gamification for measuring cyber security situational awareness. Found. Augmented Cogn. 8027, 656–665 (2013). doi:10.1007/978-3-642-39454-6_70 8. Reiners, T., Wood, L.C., Chang, V., Guetl, C., Herrington, J., Gregory, S., Teräs, H.: Operationalising gamification in an educational authentic environment. In: Kommers, P., Issa, T., Isaías, P. (eds.) International Conference on Internet Technologies and Society, pp. 93–100. IADIS Press, Perth (2012) 9. Huang, W.H.-Y., Soman, D.: Gamification of education. In: Paper Presented at Research Report Series: Behavioural Economics in Action. Rotman School of Management, Toronto, Canada (2013) 10. Gros, B.: Digital games in education: the design of games-based learning environments. J. Res. Technol. Educ. 40(1), 23–38 (2007) 11. Burke, B.: Gamify: how gamification motivates people to do extraordinary things. Bibliomotion, Brookline (2014) 12. Csikszentmihalyi, M.: Flow: The Psychology of Optimal Experience. Harper-Perennial, New York (1990) 13. Nakamura, J., Csikszentmihalyi, M.: The concept of flow. In: Snyder, C.R., Lopez, S.J. (eds.) Handbook of Positive Psychology, pp. 89–105. Oxford University Press, New York (2002) 14. Csikszentmihalyi, M.: Finding Flow: The Psychology of Engagement with Everyday Life. HarperCollins, New York (1997) 15. Csikszentmihalyi, M.: The Systems Model of Creativity: The Collected Works of Mihaly Csikszentmihalyi. Springer, Dordrecht (2015) 16. Khan, S.: The One World Schoolhouse: Education Reimagined. Hachette Digital, New York (2012) 17. Steele, J.P., Fullagar, C.J.: Facilitators and outcomes of student engagement in a college setting. J. Psychol. 143(1), 5–27 (2009). doi:10.3200/jrlp.143.1.5-27 18. Gammon, S., Lawrence, L.: Improving student experience through making assessments ‘flow’. In: Bryan, C., Clegg, K. (eds.) Innovative Assessment in Higher Education, pp. 132– 140. Routledge, New York (2006) 486 L.A. Loos and M.E. Crosby 19. Shernoff, D.J., Csikszentmihalyi, M., Shneider, B., Shernoff, E.S.: Student engagement in high school classrooms from the perspective of flow theory. School Psychol. Q. 18(2), 158– 176 (2003). doi:10.1521/scpq.18.2.158.21860 20. Beylefeld, A.A., Struwig, M.C.: A gaming approach to learning medical microbiology: students’ experiences of flow. Med. Teacher 29(9), 933–940 (2007). doi:10.1080/ 01421590701601550 21. Kiili, K.: Content creation challenges and flow experience in educational games: the IT-emperor case. Internet High. Educ. 8(3), 183–198 (2005). doi:10.1016/j.iheduc.2005.06. 001 22. Aberbach, J.D., Rockman, B.A.: Conducting and coding elite interviews. Polit. Sci. Polit. 35 (04), 673–676 (2002) 23. Boeije, H.: A purposeful approach to the constant comparative method in the analysis of qualitative interviews. Qual. Quant. 36(4), 391–409 (2002). doi:10.1023/a:1020909529486 24. Mayring, P.: Qualitative Content Analysis: Theoretical Foundation, Basic Procedures and Software Solution. Springer, Klagenfurt (2014) 25. Werbach, K.: (Re)Defining gamification: a process approach. In: Spagnolli, A., Chittaro, L., Gamberini, L. (eds.) Persuasive Technology, pp. 266–272. Springer, Switzerland (2014) 26. Agarwal, R., Karahanna, E.: Time flies when you’re having fun: cognitive absorption and beliefs about information technology usage. MIS Q. 24(4), 665–694 (2000). http://misq.org/ cat-articles/time-flies-when-you-re-having-fun-cognitive-absorption-and-beliefs-aboutinformation-technology-usage.html 27. Kiili, K.: Digital game-based learning: towards an experiential gaming model. Internet High. Educ. 8(1), 13–24 (2005). doi:10.1016/j.iheduc.2004.12.001 28. Hamari, J., Koivisto, J., Sarsa, H.: Does gamification work? A literature review of empirical studies on gamification. In: Paper Presented at the Proceedings of the 47th Hawaii International Conference on System Sciences, pp. 3025–3034. Computer Society Press, Honolulu, January, 2014. http://people.uta.fi/*kljuham/2014-hamari_et_al-does_gamification_work.pdf 29. De Byl, P.: Can digital natives level-up in a gamified curriculum? In: Brown, M., Hartnett, M., Stewart, T. (eds.) Future Challenges, Sustainable Futures, pp. 256–266. ASCILITE, New Zealand (2012) 30. Endicott-Popovsky, B.E., Popovsky, V.M.: Application of pedagogical fundamentals for the holistic development of cybersecurity professionals. ACM Inroads 5(1), 57–68 (2014). doi:10.1145/2568195.2568214 31. Vygotsky, L.: Interaction between learning and development. In: Gauvain, M., Cole, M. (eds.) Readings on the Development of Children 2004, pp. 34–41. Worth Publishers, New York (1978) 32. Shernoff, D.J., Hamari, J., Rowe, E.: Measuring flow in educational games and gamified learning environments. In: World Conference on Educational Media and Technology. AACE, Tampere, Finland (2014). https://www.researchgate.net/publication/264046054_ Measuring_Flow_in_Educational_Games_and_Gamified_Learning_Environments 33. Park, H.J., Bae, J.H.: Analysis and survey of gamification. In: Paper Presented at the Science and Engineering Research Support Society, Current Research on Game and Graphics (2287-1233), International Workshop on Game and Graphics Tasmania, Australia. SERC (2013) 34. Johnson, L., Adams, B.S., Cummins, M., Estrada, V., Freeman, A., Ludgate, H.: NMC Horizon Report: 2013 Higher Education. The New Media Consortium, Austin (2013) 35. Arnab, S., Berta, R., Earp, J., De Freitas, S., Popescu, M., Romero, M., Stanescu, I., Usart, M.: Framing the adoption of serious games in formal education. Electron. J. e-Learning 10 (2), 159–171 (2012)