Gamification Does Not Belong at a University

“Gamification Does Not Belong at a University” Adam Palmquist University of Skövde Högskolevägen 1 541 45 Skövde Telephone: +46500448894 adam.palmquist@his.se Jonas Linderoth University of Skövde Högskolevägen 1 541 45 Skövde Telephone: +46500448819 jonas.linderoth@his.se (Not presented due to the Covid-19 pandemic and the cancellation of the DiGRA 2020 conference) ABSTRACT This paper reports a case study in which some students in a large-scale gamification implementation project wrote a script that automated their progression. The incident was followed with multi-sited ethnography and analysed through the lens of Goffman’s frame analysis. Based on chat logs, mail correspondence, data on user behaviour in the learning management system, informal conversations and student interviews, the study shows that different actors have somewhat different perceptions of gamification, as they framed the incident with the script in different ways. The students saw their actions as a form of resistance and activism towards problematic game design and had a desire to uphold specific tech-student identities. The gamification designers treated the incident as an act of playfulness and display of technological skills. The university, on the other hand, framed the incident as cheating. The study highlights the need for educational institutions to be knowledgeable about games and gaming behaviour if they want to implement gamification. Keywords: Gamification, Frame Analysis, Higher Education, Goffman Proceedings of DiGRA 2020 © 2020 Authors & Digital Games Research Association DiGRA. Personal and educational classroom use of this paper is allowed, commercial use requires specific permission from the author. INTRODUCTION – GAMIFICATION AND FRAME ANALYSIS Gamification, processes in which knowledge about how to develop games can inform the design of services and applications on a general level, has been defined in somewhat different ways. One of the more commonly used definitions states that gamification is the “use of game design elements in non-game contexts” (Deterding et al. 2011). Huotari and Hamari (2012) have emphasised that this definition lacks focus on the goal of the process. They instead suggested that gamification is “a process of enhancing a service with affordances for gameful experiences in order to support user’s overall value creation.” (19). Such a definition suggests that gamification involves adding to an activity and/or artefact features that change the experience into something ‘game like’, where this transformation is not a goal in itself but a means to accomplish something else. Erving Goffman’s theory, put forth in Frame Analysis (1986), can serve here as a basis for approaching gamification. Goffman’s work has been previously utilised in the study of play and games (see Deterding 2009). The basic premise of frame analysis is that in any social situation we will ‘define the situation’ – and ask ourselves, ‘What is going on here right now?’ This definition guides our experiences as well as our actions, and we will ‘frame’ the situation (Goffman 1986). Being a socio-psychological mechanism, frames are not fixed; they change during strips of action as we re-frame and negotiate what we find is happening. Frames can be, as Goffman puts it, laminated, as many things are occurring at the same time and we move between frames; for example, interrupting a series of serious interactions in order to make a joke. Hence, gamification can be seen as a designed frame, laminated on top of an existing activity. Classical gamification with points, badges and leader boards (PBLs) is an example of how a gamification designer attempts to make an uninspiring activity more engaging (Werbach and Hunter 2012). Using Goffman’s theory as a lens, this can be expressed as building support for the users in establishing a ‘game frame’ that is intended to be laminated with the original activity. Typically, the idea is to turn a dull task or chore into something playful by creating new incentives for behaviour. In PBL design, points are rewarded to users for different requested behaviours in a system. The accumulated points are then shown on a leader board in order to compare how the users are performing against each other. Users can also pick up badges for reaching pre-defined goals, such as reaching a 7-day streak of exercising or taking vitamins 21 days in a row. While such transformation of an activity might seem straightforward on paper, frames are negotiated and established through interaction and hence in the end are dependent on the users. Users of game systems are likely to be reflexive in relation to their own position as players/users. For instance, Lantz-Andersson and Linderoth (2011) showed in an interaction study of the use of game-based learning (GBL) that students were aware of themselves as objects for the GBL designers. During their use of GBL, the students postulated what they thought that the GBL designers were thinking about them as users when they created the game system. For example, they postulated that the designers were likely to have designed the game with logical progression, hence mathematical procedures that had not been part of the game were likely not part of the solutions. The students’ propositions therefore became resources for progressing in the game. The students did not accept the designed frame and did not position themselves as users/students who were motivated by solving maths in a game. Instead, they established a meta-frame, trying to analyse the designers’ perceptions of hypothetical future students. Björk (2019) has stressed that many activities that are already gamified are systems with features that resemble games. Education is, for instance, an activity with elaborate -- 2 -- examination systems. In their classical study Up to the mark: A study of the examination game, Miller and Parlett (1974) showed how students treat their exams in an instrumental, game-like way, being completely strategic in relation to ‘what counts’. A university course has a clear, quantifiable goal; it already has what Juul (2003) labels as ‘outcome valorization rules’. Any implementation of PBLs and other gamification features in an educational setting with formal examinations will become related to this ‘examination game’. According to O’Donnell, Kappen and Fitz-Walter (2017), the second-largest area of gamification research centred around 2016 education. Many of the studies concerned higher education (Dichev and Dicheva 2017) and showed moderately positive results (Koivisto and Hamari 2019). For instance, Bonde et al. (2014) showed that there was an increase in both learning results and student engagement when gamification was used in biotech education. Hakulinen, Auvinen and Korhonen (2015) showed that achievements/badges had positive effects on students in a higher education computer science course. Domínguez et al. (2013) showed that a gamified version of a curriculum in a higher education course improved test scores on practical assignments and increased student engagement in course assignments. Barata et al. (2013) studied students’ learning in the fields of information systems and computer engineering. The study showed improvements in students’ attendance, number of downloads of lecture slides and the number of posts on the course’s forums. Knutas et al. (2019) investigated the level of student engagement when features in a Massive Open Online Course were gamified. The results showed that engagement with gamification was positively associated with desired behaviour and positive outcomes on the collaborative discussion platform. Nevertheless, there is also a significant amount of research that shows week results of gamification. Dichev and Dicheva (2017) emphasise in a critical review of 63 papers about gamification in education that there still exist grey areas in the research on gamification. As shown above, the unit of analysis varies between different studies, and displays of high engagement and motivation do not necessarily imply higher achievement. In a review of empirical literature, Majuri, Koivisto and Hamari (2018) argued that the success of gamification is contextually dependent, something they suggest future research should take into account. Gamification of education is therefore far from straightforward. As illustrated by the findings of Lantz-Andersson and Linderoth (2011), students can potentially reflect on their own position in a gamified system and guide their behaviour according to this meta awareness. This phenomenon, as well as the friction between game rules and rules in the formal educational system (Björk 2019), was observed in the case study that is reported in the present paper. The current study was based on an incident that happened in a large-scale gamification implementation project in which Author 1 was involved. As the incident initially showed some interesting frame processes that suggested anything but a straightforward acceptance of gamification, the researchers, Authors 1 and 2, decided to study it further. Following the method of multi-sited ethnography, the incident was analysed through the lens of frame analysis. The project and the students, teachers, designers, companies and universities have been anonymised. All participants studied were informed that they could withdraw their statements at any time during the study. The individuals and organisations in the protocol from the disciplinary committee, a public document, have been anonymised. -- 3 -- THE PROJECT In the last decade, completion rates in higher education have suffered from a downward trend (Vossensteyn et al. 2015). Within tertiary education in OECD countries in 2017, an average of 39% of students completed their bachelor’s programme within its duration (3 years), increasing to only 67% when given 3 additional years (OECD 2019). In the EU 2020 education strategy, one of the goals is to have more individuals completing higher education. Reducing dropout and increasing completion rates in higher education is one of the key strategies for achieving this goal. With this as a backdrop, an externally funded project was initiated as a collaboration between a gamification developer and a 200-year-old, traditional, technical university, here called TechU1. The project aimed to increase the completion rate and reduce student dropouts in higher education by using gamification. The expected result was that the students would have a greater chance of completing the course during the predetermined study period if the course material was gamified. The project hypothesised that the design principles that exist in social media and digital games could also be used to improve student retention in higher education. In the project, a gamification studio assisted the university with the design, implementation and construction of a gamified mechatronics course. Through a series of workshops and courses, the needs of the university were explored. The gamification studio specialised in developing a gamification Application Programming Interface (API) that can be implemented as a widget on top of pre-existing software applications. The results of the workshops became the basis for the implementation of the studio’s API in a specific course. It should be noted that the API in the project was designed to be implemented in miscellaneous software. Therefore, the gamification layer was integrated in a pre-existing learning management system (LMS). The gamification system was therefore separate from the actual product. The design idea was that this approach allows for non-invasive introduction of gamification into a wide range of software. THE COURSE AND THE SPECIFIC IMPLEMENTATION The course in the project was a mechatronics course, which involved four areas of automation strategy: 1) management and analysis of levels of automation, 2) humans’ part in the system, 3) flow simulations and 4) factory layout. The course ran every semester at the university and operated for 9 weeks. The course that was studied had 84 students assigned to it. Most of these students had completed several higher education courses at TechU before. In the syllabus for the course on the TechU webpage, the examination to pass the course required the student to be: • approved on compulsory group submissions • approved on mandatory quizzes • approved on compulsory parts. To pass the course, the students needed at least 80% attendance for laboratory projects, which meant that they could miss one lab; however, they still had to participate in the group assignment, handing in the results from the missed lab. The examination in the course was a mix of individual and group assignments. However, due to the gamification implementation, the teachers running the course altered the syllabus. In the gamified syllabus, the students also needed to do individual work for their grade. Grade 3 Passed all missions for each level, be level 8–10, passed a quiz, read and commented on an article with at least 50 words and made a discussion entry on another group’s submission. -- 4 -- Grade 4 All the above, be above level 10, passed two quizzes, read and commented on two articles with at least 50 words and made two discussion entries on another group’s submission. Grade 5 All the above, be above level 12, passed two quizzes, read and commented on three articles with at least 50 words and made three discussion entries on another group’s submission. Information about the grades and the course criteria associated with the gamification was not published on the TechU webpage nor presented at the course opening. The students learnt about these criteria 5 days after the course had started. What was Gamified? The course at TechU was gamified through an API that was implemented in the university’s LMS (Figure 1). The gamification elements were missions, levels, achievements and shop. Figure 1 The LMS with the gamified API layer The API added a visualisation of student progression through a mission map. This mission map was illustrated with numbered circles connected by dotted lines (Figure 2). The missions typically consisted of different subtasks to be used, such as read an article, do a quiz and write a comment on a discussion board. The mission’s module was constructed such that every day a new set of missions spawned for the students to complete. When the missions were completed, the box related to the task was checked. -- 5 -- Figure 2 The mission module The tasks in each mission could be completed in any order. Every time a task was completed, the students gained experience points (XP). When all the tasks in a mission had been completed, the student was rewarded XP. These XP were needed in order to gain a level. The levels (Figure 3) were designed to correspond with the actual course week; if the course was in its third week, the student’s level was expected to be level 3. This was designed to communicate to the students whether they were following the intended pace of the course. Figure 3 The level module When the student gained a level, they were rewarded with coins, a fictitious currency, that they could use later in a shop module. In the shop module (Figure 4), the students could buy both virtual as well as physical rewards. In the TechU implementation, students could buy: -- 6 -- • • • extended deadlines: “A one-time use voucher for extending your next upcoming deadline” limited-edition t-shirts: “A cool limited-edition TechU T-shirt”2 science avatars: “New cool science-themed avatar pack”. Figure 4 The shop module There was a limit on how many times the different items could be bought. The extended deadline could be bought twice during the entire course. The t-shirt was limited so that only the 10 students who were the first to gain enough coins could buy it. This reward structure, as well as most of the design choices, was devised based on discussions between the teachers of the course and the gamification designers. The achievements modules (Figure 5) were based on a set of different goals. For the TechU implementation, the achievements were designed to engage the students in completing different tasks in the LMS system, such as finishing quizzes (Quizzer) or commenting on other students’ work (Commenter). Figure 5 The Quizzer Achievement -- 7 -- The achievements were connected to different behaviours that the designers and the teachers perceived as important for the students’ probability of success in the course. This design resonated with the whole ambition of the implementation, to provide motivation in order to increase student retention in and reduce dropouts from the course. THE INCIDENT The incident that made the researchers conduct the present specific case study happened during the first day of the course. As mentioned above, one of the ways in which students could gain XP was to read articles. This was operationalised by the API’s recording when a certain link in the LMS was clicked on. The function was supposed to have a cooldown; however, this was mistakenly not activated in the gamified course. This made it possible for a student to click on the same link as much as they wanted, earning increasing amounts of XP. During the implementation, some students created a script that opened a new tab in their browser repeatedly. This script made it possible for them to level up at a tremendous speed and reach levels far beyond what was intended (while a reasonable level would have been between 8 and 12, one of the students who used the script reached level 9073). The incident affected both students and teachers at the university, as well as the employees at the gamification studio. METHOD As soon as the researchers became aware of the incident, they decided to design a specific case study. Departing from Goffman’s theory of frames, the study raised two research questions: How do different actors frame an incident of using a script to earn experience points in a gamified university course? What do such frames reveal about general ideas about gamification among university teachers, students and gamification developers? The study employed multi-sited ethnography (Hannertz 2003; Marcus 1995), in which the unit of analysis is followed across temporal and spatial boundaries. This method makes it possible to circumvent dichotomies such as local–global (Marcus 1995) and virtual–real (Beneito-Montagut 2011). Therefore, any context that was relevant to the research question could be followed. Multi-sited ethnography has been discussed as challenging with regard to conducting cross-context comparisons (Coleman and von Hellerman 2011). However, in this study, the researchers followed a naturally occurring incident and thus had a fixed reference point for the analysis. In total, the researchers gathered the following data: logs from the designers’ internal chat conversations, mail correspondence between students, teachers and gamification studio employees, official public documents (such as incident reports), data on the user behaviour in the API, informal conversations with university teachers, informal conversations with the gamification studio employees, and formal interviews (lasting 31 and 55 minutes, respectively) with two of the four students who wrote the script. Selections were also used from a series of interviews conducted for another study within the project, in which eight students had been asked about their acceptance of the new gamified technology in the course. RESULTS In total, the time period analysed spanned 38 weeks, from the start of the project to the disciplinary board’s decision. During this period, some specific events are noteworthy, as well as their temporal order (Table 1). -- 8 -- Project week Event Research data 1–9 Gamification project starts (designing, developing and implementing the gamified API in the course LMS) Gamification design document Course starts; presentation of the gamification project to the students (week 10, Tuesday) Recording with students from the other study 10 Interviews for the technology acceptance study begin (week 10, Wednesday) Course syllabus Data logs from gamification studio Teacher’s email to students The ‘script situation’ emerges (week 10, Wednesday) The gamification studio gains knowledge of the script; the creation of the achievement (week 10, Wednesday) Teachers at TechU receive the information (week 10, Wednesday– Thursday) Teachers contact all 84 students by email about the new course criteria (week 10, Sunday) 11 The Disciplinary Committee is contacted by TechU teachers. (week 11, Monday) Teacher’s email to students Teacher contacts the 84 students by email about the suspicion of cheating in the course (week 10, Monday) Backlog data from API Student levels reset (week 11, Monday) Interview with the script-makers (week 11, Tuesday) Investigation of the ‘situation’ by the gamification studio (week 11, Wednesday–Friday) -- 9 -- Recording with the script-makers Report from gamification studio to TechU 12 Discussions in the project group Informal discussions with the project group 13 Gamification studio meets all students on the entire course Report from gamification studio to TechU 14–18 Researchers’ informal discussions with teachers and gamification studio employees Informal discussions with teachers at TechU and gamification studio employees Interviews with students in other projects relating to the incident. Recordings with students The Disciplinary Committee hearing and conclusion Informal discussions with teachers at TechU, student ombudsman3 and gamification studio employees 38 Table 1. Timeline of the study. Overall, the results of the study show that different groups establish different frames for interpreting the incident. The different perspectives of gamification that are accentuated by the different actors reveal how different interests as well as different ideologies collide. The Students’ Frame: Resistance During the technology acceptance study, Author 1 learnt about the incident during an interview. Student 4 I expect there will be much problem as it is the first iteration. There have already been problems. Author 1 Do you want to tell me about them? Student 4 Yes, there is a bug. I suppose it is a bug, but there is an assignment where you have to open a document. It does not reset every time you open a document that gives you experience points. So you can create a script where you can click on a link several times a second or so. I have a classmate who is level 3359. Author 1 Excuse me? Student 4 It was yesterday. So within 24 hours, he had reached level 3359. It also meant that he got a lot of Coins. I don't know about these t-shirts. They are probably gone now. This interview took place 2 days after the course start. The information was passed on to the gamification studio about half an hour later. -- 10 -- Exactly how many students used the script is unknown, but data on user behaviour in the API indicates that there were eight students who used some form of script to achieve high levels (Table 2). userId level currentXp totalXp levelXp e71926ad-ac6b-4980-bcbe-4683f7c328ca 9073 6487 139647800 20011 1eec8cc2-9248-4666-81e8-a8b1157ba19f 4690 5767 59217000 16417 6c56cda6-86a7-48c0-9ee3-8e3d58271f22 1570 6263 14272500 11821 136079c7-b4ed-4289-8406-832a812bc549 1417 3394 12488200 11463 f37ee47b-3ef1-4b7d-9c37-06ac9c232318 396 3187 2377700 7818 7f12f524-c897-406a-9a93-f5d955483671 272 2417 1457400 6984 c3f272f5-d635-470e-aafc-225460b1c6ef 48 1516 150700 4140 ead2ab92-02c4-492d-b0f3-53d53d0a7abd 35 2770 100700 3761 Table 2. The gamification studio’s list of suspected script users in the course. The columns, from left to right, show the userid in the gamification API (here anonymised by the gamification studio), the user level in the gamification system, the user’s current XP in their level progression, the user’s total XP and the XP needed to gain the next level. According to the gamification studio’s incident report, the script was mainly used by four of the 84 students: Four (students) reached levels 1417–9073 (1417, 1570, 4690, 9073). Here everything points to that they used an automatically (sic) script. We have looked at the user with level 1570 and average there is an event / 1.6 seconds for about 24 hours. The loophole in the gamification design was exploited after the first day of the course. Two days after the exploits, the researcher conducting technology acceptance interviews, Author 1, was told by several students that cheating had occurred and there was a rumour among the students that someone had passed level 9000 using a script. Author 1 asked the respondents of the technology acceptance study to put him in contact with those who had built the script. After 4 days, two students (referred to as Student 1 and Student 2 in the text) made contact with Author 1 and wanted to talk about “us who were a little too efficient at leveling up” (Student 2, first mail conversation with Author 1). The two students wanted to give their version of what had occurred. The interview with them took place 3 days after they contacted Author 1. In the interview Student 1 compared the API design with a mobile game: It was introduced as a… the lecturer brought it up as service, you are to check these things you need in order to pass a mission. You can also get XP in some different ways, and I would say it felt like a mobile game that I wouldn’t have played. …it really felt -- 11 -- like a game where you were running at the spot. That’s how many mobile games are designed today, they are addictive to some and that’s how they make money. Student 1 The student’s overall perception of the gamified LMS through the interview and as illustrated above was mainly negative. Part of this criticism was tied to the perception that the design mimics mobile games that s/he found had unethical addictive design patterns and monetising models. Another objection, which was also expressed in interviews with other students, was that the students could use progression in the game in order to get extended deadlines: You can buy bonuses as well something that I and many in my class strongly criticised when it was brought up. By doing this you can get coins and with these coins you can buy extended deadline and such… And I simply do not think that Gamification belongs at a university. Student 1 This was something that Student 2 also brought up: But I feel that people talked most about was the extended deadline. It was the real ‘uh oh’. Student 2 At TechU, students with special needs may be granted extended time as an aid in the course or in an exam situation. This could be a possible reason for the students’ reaction. When a similar course adjustment appeared in the gamification shop, it seemed to spark disapproval among the students. The gamification reward extended deadline was discussed among the students in the course, not just the students using the script. I am very fond of the t-shirt idea, but I am very ambivalent about this with extra time and extended deadlines. I think like this, if you study hard to get coins then you may not need an extra deadline, but those who would need extra help and extended deadlines they may not get enough coins. It feels like it will be a positive development for those who are capable, but a negative development for those for who are incapable. Student 3 (excerpt from an interview for the technology acceptance study) Student 1 used a moral framework for criticising the gamified course. For Student 1 , using gamification was unethical and did not belong at a university, as it draws upon similar design features as the mobile game industry which s/he considered unethical. This notion was shared with other students in the gamified course. Student 3 also had complaints based on moral grounds. S/he thought that extended time for handing in assignments should be used in relation to student needs and not as a reward. Student 2 was also critical, but not on moral grounds. S/he described oneself as being mainly negative towards new ideas in the courses and considered the whole idea as laboured: I am often critical to these things or consider it work in progress. But it is not like I go ‘YES’ more like ohh are we supposed to do this now as well. ...it feels kind of laboured and hollow, done just because it was possible. Student 2 As exemplified by the quote, Student 2 utilised a ‘gamification is meaningless’ frame throughout the interview. S/he thought that the whole approach was done for its own sake and found it difficult to perceive it as adding value to their education. In the course, outcomes of the gamified parts were not initially integrated and related to the formal -- 12 -- assessment. It was not until after two course days, after the script had been used, that it was first communicated to the students that the levels they reached would affect their formal grade in the course. Student 2 pointed out that they did not consider the gamified parts as something that would be graded: We did not understand exactly what it [gamification] would contribute with when we started. We hadn’t really understood what the thought was with gamification in the course. It was not clear, as it later became, that it was connected to grades. That some things are related to how active you have been in the game. We didn’t know that when we started instead we saw it as a way of increasing the interaction with the course web and the documents. And spread it over a longer time. Student 2 The students saw the gamification as a layer on top of the ‘real’ examination game (see the discussion above) and not as something integrated into the course, and, as the analysis above demonstrates, they had objections to it. The Students’ Actions: Activism Both the students who used the script expressed that they did not favour the way that the gamified course positioned them as learners. The first thing I thought is that this sets up a way stricter learning approach than I am used to. Student 1 I am much more understanding when it comes from another approach. This time it is more; we make a game for you to learn more, instead of learning the game you are supposed to play. There is a difference. Student 2 The students felt that the design of the course tried to push them into a certain way of studying that they were not comfortable with; the intended study trajectory was not the process they were used to in previous learning situations. This made them question the system. When exploring the gamified content, the students discovered that they were rewarded XP every time they opened some of the pages in the LMS (reading material). Student 1 tells how s/he first found this design comical and then found it possible to exploit. It's like a sketch. Why have they gamified this? It is really bad. First thought. Then we thought more like hmmm if you are first to 2000 coins you can buy a t-shirt. And then it almost became a goal in itself for us, who can build the fastest system to get there. Student 1 These thoughts made the students consider creating a script and lead to an informal competition to find out who could build the most efficient script and collect the t-shirts. Student 2 was asked if s/he saw it as a game. You can say so yes, and a game we found way more exciting than the original. Student 2 The final script that Student 1 wrote was described as rather simple. -- 13 -- In the end I noticed that the system could handle approximately 20 call a second without any problems. Which gave a level every third to fourth second. Then I put it all on a loop and it was not much more to it. As a code, it was like 5–6 rows in the script. Student 1 Given the students’ moral objections to the gamified course design, their actions can be understood as a form of activism. The students did not want to take part in the gamified course and the way for them to express this was by gaming the system. During the interviews, they stressed that the intention never was to cheat. Given the fact that they used the script in order to reach so such over-exaggerated levels, it was their way of demonstrating problems with the course design. As the students said, if they had wanted to cheat, it would have been a bad idea to reach such extreme levels: If you really had wanted to cheat to get something out of it you wouldn’t have gone to level 9000 something. Student 1 No, it is not subtle or camouflaged in any way, it is level 1 to 9000 in two days. Student 2 It’s a completely different thing to cheating, everybody can do it and it is wide open. There is a limit to how much you can do it but the actual action in itself is not cheating. It is just simulating that you press a link. Student 1 Obviously we saw it as excessive. We wanted to show that we could do it. There was no way in the world we thought that this would benefit us in the long run. Student 2 The students thought that their statement would be considered as an innocent prank and at the same time make TechU and the gamification studio self-critically reconsider the course design. However, TechU and the studio framed the incident in quite different ways. The University’s Response: Cheating The teachers at the university received the information about the incident 3 days after the course had started. Four days after the script been exposed, TechU decided to reset all 84 students’ levels to 1. The students received the following mail from the course leader: Hey! Since we have encountered suspicion of cheating in the course, everyone has gone back to level 1. We hope to be able to resolve this as soon as possible. I can still see if you take quizzes and read articles etc. via canvas, so I have a check on you anyway;) You do not have to worry about the grades suffering or that I do not see what you have done so far. I hope we can solve this as soon as possible! Teacher 1 The teacher’s immediate response was to frame the incident as cheating, and thus they reported it to the disciplinary committee at TechU. As a part of the investigation, the studio was asked to provide the names of the students who had written the script. The -- 14 -- studio provided the userid (not the names, which they did not have). With these userid, the teachers were able to track the students who had made the script and to make a case. The course continued for nine more weeks. During this time, the students expressed that they were confused about what role the gamified parts of the course had. Thirty weeks after the incident, the disciplinary committee at TechU heard the case. The students were charged with 1) suspicion of misleading examination, 2) suspicion of disturbance of examination, and 3) suspicion of breaking TechU’s code of conduct. All the charges were dropped, as it could not be clarified whether the students knew that the gamified activities were graded, and hence it could not be proven that any act of insubordination had occurred. According to the protocol from the meeting, one of the students claimed that s/he had been so worried about the charges that it affected their studies. The Gamification Developers’ View and Response: Playfulness The gamification studio found out about the script 2 days after it was implemented. The studio received the information from Author 1 as well from the gamification API (Figures 6 and 7). The script triggered a notification in the system when the traffic on the server was higher than expected. Figure 6 The gamification API interface showing a spike in operations per seconds when the script was activated Figure 7 The gamification API interface showing three spikes in the storage usage on the server: 1) the gamification studio tested the implementation, 2) the course started, and 3) the script was activated. In conversations extracted from the studios’ instant messaging platform, the studio employees demonstrated that they framed the incident as a playful prank. Chat log 1 Henry 12:00 PM Hi, I hear there are many in the Cnvas [sic] course cheating. They spam certain behaviours. Henry 12:00 PM -- 15 -- What shall we do about it? Henry 12:06 PM Some apparently are LvL 80 Rosen 12:06 PM LoL Rosen 12:06 PM I cant see anything of this. Henry 12:07 PM Learnt about it from (Author 1 - Anonymous). Rosen 12:07 PM If we look at it from the bright side, they care about the system. The mood in the chat log was somewhat joyful and the statement “If we look at it from the bright side…”” contained a start for self-criticism from the developers. The studio talked about the incident as if it were a small success, because the students took the time to interact with the gamification API, but also the initial ”LOL” in the conversation (internet slang for ‘laughing out loud’), gives an indication that initial reactions were light-hearted. Approximately 20 minutes later, there was a discussion of the issue in another channel on the instant messaging platform. Chat log 2 Hooch 12:29 PM Best today, There is a bug in the Canvas implementation. You can hand in the same mission many times. Apparently some students at TechU have built a script. – There are students that are LVL 3359. C-Birch 1:03 PM Love TechU. Hooch 1:16 PM Indeed! Hooch 1:18 PM Feels a bit like “I am going to show that middle aged dad who really knows games”. Chat log 2 also had a playful tone. The mood was ironic and self-taunting. The studio sees the situation as a prank from the students which they clearly appreciate. Approximately 1 hour after the gamification studio received knowledge of the incident, the studio, after a short meeting, decided to act against the students. The studio started to discuss giving an achievement (a badge given as a reward for reaching specific goals) for the effort of writing a script (Figure 8). Chat log 3 Rosen 2:02 PM We are going to give our hackers an achievement after wiping their cheated progression. We take the suggestion with most votes at 15. My suggestions is from the classic: [link to scene from Jurassic park] @Rosen has a poll for you! -- 16 -- What shall we call the Achievement we givet [sic] to our dear hackers? 1234 1: ‘Ah ah ah, you didn't say the magic word’ 67% (4) 2: Got caught red handed! 17% (1) 3: Stop trying to hit me and hit me! 4: 40 thousand billion xp [referring to an incident in Swedish politics] 0% 5: Straight in the mother modem, the heart of the hard drive [referring to a Swedish article in a computer magazine from the 1990s] 17% (1) 6: It's Over 9000! 0% Total votes 6 Figure 8 The achievement that the students received after they passed level 99 The appreciation of the incident in the studio was taken even further, as it was suggested that the script writers, for PR reasons, should be offered summer jobs at the studio: Chat log 4 denniscraig 4:55 PM We should actually up this more and build PR. Offer summer jobs to those that hacked us? Can the local newspaper pick it up? Kurtz 5:12 PM Well hacked and hacked... But absolutely, we can do something. Tron 5:52 PM Offer summer jobs would be nice! Kurtz 6:15 PM -- 17 -- In that case I think we should make a challenge of some sort and mail them an anonymous link. The one that first solves it we take. Kurtz 6:15 PM We have their mail addresses. Kurtz 6:16 PM In this way we can see if they actually ”know” something or if they just are script kiddies. [script kiddies written in English] Here, the frame of playful appreciation collided with a notion of being tech savvy and, as Kurtz pointed out, being someone who “actually know something”. The playful frame was combined with a more elitist, hierarchical perception of technological competence (compare with Ottemo, 2015 and the concept of ‘already-passionate subject’). This display of gatekeeping implies that a more serious violation, such as an actual hack and not just a simulation of clicks on the user’s own platform, would have been ‘even better’. The studio hence framed the incident in a significantly different way compared with TechU: the episode that occurred was a prank, creating technical shortcuts is something appreciated, no harm is done, reset, and follow up the prank with an achievement. DISCUSSION The results of this study illustrate how gamification of a course in higher education is a complex affair, where different actors can potentially establish and utilise different frames. Although this was an isolated incident, the study can serve as a backdrop for generating hypotheses about people’s behaviour and perceptions of gamification in higher education. The students’ moral stance and perception of gamification as something done for its own sake could even in other cases spark resistance that leads to similar activism. This result confirms and furthers the findings of Lantz-Andersson and Linderoth (2011) that users of game systems in educational settings are aware of how they are positioned by the designers of the system. In this study, the students objected to being positioned as people who would fall for the unethical mobile game-like design and find it meaningful. One student who wrote a script expressed that gamification did not belong in the university and that it was more suitable in high schools and upper secondary education. TechU is a 200-year-old institution known for its outspoken university spirit, student orchestra and traditional activities. Appreciating gamification did not align with the self-image of being a TechU student. Resistance and activism challenged the didactic contract and gave the tricksters/cheaters/hackers high social status. They objected against an invading progressive way of teaching and protected the identity of TechU students (see also Ottemo 2015). This supports Majuri, Hamari and Koivisto’s (2018) argument that context and personal traits affect the success of gamification. Our study can also be related to Dichev and Dichevas’ (2017) review, in which they argued for the importance of mapping out the grey areas of gamification in higher education. Our study suggests the possibility of a student’s opposition to gamification in higher education. It is uncertain whether the script incident affected the learning of the majority of the 84 students; however, it is likely that it affected their attitude towards gamification. Future gamification implementation projects may benefit from considering that the implementation of a system positions a user. When trying to enhance the motivation -- 18 -- and behaviour of someone with specific design features such as PBLs, these features are built on assumptions of what motivates and drives the user. In other words, the system includes a certain role to be filled by the student. As this study shows, such a role can come into conflict with students’ viewing themselves as high performing. Here, flexible course designs where gamification elements are made optional, as a layer on top of the formal course, may be a strategy worth investigating. As game scholars such as Huizinga (Werbach and Hunter 2012; Huizinga 2004) and Caillois (1958) have emphasised, an essential quality of play is that it is voluntary. When making play mandatory, one of the defining traits of play is omitted; that is, if people are forced to play, they cannot play. Our study also shows that there can potentially be somewhat extreme variations in how different actors frame gamification. The studio’s response, giving an achievement to the students, stands in stark contrast to the teacher’s reporting the students to the disciplinary committee. Hence, future gamification projects can try to avoid these problems. In the case studied here, both TechU and the studio could have benefitted from being more aware of each other’s practices. Finally, our study stress the observation made by Björk (2019) that gamification often happens in settings where pre-existing systems have game features. How the implemented gamification system is merged with the existing systems (for instance, how levels, XP and missions relate to formal modules and grades) can, as this study shows, have a huge impact on user behaviour. This highlights the need for educational institutions to be knowledgeable about game systems. It should be noted that this is a case study made under interesting, yet special, circumstances that limit its generalisability. The findings are, as previously stated, to be seen as first steps towards generating hypotheses about gamification implementation of pre-existing courses in higher education. ACKNOWLEDGEMENTS The study was partly supported by the Swedish innovation agency, Vinnova grant number 2018-02953. ENDNOTES 1 Author one is conducting his PhD within this company where he also is employed. Author one was also the project leader for the Vinnova-project. He participated in the workshops regarding the implementation, however the final course design was made by TechU and the gamification company. Author two has no connection to the company, TechU or the project. 2 This was a physical t-shirt with the university's logo. 3 A person that can provide advice and give assistance in matters concerning the study situation. E.g cases of cheating or suitability issues. BIBLIOGRAPHY Barata, G., Gama, S., Jorge, J. and Gonçalves, D. n.d. Engaging engineering students with gamification engaging engeneering students with gamification: An empirical study. https://www.zombiesrungame.com/. Beneito-Montagut, R. 2011. “Ethnography goes online: Towards a user-centred methodology to research interpersonal communication on the Internet.” Qualitative Research 11 (6): 716–35. -- 19 -- Björk, S. 2019 (8 may). Dark patterns in the design of games. Presentation. Skövde: University of Skövde. Bonde, M. T., Makransky, G., Wandall, J., Larsen, M. V., Morsing, M., … Sommer, M. O. A. 2014. “Improving Biotech Education through Gamified Laboratory Simulations.” Nature Biotechnology 32 (7): 694–97. https://doi.org/10.1038/nbt.2955. Caillois, R. 1958. Man, play, and games. New York: The Free Press. Coleman, S. and von Hellerman, P. 2011. Multi-sited ethnography: Problems and possibilities in the translocation of research methods. London: Routledge. Deterding, S. 2009. “The game frame: Systemizing a Goffmanian approach to video game theory.” DIGRA '09 - Proceedings of the 2009 DiGRA International Conference: Breaking New Ground: Innovation in Games, Play, Practice and Theory, London: Brunell University. Deterding, S., Khaled, R., Nacke, L. E. and Dixon, D. 2011, May. “Gamification: Toward a definition.” In CHI 2011 gamification workshop proceedings. Vancouver BC, Canada, 12–15. Dichev, C., and Dicheva, D. 2017. “Gamifying education: What is known, what is believed and what remains uncertain: A critical review.” International Journal of Educational Technology in Higher Education. https://doi.org/10.1186/s41239-0170042-5. September 28–30, 2011): 9–15. Domínguez, A., Saenz-de-Navarrete, J., de-Marcos, L., Fernández-Sanz, L., Pagés, C., & Martínez-Herráiz, J.-J. 2013. “Gamifying learning experiences: Practical implications and outcomes.” Computers & Education 63: 380–392. Goffman, E. 1986. Frame analysis: an essay on the organization of experience. Boston: Northeastern Univ. Press Hannerz, U. 2003. “Being there... and there... and there! Reflections on multi-site ethnography.” Ethnography 4 (2): 201–16. Hakulinen, L., Auvinen, T. and Korhonen, A. 2015. “The effect of achievement badges on students’ behavior: An empirical study in a university-level computer science course.” International Journal of Emerging Technologies in Learning 10 (1): 18–29. https://doi.org/10.3991/ijet.v10i1.4221. Huizinga, J. 2004. Den lekande människan: (homo ludens). (2. utg.) Stockholm: Natur och kultur. Juul, J. 2003. Half–real: Video games between real rules and fictional worlds. Unpublished thesis, IT–university Copenhagen, Copenhagen. Koivisto, J., and Hamari, J. 2019. "The rise of motivational information systems: A review of gamification research." International Journal of Information Management 45: 191-210. -- 20 -- Knutas A., Hynninen, T., Wolff, A. and Khakurel, J. 2019. “Exploring the connection between gamification and student engagement in computer-supported collaboration.” Accessed July 2, 2019. http://ceur-ws.org/Vol-2359/paper1.pdf. Lantz-Andersson, A. and Linderoth, J. 2011. “The ’voice’ of absent designers.” Nordic Journal of Digital Literacy 6, (1/2): 52–74. Majuri, J., Koivisto, J. and Hamari, J. 2018. “Gamification of education and learning: A review of empirical literature.” CEUR Workshop Proceedings 2186 (May): 11–19. Marcus, G. E. 1995. “Ethnography in/of the world system: The emergence of multisited ethnography.” Annual Review of Anthropology. 24: 95–117. Miller, C. M. L. and Parlett, M. 1974. Up to the mark: A study of the examination game. London: Soc. for research into higher education. O’Donnell, N., Kappen, D. L., Fitz-Walter, Z., Deterding, S., Nacke, L. E. and Johnson, D. 2017. “How multidisciplinary is gamification research?” In Extended Abstracts Publication of the Annual Symposium on Computer-Human Interaction in Play - CHI PLAY ’17 Extended Abstracts, 445–52. New York, New York, USA: ACM Press. https://doi.org/10.1145/3130859.3131412. OECD. 2019. Education at a Glance 2019: OECD Indicators. Paris: OECD Publishing. https://doi.org/10.1787/f8d7880d-en. Ottemo, A. 2015. Gender, body, desire, and technology: Passion and instrumentality in two technical university programs. Diss. Gothenburg: University of Gothenburg Vossensteyn, H., Kottmann, A., Jongbloed, B., Kaiser, F., Cremonini, L. ... Wollscheid, S. 2015. Dropout and completion in higher education in Europe main report. Luxembourg: Publications Office. https://doi.org/10.2766/826962. Werbach, K. and Hunter, D. 2012. For the win: How game thinking can revolutionize your business. Philadelphia, PA.: Wharton Digital Press. -- 21 --