The Flipped MOOC: Using Gamification and Learning Analytics in MOOC Design—A Conceptual Approach

Abstract

Recently, research has highlighted the potential of Massive Open Online Courses (MOOCs) for education, as well as their drawbacks, which are well known. Several studies state that the main limitations of the MOOCs are low completion and high dropout rates of participants. However, MOOCs suffer also from the lack of participant engagement, personalization, and despite the fact that several formats and types of MOOCs are reported in the literature, the majority of them contain a considerable amount of content that is mainly presented in a video format. This is in contrast to the results reported in other educational settings, where engagement and active participation are identified as success factors. We present the results of a study that involved educational experts and learning scientists giving new and interesting insights towards the conceptualization of a new design approach, the flipped MOOC, applying the flipped classroom approach to the MOOCs’ design and making use of gamification and learning analytics. We found important indications, applicable to the concept of a flipped MOOC, which entails turning MOOCs from mainly content-oriented delivery machines into personalized, interactive, and engaging learning environments. Our findings support the idea that MOOCs can be enriched by the orchestration of a flipped classroom approach in combination with the support of gamification and learning analytics.

1. Introduction

“Education is a fundamental human right and essential for the exercise of all other human rights. It promotes individual freedom and empowerment and yields important development benefits” [1]. With the aim of bringing and democratizing education for all, phenomena such as Open Education and in particular Massive Open Online Courses (MOOCs) have started to grow in recent years; MOOCs could hypothetically bring education to everyone and for free. Despite this potential, MOOC studies have highlighted several limitations, such as the lack of users’ engagement and their high dropout and consequently low completion rates [2].

Recent studies suggest looking at a completion rate as an indicator of MOOC learners’ intentions [3,4], and until now the success of MOOCs has been mainly measured in terms of course completion, counting each participant who has not completed the whole course with its assignments and exams as a dropout factor. We argue that studying completion in MOOCs requires a different perspective. MOOCs are not traditional courses. The fact that they potentially can reach a massive number of people means dealing with and catering for an enormous variety of needs. From this perspective, the user interest and intention have to be considered in the design phase.

MOOCs that take the diversity of learners into account need to be personalized, engaging and motivating. Various concepts have been proposed [5,6] to improve MOOCs using community-oriented approaches. Despite the availability of different types of MOOCs, however, the majority of them still deliver learning content via video and as a result MOOCs have become content containers and repositories of material. If content is delivered in a monotonous way without taking advantage of advanced technological and didactical approaches it will be hard for users to remain engaged.

Consequently, one of the key challenges in MOOC design is to overcome the problem of being attractive to mass audiences and at the same time it has to be relevant to a large variety of individual needs, requirements, and expectations. Therefore, we conceptualize the idea of a flipped MOOC based on the concepts of flipped classroom, gamification, and learning analytic. In line with the technology-enhanced learning perspective in this article, we aim to address the issues of low student engagement, low completion rates, and high dropout rates by proposing a MOOC design solution that makes use of successfully applied pedagogical concepts known from other educational fields: flipped classroom, gamification, and learning analytics. We will briefly introduce these concepts and their current state as reported in literature in the following paragraphs.

The “flipped classroom” is a model within the blended learning delivery where direct instruction happens online and interaction is planned for classroom activities. This approach is often described in terms of Bloom’s taxonomy [7] where the first two phases (remembering and understanding) happen online through interaction with the content and direct instruction. Flipping a classroom implies focusing on interactive and active support of teacher and active learning from the students’ side. In this context, students are required to engage with the content independently. While student perception reports on the use of flipped classroom are mixed, we know that video lectures are not preferred to in-person lectures [8]. On the other hand, students do prefer interactive classroom activities. Therefore, what is mainly flipped is the active role of the teacher in passing the concepts and content knowledge, which is left up to the student, at his/her own pace. It is not the teacher that lectures and transmits his/her own vision on the specific topic but the student by reading and studying the material, while the teacher provides help to the student to develop his/her own understanding during the in-class time, where there is time to discuss.

There is some evidence suggesting that the student learning is improved by the flipped mode and that it leads to higher cognitive outcomes [9,10,11]. Other findings suggest that a MOOC-based flipped class is a good solution to promote students’ motivation and learning, but the implementation of this teaching strategy is delicate and must be very well planned [12]. Some authors indicate that the use of flipped MOOCs is promising not only in terms of effectiveness but also the motivation of the students and teachers [13] and increased satisfaction [14]. Meanwhile some authors are more skeptical and suggest that flipped learning (FL) effectiveness is yet to be proven [11]. It is our aim to transfer the concept of the flipped classroom to MOOC design in order to achieve MOOCs that are more engaging, interactive, and that provide a more personalized learning experience. A MOOC designed in such a way can be called a “flipped MOOC”. The following two concepts described here represent the building blocks for the first concept (flipped classroom), which we aim to explore to design flipped MOOCs.

“Gamification” is the application of game elements in a non-game scenario to solve a problem or induce a change in the behavior of the target population [15]. Gamification has a lot of potential in education [16]. Its effectiveness has been studied by several authors and from these studies it has been suggested that the gamified approach can generate several effects, for instance, on users’ performance [17,18,19,20,21], motivation [22,23,24], engagement and enjoyment [25,26,27]. In the last 10 years gamification has been applied in several fields, such as: marketing, trading, education, training, fitness, software engineering, etc. The highest number of applications can be found in education [28,29].

Conceptual examples of gamification applied in MOOC environments are also available, such as the approach by Chang and Wei [26], who try to address the problem of the most suitable game mechanics to enhance users’ engagement. However, as other authors highlighted, the game elements most commonly found in implemented gamification solutions are points, badges and leaderboards (known as PBL), which are rewards given to participants who mainly derive their motivational potential from social comparison theory [27,30].

Similar to flipped learning (FL) approaches, gamification also requires conscious design decisions. In particular, it is fundamental to know the problem that characterizes the context of application, in our case MOOCs, and the awareness of the expected impact that determinate game elements can generate on users’ behavior.

The third and last concept “learning analytics” (LA), is “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (Society of Learning Analytics Research (SoLAR)). Learning analytics here is introduced as an instrument that supports game elements applied in a gamification approach to deliver personalized and interactive learning experiences. As a rapidly developing field, LA promises to offer new types of data analytics, including new assessment methods [31]. On the one hand, students have increasing needs to interpret activities given by teachers and engage with study material; on the other hand, teachers need to make informed decisions about the classroom phase activities and, overall, inform the learning design [32]. Within the context of our research, LA use has twofold aims: (1) to envision how LA can support learners through meta-cognitive capabilities since students in a FL setting have more elevated necessity to reflect on their own learning than in a traditional setting [33]; and (2) to support teachers to create a feedback loop between online and classroom phases and assess the learning of students. This can be done through dashboards or retrospective analysis [34]. In the context of our approach towards designing flipped MOOCs, the role of learning analytics is to explore the potential of feeding learner-generated data into the selected game elements to foster personalization, interactivity, and engagement.

Flipping a MOOC requires planning, explicit design and orchestration of several elements (such as gamification and the learning analytics enrichment) to achieve the desirable results it can offer [12]. Our idea of flipping a MOOC has been presented during a workshop in an international conference. This idea was developed by the authors by analyzing the concepts of flipped classroom, gamification and learning analytics.

In the context of a flipped classroom, active learning strategies are required from students during the pre-classroom activities enabling them to engage and fully participate then in the classroom. In this way, discipline and self-directed learning skills are required from the student to remain engaged in these pre-classroom activities [35,36]. “Many students have under-developed self-regulation skills and need support and scaffolding to manage their learning in less familiar and more intensive settings that often characterize FL designs” [34]. This statement is even truer in the context of MOOCs in general, and the flipped one in particular. We believe that gamification can help users stay engaged and manage learning in accordance with their intentions.

In previous studies [2,29], we have involved experts from different disciplines (game design, learning sciences, and technology-enhanced learning) to assess the first selection of game elements [2,30] suitable for MOOC. As a result, we have identified ten game elements suitable to be applied in the MOOC environment designed for FL. These game elements are: (g1) empowerment; (g2) smooth learning curves; (g3) communication channels; (g4) levels; (g5) clues; (g6) goal indicators; (g7) skills tree; (g8) guilds; (g9) storytelling; and (g10) stimulated planning.

We have given these game elements to our workshop participants and presented them with the concept of flipped classroom and learning analytics. We have asked them to design a flipped MOOC and to apply the concepts explained above in their design. In this paper, we report on this study, describing the method applied and the results achieved. We then briefly discuss these results as well as the limitations of the study conducted and then draw some conclusions.

2. Materials and Methods

In the context of in an international conference dedicated to MOOC stakeholders, we have used the outcomes of the previous research, as indicated in the introduction, to design a creative expert design study, in which educational experts were briefly introduced to the concepts of MOOCs, flipped classroom, gamification, and learning analytics. Furthermore, we have presented the results of our previous work, which lead to the list of game elements presented above. We then asked the participants to apply the presented concepts in the design of a flipped MOOC aiming at creating an engaging and personalized learning experience.

2.1. Participants

Nineteen professionals with several years of experience in education participated in our study. The participants were all either practicing teachers in higher education or involved in educational research, therefore we refer to them as educational experts; however their backgrounds differ significantly with respect to the design of MOOCs, the concept of flipped classroom, or the application of gamification concepts in educational design.

About one third of the participants had prior experience in designing MOOCs or comparable experience in online learning content creation, two had experience applying a flipped classroom approach in MOOCs, and one participant had experience in an on-campus use of MOOCs. Others were more experienced in classroom-based teaching. Two of the participants had active experience in gamification, a few more were familiar with the concept of gamification previously, none were acquainted with learning analytics application.

All participants were informed that they were taking part in a study, that we were collecting the results of their participation and that their anonymized outcomes will be used to prepare a scientific publication.

2.2. Procedure

We organized the participants into small groups (4–5 participants per group) and assigned them the task of designing and conceptualizing a flipped MOOC using gamification and learning analytics applications (see Figure 1).

The participants were asked to first select a topic for their MOOC and specify how would they roughly organize the content into main sections and how much content would be required for their MOOCs.

As a next step, participants had to discuss which part of the MOOC they wanted to flip, which game elements presented to them (see

Table 1. Game element descriptions and design guideline questions.

Game Element	Description	How Do We Use It?
1. Empowerment	Enables a user to perceive that he/she can impact the situation, have a sense of control	What are the decisions an empowered learner can take?
2. Smooth Learning Curves	Keeps the user in a continuous learning progress and in mastering his/her skills	How can we smoothen learning curves for different learners?
3. Communication Channels	Enables communication via chats, forum, etc.	Which communication channels are specifically needed?
4. Levels	Part of the game in which all player actions take place until a certain goal has been reached or an end condition has been fulfilled	How do we distinguish between the different levels? What is specific for each level?
5. Clues	Hints given to users to solve a problem and/or to achieve a goal and unblock the next level	How do we balance clues (not too many, not too few, being informative without giving away too much information)?
6. Goal Indicators	Indicators, which make the goals explicit	Which goals can be selected/defined?
7. Skills tree	Enables custom configurations of a character’s abilities, often organized in branches	Which skills can we distinguish? How can we organize them in a hierarchy?
8. Guild	Associations of players who chose to come together to achieve a common goal	How can learners team up in guilds? What is the goal of each guild? How does it compete with other guilds?
9. Storytelling	The act of telling stories within the game	Which story do we want to tell?
10. Stimulated Planning	Enabling and stimulating users to plan future actions or goals and thus guiding them to define the strategy to follow in order to complete a level or the game itself	In a flipped MOOC how do you foresee the implementation of Stimulated Planning, a game elements that invite the users to plan their strategy?

) they deemed most suitable with respect to their selected topic and their specified size of the MOOC (in terms of content and structure). Based on this selection, we also asked the participants to conceptualize the game elements in order to give a first impression of a possible implementation of the game elements in the MOOC.

Finally, we asked our participants to connect their design to learning analytics. We asked them to indicate what kind of learner data is required to support the selected game elements and how this learner data could be acquired from the learners and teachers.

The whole study lasted three hours, with 30 min spent on the introduction, assignments, and group forming. Participants had then 105 min of time to work on their assignment in the groups and to prepare their mock-up. The final 45 min were used for group presentations, follow-up discussions, and a short wrap-up session.

In order to prepare the participants for their assignment, in addition to the initial presentations, we distributed information on different game elements proposed and added guiding questions, which aimed at helping the participants design or conceptualize the selected game elements in a concrete way (see Table 1).

Furthermore, participants received guiding questions for applying learning analytics to their design and for designing the whole course in terms of domain, topic, and target group (see

Table 2. Guiding questions for learning analytics and the course design.

Learning Analytics Conceptualization
For each game element selected:	Why do we use it?
	For whom?
	How do we use it?/how do we implement it?
	What data do we need to support that?
Design Phase
Domain	In which domain is our MOOC placed?
Topic	Which topic(s) should it cover?
Target group in mind	For which target group do we design it?

). Participants were asked to document and present their findings as part of the group assignment by answering the guideline questions.

3. Results

Four participant groups delivered answers to the guideline questions stating their flipped MOOC concept based on the application of gamification elements and learning analytics concepts. All the results received are provided as Supplementary Materials (see link to Supplementary Materials at the end of this article).

Each group selected their own topic, covering: “Engaging students in the 21st century” (Group 1), “Organizational models” (Group 2), “How to write a dissertation” (Group 3), and “Basic Japanese” (Group 4). All groups selected higher-education target groups including teacher training cohorts (group 1), engineering students (Group 2), undergrad and/or post grad students (Group 3), and students visiting Japan (Group 4). The chosen domains where indicated as “Education for the future” (Group 1), “Dissertation” (Group 3), and “Language Learning” (Group 4), group 2 did not indicate the domain (see

Table 3. Groups, topics, and domains.

Group	Topic	Domain
Group 1	Engaging students in the 21st century	Education for the future
Group 2	Organizational models	-
Group 3	How to write a dissertation	Dissertation
Group 4	Basic Japanese	Language Learning

).

Due to the limited time of the study, only two groups worked on the LA-related part of the assignment, while all four groups worked on the gamification part. For the same reason, the groups were not able to provide a more detailed conceptualization of their MOOCs or to dive into content related designs beyond high-level considerations.

3.1. Game Elements’ Selected

All four groups chose the game elements “Levels” and “Goal indicators” in their concepts. Three of the four groups each decided to use the game elements “Empowerment”, “Smooth learning curves”, and “Communication channels” in their concepts. Two groups decided for “Guilds”. One group has chosen “Skills tree” or “Storytelling” respectively. None of the groups picked “Clues” or “Stimulated planning” (see summary on

Table 4. Use of game elements by the groups.

Game Elements		1	2	3	4	Total
	Groups
1. Empowerment		Y	Y	N	Y	3
2. Smooth Learning Curves		N	Y	Y	Y	3
3. Communication Channels		Y	Y	N	Y	3
4. Levels		Y	Y	Y	Y	4
5. Clues		N	N	N	N	0
6. Goal Indicators		Y	Y	Y	Y	4
7. Skills tree		Y	N	N	N	1
8. Guild		N	Y	Y	N	2
9. Storytelling		N	N	N	Y	1
10. Stimulated Planning		N	N	N	N	0
Total Patterns per Group		5	6	4	6	-

). All groups decided to apply four to six of the game elements from the list (Table 1) to their gamification concept.

3.2. Game Element Design Implication

Game element: Levels. All four groups applied the game element “Levels”. However, the way levels are conceptualized vary strongly among the different design concepts. Group 1 used successful quiz results as an enabler for accessing new content boxes (and, therefore, level). Group 2 conceived these as challenges of increasing complexity. Group 3 aligned levels to educational levels (under-graduate/post-graduate) and defined metaphors as cake ingredients, which all have to be mastered to be able to bake the cake (= graduation) at the end. Group 4 assigned levels to an initial assessment, which enables learners to choose topics of interest.

Game element: Goal indicators. Also, all four groups used goal indicators. Group 1 defined a set of different goal categories (intermediate goals, personal goals, community goals), which impose different activities to be achieved. Furthermore, the group connected goal indicators to the game element “Empowerment” by allowing learners to choose their own goals and consequently individualize their learning path. Feedback on the goal achievement status should be visible to learners. Group 2 related goal indicators to challenges and linked real world organizational systems to the goal indicators. Group 3 linked goal indicators to the core learning objectives of the MOOC, where each learning session corresponds to reaching a specific goal. Group 4 linked goal indicators with levels, where each level has a specific set of goals.

Game element: Empowerment. Three groups used this game element. Group 1 designed various input and interaction methods for different learners, allowing the learners to individualize their access to the course content. Group 2 envisioned a content related simulation system, where learners can manipulate core simulation parameters. Group 4 related empowerment to the learning path by allowing learners to freely navigate the whole course after performing the initial assessment.

Game element: Smooth learning curves. Three groups decided to use smooth learning curves as a part of their design. Group 2 applied smooth learning curves to the varying and increasing complexity of the simulation system. Group 3 provided different levels of quizzes, which can be taken. Group 4 bases their smooth learning curve approach on initial basic assessments.

Game element: Communication channels. Three groups have chosen this game element. Group 1 did not however, provide a design consideration for this game element. Group 2 aimed to enable forum and chat-based communication channels in their MOOC. Group 4 proposed separate topic-related discussion rooms. Based on participant interests, different discussion rooms can be offered to the learners.

Game element: Guilds. Two of the four groups included guilds in their concept. Group 2 aimed to let learners self-organize into groups, which have to solve specific challenges together. The whole group would have received public feedback to their challenge result. Group 3 rather proposed an automatic assignment of participants to guilds based on subject chosen.

Game element: Skills tree. One group opted for this game element by providing a visualization of acquired skills to the learner according to a chosen scenario.

Game element: Storytelling. One group aimed to use storytelling for language learning, by envisioning stories to be developed by the participants in a collaborative manner. This way, the group connected storytelling with communication channels.

Game elements: Clues, Stimulated Planning. None of the groups selected one of these patterns and consequently no design concept has been provided.

3.3. Learning Analytics Concepts

Two of the groups worked on the learning analytics part of the assignment (Group 1 and Group 2) and specified what data can be used to support selected game elements and how the game elements can be affected by learning analytics.

Group 1 specified a number of data sources, which can be used for learning analytics, mainly based on course interactions (content access, time spent, activity completion, interaction, and communication), and which can support personalization of learning processes. Furthermore, this data can be used to adapt the game elements in order to address engagement and retention issues. The data can also support reflection and debriefing phases.

Group 2 aimed to support the game elements “Empowerment”, “Communication Channels”, “Levels”, “Goal indicators”, and “Guild” with specific learning analytics approaches. For the game element “Smooth Learning Curves” group 2 does not foresee learning analytics support.

Empowerment is supported by connecting simulation data with real-world data to generate immediate feedback on key performance parameters of the simulation. In the case of communication channels, learning analytics helps to assess individual contributions in relation to overall group activities. The process of leveling up is supported through the analysis of learner progress data (“Levels”), which also fosters the display of current goal achievement status (“Goal Indicators”). For “Guilds”, measuring intra-group interaction (based on chat activity) as well as the challenge completion rate is proposed.

4. Discussion

Selecting and designing game elements and applying them to educational contexts are interdisciplinary, creative, and complex processes. Game elements allow for various ways of implementation and visualization but also generate interdependencies and possible conflicts. Some game elements, such as “Empowerment”, allow for learner control in process and progress and thus support personalization, while others, i.e., “Storytelling”, rather require the following of a predefined path to provide a meaningful story. While both game elements can be motivating for learners, it is difficult to combine both game elements in one MOOC [29].

In order to assess the possible value of specific game elements for their application in flipped MOOCs, we have followed a multi-step research approach involving a literature review, expert studies with experts of various domains (learning sciences, game design, technology-enhanced learning) to narrow down the amount of game elements to a promising selection [30,37].

This previous work lead to the selected list of game elements and also resulted in the design guidelines used to utilize these elements in the context of MOOCs. Clearly, the selected game elements are not exclusive and the design guidelines given to the participants are not the only way to reflect on specific game elements and their use in MOOCs. We are aware that the guidelines might produce a specific bias to the participants. However, since the selection of game elements and the corresponding guidelines are based on previous studies, we support that they help inform the participants reflect on the utilization of specific game elements and provide orientation considering rather short timeframe of the study [2,30,37].

We are also aware that we combined complex concepts into a single intervention (MOOCs, flipped classroom, gamification, learning analytics), and thus ran the risk of acquiring hard-to-compare and hard-to-measure results. However, since the aim of this study was to understand the concept of a flipped MOOC and derive corresponding design implications from the point of view of educators, we believe that our study still contributes to the understanding of the key concepts and their application.

Based on the pre-selection, we here reported a further study with educational practitioners and educational researchers to understand better, which particular design considerations can help to implement our concept of flipped MOOCs.

With nineteen participants split in four groups, our expert study cannot, of course, conclusively rate specific game elements as better apt for the purpose than others. However, we can use the expert recommendations and their design input as indicators for the design of a flipped MOOC concept based on gamification and learning analytics. As such, this study does not contribute empirical evidence about the potential effects of the flipped MOOC approach and it does not claim to deliver the best possible conceptualization of a flipped MOOC based on the underlying gamification and learning analytics concepts. However, we argue that this study contributes to the understanding of the design of flipped MOOCs as a conceptual idea to create more interactive learning and personalized experiences with MOOCs. Based on the expertise of a group of educational experts with various backgrounds, we could indicate ways towards the design of flipped MOOCs.

With levels, goal indicators, empowerment, smooth learning curves, and communication channels as most chosen game elements, our experts clearly voted for game elements, which are engaging (levels, communication channels), motivating (goal indicators, smooth learning curves), and personalization oriented (empowerment, goal indicators). These game elements are expected to foster interaction, communication, and self-organization in the learning processes.

Our concept of flipped MOOC relies on turning MOOCs from mainly content-oriented delivery machines into personalized, interactive, and engaging learning environments with support from selected game elements and learning analytics. This comes along with a shifted perspective on success in a MOOC: while previous measures mainly accounted for completion rates or certificates earned, modern approaches take user intentions into account and calculate individual goal achievement rates [2,38,39], only with this shift in success measurement, the flipped MOOC is possible: individualizing the learning experience does not necessarily deliver comparable results in terms of completion rates. Consequently, a personalized MOOC would be rated as less successful. Taking the individual intentions, expectations, and goals into account may balance this view [39]. By selecting gamification elements, which foster these personalized experiences, our participants confirm this view.

The outcomes produced by the educational experts included in our expert design study mainly indicate the relevance of this new concept. The participants were also able to apply the presented concepts (MOOC, gamification, learning analytics, flipped classroom) into their conceptualization and design of a flipped MOOC for a self-chosen topic. All groups mainly followed the guideline questions given to them (Table 1 and Table 2) and answered in textual form. Group 3 additionally created a visual presentation of their findings presented during the discussion round.

The groups spent less of their time on the aspects related to learning analytics. This is, as participants reported, partly due to short time given to them but also due to learning analytics being introduced in relation to the gamification concepts selected. This way, participants had to first tackle the gamification design before they could address the LA-related issues involved.

The differences found in the various concepts mainly support the statement that designing game elements for learning environments is still a creative and complex routine, which is open to variations. Also, we can see that game elements chosen have been selected partly with the domain and topic of the MOOC in mind: group 1, focusing on student engagement, mainly selected game elements for communication and personalization (empowerment, communication channels, skills tree, goal indicators, and levels). Group 2, working on simulations for organizational models related their choice of game elements to the simulation experience and connected the design of their MOOC to real-world data giving feedback on simulation progress and success. Group 3 organized their MOOC about writing a dissertation on levels according to the academic procedures and thus designed a strong connection of learning process and game element. Group 4, dealing with learning the Japanese language, applied storytelling as a key game element to learning scenarios. They have incorporated collaborative approaches to develop and tell these stories as a part of the community approach, which is, in fact, one possible way to solve the abovementioned conflict between empowerment and storytelling. All of the four groups claimed that they have enhanced student engagement, community building, and interactivity was their guiding principle. Yet the selected topic and domain yielded varying concepts and designs.

5. Conclusions

As a key contribution of this article, we aim to introduce the concept of a flipped MOOC based on gamification and learning analytics as a way to provide more engaging, personalized, interactive, and community-oriented learning experiences in MOOCs. Based on a series of expert studies, we presented and discussed a set of design concepts for flipped MOOCs utilizing the mentioned concepts serving as input for the participants of our study.

While there is a clear overlap in the selected game elements educators used to design their MOOC, the concrete design considerations vary according to topic and domain considerations but also according to individual creativity and design ideas.

As a matter of fact, our study is clearly limited due to the short period of time used and due to the limited number of participants included. Nevertheless, we believe this large variety is also a confirmation of the complexity involved in applying game elements to educational processes. Furthermore, we believe that this variety is an indicator, that gamification should not be applied in a content- or topic-agnostic way, but rather support the concrete educational approach chosen by the designer of a MOOC. Exactly at this point, we see a major shift away from previously dominant content-agnostic gamification approaches that rather relied on extrinsic motivational factors triggered by game elements such as points, badges, and leaderboards, for which long-term effects are at least questionable [40]. The set of expert studies we conducted confirm our view that gamification concepts should also support intrinsic motivational factors, engagement, community-building and personalization. In line with previous results [37,41,42,43,44,45], the designs created by our participants indicate that gamification can be helpful in education only if it is carefully selected and designed chosen according to the application scenario at hand (problem to be solved/addressed, target group, topic, domain, content).

In the context of our flipped MOOC concept, we thus believe that a well-balanced application of game elements, selected to match the topic, the domain, and the target group of the MOOC and in particular the problem that needs to be solved, can deliver an engaging, personalized learning experience. Learning analytics serves as an underlying key concept to relate individual learner data to overall learner activity, which is required for many of the game elements to reveal their full potential.

As a next step, we plan to design a MOOC in accordance with the instructional design concept presented in this paper. Within that flipped MOOC we will experimentally evaluate effects on users’ engagement, community-building, and goal achievement.