Barzilai & Blau 2016

Computers & Education 70 (2014) 65–79
Contents lists available at ScienceDirect
Computers & Education

journal homepage: www.elsevier.com/locate/compedu
Scaffolding game-based learning: Impact on learning achievements,

perceived learning, and game experiences
Sarit Barzilai a, *, Ina Blau b
a
Department of Learning, Instruction, and Teacher Education, Faculty of Education, University of Haifa, Mount Carmel, Haifa 31905, Israel
b
Department of Education and Psychology, The Open University of Israel, 108 Ravutsky St., Ra’anana, Israel
a r t i c l e i n f o a b s t r a c t
Article history: One of the central challenges of integrating game-based learning in school settings is helping learners
Received 6 May 2013 make the connections between the knowledge learned in the game and the knowledge learned at school,
Received in revised form while maintaining a high level of engagement with game narrative and gameplay. The current study
2 August 2013
evaluated the effect of supplementing a business simulation game with an external conceptual scaffold,
Accepted 5 August 2013
which introduces formal knowledge representations, on learners’ ability to solve financial-mathematical
word problems following the game, and on learners’ perceptions regarding learning, flow, and enjoy-
Keywords:
ment in the game. Participants (Mage ¼ 10.10 years) were randomly assigned to three experimental
Interactive learning environments
Simulations conditions: a “study and play” condition that presented the scaffold first and then the game, a “play and
Teaching/Learning strategies study” condition, and a “play only” condition. Although no significant gains in problem-solving were
Elementary education found following the intervention, learners who studied with the external scaffold before the game per-
Improving classroom teaching formed significantly better in the post-game problem-solving assessment. Adding the external scaffold
before the game reduced learners’ perceived learning. However, the scaffold did not have a negative
impact on reported flow and enjoyment. Flow was found to significantly predict perceived learning and
enjoyment. Yet, perceived learning and enjoyment did not predict problem-solving and flow directly
predicted problem solving only in the “play and study” condition. We suggest that presenting the scaffold
may have “problematized” learners’ understandings of the game by connecting them to disciplinary
knowledge. Implications for the design of scaffolds for game-based learning are discussed.
Ó 2013 Elsevier Ltd. All rights reserved.
1. Introduction
One of the central challenges of integrating game-based learning in formal learning settings is helping learners and teachers make the
connections between the knowledge learned in the game and the knowledge learned at school (Clark et al., 2011; Habgood & Ainsworth,
2011; Quintana et al., 2004; Van Eck, 2006). A possible instructional approach to this challenge is supplementing games with external
scaffolds that support the construction of links between game content and disciplinary content (Charsky & Ressler, 2011; Garris, Ahlers, &
Driskell, 2002; Honey & Hilton, 2011). However, adding a scaffold to a game might negatively impact learners’ perceptions of learning and
enjoyment in the game (Broza & Barzilai, 2011; Charsky & Ressler, 2011). Therefore, we need a better understanding of the cognitive and
affective effects of scaffolding game-based learning in order to successfully design scaffolds which promote learning while maintaining
enjoyment and flow. In this study we set out to explore the effects of adding an external conceptual scaffold, which introduces formal
knowledge representations, to a business simulation game on learners’ ability to solve financial-mathematical word problems following the
game, and on learners’ perceptions regarding learning, flow, and enjoyment in the game.
1.1. Informal and formal knowledge representations in game-based learning
Well-designed games engage learners in authentic and enacted problem-solving and in situated meaning making (Barab et al., 2007;
Gee, 2007, 2009). Thus, one of the main challenges of learning game design is to deeply integrate or embed learning content within the
* Corresponding author.
E-mail addresses: sarit.barzilai@edu.haifa.ac.il (S. Barzilai), inabl@openu.ac.il (I. Blau).
0360-1315/$ – see front matter Ó 2013 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.compedu.2013.08.003
66 S. Barzilai, I. Blau / Computers & Education 70 (2014) 65–79
game narrative and mechanics (Clark & Martinez-Garza, 2012; Habgood & Ainsworth, 2011; Ritterfeld & Weber, 2006). However, deeply
integrating or embedding the learning content within the game, while creating opportunities for motivating and meaningful learning, may
also pose difficulties in terms of learners’ knowledge construction. Learners’ understandings of the core ideas and relationships within the
game may be tacit and enacted, and may not be spontaneously translated to explicit understandings that can be related to the knowledge
learned at school (Clark et al., 2011). Furthermore, in games knowledge is often intuitively and implicitly represented in the game visuals,
mechanics, controls, and feedback mechanisms, while at school, learners are required to engage with explicit knowledge representations
that involve formal language and symbolism (Clark et al., 2011; Clark & Martinez-Garza, 2012). This representational gap needs to be
understood in the context of the general difficulties learners face in mapping between their intuitive understandings and disciplinary
concepts and representations (Reiser, 2004). Some games address this difficulty by integrating formal knowledge representations within the
game (e.g., Clark, Nelson, D’Angelo, Slack, & Martinez-Garza, 2010; Habgood & Ainsworth, 2011; Holbert & Wilensky, 2012). However, such
formal representations are usually not available in commercial-of-the-shelf games (Sandford, Ulicsak, Facer, & Rudd, 2006) and in many
educational games as well.
A case can be made that the intuitive and informal learning enabled by games is sufficient in itself, since players who succeed in the game
demonstrate their ability to solve complex problems effectively. However, connecting intuitive understandings of the game to the abstract
and formal knowledge learned at school may serve several related objectives. First, creating such connections might help learners develop
their concrete understandings of the game into more abstract and global understandings of the game model and its underlying principles
(Clark & Martinez-Garza, 2012; Gee, 2009; Parnafes & Disessa, 2004). Second, discerning deep structures and representing problems in a
higher level of abstraction may increase the likelihood that the knowledge and skills gained from playing the game will be used in additional
contexts that do not necessarily resemble the game context in which they were learned, or, in other words, may increase the likelihood of
transfer (Bransford, Brown, & Cocking, 2000; Day & Goldstone, 2012). The concern with transfer reflects an underlying hope that the
contribution game-based learning to developing learners’ understandings and skills will extend beyond the boundaries of the game to
additional learning contexts, including formal ones. Third, teachers as well need to recognize the connections between game content and
school curricula (Kirriemuir & McFarlane, 2004; Sandford et al., 2006; Van Eck, 2006). Indeed, a recent survey indicates that curriculum-
relatedness is one of the key determining factors in teachers’ intentions to adopt digital games in formal education (De Grove,
Bourgonjon, & Van Looy, 2012). Curriculum-relatedness may be important to teachers precisely because they are concerned that stu-
dents might not relate the concepts learned in the game to the concepts they encounter in class and in their homework (Kebritchi, 2010).
Making the curricular connections of the game visible may thus provide instructional support for teachers who wish to meaningfully
integrate games in their teaching.
1.2. Scaffolding knowledge construction in game-based learning
In recent years, researchers have started exploring various approaches for bridging informal and formal knowledge representations in
game-based learning. As mentioned above, one fruitful approach to the problem is to integrate or embed formal knowledge representations
within the game (Clark & Martinez-Garza, 2012; Clark et al., 2010, 2011; Habgood & Ainsworth, 2011; Holbert & Wilensky, 2012). A second
approach to the problem is providing external scaffolds that help learners make the connections between the knowledge learned in the
game and disciplinary knowledge (Charsky & Mims, 2008; Charsky & Ressler, 2011; Garris et al., 2002; Honey & Hilton, 2011; Neulight, Kafai,
Kao, Foley, & Galas, 2007). Scaffolding occurs when a more knowledgeable peer or teacher supports learners in carrying out a task which
they cannot yet manage independently (Brown, Collins, & Newman, 1989; Wood, Bruner, & Ross, 1976). In the domain of computer-based
learning, the concept of scaffolding has been extended to refer to software-based tools that support learners in engaging in complex and
difficult tasks (Collins, 2006; Quintana et al., 2004). Reiser (2004) proposes that software tools may support learners through two com-
plementary mechanisms; Scaffolds may structure the task and reduce its complexity or they may problematize the subject matter by causing
learners to pay more attention to critical ideas and connections that might be otherwise overlooked. Problematizing the content may add
difficulties in the short term, by forcing learners to engage with the complexities of the subject matter, but it is also creates opportunities for
deeper processing and more productive learning (Reiser, 2004). A certain type of problematizing scaffold that has received much empirical
attention is advance organizers: These organizers introduce more general, abstract, and inclusive concepts, prior to learning, and thus
provide a conceptual scaffold that helps learners organize and anchor new material by integrating it with existing knowledge structures
(Ausubel, 1960; Corkill, 1992; Mayer, 1979b; Stone, 1983). More recently, it has been suggested that advance organizers can also effectively
employ examples as a means for clarifying the ideas that are to be presented in the learning material (Corkill, 1992; Gurlitt, Dummel,
Schuster, & Nückles, 2012; Mayer, 1979b).
Recent research highlights the value of external scaffolds in enhancing learning from games (Honey & Hilton, 2011). For example, Mayer,
Mautone, and Prothero (2002) demonstrated that providing learners with pictorial representations of the concepts used in the game, before
playing, helps learners perform more accurately in a geology simulation game. Such “bridging” scaffolds may also be employed following the
game. Garris et al. (2002) suggested that debriefing, that is review and analysis of events that occurred in the game, can help learners create
critical links between game representations and events and “real world” representations and events. Neulight et al. (2007) employed
teacher-facilitated classroom discussions in order to help learners connect their understandings of a virtual infectious epidemic in the game
world of Whyville to natural infectious diseases.
Adding external scaffolds to games need not involve high production costs since such scaffolds can be designed and employed by
teachers. However, we do not yet have sufficient empirical evidence regarding whether or not this design strategy can indeed successfully
impact learning from the game. Furthermore, there is also a concern that a scaffold that requires learners to reframe concepts from the game
in formal terms might negatively impact learners’ perceptions of the game and their game experiences. Charsky and Ressler (2011) found
that supplementing a game with a conceptual scaffold, in the form of concept maps, can decrease student motivation to learn through
gameplay. They suggested that this might happen because the scaffold may focus students’ attention on the difficulty of learning the
concepts, may emphasize extrinsic motivations for playing the game, and thus make gameplay less autonomous and less fun. Adding a
scaffold to a game may also impact the ways in which learners’ perceive learning in the game. Broza and Barzilai (2011) found that con-
ceptual scaffolds had a positive impact on the perceived ease of the math skills learned in the game but had a negative impact on learners’
S. Barzilai, I. Blau / Computers & Education 70 (2014) 65–79 67
evaluations of the contribution of the games to learning and on learners’ enjoyment. Hence, Broza and Barzilai proposed that scaffolding a
game may involve trade-offs between learning gains, on the one hand, and students’ perceptions regarding learning and fun in the game, on
the other hand.
1.3. Flow, enjoyment, and their relation to learning
Games draw much of their unique instructional potential by offering learning experiences that are motivating, engaging, and enjoyable
for learners (e.g., Garris et al., 2002; Malone & Lepper, 1987; Papastergiou, 2009). In the current study we focus on two facets of learners’
game experiences, flow and enjoyment, and on their relation to learning.
Flow is a state of deep concentration in which “thoughts, intentions, feelings, and all of the senses are focused on the same goal”
(Csikszentmihalyi, 1990, p. 41). Csikszentmihalyi describes flow as occurring when individuals engage in challenging activities that require
skills. Flow involves a delicate balancing of challenge and skills: Enjoyable challenges are at a level of difficulty that avoids being too easy and
hence boring or too hard and hence frustrating. Further conditions for flow are opportunities for concentration, clear goals, immediate and
appropriate feedback, and a sense of control (Csikszentmihalyi, 1990; Hoffman & Novak, 2009; Kiili, 2005b; Sweetser & Wyeth, 2005). Flow
experiences are autotelic experiences that lead to deep and satisfying enjoyment (Csikszentmihalyi, 1990). Hence, enjoyment can be
conceptualized as a consequence of flow (Hoffman & Novak, 2009; Kiili, 2005b).
Enjoyment can be generally defined as a positive reaction to an experience that involves intertwined physiological, affective, and
cognitive dimensions (Ritterfeld & Weber, 2006; Vorderer, Klimmt, & Ritterfeld, 2004). Ritterfeld and Weber (2006) suggest that in
gameplay enjoyment may result from (a) sensory delight, (b) suspense, thrill, and relief, or (c) achievement, control, and self-efficacy. It is
interesting to note that achievement and challenge are often viewed as a central source of enjoyment in digital games and as a key motive
for engaging with interactive media (Koster, 2005; Malone & Lepper, 1987; McGonigal, 2011; Vorderer et al., 2004). The fun that games offer
is “hard fun”: fun that involves hard but rewarding work (McGonigal, 2011).
Flow and enjoyment are both posited to contribute to learning (e.g., Ainley, 2006; Cordova & Lepper, 1996; Garris et al., 2002; Malone &
Lepper, 1987; Pekrun & Stephens, 2012). However, the empirical research base regarding the relation between flow, enjoyment, and learning
is still rather limited (Pekrun & Stephens, 2012). Enjoyment has generally been found to be positively associated with learning achievements
(Pekrun, Goetz, Frenzel, Barchfeld, & Perry, 2011; Pekrun, Goetz, Titz, & Perry, 2002), motivation to learn (Linnenbrink & Pintrich, 2002), and
positive classroom climate (Meyer & Turner, 2006). Flow research has so far yielded mixed results, with some studies documenting positive
effects on learning (Engeser & Rheinberg, 2008; Kiili, 2005a; Skadberg & Kimmel, 2004), and other studies showing little or no effect on
learning (Admiraal, Huizenga, Akkerman, & Dam, 2011; Kiili & Lainema, 2008). These inconsistencies may be partially due to the difficulties
in measuring the complex construct of flow and assessing its contribution to learning (Engeser & Rheinberg, 2008; Hoffman & Novak, 2009;
Pearce, Ainley, & Howard, 2005).
How might flow and enjoyment lead to improvement in learning? Flow and enjoyment may impact learning both directly and by
increasing learners’ intrinsic motivation to engage with the learning task (Linnenbrink & Pintrich, 2002; Malone & Lepper, 1987; Meyer &
Turner, 2006; Pekrun & Stephens, 2012). Positive affect and flow are proposed to (a) cause learners to voluntarily engage in learning activities
and to choose to repeat these activities (Deater-Deckard, Chang, & Evans, 2013; Ritterfeld, Cody, & Vorderer, 2009; Webster, Trevino, & Ryan,
1993); (b) lead learners to allocate attention to the learning task and thus improve learning and subsequent recall (Ainley, 2006; Engeser &
Rheinberg, 2008; Pekrun & Stephens, 2012; Ritterfeld & Weber, 2006); (c) encourage learners to persist in learning, even in the face of
difficulties, and thus to extend study time (Deater-Deckard et al., 2013; Garris et al., 2002; Ritterfeld et al., 2009); (d) motivate learners to
attempt challenging and complex tasks and strategies (Engeser & Rheinberg, 2008; Lepper & Cordova, 1992); and (e) prompt learners to explore
alternatives and to engage in elaborative, flexible, and creative thinking (Efklides & Petkaki, 2005; Ghani & Deshpande, 1994; Hoffman &
Novak, 2009; Pekrun & Stephens, 2012; Webster et al., 1993). Finally, positive affect may be related to increased and more effective self-
regulation of learning (Aspinwall, 1998; Pekrun et al., 2002, 2011).
However, there appears to be a fine line between fun that stimulates learning and fun that may distract students from learning (Adams
et al., 2008; Johnson & Mayer, 2010). Combining flow, enjoyment, and learning requires a delicate balance between entertainment and
learning (Ritterfeld & Weber, 2006). A general design principle is that the actions needed to learn the content of the game and the actions
that lead to flow and enjoyment in the game should be one and the same or at least mutually reinforcing (Habgood & Ainsworth, 2011;
Lepper & Cordova, 1992). Or, in other words, the fun of the game should be in the learning (Klopfer, Osterweil, & Salen, 2009; Koster, 2005).
1.4. Perceived learning and learning achievements
One of the difficulties in studies that examine the relations between flow, enjoyment, and learning is that these studies measure learning
in very different ways. Some studies utilize measures of learning achievements while others rely on self-report measures of perceived
learning. However, discrepancies have been found between actual achievement and performance measures versus self-judgments of
learning (e.g., Koriat & Bjork, 2005; Rhodes & Castel, 2008), especially if learning materials are presented on a computer screen (Ackerman &
Goldsmith, 2011; Ackerman & Lauterman, 2012). In contrast, others argue that individuals are able to monitor their learning quite effectively
(Metcalfe, 2009). For the purpose of this study, we therefore decided to compare learners’ achievements in a formal problem-solving
assessment with their self-reported learning.
Perceived learning refers to a retrospective evaluation of the learning experience and can be defined as a “set of beliefs and feelings one
has regarding the learning that has occurred” (Caspi & Blau, 2008, p. 327). Caspi and Blau (2008, 2011) demonstrate that perceived learning
includes two components: cognitive and socio-emotional. The cognitive component of perceived learning reflects the sense of new
knowledge has been acquired and new understanding has been achieved. The socio-emotional component reflects the degree of
involvement in the learning process, experiences and feelings, such as enjoyment or difficulty. Since this study measures the game-related
emotional components of flow and enjoyment, the investigation of perceived learning is focused on its cognitive component.
It is important to consider the differences between measuring learning using achievement assessments versus self-reported perceived
learning. First, achievements are an evaluation of learning by others, while perceived learning is a subjective evaluation of learning by
Table 1
Population characteristics.
Conditions
Play only Study and play Play and study All participants
N 52 36 94 182
Age mean (SD) 10.08 (1.70) 10.11 (1.91) 10.11 (1.65) 10.10 (1.71)
Female 53.8% 69.4% 61.7% 61.0%
Play computer games daily 78.8% 83.2% 80.9% 80.8%
Played the Shakshouka Restaurant game before 67.3% 72.2% 75.5% 72.5%
Played at home 76.9% 80.6% 83.0% 80.8%
Played alone 80.8% 94.4% 86.2% 86.3%
Played on their own initiative 88.5% 86.1% 90.4% 89.0%
learners themselves. Second, achievements reflect the percentage of correct versus incorrect answers, while the cognitive dimension of
perceived learning reflects the learner’s sense that some new knowledge has been acquired and some new understanding has been ach-
ieved, even if these subjective knowledge and understanding are in contrast to academic conventions (Caspi & Blau, 2011). A subjective
evaluation of learning may be an important measure in the context of game-based learning. Self-assessment of the learning process and
quality of knowledge acquisition involves metacognitive monitoring and evaluation, which might affect learner’s behavior by leading them
to spend more time playing the game and to recruit more cognitive resources (Efklides, 2008; Serra & Metcalfe, 2009; Veenman, 2011).
Additionally, perceived learning may be related to learner satisfaction with the learning environment (Baturay, 2010; Lee & Lee, 2008).
Therefore, in the context of game-based learning we cannot rely exclusively either on achievements or on perceived learning measures and
need to take into account both evaluations of others and self-judgments.
2. Purpose of the current study
The purpose of the study is to examine if adding an external conceptual scaffold, in the form of an online study unit which introduces
formal knowledge representations, to a business simulation game might help learners solve formal financial-mathematical word problems
following the game. Additionally, the study explores how adding such an external scaffold might affect learners’ perceived learning, flow,
and enjoyment and if learners’ perceptions regarding learning, flow, and enjoyment in the game are related to their learning achievements.
The research questions examined in the study are:
1. How does supplementing a game with an external conceptual scaffold affect learners’ ability to solve formal mathematical-financial
word problems following the game?
2. How does supplementing a game with an external conceptual scaffold affect learners’ perceptions of learning, flow, and enjoyment in
the game?
3. What are the relations between learning achievements, perceived learning, flow, and enjoyment?
We hypothesized that adding the external scaffold would (1) positively impact learners’ achievements in solving formal word problems,
(2) positively affect their perceived learning, but (3) reduce flow and enjoyment during the game. Accordingly we hypothesized that (4)
achievement in solving formal word problems would be positively related to perceived learning and negatively related to flow and
enjoyment. We had no specific hypothesis regarding the order of presentation of the scaffold and the game and therefore examined two
options: first presenting the scaffold and then playing the game, and vice versa.
3. Method
The study was conducted online in the My Money website, a program developed by the Snunit Center for the Advancement of Web-based
Learning at the Hebrew University of Jerusalem. The website includes a set of online games and accompanying online study materials that
develop financial and math skills. The main target age group of the My Money program is 8–12 year old elementary school students. The My
Money website is freely available online and is in popular use in schools and after school hours (Broza & Barzilai, 2011). The current study
focuses on one of the games included in the program: the Shakshouka1 Restaurant game. This game is a business simulation in which players
explore the concepts of cost, price, and profit by running a restaurant. The game is one of the most popular games in the My Money website.
The study was conducted among online players of the Shakshouka Restaurant game. This approach enabled data collection in a naturalistic
setting and in conditions that resemble authentic online learning conditions, thus improving ecological validity (Bronfenbrenner, 1977;
Kozlov & Johansen, 2010).
3.1. Research design
The study was conducted in a pretest-posttest control group experimental design. The participants was randomly assigned to one of
three conditions: a control group that played the game only (“play only”), a group that studied with an online scaffold and then played the
game (“study and play”), and a group that played the game and then studied with an online scaffold (“play and study”).
1
Shakshouka is a popular Mediterranean dish of eggs poached in a sauce of tomatoes, chili peppers, and onions.
Fig. 1. Setting ingredients and price in the Shakshouka Restaurant game. Text translation: “Shuki’s Secret Recipe. Choose the amount of ingredients in each shakshouka dish. The
amount of ingredients you choose will affect the price of the dish. Customers usually like a lot of everything (usually, but not always.). . Now you need to decide in which price to
sell the shakshouka. The difference between the price of the shkahouka dish and its cost is your profit! The higher the price, the more profit. But if customers will think that the
shakshouka is too expensive, they will buy less”.
3.2. Participants
Over a 5 month period all of the players who entered the Shakshouka Restaurant game were invited to participate in the study.
Participation in the study was voluntary and the participants were not offered a reward for their participation. The participants were invited
to the study in a pop-up message that appeared upon entry to the game. The message explained that the website team is conducting a study
in order to see if children are enjoying the games on the website and learning from them, and that participation in the study will help the
website team improve the games in the website.
In the time period examined the game received altogether 50,687 unique page views.2 644 players responded to the survey. Once a
player responded to the survey it was blocked in order to avoid duplicates. Here we report on findings from 182 participants whose reported
age was 6–14 and who completed the post-game assessment. Demographic data and the participants’ gaming background are provided in
Table 1. Most of the participants play computer games on a daily basis, had played the Shakshouka Restaurant game at least once before, and
were currently playing at home, on their own, and on their own initiative.
The participants were randomly assigned to one of three experimental conditions, however, differences in drop-out rates resulted in a
different number of participants in each group. 52 participants completed the “play only” condition, 36 participants completed the “study
and play” condition, and 94 participants completed the “play and study” condition. Population characteristics were similar across groups,
see Table 1. The proportion of girls in the “study and play” condition was somewhat higher than in the other conditions but this difference
was not significant, c2 (2, N ¼ 179) ¼ 2.43, p ¼ .297.
3.3. Materials
3.3.1. The Shakshouka Restaurant game

The Shakshouka Restaurant game is a market game in the style of the lemonade stand game. The goal of the game is to set up a shakshouka
restaurant and earn 10,000 shekels3 in 10 “days”. In the beginning of each day the player decides on the shakshouka ingredients and sets the
shakshouka price, see Fig. 1. The shakshouka ingredients influence the cost of the shakshouka, and that, together with the price, determines
the profit per shakshouka sold. The participants then need to order stock for producing the shakshouka. Insufficient stock may result in losing
customers. Too much stock might spoil. Once all is ready the day begins. The players watch customers coming and going. As the customers
pass the restaurant they give positive and negative feedback regarding the taste and the price of the shakshouka (e.g., “the shakshouka is too
spicy”, “the shakshouka is too expensive”), see Fig. 2. Night gradually falls and at the end of the day (which lasts about 40 s) the players can
view a summary of the customers’ feedback and of the profit they made that day. The players can then change the shakshouka recipe and price
in order to increase their profits. The challenge of the game is finding the right balance between all of these components, that is producing the
shakshouka at a reasonable cost and selling it at a price that meets market expectations and provides an optimal profit.
3.3.2. External conceptual scaffold: The Setting Prices online study unit
Although success in the Shakshouka Restaurant game hinges on finding the right balance between cost and price, these understandings
may be intuitive and tacit. Even players who play the game successfully might not necessarily form explicit abstract understandings
regarding the concepts of cost, price, and profit and their relations. The Setting Prices study unit was designed in order to scaffold the
2
The unique pageview metric in Google Analytics indicates the number of visits during which the specified page was viewed at least once.
3
The shekel is the currency of the State of Israel.
Fig. 2. Customer feedback in the Shakshouka Restaurant game. The money earned appears in the top right corner. The speech bubbles above the customers provide feedback that the
price should be lower. The panel on the bottom left corner indicates the remaining stock.
development of explicit understandings of the relations between cost, price, and profit, and in order to help learners make connections
between the Shakshouka Restaurant game and the types of financial-mathematical word problems that frequently appear in math textbooks
and in the national math exams. Thus, the Setting Prices study unit is a scaffold whose goal is to support learners’ sense making by helping
them bridge between their intuitive understandings of the game and disciplinary formalisms (see Scaffolding Guideline 1, Quintana et al.,
2004). The scaffold problematizes learning by expanding the meanings of the concepts used in the game, drawing attention to their re-
lations, and making connections between game representations and disciplinary representations (Reiser, 2004). The scaffold may also be
considered as an advance organizer because it presents the general concepts of cost, price, and profit as well as provides concrete examples
of their use in problem solving (Corkill, 1992; Mayer, 1979b).
The online study unit opens by explaining the meaning of cost, price, and profit, and by providing a mathematical equation of their
relations, see Fig. 3. The learners are then introduced to a game character named Shuki, an aspiring shakshouka restaurant owner. The
learners are asked to help Shuki make decisions regarding costs, prices, and profits. In contrast to the game, this scenario is designed as a
series of formal word problems, e.g., “If Shuki will sell each shakshouka for 9 shekels how much profit will he make from each shakshouka?”
The learners can receive hints on how to solve the word problems, and are provided with informative feedback, e.g., “Are you sure that this is
the right answer? Remember, producing the shakshouka costs Shuki 5 shekels.” To conclude, the study unit introduces the relations be-
tween cost, price, and profit using two types of formal knowledge representation that are not included in the game: a mathematical
equation and financial-mathematical word problems. However, the study unit also maintains a clear connection to the game by means of
verbal and visual references to the game narrative.
3.4. Measures
3.4.1. Formal problem-solving assessment

Learners’ ability to apply what they learned in the game in a formal context was assessed using a set of eight word problems that refer to
the relations between cost, price, and profit. See sample question in Fig. 4. We define this task as a transfer task because although the game
deals with the relations between of cost, price, and profit it does so in an informal and intuitive fashion. Presenting these relations as word
problems requires learners to apply knowledge learned through enactment in a “real world” gaming context in a formal academic context
and hence involve transfer across contexts (Barnett & Ceci, 2002). Visually, the assessment was designed as an interactive online quiz that
was styled on the graphic design of the Shakshouka Restaurant game. Face validity of the assessment was determined by two subject-matter
experts. The assessment had two counterbalanced pre- and post-versions that featured the same problems using different numbers and
names. The participants did not receive feedback regarding their performance on the assessment. The problem-solving scores for pre- and
post-tests were approximately normally distributed (Range ¼ 0.00–7.00, Mean ¼ 2.75, SD ¼ 2.00, Median ¼ 2.00, Skewness ¼ 0.63 for pre-
test, and Range ¼ 0.00–7.00, Mean ¼ 2.74, SD ¼ 1.91, Median ¼ 2.00, Skewness ¼ 0.64 for post-test).
3.4.2. Perceived learning

Perceived learning was measured using four items that relate to the cognitive aspects of perceived learning, adapted from Blau and Caspi
(2008). All items are provided in Table 2. The items were measured on a six-point scale ranging from one (very much disagree) to six (very
much agree). The perceived learning scale was found to be highly reliable (4 items; a ¼ .90). The perceived learning index was based on the
mean of the four items and was normally distributed (Range ¼ 1.00–6.00, Mean ¼ 3.11, SD ¼ 1.51, Median ¼ 3.25, Skewness ¼ 0.06).
In addition, the participants were asked to reply to an open question regarding their learning perceptions: “What did you learn from the
game?” The replies to this question were analyzed in a bottom-up grounded approach (Bryant & Charmaz, 2012). Categories were formed by
iterative reading of a sample of the answers and were then applied to the entire data set. The coding categories were not exclusive, that is
Fig. 3. An excerpt from the Setting Prices online study unit. Text translation: “As business owners, when we buy a product in 10 shekels and want to profit 5 shekels we will sell it in
15 shekels. Product cost þ profit ¼ price. Product cost – how much do the ingredients cost? (10 shekels). Profit – How much do I want to earn? (5 shekels). Price – For how much do I
sell the product? (15 shekels). So how do you set the price of a product? How do you know how much you earned? Stay with us and find out”. [Shuki’s speech bubble] “I must know
how to set the price of the shakshouka I sell, so that I will make as much profit as possible, without driving the customers away”.
each statement could be coded in more than one category. The inter-rated reliability of the coding scheme was examined by the authors
using 50% of the valid responses. Inter-rater reliability was tested using Cohen’s Kappa and was found to range from 0.66 to 1.00, average
0.86 (SD 0.11), indicating a satisfactory level of agreement (Landis & Koch, 1977).
3.4.3. Perceived game experiences

Perceived flow was measured using five items based on flow measures that were previously tested with children (Brockmyer et al., 2009;
Inal & Cagiltay, 2007). See Table 2. All items were measured on a six-point scale ranging from one (very much disagree) to six (very much
agree). The reliability of the flow scale was very good (5 items; a ¼ .89). The flow index was based on the mean of the five items and was
normally distributed (Mean ¼ 3.88, SD ¼ 1.43, Median ¼ 4.00, Skewness ¼ 0.39).
Perceived enjoyment was measured using three items adapted from Davis, Bagozzi, and Warshaw (1992) and measured on a six-point
scale ranging from one (very much disagree) to six (very much agree). The enjoyment scale was also found to be highly reliable (3 items;
Fig. 4. Sample question from the formal problem-solving assessment. Text translation: “Yossi sells a shakshouka dish for 18 shekels. Yossi profits 4 shekels for each shakshouka.
How much does it cost to produce a single shakshouka? [Multiple choices] 14 shekels, 18 shekels, 22 shekels, 4 shekels”.
Table 2
Items used to measure perceived learning, flow, and enjoyment.
Variable Items
Perceived learning I learned a lot from the game.
The game added to my knowledge.
I learned new things from the game.
The game will help me remember the things I learned.
Flow I lost track of time when I played.
I really got into the game.
I was very involved in the game.
When I played I did not think of anything else.
I was totally immersed in the game.
Enjoyment I enjoyed the game.
I had fun playing the game.
Playing the game was pleasant.
a ¼ .88). The enjoyment index was based on the mean of the five items and was approximately normally distributed (Mean ¼ 4.71, SD ¼ 1.25,
Median ¼ 5.00, Skewness ¼ 0.92).
The participants were also asked to reply to an open question regarding their perceived enjoyment: “What did you enjoy in the game?”
The coding procedure of the participants’ replies was as described above. Inter-rater reliability was found to range from 0.84 to 1.00, average
0.88 (SD 0.09).
3.5. Procedure
The players were invited to participate in the survey using a pop-up frame that appeared when they entered the Shakshouka Restaurant
game. The players could choose to skip the study and continue to the game. If they chose to participate in the study they were directed to a
special version of the game that opened on their entire screen. Leaving or closing that screen stopped the study, and the data from those
participants were not used.
The first page of the study included multiple choice background questions (age, gender, previous experience in playing the game, current
location, playing partners, reasons for playing, and gaming habits) and a random counter-balanced version of the problem-solving
assessment. The participants were then randomly assigned to one of three experimental conditions: (a) playing the game only (“play
only”), (b) studying with the online scaffold and then playing the game (“study and play”), and (c) playing the game and then studying with
the online scaffold (“play and study”).
Participants in the “play only” condition were transferred directly to the game. After completing the game (i.e. concluding 10 “days” of
play) they were transferred to a page that included the post-game assessments. This page included a second counter-balanced version the
problem-solving assessment, the perceived learning, fun, and flow questionnaire, and the two open questions (“What did you enjoy in the
game?”, “What did you learn from the game?”).
Participants in the “study and play” condition viewed the following message: “Before you start playing the shakshouka restaurant game,
we have an activity that will help you know how to succeed in the game.” These participants then continued to the online study unit. After
completing the study unit they were transferred to the game. When they completed the game they were transferred to the post-game
assessment page.
Participants in the “play and study” began by playing the game. When they completed the game they viewed the following message, “We
have an activity that will help you understand what happened in the shakshouka restaurant and to know how to succeed in the game”, and
then continued to the study unit. After completing the study unit they were transferred to the to the post-game assessment page.
The online measures were tested with adults and children for several weeks and debugged to ensure technical stability and reliability
before data collection. Questionnaire data, game data (time on game and profit made), and study unit data (replies to questions) were saved.
The participants spent 16:01 min on average (SD 02:50) playing the game and “earned” 9200 shekels on average (SD 4250).
4. Results
4.1. Impact on formal learning achievements
In order to examine the effect of the external scaffold on formal learning gains we conducted a repeated measures ANOVA with
experimental condition as a between-subjects variable, measurement time (before and after the game) as a within-subjects variable, and
problem-solving assessment score as a dependent variable. A statistically significant effect of experimental condition was found,
F(2,179) ¼ 3.49, p ¼ .033, h2p ¼ .038, but no significant effect of measurement time, F(1,179) ¼ .28, p ¼ .597, h2p ¼ .002, or interaction between
experimental condition and measurement time, F(2,179) ¼ 1.17, p ¼ .312, h2p ¼ .013 were found. Means and standard deviations of the pre and
post problem-solving assessment scores are provided in Table 3.
To interpret these results we conducted separate analyses of variance for the first and second problem-solving assessment scores. These
analyses indicated that in the pre-game problem-solving assessment there were no significant differences between experimental condi-
tions, F(2,179) ¼ 0.20, p ¼ .822, h2p ¼ .002, but in the post-game assessment there were significant differences between groups,
F(2,179) ¼ 3.38, p ¼ .036, h2p ¼ .036. Post-hoc LSD tests revealed that in the post-game problem-solving assessment the achievements of
learners in the “study and play” condition were significantly higher than the achievements of learners in the “play only” condition, p ¼ .030,
and from the achievements of in the “play and study” condition, p ¼ .014. No statistically significant differences were found between the
achievements of learners in the “play and study” condition and the “play only” condition, p ¼ .942. To summarize, although no significant
gains in formal learning were found following the intervention, the “study and play” group performed significantly better than the two other
Table 3
Problem-solving assessment score means and standard deviations.
Conditions Pre-game problem-solving score Post-game problem-solving score
M SD M SD
Play only 2.62 1.87 2.58 1.89
Study and play 2.72 2.03 3.47 1.98
Play and study 2.83 2.06 2.55 1.85
groups in the post-game problem-solving assessment. It is important to note that the problem-solving assessment scores were on the whole
rather low, indicating that the participants found the word problems difficult to solve.
We examined if other individual or contextual variables may have an impact on the post-game problem-solving assessment score. There
was no significant effect of gender, and no interaction between gender and experimental condition. Gaming habits, time spent playing the
game, and the profit made in the game were not correlated with the post-game problem-solving assessment score.
4.2. Impact on perceived learning
Experimental condition had a significant impact on perceived learning, F(2,149) ¼ 3.71, p ¼ .027, h2p ¼ .047. Post-hoc LSD tests indicated
that participants in the “play only” condition reported significantly higher perceived learning than participants in the “study and play”
condition, p ¼ .008. No statistically significant differences in perceived learning were found between the “play only” condition and the “play
and study” condition, p ¼ .110, or between the “play and study” condition and the “study and play” condition, p ¼ .113. That is, adding the
external scaffold before the game lowered learners’ perceptions of how much they had learned from the game. Means and standard de-
viations are provided in Table 4.
4.3. Impact on game experiences
One of our concerns was that adding the external scaffold to the game would result in a decrease in perceived game experiences.
However, flow and enjoyment scores were above scale mean in all conditions. See Table 4. Analysis of variance revealed no effect of
experimental condition on learners’ reports of flow, F(2,149) ¼ .01, p ¼ .990, h2p ¼ .000, or enjoyment, F(2,149) ¼ .10, p ¼ .905, h2p ¼ .001.
4.4. Modeling the relations between learning achievements, perceived learning, flow, and enjoyment
We examined the bivariate correlations between learners’ post-game problem-solving scores, perceived learning, flow, and enjoyment.
Among all participants, perceived learning, flow, and enjoyment were found to be strongly and significantly correlated. However the post-
game problem-solving assessment scores were not correlated to perceived learning, flow, or enjoyment. See Table 5.
In order to further examine the relations between learning achievements, perceived learning, flow, and enjoyment we used AMOS 21
software to create a structural equation model based on maximum-likelihood estimation. Missing values were imputed using AMOS
regression imputation. We found one model that adequately fitted our data and had excellent fit indices (c2 ¼ 5.07, df ¼ 4, p ¼ .280, NFI ¼ .99
CFI ¼ 1.00 RMSEA ¼ .03). The model posits that perceived learning and enjoyment are consequences of flow and examines the direct and
indirect contribution of flow to learners’ post-game problem-solving assessment score. The standardized solution of the model is provided
in Fig. 5. In all three conditions flow was found to significantly predict perceived learning and enjoyment. However, perceived learning and
enjoyment did not predict problem-solving. Flow directly predicted problem solving only in the play and study condition but only 6% of
variance in the problem-solving assessment score was explained.
4.5. Exploring the meanings of “learning” and “enjoyment”
To gain a deeper understanding of what the learners mean by “learning” and by “enjoyment”, we analyzed the participants’ replies to the
two open questions: “What did you learn from the game?” and “What did you enjoy in the game?” 119 participants (65% of the participants)
responded to these questions. Out of these replies 82 were coded as informative. Code definitions and examples are provided in Appendix A.
Frequencies are shown in Fig. 6.
What struck us as we analyzed the participants’ replies were the numerous resemblances between learners’ perceptions of what they
enjoyed in the game and what they learned from the game. Hence, we created parallel codes for the participants’ descriptions of learning
and enjoyment in the game, see Appendix A. The most frequently recurring category in the participants’ replies was that what they learned
from the game and what they enjoyed in the game was managing a business. The participants reported that they learned how to run a
restaurant (e.g., “I learned how to manage a restaurant”), and also reported learning specific business skills (e.g., “That every income has an
expense”, “That you need to save money”, “That you can’t exaggerate the prices”). The participants reported enjoying managing the business
in general, (e.g., “Managing the restaurant”, “Setting up my own business”), as well as enjoying specific aspects of business management,
Table 4
Perceived learning, flow, and enjoyment score means and standard deviations.
Conditions Perceived learning Flow Enjoyment
M SD M SD M SD
Play only 3.51 1.47 3.91 1.23 4.67 1.30
Study and play 2.54 1.47 3.86 1.71 4.80 1.29
Play and study 3.07 1.50 3.88 1.46 4.69 1.21
Table 5
Correlations between problem solving assessment, perceived learning, flow, and enjoyment.
Variable 1 2 3 4
1. Problem-solving assessment – .11 .01 .05
2. Perceived learning – .61*** .50***
3. Flow – .73***
4. Enjoyment –
Note. N ¼ 152 due to missing values, ***p < .001 (two-tailed).
(e.g., “The idea of making a profit”, “Filling the stock”). These parallels may help explain the high correlation between perceived learning and
game experiences: For many learners enjoyment and learning in the game were not clearly separable from each other but were rather two
aspects of the experience of managing the restaurant.
In contrast, learning math skills was mentioned much less often than learning to manage a business, c2 (1, N ¼ 57) ¼ 29.5, p < .001. That is, the
participants perceived the game to be more about learning business skills than about learning math skills. However, math skills were referred to,
by a few participants, also as a source of fun in the game, indicating that in this aspect too, learning and enjoyment can be interrelated.
Another specific aspect of managing the business that participants repeatedly referred to as pleasurable was decision making: The
players enjoyed making choices, seeing their outcomes, and correcting them in response, (e.g., “I enjoyed deciding on the menu”).
Correspondingly, several participants also referred to learning how to make decisions, (e.g., “It is much clearer to me now how business
owners set prices”). Participants also, quite obviously, enjoyed succeeding in the game, namely getting positive feedback from their cus-
tomers and making money, (e.g., “Seeing that people like the food I made”, “Catching the customers and getting money”). Accordingly,
several participants referred to learning how to succeed as what they learned from the game, (e.g., “Even if you bake something and it does
not succeed you need to keep trying the recipe until it succeeds!!!”).
The participants also enjoyed specific games experiences, such as flow, pleasure, interest, excitement, or independence (e.g., “I was
immersed in the game”, “Every time I raised the price I felt suspense”). Gameplay, design, game narrative, and specific mechanics were
explicitly noted by some of the participants as a source of enjoyment (e.g., “The easy interface”, “The competition”). Some participants
referred to game experiences not only as a source of fun but also as something they had learned as well. For example, participants reported
Fig. 5. Structural equation model of the relations between problem solving, perceived learning, flow, and enjoyment. Standardized coefficients appear on the arrows. Squared
multiple correlations appear in the boxes. Dashed arrows indicated non-significant regression weights. *p < .05, ***p < .001.
Fig. 6. Perceptions of learning and enjoyment. Frequency of coding categories of the replies to the open questions: “What did you learn from the game?” and “What did you enjoy in
the game?”
learning new experiences, such as that running a business can be enjoyable, or that learning math can be fun (e.g., “I learned that selling is
fun”, “I learned that learning math can be fun”, “I learned to be responsible”).
Lastly, there were several participants who explicitly referred to the learning itself as enjoyable (e.g., “It was fun learning. You don’t feel
that it’s hard to learn.”). Correspondingly, two participants reported that what they learned in the game was how to learn, for example, that
they learned how to use the feedback provided by the game in order to improve their decision making (e.g., “According to the mistakes you
make you can know what do to right the next time”).
5. Discussion
This study examined the impact of adding an external conceptual scaffold, which introduces formal knowledge representations, to a business
simulation game on learners’ ability to solve formal financial-mathematical word problems following the game and on learners’ perceived
learning, flow, and enjoyment. We also examined the relations between learning achievements, perceived learning, flow, and enjoyment.
Although no significant gains in formal problem-solving assessment were found following the intervention, learners who studied with the
external scaffold before the game performed significantly better in the post-game assessment. Ironically, adding the external scaffold before the
game also reduced learners’ perceived learning from the game. However, the scaffold did not have a negative impact on learners’ reported flow
and enjoyment. Flow significantly predicted perceived learning and enjoyment. Yet, perceived learning and enjoyment did not predict problem-
solving and flow directly predicted problem solving only when the scaffold was presented after the game. In the ensuing paragraphs we offer
interpretations of these findings, discuss the limitations of the study, suggest ideas for future research, and explore design implications.
One of the striking findings of this study is that although learners were successful in the game, as indicated by their high average “earnings”,
their ability to solve financial-mathematical word problems, which related to the game content, was relatively low. We believe that the gap
between learners’ success in the game and their ability to solve game-related word problems reflects a gap between the tacit and enacted un-
derstandings of making a profit in the game and the explicit understandings of the concepts of cost, price, profit, and their relations. Understanding
of the informal and intuitive knowledge representations embedded in the game does not simply transfer to understanding of formal disciplinary
representations of the same content. This might happen because learners may not form generalizations and develop global understandings of
underlying principles, as they play the game (Parnafes & Disessa, 2004). Additionally, learners might not spontaneously create connections be-
tween the game concepts and disciplinary concepts (Clark & Martinez-Garza, 2012; Clark et al., 2011; Quintana et al., 2004; Reiser, 2004).
There was no significant gain in participants’ ability to solve financial-mathematical word problems following the intervention. It is
important to note that the intervention was relatively short – a single learning session. A longer intervention might be required for sig-
nificant gains in such a transfer task. However, adding an external conceptual scaffold before the game did result in significantly better post-
game scores than playing without the scaffold or presenting the scaffold after the game. We see this as a positive indicator of the potential
efficacy of external scaffolds for helping learners form connections between game knowledge and formal school knowledge. Obviously,
more research would be needed in order support this finding.
Interestingly, presenting the external conceptual scaffold before the game was more efficacious than presenting it after the game. There may
be two complementary explanations for this difference. First, explicitly presenting the concepts of cost, price, profit, and their relations, before the
game, might have prompted the learners to form connections between these concepts and game actions and feedback, as they played the game
(Holbert & Wilensky, 2012). Furthermore, the scaffold may have caused the learners to relate the gameplay to more general and inclusive
disciplinary concepts and may have thus functioned as an advance organizer that facilitates assimilation of new material with existing knowledge
structures (Corkill,1992; Mayer,1979b). In contrast, presenting the scaffold after the game would have missed such an opportunity for knowledge
building during gameplay. This finding seems to correspond to the findings of advance organizer research that indicate that such organizers are
more efficacious when presented before learning (Corkill, 1992; Mayer, 1979a). Second, participants might have experienced fatigue and con-
centration loss after completing the game and this may have had a negative impact on the quality of learning with the scaffold following the game.
Adding the external conceptual scaffold before the game also led to a decrease in learners’ perceptions regarding how much they learned
from the game. Previous studies show that perceived learning is often not a good indicator of learning and may be higher or lower than actual
achievement (e.g., Koriat & Bjork, 2005). Perhaps the higher cognitive effort required in the scaffolding condition reduced learners’ illusions of
understanding (Jacoby, Bjork, & Kelley, 1994) and lead to better calibration of learners’ assessments of how much they learned (Paik & Schraw,
2012). According to this explanation, the scaffold succeeded in problematizing the game content (Reiser, 2004) by drawing learners attention to
the similarities and differences between what is learned in the game and what is learned at school and therefore created a sense of difficulty that
reduced perceived learning (see also, D’Mello, Lehman, Pekrun, & Graesser, 2013; Graesser, Chipman, Leeming, & Biedenbach, 2009). However,
the difficulties that learners experienced due to the scaffold may be “desirable difficulties” that create conditions for more productive learning
(Bjork, 1994; Linn, Chang, Chiu, Zhang, & McElhaney, 2010; Reiser, 2004). Additionally, more accurate self-evaluation of knowledge acquisition
following the game might lead to better control of cognitive processes and more effective choice of learning strategies (Efklides, 2008; Serra &
Metcalfe, 2009). Yet, our findings also suggest a second possible interpretation of these results, namely, that the scaffold might have had an effect
on how learners interpret the meaning of “learning” in the game. As the replies to the open question indicate, the participants largely interpreted
learning in the game as a learning how to run a business. The external scaffold, by framing the game as a math-related activity, may have caused
the participants to expand or shift their interpretation of what it means to learn in the game.
Contrary to our hypothesis, adding the external scaffold did not reduce learners’ perceptions of flow and enjoyment in the game. Two
tentative explanations can be offered. First, in our study the scaffold was presented either before or after the game and thus did not interrupt
gameplay. Second, the scaffold was designed as part of the game narrative: The study unit invited the learners to help a game character
overcome the challenges posed by the game. Hence, the learners may have viewed the study unit as an extension of the game narrative
rather than as an interruption. Indeed, some of the responses to the open questions lead us to hypothesize that the participants found the
study unit itself enjoyable.
Finally, this study raises interesting questions regarding the relations between game experiences, such as flow and enjoyment, and
learning from the game. Previous research suggests that enjoyment and flow are by large positively related with learning (e.g., Garris et al.,
2002; Malone & Lepper, 1987; Pekrun & Stephens, 2012; Webster et al., 1993). Indeed, in the current study, perceived learning, flow, and
enjoyment were strongly correlated among all participants. Furthermore, flow was found to significantly predict both perceived learning
and enjoyment. This finding is supported by participants’ answers to the open questions regarding learning and enjoyment in the game:
Learners’ replies indicate that the source of perceived learning and enjoyment in the game is one and the same – the challenge of suc-
cessfully running the restaurant. These findings support the view that learning experiences and game experience are highly interrelated
aspects of game-based learning (Koster, 2005; Ritterfeld & Weber, 2006).
It is, therefore, interesting to reflect on the somewhat surprising finding that perceived learning, flow, and enjoyment were generally
uncorrelated to actual learning achievements following the game. We suggest that a key cause for this disconnect, besides the above
mentioned possible illusions of understanding, may be the gap between the meaning that the participants’ attributed to “learning”, (i.e.,
learning how to manage the restaurant), and the type of learning measured by the problem-solving assessment (i.e., learning how to solve
financial-mathematical word problems). Thus, there are multiple meanings of “learning” at play in the game: informal learning compared
to formal learning and perceived learning compared to actual learning.
Flow, in our study, predicted learners’ perceptions regarding informal learning from the game but, generally, did not substantially impact
their formal learning achievements. Only when the scaffold was presented after the game did flow appear to have some impact on problem-
solving. Perhaps higher levels of flow experiences while playing the game increased the likelihood that learners would also continue to
meaningfully engage with the scaffold and with the ensuing problem-solving assessment. This finding suggests that flow may contribute,
under certain conditions, to formal learning achievements by preparing and motivating learners for future learning.
5.1. Limitations and suggestions for future research
Several limitations of the study should be taken into account in the interpretation of the results. First, because the study was conducted
online with volunteers we were limited to performing the study in a single session. This caused the intervention to be relatively short. A recent
meta-analysis of the effectiveness of game-based learning suggests that multiple playing sessions may be more effective than a single session
of playing (Wouters, van Nimwegen, van Oostendorp, & van der Spek, 2013). Second, the study was conducted with participants who
voluntarily chose to play the game and to participate in the study. Thus, the results may not necessarily generalize to the student population at
large and to learning that takes place in non-voluntary settings. Third, there were substantial differences in drop-out rates between the three
experimental conditions. Presenting the study unit first resulted in higher drop-out rates than playing the game first. However, we found no
differences between the three groups in prior problem-solving abilities, participant background, or perceptions of flow and enjoyment.
A possible approach for addressing these limitations and extending the current study can be to conduct a similar study at school with
whole classes. A school-based study might also offer insights regarding differences between game-based learning that takes place in and out
of school. Additionally, in order to gain deeper understanding of how adding external conceptual scaffolds to digital games impacts learning,
quantitative assessment should be complemented by qualitative methods such as think aloud protocols and interviews. Some interesting
questions that might be explored in future studies are: Why and how does adding an external scaffold affect the ways in which learners
construct knowledge during and following a game? What additional aspects of the learning context may impact learners’ ability to make
connections between game knowledge and school knowledge? And how can teachers assist learners in making such connections?
There are many open questions regarding scaffold design for game-based learning. This study examined a certain type of problematizing
scaffold which was presented either before or after the game. It could be worthwhile to also examine additional scaffolding mechanisms,
e.g., structuring scaffolds, as well as different timing and spacing of scaffold presentation. More studies would be needed in order to identify
which scaffold characteristics might most effectively promote learning while maintaining flow and enjoyment.
As these questions are examined, we suggest that closer attention should be paid to the various meanings of learning in the game, since
learning in a game may occur in diverse ways and take on multiple meanings. Furthermore, learners’ views of what is learned from the game
may be different from those of the researchers or of their teachers.
5.2. Implications for the design of game-based learning
The findings of the study suggest that learners may require instructional assistance in order to bridge between the tacit and enacted
knowledge representations embedded in games and the formal knowledge representations used at school. The study supports the design
recommendation to use external scaffolds in order to facilitate deeper learning from games (Honey & Hilton, 2011). The order of pre-
sentation effect found in this study requires further testing before any conclusive recommendations can be made. However, the current
results suggest that the order of presentation of the external scaffold and the game might have importance, and that debriefing after the
game (e.g., Garris et al., 2002) might not be sufficient for helping learners form connections between the game and what they learn at
school.
The design of computer-based scaffolds is a topic that has received considerable theoretical and empirical attention and several frameworks
suggests scaffolding guidelines which may be highly-relevant for the design of scaffolds for digital games (e.g., Graesser, McNamara, & VanLehn,
2005; Kali, Linn, & Roseman, 2008; van Merrienboer, Kirschner, & Kester, 2003; Quintana et al., 2004; Quintana, Zhang, & Krajcik, 2005; Reiser,
2004; Shapiro, 2008). However, designing scaffolds for games also poses unique challenges that arise from the special characteristics of this
medium and may thus call for additional design principles (cf. Charsky & Mims, 2008; Garris et al., 2002). The findings of this study lead us to
tentatively suggest one such guideline: Perhaps in order to successfully bridge between game knowledge and school knowledge, a scaffold for
game-based learning should be a middle-ground between game-based learning and formal learning. Or, in other words, the scaffold should be
designed both for learning and for enjoyment. This can be achieved by introducing relevant disciplinary concepts and representations in ways that
make their relevance to the game narrative and to success in the game explicit. For example, curricular concepts may be tied to the game content or
may be introduced by a game character. Learners can be asked to “help” one of the game characters solve a problem in the game by applying
disciplinary knowledge. Game visuals and interactive elements can be incorporated in the scaffolds in order to make them visually related to the
game world. In these ways, among others, scaffolds can be designed to be part of the meta-game surrounding the game (Gee & Hayes, 2011; Salen &
Zimmerman, 2003) and thus offer meaningful ways for learners to connect their understandings of the game with curricular concepts and
principles.
Acknowledgments
The game and additional learning materials employed in this study were designed by the first author and by Orit Broza for the Snunit
Center for the Advancement of Web-based Learning at the Hebrew University of Jerusalem. We are grateful to Revital Rubin, the director of
the Snunit center, and to the center’s dedicated staff members, Orit Broza, Hagit Ohana, Ivan Chernykh, Yaffa Berger, Sharon Naor-Tager,
Moshe Sarbatka, and Ayelet Weizman, for their assistance in developing the materials and measures used in this study.
Appendix A. Coding scheme of the replies to open questions: “What did you learn from the game?” and “What did you enjoy in
the game?”
Code Definition Examples

Perceptions of learning
Business Learns how to run a business or a restaurant in general, “I learned how to manage a business”, “How to manage
management or learns specific aspects of running a business such as a restaurant”, “That every income has an expense”,
specific business or financial skills. “That you need to save money”, “That you can’t exaggerate the prices”
Learning Explicitly reports feelings and experiences such as fun, “I learned that selling is fun”, “I learned to be responsible”, “I learned
experiences enjoyment, flow, interest, boredom, excitement, suspense, that learning math can be fun”
independence, and responsibility.
Success Learns how to succeed in the game and in managing “I learned how to make the perfect shakshouka recipe”, “Even if you
the restaurant. Specifically, learns to how to make a profit bake something and it does not succeed you need to keep trying
or to satisfy the customers. the recipe until it succeeds!!!”
Decision Learns to make decisions. For example, learns how to choose “It is much clearer to me now how business owners set prices”,
Making a recipe and to set prices. “How to set prices”
Learning Explicitly reports on learning and on learning how to learn. “According to the mistakes you make you can know what do
to right the next time.”
Math Learns math skills. “Math”, “How to calculate a profit”, “Calculating"
Perceptions of enjoyment
Business Enjoys running a business or a restaurant in general, or enjoys “Running a business”, “Setting up my own business”, “Creating a restaurant”
management specific aspects of running a business such as specific business “The idea of making a profit”, “Filling the stock”
or financial skills.
Game Explicit reports feelings and experiences such as fun, enjoyment, “I wasn’t bored”, “I was immersed in the game”, “Every time I raised
experiences flow, interest, boredom, excitement, suspense, independence, the price I felt suspense”, “I enjoyed creating things by myself”
and responsibility.
Gameplay Enjoys various aspects of the gameplay and game design, such “The competition”, “The easy interface”, “Seeing the results
as game narrative, game mechanics, interface, and controls. and seeing what’s missing”
Success Enjoys succeeding in the game and in managing the restaurant. “Seeing that people like the food I made”, “Catching the customers
Specifically, enjoys making a profit or getting positive feedback and getting money”, “Making a profit”, “Finding the “winning” recipe”
from the customers.
Decision Enjoys making decisions and the decision making process. “I enjoyed deciding on the menu”, “Setting the final recipe
making For example, enjoys choosing a recipe and setting prices. and the cost”, “People are always unhappy with something and you
need to think what to improve”
Learning Enjoys learning, gaining new knowledge and skills, “It was fun learning. You don’t feel that it’s hard to learn.”, “I enjoyed
and self-regulated learning. getting a lot of stuff and information that I did not know before”
Math Enjoys math skills such as calculation. “The calculations”, “I enjoyed that the calculations were a little bit hard”
Note: Categories were not exclusive, that is each statement could be coded in more than one category.
References
Ackerman, R., & Goldsmith, M. (2011). Metacognitive regulation of text learning: on screen versus on paper. Journal of Experimental Psychology: Applied, 17(1), 18–32.
Ackerman, R., & Lauterman, T. (2012). Taking reading comprehension exams on screen or on paper? A metacognitive analysis of learning texts under time pressure. Computers
in Human Behavior, 28(5), 1816–1828.
Adams, W. K., Reid, S., LeMaster, R., McKagan, S. B., Perkins, K. K., Dubson, M., et al. (2008). A study of educational simulations part I – engagement and learning. Journal of
Interactive Learning Research, 19(3), 397–419.
Admiraal, W., Huizenga, J., Akkerman, S., & Dam, G. t (2011). The concept of flow in collaborative game-based learning. Computers in Human Behavior, 27(3), 1185–1194.
Ainley, M. (2006). Connecting with learning: motivation, affect and cognition in interest processes. Educational Psychology Review, 18(4), 391–405.
Aspinwall, L. (1998). Rethinking the role of positive affect in self-regulation. Motivation and Emotion, 22(1), 1–32.
Ausubel, D. P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51(5), 267.
Barab, S., Zuiker, S., Warren, S., Hickey, D., Ingram-Goble, A., Kwon, E.-J., et al. (2007). Situationally embodied curriculum: relating formalisms and contexts. Science Education,
91(5), 750–782.
Barnett, S. M., & Ceci, S. J. (2002). When and where do we apply what we learn? A taxonomy for far transfer. Psychological Bulletin, 128(4), 612–637.
Baturay, M. H. (2010). Relationships among sense of classroom community, perceived cognitive learning and satisfaction of students at an e-learning course. Interactive
Learning Environments, 19(5), 563–575.
Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe, & A. P. Shimamura (Eds.), Metacognition: Knowing about knowing
(pp. 185–205). Cambridge, MA: MIT Press.
Blau, I., & Caspi, A. (2008). Do media richness and visual anonymity influence learning? A comparative study using skypeÔ. In Y. Eshet-Alkalai, A. Caspi, & N. Geri (Eds.), Learning
in the technological era: Proceedings of the third chais conference on instructional technologies research 2008 (pp. 18–25). Ra’anana, Israel: The Open University of Israel.
Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brain, mind, experience, and school. Washington, DC: The National Academies Press.
Brockmyer, J. H., Fox, C. M., Curtiss, K. A., McBroom, E., Burkhart, K. M., & Pidruzny, J. N. (2009). The development of the game engagement questionnaire: a measure of
engagement in video game-playing. Journal of Experimental Social Psychology, 45(4), 624–634.
Bronfenbrenner, U. (1977). Toward an experimental ecology of human development. American Psychologist, 32(7), 513–531.
Brown, J. S., Collins, A., & Newman, S. E. (1989). Cognitive apprenticeship: teaching the crafts of reading, writing, and mathematics. In L. B. Resnick (Ed.), Knowing, learning, and
instruction: Essays in honor of robert glaser (pp. 453–494). Hillsdale, NJ: Lawrence Erlbaum Associates.
Broza, O., & Barzilai, S. (2011). When the mathematics of life meets school mathematics: playing and learning on the “My Money” website. In Y. Eshet-Alkalai, A. Caspi,
S. Eden, N. Geri, & Y. Yair (Eds.), Learning in the Technological Era: Proceedings of the 6th Chais conference on instructional technologies research (pp. 92–100). Ra’anana, Israel:
The Open University of Israel.
Bryant, A., & Charmaz, K. (2012). Grounded theory and psychological research. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), Research
designs: Quantitative, qualitative, neuropsychological, and biological: Vol. 2. Apa handbook of research methods in psychology (pp. 39–56). Washington, DC: American
Psychological Association.
Caspi, A., & Blau, I. (2008). Social presence in online discussion groups: testing three conceptions and their relations to perceived learning. Social Psychology of Education, 11(3),
323–346.
Caspi, A., & Blau, I. (2011). Collaboration and psychological ownership: how does the tension between the two influence perceived learning? Social Psychology of Education,
14(2), 283–298.
Charsky, D., & Mims, C. (2008). Integrating commercial off-the-shelf video games into school curriculums. TechTrends, 52(5), 38–44.
Charsky, D., & Ressler, W. (2011). “Games are made for fun”: lessons on the effects of concept maps in the classroom use of computer games. Computers and Education, 56(3),
604–615.
Clark, D. B., & Martinez-Garza, M. (2012). Prediction and explanation as design mechanics in conceptually-integrated digital games to help players articulate the tacit un-
derstandings they build through gameplay. In C. Steinkuhler, K. Squire, & S. Barab (Eds.), Games, learning, and society: Learning and meaning in the digital age. Cambridge:
Cambridge University Press.
Clark, D. B., Nelson, B. C., Chang, H.-Y., Martinez-Garza, M., Slack, K., & D’Angelo, C. M. (2011). Exploring newtonian mechanics in a conceptually-integrated digital game:
comparison of learning and affective outcomes for students in taiwan and the united states. Computers and Education, 57(3), 2178–2195.
Clark, D. B., Nelson, B. C., D’Angelo, C. M., Slack, K., & Martinez-Garza, M. (2010). Surge: integrating Vygotsky’s spontaneous and instructed concepts in a digital game. In
Proceedings of the ninth international conference of the learning sciences, Chicago, IL.
Collins, A. (2006). Cognitive apprenticeship. In K. R. Sawyer (Ed.), The cambridge handbook of the learning sciences (pp. 47–60). Cambridge, UK: Cambridge University Press.
Cordova, D. I., & Lepper, M. R. (1996). Intrinsic motivation and the process of learning: beneficial effects of contextualization, personalization, and choice. Journal of Educational
Psychology, 88(4), 715–730.
Corkill, A. (1992). Advance organizers: facilitators of recall. Educational Psychology Review, 4(1), 33–67.
Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. New York, NY: Harper & Row.
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1992). Extrinsic and intrinsic motivation to use computers in the workplace. Journal of Applied Social Psychology, 22(14), 1111–1132.
Day, S. B., & Goldstone, R. L. (2012). The import of knowledge export: connecting findings and theories of transfer of learning. Educational Psychologist, 47(3), 153–176.
Deater-Deckard, K., Chang, M., & Evans, M. E. (2013). Engagement states and learning from educational games. New Directions for Child and Adolescent Development, 2013(139),
21–30.
De Grove, F., Bourgonjon, J., & Van Looy, J. (2012). Digital games in the classroom? A contextual approach to teachers’ adoption intention of digital games in formal education.
Computers in Human Behavior, 28(6), 2023–2033.
D’Mello, S., Lehman, B., Pekrun, R., & Graesser, A. C. (2013). Confusion can be beneficial for learning. Learning and Instruction (in press).
Efklides, A. (2008). Metacognition: defining its facets and levels of functioning in relation to self-regulation and co-regulation. European Psychologist, 13(4), 277–287.
Efklides, A., & Petkaki, C. (2005). Effects of mood on students’ metacognitive experiences. Learning and Instruction, 15(5), 415–431.
Engeser, S., & Rheinberg, F. (2008). Flow, performance and moderators of challenge-skill balance. Motivation and Emotion, 32(3), 158–172.
Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: a research and practice model. Simulation & Gaming, 33(4), 441–467.
Gee, J. P. (2007). What video games have to teach us about learning and literacy. New York, NY: Palgrave Macmillian.
Gee, J. P. (2009). Deep learning properties of good digital games: how far can they go? In U. Ritterfeld, M. J. Cody, & P. Vorderer (Eds.), Serious games: Mechanisms and effects
(pp. 67–82) New York, NY: Routledge.
Gee, J. P., & Hayes, E. (2011). Nurturing affinity spaces and game-based learning. In C. Steinkuehler, K. Squire, & S. Barab (Eds.), Games, learning, and society: Learning and
meaning in the digital age (pp. 129–153). New York, NY: Cambridge University Press.
Ghani, J. A., & Deshpande, S. P. (1994). Task characteristics and the experience of optimal flow in humandcomputer interaction. The Journal of Psychology, 128(4), 381–391.
Graesser, A. C., Chipman, P., Leeming, F., & Biedenbach, S. (2009). Deep learning and emotion in serious games. In U. Ritterfeld, M. J. Cody, & P. Vorderer (Eds.), Serious games:
Mechanisms and effects (pp. 81–100). New York, NY: Routledge.
Graesser, A. C., McNamara, D. S., & VanLehn, K. (2005). Scaffolding deep comprehension strategies through point&query, autotutor, and istart. Educational Psychologist, 40(4),
225–234.
Gurlitt, J., Dummel, S., Schuster, S., & Nückles, M. (2012). Differently structured advance organizers lead to different initial schemata and learning outcomes. Instructional
Science, 40(2), 351–369.
Habgood, M. P. J., & Ainsworth, S. E. (2011). Motivating children to learn effectively: exploring the value of intrinsic integration in educational games. Journal of the Learning
Sciences, 20(2), 169–206.
Hoffman, D. L., & Novak, T. P. (2009). Flow online: lessons learned and future prospects. Journal of Interactive Marketing, 23(1), 23–34.
Holbert, N. R., & Wilensky, U. (2012). Designing video games that encourage players to integrate formal representations with informal play. In Proceedings of the 10th in-
ternational conference of the learning sciences (ICLS), Sydney, Australia.
Honey, M. A., & Hilton, M. (2011). Learning science through computer games and simulations. Committee on Science Learning: Computer Games, Simulations, and Education,
National Research Council.
Inal, Y., & Cagiltay, K. (2007). Flow experiences of children in an interactive social game environment. British Journal of Educational Technology, 38(3), 455–464.
Jacoby, L., Bjork, R., & Kelley, C. M. (1994). Illusions of comprehension, competence, and remembering. In D. Druckman, & R. Bjork (Eds.), Learning, remembering, believing:
Enhancing human performance (pp. 57–80). Washington, DC: National Academy Press.
Johnson, C. I., & Mayer, R. E. (2010). Applying the self-explanation principle to multimedia learning in a computer-based game-like environment. Computers in Human
Behavior, 26(6), 1246–1252.
Kali, Y., Linn, M., & Roseman, J. E. (2008). Designing coherent science education: Implications for curriculum, instruction, and policy. New York, NY: Teachers College Press.
Kebritchi, M. (2010). Factors affecting teachers’ adoption of educational computer games: a case study. British Journal of Educational Technology, 41(2), 256–270.
Kiili, K. (2005a). Content creation challenges and flow experience in educational games: the it-emperor case. The Internet and Higher Education, 8(3), 183–198.
Kiili, K. (2005b). Digital game-based learning: towards an experiential gaming model. The Internet and Higher Education, 8(1), 13–24.
Kiili, K., & Lainema, T. (2008). Foundation for measuring engagement in educational games. Journal of Interactive Learning Research, 19(3), 469–488.
Kirriemuir, J., & McFarlane, A. (2004). Literature review in games and learning. Bristol, UK: Futurelab.
Klopfer, E., Osterweil, S., & Salen, K. (2009). Moving learning games forward: obstacles, opportunities, and openness. The Education Arcade: MIT.
Koriat, A., & Bjork, R. A. (2005). Illusions of competence in monitoring one’s knowledge during study. Journal of Experimental Psychology: Learning, Memory, and Cognition,
31(2), 187–194.
Koster, R. (2005). A theory of fun for game design. Scottsdale, AZ: Paraglyph Press.
Kozlov, M. D., & Johansen, M. K. (2010). Real behavior in virtual environments: psychology experiments in a simple virtual-reality paradigm using video games. Cyberp-
sychology, Behavior, and Social Networking, 13(6), 711–714.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.
Lee, J.-K., & Lee, W.-K. (2008). The relationship of e-learner’s self-regulatory efficacy and perception of e-learning environmental quality. Computers in Human Behavior, 24(1),
32–47.
Lepper, M. R., & Cordova, D. I. (1992). A desire to be taught: instructional consequences of intrinsic motivation. Motivation and Emotion, 16(3), 187–208.
Linn, M. C., Chang, H.-Y., Chiu, J., Zhang, H., & McElhaney, K. (2010). Can desirable difficulties overcome deceptive clarity in scientific visualizations? In A. S. Benjamin (Ed.),
Successful remembering and successful forgetting: A festschrift in honor of Robert A. Bjork (pp. 235–258) New York, NY: Psychology Press.
Linnenbrink, E. A., & Pintrich, P. R. (2002). Achievement goal theory and affect: an asymmetrical bidirectional model. Educational Psychologist, 37(2), 69–78.
Malone, T. W., & Lepper, M. R. (1987). Making learning fun: a taxonomy of intrinsic motivations for learning. In R. E. Snow, & M. J. Farr (Eds.), Vol. 3. Aptitude, learning, and
instruction (pp. 223–253). Hillsdale, NJ: Lawrence Erlbaum Associates.
Mayer, R. E. (1979a). Can advance organizers influence meaningful learning? Review of Educational Research, 49(2), 371–383.
Mayer, R. E. (1979b). Twenty years of research on advance organizers: assimilation theory is still the best predictor of results. Instructional Science, 8(2), 133–167.
Mayer, R. E., Mautone, P., & Prothero, W. (2002). Pictorial aids for learning by doing in a multimedia geology simulation game. Journal of Educational Psychology, 94(1), 171–
185.
McGonigal, J. (2011). Reality is broken: Why games make us better and how they can change the world. New York, NY, US: Penguin Press.
van Merrienboer, J. J. G., Kirschner, P. A., & Kester, L. (2003). Taking the load off a learner’s mind: instructional design for complex learning. Educational Psychologist, 38(1), 5–13.
Metcalfe, J. (2009). Metacognitive judgments and control of study. Current Directions in Psychological Science, 18(3), 159–163.
Meyer, D., & Turner, J. (2006). Re-conceptualizing emotion and motivation to learn in classroom contexts. Educational Psychology Review, 18(4), 377–390.
Neulight, N., Kafai, Y., Kao, L., Foley, B., & Galas, C. (2007). Children’s participation in a virtual epidemic in the science classroom: making connections to natural infectious
diseases. Journal of Science Education and Technology, 16(1), 47–58.
Paik, E. S., & Schraw, G. (2012). Learning with animation and illusions of understanding. Journal of Educational Psychology, 105(2), 278–289.
Papastergiou, M. (2009). Digital game-based learning in high school computer science education: impact on educational effectiveness and student motivation. Computers and
Education, 52(1), 1–12.
Parnafes, O., & Disessa, A. (2004). Relations between types of reasoning and computational representations. International Journal of Computers for Mathematical Learning, 9(3),
251–280.
Pearce, J. M., Ainley, M., & Howard, S. (2005). The ebb and flow of online learning. Computers in Human Behavior, 21(5), 745–771.
Pekrun, R., Goetz, T., Frenzel, A. C., Barchfeld, P., & Perry, R. P. (2011). Measuring emotions in students’ learning and performance: the achievement emotions questionnaire
(aeq). Contemporary Educational Psychology, 36(1), 36–48.
Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: a program of qualitative and quantitative
research. Educational Psychologist, 37(2), 91–105.
Pekrun, R., & Stephens, E. J. (2012). Academic emotions. In K. R. Harris, S. Graham, T. Urdan, J. M. Royer, & M. Zeidner (Eds.), Individual differences and cultural and contextual
factors: Vol. 2. Apa educational psychology handbook (pp. 3–31). Washington, DC, US: American Psychological Association.
Quintana, C., Reiser, B. J., Davis, E. A., Krajcik, J., Fretz, E., Duncan, R. G., et al. (2004). A scaffolding design framework for software to support science inquiry. Journal of the
Learning Sciences, 13(3), 337–386.
Quintana, C., Zhang, M., & Krajcik, J. (2005). A framework for supporting metacognitive aspects of online inquiry through software-based scaffolding. Educational Psychologist,
40(4), 235–244.
Reiser, B. J. (2004). Scaffolding complex learning: the mechanisms of structuring and problematizing student work. Journal of the Learning Sciences, 13(3), 273–304.
Rhodes, M. G., & Castel, A. D. (2008). Memory predictions are influenced by perceptual information: evidence for metacognitive illusions. Journal of Experimental Psychology:
General, 137(4), 615.
Ritterfeld, U., Cody, M. J., & Vorderer, P. (2009). Introduction. In U. Ritterfeld, M. J. Cody, & P. Vorderer (Eds.), Serious games: Mechanisms and effects (pp. 3–9). New York, NY:
Routledge.
Ritterfeld, U., & Weber, R. (2006). Video games for entertainment and education. In P. V. J. Bryant (Ed.), Playing video games: Motives, responses, and consequences (pp. 399–
413). Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers.
Salen, K., & Zimmerman, E. (2003). Rules of play: Game design fundamentals. Cambridge, MA: MIT Press.
Sandford, R., Ulicsak, M., Facer, K., & Rudd, T. (2006). Teaching with games: Using commercial off the-shelf computer games in formal education. Bristol, UK: Futurelab.
Serra, M. J., & Metcalfe, J. (2009). Effective implementation of metacognition. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Handbook of metacognition in education (pp.
278–298). Mahwah, NJ: Erlbaum.
Shapiro, A. (2008). Hypermedia design as learner scaffolding. Educational Technology Research and Development, 56(1), 29–44.
Skadberg, Y. X., & Kimmel, J. R. (2004). Visitors’ flow experience while browsing a web site: its measurement, contributing factors and consequences. Computers in Human
Behavior, 20(3), 403–422.
Stone, C. L. (1983). A meta-analysis of advance organizer studies. The Journal of Experimental Education, 51(4), 194–199.
Sweetser, P., & Wyeth, P. (2005). Gameflow: a model for evaluating player enjoyment in games. Computers in Entertainment, 3(3), 1–24.
Van Eck, R. (2006). Digital game-based learning: it’s not just the digital natives who are restless. Educause Review, 41(2), 16–30.
Veenman, M. V. J. (2011). Learning to self-monitor and self-regulate. In R. E. Mayer, & P. A. Alexander (Eds.), Handbook of research on learning and instruction (pp. 197–218).
New York, NY: Routledge.
Vorderer, P., Klimmt, C., & Ritterfeld, U. (2004). Enjoyment: at the heart of media entertainment. Communication Theory, 14(4), 388–408.
Webster, J., Trevino, L. K., & Ryan, L. (1993). The dimensionality and correlates of flow in human-computer interactions. Computers in Human Behavior, 9(4), 411–426.
Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2), 89–100.
Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of
Educational Psychology, 105(2), 249–265.

Barzilai & Blau 2016

Uploaded by

Document Informationclick to expand document information

Copyright:

Available Formats

Barzilai & Blau 2016

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Barzilai & Blau 2016

Uploaded by

Copyright:

Available Formats

Computers & Education 70 (2014) 65–79

Contents lists available at ScienceDirect

Computers & Education

Scaffolding game-based learning: Impact on learning achievements,

1.1. Informal and formal knowledge representations in game-based learning

1.2. Scaffolding knowledge construction in game-based learning

1.3. Flow, enjoyment, and their relation to learning

1.4. Perceived learning and learning achievements

2. Purpose of the current study

3.1. Research design

3.3.1. The Shakshouka Restaurant game

3.4.1. Formal problem-solving assessment

3.4.2. Perceived learning

3.4.3. Perceived game experiences

4.1. Impact on formal learning achievements

Conditions Pre-game problem-solving score Post-game problem-solving score

4.2. Impact on perceived learning

4.3. Impact on game experiences

4.5. Exploring the meanings of “learning” and “enjoyment”

Conditions Perceived learning Flow Enjoyment

Note. N ¼ 152 due to missing values, ***p < .001 (two-tailed).

5.1. Limitations and suggestions for future research

5.2. Implications for the design of game-based learning

Code Deﬁnition Examples

You might also like