Replication and Extension of Video Game Demand Scale with a Turkish-Speaking Gamer Population

The original Video Game Demand Scale was developed in English, and it consisted of 26 items (see [13]; scale and underlying data available freely online at https://osf.io/x5jch/). The results of that validation reported five perceived demands, which included the cognitive, emotional, and social demands as expected, as well as two distinct types of physical demand that emerged during factor analyses: exertional (related to energy expenditure), and controller (considerations of the controller's usability, which corresponds to the effectiveness, efficiency, and satisfaction). More importantly, scores on the five demand dimensions were correlated as expected with relevant measures of gaming experiences. For example, increased cognitive and exertional demand were associated with higher assessments of task load on the NASA-Task Load Index (NASA-TLX [28]). Discrete demand dimensions also correlated uniquely with their associated perceptions of effort expenditure (i.e., mental effort was correlated with cognitive demand, physical effort was correlated with exertional demand, see [53]). Finally, demands were useful in distinguishing between a variety of gaming evaluations. For example, increased emotional demands were associated with increased feelings of appreciation from the in-game experience, as well as increased narrative ratings and increased positive affect towards the gaming experience [6] [16] [51]. Controller intuitiveness (an indicator of decreased controller demand) was inversely related to perceptions of gameplay and game control ratings as well as overall enjoyment and immersion, suggesting that overly demanding game controls are detrimental to game experiences [5] [15]. Social demands were strongly correlated with feelings of in-game relatedness, exertional demands were negatively associated with enjoyment, and cognitive demands increased feelings of autonomy and arousal.

This five-factor solution was further validated for use in VR-based video games [43], as well as for use in other regions, with translation and validation in German (VGDS-G; [37]) and Mandarin Chinese (VGDS-C; [10]). The German version replicated the original results, providing initial support for the scale. VGDS-C replicated the original results as well, although a 22-item shorter version showed a slightly better global model fit—most likely attributed to subtle language differences. Overall, previous replications and validations of the five-factor VGDS have been successful.

As with many types of empirical investigation, scale development involves collecting data from an initial group. However, those initial tests need replication with different and unique samples to provide convincing evidence of the scale's broad applicability and robustness. Likewise, multiple studies are needed to establish replicating patterns of data necessary for building knowledge [46]. Replication failures were at the heart of the so-called “replication crisis” in psychological sciences, in which less than one-third of published studies were able to be replicated when tested on novel samples [52]. Among the recommendations towards best practices to avoid these types of crises include collaborations between diverse authorship teams and designing direct replications which apply original designs with no alteration [36] [58]. The current study adopts both recommendations.

This replication crisis has been noted among the CHI community as well [7] [18]. Especially given that HCI domain traditionally rewards novel contributions (such as novel designs or system interfaces [70]), it can be difficult to find replication studies in the published record, and those existing replications are not initially planned or framed as such [33]. In response to these concerns, the RepliCHI workshop series was dedicated towards encouraging replication research [71], although as a community CHI generally lacks direct replications and policies around such work [2]. Here, our study is conducted in direct response to these critiques and calls for action.

Along with addressing a dearth of replications in science broadly and CHI specifically, the current study is also conducted in response to the well-known overreliance on so-called “WEIRD” populations in science (i.e., Western, educated, industrialized, rich, democratic; see [30]). Oversampling from WEIRD populations is concerning because it fails to consider potential variance among populations known to differ in terms of their perceptions, attitudes, motivations, and reasoning (among other cultural differences, see [31]). With the overabundance of research accumulated in decades with WEIRD populations, replications on samples outside these populations provide necessary novel contributions and potential corrections to how concepts are understood on a global scale [57].

Similar to the replication crisis, a reliance on WEIRD samples has also been recognized within the HCI community [62]. In an analysis of recently published empirical research in CHI proceedings from 2016 to 2020, 73% were based on Western samples that collectively represented less than 12% of the world population [44]. Although some diversity dimensions have been increasing in HCI research lately (such as having more diverse authorship teams), most research is still being shaped within WEIRD contexts [32]. Therefore, there is a need for increased efforts towards extending our findings to different countries and cultures to further the extent to which our research is applicable to various global communities.

Broadly speaking (and as with many global peoples), gaming is a popular activity among Turkish-speaking people. For example, social board gaming is a well-established tradition for Turkish people—it is easy to find groups playing Okey (described as being similar to Rummikub and Mahjong; [50]) and Tavla (Backgammon). During 1980s and 1990s, public arcades (often called “Atari Halls”) and Internet cafes focused on gaming were booming in the country as well [72]. As personal computers became more affordable and pervasive in households, digital video gaming has also become a prevalent hobby. As of this writing, total annual revenue of gaming exceeds 21.86 billion Turkish lira (about $1.2 billion) [22]. Recent reports show that gaming has become increasingly popular in Türkiye, with a nearly 17% growth in gamers from 2020 to 2021 and likewise, a 10% growth in Turkish game publishers during that same time frame [22]. These growth figures are reflective of Türkiye having been recognized as the fastest-growing global gaming market [54].

Although Turkish is not as commonly spoken globally (14th in terms of the global population's native speakers), Bhutada [4] reported that Turkish is the 4th most used language on the internet, which would be relevant for scale translation research in an increasingly online experience such as video gaming. Additionally, Türkiye has a unique position in which it borders Middle Eastern and European countries, creating a cross section of those cultures. This means that although one can assume similarities between the Western population and Turkish culture, it is also possible to see differentiation relevant to understanding non-WEIRD populations. Taken together, these makes Turkish localization and replication efforts especially important.

In this study we focus on Turkish-speaking video game players to validate and further expand the original VGDS. We tested the five-factor model of interactivity-as-demand as experienced by players, demonstrating that the a priori measurement model replicated with this novel population. Additionally, we examined correlations of discrete gaming dimensions and relevant game evaluations, and again found several points of replication such that specific gaming outcomes are influenced by unique perceptions of gaming demands. Along with adding further evidence in support of the interactivity-as-demand model, this research provides a useful metric for understanding gamers in the fastest-growing global games market.

From an initial response rate of N = 506 who accessed the shared survey URL, w removed n = 3 who did not name a focal video game, and n = 6 who did not complete all survey items for the VGDS-T; we also removed n = 287 who did not complete the survey through to the final demographic question (i.e., they left the survey early). Median time of completion for the survey was 1146 s (about 19 minutes) and removed n = 2 for completing the survey too quickly (under five minutes) and n = 15 who completed the survey too slowly (over sixty minutes); nine more participants were removed for self-reporting being under the age of 18. Our final sample for analysis was N = 184. Participation was incentivized with a drawing for a 1000 Turkish lira Amazon.com.tr gift card (one card for every 50 participants), and two of the three gift cards were claimed by prizewinners.

Participants reported k = 119 different video game titles, with the following titles being mentioned by at least 5 participants: Valorant (11), League of Legends (9), World of Warcraft (7), Stardew Valley (6), Counter Strike: Global Offensive (6), Stray (5).

About 93% of participants volunteered to answer at least some demographic questions (they were given the option to skip these). Of those who answered, 89% self-identified as male and 95% self-identified as Turkish, and the average age of a respondent to the survey was 25.2 years old (SD = 7.14), ranging from 18 to 69 years old. In terms of education, 12% of the participants had advanced degrees (Master's or PhD), 53% had undergraduate degrees, and the rest had either high school degrees or had completed only some formal education. All demographic questions were presented as open-ended and voluntary.

Potential respondents were invited to respond to a solicitation with the subject line, “Üniversite araştırmacılarıyız ve oyun deneyimlerinizle ilgili fikirlerinizi almak istiyoruz” (“We are university gaming researchers, and we'd like feedback on your gaming experiences”). On clicking the shared URL, a brief summary and informed consent document was presented discussing the purpose of the survey. Then, participants were asked to recall a recent gaming eperience and were given the survey items in the following order: VGDS-T items, measruement validity items (replicated from prior VGDS studies [10] [13] [37]), and an optional demographics self-reporting session. Survey responses were automatically logged throughout the online session. At the conclusion of the survey, respondents were given a hyperlink to a separate secured URL that included information on how to apply for the study lottery (the Amazon.com.tr cards, via validated email address). Drawing data was stored separately from survey response data. This study procedure was marked as exempt from institutional review by [host institution removed for anonymous peer review].

The 26 items from the original VGDS scale (VGDS-E) were submitted to a translation/back-translation method [14] to prepare them for a target Turkish-speaking population. First, the lead author of the current study (a native Turkish speaker also fluent in English) translated the VGDS-E into Turkish. Then, another translator—a native Turkish speaker fluent in English, independent from the research team, and with no prior knowledge of the VGDS-E—back-translated the Turkish version to English. Here, we aimed to utilize lead author's emic (insider, with intimate knowledge of video games and VGDS) point of view, whereas the other translator acted as the etic (outsider, with nominal knowledge of video games and no HCI experience) perspective [26] [27]. Then, two translators examined and discussed potential differences and nuances between their translations, consulted with the original authors of VGDS-E, and reconciled any differences for the final Turkish version. The readability of the Turkish scale was examined using the Ateşman Readability Index [1] [64] (similar to the Flesch reading ease score for English - [56]). Readability analysis determined that the average scores of the items were found to be 70.69 (SD = 15.35), which corresponds to a readability by 7^th and 8^th grades (13- and 14-year-olds). At the same time, our text as a whole was found to be suitable for students of 9^th and 10^th grades, corresponding to a scale comprehensible to 15- and 16-year-olds. Given that our participants were all older than 18 years old, and absent any other indicators of reading difficulty (such as feedback from participants), we accepted that our scale was readable to participants. Translation/back-translation documents are included online at (https://osf.io/7xydu/?view_only=8adfd784e69e470ca732a81c54f38268), as well as a complete Ateşman Readability analysis for each individual item. We also include the translated scale items in Appendix A.

All validation measures were pulled from identical items in prior VGDS research, and translated to Turkish following the same procedures as above. For convergent validity tests, we assessed players’ perceptions of task load using the NASA Task Load Index [28]. For predictive validity tests, we assessed self-assessments of the perceived mental, emotional, physical, and social effort expended during gameplay [53]. Finally for concurrent validity, we considered correlations with self-reported enjoyment and appreciation of the experience [51], assessments of need satisfaction stemming from gameplay (focusing on dimensions of autonomy, competence, and relatedness given prior research demonstrating their relevance for gaming experiences via self-determination theory in a variety of contexts; [38] [39] [40] [55] [65]), as well as discrete ratings of the video game's narrative, controls, sound, graphics, and overall game quality [51]. The descriptive statistics for these measures, as well as internal consistency scores for multiple-item measures (calculated using McDonald's ω, [29]) are reported in Table 1.

The complete 26-item VGDS-T showed an adequate fit to the expected five-factor structure, χ²(289) = 565.46, p < .001, SRMR = .075, RMSEA = .072 (90% CI .063, .081), CFI = .901, (for diagnostic cut-off criteria for each measure, see [9])—only CFI fell below an optimal value ≥ .95; see Table 2.

As CFA is a confirmatory hypothesis test, we avoided atheoretical post-hoc alterations to our measurement model that would erroneously capitalize on sample-specific variance [23]. However, we did consider the potential for the 22-item VGDS-C to be a superior fit. Notably, the four items removed in the 22-item VGDS-C had middling parameter estimates in the 26-item CFA of VGDS-T: two associated with cognitive demands, one associated with controller demand (the lone non-recoded item), and one associated with exertional demand (an item referencing “strain:” see Table 2). Indeed, the 22-item VGDS-T (a direct replication of the VGDS-C) reported slightly stronger fit global fit indices, χ2(199) = 374.12, p < .001, SRMR = .068, RMSEA = .069 (90% CI .058, .080), CFI = .926. Moreover, directly comparison of relative fit between the models suggested the 22-item measurement model (AIC = 14696, BIC = 14869) to be superior to the 26-item model (AIC = 17323, BIC = 17522); Δχ2 (90) = 191.34, p < .001; as expected, both models were superior to a unidimensional VGDS (AIC = 18962, BIC = 19128).

Most critically is that even with nominal differences between the VGDS-E (26 items) and VGDS-C (22 items) applied to a Turkish-speaking sample, the five factors of demands replicated as expected—cognitive, emotional, controller, exertional, and social demands. This data provides compelling evidence that gamers from various populations have shared perceptions of these demands and can readily distinguish between them in survey research.

In replicating prior validation work on the VGDS, convergent, predictive, and concurrent validity tests were calculated in three separate structural equation models. For ease of interpretation, results are presented in summary below in-text, with Table 3 offering the overall results. To avoid over-interpretation of statistically significant results due to alpha-inflation, only those results at the p <.001 level or greater are discussed below (and presented in Table 3, using standardized beta weights to aid in comparison across variables with different response scales). Validation tests below are performed with dimension scores from the 22-item version of VGDS, in favoring the parsimonious model solution [67].

Results showed that the five-factor structure from the interactivity-as-demand model was replicated among a novel sample of gamers. Items measuring the perceived cognitive, emotional, controller, exertional, and social demands of video games fit both a 26-item and 22-item scale, with the latter (previously validated with a Mandarin Chinese-speaking sample) reflecting a slightly stronger fit for the data than the former (previously validated with English- and German-speaking samples).

The measurement validity tests replicated several findings from prior VGDS literature, offering further evidence of the scale's measurement validity. For example, both cognitive and exertional demand were strongly associated with perceived overall task demand—an expected given that NASA-TLX questions tap more centrally into questions about cognitive and physical dimensions (e.g., questions about the how “mentally demanding” and “physically demanding” comprise two of the scale items, and others ask about more generalized effort and attentional demand [28]). Moreover, single item effort questions were found to be correlated to their respective demand subscales. For instance, mental effort was associated with cognitive demand, emotional effort was associated with emotional demand, exertional demand was associated with physical effort, and social demand was associated with social effort. These replications are key insofar as they demonstrate the discrete nature of perceived demand dimensions. In addition to the task load and effort related single-item questions, we also examined how demand scales relate to a broader set of game experience measures. Cognitive demand was found to be associated with autonomy appreciation—those deeper and more contemplative reactions to gaming experiences [6] [16], suggesting that more introspective reactions to gaming content require a fair amount of cognitive processing. Associations between cognitive demand and autonomy emerged as well, suggesting an interplay between game challenges and being allowed a greater sense of volition over one's in-game actions (especially relevant in sandbox-style video games, open-world gaming experiences, and battle arena games that encourage a wider variety of strategies). As expected, emotional demand was more strongly associated with appreciation, as well as assessments of autonomy, enjoyment, relatedness, and narrative rating—all replicating previous VGDS studies. Controller demand was strongly and negatively correlated with control rating, suggesting for example that demanding controller interfaces are those not well-mapped for gameplay; [42] [48]. Another expected finding was the strong association between social demand and relatedness, confirming prior work on the centrality of sociality for some gaming experiences [34] [59].

There were several other interesting findings to note, some unique to the current study. First, social demand was found to be associated with the NASA-TLX measure, and this finding had only previously been reported with the original VGDS-E research. One explanation is that games played in this study reflected more social experiences. For instance, Valorant, League of Legends, World of Warcraft, and Counter Strike: Global Offensive were the most common games played by our participants. These games not only require high levels of cognitive demand, but also are team-based games which require coordination among team members and as such, social interaction is a key component of gameplay (re: the task at hand). Second, overall game ratings were not associated with any one demand dimension at the p < .001 level, although the strongest association was found with emotional demand–this association did not exist in the German replication but was found in original English and Mandarin cases. One possibility here is that more complex statistical modeling could be required to tease out these patterns of covariance, as the structural equation models uses here model all demand dimensions as exogenous variables and all game evaluations as outcome measures in parallel to each other. These paths could be especially important within specific games—for example, massively multiplayer online games might be rated highly when specific features of those games increase cognitive and social demands, whereas a turn-based role-playing game might be rated more highly with a combination of cognitive and emotional demands (with a simplified menu-driven control system). Such associations could not be tested in the current data, as no one game emerged with large enough sample size to provide for a statistically robust secondary analysis. Finally, mental effort and emotional demand were strongly and negatively correlated—this finding had not been reported in any prior VGDS research. This might be due to a potential conflict between cognitive and emotional demands insofar as games that require a lot of problem-solving and challenge focus might pull away from players deeper emotional connection with the experience (i. e., a focus on problem-solving over poignancy [6] [51]), although this association could also be a sample-specific artifact (i.e., a Type 1 statistical conclusions error).

A key contribution of this research is our focus on a Turkish-speaking sample not commonly found in published HCI or video gaming research. Within the WEIRD framework [31], Türkiye is classified as a highly educated (E) upper middle status wealth (R) country, but also seen as a non-Western nation (W) with a hybrid democracy (D; not fully democratic but also not fully authoritarian) with an emerging-but-smaller economy (I). In totality, Türkiye can be understood as a non-WEIRD country for purposes of data collection [3]. From a cultural perspective as classified by [49], Türkiye is situated as rather distant from the United States, Germany, and Taiwan—all samples in prior VGDS research. That the five-factor structure of VGDS held in this somewhat culturally distinct sample is further evidence that gaming experiences maybe more universal or at least, players from a variety of backgrounds can still articulate and understand the variable cognitive, emotional, controller, exertional, and social demands of their gaming experiences.

This study is useful for HCI researchers in several ways. Given the rather large body of player psychology and user experience research within the HCI community, providing models that help better understand player experiences for a broad set of gaming experiences and gamified products is important for researchers to build on. A benefit of VGDS is that it is applicable to other gaming-related applications (such as VR-based games [43]) and the variable demand perceptions are relevant for a variety of interactive experiences. Establishing the reliability and validity of measurements through replication and extension provides confidence that those metrics useful for scholars and developers seeking to utilize them in their own research. Translation research is critical to providing metrics for observing and examining the experiences of under-represented populations—here, a Turkish-speaking population—and how they may (or in this case, may not) differ from other widely studied populations [44].

Although our findings are broadly supportive of our goals, they should be interpreted within the context of key limitations that require further investigation.

For example, although we sampled from a non-WEIRD population, the demographics of our sample skewed towards self-identifying males—a similar issue in prior VGDS research relying on social media posts (such as Reddit forums) for data collection. Such sampling is an incomplete picture of the tapestry of gamers playing and likewise, online forums such as Reddit are sites of toxic masculinity [45]. It is essential for future research to engage in more targeted and nuanced sampling. Likewise, the sample size for this study was considerably smaller than the sample for other VGDS validation studies (all sampling more than N = 500 participants), which raises questions of statistical power. That said, the five-factor structure still fit (in both 26- and 22-item configurations) and expected validation findings replicated past work. Given the paucity of research from an HCI perspective broadly (and gaming specifically) in a Turkish context [44], we see this study as an initial foray into the region. Future studies must consider more robust ways of engaging Turkish gamers, especially to be more inclusive for non-male participants. For example, future work might pursue data collection venues other than social media, such as universities or gaming conventions, or work with partners that can especially help target a variety of players (such as Turkish startup Spyke Games, known for its diverse and inclusive mission: https://spykegames.com/). Additionally, although our data do suggest that VGDS-T items were comprehended in Turkish as intended, more thorough translation methods could be used. Related to this, future research could be combined with inductive methods to further understand how Turkish speakers are interpreting scale items, as well as the broader notions of gaming demand.

Participants reported on a diverse set of video games, which allows some confidence in the generalizability of demand perceptions for myriad gaming experience. Likewise, asking players to recall “recent” rather than “favorite” gaming memories was an attempt to increase variance among the valence of player experiences, and decrease variance in the temporal distance of those memories (i.e., presuming gamers involved in reddit communities to be semi-regular gamers themselves). However, temporal distance is an enduring concern of any self-report/recollection research method. Future work would benefit from administering VGDS immediately after gameplay. Beyond controlling temporal distance, collecting demand data immediately following gameplay could allow for research unpacking discrete features of a video game that lead to variance in demand perceptions—a “demand signature” of sorts, for video games.

Finally, although we did focus on a Turkish-speaking population mostly residing in Türkiye, the survey itself does not specifically engage potentially unique aspects of Turkish culture. Likewise, many of the games mentioned were those that have global appeal, such as Valorant, League of Legends, and World of Warcraft. It would be compelling to examine this data in concert with specific cultural indicators, such as playing Turkish-made video games or versions of traditional games from the region.