Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3573382.3616076acmconferencesArticle/Chapter ViewAbstractPublication Pageschi-playConference Proceedingsconference-collections
research-article

Preliminary Study of the Performance of the miniPXI when Measuring Player Experience throughout Game Development

Published: 06 October 2023 Publication History
  • Get Citation Alerts
  • Abstract

    Short questionnaires, using only a few items for measuring user experience related constructs, have been used in a variety of domains. In the field of games, the miniPXI is such a validated short version of the player experience inventory (PXI), containing 11 single items to measure 11 different PX-related constructs. Previous validations of the miniPXI were carried out in an experimental setting with existing, fully finished games. In this study, we conduct a preliminary investigation of the potential of miniPXI to evaluate prototypes during game development. We explore differences in PX across two iterations of nine games prototypes, based on input from 16 participants. Findings suggest that the miniPXI is capable of detecting differences between the two prototype versions. In addition, at the level of individual games, the miniPXI is effective at identifying differences in nearly all PX dimensions. However, we also find limited use for the single enjoyment item, and suggest that including alternative measures such as the Net Promotor Score may be more useful. Overall, this work suggests that the miniPXI has the potential to evaluate different iterations of game prototypes, starting from the earliest stages of game development.

    Supplementary Material

    ZIP file contains the following: 1: A PDF file with two Annexures Annex A: miniPXI and NPS items used in the survey Annex B: R Code with Mixed Effect Models for both research questions 2: Data Files Folder 14 CSV files with data in a Long format for individual miniPXI constructs, composite data of Functional and Psychosocial level constructs, and NPS rating Presenting at CHI PLAY 2023 (playcomp23wip-p1023-p-Archive.zip)

    References

    [1]
    Vero Vanden Abeele, Katta Spiel, Lennart Nacke, Daniel Johnson, and Kathrin Gerling. 2020. Development and validation of the player experience inventory: A scale to measure player experiences at the level of functional and psychosocial consequences. International Journal of Human Computer Studies 135, June 2019 (2020), 102370. https://doi.org/10.1016/j.ijhcs.2019.102370
    [2]
    Ahmad Azadvar and Alessandro Canossa. 2018. UPEQ: Ubisoft Perceived Experience Questionnaire: A Self-Determination Evaluation Tool for Video Games. In Proceedings of the 13th International Conference on the Foundations of Digital Games(FDG ’18). Association for Computing Machinery, New York, NY, USA, Article 5, 7 pages. https://doi.org/10.1145/3235765.3235780
    [3]
    Sven Baehre, Michele O’Dwyer, Lisa O’Malley, and Nick Lee. 2022. The use of Net Promoter Score (NPS) to predict sales growth: insights from an empirical investigation. Journal of the Academy of Marketing Science 50, 1 (01 Jan 2022), 67–84. https://doi.org/10.1007/s11747-021-00790-2
    [4]
    Douglas Bates, Martin Mächler, Ben Bolker, and Steve Walker. 2015. Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software 67, 1 (2015), 1–48. https://doi.org/10.18637/jss.v067.i01
    [5]
    Regina Bernhaupt. 2015. Game User Experience Evaluation. Springer, Cham, Switzerland.
    [6]
    Jeanne H Brockmyer, Christine M Fox, Kathleen A Curtiss, Evan McBroom, Kimberly M Burkhart, and Jacquelyn N Pidruzny. 2009. The development of the Game Engagement Questionnaire: A measure of engagement in video game-playing. Journal of Experimental Social Psychology 45, 4 (2009), 624–634. https://doi.org/10.1016/j.jesp.2009.02.016
    [7]
    M.-T. Cheng, H.-C. She, and L A Annetta. 2015. Game immersion experience: its hierarchical structure and impact on game-based science learning. Journal of Computer Assisted Learning 31, 3 (2015), 232–253. https://doi.org/10.1111/jcal.12066
    [8]
    Sung Hyeon Cheon and Johnmarshall Reeve. 2015. A classroom-based intervention to help teachers decrease students’ amotivation. Contemporary Educational Psychology 40 (Jan. 2015), 99–111. https://doi.org/10.1016/j.cedpsych.2014.06.004
    [9]
    Jacob Cohen. 2013. Statistical Power Analysis for the Behavioral Sciences. Routledge. https://doi.org/10.4324/9780203771587
    [10]
    Ciprian M Crainiceanu and David Ruppert. 2004. Likelihood ratio tests in linear mixed models with one variance component. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 66, 1 (2004), 165–185.
    [11]
    Alena Denisova, A. Imran Nordin, and Paul Cairns. 2016. The convergence of player experience questionnaires. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play. Association for Computing Machinery, New York, NY, USA, 33–37. https://doi.org/10.1145/2967934.2968095
    [12]
    Christyn L Dolbier, Judith A Webster, Katherine T McCalister, Mark W Mallon, and Mary A Steinhardt. 2005. Reliability and Validity of a Single-Item Measure of Job Satisfaction. American Journal of Health Promotion 19, 3 (2005), 194–198. https://doi.org/10.4278/0890-1171-19.3.194
    [13]
    Aimee L. Drolet and Donald G. Morrison. 2001. Do We Really Need Multiple-Item Measures in Service Research?Journal of Service Research 3, 3 (Feb. 2001), 196–204. https://doi.org/10.1177/109467050133001
    [14]
    Aqeel Haider, Casper Harteveld, Daniel Johnson, Max V. Birk, Regan L. Mandryk, Magy Seif El-Nasr, Lennart E. Nacke, Kathrin Gerling, and Vero Vanden Abeele. 2022. MiniPXI: Development and Validation of an Eleven-Item Measure of the Player Experience Inventory. Proc. ACM Hum.-Comput. Interact. 6, CHI PLAY, Article 244 (oct 2022), 26 pages. https://doi.org/10.1145/3549507
    [15]
    Yu-Guan Hsieh, Kimon Antonakopoulos, and Panayotis Mertikopoulos. 2021. Adaptive learning in continuous games: Optimal regret bounds and convergence to Nash equilibrium. In Conference on Learning Theory. PMLR, 2388–2422.
    [16]
    Charlene Jennett, Anna L. Cox, Paul Cairns, Samira Dhoparee, Andrew Epps, Tim Tijs, and Alison Walton. 2008. Measuring and defining the experience of immersion in games. International Journal of Human Computer Studies 66, 9 (Sept. 2008), 641–661. https://doi.org/10.1016/j.ijhcs.2008.04.004
    [17]
    Joseph R. Keebler, William J. Shelstad, Dustin C. Smith, Barbara S. Chaparro, and Mikki H. Phan. 2020. Validation of the GUESS-18: A Short Version of the Game User Experience Satisfaction Scale (GUESS). J. Usability Studies 16, 1 (nov 2020), 49–62.
    [18]
    Jeanine Krath, Maximilian Altmeyer, Gustavo F. Tondello, and Lennart E. Nacke. 2023. Hexad-12: Developing and Validating a Short Version of the Gamification User Types Hexad Scale. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 677, 18 pages. https://doi.org/10.1145/3544548.3580968
    [19]
    Alexandra Kuznetsova, Per B. Brockhoff, and Rune H. B. Christensen. 2017. lmerTest Package: Tests in Linear Mixed Effects Models. Journal of Statistical Software 82, 13 (2017), 1–26. https://doi.org/10.18637/jss.v082.i13
    [20]
    Robert Loo. 2002. A caveat on using single‐item versus multiple‐item scales. Journal of Managerial Psychology 17, 1 (01 Jan 2002), 68–75. https://doi.org/10.1108/02683940210415933
    [21]
    Pejman Mirza-Babaei, Lennart Nacke, Geraldine Fitzpatrick, Gareth White, Graham McAllister, and Nick Collins. 2012. Biometric Storyboards: Visualising Game User Research Data. In CHI ’12 Extended Abstracts on Human Factors in Computing Systems (Austin, Texas, USA) (CHI EA ’12). Association for Computing Machinery, New York, NY, USA, 2315–2320. https://doi.org/10.1145/2212776.2223795
    [22]
    Mark S. Nagy. 2002. Using a single-item approach to measure facet job satisfaction. Journal of Occupational and Organizational Psychology 75, 1 (2002), 77–86. https://doi.org/10.1348/096317902167658 arXiv:https://bpspsychub.onlinelibrary.wiley.com/doi/pdf/10.1348/096317902167658
    [23]
    Frederick F Reichheld. 2003. The one number you need to grow. Harvard business review 81, 12 (2003), 46–55.
    [24]
    Richard M Ryan, C Scott Rigby, and Andrew Przybylski. 2006. The Motivational Pull of Video Games: A Self-Determination Theory Approach. Motivation and Emotion 30, 4 (2006), 344–360. https://doi.org/10.1007/s11031-006-9051-8
    [25]
    Jeffrey M. Stanton, Evan F. Sinar, William K. Balzer, Amanda L. Julian, Paul Thoresen, Shahnaz Aziz, Gwenith G. Fisher, and Patricia C. Smith. 2002. Development of a compact measure of job satisfaction: The abridged Job Descriptive Index. Educational and psychological measurement 62, 1 (2002), 173–191. Publisher: Sage Publications Sage CA: Los Angeles, CA.
    [26]
    Team, R Core and others. 2013. R: A language and environment for statistical computing.
    [27]
    Margaret Verkuyl, Naza Djafarova, Paula Mastrilli, and Lynda Atack. 2022. Virtual Gaming Simulation: Evaluating Players’ Experiences. Clinical Simulation in Nursing 63 (2022), 16–22. https://doi.org/10.1016/j.ecns.2021.11.002
    [28]
    J P Wanous, A E Reichers, and M J Hudy. 1997. Overall job satisfaction: how good are single-item measures?The Journal of applied psychology 82, 2 (apr 1997), 247–252. https://doi.org/10.1037/0021-9010.82.2.247

    Cited By

    View all
    • (2024)Assessing the Effects of Various Gaming Platforms on Players’ Affective States and Workloads through ElectroencephalogramElectronics10.3390/electronics1311204313:11(2043)Online publication date: 23-May-2024

    Index Terms

    1. Preliminary Study of the Performance of the miniPXI when Measuring Player Experience throughout Game Development

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CHI PLAY Companion '23: Companion Proceedings of the Annual Symposium on Computer-Human Interaction in Play
      October 2023
      370 pages
      ISBN:9798400700293
      DOI:10.1145/3573382
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 06 October 2023

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. games user research
      2. player experience evaluation
      3. short questionnaire
      4. validity analysis

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      CHI PLAY '23
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 421 of 1,386 submissions, 30%

      Upcoming Conference

      CHI PLAY '24
      The Annual Symposium on Computer-Human Interaction in Play
      October 14 - 17, 2024
      Tampere , Finland

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)122
      • Downloads (Last 6 weeks)4
      Reflects downloads up to 26 Jul 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Assessing the Effects of Various Gaming Platforms on Players’ Affective States and Workloads through ElectroencephalogramElectronics10.3390/electronics1311204313:11(2043)Online publication date: 23-May-2024

      View Options

      Get Access

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media