research-article

Personalized Serious Games for Cognitive Intervention with Lifelog Visual Analytics

Authors:

Vigneshwaran Subbaraju,

Chee How Cheong,

Munirah Bashir,

Joo-Hwee LimAuthors Info & Claims

MM '18: Proceedings of the 26th ACM international conference on Multimedia

Pages 328 - 336

https://doi.org/10.1145/3240508.3240598

Published: 15 October 2018 Publication History

Abstract

This paper presents a novel serious game app and a method to cre- ate and integrate personalized game content based on lifelog visual analytics. The main objective is to extract personalized content from visual lifelogs, integrate it into mobile games, and evaluate the effect of personalization on user experience. First, a suite of visual analysis methods is proposed to extract semantic informa- tion from visual lifelogs and discover the association among the lifelog entities. The outcome is dataset that contains augmented and personal lifelog images. Next, a mobile game app is developed that makes use of the dataset as game content. Finally, an experiment is conducted to evaluate user gameplay behaviors in the wild over three months, where a mixture of generic and personalized game content is deployed. It is observed that user adherence is heightened by personalized game content as compared to generic content. Also observed is a higher enjoyment level in personalized than generic game content. The result provides the first empirical evidence of the effect of personalized games on user adherence and preference for cognitive intervention. This work paves the way for effective cognitive training with user-generated content.

References

[1]

J. A. Anguera, J. Boccanfuso, J. L. Rintoul, O. Al-Hashimi, F. Faraji, J. Janowich, E. Kong, Y. Larraburo, C. Rolle, E. Johnston, and A. Gazzaley. Video game training enhances cognitive control in older adults. Nature, 510(97--101), 2013.

[2]

J. A. Anguera and A. Gazzaley. Video games, cognitive exercises, and hte enhancement of cognitive abilities. Current Opinion in Behavioral Sciences, 4:160--165, 2015.

[3]

P. Belchior, M.Marsiske, S. Sisco, A. Yam, and W. Mann. Older adults' engagement with a video game training program. Act. Adapt. Aging, 36:269--279, 2012.

[4]

A. Bermingham, J. O'Rourke, C. Gurrin, R. Collins, K. Irving, and A. F. Smeaton. Automatically recommending multimedia content for use in group reminiscence therap. In MIIRH'13, pages 49--58, New York, NY, USA, 2013. ACM.

Digital Library

[5]

E. Berry, N. Kapur, L. Williams, S. Hodges, P. Watson, G. Smyth, J. Srinivasan, R. Smith, B. Wilson, and K. Wood. The use of a wearable camera, sensecam, as a pictorial diary to improve autobiographical memory in a patient with limbic encephalitis: A preliminary report. Neuropsychol Rehabil, 17(4--5):582--601, 2007.

[6]

K. A. Blocker, T. J. Wright, and W. R. Boot. Gaming preferences of aging generations. Gerontechnology, 12(3):174--184, 2014.

[7]

M. Bolanos, M. Dimiccoli, and P. Radeva. Toward storytelling from visual lifelogging: An overview. IEEE Trans. Human--Mach. Syst., 47:77--90, 2017.

[8]

W. R. Boot, D. Souders, N. Charness, K. Blocker, N. Roque, and T. Vitale. The gamification of cognitive training: Older adults' perceptions of and attitudes toward digital game-based interventions. In J. Zhou and G. Salvendy, editors, Lecture Notes in Computer Science, volume 9754 of Human Aspects of IT for the Aged Population. Design for Aging, ITAP 2016. Springer, Cham, 2016.

[9]

Y. Chen and G. J. Jones. Augmenting human memory using personal lifelogs. In AH'10. ACM, 2010.

Digital Library

[10]

M. Cotelli, R. Manenti, O. Zanetti, and C. Miniussi. Non-pharmacological intervention for memory decline. Frontiers in Human Neuroscience, 6:No. 46, 2012.

[11]

M. Csikszentmihalyi. Flow: The psychology of the optimal experience. New York: Harper & Row., 1990.

[12]

J. R. Finley, W. F. Brewer, and A. S. Benjamin. The effects of end-of-day picture review and a sensor-based picture capture procedure on autobiographical memory using sensecam. Memory, 19(7):796--807, 2011.

[13]

A. Garcia del Molino. First person view video summarization subject to the user needs. In MM '16, pages 1440--1444. ACM, 2016.

Digital Library

[14]

N. J. Gates and P. Sachdev. Is cognitive training an effective treatment for preclinical and early alzheimer's disease? Journal of Alzheimer's Disease, 42:S551--S559, 2014.

[15]

G. Gowans, J. Campbell, N. Alm, R. Dye, A. Astell, and M. Ellis. Designing a multimedia conversation aid for reminiscence therapy in dementia care environments. CHI EA '04, pages 825--836, New York, NY, USA, 2004. ACM.

Digital Library

[16]

K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR' 16, pages 770--778.

[17]

G. Kim, L. Sigal, and E. Xing. Joint summarization of large-scale collections of web images and videos for storyline reconstruction. In CVPR, 2014.

Digital Library

[18]

A. M. Kueider, J. M. Parisi, A. L. Gross, and G. W. Rebok. Computerized cognitive training with older adults: A systematic review. PLoS ONE, 7(7):e40588, 2012.

[19]

A. Lampit, H. Hallock, and M. Valenzuela. Computerized cognitive training in cognitively healthy older adults: A systematic review and meta-analysis of effect modifiers. PLOS Medicine, 11(11):e1001756, 2014.

[20]

M. Lee and K. Dey. Providing good memory cues for people with episodic memory impairment. In ASSETS'07, 2007.

Digital Library

[21]

T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and L. Zitnick. Microsoft coco: Common objects in context. In ECCV'14.

[22]

Z. Lu and K. Grauman. Story-driven summarization for egocentric video. 2013.

[23]

H. W. Mahncke, B. B. Connor, J. Appelman, O. N. Ahsanuddin, J. L. Hardy, R. A. Wood, N. M. Joyce, T. Boniske, S. M. Atkins, and M. M. Merzenich. Memory enhancement in healthy older adults using a brain plasticity-based training program: A randomized, controlled study. PNAS, 103(33):12523--28, 2006.

[24]

V. Manera, G. Ben-Sadoun, and T. Aalbers. Recommendations for the use of serious games in neurodegenerative disorders: 2016 delphi panel. Frontiers in Psychology, 8:1243:1--10, 2017.

[25]

A. McLaughlin, M. Gandy, J. Allaire, and L. Whitlock. Putting fun into aging? overcoming usability and motivational issues in video games for older adults. Ergonomics in Design, 20(13--20), 2012.

[26]

A. Mora, C. González, and J. Arnedo-Moreno. Gamification of cognitive training: A crowdsourcing- inspired approach for older adults. In Interacción '16, page Article No. 5, 2016.

Digital Library

[27]

A. C. Oei and M. D. Patterson. Enhancing cognition with video games: A multiple game training study. PLOS ONE, 8(3):e58546, 2013.

[28]

G. Oliveira-Barra, M. Bola nos, E. Talavera, A. Due nas, O. Gelonch, and M. Garolera. Serious games application for memory training using egocentric images. Sept. 11--15 2017.

[29]

C. Peretz, A. D. Korczyn, E. Shatil, V. Aharonson, S. Birnboim, and N. Giladi. Computer-based, personalized cognitive training versus classical computer games: A randomized double-blind prospective trial of cognitive stimulation. Neruoepidemiology, 36:91--99., 2011.

[30]

V. Pieramico, R. Esposito, S. Cesinaro, V. Frazzini, and S. L. Sensi. Effects of non-pharmacological or pharmacological interventions on cognition and brain plasticity of aging individuals. Front in Syst Neurosci, 8:153: 1--10, 2014.

[31]

S. Ren, K. He, R. Girshick, and J. Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6):1137 -- 1149, 2017.

Digital Library

[32]

J. P. Salmon, S. M. Dolan, R. S. Drake, G. C. Wilson, R. M. Klein, and G. A. Eskes. A survey of video game preferences in adults: Building better games for older adults. Entertainment Computing, 21:45--64, 2017.

[33]

V. Sarne-Fleischmann, N. Tractinsky, T. Dwolatzky, and I. Rief. Personalized reminiscence therapy for patients with alzheimer's disease using a computerized system. In PETRA '11, pages 48:1--48:4. ACM, 2011.

Digital Library

[34]

A. Sellen and S. Whittaker. Beyond total capture: A constructive critique of lifelogging. Communications of the ACM, 53(5):70--77, 2010.

Digital Library

[35]

E. Shatil. Does combined cognitive training and physical activity training enhance cognitive abilities more than either alone? a four-condition randomized controlled trial among healthy older adults. Front. Aging Neurosci., 5:8:1--12, 2013.

[36]

E. Shatil, A. Metzer, O. Horvitz, and A. Miller. Home-based personalized cognitive training in ms patients: A study of adherence and cognitive performance. NeuroRehabilitation, 26:143--153, 2010.

[37]

A. R. Silva, S. Pinho, L. M. Macedo, and C. J. Moulin. Benefits of sensecam review on neuropsychological test performance. Am J Prev Med, 44(3):302--307, 2013.

[38]

P. Siriaraya and C. S. Ang. Recreating living experiences from past memories through virtual worlds for people with dementia. In CHI'14, pages 3977--3986, 2014.

Digital Library

[39]

V. Subbaraju, Q. Xu, B. Mandal, L. Li, and J.-H. Lim. An empirical approach for automatic face clustering on personal lifelogging images. In ICSIP'17, pages 127--131.

[40]

C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. In AAAI'17, pages 4278--4284.

[41]

P. Wang and A. F. Smeaton. Using visual lifelogs to automatically characterize everyday activities. Information Sciences, 230:147--161, 2013.

Digital Library

[42]

S. L. Willis, S. L. Tennstedt, M. Marsiske, K. Ball, J. Elias, K. M. Koepke, J. N. Morris, G. W. Rebok, F. W. Unverzagt, A. M. Stoddard, and E. Wright. Long-term effects of cognitive training on everyday functional outcomes in older adults. JAMA, 296(23):2805--2814, 2006.

[43]

Q. Xu, V. Subbaraju, A. G. del Molino, J. Lin, F. Fang, J. Lim, L. Li, and V. Chandrasekhar. Visualizing personal lifelog data for deeper insights at the ntcir-13 lifelog-2 task. In NTCIR17, pages 33--39, 2017.

Cited By

Carvalho CTeran LMota MPereira R(2024)Technologies to Support Adaptable Game Design: A Systematic Mapping StudyJournal of the Brazilian Computer Society10.5753/jbcs.2024.309030:1(69-101)Online publication date: 26-Apr-2024
https://doi.org/10.5753/jbcs.2024.3090
Carvalho CTeran LMota MPereira RSantos CBento Villela MRios da Hora Rodrigues Kde Gois Ribeiro Darin T(2022)A systematic mapping study on digital game adaptation dimensionsProceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems10.1145/3554364.3559122(1-14)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3554364.3559122
Bassano CChessa MSolari F(2022)Visualization and Interaction Technologies in Serious and Exergames for Cognitive Assessment and Training: A Survey on Available Solutions and Their ValidationIEEE Access10.1109/ACCESS.2022.321056210(104295-104312)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3210562
Show More Cited By

Index Terms

Personalized Serious Games for Cognitive Intervention with Lifelog Visual Analytics
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
  2. Interaction design
    1. Interaction design process and methods
2. Software and its engineering
  1. Software organization and properties
    1. Contextual software domains
      1. Virtual worlds software
        Interactive games

Recommendations

Monitoring Cognitive Performance with a Serious Game: A Longitudinal Case Study on Online Cognitive Assessment Using Serious Games
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

This case study involves: the design and evaluation of serious games; the use of longitudinal research and remote testing in an international setting. Current methods for cognitive assessment tend to be inconvenient, costly and infrequently performed. ...
Voluntary Play in Serious Games
GALA 2015: Revised Selected Papers of the 4th International Conference on Games and Learning Alliance - Volume 9599

Voluntariness is an important feature of games. Serious game designers intend to generate engaging gameplay, which implies that voluntary play should be equally important for serious games as for entertainment games. This paper describes the outcome of ...
Gamifying Serious Games: Modding Modern Board Games to Teach Game Potentials
Gaming, Simulation and Innovations: Challenges and Opportunities
Abstract
The growing interest in game-based approaches is evident. But establishing Gamification and Serious Game processes are challenging, especially when there are few available resources and game design knowledge. Low-tech and low-budget games can be ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '18: Proceedings of the 26th ACM international conference on Multimedia

October 2018

2167 pages

ISBN:9781450356657

DOI:10.1145/3240508

General Chairs:
Susanne Boll
University of Oldenburg, Germany
,
Kyoung Mu Lee
Seoul National University, Korea
,
Jiebo Luo
University of Rochester, USA
,
Wenwu Zhu
Tsinghua University, China
,
Program Chairs:
Hyeran Byun
Yonsei University, Korea
,
Chang Wen Chen
State Univ. Of New York at Buffalo, USA
,
Rainer Lienhart
University of Augsburg, Germany
,
Tao Mei
JD AI, China

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

A*STAR JCO

Conference

MM '18

Sponsor:

SIGMM

MM '18: ACM Multimedia Conference

October 22 - 26, 2018

Seoul, Republic of Korea

Acceptance Rates

MM '18 Paper Acceptance Rate 209 of 757 submissions, 28%;

Overall Acceptance Rate 995 of 4,171 submissions, 24%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
341
Total Downloads

Downloads (Last 12 months)55
Downloads (Last 6 weeks)7

Reflects downloads up to 26 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Carvalho CTeran LMota MPereira R(2024)Technologies to Support Adaptable Game Design: A Systematic Mapping StudyJournal of the Brazilian Computer Society10.5753/jbcs.2024.309030:1(69-101)Online publication date: 26-Apr-2024
https://doi.org/10.5753/jbcs.2024.3090
Carvalho CTeran LMota MPereira RSantos CBento Villela MRios da Hora Rodrigues Kde Gois Ribeiro Darin T(2022)A systematic mapping study on digital game adaptation dimensionsProceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems10.1145/3554364.3559122(1-14)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3554364.3559122
Bassano CChessa MSolari F(2022)Visualization and Interaction Technologies in Serious and Exergames for Cognitive Assessment and Training: A Survey on Available Solutions and Their ValidationIEEE Access10.1109/ACCESS.2022.321056210(104295-104312)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3210562
Sajjadi PEwais ADe Troyer O(2022)Individualization in serious games: A systematic review of the literature on the aspects of the players to adapt toEntertainment Computing10.1016/j.entcom.2021.10046841(100468)Online publication date: Mar-2022
https://doi.org/10.1016/j.entcom.2021.100468
Xu QFang Fdel Molino ASubbaraju VLim JRanzato MBeygelzimer ADauphin YLiang PVaughan J(2021)Predicting event memorability from contextual visual semanticsProceedings of the 35th International Conference on Neural Information Processing Systems10.5555/3540261.3541979(22431-22442)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.5555/3540261.3541979
Ardakan MVahed Z(2019)A conceptual framework for cognitive game design analysis(CGDA)2019 International Serious Games Symposium (ISGS)10.1109/ISGS49501.2019.9046983(81-88)Online publication date: Dec-2019
https://doi.org/10.1109/ISGS49501.2019.9046983

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents