short-paper

Open access

An open platform for full-body multisensory serious-games to teach geometry in primary school

Authors:

Gualtiero VolpeAuthors Info & Claims

MIE 2017: Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education

Pages 49 - 52

https://doi.org/10.1145/3139513.3139523

Published: 13 November 2017 Publication History

Abstract

Recent results from psychophysics and developmental psychology show that children have a preferential sensory channel to learn specific concepts. In this work, we explore the possibility of developing and evaluating novel multisensory technologies for deeper learning of arithmetic and geometry. The main novelty of such new technologies comes from the renewed understanding of the role of communication between sensory modalities during development that is that specific sensory systems have specific roles for learning specific concepts. Such understanding suggests that it is possible to open a new teaching/learning channel, personalized for each student based on the child’s sensory skills. Multisensory interactive technologies exploiting full-body movement interaction and including a hardware and software platform to support this approach will be presented and discussed. The platform is part of a more general framework developed in the context of the EU-ICT-H2020 weDRAW Project that aims to develop new multimodal technologies for multisensory serious-games to teach mathematics concepts in the primary school.

References

[1]

Camurri A., Mazzarino B., and Volpe G. 2004. Expressive Interfaces. Cognition Technology and Work 6, 1 (2004), 15-22.

Digital Library

[2]

Camurri A., Mazzarino B., Ricchetti M., Timmers R., and Volpe G. 2004. Multimodal analysis of expressive gesture in music and dance performance. In Gesture-Based Communication in Human-Computer Interaction, Springer Berlin Heidelberg (Ed.). 20-39.

[3]

Camurri A., Hashimoto S., Richetti M., Ricci A., Suzuki K., Trocca R., and Volpe G. 2000. EyesWeb: Toward Gesture and Affect Recognition in Interactive Dance and Music Systems. Computer Music Journal (2000), 57-69.

Digital Library

[4]

Normal D. 1993. Things that make us smart: Defending human attributes in the age of the machine. Addison-Wesley, New York, NY.

Digital Library

[5]

Flaugnacco E., Lopez L., Terribili C., Zoia S., Buda S., Tilli S., Monasta L., Montico M., Sila A., Ronfani L., and SchÃűn D. 2014. Rhythm perception and production predict reading abilities in developmental dyslexia. Frontiers in Human MIE’17, November 13, 2017, Glasgow, UK S.Ghisio et al. Neuroscience 8, 392 (2014), 1-14.

[6]

Dubus G. and Bresin R. 2013. A Systematic Review of Mapping Strategies for the Sonification of Physical Quantities. PLoS ONE 8, 12 (2013).

[7]

Fernald G. 1921. The effect of kinesthetic factors in development of word recognition in the case of non-readers. Journal of Educational Research 4 (1921).

[8]

Varni G. and Dubus G. et al. 2012. Interactive sonification of synchronisation of motor behaviour in social active listening to music with mobile devices. Journal on Multimodal User Interfaces 5 (2012), 157-173.

[9]

Barrett M. and Newell F. 2015. Task-Specific, Age Related Effects in the Cross-Modal Identification and Localisation of Objects. Multisensory research 28, 1-2 (2015), 111-151.

[10]

Ernst M. and Banks M. 2002. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415 (2002), 429-433.

[11]

Gori M., Del Viva M., Sandini G., and Burr D. C. 2008. Young children do not integrate visual and haptic form information. Current Biology 18, 9 (2008), 694-8.

[12]

Seitz A. R., Kim R., and Shams L. 2006. Sound facilitates visual learning. Current Biology 16 (2006), 1422-1427.

[13]

Kim R. S. 2008. Benefits of stimulus congruency for multisensory facilitation of visual learning. PLoS One 3, e.1532 (2008).

[14]

A. Strauss and L. E. Lehtinen. 1947. Psychopathology and Education of the Brain-Injured Child. Grune and Stratton.

[15]

Orton S. T. 1928. Specific reading disability - strephosymbolia. Journal of American Medical Association 90 (1928), 1095-1099.

Cited By

Nown TUpadhyay PKerr AAndonovic ITachtatzis CGrealy M(2023)A Mapping Review of Real-Time Movement Sonification Systems for Movement RehabilitationIEEE Reviews in Biomedical Engineering10.1109/RBME.2022.318784016(672-686)Online publication date: 2023
https://doi.org/10.1109/RBME.2022.3187840

Index Terms

An open platform for full-body multisensory serious-games to teach geometry in primary school
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. HCI design and evaluation methods
      1. Laboratory experiments

Recommendations

Enhancing children understanding of mathematics with multisensory technology
MOCO '18: Proceedings of the 5th International Conference on Movement and Computing

Recent results from psychophysics and developmental psychology show that children do not integrate and use the same sensory modalities in the same way, but they rather have a preferential sensory channel to learn specific concepts. In schools, however, ...
A multimodal serious-game to teach fractions in primary school
MIE 2017: Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education

Multisensory learning is considered a relevant pedagogical framework for education since a very long time and several authors support the use of a multisensory and kinesthetic approach in children learning. Moreover, results from psychophysics and ...
Exploiting multimodal integration in adaptive interactive systems and game-based learning interfaces
MOCO '18: Proceedings of the 5th International Conference on Movement and Computing

The main purpose of my work is to investigate multisensory and multimodal integration in the design and development of adaptive systems and interfaces for game-based learning applications in the areas of education and rehabilitation. To this aim, I ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MIE 2017: Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education

November 2017

75 pages

ISBN:9781450355575

DOI:10.1145/3139513

Copyright © 2017 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

H2020-EU.2.1.1. - INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Conference

ICMI '17

Sponsor:

SIGCHI

ICMI '17: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

November 13, 2017

Glasgow, UK

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
447
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)9

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nown TUpadhyay PKerr AAndonovic ITachtatzis CGrealy M(2023)A Mapping Review of Real-Time Movement Sonification Systems for Movement RehabilitationIEEE Reviews in Biomedical Engineering10.1109/RBME.2022.318784016(672-686)Online publication date: 2023
https://doi.org/10.1109/RBME.2022.3187840

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents