research-article

Designing Embodied Musical Interaction for Children with Autism

Author:

Grazia RagoneAuthors Info & Claims

ASSETS '20: Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility

Article No.: 104, Pages 1 - 4

https://doi.org/10.1145/3373625.3417077

Published: 29 October 2020 Publication History

Abstract

This paper describes the design, implementation, and pilot evaluation of an interface to support embodied musical interaction for children with Autism Spectrum Conditions (ASC), in the context of music therapy sessions. Previous research suggests music and movement therapies are powerful tools for supporting children with autism in their development of communication, expression, and motor skills. OSMoSIS (Observation of Social Motor Synchrony with an Interactive System) is an interactive musical system which tracks body movements and transforms them into sounds using the Microsoft Kinect motion capture system. It is designed so that, regardless of motor abilities, children can generate sounds by moving in the environment either freely or guided by a facilitator. OSMoSIS was inspired by the author's experiences as a music therapist and supports observation of Social Motor Synchrony to allow facilitators and researchers to record and investigate this aspect of the therapy sessions. It converts movements into sounds using Microsoft Kinect body tracking, in the context of an interactive game. From our preliminary testing with 11 children with autism (aged 5 – 11 years old), we observed that our design actively connects children, who displayed a notable increase in engagement and interaction when the system was used.

References

[1]

American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders. American Psychiatric Publishing.

[2]

Bacon, A., Beaman, C. P., & Liu, F. (2020). An Exploratory Study of Imagining Sounds and ‘Hearing’ Music in Autism. Journal of Autism and Developmental Disorders, 50(4), 1123–1132.

[3]

Chawarska K, Shic F, Macari S, 18-month predictors of later outcomes in younger siblings of children with autism spectrum disorder: a baby siblings research consortium study. J Am Acad Child Adolesc Psychiatry. 2014;53(12):1317-1327.e1.

[4]

Cibrian, F.L., Pena, O., Ortega, D., Tentori, M., 2017 BendableSound: An elastic multisensory surface using touch-based interactions to assist children with severe autism during music therapy. Int J Hum-CompStudies. 107, May (2017), 22–37.

Digital Library

[5]

Cibrian, F.L., Vazquez, V., Cardenas, C., Tentori, M.,(2016). Designing a Musical Fabric-Based Surface to Encourage Children with Autism to Practice Motor Movements. In Proceeding of the 6th Mexican Conference on Human Computer Interaction, September (2016).

[6]

Colombo Dougovito, M., Block, M. E., (2017). Fundamental Motor Skill Interventions for Children and Adolescents on the Autism Spectrum: a Literature Review. Journal of Autism and Developmental Disorders volume 6, 159–171(2019 )

[7]

Forti, S., Colombo, B., Clark, J., Bonfanti, A., Molteni, S., Crippa, A., Antonietti, A., & Molteni, M. (n.d.). Soundbeam imitation intervention: Training children with autism to imitate meaningless body gestures through music. Advances in Autism

[8]

Frid, E. Accessible Digital Musical Instruments—A Review of Musical Interfaces in Inclusive Music Practice. Multimodal Technologies Interact. 2019, 3, 57.

[9]

Frid, E., Breson, R., Alborno, P., Elblaus, L., (2016) Interactive Sonification of Spontaneous Movement of Children-Cross-Modal aping and the Perception of Body Movement Qualities through Sound. Frontiers in Neuroscience. 10, November (2016)

[10]

Fournier, K. A., Hass, K. J., Naik, S. K., Lodha, N., Caraugh, J. H., (2010). Motor coordination in autism spectrum disorders: a synthesis and meta-analysis, Journal Autism Developmental Disorder.

[11]

Herrera, G., Casas, X., Sevilla, J., Rosa, L., Pardo, C., Plaza, J., Jordan, R., Le Groux, S., (2012). Pictogram Room: Natural Interaction Technologies to Aid in the Development of Children with Autism. Annuary of Clinical and Health Psychology, Vol. 8, pagg 39-44.

[12]

Hossein H., M., Khademi, M., (2016), A Review on Technical and Clinical Impact of Microsoft Kinect on Physical Therapy and Rehabilitation, Journal of Medical Engineering.

[13]

Leekam, S. (2016). Social cognitive impairment and autism: What are we trying to explain? Philosophical Transactions of the Royal Society B: Biological Sciences. Royal Society, 371(1686).

[14]

Leman, M., (2007). Embodied Music Cognition and Mediation Technology. Cambridge, MA:MIT Press.

[15]

James, R., Sigafoos, J., Green, V., Lancioni, G., O'Reilly, M., (2015). Music Therapy for Individuals with Autism Spectrum Disorder: a Systematic Review. Review Journal of Autism and Developmental Disorders. 2, 1, 39 – 54.

[16]

Keller, P.E., Rieger, M., (2009). Special issue–musical movement and synchronization. Music Perception. 26, 397–400.

[17]

Magee, W.L., (2006), Electronic technologies in clinical music therapy: a survey of practice and attitudes. Technology and Disability, 18(3), 139-146.

[18]

Mike Oliver. 2013. The social model of disability: Thirty years on. Disability & Society 28, 7 (2013), 1024–1026.

[19]

Ramseyer, F. T. (2019). Motion Energy Analysis (MEA). A primer on the assessment of motion from video. Journal of Counseling Psychology.

[20]

Rinehart, N, Bellgrove, M., Tonge, B., Brereton, A., Howells-Rankin, D., Bradshaw, J., (2006), An examination of movement kinematics in young people with high-functioning autism and Asperger's disorder: further evidence for a motor planning deficit. Journal of Autism and Developmental Disorders. 36, 6. Pag. 757-767.

[21]

Samuels, K. R., Enabling Creativity: Inclusive Music Interfaces and Practices. In Proceedings of the International Conference on Live Interfaces (ICLI), Lisbon, Portugal, 19–23 November 2016.

[22]

Srinivasan, S. M., & Bhat, A. N. (2013). A review of “music and movement” therapies for children with autism: Embodied interventions for multisystem development. Frontiers in Integrative Neuroscience, 7.

[23]

Sigrist, R., Rauter, G., Marchal-Crespo, L., Riener, T., Wolf, P., (2014). Sonification and haptic feedback in addition to visual feedback enhances complex motor task learning. Experimental Brain Research. 233,3 (2014), 909-925.

[24]

Teitelbaum, P., Teitelbaum, O., Nye, J., Fryman, J., Maurer, R.G., (1998). Movement analysis in infancy may be useful for early diagnosis of autism, Proceedings of the National Academy of Sciences of the United States of America, 95, 23, pag. 13982-13987, November 1998.

Cited By

Ferreira RCastro T(2024)Participatory and Inclusive Design Models from the Perspective of Universal Design for Children with Autism: A Systematic ReviewEducation Sciences10.3390/educsci1406061314:6(613)Online publication date: 6-Jun-2024
https://doi.org/10.3390/educsci14060613
Bennett CShelby RRostamzadeh NKane S(2024)Painting with Cameras and Drawing with Text: AI Use in Accessible CreativityProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675644(1-19)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675644
Sun JYang JZhou GJin YGong J(2024)Understanding Human-AI Collaboration in Music Therapy Through Co-Design with TherapistsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642764(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642764
Show More Cited By

Index Terms

Designing Embodied Musical Interaction for Children with Autism
1. Applied computing
2. Human-centered computing
  1. Human computer interaction (HCI)

Index terms have been assigned to the content through auto-classification.

Recommendations

Designing a Musical Fabric-Based Surface to Encourage Children with Autism to Practice Motor Movements
MexIHC'16: Proceedings of the 6th Mexican Conference on Human-Computer Interaction

Children with autism have difficulties in controlling the strength and timing of their movements. Neurologic music therapy uses musical instruments to promote motor development in children with autism. However, most musical instruments are not ...
Understanding the musical interaction of children with autism spectrum disorder using elastic display
Abstract
Musical interaction greatly benefits children’s development. However, case studies reporting musical interactions with children, especially those with neurodevelopmental disorders, like autism spectrum disorders (ASD), are scarce and urgently ...
Designing a deformable musical surface for children with autism
UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct

Children with autism with sensorimotor impairments found difficult to control their movements when responding to sensory stimuli. Neurologic music therapy, combining music and movement helps individuals with disabilities to better control their ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ASSETS '20: Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility

October 2020

764 pages

ISBN:9781450371032

DOI:10.1145/3373625

Editors:
Tiago Guerreiro
University of Lisbon, Portugal
,
Hugo Nicolau
INESC-ID, Portugal
,
Karyn Moffatt
McGill University, Canada

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGACCESS: ACM Special Interest Group on Accessible Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

ASSETS '20

Sponsor:

SIGACCESS

ASSETS '20: The 22nd International ACM SIGACCESS Conference on Computers and Accessibility

October 26 - 28, 2020

Virtual Event, Greece

Acceptance Rates

ASSETS '20 Paper Acceptance Rate 46 of 167 submissions, 28%;

Overall Acceptance Rate 436 of 1,556 submissions, 28%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
318
Total Downloads

Downloads (Last 12 months)62
Downloads (Last 6 weeks)2

Reflects downloads up to 11 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ferreira RCastro T(2024)Participatory and Inclusive Design Models from the Perspective of Universal Design for Children with Autism: A Systematic ReviewEducation Sciences10.3390/educsci1406061314:6(613)Online publication date: 6-Jun-2024
https://doi.org/10.3390/educsci14060613
Bennett CShelby RRostamzadeh NKane S(2024)Painting with Cameras and Drawing with Text: AI Use in Accessible CreativityProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675644(1-19)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675644
Sun JYang JZhou GJin YGong J(2024)Understanding Human-AI Collaboration in Music Therapy Through Co-Design with TherapistsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642764(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642764
Wu JFan MSheng LSun G(2023)Exploring the design space of virtual tutors for children with autism spectrum disorderEducation and Information Technologies10.1007/s10639-023-11838-x28:12(16531-16560)Online publication date: 16-May-2023
https://dl.acm.org/doi/10.1007/s10639-023-11838-x
Monarca ITentori MCibrian F(2023)Understanding the musical interaction of children with autism spectrum disorder using elastic displayPersonal and Ubiquitous Computing10.1007/s00779-022-01703-y27:5(1843-1860)Online publication date: 3-Feb-2023
https://dl.acm.org/doi/10.1007/s00779-022-01703-y
Iijima RShitara AOchiai Y(2022)Designing Gestures for Digital Musical Instruments: Gesture Elicitation Study with Deaf and Hard of Hearing PeopleProceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3517428.3544828(1-8)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1145/3517428.3544828
Barbareschi GInakage M(2022)Assistive or Artistic Technologies? Exploring the Connections between Art, Disability and Wheelchair UseProceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3517428.3544799(1-14)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1145/3517428.3544799
Ragone GHowland KBrulé E(2022)Evaluating Interactional Synchrony in Full-Body Interaction with Autistic ChildrenProceedings of the 21st Annual ACM Interaction Design and Children Conference10.1145/3501712.3529729(1-12)Online publication date: 27-Jun-2022
https://dl.acm.org/doi/10.1145/3501712.3529729
Qi DShibasaki MKamiyama YTanaka SKawasaki BMitsuhashi CPai YMinamizawa K(2022)Furekit: Wearable Tactile Music Toolkit for Children with ASDHaptics: Science, Technology, Applications10.1007/978-3-031-06249-0_35(310-318)Online publication date: 20-May-2022
https://doi.org/10.1007/978-3-031-06249-0_35
Nicolau HMoffatt KGuerreiro T(2021)Overview of ASSETS 2020ACM SIGACCESS Accessibility and Computing10.1145/3477315.3477316(1-6)Online publication date: 22-Jul-2021
https://dl.acm.org/doi/10.1145/3477315.3477316
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten