research-article

Motor Variability in Complex Gesture Learning: Effects of Movement Sonification and Musical Background

Authors:

Michelle Agnes Magalhaes,

Wendy E. Mackay,

Michel Beaudouin-Lafon,

Frédéric BevilacquaAuthors Info & Claims

ACM Transactions on Applied Perception (TAP), Volume 19, Issue 1

Article No.: 2, Pages 1 - 21

https://doi.org/10.1145/3482967

Published: 06 January 2022 Publication History

Abstract

With the increasing interest in movement sonification and expressive gesture-based interaction, it is important to understand which factors contribute to movement learning and how. We explore the effects of movement sonification and users’ musical background on motor variability in complex gesture learning. We contribute an empirical study in which musicians and non-musicians learn two gesture sequences over three days, with and without movement sonification. Results show the interlaced interaction effects of these factors and how they unfold in the three-day learning process. For gesture 1, which is fast and dynamic with a direct “action-sound” sonification, movement sonification induces higher variability for both musicians and non-musicians on day 1. While musicians reduce this variability to a similar level as no auditory feedback condition on day 2 and day 3, non-musicians remain to have significantly higher variability. Across three days, musicians also have significantly lower variability than non-musicians. For gesture 2, which is slow and smooth with an “action-music” metaphor, there are virtually no effects. Based on these findings, we recommend future studies to take into account participants’ musical background, consider longitudinal study to examine these effects on complex gestures, and use awareness when interpreting the results given a specific design of gesture and sound.

References

[1]

Jack A. Adams. 1971. A closed-loop theory of motor learning. J. Motor Behav. 3, 2 (1971), 111–150.

[2]

Sarah Fdili Alaoui, Baptiste Caramiaux, Marcos Serrano, and Frédéric Bevilacqua. 2012. Movement qualities as interaction modality. In Proceedings of the Designing Interactive Systems Conference. ACM, New York, NY, 761–769.

Digital Library

[3]

Jessalyn Alvina, Joseph Malloch, and Wendy Mackay. 2016. Expressive keyboards: Enriching gesture-typing on mobile devices. In Proceedings of the 29th ACM Symposium on User Interface Software and Technology (UIST’16). ACM, ACM, 583–593. https://doi.org/10.1145/2984511.2984560

Digital Library

[4]

Frédéric Bettens and Todor Todoroff. 2009. Real-time dtw-based gesture recognition external object for max/msp and puredata. In Proceedings of the SMC Sound and Music Computing Conference. 30–35.

[5]

Frédéric Bevilacqua, Eric O. Boyer, Jules Françoise, Olivier Houix, Patrick Susini, Agnès Roby-Brami, and Sylvain Hanneton. 2016. Sensori-motor learning with movement sonification: perspectives from recent interdisciplinary studies. Front. Neurosci. 10 (2016), 385.

[6]

Frédéric Bevilacqua, Baptiste Caramiaux, and Jules Françoise. 2016. Perspectives on Real-time Computation of Movement Coarticulation. In Proceedings of the 3rd International Symposium on Movement and Computing (MOCO’16). ACM, New York, NY, 35:1–35:5.

Digital Library

[7]

Frederic Bevilacqua, Fabrice Guédy, Norbert Schnell, Emmanuel Fléty, and Nicolas Leroy. 2007. Wireless sensor interface and gesture-follower for music pedagogy. In Proceedings of the 7th International Conference on New Interfaces for Musical Expression (NIME’07). ACM, New York, NY, 124–129. https://doi.org/10.1145/1279740.1279762

Digital Library

[8]

Frédéric Bevilacqua, Norbert Schnell, Jules Françoise, Éric O. Boyer, Diemo Schwarz, and Baptiste Caramiaux. 2017. Designing action-sound metaphors using motion sensing and descriptor-based synthesis of recorded sound materials. In The Routledge Companion to Embodied Music Interaction, Micheline Lesaffre, Pieter-Jan Maes, and Marc Leman (Eds.). Taylor & Francis, New York, NY, 391–401.

[9]

Eric O. Boyer, Frederic Bevilacqua, Emmanuel Guigon, Sylvain Hanneton, and Agnes Roby-Brami. 2020. Modulation of ellipses drawing by sonification. Exper. Brain Res. 238 (2020), 1011–1024.

[10]

Daniel A. Braun, Ad Aertsen, Daniel M. Wolpert, and Carsten Mehring. 2009. Motor task variation induces structural learning. Curr. Biol. 19, 4 (Feb. 2009), 352–7.

[11]

Roberto Bresin, Maurizio Mancini, Ludvig Elblaus, and Emma Frid. 2020. Sonification of the self vs. sonification of the other: Differences in the sonification of performed vs. observed simple hand movements. Int. J. Hum.-Comput. Stud. 144 (2020), 102500.

[12]

Antonio Camurri, Corrado Canepa, Nicola Ferrari, Maurizio Mancini, Radoslaw Niewiadomski, Stefano Piana, Gualtiero Volpe, Jean-Marc Matos, Pablo Palacio, and Muriel Romero. 2016. A system to support the learning of movement qualities in dance: A case study on dynamic symmetry. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. ACM, New York, NY, 973–976.

Digital Library

[13]

Baptiste Caramiaux, Alessandro Altavilla, Scott G. Pobiner, and Atau Tanaka. 2015. Form follows sound: Designing interactions from sonic memories. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15). ACM, New York, NY, 3943–3952.

Digital Library

[14]

Baptiste Caramiaux, Frédéric Bevilacqua, Marcelo M Wanderley, and Caroline Palmer. 2018. Dissociable effects of practice variability on learning motor and timing skills. PLoS One 13, 3 (2018), 18 pages.

[15]

Per-Erik Danielsson. 1980. Euclidean distance mapping. Comput. Graph. Image Process. 14, 3 (1980), 227–248.

[16]

Jérémy Danna, Maureen Fontaine, Vietminh Paz-Villagrán, Charles Gondre, Etienne Thoret, Mitsuko Aramaki, Richard Kronland-Martinet, Sølvi Ystad, and Jean-Luc Velay. 2015. The effect of real-time auditory feedback on learning new characters. Hum. Movement Sci. 43 (2015), 216–228.

[17]

Jérémy Danna and Jean-Luc Velay. 2017. Handwriting movement sonification: Why and how?IEEE Trans. Hum.-Mach. Syst. 47, 2 (2017), 299–303.

[18]

Jérémy Danna, Jean-Luc Velay, Vietminh Paz-Villagrán, Annabelle Capel, Céline Petroz, Charles Gondre, Etienne Thoret, Mitsuko Aramaki, Solvi Ystad, and Richard Kronland-Martinet. 2013. Handwriting Movement Sonification for the Rehabilitation of Dysgraphia. In Proceedings of the 10th International Symposium on Computer Music Multidisciplinary Research (CMMR’13). 200–208.

[19]

Keith Davids, Paul Glazier, Duarte Araujo, and Roger Bartlett. 2003. Movement systems as dynamical systems. Sports Med. 33, 4 (2003), 245–260.

[20]

Ashesh K. Dhawale, Maurice A. Smith, and Bence P. Ölveczky. 2017. The role of variability in motor learning. Annu. Rev. Neurosci. 40, 1 (2017), 479–498.

[21]

David H. Douglas and Thomas K. Peucker. 2011. Algorithms for the Reduction of the Number of Points Required to Represent a Digitized Line or Its Caricature. John Wiley & Sons, Ltd, New York, NY, 15–28.

[22]

Paul Dourish. 2004. Where the Action Is: The Foundations of Embodied Interaction. MIT Press, Cambridge, MA.

[23]

Gaël Dubus and Roberto Bresin. 2013. A systematic review of mapping strategies for the sonification of physical quantities. PLoS One 8, 12 (2013), e82491.

[24]

Gaël Dubus and Roberto Bresin. 2015. Exploration and evaluation of a system for interactive sonification of elite rowing. Sports Eng. 18, 1 (2015), 29–41.

[25]

J. F. Dyer, P. Stapleton, and M. W. M. Rodger. 2017. Advantages of melodic over rhythmic movement sonification in bimanual motor skill learning. Exp. Brain Res. 235, 10 (2017), 3129–3140.

[26]

Alfred O. Effenberg. 2005. Movement sonification: Effects on perception and action. IEEE Multimedia 12, 2 (2005), 53–59.

Digital Library

[27]

Alfred O. Effenberg, Ursula Fehse, Gerd Schmitz, Bjoern Krueger, and Heinz Mechling. 2016. Movement sonification: effects on motor learning beyond rhythmic adjustments. Front. Neurosci. 10 (2016), 219.

[28]

Paul M. Fitts. 1954. The information capacity of the human motor system in controlling the amplitude of movement.J. Exp. Psychol. 47, 6 (1954), 381.

[29]

Paul M. Fitts and Michael I. Posner. 1967. Human Performance. Brooks/Cole, Belmont, CA.

[30]

Jules Françoise, Olivier Chapuis, Sylvain Hanneton, and Frédéric Bevilacqua. 2016. SoundGuides: Adapting continuous auditory feedback to users. In Proceedings of the CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA’16). ACM, New York, NY, 2829–2836. https://doi.org/10.1145/2851581.2892420

Digital Library

[31]

Karmen Franinović and Stefania Serafin. 2013. Sonic Interaction Design. MIT Press, Cambridge, MA.

Digital Library

[32]

R. I. Godøy and M. Leman (Eds.). 2009. Musical Gestures: Sound, Movement and Meaning. Routledge, New York, NY.

[33]

Rolf Inge Godøy and Marc Leman. 2010. Musical Gestures: Sound, Movement, and Meaning. Routledge, New York, NY.

[34]

J. Guerra, L. Smith, D. Vicinanza, B. Stubbs, N. Veronese, and G. Williams. 2020. The use of sonification for physiotherapy in human movement tasks: A scoping review. Sci. Sports 35, 3 (2020), 119–129.

[35]

Richard W. Hamming. 1950. Error detecting and error correcting codes. Bell Syst. Techn. J. 29, 2 (1950), 147–160.

[36]

Luke S. Hopper, Andries L. Weidemann, and Janet Karin. 2018. The inherent movement variability underlying classical ballet technique and the expertise of a dancer. Res. Dance Educ. 19, 3 (2018), 229–239.

[37]

Jessica Hummel, Thomas Hermann, Christopher Frauenberger, and Tony Stockman. 2010. Interactive sonification of german wheel sports. In Proceedings of Interactive Sonification Workshop: Human Interaction with Auditory Displays (ISon’10). 17–22.

[38]

A. R. Jensenius, M. M. Wanderley, R. I. Godøy, and M. Leman. 2009. Musical gestures: Concepts and methods in research. In Musical Gestures: Sound, Movement, and Meaning. Routledge, New York, NY.

[39]

Richard Kirby. 2009. Development of a real-time performance measurement and feedback system for alpine skiers. Sports Technol. 2, 1–2 (2009), 43–52.

[40]

John W. Krakauer and Pietro Mazzoni. 2011. Human sensorimotor learning: Adaptation, skill, and beyond. Curr. Opin. Neurobiol. 21, 4 (2011), 636–644.

[41]

Guillaume Lemaitre, Olivier Houix, Patrick Susini, Yon Visell, and Karmen Franinović. 2012. Feelings elicited by auditory feedback from a computationally augmented artifact: The flops. IEEE Trans. Affect. Comput. 3, 3 (2012), 335–348.

Digital Library

[42]

Marc Leman. 2007. Embodied Music Cognition and Mediation Technology. MIT Press, Cambridge, MA.

Digital Library

[43]

Marc Leman. 2016. The Expressive Moment: How Interaction (with Music) Shapes Human Empowerment. MIT Press, Cambridge, MA.

[44]

Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Sov. Phys. Doklady 10, 8 (1966), 707–710.

[45]

Pieter-Jan Maes, Valerio Lorenzoni, and Joren Six. 2019. The SoundBike: Musical sonification strategies to enhance cyclists’ spontaneous synchronization to external music. J. Multimodal User Interfaces 13, 3 (2019), 155–166.

[46]

Richard A. Magill and David I. Anderson. 2007. Motor Learning and Control: Concepts and Applications. Vol. 11. McGraw–Hill, New York, NY.

[47]

Claire F. Michaels and Claudia Carello. 1981. Direct Perception. Prentice-Hall, Englewood Cliffs, NJ.

[48]

Benjamin Matuszewski, Michelle Agnes Magalhaes, and Frédéric Bevilacqua. 2020. Constella(c)tions. Technical Report. STARTS Residency Public Report.

[49]

Florian Mueller and Katherine Isbister. 2014. Movement-based game guidelines. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 2191–2200.

Digital Library

[50]

Mauro Murgia, Federica Corona, Roberta Pili, Fabrizio Sors, Tiziano Agostini, Carlo Casula, Massimiliano Pau, and Marco Guicciardi. 2015. Rhythmic auditory stimulation (RAS) and motor rehabilitation in Parkinson’s disease: New frontiers in assessment and intervention protocols. Open Psychol. J. 8, Suppl 3: M7 (2015), 220–229.

[51]

Elena Nava and Ana Tajadura-Jiménez. 2020. Auditory-induced body distortions in children and adults. Sci. Rep. 10, 1 (2020), 1–14.

[52]

John G. Neuhoff, Rebecca Knight, and Joseph Wayand. 2002. Pitch change, sonification, and musical expertise: Which way is up? In Proceedings of the International Conference on Auditory Display. Georgia Institute of Technology, Atlanta, GA.

[53]

Joseph W. Newbold, Nadia Bianchi-Berthouze, Nicolas E. Gold, Ana Tajadura-Jiménez, and Amanda CdC Williams. 2016. Musically informed sonification for chronic pain rehabilitation: Facilitating progress & avoiding over-doing. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI’16). Association for Computing Machinery, New York, NY, 5698–5703.

Digital Library

[54]

Karl M. Newell and Andrew B. Slifkin. 1998. The nature of movement variability. In Motor Behavior and Human Skill: A Multidisciplinary Approach. Human Kinetics, Champaign, IL, 143–160.

[55]

Radoslaw Niewiadomski, Maurizio Mancini, Andrea Cera, Stefano Piana, Corrado Canepa, and Antonio Camurri. 2019. Does embodied training improve the recognition of mid-level expressive movement qualities sonification?J. Multimodal User Interfaces 13, 3 (2019), 191–203.

[56]

Nikou Nikmaram, Daniel S. Scholz, Michael Großbach, Simone B. Schmidt, Jakob Spogis, Paolo Belardinelli, Florian Müller-Dahlhaus, Jörg Remy, Ulf Ziemann, Jens D. Rollnik, and Eckart Altenmüller. 2019. Musical sonification of arm movements in stroke rehabilitation yields limited benefits. Front. Neurosci. 13 (2019), 1378.

[57]

Nikou Nikmaram, Daniel S. Scholz, Michael Großbach, Simone B. Schmidt, Jakob Spogis, Paolo Belardinelli, Florian Müller-Dahlhaus, Jörg Remy, Ulf Ziemann, Jens D. Rollnik, and Eckart Altenmüller. 2019. Musical sonification of arm movements in stroke rehabilitation yields limited benefits. Front. Neurosci. 13 (2019), 1378.

[58]

Benjamin O’Brien, Brett Juhas, Marta Bieńkiewicz, Frank Buloup, Lionel Bringoux, and Christophe Bourdin. 2020. Online sonification for golf putting gesture: Reduced variability of motor behaviour and perceptual judgement. Exp. Brain Res. 238, 4 (2020), 883–895.

[59]

Katerina El Raheb, Marina Stergiou, Akrivi Katifori, and Yannis Ioannidis. 2019. Dance interactive learning systems: A study on interaction workflow and teaching approaches. ACM Comput. Surv. 52, 3 (2019), 1–37.

Digital Library

[60]

Urs Ramer. 1972. An iterative procedure for the polygonal approximation of plane curves. Comput. Graphics Image Process. 1, 3 (1972), 244–256. https://doi.org/10.1016/S0146-664X(72)80017-0

[61]

P. Ripollés, N. Rojo, J. Grau-Sánchez, J. L. Amengual, E. Càmara, J. Marco-Pallarés, M. Juncadella, L. Vaquero, F. Rubio, E. Duarte, et al. 2016. Music supported therapy promotes motor plasticity in individuals with chronic stroke. Brain Imaging Behav. 10, 4 (2016), 1289–1307.

[62]

Matthew W. M. Rodger and Cathy M. Craig. 2016. Beyond the metronome: Auditory events and music may afford more than just interval durations as gait cues in Parkinson’s disease. Front. Neurosci. 10 (2016), 272.

[63]

Giulio Rosati, Antonio Rodà, Federico Avanzini, and Stefano Masiero. 2013. On the role of auditory feedback in robot-assisted movement training after stroke: Review of the literature. Comput. Intell. Neurosci. (2013), 11.

Digital Library

[64]

Emanuele Ruffaldi, Alessandro Filippeschi, Antonio Frisoli, Oscar Sandoval, Carlo Alberto Avizzano, and Massimo Bergamasco. 2009. Vibrotactile perception assessment for a rowing training system. In Proceedings of the 3rd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (World Haptics’09). IEEE, Piscataway, NJ, 350–355.

Digital Library

[65]

Abraham Savitzky and Marcel J. E. Golay. 1964. Smoothing and differentiation of data by simplified least squares procedures.Anal. Chem. 36, 8 (1964), 1627–1639.

[66]

Nina Schaffert, Thenille Braun Janzen, Klaus Mattes, and Michael H. Thaut. 2019. A review on the relationship between sound and movement in sports and rehabilitation. Front. Psychol. 10 (2019), 244.

[67]

Andrea Schiavio and Eckart Altenmüller. 2015. Exploring music-based rehabilitation for Parkinsonism through embodied cognitive science. Front. Neurol. 6 (2015), 217.

[68]

Richard A. Schmidt. 1975. A schema theory of discrete motor skill learning.Psychol. Rev. 82, 4 (1975), 225.

[69]

Richard A. Schmidt. 1991. Frequent augmented feedback can degrade learning: Evidence and interpretations. In Tutorials in Motor Neuroscience. Springer, Heidelberg, 59–75.

[70]

Richard A. Schmidt, Timothy D. Lee, Carolee Winstein, Gabriele Wulf, and Howard N. Zelaznik. 2018. Motor Control and Learning: A Behavioral Emphasis. Human Kinetics, Champaign, IL.

[71]

Gerd Schmitz, Bahram Mohammadi, Anke Hammer, Marcus Heldmann, Amir Samii, Thomas F. Münte, and Alfred O. Effenberg. 2013. Observation of sonified movements engages a basal ganglia frontocortical network. BMC Neurosci. 14, 1 (2013), 32.

[72]

Ludovic Seifert, Chris Button, and Keith Davids. 2013. Key properties of expert movement systems in sport. Sports Med. 43, 3 (2013), 167–178.

[73]

Ludovic Seifert, Hugues Leblanc, Romain Herault, John Komar, Chris Button, and Didier Chollet. 2011. Inter-individual variability in the upper–lower limb breaststroke coordination. Hum. Movement Sci. 30, 3 (2011), 550–565.

[74]

Roland Sigrist, Georg Rauter, Robert Riener, and Peter Wolf. 2013. Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review. Psychon. Bull. Rev. 20, 1 (2013), 21–53.

[75]

Michael J. Spilka, Christopher J. Steele, and Virginia B. Penhune. 2010. Gesture imitation in musicians and non-musicians. Exp. Brain Res. 204, 4 (2010), 549–558.

[76]

Nicholas Stergiou and Leslie M. Decker. 2011. Human movement variability, nonlinear dynamics, and pathology: Is there a connection?Hum. Movement Sci. 30, 5 (2011), 869–888.

[77]

Sandra Sülzenbrück and Herbert Heuer. 2011. Type of visual feedback during practice influences the precision of the acquired internal model of a complex visuo-motor transformation. Ergonomics 54, 1 (2011), 34–46.

[78]

Ana Tajadura-Jimenez, Nadia Bianchi-Berthouze, Enrico Furfaro, and Frederic Bevilacqua. 2015. Sonification of surface tapping changes behavior, surface perception, and emotion. IEEE MultiMedia 22, 1 (2015), 48–57. https://doi.org/10.1109/MMUL.2015.14

Digital Library

[79]

Ana Tajadura-Jiménez, Maria Vakali, Merle T. Fairhurst, Alisa Mandrigin, Nadia Bianchi-Berthouze, and Ophelia Deroy. 2018. Author correction: Contingent sounds change the mental representation of one’s finger length. Sci. Rep. 8, 1 (2018), 4875.

[80]

Atau Tanaka. 2019. Embodied musical interaction. In New Directions in Music and Human-Computer Interaction. Springer, Heidelberg, 135–154.

[81]

Michael Thaut and Volker Hoemberg. 2014. Handbook of Neurologic Music Therapy. Oxford University Press, Oxford, UK.

[82]

Carlos Trenado, Anaí González-Ramírez, Victoria Lizárraga-Cortés, Nicole Pedroarena Leal, Elias Manjarrez, and Diane Ruge. 2019. The potential of trial-by-trial variabilities of ongoing-EEG, evoked potentials, event related potentials and fMRI as diagnostic markers for neuropsychiatric disorders. Front. Neurosci. 12 (2019), 850.

[83]

William E. Winkler. 1990. String comparator metrics and enhanced decision rules in the Fellegi-Sunter model of record linkage. In Proceedings of the Survey Research Methods Section. American Statistical Association, Alexandria, VA, 354–359.

[84]

Daniel M. Wolpert, Jörn Diedrichsen, and J. Randall Flanagan. 2011. Principles of sensorimotor learning. Nat. Rev. Neurosci. 12, 12 (2011), 739–751.

[85]

Howard G. Wu, Yohsuke R. Miyamoto, Luis Nicolas Gonzalez Castro, Bence P. Ölveczky, and Maurice A. Smith. 2014. Temporal structure of motor variability is dynamically regulated and predicts motor learning ability. Nat. Neurosci. 17, 2 (2014), 312–321.

[86]

Gabriele Wulf. 2013. Attentional focus and motor learning: A review of 15 years. Int. Rev. Sport Exercise Psychol. 6, 1 (2013), 77–104.

[87]

Gabriele Wulf and Charles H. Shea. 2002. Principles derived from the study of simple skills do not generalize to complex skill learning. Psychon. Bull. Rev. 9, 2 (2002), 185–211.

[88]

Goh Yamamoto, Kensuke Shiraki, Masami Takahata, Yutaka Sakane, and Yoichi Takebayashi. 2004. Multimodal knowledge for designing new sound environments. In Proceedings of the Workshop “Mobile HCI and Sound Workshop” of the 6th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI’04).

Cited By

Barkasi MBansal AJörges BHarris L(2024)Online reach adjustments induced by real-time movement sonificationHuman Movement Science10.1016/j.humov.2024.10325096(103250)Online publication date: Aug-2024
https://doi.org/10.1016/j.humov.2024.103250
Li SMa YZhang YXie Y(2024)Towards Enhanced Energy Aware Resource Optimization for Edge Devices Through Multi-cluster Communication SystemsJournal of Grid Computing10.1007/s10723-024-09773-322:2Online publication date: 6-Jun-2024
https://dl.acm.org/doi/10.1007/s10723-024-09773-3
Peyre IRoby-Brami ASegalen MGiron ACaramiaux BMarchand-Pauvert VPradat-Diehl PBevilacqua F(2023)Effect of sonification types in upper-limb movement: a quantitative and qualitative study in hemiparetic and healthy participantsJournal of NeuroEngineering and Rehabilitation10.1186/s12984-023-01248-y20:1Online publication date: 5-Oct-2023
https://doi.org/10.1186/s12984-023-01248-y
Show More Cited By

Index Terms

Motor Variability in Complex Gesture Learning: Effects of Movement Sonification and Musical Background
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. Interaction devices
      1. Sound-based input / output

Recommendations

Somatic Sonification in Dance Performances. From the Artistic to the Perceptual and Back
MOCO '20: Proceedings of the 7th International Conference on Movement and Computing

Since the end of the 1980s, interactive musical systems have played an increasingly relevant role in dance performances. More recently, the use of interactive auditory feedback for sensorimotor learning such as movement sonification has gained currency ...
Visual cues effect on the impact of sonification on movement
MOCO '18: Proceedings of the 5th International Conference on Movement and Computing

When designing movement sonification, its effect on peoples' movement must be considered. Recent work has shown that using musical expectancy within movement sonification can impact the way people move. However, this body of work has also found that ...
Learning musical instrument skills through interactive sonification
NIME '06: Proceedings of the 2006 conference on New interfaces for musical expression

Interactive visualisation methods have been used to understand real-time acoustic analysis for the purpose of learning musical skills. However, interactive sonification has not often been considered, perhaps because it is assumed the musician cannot ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Applied Perception

ACM Transactions on Applied Perception Volume 19, Issue 1

January 2022

75 pages

ISSN:1544-3558

EISSN:1544-3965

DOI:10.1145/3505204

Editor:
Bobby Bodenheimer
Vanderbilt University, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 January 2022

Accepted: 01 August 2021

Revised: 01 January 2021

Received: 01 May 2020

Published in TAP Volume 19, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

ELEMENT – Enabling Learnability in Embodied Movement Interaction
European Research Council (ERC)
Insead-Sorbonne University Behavioral Lab.

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
541
Total Downloads

Downloads (Last 12 months)110
Downloads (Last 6 weeks)7

Reflects downloads up to 23 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Barkasi MBansal AJörges BHarris L(2024)Online reach adjustments induced by real-time movement sonificationHuman Movement Science10.1016/j.humov.2024.10325096(103250)Online publication date: Aug-2024
https://doi.org/10.1016/j.humov.2024.103250
Li SMa YZhang YXie Y(2024)Towards Enhanced Energy Aware Resource Optimization for Edge Devices Through Multi-cluster Communication SystemsJournal of Grid Computing10.1007/s10723-024-09773-322:2Online publication date: 6-Jun-2024
https://dl.acm.org/doi/10.1007/s10723-024-09773-3
Peyre IRoby-Brami ASegalen MGiron ACaramiaux BMarchand-Pauvert VPradat-Diehl PBevilacqua F(2023)Effect of sonification types in upper-limb movement: a quantitative and qualitative study in hemiparetic and healthy participantsJournal of NeuroEngineering and Rehabilitation10.1186/s12984-023-01248-y20:1Online publication date: 5-Oct-2023
https://doi.org/10.1186/s12984-023-01248-y
Zheng JLiu YFeng YXu HZhang M(2023)Contrastive Attention-guided Multi-level Feature Registration for Reference-based Super-resolutionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/361649520:2(1-21)Online publication date: 18-Oct-2023
https://dl.acm.org/doi/10.1145/3616495
Wei JYang YXu XSong JWang GShen H(2023)Less is Better: Exponential Loss for Cross-Modal MatchingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.324975433:9(5271-5280)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1109/TCSVT.2023.3249754
Turner CGoubault EMaso FBegon MVerdugo F(2023)The influence of proximal motor strategies on pianists' upper-limb movement variabilityHuman Movement Science10.1016/j.humov.2023.10311090(103110)Online publication date: Aug-2023
https://doi.org/10.1016/j.humov.2023.103110
Liu YKim K(2023)An artificial-intelligence-driven product design framework with a synergistic combination of Genetic Algorithm and Particle Swarm OptimizationSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-023-09223-427:23(17621-17638)Online publication date: 23-Sep-2023
https://dl.acm.org/doi/10.1007/s00500-023-09223-4
Ley-Flores JAlshami ESingh ABevilacqua FBianchi-Berthouze NDeroy OTajadura-Jiménez A(2022)Effects of pitch and musical sounds on body-representations when moving with soundScientific Reports10.1038/s41598-022-06210-x12:1Online publication date: 17-Feb-2022
https://doi.org/10.1038/s41598-022-06210-x

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents