research-article

Form Follows Sound: Designing Interactions from Sonic Memories

Authors: Baptiste Caramiaux, Alessandro Altavilla, Scott G. Pobiner, Atau TanakaAuthors Info & Claims

CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Pages 3943 - 3952

https://doi.org/10.1145/2702123.2702515

Published: 18 April 2015 Publication History

Abstract

Sonic interaction is the continuous relationship between user actions and sound, mediated by some technology. Because interaction with sound may be task oriented or experience-based it is important to understand the nature of action-sound relationships in order to design rich sonic interactions. We propose a participatory approach to sonic interaction design that first considers the affordances of sounds in order to imagine embodied interaction, and based on this, generates interaction models for interaction designers wishing to work with sound. We describe a series of workshops, called Form Follows Sound, where participants ideate imagined sonic interactions, and then realize working interactive sound prototypes. We introduce the Sonic Incident technique, as a way to recall memorable sound experiences. We identified three interaction models for sonic interaction design: conducting; manipulating; substituting. These three interaction models offer interaction designers and developers a framework on which they can build richer sonic interactions.

Supplementary Material

MP4 File (p3943-caramiaux.mp4)

Download
136.89 MB

References

[1]

Altavilla, A., Caramiaux, B., and Tanaka, A. Towards Gestural Sonic Affordances. Proc. NIME'13, (2013), 61--64.

[2]

Augoyard, J.-F. and Torgue, H. Sonic Experience: A Guide To Everyday Sounds. McGill-Queen's Press - MQUP, 2006.

[3]

Avanzini, F., De Götzen, A., Spagnol, S., and Rodá, A. Integrating auditory feedback in motor rehabilitation systems. Proc. International Conference on Multimodal Interfaces for Skills Transfer (SKILLS), (2009), 53--58.

[4]

Beaudouin-Lafon, M. Instrumental interaction: an interaction model for designing post-WIMP user interfaces. Proc. CHI, (2000), 446--453.

Digital Library

[5]

Beaudouin-Lafon, M. Designing interaction, not interfaces. Proc. Working Conference on Advanced Visual Interfaces AVI, (2004), 15--22.

Digital Library

[6]

Burns, C., Dishman, E., Johnson, B., and Verplank, B. "Informance": Min(d)ing future contexts for scenario-based interaction design. Presented at BayCHI. 1995.

[7]

Caramiaux, B., Bevilacqua, F., Bianco, T., Schnell, N., Houix, O., and Susini, P. The Role of Sound Source Perception in Gestural Sound Description. ACM Transactions on Applied Perception 11, 1 (2014), 1--19.

Digital Library

[8]

Caramiaux, B., Françoise, J., Bevilacqua, F., and Schnell, N. Mapping Through Listening. Computer Music Journal 38, 3 (2014), 34--48.

Digital Library

[9]

Caramiaux, B. and Tanaka, A. Machine Learning of Musical Gestures. Proc. NIME, (2013), 513--518.

[10]

Dourish, P. Where the action is: the foundations of embodied interaction. The MIT Press, 2001.

[11]

Ekman, I., Rinott, M., and Box, P.O. Using Vocal Sketching for Designing Sonic Interactions. Proc. DIS, (2010), 123--131.

Digital Library

[12]

Flanagan, J.C. The critical incident technique. Psychological bulletin 51, 4 (1954), 327--58.

[13]

Franinović, K., Gaye, L., and Behrendt, F. Exploring Sonic Interaction with Artifacts in Everyday Contexts. Proc. ICAD, (2008), 1--4.

[14]

Franinović, K., Hug, D., and Visell, Y. Sound Embodied: Explorations of sonic interaction design for everyday objects in a workshop setting. Proc. ICAD, (2007), 334--341.

[15]

Franinović, K. and Salter, C. The Experience of Sonic Interaction. In S. Franinović, Karmen, Serafin, ed., Sonic Interaction Design. MIT Press, 2013, 39--75.

[16]

Franinović, K. and Serafin, S. Sonic Interaction Design. Mit Press, 2013.

[17]

Gaver, W. The SonicFinder: An Interface That Uses Auditory Icons. Human-Computer Interaction 4, 1 (1989), 67--94.

Digital Library

[18]

Gaver, W.W. What in the World Do We Hear?: An Ecological Approach to Auditory Event Perception. Ecological Psychology 5, 1 (1993), 1--29.

[19]

Goodman, S. Sonic Warfare: Sound, Affect, and the Ecology of Fear. MIT Press, 2009.

Digital Library

[20]

Hartson, H.R. and Castillo, J.C. Remote evaluation for post-deployment usability improvement. Proc. Working Conference on Advanced Visual Interfaces AVI, (1998), 22--29.

Digital Library

[21]

Henriques, J.F. The Vibrations of Affect and their Propagation on Night Out on Kingston's Dancehall scene. Body Society 16, (2010), 57--89.

[22]

Hermann, T. and Hunt, A. An introduction to interactive sonification. IEEE Multimedia 12, 2005, 20--24.

Digital Library

[23]

Hermann, T. Sonification for Exploratory Data Analysis. University of Bielefeld, 2002.

[24]

Houix, O., Gutierrez, F., Susini, P., and Misdariis, N. Participatory Workshops: Everyday Objects and Sound Metaphors. Proc. CMMR, (2013), 41--53.

[25]

Hug, D. Using a systematic design process to investigate narrative sound design strategies for interactive commodities. Proc. ICAD, (2009), 1--8.

[26]

Kramer, G., Walker, B., and Bargar, R. Sonification report: Status of the field and research agenda. International Community for Auditory Display, (1999).

[27]

Leman, M., Desmet, F., Styns, F., Van Noorden, L., and Moelants, D. Sharing musical expression through embodied listening: A case study based on Chinese Guqin music. Music Perception 26, 3 (2009), 263--278.

[28]

MacKay, W. Using Video to Support Interaction Design. Tutorial CHI, 2002.

[29]

Maes, P.-J., Leman, M., Palmer, C., and Wanderley, M.M. Action-based effects on music perception. Frontiers in psychology 4, 1 (2014).

[30]

Norman, D.A. and Verganti, R. Incremental and Radical Innovation: Design Research versus Technology and Meaning Change. Design Issues 30, 1 (2014), 78--96.

[31]

Pauletto, S., Hug, D., Barrass, S., and Luckhurst, M. Integrating Theatrical Strategies into Sonic Interaction Design. Proc. Audio Mostly, (2009), 1--6.

[32]

Rasamimanana, N., Bevilacqua, F., Bloit, J., et al. The Urban Musical Game: Using Sport Balls As Musical Interfaces. Proc. CHI EA'12, (2012), 1027--1030.

Digital Library

[33]

Roby-Brami, A., Van Zandt-Escobar, A., Jarrassé, N., et al. Toward the use of augmented auditory feedback for the rehabilitation of arm movements in stroke patients. Annals of Physical and Rehabilitation Medicine 57, (2014), e4--e5.

[34]

Rocchesso, D., Polotti, P., and Monache, S.D. Designing continuous sonic interaction. International Journal of Design 3, 3 (2009), 13--25.

[35]

Shneiderman, B. Designing the User Interface: Strategies for Effective Human-Computer Interaction. Reading, MA: Addison-Wesley, 1998.

Digital Library

[36]

Svanaes, D. and Seland, G. Putting the users center stage: Role playing and low-fi prototyping enable end users to design mobile systems. Proc. CHI, (2004), 479--486.

Digital Library

[37]

Tanaka, A., Altavilla, A., and Spowage, N. Gestural Musical Affordances. Proc. SMC, (2012), 318--325.

[38]

Varela, F.J., Thompson, E.T., and Rosch, E. The Embodied Mind: Cognitive Science and Human Experience. MIT press, 1991.

[39]

Zatorre, R.J., Chen, J.L., and Penhune, V.B. When the brain plays music: auditory--motor interactions in music perception and production. Nature Reviews Neuroscience 8, 7 (2007), 547--558.

Cited By

Zorea MKushi K(2024)Empathic Directions for The Sounds of Parenting Technology: The Case of Baby MonitorsProceedings of the 3rd Empathy-Centric Design Workshop: Scrutinizing Empathy Beyond the Individual10.1145/3661790.3661792(8-13)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3661790.3661792
Zorea MKushi K(2024)All Is Heard: Reimagining The Sounds of Home With CareProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660686(2708-2718)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660686
Kim YLee MPeljhan MKuchera-Morin JHöllerer T(2024)Spatial Orchestra: Locomotion Music Instruments through Spatial ExplorationExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3648659(1-5)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3648659
Show More Cited By

Index Terms

Form Follows Sound: Designing Interactions from Sonic Memories

Recommendations

Sound to your objects: a novel design approach to evaluate orangutans' interest in sound-based stimuli
ACI '16: Proceedings of the Third International Conference on Animal-Computer Interaction

Orangutans show interest in sound-based stimuli, but the auditory enrichment they are usually provided with is either based on human-music, or does not allow for control and choice. In this work-in-progress paper, we describe the design of sound-based ...
Toward the design and evaluation of continuous sound in tangible interfaces: The Spinotron

This paper reports on an approach to the design of continuous sonic feedback in tangible interfaces, and on quantitative evaluation methods intended to guide such design tasks. The issues it addresses may be of central relevance to areas of the emerging ...
Basic exploration of narration and performativity for sounding interactive commodities
HAID'10: Proceedings of the 5th international conference on Haptic and audio interaction design

We present an exploration in sonic interaction design, aimed at integrating the power of narrative sound design with the sonic aesthetics of a physics-based sound synthesis. The emerging process is based on interpretation, and can represent a novel tool ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

April 2015

4290 pages

ISBN:9781450331456

DOI:10.1145/2702123

General Chairs:
Bo Begole
Huawei, USA
,
Jinwoo Kim
Yonsei University, Korea
,
Program Chairs:
Kori Inkpen
Microsoft Research, USA
,
Woontack Woo
KAIST, Korea

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 April 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Provost's Office Parsons School of Design Strategies
European Union's Seventh Framework Programme

Conference

CHI '15

Sponsor:

SIGCHI

CHI '15: CHI Conference on Human Factors in Computing Systems

April 18 - 23, 2015

Seoul, Republic of Korea

Acceptance Rates

CHI '15 Paper Acceptance Rate 486 of 2,120 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
760
Total Downloads

Downloads (Last 12 months)76
Downloads (Last 6 weeks)9

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zorea MKushi K(2024)Empathic Directions for The Sounds of Parenting Technology: The Case of Baby MonitorsProceedings of the 3rd Empathy-Centric Design Workshop: Scrutinizing Empathy Beyond the Individual10.1145/3661790.3661792(8-13)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3661790.3661792
Zorea MKushi K(2024)All Is Heard: Reimagining The Sounds of Home With CareProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660686(2708-2718)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660686
Kim YLee MPeljhan MKuchera-Morin JHöllerer T(2024)Spatial Orchestra: Locomotion Music Instruments through Spatial ExplorationExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3648659(1-5)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3648659
Zorea MKushi K(2023)Audible Imagery: Creative Contemplations on the Sounds of HomeProceedings of the 15th Conference on Creativity and Cognition10.1145/3591196.3593366(150-161)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3591196.3593366
Bang TFdili Alaoui S(2023)Suspended Circles: Soma Designing a Musical InstrumentProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581488(1-15)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581488
Escamilla AMelenchón JMonzo CMoran J(2023)User Clustering Visualization and Its Impact on Motion-Based Interaction DesignHuman-Computer Interaction10.1007/978-3-031-35596-7_4(47-63)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-35596-7_4
Johansen Svan Berkel NFritsch J(2022)Characterising Soundscape Research in Human-Computer InteractionProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533458(1394-1417)Online publication date: 13-Jun-2022
https://dl.acm.org/doi/10.1145/3532106.3533458
Liu WMagalhaes MMackay WBeaudouin-Lafon MBevilacqua F(2022)Motor Variability in Complex Gesture Learning: Effects of Movement Sonification and Musical BackgroundACM Transactions on Applied Perception10.1145/348296719:1(1-21)Online publication date: 6-Jan-2022
https://dl.acm.org/doi/10.1145/3482967
Willett WAseniero BCarpendale SDragicevic PJansen YOehlberg LIsenberg P(2022)Perception! Immersion! Empowerment! Superpowers as Inspiration for VisualizationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.311484428:1(22-32)Online publication date: Jan-2022
https://doi.org/10.1109/TVCG.2021.3114844
Hilton CPlant NGonzález Díaz CPerry PGibson RMartelli BZbyszynski MFiebrink RGillies M(2021)InteractML: Making machine learning accessible for creative practitioners working with movement interaction in immersive mediaProceedings of the 27th ACM Symposium on Virtual Reality Software and Technology10.1145/3489849.3489879(1-10)Online publication date: 8-Dec-2021
https://dl.acm.org/doi/10.1145/3489849.3489879
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents