research-article

Shaping Sounds: The Role of Gesture in Collaborative Spatial Music Composition

Authors:

Nick Bryan-Kinns,

Patrick G.T. Healey,

Mathieu BarthetAuthors Info & Claims

C&C '19: Proceedings of the 2019 Conference on Creativity and Cognition

Pages 121 - 132

https://doi.org/10.1145/3325480.3325493

Published: 13 June 2019 Publication History

Abstract

This paper presents an observational study of collaborative spatial music composition. We uncover the practical methods two experienced music producers use to coordinate their understanding of multi-modal and spatial representations of music as part of their workflow. We show embodied spatial referencing as a significant feature of the music producers' interactions. Our analysis suggests that gesture is used to understand, communicate and form action through a process of shaping sounds in space. This metaphor highlights how aesthetic assessments are collaboratively produced and developed through coordinated spatial activity. Our implications establish sensitivity to embodied action in the development of collaborative workspaces for creative, spatial-media production of music.

References

[1]

2014. Dolby Atmos Next Generation Audio for Cinema . Technical Report 3. Dolby Laboratories. http://www.dolby.com/us/en/professional/cinema/products/dolby-atmos-next-generation-audio-for-cinema-white-paper.pdf

[2]

2017. Dolby Atmos production suite guide. Technical Report April. Dolby Laboratories.

[3]

Saul Albert. 2017. Research Methods: Conversation Analysis . In The Routledge Handbook of Discourse Processes. 99.

[4]

Martha W Alibali. 2005. Gesture in Spatial Cognition: Expressing, Communicating, and Thinking About Spatial Information . Spatial Cognition and Computation April 2013 (2005), 37--41.

[5]

Enda Bates and Francis M. Boland. 2016. Spatial Music, Virtual Reality and 360 Media . AES Conference on Audio for Virtual and Augmented Reality (2016), 1--8.

[6]

Joe Bennett. 2012. Constraint, Collaboration and Creativity in Popular Songwriting Teams . In The Act of Musical Composition: Studies in the Creative Process, D. Collins (Ed.). SEMPRE Studies in the Psychology of Music, Farnham, Chapter 6, 139--169.

[7]

Joe Bennett. 2014. Collaborative Songwriting -- The Ontology Of Negotiated Creativity In Popular Music Studio Practice . Journal on the Art of Record Production 1 (2014), 1--8.

[8]

Phillip Brooker and Wes Sharrock. 2013. Remixing Music Together : The Use and Abuse of Virtual Studio Software as a Hobby . In Ethnomethodology at Play. Directions in Ethnomethodology and Conversation Analysis, Peter Tolmie (Ed.). Ashgate, London, 135--155.

[9]

Phillip Brooker and Wes Sharrock. 2016. Collaborative Music-Making with Digital Audio Workstations: The n-th Member as a Heuristic Device for Understanding the Role of Technologies in Audio Composition . Symbolic Interaction 39, 3 (2016), 463--483.

[10]

Nick Bryan-Kinns. 2012. Mutual engagement in social music making . Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering 78 LNICST, 0 (2012), 260--266.

[11]

David Byrne. 2012. How music works. (2012). http://site.ebrary.com/id/10602980

[12]

Andy Clark. 1998. Being There: Putting Brain, Body, and World Together Again . 292 pages.

Digital Library

[13]

Andy Crabtree, Peter Tolmie, and Mark Rouncefield. 2012. Doing Design Ethnography . Springer. 205 pages.

[14]

William J Davies. 2015. Cognition of soundscapes and other complex acoustic scenes. In Inter Noise. San Francisco.

[15]

Jé ré mie Garcia, Thibaut Carpentier, and Jean Bresson. 2017. Interactive-Compositional Authoring of Sound Spatialization . Journal of New Music Research 46, 1 (2017), 20.

[16]

Gabriela Goldschmidt. 1991. The Dialectics of Sketching . Creativity Research Journal 4, 2 (1991), 123--143.

[17]

Charles Goodwin. 2003. Pointing as situated practice . Pointing: Where Language, Culture, and Cognition Meet (2003), 217--241.

[18]

Charles Goodwin and Marjorie Harness Goodwin. 1992. Assessments and Construction of Context . In Rethinking Context, Alessandro Duranti and Charles Goodwin (Eds.). Cambridge University Press, 147--190.

[19]

Maya Gratier. 2008. Grounding in musical interaction: Evidence from jazz performances . Musicae Scientiae (2008), 71--110.

[20]

Patrick G. T. Healey and S.a. Battersby. 2009. The interactional geometry of a three-way conversation . Proceedings of the 31 st Annual Conference of the Cognitive Science Society (2009), 785--790. http://141.14.165.6/CogSci09/papers/138/paper138.pdf

[21]

Patrick G. T. Healey, Joe Leach, and Nick Bryan-Kinns. 2005. Inter-play: Understanding group music improvisation as a form of everyday interaction. In Proceedings of the 1st International Forum on Less is More-Simple Computing in an Age of Complexity.

[22]

Patrick G. T. Healey and Charlotte R. Peters. 2007. The conversational organisation of drawing . 1st International Workshop on Pen-Based Learning Technologies, PLT 2007 (2007).

Digital Library

[23]

Claude Heath and Patrick G. T. Healey. 2011. Making Space for Interaction: Architect's Design Dialogues. In Gesture Workshop. Athens, Greece.

Digital Library

[24]

Christian Heath, Jon Hindmarsh, and Paul Luff. 2010. Video in Qualitative Research . SAGE.

[25]

Christian Heath, Marina Jirotka, Paul Luff, and Jon Hindmarsh. 1995. The Individual and the Collaborative : the Interactional Organisation of Trading in a City Dealing Room . Journal of Computer Supported Cooperative Work 3, 1 (1995), 147--165.

Digital Library

[26]

Christian Heath and Paul Luff. 1991. Collaborative activity and technological design: Task coordination in London Underground control rooms . Proceedings of the Second European Conference on Computer-Supported Cooperative Work (1991), 65--80.

Digital Library

[27]

Christian Heath and Paul Luff. 1992. Collaboration and control: Crisis Management and Multimedia Technology . Computer Supported Cooperative Work: CSCW: An International Journal 1 (1--2), 1990 (1992), 69--94.

[28]

John Heritage and Geoffrey Raymond. 2005. The Terms of Agreement: Indexing Epistemic Authority and Subordination in Talk-in-Interaction . Social Psychology Quarterly 68, 1 (2005), 15--38.

[29]

James Hollan, Edwin Hutchins, and David Kirsh. 2000. Distributed cognition: Toward a new foundation for human-computer interaction research . Transactions on Computer-Human Interaction 7, 2 (jun 2000), 174--196.

Digital Library

[30]

Jay Jantz, Adam Molnar, and Ramses Alcaide. 2017. A brain-computer interface for extended reality interfaces. In SIGGRAPH '17 ACM SIGGRAPH 2017 VR Village, Vol. 13. 73--84.

Digital Library

[31]

Brigitte Jordan and Austin Henderson. 1995. interaction analysis: Foundations and practice . Journal of the Learning Sciences 4, 1 (1995), 39--103.

[32]

Seokmin Kang, Barbara Tversky, and John B. Black. 2015. Coordinating Gesture, Word, and Diagram: Explanations for Experts and Novices . Spatial Cognition and Computation 15, 1 (2015), 1--26.

[33]

Adam Kendon. 2004. Gesture: Visible action as utterance . Cambridge University Press.

[34]

Robin Laney, Chris Dobbyn, Anna Xambó, Mattia Schirosa, Dorothy Miell, Karen Littleton, and Nick Dalton. 2010. Issues and techniques for collaborative music making on multi-touch surfaces . 7th Sound and Music Computing Conference (2010).

[35]

Glenn McGarry, Peter Tolmie, Steve Benford, Chris Greenhalgh, and Alan Chamberlain. 2017. "They're all going out to something weird": Workflow, Legacy and Metadata in the Music Production Process . Proceedings of the 2017 ACM Conference on Computer-Supported Cooperative Work and Social Computing (2017).

Digital Library

[36]

Sean McGrath, Alan Chamberlain, and Steve Benford. 2016. Making music together : an exploration of amateur and pro-am Grime music production. In Audio Mostly, Vol. 2016. 4--6.

Digital Library

[37]

F Melchior, A Churnside, and S Spors. 2012. Emerging Technology Trends in Spatial Audio . SMPTE Motion Imaging Journal 121, 6 (2012), 95--100.

[38]

Keith Murphy. 2003. Building Meaning in Interaction: Rethinking Gesture Classifications . Crossroads of Language Interaction and Culture 2003 UC Regents CA Vol 5. p. 27--47 2005 5, 1972 (2003), 1--10.

[39]

Keith M. Murphy. 2005. Collaborative imagining: The interactive use of gestures, talk, and graphic representation in architectural practice . Semiotica 2005, 156 (2005), 113--145.

[40]

Shahin Nabavian and Nick Bryan-Kinns. 2006. Analysing group creativity: A distributed cognitive study of joint music composition. In Proceedings of cognitive science. 1856--1861. http://www.eecs.qmul.ac.uk/ nickbk/papers/AnalysingGroupCreativityA

[41]

Matt Rahaim. 2008. Gesture and melody in Indian vocal music . Gesture 8, 3 (2008), 325--347.

[42]

Francis Rumsey. 2001. Spatial Audio . 240 pages.

[43]

Francis Rumsey. 2016. Virtual Reality: Mixing Rendering, Believabiity . J. Audio Eng. Soc 64, 12 (2016), 1073--1077. http://www.aes.org/e-lib/browse.cfm?elib=18538

[44]

R. K. Sawyer and S DeZutter. 2009. Distributed Creativity: How Collective Creations Emerge From Collaboration . Psychology of Aesthetics, Creativity, and the Arts 3, 2 (2009), 81--92.

[45]

Mike Scaife and Yvonne Rogers. 1996. External cognition: How do graphical representations work? International Journal of Human Computer Studies 45, 2 (1996), 185--213.

Digital Library

[46]

Jean-Baptiste Thiebaut and Patrick G. T. Healey. 2007. Sketching Musical Compositions . Cognitive Science Society Journal (2007), 1079--1084.

[47]

Jean-baptiste Thiebaut, Patrick G. T. Healey, and Nick Bryan-Kinns. 2008. Drawing Electroacoustic Music. In ICMC.

[48]

Barbara Tversky. 2014. The Cognitive Design of Tools of Thought . Review of Philosophy and Psychology 6, 1 (2014), 99--116.

[49]

Barbara Tversky. 2017. Gestures can create diagrams (that are neither imagistic nor analog) . Behavioral and Brain Sciences 40 (2017), e73.

[50]

Barbara Tversky, Azadeh Jamalian, Valeria Giardino, Seokmin Kang, and Angela Kessell. 2013. Comparing Gestures And Diagrams. In Proceedings of the 10th International Gesture Workshop.

[51]

Robin Wolff, Dave J. Roberts, Anthony Steed, and Oliver Otto. 2007. A review of telecollaboration technologies with respect to closely coupled collaboration . International Journal of Computer Applications in Technology 29, 1 (2007), 11--26.

Digital Library

[52]

Anna Xambó, Robin Laney, Chris Dobbyn, and Sergi Puig Jordà. 2012. Towards a taxonomy for video analysis on collaborative musical tabletops. In Proceedings of BCS HCI 2012 Workshops: Video Analysis Techniques for Human-Computer Interaction. 1--4.

Cited By

Saitis CDel Sette BShier JTian HZheng SSkach SReed CFord C(2024)Timbre Tools: Ethnographic Perspectives on Timbre and Sonic Cultures in Hackathon DesignsProceedings of the 19th International Audio Mostly Conference: Explorations in Sonic Cultures10.1145/3678299.3678322(229-244)Online publication date: 18-Sep-2024
https://dl.acm.org/doi/10.1145/3678299.3678322
Ong Y(2023)Quest Driven Spatial Songs: Exploring how narrative structures of Quests in Games can be mapped onto songs.SIGGRAPH Asia 2023 Doctoral Consortium10.1145/3623053.3623367(1-3)Online publication date: 28-Nov-2023
https://dl.acm.org/doi/10.1145/3623053.3623367
Deacon TBarthet M(2023)Invoke: A Collaborative Virtual Reality Tool for Spatial Audio Production Using Voice-Based Trajectory SketchingProceedings of the 18th International Audio Mostly Conference10.1145/3616195.3616211(161-168)Online publication date: 30-Aug-2023
https://dl.acm.org/doi/10.1145/3616195.3616211
Show More Cited By

Index Terms

Shaping Sounds: The Role of Gesture in Collaborative Spatial Music Composition
1. Human-centered computing
  1. Collaborative and social computing
    1. Collaborative and social computing theory, concepts and paradigms
      1. Collaborative content creation

Recommendations

‘shaping electronic sounds like clay’: The historical situation and aesthetic position of electroacoustic music at the zkm | institute for music and acoustics

Twenty years after the founding of the Karlsuhe ZKM | Center for Art and Media and its Institute for Music and Acoustics, we reexamine the institute’s position with regard to both aesthetic approaches and the 20 years of a reunified Germany. When the ...
The choreographer's notebook: a video annotation system for dancers and choreographers
C&C '11: Proceedings of the 8th ACM conference on Creativity and cognition

We present a collaborative tool for choreographers and dancers, the Choreographer's Notebook, which allows multimodal annotation of rehearsal videos during the dance production process. The dance production process is a creative process in which ...
Collaborative Artificial Intelligence in Music Production
AM '18: Proceedings of the Audio Mostly 2018 on Sound in Immersion and Emotion

The use of technology has revolutionized the process of music composition, recording, and production in the last 30 years. One fusion of technology and music that has been longstanding is the use of artificial intelligence in the process of music ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

C&C '19: Proceedings of the 2019 Conference on Creativity and Cognition

June 2019

745 pages

ISBN:9781450359177

DOI:10.1145/3325480

General Chairs:
Steven Dow
UC San Diego, USA
,
Mary Lou Maher
UNC Charlotte, USA
,
Program Chairs:
Andruid Kerne
Texas A&M University
,
Celine Latulipe
UNC Charlotte, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 June 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

C&C '19

Sponsor:

SIGCHI

C&C '19: Creativity and Cognition

June 23 - 26, 2019

CA, San Diego, USA

Acceptance Rates

C&C '19 Paper Acceptance Rate 30 of 101 submissions, 30%;

Overall Acceptance Rate 108 of 371 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
322
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Saitis CDel Sette BShier JTian HZheng SSkach SReed CFord C(2024)Timbre Tools: Ethnographic Perspectives on Timbre and Sonic Cultures in Hackathon DesignsProceedings of the 19th International Audio Mostly Conference: Explorations in Sonic Cultures10.1145/3678299.3678322(229-244)Online publication date: 18-Sep-2024
https://dl.acm.org/doi/10.1145/3678299.3678322
Ong Y(2023)Quest Driven Spatial Songs: Exploring how narrative structures of Quests in Games can be mapped onto songs.SIGGRAPH Asia 2023 Doctoral Consortium10.1145/3623053.3623367(1-3)Online publication date: 28-Nov-2023
https://dl.acm.org/doi/10.1145/3623053.3623367
Deacon TBarthet M(2023)Invoke: A Collaborative Virtual Reality Tool for Spatial Audio Production Using Voice-Based Trajectory SketchingProceedings of the 18th International Audio Mostly Conference10.1145/3616195.3616211(161-168)Online publication date: 30-Aug-2023
https://dl.acm.org/doi/10.1145/3616195.3616211
Montague LBarthet M(2023)Collaboration on the Tracks: Ethnographically-Informed Design for Computer-Assisted Music Collaboration between Producers and PerformersProceedings of the 15th Conference on Creativity and Cognition10.1145/3591196.3596619(391-392)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3591196.3596619
Deacon THealey PBarthet M(2022)“It’s cleaner, definitely”: Collaborative Process in Audio ProductionComputer Supported Cooperative Work10.1007/s10606-022-09448-132:3(475-505)Online publication date: 14-Nov-2022
https://dl.acm.org/doi/10.1007/s10606-022-09448-1
Deacon TBarthet M(2022)Spatial Design Considerations for Interactive Audio in Virtual RealitySonic Interactions in Virtual Environments10.1007/978-3-031-04021-4_6(181-217)Online publication date: 14-Oct-2022
https://doi.org/10.1007/978-3-031-04021-4_6

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents