TOUCHtr4ck

TOUCHtr4ck: Democratic Collaborative Music Anna Xambó Music Computing Lab The Open University Milton Keynes, UK a.xambo@open.ac.uk Robin Laney Music Computing Lab The Open University Milton Keynes, UK r.c.laney@open.ac.uk Chris Dobbyn Music Computing Lab The Open University Milton Keynes, UK c.h.dobbyn@open.ac.uk ABSTRACT When electronic musicians compose collaboratively, they typically use their own single-user musical controllers. It may, therefore, be useful to develop novel controllers that support collaborative workflows and democratic principles. After describing the design principles for developing such controllers, we present TOUCHtr4ck, a prototype multitouch system designed to facilitate such democratic relationships. Informal testing has revealed that this approach does facilitate democratic and collaborative music making, and can produce creative musical results. Author Keywords Collaboration, HCI, musical interface, tabletop groupware ACM Classification Keywords H.5.5 Information Interfaces and Presentation: Sound and Music Computing—Systems; H.5.3 Information Interfaces and Presentation: Group and Organization Interfaces—Collaborative computing General Terms Design, Experimentation INTRODUCTION The practice of composing electronic music tends to be an individual activity, even though musical composition and music making may also have a social dimension. Curiously, among the vast market of musical controllers available for musicians, few are designed for facilitating collaboration in the creative process. A study of the compositional processes of electronic musicians [5] found that this community would prefer a free and exploratory approach during the initial stages—when an idea can still change substantially—, which is not addressed by the traditional digital audio workstation (DAW) software they use. Within a spectrum from exploration to linearity in the creative process, interactive composition systems [2] become a trade-off between the two approaches, hence they offer an explorative approach under Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. TEI’11, January 22–26, 2011, Funchal, Portugal. Copyright 2011 ACM 978-1-4503-0478-8/11/01...$10.00. Figure 1. The TOUCHtr4ck prototype. certain constraints. Using these real-time computer music systems, the process of composing and performing music happens simultaneously. With Ableton Live, for example, both processes can become one. But, even though collaborations can emerge from laptop ensembles using this software [7], musical dialogues in real time are more fully exploited by tangible user interfaces (TUI) such as the reacTable [11] or the Squeezables [20]. In our opinion, approaching the aspects that facilitate a musical controller to be more democratic is a key element for successful collaboration1 . An early exploration of democratic collaborative electronic music making is Mikrophonie I (1963) by Stockhausen (reported by Blaine and Fels [18]). This piece is meant to be played by six musicians in three pairs: one pair of percussionists plays a tam-tam instrument; another pair records the resulting sounds with microphones; and the third pair applies filters to the output. Although the musicians follow an instructional score, and thus the performance requires musical expertise, it can be seen as a seminal example of a democratic piece because each performer’s musical influence depends on the rest of the team. We think there is a need to address more thoroughly the design of interfaces that enhance these democratic, collabora1 The term democratic collaborative music is used to reference the process of creating music in collaboration, where all kinds of participants (experts and novices) can shape and dialogue their musical ideas under equal means. tive musical creative processes. According to [13], interface design can support democratized performances by suggesting processes, environments and relationships. Furthermore, among the different genres, electronic music can be one of the most democratic in terms of the musical knowledge required: the extensive use of digital musical tools has contributed to a new aesthetic that also allows musicians with little technical knowledge to produce music easily, and to create music from novel and experimental processes [4, 1]. In consequence, we have developed TOUCHtr4ck—a multitouch tabletop prototype intended for democratic collaborative music making—with the aim of allowing groups to perform music together in real time using both exploration and discovery (see Figure 1). In the following section, some background on both computer-supported collaborative work and collaborative musical experiences is given. After that, an overview of some design principles for democratic collaborative music making are presented. Next, the design process for the prototype is introduced: concept, interface design, implementation and interaction mappings. Finally, an informal test of the approach is discussed and future work highlighted. proving the experience of playing interactive composition systems [2], either individually or in collaboration. Jordà [9, 10] points out the relationship between new musical controllers and new music-making paradigms. According to Winkler [22], interactive music interfaces must provide feedback and interaction support. Cook [3] suggests some artistic principles such as “instant music, subtlety later” or “make a piece, not an instrument or a controller”. Some of these are particularly relevant in collaborative interaction, e.g., the use of certain metaphors such as “catch and throw” facilitates the idea of a dialogue between several musicians, in which the musical material is received, modified and sent in real time [21]. In the context of democratic collaborative network music, new design principles related to engagement have also been recommended, such as facilitating the awareness of contributions, relationships between performers or dialogues mediated by technology [13]. Musical tabletops present an ideal setting for collaborative engagement because they make possible visual feedback, individual vs. shared spaces, and real-time multi-dimensional interaction [10, 14]. DESIGN PROCESS RELATED WORK Computer-supported collaborative work Interactive tabletops have been studied within the field of computer-supported collaborative work (CSCW). In the CSCW literature, facilitating the awareness of others [8] and enriching of existing work processes [6] are factors that have been considered when approaching co-located collaboration. Thus, our approach focuses on enhancing the workflow of collaborative music creation with special attention to facilitating communication among musicians by supporting the awareness of others. Collaborative musical experiences Collaborative musical experiences do not necessarily imply a democratic setting. In the iltur system, for example, expert musicians keep playing their own traditional instruments whereas novice musicians can record and manipulate these recordings in real time; this tends to mean that the collaboration is hierarchical [19]. Another example are Sound Toys, which are playful, explorative and collaborative musical objects designed specifically for novices [17]. The reacTable [11] or the Squeezables [20] propose to attract both novices and experts alike, although either their interface complexity or simplicity can bias. According to [14], attracting both experts and novices alike implies keeping a balance between ease of use and constraints more suitable to novices, with the presence of sufficient musical features to allow personal musical expressivity more suitable to experts. TOUCHtr4ck is intended to be a playful and exploratory musical controller that facilitates democratic relationships between performers. Thus, we are interested in providing both a highly engaging social experience, but also a satisfactory musical one. DESIGN PRINCIPLES In the literature on musical interfaces, design principles for musical controllers have been outlined with the aim of im- Concept The design of TOUCHtr4ck has been informed by our previous experience of designing and evaluating similar prototypes [14, 23]. The tr4ck prototype was designed for a PDA as an exploratory musical controller for recording and manipulating up to four sounds in sync. It provided positive results in terms of ease of use, productivity and satisfactory musical output, although it was limited to a single player [23]. In [14] we developed and evaluated a minimal tabletop interface for collaborative music making using multi-touch interaction. The prototype evaluated in this study proved to be useful for collaborative engagement, although more musical features, awareness and control features (e.g. individual vs. shared controls) were requested by users. Thus the question arose of whether a more democratic collaborative musical controller, combining the functionality of the former with the collaborative music making aspects of the latter, was possible. RECORDER/PLAYER tr4ck TRANSFORMER/MIXER RECORDER/PLAYER TOUCHtr4ck Figure 2. Design process for TOUCHtr4ck. According to the previous experience reported above, and considering the design principles cited in the previous section, we established three main objectives for the design of a democratic collaborative musical controller, the TOUCHtr4ck prototype. Firstly, it should be exploratory, similar to the single-user tr4ck, by allowing instant and realtime shaping of unexpected musical ideas (in contrast to a more linear and sequential approach). Secondly, the interface design should invite democratic and interdependent collaboration, similar to Mikrophonie I, where there is a realtime division of labour, between producing, recording and transforming sounds (see Figure 2). Thirdly, the interface design should provide collaborative engagement, such as in our previous minimal tabletop interface. For that purpose, a similar setting of tabletop multi-touch interaction should be maintained. Accordingly, the interface design should be easy to use, but would incorporate both discrete and continuous actions in order to engage either experts or novices. Consistent with these three objectives, the TOUCHtr4ck prototype allows musicians to record up to four samples and mix these collaboratively on a multi-touch tabletop surface. The prototype offers a plug and play approach, where sounds can be recorded and modified, and disruption of the workflow between editing and mixing is avoided by looping all tracks and showing changes in real time, which seems ideal for exploration. Democratic collaboration is supported by dividing these two main tasks into two modules which can be executed in parallel, and the possibility of modifying others’ contributions. Additionally, the awareness of others is provided by real-time visual feedback. Finally, collaborative engagement seems to be associated to the personal motivation and the control level of the interface, hence using basic UI controllers (both discrete and continuous such as knobs, sliders or buttons) is intended to afford ease of learning and use at a general level. Interface design The interface design displays a number of circles with different UI controllers inside each (see Figure 3). Each circle represents a task such as recording/playing or transforming/mixing. The large circle on the right of Figure 3 shows four tracks. For each track it is possible to play, record or stop a sample, as well as to modify its volume. The large circle on the left allows participants to modify global con- trols such as the global volume and to manipulate a set of filters such as band-pass filter or reverb, among others. Filters affect only the tracks that are in play mode, and they are applied in sequence to the global output. Given that all the tracks are looping, and start at the same time, the circle in the middle indicates the start of the global loop with a bright yellow light pulse. This circle also permits changes in the global pitch shift. Implementation The TOUCHtr4ck prototype was developed using open source tools. The computer vision framework reacTIVision [12] was used for the multi-touch finger tracking, and the table hardware (e.g. infrared illumination, camera and projector) was built according to the requirements of this framework. The audio software was built using the programming language for real-time audio synthesis SuperCollider 3 [15], and the graphics and control management with the programming language Processing [16]. Thus, the prototype is divided into model, view and control modules: the sound synthesis engine (e.g. playing, recording and transforming sound) is defined and managed in SuperCollider (the model), whilst the graphic interface and the interaction control of the TUIO messages sent by reacTIVision are managed in Processing (the view and the controller). Interaction mappings Three main interaction factors are identified for a democratic collaborative music making on multi-touch surfaces: awareness of others’ actions; modifiability of others’ actions; and the distinction of users’ musical expertise. Awareness The awareness of others’ actions is supported by the division of labour incorporated in the interface design. Each of the large circles is devoted to either editing or mixing, so its proximity to the subject indicates who is in charge of each task. Moreover, audiovisual feedback indicates the state of the tracks or filters, that is, whether they are mute, active or selected. Modifiability Modiability of others’ actions is allowed through the use of the shared controls such as the global pitch shift. In addition, the division between the tasks implies that users can only have partial control of the musical result. Expertise VOLUME TRACK B RV HP F BPF DELAY PLAY BUTTON STOP BUTTON RECORD BUTTON FILTER FILTER FILTER FILTER GLOBAL PITCH SHIFT GLOBAL VOLUME ACTIVE TRACK RECORDING TRACK Figure 3. Interface design. MUTE TRACK AWARENESS Each user is able to add or reduce the number of tracks or filters in order to adapt to his or her expertise, within the limits of the interface design. This flexibility permits novices and experts alike to use the system, whilst maintaining, as much as possible, a democratic setting. DISCUSSION AND FUTURE WORK Informal testing was done with two expert musicians using the prototype as a proof-of-concept. The two participants formerly played in a band together, and currently they make electronic music using mainly DAW workstations, along with individual musical controllers. Both interacted with the prototype for an interval of ten minutes. A playful attitude was observed during the whole session. Moreover, the musicians contributed similarly, using both shared and individual controls. After the session, an informal discussion was carried out, in which both described the prototype as an experimental tool not fully controllable, and which provided unexpected results that can be useful when composing music. They both agreed on the ease of use of the prototype, although commented about the need of more accuracy when recording. In summary, we have provided a set of design principles for democratic collaborative music on interactive systems, and we have built a prototype upon these. After an informal evaluation, promising results have been obtained: Firstly, collaborative experience has been facilitated by awareness and modifiability of others’ actions using a shareable interface; and secondly, experts found the musical controller playful and experimental, an approach which tends to be offered to novices only. Future work will involve, on the one hand, carrying out a formal testing with more users, in order to strengthen the design concept; and, on the other hand, to improve the prototype by providing more accuracy of control, more support for relationships between performers and more awareness of contributions. Furthermore, the benefits of a flexible design should be examined precisely. As a final remark, this approach specifically enhances the relationships between performers because collaborative work processes are facilitated which, in turn, affect the musical output. ACKNOWLEDGEMENTS We thank Gerard Roma, Josep Guasch, Andrew Milne and Adam Linson for their expert contributions; and the Open University for the opportunity to carry out this study. REFERENCES 1. K. Cascone. The aesthetics of failure: “Post-digital” tendencies in contemporary computer music. Computer Music Journal, 24(4):12–18, 2000. 2. J. Chadabe. Interactive composing: An overview. Computer Music Journal, 8(1):22–28, 1984. 3. P. Cook. Principles for designing computer music controllers. In Proc. NIME ’01, pages 1–4, 2001. 4. C. Cox and D. Warner. Audio Culture: Readings in Modern Music. Continuum, New York, 2004. 5. S. Gelineck and S. Serafin. From idea to realization understanding the compositional processes of electronic musicians. In Proc. Audio Mostly 2009, 2009. 6. J. Halloran, Y. Rogers, T. Rodden, and I. Taylor. Creating new user experiences to enhance collaboration. In Proc. INTERACT’03, pages 479 – 486, 2003. 7. S. Heinz and S. O’Modhrain. Designing a shareable musical TUI. In Proc. NIME ’10, pages 339–342, 2010. View publication stats 8. E. Hornecker, P. Marshall, N. S. Dalton, and Y. Rogers. Collaboration and interference: Awareness with mice or touch input. In Proc. CSCW ’08, pages 167–176. ACM, 2008. 9. S. Jordà. Fmol: Toward user-friendly, sophisticated new musical instruments. Computer Music Journal, 26(3):23–39, 2002. 10. S. Jordà. On stage: the reactable and other musical tangibles go real. International Journal of Arts and Technology, 1(3–4):268–287, 2008. 11. S. Jordà, G. Geiger, M. Alonso, and M. Kaltenbrunner. The reacTable: Exploring the synergy between live music performance and tabletop tangible interfaces. In Proc. TEI’07, 2007. 12. M. Kaltenbrunner and R. Bencina. reacTIVision: A computer-vision framework for table-based tangible interaction. In Proc. TEI ’07, pages 69–74. ACM, 2007. 13. D. Kim-Boyle. Network musics - Play, engagement and the democratization of performance. In Proc. NIME ’08, pages 3–8, 2008. 14. R. Laney, C. Dobbyn, A. Xambó, M. Schirosa, D. Miell, K. Littleton, and S. Dalton. Issues and techniques for collaborative music making on multi-touch surfaces. In Proc. SMC’10, pages 146–153, 2010. 15. J. McCartney. Rethinking the computer music language: SuperCollider. Computer Music Journal, 26(4):61–68, 2002. 16. C. Reas and B. Fry. Processing: A Programming Handbook for Visual Designers and Artists. MIT Press, 2007. 17. D. Robson. PLAY!: Sound toys for non-musicians. Computer Music Journal., 26(3):50–61, 2002. 18. B. Tina and F. Sidney. Collaborative musical experiences for novices. Journal of New Music Research, 32(4):411 – 428(18), December 2003. 19. G. Weinberg and S. Driscoll. iltur: Connecting novices and experts through collaborative improvisation. In Proc. NIME ’05, pages 17–22, 2005. 20. G. Weinberg and S.-L. Gan. The Squeezables: Toward an expressive and interdependent multi-player musical instrument. Computer Music Journal, 25(2):37–45, 2001. 21. D. Wessel and M. Wright. Problems and prospects for intimate musical control of computers. Computer Music Journal, 26(3):11–22, 2002. 22. T. Winkler. Composing Interactive Music. MIT Press, Cambridge, MA, USA, 1998. 23. A. Xambó. Interfaces for sketching musical compositions. Master’s thesis, Universitat Pompeu Fabra, 2008.