Sonic Interaction Design

Embodied sound design is a process of sound creation that involves the designer’s vocal apparatus and gestures. The possibilities of vocal sketching were investigated by means of an art installation. An artist–designer interpreted several vocal self-portraits and rendered the corresponding synthetic sketches by using physics-based and concatenative sound synthesis. Both synthesis techniques afforded a broad range of artificial sound objects, from concrete to abstract, all derived from natural vocalisations. The vocal-to-synthetic transformation process was then automated in SEeD, a tool allowing to set and play interactively with physics- or corpus-based sound models. The voice-driven process and tool, developed and evaluated through design exercises, show how an embodied sound sketching system can work in supporting the externalisation of sonic concepts.

Advancements in embedded computer platforms have allowed data to be collected and shared between objects—or smart devices—in a network. While this has resulted in highly functional outcomes in fields such as automation and monitoring, there are also implications for artistic and expressive systems. In this paper we present a pluralistic approach to incorporating environmental factors within the field of site-responsive sonic art using embedded audio and data processing techniques. In particular, we focus on the role of such systems within an ecosystemic framework, both in terms of incorporating systems of living organisms, as well as sonic interaction design. We describe the implementation of such a system within a large-scale site-responsive sonic art installation that took place in the subtropical desert climate of Arizona in 2017.

https://vimeo.com/10722818

Architecture is not only that which is built. Architecture is made up of different aspects, both material and immaterial. The atmosphere, the sounds, the smells, the possibility of interaction between human bodies: these all constitute characteristics of space, characteristics that are assuming an increasing importance within architectural research worldwide. Within this field of thought about “Architecture Beyond Building”, we focus on one particular non-material spatial phenomenon that lies ‘beyond the built’: sound.

We have created, thus, an interactive sonic map of Athens, which presents, in an unexpected way, fragments of the atmosphere of the city. What would a non-visual map look like? What would it feel like if you wandered within a forest of headphones, playing sounds from different places in Athens? How would it feel if you found yourself in a ‘map’ that only appeared when you walked in it? What would it be like if
the map only appeared when you invited one more person to be with you?

The Greek pavilion addresses these questions through an atmospheric interactive ‘game’, presenting fragments of sounds and visual sequences of Athens. The visitor recreates the space around him through his own presence and movement. The map appears only where he walks, and/or when he invites one more person to sit next to him. The bodies of the visitors react with one another and with the space itself, creating a dynamic, changing field.

This walk in the pavilion takes you “out there”, through invisible Athens. The pavilion brings forth the aspects of architecture that are ‘beyond the material’: the ‘beyond the built’, the almost unreachable, elusive aspects of space, such as sound, non-visual senses, atmosphere. It challenges, thus, the limits of architecture, the limits of what can be mapped and re-located and what cannot…

Contents
Michalis Liapis: Foreword
A. Karandinou, C. Achtypi, S. Giamarelos: Introduction

Athens by Sound project-relevant
Ιntothepill: Notes from the discussions related to the Athens by Sound project
Katie Lloyd Thomas: Sounding the grid. Immateriality and its hidden supports
Martin Parker: Water does not run uphill unless it is pushed. Maxims for thinking about sound in the context of architecture, installation art and technology. A response to Athens by Sound
Panayiotis Tournikiotis: Interview

Τheoretical framework - broader discussions
Material - immaterial in architecture
Mark Wigley: The Architecture of Atmosphere
Dorian Wiszniewski: Beyond Architecture. Architecture and Immateriality
Leslie Kavanaugh: A place to stand
Stephen Cairns: Architecture’s Intertwining
Jonathan Hill: An Infinity of Interpretations
Vassilis Ganiatsas: Interview
Anastasios Kotsiopoulos: The debatable appeal of structured matter in architecture
Constance Classen: Sensuous Worlds. Accessing the Invisible Environment
Stavros Stavrides: Inhabiting Rhythms
Ole Bouman: Designing to Socialise

Technology - space - architecture
William Mitchell: Digital Porosity
Richard Coyne: Interview
Neil Spiller: Towards a Theory of the Post-Digital in Architecture
Kas Oosterhuis, Nora Schueler: Fine-tuning the City
Zissis Kotionis: Interview
Stelarc: Extra Ear. Engineering Internet Organs
Andreas Angelidakis: Interview
Aristide Antonas: Interview

Mapping - recording
Slavoj Žižek: Passions of the Real, Passions of Semblance
Nikolaos Laskaris: The Depiction of Structured Space in the Language
of the Plastic Arts
Argyris Rokas: Interview
Andreas Kourkoulas: Interview
John Peponis: Plotting the Human Voice
Yorgos Ioannou: Ommonia 1980

Urban sounds
Yorgos Tzirtzilakis: Interview. The Dissolution of Architecture. Smooth Space, Mapping and Soundgraphy
Konstantinos Vita: Symphony of Space
Dionyssis Kapsalis: Interview
United Visual Artists: Interview
Platon Rivellis: Interview
Dimitris Filippidis: Interview

Recordings

Biographies

Acknowledgements

Contributors
Greek National Participation to the 11th International Architecture Exhibition La Biennale di Venezia "Out there. Architecture beyond building" (2008)

Organised by:
Hellenic Ministry of Culture [yppo.gr]
General Directorate of Modern Culture
Directorate of Visual Arts
Department for the Promotion of Contemporary Art

Curators:
Anastasia Karandinou
Christina Achtypi
Stylianos Giamarelos

Video works by Intothepill net [intothepill.net]
Artists:
Yiannis Grigoriadis
Yiannis Isidorou
Lina Theodorou

Sound Recording / Sound Design
Dimitris Miyakis [movement.gr]
Vangelis Lympouridis

Exhibition Graphics / Catalogue Design
Company [company-london.com]

Design and Implementation of interactive environment
2monochannels [2monochannels.com]
Audiovisual and interactive systems design / acoustic design / construction supervision
Iraklis Lampropoulos
Giorgos Lampropoulos
Software programming
Vassilis Boukis
Electronic subsystem design
Michail Kritsotakis
Electrical Design
Giorgos Satolias

Interconnection of interactive elements
Vangelis Lympouridis [inter-axions.com]
Dimitris Miyakis

Light design
L+DG lighting architects [lightingdg.com]
Thomas Gravanis
Christina Frangeti

Construction
Michail Gavrilos [gavrilos.gr]

Digital printing
Polichromo [polichromo.com]

Translations
Rachel Howard
Nikos Masourides

Catalogue photographs
Intothepill – Internet video platform

Catalogue published by
futura publications

Marketing communication
Chryssa Vrouzi
Communication associate
Katerina Stamidi

Photographer
Cathy Cunliffe [cathycunliffeΑΤgmail.com]

For their financial and material support for the Greek participation at the 11th International Exhibition of Architecture, La Biennale di Venezia, we express our deepest thanks to the sponsors:

Alexander S. Onassis Public Benefit Foundation [onassis.gr]
Akzonobel [akzonobel.com]
Carteco - Architectural Materials & Design [carteco.gr]
L+DG Lighting Architects [lightingdg.com]
Plaisio [plaisio.gr]
Polichromo Advertising Applications [polichromo.com]
iGuzzini illuminazione [iguzzini.com]
Diathlasis Architectural Lighting [diathlasis.gr]

Sonic interaction is the continuous relationship between user actions and sound, mediated by some technology. Because interaction with sound may be task oriented or experience-based it is important to understand the nature of action-sound relationships in order to design rich sonic interactions. We propose a participatory approach to sonic interaction design that first considers the affordances of sounds in order to imagine embodied interaction, and based on this, generates interaction models for interaction designers wishing to work with sound. We describe a series of workshops, called Form Follows Sound, where participants ideate imagined sonic interactions, and then realize working interactive sound prototypes. We introduce the Sonic Incident technique, as a way to recall memorable sound experiences. We identified three interaction models for sonic interaction design: conducting; manipulating; substituting. These three interaction models offer interaction designers and developers a framework on which they can build richer sonic interactions.

The richness of crossmodal feedback in car driving makes it an en- gaging, complex, yet “natural” activity. Audition plays an impor- tant role, as the engine sound, perceived in the cabin, conveys rel- evant cues about the vehicle motion. In this paper, we introduce a procedural and physically informed model for synthetic combustion engine sound, as an effective, flexible and computationally efficient alternative to sample-based and analysis/resynthesis approaches. The sound model, currently being developed as Max/MSP exter- nal, has been integrated in GeneCars, a driving simulator environ- ment for industrial sound design, and SkAT Studio, a demonstration framework for the rapid creation of audio processing workflows.

Shape-retaining freely-deformable interfaces can take innumerable distinct shapes, and creating specific target configurations can be a challenge. In this paper, we investigate how audio can guide a user in this process, through the use of either musical or metaphoric sounds. In a formative user study, we found that sound encouraged action possibilities and made the affordances of the interface perceivable. We also found that adding audio as a modality along with vision and touch, made a positive contribution to guiding users’ interactions with the interface.

This paper presents a knowledge-based, data-driven method for using data describing action-sound couplings collected from a group of people to generate multiple complex map-pings between the performance movements of a musician and sound synthesis. This is done by using a database of multimodal motion data collected from multiple subjects coupled with sound synthesis parameters. A series of sound stimuli is synthesised using the sound engine that will be used in performance. Multimodal motion data is collected by asking each participant to listen to each sound stimulus and move as if they were producing the sound using a musical instrument they are given. Multimodal data is recorded during each performance, and paired with the synthesis parameters used for generating the sound stimulus. The dataset created using this method is then used to build a topological representation of the performance movements of the subjects. This representation is then used to interactively generate training data for machine learning algorithms , and define mappings for real-time performance. To better illustrate each step of the procedure, we describe an implementation involving clarinet, motion capture, wear-able sensor armbands, and waveguide synthesis.