Sound Spatialisation

This project presents a model to sonify data. It uses a multichannel audio system to create a spatial illusion of the presence of the sound source. This in order to present with a way of translating information from other domains into sound, at the same time that the spatial position conveys information also from this data.

The PhD investigates the creation of closeness and immediacy through composition, exploring the processes of capturing, processing and composing sound materials, their spatialisation both during production and performance, and the sound materials' contexts. It is suggested that closeness can be understood spatially, temporally and in addition as being familiar with sounds and musical languages; whereas immediacy adds the meaning of being involved at some level in the shaping or decoding of the meaning of sounds of a composition. In that sense closeness and immediacy together form entry-points for the listener to make him/her become engaged in the compositional narrative.Seven original acousmatic and mixed media works are presented in the portfolio. These are stone and metal, empty rooms, weave/unravel, skalna, pulses, beeps and triptych. The pieces rely on found sounds and their referential qualities, with both informing the compositional methodologies. They also borrow elements from soundscape composition, electronic music and film music. Over the development of the portfolio the inclusion of elements of other genres of music became a valuable source of inspiration and shaped the compositional methodology which lead to the development of a unique, personal style of composition.Three of the PhD’s compositions – pulses, beeps and triptych – investigate the musical opportunities of an acousmatic take on stems to improve the flexibility and perceived depth of spatialisation. The spatial layers of the compositions are split into parts of a soundfile. These parts can be mapped according to specific rules to the number of loudspeakers available. The portfolio pieces demonstrate that composing in spatial stems enhances spatial depth as close and distant sounds can be reproduced independently of each other on dedicated loudspeakers at the same time. The sounds of the distant loudspeakers merge with the acoustic properties of the performance space and therefore assist in making the composed spaces credible.In addition to the compositions, one original software tools are presented in the portfolio (PLib), as well as a substantial contribution to an existing tool (MANTIS Diffusion System). They aim to facilitate the production and performance of electroacoustic music. Their application and potential is briefly discussed in the commentary.

The audiovisual history charted in chapter two is here revoiced in terms of spatial expansion. It is argued that video technology did not initiate a new form of creative engagement with its performance spaces, but rather represented the peak of a spatial expansion that had been gathering pace throughout the twentieth century. As musicians explored their spatial parameters and artists included time in their work, attention was drawn to traditional viewing and listening procedures. Drawing on the theories of László Moholy-Nagy, Siegfried Giedion, Christopher Small and Brian O’Doherty, this chapter compares the conventions of gallery exhibition and display, viewing procedures and audience behaviour with the customs and aesthetics of listening in the traditional concert hall. Performance and installation art, aleatoric music and communal composition are used to propose a theory of spatialised creativity, audience activation and performativity.

Timbral spatialisation is a signal processing technique that involves the spatial treatment of all individual spectral bands extracted from a source sound. Previous research proposed that Wave Terrain Synthesis can be used as an effective bridging control structure for timbral spatialisation, enabling gestural control of the thousands of panning parameters required [18]. This paper develops on the possibilities and challenges of establishing a spatial language for timbral spatialisation in live computer music, addressing problems and ideas in pertinent writings on the notion of space, spectromorphology, spatial motion, and immersiveness by Smalley, Wishart, Normandeau, Rumsey, Kendall, and Sazdov.

Explorations of a new mapping strategy for spectral spatial- isation demonstrate a concise and flexible control of both spatiomorphology and spectromorphology. With the crea- tion of customized software by the author for audio-rate histograms, spectral processing function smoothing, spec- tral centroid width modulation, audio-rate distance-based amplitude panning, audio-rate ambisonic equivalent pan- ning, a growing library of audio trajectory functions, and an assortment of spectral transformation functions, this article aims to explain the rationale of this process.

Stem-based multichannel compositions promise greater flexibility in different listening situations and therefore improve the chances of a rewarding listening experience. The idea in stems is to separate materials or spatial layers into a collection of sound files which can be reproduced in various ways depending on the situation's needs. Also, splitting musical layers into separate sound files could result in higher spatial depth and improved off-centre listening. However, composing and performing stem-based compositions poses its own challenges which shall be laid out in the paper. The discussion will focus on the benefits, limitations and compositional methodology of spatial stems.

Nous présentons ici un instrument conçu pour le projet Vertiges d’espace et destiné à la diffusion sonore dans le contexte d’une performance de musique électroacoustique improvisée. Cet instrument a été expérimenté en concert et les premiers aboutissements nous permettent d’entrevoir des solutions concernant l’équilibre entre le nombre de contrôles nécessaires, le caractère et la gestion de la mise en espace envisagée.

This practice-based PhD set out to design and apply novel multi-channel and nonintrusive electroencephalographic wave analysers, along with environmental sensors, to provide extensive compositional biofeedback in gallery and non-gallery settings as well as public spaces where Virtual Reality, Augmented Reality and/or Mixed Reality technologies may also possibly be in use. There is a current gap in knowledge about the nature and behaviours of auditory processes and interactions in both sound art performances and in-built environment situations (whether in the art gallery or any other type of space); some research has been conducted but this unique combination of concerns has not to date been investigated in either practice or theory in the manner undertaken in this thesis. The methodology of the research is practice-based experimentation to contrive and assess sound art, in situ, ranging from small venues to outdoor public spaces. The intention is to better characterise sensory feedback established from a practice-based research to a comprehensive explorative framework. The intended result - as achieved in this PhD - is to map a multitude of possible correlations and interactions to create responsive immersive conditions. The intended outcome - also achieved - is to develop a new method for compositional design and perceptual analysis of relevance and applicability to a variety of spaces. The compositional aspect of this novel research project has built upon Sound Art and Sensorial Practice, and has woven in and informed this practice with theory and critical frameworks to help future practitioners and scholars engage with the work and take it further in their own ways, for their own purposes. The thesis as a whole delivers a written dissertation that builds upon and shares new knowledge gained from the practice-based study, and offers a new and novel set of techniques that future audio-visual artists and designers alike may choose to apply, in order to induce a sensorial dimension in their own works, whereas future scholars may build upon the methodology and the examples provided as a starting point for future Sound Art Practice research. The Sound Art practice is itself submitted as an integral part of the thesis and should be listened to in conjunction with reading of the materials. Overall, the perceptual findings inform a research methodology and educational foundation underpinning a new basis for research and development in the fields of Sound Art, Acoustics and Architecture.

Au début des années 1950, les recherches de José Bernhart et Jean
Wilfrid Garrett sur le système de diffusion stéréophonique donnent
lieu à la première diffusion stéréophonique à la radio : celle de la pièce
de théâtre radiophonique, Une larme du diable, pièce de Théophile
Gautier, le 19 juin 1950. Pour bénéficier d’une écoute stéréophonique,
les auditeurs devaient disposer de deux postes de radio, réglés sur deux
chaînes différentes. L’adaptation radiophonique de la pièce par Jean
Forest est mise en ondes sous la direction artistique de René Clair.
« Par quels artifices aurait-on pu représenter au théâtre des êtres, des
choses, ou des substances aussi dénués de pesanteur et qui ne prennent
corps que grâce au prestige de la poésie ? Il a fallu attendre jusqu’à ce
jour pour imaginer ces artifices, et c’est la radio qui nous en fournit
le secret ». René Clair voit dans la radio quasiment le seul média permettant de mettre en scène l’imaginaire et la poésie d’Une larme du
diable. Les images sonores se multiplient et s’enrichissent grâce aux
techniques d’enregistrement à l’aide d’une tête artificielle pour la captation, suivi d’une diffusion stéréophonique à la radio. Les différents
espaces sonores traversés pendant la pièce viennent se superposer à
l’espace domestique de l’auditeur, et l’ensemble forme ainsi ce que
l’on pourrait définir comme une cospatialité sonore. D’autres formes
de cospatialité sonore sont aussi présentes dans les étapes mêmes de la
création d’œuvres de théâtre radiophonique : de l’écriture à la prise de
son et la mise en espace des sources radiophoniques, mais aussi dans
l’espace même de la « scène imaginaire » de l’auditeur. Cette étude se
propose ici de décrire les modalités techniques d’enregistrement à la
radio et de diffusion sonore stéréophonique ouvrant à la cospatialité
sonore.

Sound spatialisation is an important component in interactive per- formances as well as in game audio and virtual or mixed reality systems. HCI practices are increasingly focused on creating a natural user experience and embodied interaction through gestu- ral control. Body movements that coincide with sounds consist of both performed ‘sound producing‘ gestures, and ancillary and communicative movements. Thus, different gestural typologies may relate to the same audio source. Furthermore, gestures may depend on the context in which they have been expressed; in other words, they can carry different semantic or semiotic meanings in relationship to the situation and environment or reality in which they have been enacted. In order to explore these research themes, we are developing gSPAT: a software and hardware system able to drive live sound spatialisation for interactive audio performance using gestural control based on human-meaningful gesture-sound relationships. The ultimate aim is to provide a highly natural and musically expressive sound spatialisation experience for the per- former. Here we describe three experiments conducted to explore the possible directions for the future of gSPAT’s development. The tests employ a range of practice-based and ethnographic research methods to establish applicability, naturalness and usability across a range of approaches to the interaction design of the system.