research-article

Audiovisual granular synthesis: micro relationships between sound and image

Authors:

Stefan GreuterAuthors Info & Claims

IE '13: Proceedings of The 9th Australasian Conference on Interactive Entertainment: Matters of Life and Death

Article No.: 8, Pages 1 - 7

https://doi.org/10.1145/2513002.2513568

Published: 30 September 2013 Publication History

Abstract

The real-time audio-visual instrument presented in this paper enables a unified environment for composition and performance through audio-visual granular synthesis. The instrument empowers the user to reconstruct grains into new experiences and make possible real-time improvisation of the material at the granular level. The user can form audio-visual relationships through interacting with and observing perceptual similarities between sound and image at the micro and macro level. These relationships provide the user with a unified parameter for audio-visual manipulation. Throughout the paper I discuss and illustrate the design of the instrument, its performance potential and its innovative application in the field of interactive entertainment, particularly in the performance work of DJs and VJs.

References

[1]

Akenine-Moller, T & Haines, E 2002, Real-Time Rendering, 2^nd edn, AK Peters, Natick Massachusetts.

Digital Library

[2]

Alves, B 2012, 'Consonance and Dissonance in Visual Music', Organised Sound, vol. 17, no. 2, pp. 114--119.

Digital Library

[3]

Granular Synthesis: Modell 5. Motion Control 1996, DVD, HTBA Hull, Germany, 4 February.

[4]

Jones, R 2008, New Eyes for the Mind, The Cinematic Experience, viewed 5 March 2009, <http://2uptech.com/papers/new_eyes.pdf>

[5]

Jordan, M & McLaren, N 1953, 'Notes on Animated Sound', The Quarterly of Film Radio and Television, vol. 7, no. 3, pp. 223--229.

[6]

Kamien, R 2010, Music: an appreciation, 10^th edn, McGraw-Hill Higher Education, Australia and New Zealand.

[7]

McDonnell, M 2007, Visual Music, Sounding Visual, viewed 20 March 2013, http://www.soundingvisual.com/visualmusic/VisualMusicEssay.pdf

[8]

MetaSynth 5 for OSX, software, version 5.01, U&I Software, Paris.

[9]

Moritz, W 1986, Towards an Aesthetics of Visual Music, Center for Visual Music, viewed 10 April 2011, <http://www.centerforvisualmusic.org/TAVM.htm>

[10]

openFrameworks 007, open source code, version 0.7.4.

[11]

Penner, R 2002, Programming Macromedia Flash, 1^st edn, McGraw-Hill/Osborne, Berkeley California.

Digital Library

[12]

Roads, C 2001, Microsound, 1^st edn, MIT Press, Cambridge Massachusetts.

[13]

Truax, B 1988, 'Real-Time Granular Synthesis with a Digital Signal Processor', Computer Music Journal, vol. 12, no. 2, pp. 14--26.

[14]

Truax, B 1990, 'Composing with Real-Time Granular Sound', Perspectives of New Music, vol. 28, no. 2, pp. 120--134.

[15]

TVestroy, 2009, Extrait, Audiovisual Performance, STRP Festival, NL, viewed 20 October 2011, <http://www.youtube.com/watch?v=lhHmncmy2Vs>

[16]

Ward, O 2006, 'The man that heard his paintbox hiss', The Telegraph, 10 June, viewed 10 July 2012, <http://www.telegraph.co.uk/culture/art/3653012/The-man-who-heard-his-paintbox-hiss.html>

Cited By

Xu S(2023)AI Color Organ: Piano Music Visualization using Onset Detection and HistoGANHighlights in Science, Engineering and Technology10.54097/hset.v39i.653939(274-279)Online publication date: 1-Apr-2023
https://doi.org/10.54097/hset.v39i.6539
Ruiz-del Olmo FVertedor-Romero JAlonso-Calero J(2019)Creación sonora y nuevas tendencias artísticas en el siglo XXI: Algoritmos, música electrónica y AlgoraveArte, Individuo y Sociedad10.5209/ARIS.6082531:2(425-440)Online publication date: 11-Mar-2019
https://doi.org/10.5209/ARIS.60825

Index Terms

Audiovisual granular synthesis: micro relationships between sound and image
1. Software and its engineering
  1. Software notations and tools
    1. General programming languages
      1. Language features

Recommendations

Creative Control of Granular Synthesis Using Fluid Simulation & Motion Tracking
MOCO '14: Proceedings of the 2014 International Workshop on Movement and Computing

This paper describes the development of an audio-visual performance system which applies 'reality based interaction' techniques. The real-time gestures and sounds of a musician playing an acoustic instrument are tracked and translated into forces which ...
Audio-visual granular synthesis performance demo
IE '13: Proceedings of The 9th Australasian Conference on Interactive Entertainment: Matters of Life and Death

In this paper, I present a prototype of my audio-visual granular synthesis instrument Kortex. The instrument enables real-time improvisation of audio-visual material in a performance context. Granular synthesis is a processing technique that segments ...
RhythmSynthesis: visual music instrument
C&C '11: Proceedings of the 8th ACM conference on Creativity and cognition

Originating as an investigation into the relationships between rhythm and technology, RhythmSynthesis applies color, shape, and sound to demonstrate how our understanding of visual music, computation, and tangible, audio-visual interactions can be ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

IE '13: Proceedings of The 9th Australasian Conference on Interactive Entertainment: Matters of Life and Death

September 2013

243 pages

ISBN:9781450322546

DOI:10.1145/2513002

Conference Chairs:
Stefan Greuter
RMIT University
,
Christian McCrea
RMIT University
,
Program Chairs:
Florian Mueller
RMIT University
,
Larissa Hjorth
RMIT University
,
Deborah Richards
Macquarie University

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Macquarie University-Sydney

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 September 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IE'2013

Sponsor:

IE'2013: The 9th Australasian Conference on Interactive Entertainment -

September 30 - October 1, 2013

Melbourne, Australia

Acceptance Rates

IE '13 Paper Acceptance Rate 20 of 51 submissions, 39%;

Overall Acceptance Rate 64 of 148 submissions, 43%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
222
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)2

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xu S(2023)AI Color Organ: Piano Music Visualization using Onset Detection and HistoGANHighlights in Science, Engineering and Technology10.54097/hset.v39i.653939(274-279)Online publication date: 1-Apr-2023
https://doi.org/10.54097/hset.v39i.6539
Ruiz-del Olmo FVertedor-Romero JAlonso-Calero J(2019)Creación sonora y nuevas tendencias artísticas en el siglo XXI: Algoritmos, música electrónica y AlgoraveArte, Individuo y Sociedad10.5209/ARIS.6082531:2(425-440)Online publication date: 11-Mar-2019
https://doi.org/10.5209/ARIS.60825

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents