An Intuitive Interface for Digital Synthesizer by Pseudo-intention Learning
Pages 39 - 44
Abstract
Digital musical instruments are essential technologies in modern musical composition and performance. However, the interface of the synthesizer is not intuitive enough and require extra knowledge because of the parameters. To address this problem, we propose pseudo-intention learning: a novel data collection method for supervised learning in musical instrument development. Pseudo-intention learning collects a data set of the paired target tone and input performed by the user. We developed a conversion framework that reflects the composer's intention by combining standard convolutional neural network and pseudo-intention learning. As a proof of concept, we constructed an interface that can freely manipulate the sound source of a digital snare drum and demonstrated its effectiveness with a pilot study. We confirmed that the tone parameters generated by our system reflected the user's intention. We also discuss applying this method to richer musical expression.
References
[1]
2010. LiveOSC: Complete Control of Ableton Live Using Your iPad or iPod. https://livecontrol.q3f.org/ableton-liveapi/liveosc/. (Accessed on 06/14/2019).
[2]
2015. The Sensel Morph. https://sensel.com/morph. (Accessed on 06/14/2019).
[3]
Ableton. 2019. Ableton Live 10. https://www.ableton.com/en/live/. (Accessed on 06/14/2019).
[4]
Frédéric Bevilacqua, Remy Müller, and Norbert Schnell. 2005. MnM: a Max/MSP mapping toolbox. 85--88.
[5]
Frédéric Bevilacqua, Bruno Zamborlin, Anthony Sypniewski, Norbert Schnell, Fabrice Guédy, and Nicolas H. Rasamimanana. 2009. Continuous Realtime Gesture Following and Recognition. Gesture in Embodied Communication and Human-Computer Interaction 5394, 73--84. https://doi.org/10.1007/978-3-642-12553-9_7
[6]
Ole Martin Bjørndalen. 2013. Mido: MIDI Objects for Python. https://mido.readthedocs.io/en/latest/. (Accessed on 06/14/2019).
[7]
Chris Donahue, Ian Simon, and Sander Dieleman. 2019. Piano Genie. In Proceedings of the 24th International Conference on Intelligent User Interfaces (IUI '19). ACM, New York, NY, USA, 160--164. https://doi.org/10.1145/3301275.3302288
[8]
Rebecca Fiebrink and Perry R Cook. 2010. The Wekinator: a system for real-time, interactive machine learning in music. In Proceedings of The Eleventh International Society for Music Information Retrieval Conference (ISMIR 2010)(Utrecht).
[9]
Rebecca Fiebrink, Dan Trueman, and Perry R Cook. 2009. A Meta-Instrument for Interactive, On-the-Fly Machine Learning. In NIME. 280--285.
[10]
Jules FranÃğoise. 2015. Motion-Sound Mapping by Demonstration. Ph.D. Dissertation. https://doi.org/10.13140/RG.2.1.5035.0248
[11]
Nicholas Edward Gillian, R. Benjamin Knapp, and M. Sile O'Modhrain. 2011. A Machine Learning Toolbox For Musician Computer Interaction. In NIME.
[12]
Rolf Inge Godøy, Egil Haga, and Alexander Refsum Jensenius. 2006. Playing "Air Instruments": Mimicry of Sound-Producing Gestures by Novices and Experts. In Gesture in Human-Computer Interaction and Simulation, Sylvie Gibet, Nicolas Courty, and Jean-François Kamp (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 256--267.
[13]
Rolf Inge Godøy and Alexander Refsum Jensenius. 2009. Body Movement in Music Information Retrieval. In ISMIR.
[14]
Sanghwa Hong, Eunseok Jeong, Seongkook Heo, and Byungjoo Lee. 2018. FD-Sense: Estimating Young's Modulus and Stiffness of End Effectors to Facilitate Kinetic Interaction on Touch Surfaces. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST '18). ACM, New York, NY, USA, 809--823. https://doi.org/10.1145/3242587.3242644
[15]
Tejaswinee Kelkar and Alexander Refsum Jensenius. 2017. Representation Strategies in Two-handed Melodic Sound-Tracing. In Proceedings of the 4th International Conference on Movement Computing (MOCO '17). ACM, New York, NY, USA, Article 11, 4 pages. https://doi.org/10.1145/3077981.3078050
[16]
Charles P. Martin and Jim Tørresen. 2019. An Interactive Musical Prediction System with Mixture Density Recurrent Neural Networks. CoRR abs/1904.05009 (2019). arXiv:1904.05009 http://arxiv.org/abs/1904.05009
[17]
Masataka Niwa, Hiroyuki Iizuka, Hideyuki Ando, and Taro Maeda. 2012. Tsumori Control: Manipulate Robot with detection and transmission of an Archetype of Behavioral Intention. Transactions of the Virtual Reality Society of Japan 17, 1 (2012), 3--10. https://doi.org/10.18974/tvrsj.17.1_
[18]
Kristian Nymoen, Baptiste Caramiaux, Mariusz Kozak, and Jim Torresen. 2011. Analyzing Sound Tracings - A Multimodal Approach to Music Information Retrieval. MM' 11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops - MIRUM 2011 Workshop, MIRUM'11, 39--44. https://doi.org/10.1145/2072529.2072541
[19]
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. (2017).
[20]
PyPI. 2019. pylive. https://pypi.org/project/pylive/. (Accessed on 06/14/2019).
[21]
Udit Roy, Tejaswinee Kelkar, and Bipin Indurkhya. 2014. TrAP: An Interactive System to Generate Valid Raga Phrases from Sound-Tracings.
[22]
Munehiko Sato, Ivan Poupyrev, and Chris Harrison. 2012. Touché: Enhancing Touch Interaction on Humans, Screens, Liquids, and Everyday Objects. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 483--492. https://doi.org/10.1145/2207676.2207743
[23]
Hugo Scurto and Rebecca Fiebrink. 2016. Grab-and-Play Mapping: Creative Machine Learning Approaches for Musical Inclusion and Exploration. (01 2016).
[24]
Jeff Snyder and Danny Ryan. 2014. The Birl: An Electronic Wind Instrument Based on an Artificial Neural Network Parameter Mapping Structure.
[25]
Y. Zhang and Z. Duan. 2016. IMISOUND: An unsupervised system for sound query by vocal imitation. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2269--2273. https://doi.org/10.1109/ICASSP.2016.7472081
Index Terms
- An Intuitive Interface for Digital Synthesizer by Pseudo-intention Learning
Recommendations
ToCoPlay: graphical multi-touch interaction for composing and playing music
INTERACT'11: Proceedings of the 13th IFIP TC 13 international conference on Human-computer interaction - Volume Part IIIWith the advent of electronic music and computers, the human-sound interface is liberated from the specific physical constraints of traditional instruments, which means that we can design musical interfaces that provide arbitrary mappings between human ...
Guitar man: (an implementation of a rhythm game cooperative musical performance system with actual musical instruments)
SIGGRAPH Asia '08: ACM SIGGRAPH ASIA 2008 educators programmeThis study proposes a method for learning musical instruments using computer graphics. The conventional musical instrument learning systems have been considered as simple game controllers or simple score processors. This study produced an education ...
ism: improvisation supporting system based on melody correction
NIME '04: Proceedings of the 2004 conference on New interfaces for musical expressionIn this paper, we describe a novel improvisation supporting system based on correcting musically unnatural melodies. Since improvisation is the musical performance style that involves creating melodies while playing, it is not easy even for the people ...
Comments
Information & Contributors
Information
Published In
September 2019
310 pages
ISBN:9781450372978
DOI:10.1145/3356590
Copyright © 2019 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
In-Cooperation
- The University of Nottingham: The University of Nottingham
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 18 September 2019
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
AM'19
Acceptance Rates
AM '19 Paper Acceptance Rate 25 of 49 submissions, 51%;
Overall Acceptance Rate 177 of 275 submissions, 64%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 142Total Downloads
- Downloads (Last 12 months)5
- Downloads (Last 6 weeks)0
Reflects downloads up to 10 Aug 2024
Other Metrics
Citations
Cited By
View allView Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in