research-article

A Smart Mirror for Music Conducting Exercises

Authors:

Andrea Salgian,

David Vickerman,

David VassalloAuthors Info & Claims

Thematic Workshops '17: Proceedings of the on Thematic Workshops of ACM Multimedia 2017

Pages 544 - 549

https://doi.org/10.1145/3126686.3129333

Published: 23 October 2017 Publication History

Abstract

Music conductors are at a disadvantage compared to musicians playing an instrument, in that they don't receive immediate feedback during individual practice. Since the orchestra is not always readily available, beginner conductors practice in front of a mirror and are left to judge their own performance, which can be a pretty difficult and subjective task.

In this paper we present a low-cost computer-based system that, in addition to the mirror image, provides users with real time feedback about musical conducting performance. The system uses a Microsoft Kinect to track upper body joint coordinates, and a Myo armband to provide more exact arm positioning data. Using these devices, we are able to detect common mistakes such as swaying, rocking, excessive hinge movement, mirroring, and incorrect palm position, as well as measure tempo and classify the articulation type as staccato or legato.

The system has been well received by conducting students and their instructor, as it is very easy to set up and use, and it allows them to practice by themselves, without an orchestra.

References

[1]

T. Baba, M. Hashida, and H. Katayose. 2010. "VirtualPhilharmony": A Conducting System with Heuristics of Conducting an Orchestra New Interfaces for Musical Expression (NIME). Sydney, Australia.

[2]

R. Behringer. 2005. Conducting Digitally Stored Music by Computer Vision Tracking Proceedings of the First International Conference on Automated Production of Cross Media Content for Multi-Channel Distribution (AXMEDIS?05). Florence, Italy.

Digital Library

[3]

Diana Hollinger and Jill M. Sullivan. 2007. The Effects of Technology-Based Conducting Practice on Skill Achievement in Novice Conductors. Research and Issues in Music Education Vol. 5, 1 (2007).

[4]

P. Kolesnik and M. Wanderley. 2004. Recognition, Analysis and Performance with Expressive Conducting Gestures Proceedings of the International Computer Music Conference. Miami, FL.

[5]

Thalmic Labs. 2015. Myo Armband. https://www.myo.com/. (2015). Accessed:2017-04-14.

[6]

Y. K. Lim and W. S. Yeo. 2014. Smartphone-based Music Conducting. In New Interfaces for Musical Expression (NIME). London, UK.

[7]

T. Marrin and J. Paradiso. 1997. The digital baton: A versatile performance instrument Proceedings of the International Computer Music Conference. Thessaloniki, Greece.

[8]

Microsoft. 2013. Meet Kinect for Windows. https://developer.microsoft.com/en-us/windows/kinect. (2013). Accessed: 2017-04-14.

[9]

D. Murphy, T. H. Andersen, and K. Jensen. 2003. Conducting audiofiles via computer vision. In Proceedings of the 5th International Gesture Workshop, LNAI. Genoa, Italy.

[10]

T. M. Nakra, Y. Ivanov, P. Smaragdis, and C. Ault. 2009. The UBS Virtual Maestro: an Interactive Conducting System New Interfaces for Musical Expression (NIME). Pittsburgh, PA.

[11]

Jerry Nowak. 2002. Conducting the music, not the musicians. Carl Fischer, New York.

[12]

A. Salgian, L. Hsu, N. Milkosky, and D. Vickerman. 2015. Conductor Tutoring Using the Microsoft Kinect. In International Symposium on Visual Computing - Advances in Visual Computing. Pittsburgh, PA.

[13]

A. Salgian and D. Vickerman. 2016. Computer-Based Tutoring for Conducting Students. Proceedings of the International Computer Music Conference. Utrecht, Netherlands.

[14]

Dilip Swaminathan, Harvey Thornburg, Todd Ingalls, Stjepan Rajko, Jodi James, Ellen Campana, Kathleya Afanador, and Randal Leistikow. 2008. Computer Music Modeling and Retrieval. Sense of Sounds. Springer-Verlag, Berlin, Heidelberg, Chapter Capturing Expressive and Indicative Qualities of Conducting Gesture: An Application of Temporal Expectancy Models, 34--55.

Digital Library

[15]

S. Usa and Y. Mochida. 1998. A conducting recognition system on the model of musicians' process. J. Acoust. Soc. Jpn., Vol. 19, 4 (1998).

[16]

A. Wilson and A. Bobick. 1999. Realtime online adaptive gesture recognition. In Proceedings of the International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems. Corfu, Greece.

Digital Library

Cited By

Aziz M(2021)DESIGN AND DEVELOPMENT OF RESOLDEPMIRROR: A SMART MIRROR FOR RESOLVING DEPRESSIONJOURNAL OF MECHANICS OF CONTINUA AND MATHEMATICAL SCIENCES10.26782/jmcms.2021.01.0000316:1Online publication date: 26-Jan-2021
https://doi.org/10.26782/jmcms.2021.01.00003
Cui XLiao MLv PZheng C(2019)Sequencing the musical sections with deep learning2019 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2019.8851935(1-7)Online publication date: Jul-2019
https://doi.org/10.1109/IJCNN.2019.8851935

Index Terms

A Smart Mirror for Music Conducting Exercises
1. Applied computing
  1. Arts and humanities
    1. Performing arts
  2. Education
    1. Computer-assisted instruction
2. Human-centered computing
  1. Interaction design
    1. Interaction design process and methods
      1. Activity centered design

Recommendations

Music Control in an Interactive Conducting System Using Kinect

Music conducting is the art of directing musical ensembles with hand gestures to personalize and diversify a piece of music. Although the ability to successfully perform a musical piece demands intense training and coordination for the conductor and the ...
An analysis of basic expressive qualities in instrumental conducting
MOCO '15: Proceedings of the 2nd International Workshop on Movement and Computing

Advances in sensing technology make the task of quantifying expressive human body movement more feasible than ever before. Success will enable breakthroughs in Human-computer interaction (HCI) and control paradigms. In most areas, however, expressivity ...
Conducting Digitally Stored Music by Computer Vision Tracking
AXMEDIS '05: Proceedings of the First International Conference on Automated Production of Cross Media Content for Multi-Channel Distribution

In this paper, a method for controlling electronic digital music instruments is proposed, based on visual capture of baton and hand motion of a conductor. This approach is suitable for being applied in mixed ensembles of human musicians and electronic ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

Thematic Workshops '17: Proceedings of the on Thematic Workshops of ACM Multimedia 2017

October 2017

558 pages

ISBN:9781450354165

DOI:10.1145/3126686

Program Chairs:
Wanmin Wu
Google, USA
,
Jianchao Yang
Snap Inc., USA
,
Qi Tian
The University of Texas at San Antonio, USA
,
Roger Zimmermann
National University of Singapore, Singapore

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '17

Sponsor:

SIGMM

MM '17: ACM Multimedia Conference

October 23 - 27, 2017

California, Mountain View, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
141
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)1

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Aziz M(2021)DESIGN AND DEVELOPMENT OF RESOLDEPMIRROR: A SMART MIRROR FOR RESOLVING DEPRESSIONJOURNAL OF MECHANICS OF CONTINUA AND MATHEMATICAL SCIENCES10.26782/jmcms.2021.01.0000316:1Online publication date: 26-Jan-2021
https://doi.org/10.26782/jmcms.2021.01.00003
Cui XLiao MLv PZheng C(2019)Sequencing the musical sections with deep learning2019 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2019.8851935(1-7)Online publication date: Jul-2019
https://doi.org/10.1109/IJCNN.2019.8851935

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents