Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
  • Bill Manaris is a computer science researcher, educator, and musician. He is Professor of Computer Science, and Direc... moreedit
Making Music with Computers: Creative Programming in Python introduces important concepts and skills necessary to generate music with computers. It interweaves computing pedagogy with musical concepts and creative activities, showing... more
Making Music with Computers: Creative Programming in Python introduces important concepts and skills necessary to generate music with computers. It interweaves computing pedagogy with musical concepts and creative activities, showing students how to integrate the creativity and design of the arts with the mathematical rigor and formality of computer science. The book provides an introduction to creative software development in the Python programming language.
This is the fifth year of publication of an annual survey of departments offering degree programs accredited by the Computer
Development environments play a crucial role for novice programmers. Not only do they act as interface to type in and execute programs, but a programming environment is also responsible for reporting errors, managing in- and output when... more
Development environments play a crucial role for novice programmers. Not only do they act as interface to type in and execute programs, but a programming environment is also responsible for reporting errors, managing in- and output when the program is running, or offering the programmer access to the underlying notional machine. In recent years several new educational programming environments for Python have been presented. However, the important issue of reporting errors has rarely been addressed and evaluations often hint that students main issue is the poor quality of Python's error messages. We have therefore written an educational Python environment with enhanced error messages. This paper presents the design and rationale of its three primary features: modifications to Python, enhanced error messages, and the visual debugger.
Algorithms have existed for at least 2,000 years (e.g., Euclid's algorithm). In music and art, algorithms appear as early as Guido d'Arezzo (ca. 1000 A.D.), and in compositions by Bach, Mozart, John Cage, Iannis Xenakis, among... more
Algorithms have existed for at least 2,000 years (e.g., Euclid's algorithm). In music and art, algorithms appear as early as Guido d'Arezzo (ca. 1000 A.D.), and in compositions by Bach, Mozart, John Cage, Iannis Xenakis, among others. Modern examples include data sonification for scientific or aesthetic purposes, such as sonifying biosignals, images, orbits of planets, and human movement (e.g., dance), among others. This talk will focus on Computing in the Arts (CITA), an NSF-funded model curriculum, which combines creativity, problem solving, and computer programming to prepare students for graduate school and careers in technology and arts industries of the 21st century. CITA is part of the new movement to combine art and design with science, technology, engineering and math (STEM + Art = STEAM). Several examples will be presented, including: • SoundMorpheus (an innovative interface for positioning sounds via arm movements); • Diving into Infinity (a motion-based system wh...
We discuss the application of the Zipf–Mandelbrot law to musical pieces encoded in MIDI. Specifically, we have identified a set of metrics on which to apply this law. These metrics include pitch of musical events, duration of musical... more
We discuss the application of the Zipf–Mandelbrot law to musical pieces encoded in MIDI. Specifically, we have identified a set of metrics on which to apply this law. These metrics include pitch of musical events, duration of musical events, the combination of pitch and duration of musical events, and several others. Our hypothesis is that these metrics will allow us to computationally identify and perhaps emphasize aesthetic aspects of music. We have developed a system that automates calculation of these metrics in MIDIencoded pieces. Using this tool we conducted a study on a corpus of 220 pieces from baroque, classical, romantic, 12tone, jazz, rock, DNA strings, and random music. Our results support the above hypothesis. We discuss limitations, and how to minimize the potential for statistical error through composite metrics. We present possible applications and an overview of preliminary work in computer–aided music composition. 1.
Esperanto is a constructed natural language, which was intended to be an easy-to-learn lingua franca. Zipf's law models the statistical proportions of various phenomena in human ecology, including natural languages. Given Esperanto’s... more
Esperanto is a constructed natural language, which was intended to be an easy-to-learn lingua franca. Zipf's law models the statistical proportions of various phenomena in human ecology, including natural languages. Given Esperanto’s artificial origins, one wonders how “natural” it appears, relative to other natural languages, in the context of Zipf’s law. To explore this question, we collected a total of 283 books from six languages: English, French, German, Italian, Spanish, and Esperanto. We applied Zipf-based metrics on our corpus to extract distributions for word, word distance, word bigram, word trigram, and word length for each book. Statistical analyses show that Esperanto’s statistical proportions are similar to those of other languages. We then trained artificial neural networks (ANNs) to classify books according to language. The ANNs achieved high accuracy rates (86.3 % to 98.6%). Subsequent analysis identified German as having the most unique proportions, followed b...
This project applies Zipf's law on musical pieces encoded in MIDI. Our hypothesis is that this will allow us to identify musical pieces that humans find “pleasing, beautiful, harmonious.” Specifically, we have identified an initial... more
This project applies Zipf's law on musical pieces encoded in MIDI. Our hypothesis is that this will allow us to identify musical pieces that humans find “pleasing, beautiful, harmonious.” Specifically, we have identified an initial set of attributes (metrics) of music pieces on which to apply Zipf’s law. These metrics include pitch of musical events, duration of musical events, the combination of pitch and duration of musical events, and several others. Our preliminary results—derived mostly by tedious manual processing—are encouraging. We are working on automating our metrics so that we can test our hypothesis on a wide variety of music genres (baroque, classical, 20 th century, blues, jazz, etc.). If this project is successful, we plan to investigate how such metrics may be used to
The practice of coding music live challenges computing conventions with regard to developmental agility. The computational representation of music likewise challenges musicians to articulate their practice in new ways. In this article we... more
The practice of coding music live challenges computing conventions with regard to developmental agility. The computational representation of music likewise challenges musicians to articulate their practice in new ways. In this article we describe the development, teaching and use of the JythonMusic environment designed to meet these challenges head on. JythonMusic, written in Python, is an open source project for music making and creative programming activities intended for musicians and programmers, of all levels and backgrounds. JythonMusic supports algorithmic music composition, dynamic coding and musical performativity including live coding. This article examines the functionality of JythonMusic for composition and performance projects in the context of university courses that combine computer science and music. The advantages of this medium for music and programming instruction are demonstrated through several projects and a live coding case study, leading to a series of observations and proposals concerning the advantages of coding in music pedagogy.
This paper focuses on late-breaking results from a multiyear research project investigating Zipf’s Law in the context of music information retrieval and data mining. This research project has produced Armonique
We present an innovative sound spatialization and shaping interface, called SoundMorpheus, which allows the placement of sounds in space, as well as the altering of sound characteristics, via arm movements that resemble those of a... more
We present an innovative sound spatialization and shaping interface, called SoundMorpheus, which allows the placement of sounds in space, as well as the altering of sound characteristics, via arm movements that resemble those of a conductor. The interface displays sounds (or their attributes) to the user, who reaches for them with one or both hands, grabs them, and gently or forcefully sends them around in space, in a 360° circle. The system combines MIDI and traditional instruments with one or more myoelectric sensors. These components may be physically collocated or distributed in various locales connected via the Internet. This system also supports the performance of acousmatic and electronic music, enabling performances where the traditionally central mixing board, need not be touched at all (or minimally touched for calibration). Finally, the system may facilitate the recording of a visual score of a performance, which can be stored for later playback and additional manipulatio...
Abstract. We present Harmonic Navigator (HN), a system for navigating and exploring harmonic spaces extracted from large musical corpora, to be used in music composition and performance. A harmonic space is a set of harmonies (chords) and... more
Abstract. We present Harmonic Navigator (HN), a system for navigating and exploring harmonic spaces extracted from large musical corpora, to be used in music composition and performance. A harmonic space is a set of harmonies (chords) and transitions between harmonies found in a music corpus. By navi-gating this space, the user can derive new harmonic progressions, which have correct voice leading. HN is controllable via a Kinect gesture interface. To aid the user, the system also incorporates stochastic and evolutionary techniques. HN offers for two primary modes of interaction: a harmonic transition selector, called harmonic palette, which utilizes a GUI to navigate harmonic transitions in a front-to-back manner; and a harmonic-flow scrubber, which presents a global overview of a harmonic flow and allows the user to perform common audio scrubbing and editing tasks. Both GUIs use colors to indicate harmonic density based on Legname’s density degree theory.
This paper presents results on a user interface model for providing universal access to mobile computing devices.
Computer users with motor impairments find it difficult and, in many cases, impossible to access PC functionality through the physical keyboard-and-mouse interface. Studies show that even able-bodied users experience similar difficulties... more
Computer users with motor impairments find it difficult and, in many cases, impossible to access PC functionality through the physical keyboard-and-mouse interface. Studies show that even able-bodied users experience similar difficulties when interacting with mobile devices; this is due to the reduced size/usability of the input interfaces. Advances in speech recognition have made it possible to design speech interfaces for alphanumeric data entry and indirect manipulation (cursor control). Although several related commercial applications exist, such systems do not provide a complete solution for arbitrary keyboard and mouse access, such as the access needed for, say, typing, compiling, and executing a C++ program.
A study on a 220-piece corpus (baroque, classical, romantic, 12tone, jazz, rock, DNA strings, and random music) reveals that aesthetically pleasing music may be describable under the Zipf-Mandelbrot law. Various Zipf-based metrics have... more
A study on a 220-piece corpus (baroque, classical, romantic, 12tone, jazz, rock, DNA strings, and random music) reveals that aesthetically pleasing music may be describable under the Zipf-Mandelbrot law. Various Zipf-based metrics have been developed and evaluated. Some focus on musictheoretic attributes such as pitch, pitch and duration, melodic intervals, and harmonic intervals. Others focus on higher-order attributes and fractal aspects of musical balance. Zipf distributions across certain dimensions appear to be a necessary, but not sufficient condition for pleasant music. Statistical analyses suggest that combinations of Zipf-based metrics might be used to identify genre and/or composer. This is supported by a preliminary experiment with a neural network classifier. We describe an evolutionary music framework under development, which utilizes Zipf-based metrics as fitness functions.
Abstract—Monterey Mirror is an interactive stochastic music generator based on Markov models, genetic algorithms, and power-law metrics for music information retrieval. It combines the predictive power of Markov models with the innovative... more
Abstract—Monterey Mirror is an interactive stochastic music generator based on Markov models, genetic algorithms, and power-law metrics for music information retrieval. It combines the predictive power of Markov models with the innovative power of genetic algorithms, using power-law metrics for fitness evaluation. These metrics have been developed and refined in a decade-long project, which explores music information retrieval based on Zipf’s law and related power laws. We describe the architecture of Monterey Mirror, which can generate musical responses based on aesthetic variations of user input. We also explore how such a system may be used as a musical metainstrument / environment in avant-garde music composition and performance projects.
We present a music information retrieval approach based on power laws. Research in cognitive science and neuroscience reveals connections between power laws, human cognition, and human physiology. Empirical studies also demonstrate... more
We present a music information retrieval approach based on power laws. Research in cognitive science and neuroscience reveals connections between power laws, human cognition, and human physiology. Empirical studies also demonstrate connections between power laws and human aesthetics. We utilize 250+ power-law metrics to extract statistical proportions of music-theoretic and other attributes of music pieces. We discuss an experiment where artificial neural networks classify 2,000 music pieces, based on aesthetic preferences of human listeners, with 90.70 % accuracy. Also, we present audio results from a music information retrieval experiment, in which a music search engine prototype retrieves music based on “aesthetic ” similarity from a corpus of 15,200+ pieces. These results suggest that power-law metrics are a promising model of music aesthetics, as they may be capturing statistical properties of the human hearing apparatus. 1.
This paper presents results from an on-going MIR study utilizing hundreds of melodic and timbre features based on power laws for content-based similarity retrieval. These metrics are incorporated into a music search engine prototype,... more
This paper presents results from an on-going MIR study utilizing hundreds of melodic and timbre features based on power laws for content-based similarity retrieval. These metrics are incorporated into a music search engine prototype, called Armonique. This prototype is used with a corpus of 9153 songs encoded in both MIDI and MP3 to identify pieces similar to and dissimilar from selected songs. The MIDI format is used to extract various powerlaw features measuring proportions of music-theoretic and other attributes, such as pitch, duration, melodic intervals, and chords. The MP3 format is used to extract power-law features measuring proportions within FFT power spectra related to timbre. Several assessment experiments have been conducted to evaluate the effectiveness of the similarity model. The results suggest that power-law metrics are very promising for content-based music querying and retrieval, as they seem to correlate with aspects of human emotion and aesthetics. 1.
Where shall you seek beauty, and how shall you find her unless she herself be your way and your guide? And how shall you speak of her except she be the weaver of your speech? —Kahlil Gibran, The Prophet, p. 74 This project applies... more
Where shall you seek beauty, and how shall you find her unless she herself be your way and your guide? And how shall you speak of her except she be the weaver of your speech? —Kahlil Gibran, The Prophet, p. 74 This project applies Zipf's law on musical pieces encoded in MIDI. Our hypothesis is that this will allow us to identify musical pieces that humans find “pleasing, beautiful, harmonious. ” Specifically, we have identified an initial set of attributes (metrics) of music pieces on which to apply Zipf’s law. These metrics include pitch of musical events, duration of musical events, the combination of pitch and duration of musical events, and several others. Our preliminary results—derived mostly by tedious manual processing—are encouraging. We are working on automating our metrics so that we can test our hypothesis on a wide variety of music genres (baroque, classical, 20 th century, blues, jazz, etc.). If this project is successful, we plan to investigate how such metrics m...
This paper presents results on a user interface model for providing universal access to mobile computing devices. The model uses a continuous speech understanding engine to provide access to a virtual keyboard and mouse through speech... more
This paper presents results on a user interface model for providing universal access to mobile computing devices. The model uses a continuous speech understanding engine to provide access to a virtual keyboard and mouse through speech input. This research has been targeted towards users with permanent motor disabilities. However, these results also apply to able-bodied users with temporary, taskinduced motor disabilities, such as users performing alphanumeric data entry through a cellular phone keypad. The proposed solution might complement (or even replace) miniaturized keyboards and other physical keyboard alternatives, such as stylus-type "soft" keyboards. Since it only requires a microphone (and perhaps a speaker for feedback) which are already included in many mobile devices, it may allow such devices to shrink considerably in size, as alphanumeric input is no longer bound to a physical area. The paper describes the underlying architecture employed by the system. It p...
Monterey Mirror is an interactive stochastic music generator based on Markov models, genetic algorithms, and power-law metrics for music information retrieval. It combines the predictive power of Markov models with the innovative power of... more
Monterey Mirror is an interactive stochastic music generator based on Markov models, genetic algorithms, and power-law metrics for music information retrieval. It combines the predictive power of Markov models with the innovative power of genetic algorithms, using power-law metrics for fitness evaluation. These metrics have been developed and refined in a decade-long project, which explores music information retrieval based on Zipf’s law and related power laws. We describe the architecture of Monterey Mirror, which can generate musical responses based on aesthetic variations of user input. We also explore how such a system may be used as a musical metainstrument / environment in avant-garde music composition and performance projects.
We present a novel, real-time system for exploring harmonic spaces of musical styles, to generate music in collaboration with human performers utilizing gesture devices (such as the Kinect) together with MIDI and OSC instruments /... more
We present a novel, real-time system for exploring harmonic spaces of musical styles, to generate music in collaboration with human performers utilizing gesture devices (such as the Kinect) together with MIDI and OSC instruments / controllers. This corpus-based environment incorporates statistical and evolutionary components for exploring potential flows through harmonic spaces, utilizing power-law (Zipf-based) metrics for fitness evaluation. It supports visual exploration and navigation of harmonic transition probabilities through interactive gesture control. These probabilities are computed from musical corpora (in MIDI format). Herein we utilize the Classical Music Archives 14,000+ MIDI corpus, among others. The user interface supports real-time exploration of the balance between predictability and surprise for musical composition and performance, and may be used in a variety of musical contexts and applications.
Artificial Intelligence (AI) research has traditionally focused on exploring and modeling the left-brain side of human intelligence: science, math, logic, engineering, etc. This focus is probably correlated with the academic background of... more
Artificial Intelligence (AI) research has traditionally focused on exploring and modeling the left-brain side of human intelligence: science, math, logic, engineering, etc. This focus is probably correlated with the academic background of the typical AI researcher: heavily grounded in mathematics and computer science. Accordingly, the AI literature follows this trend, with the exception of a few specialized journals such as MIT’s Leonardo and Computer Music Journal.
Specter combines music information retrieval (MIR) with sound spatialization to provide a simple, yet versatile environment to experiment with sound spatialization for music composition and live performance. Through various interfaces and... more
Specter combines music information retrieval (MIR) with sound spatialization to provide a simple, yet versatile environment to experiment with sound spatialization for music composition and live performance. Through various interfaces and sensors, users may position sounds at arbitrary locations and trajectories in a three-dimensional plane. The system utilizes the JythonMusic environment for symbolic music processing, music information retrieval, and live audio manipulation. It also incorporates Iannix, a 3D graphical, open-source sequencer, for realtime generation, manipulation, and storing of sound trajectory scores. Finally, through Glaser, a sound manipulation instrument, Specter renders the various sounds in space. The system architecture supports different sound spatialization techniques including Ambisonics and Vector Based Amplitude Panning. Various interfaces are discussed, including a Kinect-based sensor system, a LeapMotion-based hand-tracking interface, and a smartphone...
A novel approach to the production of evolutionary art is presented. This approach is based on the promotion of an arms race between an adaptive classifier and an evolutionary computation system. An artificial neural network is trained to... more
A novel approach to the production of evolutionary art is presented. This approach is based on the promotion of an arms race between an adaptive classifier and an evolutionary computation system. An artificial neural network is trained to discriminate among images previously created by the evolutionary engine and famous paintings. Once training is over, evolutionary computation is used to generate images that the neural network classifies as paintings. The images created throughout the evolutionary run are added to the training set and the process is repeated. This iterative process leads to the refinement of the classifier and forces the evolutionary algorithm to explore new paths. The experimental results attained across iterations are presented and analyzed. Validation tests were conducted in order to assess the changes induced by the refinement of the classifier and to identify the types of images that are difficult to classify. Taken as a whole, the experimental results show th...
In this paper we propose a framework for the development of artificial art critics, consisting of an architecture and a validation methodology. The architecture includes two modules: a feature extractor, which performs a pre-processing of... more
In this paper we propose a framework for the development of artificial art critics, consisting of an architecture and a validation methodology. The architecture includes two modules: a feature extractor, which performs a pre-processing of the artwork, extracting several measurements and characteristics; and an evaluator, which, based on the output of the feature extractor, classifies the artwork according to a certain criteria. The validation methodology comprises several stages, ranging from author and style discrimination to the integration of the artificial art critic in a dynamic multi-agent environment, which includes human agents. Finally, we present experimental results obtained by the implementation of the proposed framework.
We present a method for combining data from multiple Kinect motion-capture sensors into a common coordinate system. Kinect sensors offer a cheaper, potentially less accurate alternative for full-body motion tracking. By incorporating... more
We present a method for combining data from multiple Kinect motion-capture sensors into a common coordinate system. Kinect sensors offer a cheaper, potentially less accurate alternative for full-body motion tracking. By incorporating multiple sensors into a multiscopic system, we address potential accuracy and recognition flaws caused by individual sensor conditions, such as occlusions and space limitations, and increase the overall accuracy of skeletal data tracking. We merge data from multiple Kinects using a custom calibration algorithm, called K-Multiscope. K-Multiscope generates an affine transform for each of the available sensors, and thus combines their data into a common 3D coordinate system. We have incorporated this algorithm into Kuatro, a skeletal data pipeline designed earlier to simplify live motion capture for use in music interaction experiences and installations. In closing, we present Liminal Space, a live duet performance for cello and dance, which utilizes the K...
JythonMusic is a software environment for developing interactive musical experiences and systems. It is based on jMusic, a software environment for computer-assisted composition, which was extended within the last decade into a more... more
JythonMusic is a software environment for developing interactive musical experiences and systems. It is based on jMusic, a software environment for computer-assisted composition, which was extended within the last decade into a more comprehensive framework providing composers and software developers with libraries for music making, image manipulation, building graphical user interfaces, and interacting with external devices via MIDI and OSC, among others. This environment is free and open source. It is based on Python, therefore it provides more economical syntax relative to Java- and C/C++-like languages. JythonMusic rests on top of Java, so it provides access to the complete Java API and external Javabased libraries as needed. Also, it works seamlessly with other software, such as PureData, Max/MSP, and Processing. The paper provides an overview of important JythonMusic libraries related to constructing interactive musical experiences. It demonstrates their scope and utility by su...
We describe a Kinect-based interface for navigating M.C. Escher’s works. Our interface is based on the Kuatro, a framework for developing motion-based interactive virtual environments. Kuatro utilizes the Model-View Controller (MVC)... more
We describe a Kinect-based interface for navigating M.C. Escher’s works. Our interface is based on the Kuatro, a framework for developing motion-based interactive virtual environments. Kuatro utilizes the Model-View Controller (MVC) architecture and Open Sound Control (OSC) to provide an expandable environment for motion-sensor based installations for composers, artists, and interaction designers. We present a case study based on “Print Gallery”, an intriguing, self-similar work created by M.C. Escher in 1956. Our interaction design involves a Kinect sensor, a video projector, a Kuatro server, and a screen; it allows a user to zoom in and out, as well as rotate the image to reveal its self-similarity, by navigating prerecorded video material. This material is based on previous mathematical analyses of “Print Gallery” to reveal / explain the artist’s depiction of infinity. We discuss adapting this approach to other M.C. Escher works involving infinity.
We discuss the application of the Zipf–Mandelbrot law to musical pieces encoded in MIDI. Specifically, we have identified a set of metrics on which to apply this law. These metrics include pitch of musical events, duration of musical... more
We discuss the application of the Zipf–Mandelbrot law to musical pieces encoded in MIDI. Specifically, we have identified a set of metrics on which to apply this law. These metrics include pitch of musical events, duration of musical events, the combination of pitch and duration of musical events, and several others. Our hypothesis is that these metrics will allow us to computationally identify and perhaps emphasize aesthetic aspects of music. We have developed a system that automates calculation of these metrics in MIDIencoded pieces. Using this tool we conducted a study on a corpus of 220 pieces from baroque, classical, romantic, 12tone, jazz, rock, DNA strings, and random music. Our results support the above hypothesis. We discuss limitations, and how to minimize the potential for statistical error through composite metrics. We present possible applications and an overview of preliminary work in computer–aided music composition.
Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references. ... WN Locke and AD Booth, eds.,... more
Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references. ... WN Locke and AD Booth, eds., <i>Machine Translation of Languages,</i> Technology Press ...
The connection between aesthetics and numbers dates back to pre-Socratic times. Pythagoras, Plato, and Aristotle worked on quantitative expressions of proportion and beauty such as the golden ratio. Pythagoreans, for instance,... more
The connection between aesthetics and numbers dates back to pre-Socratic times. Pythagoras, Plato, and Aristotle worked on quantitative expressions of proportion and beauty such as the golden ratio. Pythagoreans, for instance, quantified" harmonious" ...
... Nikolaos G. Bourbalus+* and Bill Manark ... 1. D.Na.rasimhalu and S.Christodoulakis, Multimedia Information Systems: the unfolding of a reality, IEEE Computer, 24,10,1991 2. V.Gudivada & V.Raghavan,Content based image... more
... Nikolaos G. Bourbalus+* and Bill Manark ... 1. D.Na.rasimhalu and S.Christodoulakis, Multimedia Information Systems: the unfolding of a reality, IEEE Computer, 24,10,1991 2. V.Gudivada & V.Raghavan,Content based image retrieval systems,IEEE Computer,28,9, 1995 3. L.O ...
Research Interests:
ABSTRACT In the twenty-first century, computers have become indispensable in music making, distribution, performance, and consumption. Making Music with Computers: Creative Programming in Python introduces important concepts and skills... more
ABSTRACT In the twenty-first century, computers have become indispensable in music making, distribution, performance, and consumption. Making Music with Computers: Creative Programming in Python introduces important concepts and skills necessary to generate music with computers. It interweaves computing pedagogy with musical concepts and creative activities, showing students how to integrate the creativity and design of the arts with the mathematical rigor and formality of computer science. The book provides an introduction to creative software development in the Python programming language. It uses innovative music-creation activities to illustrate introductory computer programming concepts, including data types, algorithms, operators, iteration, lists, functions, and classes. The authors also cover GUIs, event-driven programming, big data, MIDI programming, client–server programming, recursion, fractals, and complex system dynamics.

And 181 more