Online Symbolic Music Alignment With Offline Reinforcement Learning
Creators
Description
Symbolic Music Alignment is the process of matching performed MIDI notes to corresponding score notes. In this paper, we introduce a reinforcement learning (RL)- based online symbolic music alignment technique. The RL agent — an attention-based neural network — itera- tively estimates the current score position from local score and performance contexts. For this symbolic alignment task, environment states can be sampled exhaustively and the reward is dense, rendering a formulation as a simpli- fied offline RL problem straightforward. We evaluate the trained agent in three ways. First, in its capacity to identify correct score positions for sampled test contexts; second, as the core technique of a complete algorithm for symbolic online note-wise alignment; and finally, as a real-time sym- bolic score follower. We further investigate the pitch-based score and performance representations used as the agent's inputs. To this end, we develop a second model, a two- step Dynamic Time Warping (DTW)-based offline align- ment algorithm leveraging the same input representation. The proposed model outperforms a state-of-the-art refer- ence model of offline symbolic music alignment.
Files
000075.pdf
Files
(591.6 kB)
Name | Size | Download all |
---|---|---|
md5:c68c48470b58c7d3d5fc5296a4414ca5
|
591.6 kB | Preview Download |