Abstract
We propose a novel approach for humming transcription that combines a CNN-based architecture with a dynamic programming-based post-processing algorithm, utilizing the recently introduced HumTrans dataset. We identify and address inherent problems with the offset and onset ground truth provided by the dataset, offering heuristics to improve these annotations, resulting in a dataset with precise annotations that will aid future research. Additionally, we compare the transcription accuracy of our method against several others, demonstrating state-of-the-art (SOTA) results. All our code and corrected dataset is available at https://github.com/shubham-gupta-30/humming_transcription.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Balhar, J., Hajic, J., Jr.: Melody extraction using a harmonic harmonic convolutional neural network. Dimensions 10(16x10x540), 16x10x360
Bittner, R.M., Bosch, J.J., Rubinstein, D., Meseguer-Brocal, G., Ewert, S.: A lightweight instrument-agnostic model for polyphonic note transcription and multipitch estimation. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Singapore (2022)
Bittner, R.M., Bosch, J.J., Rubinstein, D., Meseguer-Brocal, G., Ewert, S.: A lightweight instrument-agnostic model for polyphonic note transcription and multipitch estimation. In: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 781–785. IEEE (2022)
Böck, S., Krebs, F., Schedl, M.: Evaluating the online capabilities of onset detection methods. In: ISMIR, pp. 49–54 (2012)
Donahue, C., Thickstun, J., Liang, P.: Melody transcription via generative pre-training. In: ISMIR (2022)
Google: Song stuck in your head? just hum to search. https://blog.google/products/search/hum-to-search/. Accessed 12 Dec 2023
Hsu, J.Y., Su, L.: VOCANO: a note transcription framework for singing voice in polyphonic music. In: Proceedings of the International Society of Music Information Retrieval Conference (ISMIR) (2021)
Liu, S., Li, X., Li, D., Shan, Y.: HumTrans: a novel open-source dataset for humming melody transcription and beyond. arXiv preprint arXiv:2309.09623 (2023)
Raffel, C., et al.: Mir_eval: a transparent implementation of common MIR metrics. In: ISMIR, vol. 10, p. 2014 (2014)
Kum, S., Lee, J., Kim, K.L., Kim, T., Nam, J.: Pseudo-label transfer from frame-level to note-level in a teacher-student framework for singing transcription from polyphonic music. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2022)
Shansongliu: Humtrans. https://github.com/shansongliu/HumTrans. Accessed 12 Dec 2023
Wang, J.Y., Jang, J.S.R.: On the preparation and validation of a large-scale dataset of singing transcription. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 276–280. IEEE (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendices
A Heuristic Algorithm for Better Ground Truth Annotations
We reproduce the python code used to create the waveform envelopes as mentioned in Sect. 2.1. We want our waveform envelops to be as close to the original general waveform shape as possible and thus we utilize a simple heuristic algorithm that locally computes the maximum of the waveform preceding a point of interest and the maximum of the waveform following a point of interest. We found we get a very tight hugging envelope when we take a min of these two values.
We found that the envelope calculated using the above method could still be improved if we calculated the envelope of the envelope again. Having now obtained a tight envelope of the waveform, we now calculate the threshold to use for this waveform to measure the onset and offset boundaries. We further clean these onsets and offsets by disregarding any silences that are too small (the method adjust_onsets_offsets in the code below). Finally, we check if only consider this waveform for training or testing purposes if we get the right number of notes through this heuristic, otherwise we disregard this sample altogether. The code to do this is reproduced below.
B Dynamic Programming Postprocessing
The code for computing a path using dynamic programming as detailed in Sect. 2.4 is reproduced below. Note that the method clean_path simply performs a heuristic cleaning on the paths discovered by the dynamic programming solution so that they are more meaningful and make sense.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gupta, S., Gomez-Sarmiento, I.N., Mezdari, F.A., Ravanelli, M., Subakan, C. (2024). Dynamic HumTrans: Humming Transcription Using CNNs and Dynamic Programming. In: Suen, C.Y., Krzyzak, A., Ravanelli, M., Trentin, E., Subakan, C., Nobile, N. (eds) Artificial Neural Networks in Pattern Recognition. ANNPR 2024. Lecture Notes in Computer Science(), vol 15154. Springer, Cham. https://doi.org/10.1007/978-3-031-71602-7_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-71602-7_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-71601-0
Online ISBN: 978-3-031-71602-7
eBook Packages: Computer ScienceComputer Science (R0)
