research-article

Mixtape: Using Real-Time User Feedback to Navigate Large Media Collections

Authors:

Luciana Fujii Pontello,

Pedro H. F. Holanda,

Bruno Guilherme,

João Paulo V. Cardoso,

Olga Goussevskaia,

Ana Paula Couto Da SilvaAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 13, Issue 4

Article No.: 50, Pages 1 - 22

https://doi.org/10.1145/3105969

Published: 01 August 2017 Publication History

Abstract

In this work, we explore the increasing demand for novel user interfaces to navigate large media collections. We implement a geometric data structure to store and retrieve item-to-item similarity information and propose a novel navigation framework that uses vector operations and real-time user feedback to direct the outcome. The framework is scalable to large media collections and is suitable for computationally constrained devices. In particular, we implement this framework in the domain of music. To evaluate the effectiveness of the navigation process, we propose an automatic evaluation framework, based on synthetic user profiles, which allows us to quickly simulate and compare navigation paths using different algorithms and datasets. Moreover, we perform a real user study. To do that, we developed and launched Mixtape, a simple web application that allows users to create playlists by providing real-time feedback through liking and skipping patterns.

References

[1]

Amr Ahmed, Nino Shervashidze, Shravan Narayanamurthy, Vanja Josifovski, and Alexander J. Smola. 2013. Distributed large-scale natural graph factorization. In Proceedings of the 22nd International Conference on World Wide Web (WWW’13). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 37--48.

Digital Library

[2]

Kristina Andersen and Peter Knees. 2016. Conversations with expert users in music retrieval and research challenges for creative MIR. In Proceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR’16), ISMIR, 122--128. https://wp.nyu.edu/ismir2016/wp-content/uploads/sites/2294/2016/07/246_Paper.pdf.

[3]

J.-J. Aucouturier, François Pachet, and Mark Sandler. 2005. “The way it Sounds”: Timbre models for analysis and retrieval of music signals. IEEE Trans. Multimed. 7, 6 (2005), 1028--1035.

Digital Library

[4]

Mikhail Belkin and Partha Niyogi. 2003. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 6 (June 2003), 1373--1396.

Digital Library

[5]

Pedro Cano, Markus Koppenberger, and Nicolas Wack. 2005. Content-based music audio recommendation. In Proceedings of the 13th Annual ACM International Conference on Multimedia. ACM, 211--212.

Digital Library

[6]

João Paulo V. Cardoso, Luciana Fujii Pontello, Pedro H. F. Holanda, Bruno Guilherme, Olga Goussevskaia, and Ana Paula Couto da Silva. 2016. Mixtape: Direction-based navigation in large media collections. In Proceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR’16). ISMIR, 454--460.

[7]

Shuo Chen, Josh L. Moore, Douglas Turnbull, and Thorsten Joachims. 2012. Playlist prediction via metric embedding. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’12). ACM, New York, NY, 714--722.

Digital Library

[8]

Shuo Chen, Jiexun Xu, and Thorsten Joachims. 2013. Multi-space probabilistic sequence modeling. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’13). ACM, New York, NY, 865--873.

Digital Library

[9]

Chia-Hao Chung, Jing-Kai Lou, and Homer Chen. 2016. A latent representation of users, sessions, and songs for listening behavior analysis. In Proceedings of the 17th International Society for Music Information Retrieval Conference (ISMIR’16). ISMIR, 323--329.

[10]

Trevor F. Cox and M. A. A. Cox. 2000. Multidimensional Scaling. Chapman and Hall/CRC.

[11]

Vin De Silva and Joshua B. Tenenbaum. 2004. Sparse Multidimensional Scaling Using Landmark Points. Technical Report. Technical report, Stanford University.

[12]

Katayoun Farrahi, Markus Schedl, Andreu Vall, David Hauger, and Marko Tkalcic. 2014. Impact of listening behavior on music recommendation. In Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR’14), 483--488.

[13]

Benjamin Fields, Christophe Rhodes, Michael A. Casey, and Kurt Jacobson. 2008. Social playlists and bottleneck measurements: Exploiting musician social graphs using content-based dissimilarity and pairwise maximum flow values. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR’08), Juan Pablo Bello, Elaine Chew, and Douglas Turnbull (Eds.). ISMIR, 559--564.

[14]

Arthur Flexer. 2014. On inter-rater agreement in audio music similarity. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR’14). ISMIR.

[15]

Olga Goussevskaia, Michael Kuhn, Michael Lorenzi, and Roger Wattenhofer. 2008. From web to map: Exploring the world of music. In Proceedings of the Conference on Web Intelligence and Intelligent Agent Technology (WIIAT’08), Vol. 1.

Digital Library

[16]

Pedro Holanda, Bruno Guilherme, Joo Paulo Cardoso, Ana Paula Couto da Silva, and Olga Goussevskaia. 2015a. Mapeando o universo da midia usando dados gerados por usuarios em redes sociais online. In Proceedings of the Brazilian Symposium on Computer Networks and Distributed Systems (SBRC’15). 1--12.

[17]

Pedro Holanda, Bruno Guilherme, Ana Paula Couto da Silva, and Olga Goussevskaia. 2015b. TV goes social: Characterizing user interaction in an online social network for TV fans. In Proceedings of the 15th International Conference on Engineering the Web in the Big Data Era (ICWE’15), 182--199.

Digital Library

[18]

Ioannis Konstas, Vassilios Stathopoulos, and Joemon M Jose. 2009. On social networks and collaborative recommendation. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 195--202.

Digital Library

[19]

Michael Kuhn, Roger Wattenhofer, and Samuel Welten. 2010. Social audio features for advanced music retrieval interfaces. In Proceedings of the International Conference on Multimedia. ACM, 411--420.

Digital Library

[20]

Quoc V. Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In Proceedings of the 31th International Conference on Machine Learning (ICML’14), 1188--1196.

Digital Library

[21]

Beth Logan. 2002. Content-based playlist generation: Exploratory experiments. In Proceedings of 3rd International Conference on Music Information Retrieval.

[22]

Beth Logan and Ariel Salomon. 2001. A music similarity function based on signal analysis. Proceedings of the 2001 IEEE International Conference on Multimedia and Expo, 190.

[23]

Franois Maillet, Douglas Eck, Guillaume Desjardins, and Paul Lamere. 2009. Steerable playlist generation by learning song similarity from radio station playlists. In Proceedings of the 10th International Society for Music Information Retrieval Conference. 345--350.

[24]

Brian McFee, Luke Barrington, and Gert Lanckriet. 2012. Learning content similarity for music recommendation. IEEE Trans. Aud. Speech Lang. Process. 20, 8 (2012), 2207--2218.

Digital Library

[25]

Joshua L Moore, Shuo Chen, Thorsten Joachims, and Douglas Turnbull. 2012. Learning to embed songs and tags for playlist prediction. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR’12). 349--354.

[26]

Joshua L. Moore, Shuo Chen, Douglas Turnbull, and Thorsten Joachims. 2013. Taste over time: The temporal dynamics of user preferences. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR’13), Alceu de Souza Britto Jr., Fabien Gouyon, and Simon Dixon (Eds.). 401--406.

[27]

Elias Pampalk, T. Pohle, and G. Widmer. 2005. Dynamic playlist generation based on skipping behavior. In Proceedings of the International Symposium on Music Information Retrieval.

[28]

Maria Panteli, Emmanouil Benetos, and Simon Dixon. 2016. Learning a feature space for similarity in world music. In Proceedings of the 17th International Society for Music Information Retrieval Conference, (ISMIR’16). 538--544.

[29]

Minsu Park, Ingmar Weber, Mor Naaman, and Sarah Vieweg. 2015. Understanding musical diversity via online social media. In Proceedings of the 9th International AAAI Conference on Weblogs and Social Media (ICWSM’15). 1--10.

[30]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 701--710.

Digital Library

[31]

Andriy Shepitsen, Jonathan Gemmell, Bamshad Mobasher, and Robin Burke. 2008. Personalized recommendation in social tagging systems using hierarchical clustering. In Proceedings of the 2008 ACM Conference on Recommender Systems. ACM, 259--266.

Digital Library

[32]

Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. LINE: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web. 1067--1077.

Digital Library

[33]

J. B. Tenenbaum, V. Silva, and J. C. Langford. 2000. A global geometric framework for nonlinear dimensionality reduction. Science 290, 5500 (2000), 2319--2323.

[34]

Douglas R. Turnbull, Justin A. Zupnick, Kristofer B. Stensland, Andrew R. Horwitz, Alexander J. Wolf, Alexander E. Spirgel, Stephen P. Meyerhofer, and Thorsten Joachims. 2014. Using personalized radio to enhance local music discovery. In Proceedings of the Conference on Extended Abstracts on Human Factors in Computing Systems (CHIEA’14). ACM, New York, NY, 2023--2028.

Digital Library

[35]

Aaron Van den Oord, Sander Dieleman, and Benjamin Schrauwen. 2013. Deep content-based music recommendation. In Proceedings of the Conference on Advances in Neural Information Processing Systems.

[36]

Laurens J. P. Van der Maaten, Eric O. Postma, and H. Jaap van den Herik. 2009. Dimensionality reduction: A comparative review. J. Mach. Learn. Res. 10, 1--41 (2009), 66--71.

Cited By

Gabbolini GBridge D(2024)Surveying More Than Two Decades of Music Information Retrieval Research on PlaylistsACM Transactions on Intelligent Systems and Technology10.1145/368839815:6(1-68)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3688398
Almeida MVieira CMelo PAssunção R(2019)Random Playlists Smoothly Commuting Between StylesACM Transactions on Multimedia Computing, Communications, and Applications10.1145/336174215:4(1-20)Online publication date: 16-Dec-2019
https://dl.acm.org/doi/10.1145/3361742
Kamehkhosh IBonnin GJannach D(2019)Effects of recommendations on the playlist creation behavior of usersUser Modeling and User-Adapted Interaction10.1007/s11257-019-09237-4Online publication date: 22-May-2019
https://doi.org/10.1007/s11257-019-09237-4

Index Terms

Mixtape: Using Real-Time User Feedback to Navigate Large Media Collections
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object recognition

Recommendations

Musical fingerprints: collaboration around home media collections
GROUP '09: Proceedings of the 2009 ACM International Conference on Supporting Group Work

As people collect more and more digital music, photos, and video, the growing scale of the collection challenges how families share and collaborate around home media collections. We studied the intersection between physical and digital media ...
Vision-Based Humanoid Robot Navigation in a Featureless Environment
MCPR 2015: Proceedings of the 7th Mexican Conference on Pattern Recognition - Volume 9116

One of the most basic tasks for any autonomous mobile robot is that of safely navigating from one point to another e.g. service robots should be able to find their way in different kinds of environments. Typically, vision is used to find landmarks in ...
Content-boosted matrix factorization for recommender systems: experiments with recipe recommendation
RecSys '11: Proceedings of the fifth ACM conference on Recommender systems

The Netflix prize has rejuvenated a widespread interest in the matrix factorization approach for collaborative filtering. We describe a simple algorithm for incorporating content information directly into this approach. We present experimental evidence ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 13, Issue 4

November 2017

362 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3129737

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 August 2017

Accepted: 01 June 2017

Revised: 01 May 2017

Received: 01 October 2016

Published in TOMM Volume 13, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Brazilian agencies CNPq, CAPES and Fapemig

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
163
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gabbolini GBridge D(2024)Surveying More Than Two Decades of Music Information Retrieval Research on PlaylistsACM Transactions on Intelligent Systems and Technology10.1145/368839815:6(1-68)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3688398
Almeida MVieira CMelo PAssunção R(2019)Random Playlists Smoothly Commuting Between StylesACM Transactions on Multimedia Computing, Communications, and Applications10.1145/336174215:4(1-20)Online publication date: 16-Dec-2019
https://dl.acm.org/doi/10.1145/3361742
Kamehkhosh IBonnin GJannach D(2019)Effects of recommendations on the playlist creation behavior of usersUser Modeling and User-Adapted Interaction10.1007/s11257-019-09237-4Online publication date: 22-May-2019
https://doi.org/10.1007/s11257-019-09237-4

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents