poster

Multi-space probabilistic sequence modeling

Authors:

Thorsten JoachimsAuthors Info & Claims

KDD '13: Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 865 - 873

https://doi.org/10.1145/2487575.2487632

Published: 11 August 2013 Publication History

Abstract

Learning algorithms that embed objects into Euclidean space have become the methods of choice for a wide range of problems, ranging from recommendation and image search to playlist prediction and language modeling. Probabilistic embedding methods provide elegant approaches to these problems, but can be expensive to train and store as a large monolithic model. In this paper, we propose a method that trains not one monolithic model, but multiple local embeddings for a class of pairwise conditional models especially suited for sequence and co-occurrence modeling. We show that computation and memory for training these multi-space models can be efficiently parallelized over many nodes of a cluster. Focusing on sequence modeling for music playlists, we show that the method substantially speeds up training while maintaining high model quality.

References

[1]

Y. Bengio, H. Schwenk, J.-S. Senécal, F. Morin, and J.-L. Gauvain. Neural probabilistic language models. Innovations in Machine Learning, pages 137--186, 2006.

[2]

O. Dekel, R. Gilad-Bachrach, O. Shamir, and L. Xiao. Optimal distributed online prediction using mini-batches. The Journal of Machine Learning Research (JMLR), 13:165--202, 2012.

Digital Library

[3]

E. Gabriel, G. E. Fagg, G. Bosilca, T. Angskun, J. J. Dongarra, J. M. Squyres, V. Sahay, P. Kambadur, B. Barrett, A. Lumsdaine, R. H. Castain, D. J. Daniel, R. L. Graham, and T. S. Woodall. Open MPI: Goals, concept, and design of a next generation MPI implementation. In Proceedings, 11th European PVM/MPI Users' Group Meeting, pages 97--104, Budapest, Hungary, September 2004.

[4]

A. Globerson, G. Chechik, F. Pereira, et al. Euclidean embedding of co-occurrence data. Journal of Machine Learning Research (JMLR), 2007.

Digital Library

[5]

G. Hinton and S. Roweis. Stochastic neighbor embedding. Advances in Neural Information Processing Systems (NIPS), 15:833--840, 2002.

[6]

E. H. Huang, R. Socher, C. D. Manning, and A. Y. Ng. Improving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL): Long Papers-Volume 1, pages 873--882. Association for Computational Linguistics, 2012.

Digital Library

[7]

G. Karypis and V. Kumar. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on Scientific Computing, 20(1):359--392, 1998.

Digital Library

[8]

M. Khoshneshin and W. N. Street. Collaborative filtering via euclidean embedding. In Proceedings of the fourth ACM conference on Recommender systems (RecSys), pages 87--94. ACM, 2010.

Digital Library

[9]

J. Kleinberg and E. Tardos. Algorithm design. Pearson Education India, 2006.

Digital Library

[10]

Y. Maron, M. Lamar, and E. Bienenstock. Sphere embedding: An application to part-of-speech induction. In Neural Information Processing Systems Conference (NIPS), 2010.

[11]

A. Mnih and G. Hinton. Three new graphical models for statistical language modelling. In Proceedings of the 24th International Conference on Machine learning (ICML), pages 641--648. ACM, 2007.

Digital Library

[12]

J. Moore, Shuo Chen, T. Joachims, and D. Turnbull. Learning to embed songs and tags for playlist prediction. In International Conference on Music Information Retrieval (ISMIR), pages 349--354, 2012.

[13]

A. Ng, M. Jordan, Y. Weiss, et al. On spectral clustering: Analysis and an algorithm. Advances in Neural Information Processing Systems (NIPS), 2:849--856, 2002.

[14]

F. Niu, B. Recht, C. Ré, and S. J. Wright. Hogwild!: A lock-free approach to parallelizing stochastic gradient descent. arXiv preprint arXiv:1106.5730, 2011.

[15]

S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme. BPR: Bayesian personalized ranking from implicit feedback. In J. Bilmes and A. Y. Ng, editors, Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (UAI), pages 452--461. AUAI Press, 2009.

Digital Library

[16]

S. Rendle, C. Freudenthaler, and L. Schmidt-Thieme. Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the 19th international conference on World Wide Web (WWW), pages 811--820. ACM, 2010.

Digital Library

[17]

S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323--2326, 2000.

[18]

Shuo Chen, J. Moore, D. Turnbull, and T. Joachims. Playlist prediction via metric embedding. In Proceedings of the 18th ACM SIGKDD international conference on knowledge discovery and data mining (KDD), pages 714--722. ACM, 2012.

Digital Library

[19]

J. B. Tenenbaum, V. De Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319--2323, 2000.

[20]

K. Q. Weinberger, B. D. Packer, and L. K. Saul. Nonlinear dimensionality reduction by semidefinite programming and kernel matrix factorization. In Proceedings of the tenth international workshop on artificial intelligence and statistics, pages 381--388, 2005.

[21]

D. Zhou, S. Zhu, K. Yu, X. Song, B. L. Tseng, H. Zha, and C. L. Giles. Learning multiple graphs for document recommendations. In Proceedings of the 17th international conference on World Wide Web (WWW), pages 141--150. ACM, 2008.

Digital Library

[22]

M. Zinkevich, M. Weimer, A. Smola, and L. Li. Parallelized stochastic gradient descent. Advances in Neural Information Processing Systems (NIPS), 23(23):1--9, 2010.

Cited By

Zhang LWang PLi JXiao ZShi H(2021)Attentive Hybrid Recurrent Neural Networks for sequential recommendationNeural Computing and Applications10.1007/s00521-020-05643-7Online publication date: 4-Jan-2021
https://doi.org/10.1007/s00521-020-05643-7
Quadrana MCremonesi PJannach D(2018)Sequence-Aware Recommender SystemsACM Computing Surveys10.1145/319061651:4(1-36)Online publication date: 6-Jul-2018
https://dl.acm.org/doi/10.1145/3190616
Schedl MZamani HChen CDeldjoo YElahi M(2018)Current challenges and visions in music recommender systems researchInternational Journal of Multimedia Information Retrieval10.1007/s13735-018-0154-27:2(95-116)Online publication date: 5-Apr-2018
https://doi.org/10.1007/s13735-018-0154-2
Show More Cited By

Index Terms

Multi-space probabilistic sequence modeling
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches

Recommendations

Playlist prediction via metric embedding
KDD '12: Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining

Digital storage of personal music collections and cloud-based music services (e.g. Pandora, Spotify) have fundamentally changed how music is consumed. In particular, automatically generated playlists have become an important mode of accessing large ...
A multi-level parallelization concept for high-fidelity multi-block solvers
SC '97: Proceedings of the 1997 ACM/IEEE conference on Supercomputing

The integration of high-fidelity Computational Fluid Dynamics (CFD) analysis tools with the industrial design process benefits greatly from the robust implementations that are transportable across a wide range of computer architectures. In the present ...
A New Reversible and high capacity data hiding technique for E-healthcare applications

A high capacity and reversible data hiding technique capable of tamper detection and localisation of medical images has been proposed in this paper. Image interpolation has been used to scale up the original image to obtain the cover image. The cover ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '13: Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining

August 2013

1534 pages

ISBN:9781450321747

DOI:10.1145/2487575

Editors:
Rayid Ghani
University of Chicago
,
Ted E. Senator
SAIC
,
Paul Bradley
MethodCare, Inc.
,
Rajesh Parekh
Groupon
,
Jingrui He
Stevens Institute of Technology
,
General Chairs:
Robert L. Grossman
University of Chicago and Open Data Group
,
Ramasamy Uthurusamy
General Motors Corporation (retired)
,
Program Chairs:
Inderjit S. Dhillon
University of Texas
,
Yehuda Koren
Google

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 August 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

KDD' 13

Sponsor:

KDD' 13: The 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 11 - 14, 2013

Illinois, Chicago, USA

Acceptance Rates

KDD '13 Paper Acceptance Rate 125 of 726 submissions, 17%;

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
418
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang LWang PLi JXiao ZShi H(2021)Attentive Hybrid Recurrent Neural Networks for sequential recommendationNeural Computing and Applications10.1007/s00521-020-05643-7Online publication date: 4-Jan-2021
https://doi.org/10.1007/s00521-020-05643-7
Quadrana MCremonesi PJannach D(2018)Sequence-Aware Recommender SystemsACM Computing Surveys10.1145/319061651:4(1-36)Online publication date: 6-Jul-2018
https://dl.acm.org/doi/10.1145/3190616
Schedl MZamani HChen CDeldjoo YElahi M(2018)Current challenges and visions in music recommender systems researchInternational Journal of Multimedia Information Retrieval10.1007/s13735-018-0154-27:2(95-116)Online publication date: 5-Apr-2018
https://doi.org/10.1007/s13735-018-0154-2
Ludewig MJannach D(2018)Evaluation of session-based recommendation algorithmsUser Modeling and User-Adapted Interaction10.1007/s11257-018-9209-628:4-5(331-390)Online publication date: 1-Dec-2018
https://dl.acm.org/doi/10.1007/s11257-018-9209-6
Silva Gde Oliveira Lde Medeiros RGoussevskaia OBenevenuto FSheth ANgonga AWang yChang EŚlęzak DFranczyk BAlt RTao X(2017)Characterizing internet radio stations at scaleProceedings of the International Conference on Web Intelligence10.1145/3106426.3106540(670-677)Online publication date: 23-Aug-2017
https://dl.acm.org/doi/10.1145/3106426.3106540
Pontello LHolanda PGuilherme BCardoso JGoussevskaia OSilva A(2017)MixtapeACM Transactions on Multimedia Computing, Communications, and Applications10.1145/310596913:4(1-22)Online publication date: 1-Aug-2017
https://dl.acm.org/doi/10.1145/3105969
Wu TGleich DMatwin SYu SFarooq F(2017)Retrospective Higher-Order Markov Processes for User TrailsProceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining10.1145/3097983.3098127(1185-1194)Online publication date: 13-Aug-2017
https://dl.acm.org/doi/10.1145/3097983.3098127
Yu WLi LXu XWang DWang JChen S(2017)ProductRec: Product Bundle Recommendation Based on User's Sequential Patterns in Social Networking Service Environment2017 IEEE International Conference on Web Services (ICWS)10.1109/ICWS.2017.127(301-308)Online publication date: Jun-2017
https://doi.org/10.1109/ICWS.2017.127
Yu WLi LLi FZhang JHu F(2017)Exploring Latent Bundles from Social Behaviors for Personalized RankingKnowledge Science, Engineering and Management10.1007/978-3-319-63558-3_31(371-379)Online publication date: 19-Jul-2017
https://doi.org/10.1007/978-3-319-63558-3_31
Zhenjiang DJia DXiaohui JYongli W(2017)RTMatch: Real-Time Location Prediction Based on Trajectory Pattern MatchingDatabase Systems for Advanced Applications10.1007/978-3-319-55705-2_8(103-117)Online publication date: 22-Mar-2017
https://doi.org/10.1007/978-3-319-55705-2_8
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten