research-article

Recommending the Video to Watch Next: An Offline and Online Evaluation at YOUTV.de

Authors:

Panagiotis Symeonidis,

Dmitry Chaltsev,

Philip Giuliani,

Daniel Morandini,

Andreas Unterhuber,

Markus ZankerAuthors Info & Claims

RecSys '20: Proceedings of the 14th ACM Conference on Recommender Systems

Pages 299 - 308

https://doi.org/10.1145/3383313.3412257

Published: 22 September 2020 Publication History

Abstract

The task “recommend a video to watch next?” has been in the focus of recommender systems’ research for a long time. However, adequately exploiting the clues hidden in the sequences of actions of user sessions in order to reveal users’ short-term intentions moved only recently into the focus of research. Based on a real-world application scenario, in this paper, we propose a Markov Chain-based transition probability matrix to efficiently reveal the short-term preferences of individuals. We experimentally evaluated our proposed method by comparing it against state-of-the-art algorithms in an offline as well as a live evaluation setting. In both cases our method not only demonstrated its superiority over its competitors, but exposed a clearly stronger engagement of users on the platform. In the online setting, our method improved the click-through rate by up to 93.61%. This paper therefore contributes real-world evidence for improving the recommendation effectiveness, by considering sequence-awareness, since capturing the short-term preferences of users is crucial in the light of items with a short life span such as tv programs (news, tv shows, etc.).

References

[1]

Shuo Chen, Joshua L. Moore, Douglas Turnbull, and Thorsten Joachims. 2012. Playlist prediction via metric embedding. In The 18th ACM International Conference on Knowledge Discovery and Data Mining, KDD’12, Beijing, China. ACM, 714–722. https://doi.org/10.1145/2339530.2339643

Digital Library

[2]

Abhinandan S Das, Mayur Datar, Ashutosh Garg, and Shyam Rajaram. 2007. Google news personalization: scalable online collaborative filtering. In Proceedings of the 16th international conference on World Wide Web. ACM, 271–280.

Digital Library

[3]

Cagdas Esiyok, Benjamin Kille, Brijnesh Johannes Jain, Frank Hopfgartner, and Sahin Albayrak. 2014. Users’ reading habits in online news portals. In Fifth Information Interaction in Context Symposium, IIiX ’14, Regensburg, Germany. ACM, 263–266. https://doi.org/10.1145/2637002.2637038

Digital Library

[4]

Florent Garcin, Christos Dimitrakakis, and Boi Faltings. 2013. Personalized news recommendation with context trees. In Proceedings of the 7th ACM conference on Recommender systems, RecSys’13, Hong Kong, China. ACM, 105–112. https://doi.org/10.1145/2507157.2507166

Digital Library

[5]

Florent Garcin, Boi Faltings, Olivier Donatsch, Ayar Alazzawi, Christophe Bruttin, and Amr Huber. 2014. Offline and online evaluation of news recommender systems at swissinfo. ch. In Proceedings of the 8th ACM Conference on Recommender systems. ACM, 169–176.

Digital Library

[6]

Carlos A. Gomez-Uribe and Neil Hunt. 2015. The Netflix Recommender System: Algorithms, Business Value, and Innovation. ACM Trans. Manage. Inf. Syst. 6, 4, Article 13 (Dec. 2015), 19 pages. https://doi.org/10.1145/2843948

Digital Library

[7]

Qi He, Daxin Jiang, Zhen Liao, Steven C. H. Hoi, Kuiyu Chang, Ee-Peng Lim, and Hang Li. 2009. Web Query Recommendation via Sequential Query Prediction. In Proceedings of the 25th International Conference on Data Engineering, ICDE’09, Shanghai, China. IEEE, 1443–1454. https://doi.org/10.1109/ICDE.2009.71

Digital Library

[8]

Balázs Hidasi and Alexandros Karatzoglou. 2017. Recurrent Neural Networks with Top-k Gains for Session-based Recommendations. Proceedings of the 27th ACM International Conferernce on Information and Knowledge Management (2017).

[9]

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based Recommendations with Recurrent Neural Networks. In Proceedings of the International Conference on Learning Representations (ICLR 2016), Vol. abs/1511.06939. arxiv:1511.06939http://arxiv.org/abs/1511.06939

[10]

Balázs Hidasi, Massimo Quadrana, Alexandros Karatzoglou, and Domonkos Tikk. 2016. Parallel Recurrent Neural Network Architectures for Feature-rich Session-based Recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems(RecSys ’16). ACM, New York, NY, USA, 241–248. https://doi.org/10.1145/2959100.2959167

Digital Library

[11]

Dietmar Jannach and Michael Jugovac. 2019. Measuring the business value of recommender systems. ACM Transactions on Management Information Systems (TMIS) 10, 4(2019), 1–23.

Digital Library

[12]

Dietmar Jannach, Lukas Lerche, and Michael Jugovac. 2017. Adaptation and Evaluation of Recommendations for Short-term Shopping Goals. In Proceedings of the ninth ACM Conference on Recommender Systems(RecSys ’15). ACM, New York, NY, USA.

[13]

D. Jannach and M. Ludewig. 2017. When Recurrent Neural Networks Meet the Neighborhood for Session-Based Recommendation. In Proceedings of the Eleventh ACM Conference on Recommender Systems(RecSys ’17). ACM, New York, NY, USA, 306–310. http://doi.acm.org/10.1145/3109859.3109872

[14]

Dietmar Jannach and Malte Ludewig. 2017. When recurrent neural networks meet the neighborhood for session-based recommendation. In Proceedings of the Eleventh ACM Conference on Recommender Systems. 306–310.

Digital Library

[15]

Benjamin Kille, Andreas Lommatzsch, Frank Hopfgartner, Martha Larson, and Arjen P de Vries. 2017. A stream-based resource for multi-dimensional evaluation of recommender algorithms. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1257–1260.

Digital Library

[16]

Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017. Neural attentive session-based recommendation. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 1419–1428.

Digital Library

[17]

Lei Li, Li Zheng, Fan Yang, and Tao Li. 2014. Modeling and broadening temporal user interest in personalized news recommendation. Expert Systems with Applications 41, 7 (2014), 3168 – 3177.

Digital Library

[18]

Jiahui Liu, Peter Dolan, and Elin Rønby Pedersen. 2010. Personalized news recommendation based on click behavior. In Proceedings of the 15th international conference on Intelligent user interfaces. ACM, 31–40.

Digital Library

[19]

Qiao Liu, Yifu Zeng, Refuoe Mokhosi, and Haibin Zhang. 2018. STAMP: short-term attention/memory priority model for session-based recommendation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1831–1839.

Digital Library

[20]

Malte Ludewig and Dietmar Jannach. 2018. Evaluation of session-based recommendation algorithms. User Modeling and User-Adapted Interaction 28, 4-5 (2018), 331–390.

Digital Library

[21]

Brian McFee and Gert R. G. Lanckriet. 2011. The Natural Language of Playlists. In Proceedings of the 12th International Society for Music Information Retrieval Conference, ISMIR’11, Miami, Florida, USA. ISMIR, 537–542. http://ismir2011.ismir.net/papers/PS4-11.pdf

[22]

Omar Moling, Linas Baltrunas, and Francesco Ricci. 2012. Optimal radio channel recommendations with explicit and implicit feedback. In Sixth ACM Conference on Recommender Systems, RecSys’12, Dublin, Ireland. ACM, 75–82. https://doi.org/10.1145/2365952.2365971

Digital Library

[23]

Ioannis K. Paparrizos, Berkant Barla Cambazoglu, and Aristides Gionis. 2011. Machine learned job recommendation. In Proceedings of the 2011 ACM Conference on Recommender Systems, RecSys’11, Chicago, IL, USA. ACM, 325–328. https://doi.org/10.1145/2043932.2043994

Digital Library

[24]

Massimo Quadrana, Paolo Cremonesi, and Dietmar Jannach. 2018. Sequence-aware recommender systems. ACM Computing Surveys (CSUR) 51, 4 (2018), 1–36.

Digital Library

[25]

Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, and Paolo Cremonesi. 2017. Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks. In Proceedings of the Eleventh ACM Conference on Recommender Systems(RecSys ’17). ACM, New York, NY, USA, 130–137. https://doi.org/10.1145/3109859.3109896

Digital Library

[26]

Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized Markov chains for next-basket recommendation. In Proceedings of the 19th International Conference on World Wide Web, WWW’10, Raleigh, North Carolina, USA. ACM, 811–820. https://doi.org/10.1145/1772690.1772773

Digital Library

[27]

P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom, and J. Riedl. 1994. GroupLens: An open architecture for collaborative filtering on netnews. In Proceedings of the Computer Supported Collaborative Work Conference. 175–186.

[28]

Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based Collaborative Filtering Recommendation Algorithms. In Proceedings of the 10th International Conference on World Wide Web(WWW ’01). ACM, New York, NY, USA, 285–295. https://doi.org/10.1145/371920.372071

Digital Library

[29]

Guy Shani, David Heckerman, and Ronen I. Brafman. 2005. An MDP-Based Recommender System. In Journal of Machine Learning Research (2005), Vol. 6. JMLR, 1265–1295. http://www.jmlr.org/papers/v6/shani05a.html

[30]

Elena Smirnova and Flavian Vasile. 2017. Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks. In Proceedings of the 2Nd Workshop on Deep Learning for Recommender Systems(DLRS 2017). ACM, New York, NY, USA, 2–9. https://doi.org/10.1145/3125486.3125488

Digital Library

[31]

Yong Kiam Tan, Xinxing Xu, and Yong Liu. 2016. Improved Recurrent Neural Networks for Session-based Recommendations. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems(DLRS 2016). ACM, New York, NY, USA, 17–22. https://doi.org/10.1145/2988450.2988452

Digital Library

[32]

J. Vinagre, A.M. Jorge, and J. Gama. 2014. Evaluation of recommender systems in streaming environments. In Workshop on Recommender Systems Evaluation: Dimensions and Design (REDD 2014), held in conjunction with RecSys 2014. Silicon Valley, USA.

[33]

Fajie Yuan, Alexandros Karatzoglou, Ioannis Arapakis, Joemon M Jose, and Xiangnan He. 2019. A simple convolutional generative network for next item recommendation. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining. 582–590.

Digital Library

[34]

Andrew Zimdars, David Maxwell Chickering, and Christopher Meek. 2001. Using Temporal Data for Making Recommendations. In Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence(UAI 2001). San Francisco, CA, USA, 580–588.

Digital Library

Cited By

Bauer CZangerle ESaid A(2024)Exploring the Landscape of Recommender Systems Evaluation: Practices and PerspectivesACM Transactions on Recommender Systems10.1145/36291702:1(1-31)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3629170
Chen TLi CMa M(2024)A User-Based Collaborative Filtering Algorithm Considering Interest Attenuation for Short Video Recommendation2024 5th International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT)10.1109/AINIT61980.2024.10581527(677-680)Online publication date: 29-Mar-2024
https://doi.org/10.1109/AINIT61980.2024.10581527
Liu YAchananuparp PLim E(2024)Non-binary evaluation of next-basket food recommendationUser Modeling and User-Adapted Interaction10.1007/s11257-023-09369-834:1(183-227)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s11257-023-09369-8
Show More Cited By

Recommending the Video to Watch Next: An Offline and Online Evaluation at YOUTV.de
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals

Recommendations

Next-item Recommendations in Short Sessions
RecSys '21: Proceedings of the 15th ACM Conference on Recommender Systems

The changing preferences of users towards items trigger the emergence of session-based recommender systems (SBRSs), which aim to model the dynamic preferences of users for next-item recommendations. However, most of the existing studies on SBRSs are ...
Long-tail Session-based Recommendation
RecSys '20: Proceedings of the 14th ACM Conference on Recommender Systems

Session-based recommendation focuses on the prediction of user actions based on anonymous sessions and is a necessary method in the lack of user historical data. However, none of the existing session-based recommendation methods explicitly takes the ...
Recommending new movies: even a few ratings are more valuable than metadata
RecSys '09: Proceedings of the third ACM conference on Recommender systems

The Netflix Prize (NP) competition gave much attention to collaborative filtering (CF) approaches. Matrix factorization (MF) based CF approaches assign low dimensional feature vectors to users and items. We link CF and content-based filtering (CBF) by ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

RecSys '20: Proceedings of the 14th ACM Conference on Recommender Systems

September 2020

796 pages

ISBN:9781450375832

DOI:10.1145/3383313

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 September 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Autonomous Province of Bolzano - South Tyrol

Conference

RecSys '20

Sponsor:

RecSys '20: Fourteenth ACM Conference on Recommender Systems

September 22 - 26, 2020

Virtual Event, Brazil

Acceptance Rates

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Upcoming Conference

RecSys '24

Sponsor:
sigchi

18th ACM Conference on Recommender Systems

October 14 - 18, 2024

Bari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
1,012
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)1

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bauer CZangerle ESaid A(2024)Exploring the Landscape of Recommender Systems Evaluation: Practices and PerspectivesACM Transactions on Recommender Systems10.1145/36291702:1(1-31)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3629170
Chen TLi CMa M(2024)A User-Based Collaborative Filtering Algorithm Considering Interest Attenuation for Short Video Recommendation2024 5th International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT)10.1109/AINIT61980.2024.10581527(677-680)Online publication date: 29-Mar-2024
https://doi.org/10.1109/AINIT61980.2024.10581527
Liu YAchananuparp PLim E(2024)Non-binary evaluation of next-basket food recommendationUser Modeling and User-Adapted Interaction10.1007/s11257-023-09369-834:1(183-227)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s11257-023-09369-8
Su TXi WXing XWang C(2023)MCRec: Multi-channel Gated Gifts Recommendation2023 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM58522.2023.00064(548-557)Online publication date: 1-Dec-2023
https://doi.org/10.1109/ICDM58522.2023.00064
Wu LHe XWang XZhang KWang M(2022)A Survey on Accuracy-oriented Neural Recommendation: From Collaborative Filtering to Information-rich RecommendationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.3145690(1-1)Online publication date: 2022
https://doi.org/10.1109/TKDE.2022.3145690
Bernardis CDacrema MMaurera FQuadrana MScriminaci MCremonesi P(2022)From Data Analysis to Intent-Based Recommendation: An Industrial Case Study in the Video DomainIEEE Access10.1109/ACCESS.2022.314843410(14779-14796)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3148434
Kumar CChowdary C(2022)OPHAencoder: An unsupervised approach to identify groups in group recommendationsComputing10.1007/s00607-022-01103-3104:12(2635-2657)Online publication date: 6-Jul-2022
https://doi.org/10.1007/s00607-022-01103-3
Gharahighehi AVens C(2021)Personalizing Diversity Versus Accuracy in Session-Based Recommender SystemsSN Computer Science10.1007/s42979-020-00399-22:1Online publication date: 15-Jan-2021
https://doi.org/10.1007/s42979-020-00399-2
Vrochidis ADimitriou NKrinidis SPanagiotidis SParcharidis STzovaras D(2021)A Multi-modal Audience Analysis System for Predicting Popularity of Online VideosProceedings of the 22nd Engineering Applications of Neural Networks Conference10.1007/978-3-030-80568-5_38(465-476)Online publication date: 1-Jul-2021
https://doi.org/10.1007/978-3-030-80568-5_38
Jannach DZanker M(2012)Value and Impact of Recommender SystemsRecommender Systems Handbook10.1007/978-1-0716-2197-4_14(519-546)Online publication date: 24-Feb-2012
https://doi.org/10.1007/978-1-0716-2197-4_14

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents