poster

Evaluating the dynamic properties of recommendation algorithms

Author:

Robin BurkeAuthors Info & Claims

RecSys '10: Proceedings of the fourth ACM conference on Recommender systems

Pages 225 - 228

https://doi.org/10.1145/1864708.1864753

Published: 26 September 2010 Publication History

Get Access

Abstract

Collaborative recommendation algorithms are typically evaluated on a static matrix of user rating data. However, when users experience a recommender system, it is dynamic, constantly evolving as new items and new users arrive. The dynamic properties of collaborative recommendation have become important as prediction algorithms based on the interactions of rating histories have been proposed, and as researchers seek to understand problems of robustness and maintenance in rating databases.

This paper proposes a new evaluation method for the dynamic aspects of collaborative algorithms, the "temporal leave-one-out" approach, which can provide insight into both user-specific and system-level evolution of recommendation behavior. As a case study, the methodology is applied to the Influence Limiter algorithm [12], showing that its robustness to attack comes at a high accuracy cost.

References

[1]

]]R. Burke, B. Mobasher, C. Williams, and R. Bhaumik. Detecting profile injection attacks in collaborative recommender systems. In Proc. of the IEEE J. Conf. on E-Commerce Technology and Enterprise Computing (CEC/EEE 2006), Palo Alto, CA, June 2006.

Digital Library

Google Scholar

[2]

]]D. Cosley, S. K. Lam, I. Albert, J. Konstan, and J. Riedl. Is seeing believing? how recommender systems influence users' opinions. In Conf. on Human Factors in Computing Systems (CHI 2003), pages 585--592, 2003.

Digital Library

Google Scholar

[3]

]]J. Herlocker, J. Konstan, A. Borchers, and J. Riedl. An algorithmic framework for performing collaborative filtering. In Proc. of the 22nd ACM Conf. on Research and Development in Information Retrieval (SIGIR'99), Berkeley, CA, August 1999.

Digital Library

Google Scholar

[4]

]]J. Herlocker, J. Konstan, L. G. Tervin, and J. Riedl. Evaluating collaborative filtering recommender systems. ACM Trans. on Information Systems, 22(1):5--53, 2004.

Digital Library

Google Scholar

[5]

]]Y. Koren. Collaborative filtering with temporal dynamics. In KDD '09: Proc. of the 15th ACM Int. Conf. on Knowledge Discovery and Data Mining, pages 447--456, New York, NY, USA, 2009. ACM.

Digital Library

Google Scholar

[6]

]]N. Lathia, S. Hailes, and L. Capra. Temporal collaborative filtering with adaptive neighbourhoods. In SIGIR '09: Proc. of the 32nd Int. ACM Conf. on Research and Development in Information Retrieval, pages 796--797, New York, NY, USA, 2009. ACM.

Digital Library

Google Scholar

[7]

]]Z. Lu, D. Agarwal, and I. S. Dhillon. A spatio-temporal approach to collaborative filtering. In RecSys '09: Proc. of the Third ACM Conf. on Recommender Systems, pages 13--20, New York, NY, USA, 2009. ACM.

Digital Library

Google Scholar

[8]

]]B. Mobasher, R. Burke, R. Bhaumik, and C. Williams. Towards trustworthy recommender systems: An analysis of attack models and algorithm robustness. ACM Trans. on Internet Technology, 7(4), 2007.

Digital Library

Google Scholar

[9]

]]B. K. Mohan, B. J. Keller, and N. Ramakrishnan. Scouts, promoters, and connectors: The roles of ratings in nearest-neighbor collaborative filtering. ACM Trans. Web, 1(2):8, 2007.

Digital Library

Google Scholar

[10]

]]M. O'Mahony, N. Hurley, N. Kushmerick, and G. Silvestre. Collaborative recommendation: A robustness analysis. ACM Trans. on Internet Technology, 4(4):344--377, 2004.

Digital Library

Google Scholar

[11]

]]S.-T. Park and W. Chu. Pairwise preference regression for cold-start recommendation. In RecSys '09: Proc. of the Third ACM conference on Recommender Systems, pages 21--28, New York, NY, USA, 2009. ACM.

Digital Library

Google Scholar

[12]

]]P. Resnick and R. Sami. The influence limiter: provably manipulation-resistant recommender systems. In RecSys '07: Proc. of the 2007 ACM Conf. on Recommender Systems, pages 25--32, Minneapolis, October 2007.

Digital Library

Google Scholar

[13]

]]B. M. Sarwar, G. Karypis, J. Konstan, and J. Riedl. Analysis of recommender algorithms for e-commerce. In Proceedings of the 2nd ACM E-Commerce Conference (EC'00), Minneapolis, MN, October 2000.

Digital Library

Google Scholar

[14]

]]A. I. Schein, A. Popescul, L. H. Ungar, and D. M. Pennock. Methods and metrics for cold-start recommendations. In SIGIR '02: Proc. of the 25th Int. ACM Conf. on Research and Development in Information Retrieval, pages 253--260, New York, NY, USA, 2002. ACM.

Digital Library

Google Scholar

Cited By

View all

Klimashevskaia AJannach DElahi MTrattner C(2024)A survey on popularity bias in recommender systemsUser Modeling and User-Adapted Interaction10.1007/s11257-024-09406-0Online publication date: 1-Jul-2024
https://doi.org/10.1007/s11257-024-09406-0
Ji YSun AZhang JLi C(2023)A Critical Study on Data Leakage in Recommender System Offline EvaluationACM Transactions on Information Systems10.1145/356993041:3(1-27)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1145/3569930
Zangerle EBauer C(2022)Evaluating Recommender Systems: Survey and FrameworkACM Computing Surveys10.1145/355653655:8(1-38)Online publication date: 23-Dec-2022
https://dl.acm.org/doi/10.1145/3556536
Show More Cited By

Index Terms

Evaluating the dynamic properties of recommendation algorithms
1. Computing methodologies
  1. Machine learning
2. Information systems
  1. Information retrieval

Recommendations

A new approach to evaluating novel recommendations
RecSys '08: Proceedings of the 2008 ACM conference on Recommender systems

This paper presents two methods, named Item- and User-centric, to evaluate the quality of novel recommendations. The former method focuses on analyzing the item-based recommendation network. The aim is to detect whether the network topology has any ...
New Recommendation Techniques for Multicriteria Rating Systems

Traditional single-rating recommender systems have been successful in a number of personalization applications, but the research area of multicriteria recommender systems has been largely untouched. Taking full advantage of multicriteria ratings in ...
A framework for diversifying recommendation lists by user interest expansion

Recommender systems have been widely used to discover users' preferences and recommend interesting items to users during this age of information overload. Researchers in the field of recommender systems have realized that the quality of a top-N ...

Comments

Information & Contributors

Information

Published In

RecSys '10: Proceedings of the fourth ACM conference on Recommender systems

September 2010

402 pages

ISBN:9781605589060

DOI:10.1145/1864708

General Chairs:
Xavier Amatriain
Telefonica Research, Spain
,
Marc Torrens
Strands, Spain
,
Program Chairs:
Paul Resnick
University of Michigan, USA
,
Markus Zanker
Alpen-Adria-Universität Klagenfurt, Austria

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 September 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

RecSys '10

Sponsor:

SIGCHI

RecSys '10: Fourth ACM Conference on Recommender Systems

September 26 - 30, 2010

Barcelona, Spain

Acceptance Rates

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Upcoming Conference

RecSys '24

Sponsor:
sigchi

18th ACM Conference on Recommender Systems

October 14 - 18, 2024

Bari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
643
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)2

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Klimashevskaia AJannach DElahi MTrattner C(2024)A survey on popularity bias in recommender systemsUser Modeling and User-Adapted Interaction10.1007/s11257-024-09406-0Online publication date: 1-Jul-2024
https://doi.org/10.1007/s11257-024-09406-0
Ji YSun AZhang JLi C(2023)A Critical Study on Data Leakage in Recommender System Offline EvaluationACM Transactions on Information Systems10.1145/356993041:3(1-27)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1145/3569930
Zangerle EBauer C(2022)Evaluating Recommender Systems: Survey and FrameworkACM Computing Surveys10.1145/355653655:8(1-38)Online publication date: 23-Dec-2022
https://dl.acm.org/doi/10.1145/3556536
Ma BLu MTaniguchi YKonomi S(2021)CourseQ: the impact of visual and interactive course recommendation in university environmentsResearch and Practice in Technology Enhanced Learning10.1186/s41039-021-00167-716:1Online publication date: 30-Jun-2021
https://doi.org/10.1186/s41039-021-00167-7
Jiang WLin FZhang JYang CZhang HCui Z(2021)Dynamic Sequential Recommendation: Decoupling User Intent from Temporal Context2021 International Conference on Data Mining Workshops (ICDMW)10.1109/ICDMW53433.2021.00010(18-26)Online publication date: Dec-2021
https://doi.org/10.1109/ICDMW53433.2021.00010
Gharahighehi AVens CPliakos K(2021)Fair multi-stakeholder news recommender system with hypergraph rankingInformation Processing and Management: an International Journal10.1016/j.ipm.2021.10266358:5Online publication date: 1-Sep-2021
https://dl.acm.org/doi/10.1016/j.ipm.2021.102663
Frigó EKocsis L(2021)Online convex combination of ranking modelsUser Modeling and User-Adapted Interaction10.1007/s11257-021-09306-732:4(649-683)Online publication date: 6-Nov-2021
https://doi.org/10.1007/s11257-021-09306-7
Frigó EKocsis LBogers TSaid ABrusilovsky PTikk D(2019)Online ranking combinationProceedings of the 13th ACM Conference on Recommender Systems10.1145/3298689.3346993(12-19)Online publication date: 10-Sep-2019
https://dl.acm.org/doi/10.1145/3298689.3346993
Afoudi YLazaar MAl Achhab M(2019)Collaborative Filtering Recommender SystemAdvanced Intelligent Systems for Sustainable Development (AI2SD’2018)10.1007/978-3-030-11928-7_30(332-345)Online publication date: 7-Mar-2019
https://doi.org/10.1007/978-3-030-11928-7_30
Park SLee K(2018)A noble approach to effective Recommender System using Graph EmbeddingJournal of Digital Contents Society10.9728/dcs.2018.19.10.191919:10(1919-1925)Online publication date: 31-Oct-2018
https://doi.org/10.9728/dcs.2018.19.10.1919
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A new approach to evaluating novel recommendations

New Recommendation Techniques for Multicriteria Rating Systems

A framework for diversifying recommendation lists by user interest expansion