research-article

Correlating perception-oriented aspects in user-centric recommender system evaluation

Authors:

Alan Said,

Brijnesh J. Jain,

Andreas Lommatzsch,

Sahin AlbayrakAuthors Info & Claims

IIIX '12: Proceedings of the 4th Information Interaction in Context Symposium

Pages 294 - 297

https://doi.org/10.1145/2362724.2362778

Published: 21 August 2012 Publication History

Get Access

Abstract

Research on recommender systems evaluation generally measures the quality of the algorithm, or system, offline, i.e. based on some information retrieval metric, e.g. precision or recall. The metrics do however not always reflect the users' perceptions of the recommendations. Perception-related values are instead often measured through user studies, however the bulk of the work on recommender systems is evaluated through offline analysis. In the work presented in this paper we choose to neglect the quality of the recommender system and instead focus on the similarity of aspects related to users' perception of recommender systems. Based on a user study (N = 132) we show the correlation of concepts such as usefulness, ratings, obviousness, and serendipity from the users' perspectives.

References

[1]

Dirk Bollen, Bart P. Knijnenburg, Martijn C. Willemsen, and Mark Graus, 'Understanding choice overload in recommender systems', in Proc. of the 4th ACM conference on Recommender systems, RecSys '10, pp. 63--70, New York, NY, USA, (2010). ACM.

Digital Library

Google Scholar

[2]

Laurent Candillier, Kris Jack, F Fessant, and Frank Meyer, 'State-of-the-art recommender systems', Collaborative and Social Information Retrieval and AccessTechniques for Improved User Modeling, 1--22, (2009).

Google Scholar

[3]

Dan Cosley, Shyong K. Lam, Istvan Albert, Joseph A. Konstan, and John Riedl, 'Is seeing believing?: how recommender system interfaces affect users' opinions', in Proc. of the SIGCHI conference on Human factors in computing systems, CHI '03, pp. 585--592, New York, NY, USA, (2003). ACM.

Digital Library

Google Scholar

[4]

Jonathan L. Herlocker, Joseph A. Konstan, Loren G. Terveen, and John T. Riedl, 'Evaluating collaborative filtering recommender systems', Trans. Inf. Syst., 22(1), (2004).

Digital Library

Google Scholar

[5]

Rong Hu and Pearl Pu, 'Enhancing recommendation diversity with organization interfaces', in Proc. of the 16th Intl. conference on Intelligent user interfaces, IUI '11, pp. 347--350, New York, NY, USA, (2011). ACM.

Digital Library

Google Scholar

[6]

Bart Knijnenburg, Martijn Willemsen, Zeno Gantner, Hakan Soncu, and Chris Newell, 'Explaining the user experience of recommender systems', User Modeling and User-Adapted Interaction, 22, 441--504, (2012).

Digital Library

Google Scholar

[7]

Bart P. Knijnenburg, Martijn C. Willemsen, and Alfred Kobsa, 'A pragmatic procedure to support the user-centric evaluation of recommender systems', in Proc. of the 5th ACM conference on Recommender systems, RecSys '11, pp. 321--324, New York, NY, USA, (2011). ACM.

Digital Library

Google Scholar

[8]

Sean M. McNee, John Riedl, and Joseph A. Konstan, 'Solving the apparent diversity-accuracy dilemma of recommender systems', number 10, pp. 4511--4515, New York, NY, USA, (1998).

Google Scholar

[9]

Alan Said, Benjamin Kille, Brijnesh J. Jain, and Sahin Albayrak, 'Increasing diversity through furthest neighbor-based recommendation', in Proc. of the WSDM'12 Workshop on Diversity in Document Retrieval (DDR'12), (2012).

Google Scholar

[10]

Kirsten Swearingen and Rashmi Sinha, 'Beyond Algorithms: An HCI Perspective on Recommender Systems', in ACM SIGIR Workshop on Recommender Systems, (2001).

Google Scholar

Cited By

View all

Binst B(2024)How to Evaluate Serendipity in Recommender Systems: the Need for a SerendiptionnaireProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688017(1335-1341)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688017
McCay-Peet LToms E(2017)Researching Serendipity in Digital Information EnvironmentsSynthesis Lectures on Information Concepts, Retrieval, and Services10.2200/S00790ED1V01Y201707ICR0599:6(i-91)Online publication date: 28-Sep-2017
https://doi.org/10.2200/S00790ED1V01Y201707ICR059
Fuhr NKamps JKraaij WVerberne S(2012)Report on IIiX'12ACM SIGIR Forum10.1145/2492189.249219447:1(22-30)Online publication date: 7-Jun-2012
https://dl.acm.org/doi/10.1145/2492189.2492194

Index Terms

Correlating perception-oriented aspects in user-centric recommender system evaluation
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking
    2. Retrieval tasks and goals
      1. Document filtering
      2. Information extraction
  2. Information storage systems

Recommendations

User-centric evaluation of a K-furthest neighbor collaborative filtering recommender algorithm
CSCW '13: Proceedings of the 2013 conference on Computer supported cooperative work

Collaborative filtering recommender systems often use nearest neighbor methods to identify candidate items. In this paper we present an inverted neighborhood model, k-Furthest Neighbors, to identify less ordinary neighborhoods for the purpose of ...
KMulE: a framework for user-based comparison of recommender algorithms
IUI '12: Proceedings of the 2012 ACM international conference on Intelligent User Interfaces

Collaborative Filtering Recommender Systems come in a wide variety of variants. In this paper we present a system for visualizing and comparing recommendations provided by different collaborative recommendation algorithms. The system utilizes a set of ...
A Clustering Approach for Personalizing Diversity in Collaborative Recommender Systems
UMAP '17: Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization

Much of the focus of recommender systems research has been on the accurate prediction of users' ratings for unseen items. Recent work has suggested that objectives such as diversity and novelty in recommendations are also important factors in the ...

Comments

Information & Contributors

Information

Published In

IIIX '12: Proceedings of the 4th Information Interaction in Context Symposium

August 2012

347 pages

ISBN:9781450312820

DOI:10.1145/2362724

General Chairs:
Jaap Kamps,
Wessel Kraaij,
Program Chair:
Norbert Fuhr
University of Duisburg-Essen

In-Cooperation

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 August 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IIiX'12

Sponsor:

University of Amsterdam

IIiX'12: Information Interaction in Context: 2012

August 21 - 24, 2012

Nijmegen, The Netherlands

Acceptance Rates

Overall Acceptance Rate 21 of 45 submissions, 47%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
134
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Binst B(2024)How to Evaluate Serendipity in Recommender Systems: the Need for a SerendiptionnaireProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688017(1335-1341)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688017
McCay-Peet LToms E(2017)Researching Serendipity in Digital Information EnvironmentsSynthesis Lectures on Information Concepts, Retrieval, and Services10.2200/S00790ED1V01Y201707ICR0599:6(i-91)Online publication date: 28-Sep-2017
https://doi.org/10.2200/S00790ED1V01Y201707ICR059
Fuhr NKamps JKraaij WVerberne S(2012)Report on IIiX'12ACM SIGIR Forum10.1145/2492189.249219447:1(22-30)Online publication date: 7-Jun-2012
https://dl.acm.org/doi/10.1145/2492189.2492194

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

User-centric evaluation of a K-furthest neighbor collaborative filtering recommender algorithm

KMulE: a framework for user-based comparison of recommender algorithms

A Clustering Approach for Personalizing Diversity in Collaborative Recommender Systems

Comments

Information

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations