Evaluating unfairness of popularity bias in recommender systems: : A comprehensive user-centric analysis

The popularity bias problem is one of the most prominent challenges of recommender systems, i.e., while a few heavily rated items receive much attention in presented recommendation lists, less popular ones are underrepresented even if they would be of close interest to the user. This structural tendency of recommendation algorithms causes several unfairness issues for most of the items in the catalog as they are having trouble finding a place in the top-N lists. In this study, we evaluate the popularity bias problem from users’ viewpoint and discuss how to alleviate it by considering users as one of the major stakeholders. We derive five critical discriminative features based on the following five essential attributes related to users’ rating behavior, (i) the interaction level of users with the system, (ii) the overall liking degree of users, (iii) the degree of anomalous rating behavior of users, (iv) the consistency of users, and (v) the informative level of the user profiles, and analyze their relationships to the original inclinations of users toward item popularity. More importantly, we investigate their associations with possible unfairness concerns for users, which the popularity bias in recommendations might induce. The analysis using ten well-known recommendation algorithms from different families on four real-world preference collections from different domains reveals that the popularity propensities of individuals are significantly correlated with almost all of the investigated features with varying trends, and algorithms are strongly biased towards popular items. Especially, highly interacting, selective, and hard-to-predict users face highly unfair, relatively inaccurate, and primarily unqualified recommendations in terms of beyond-accuracy aspects, although they are major stakeholders of the system. We also analyze how state-of-the-art popularity debiasing strategies act to remedy these problems. Although they are more effective for mistreated groups in alleviating unfairness and improving beyond-accuracy quality, they mostly fail to preserve ranking accuracy.