tutorial

Fairness in Recommender Systems: Evaluation Approaches and Assurance Strategies

Authors:

Guandong XuAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data, Volume 18, Issue 1

Article No.: 10, Pages 1 - 37

https://doi.org/10.1145/3604558

Published: 10 August 2023 Publication History

Abstract

With the wide application of recommender systems, the potential impacts of recommender systems on customers, item providers and other parties have attracted increasing attention. Fairness, which is the quality of treating people equally, is also becoming important in recommender system evaluation and algorithm design. Therefore, in the past years, there has been a growing interest in fairness measurement and assurance in recommender systems. Although there are several reviews on related topics, such as fairness in machine learning and debias in recommender systems, they do not present a systematic view on fairness in recommender systems, which is context aware and has a multi-sided meaning. Therefore, in this review, the concept of fairness is discussed in detail in the various contexts of recommender systems. Specifically, a comprehensive framework to classify fairness metrics is proposed from four dimensions, i.e., Fairness for Whom, Demographic Unit, Time Frame, and Quantification Method. Then the strategies for eliminating unfairness in recommendations, fairness in different recommendation tasks and datasets are reviewed and summarized. Finally, the challenges and future work are discussed.

References

[1]

Fabian Abel, Yashar Deldjoo, Mehdi Elahi, and Daniel Kohlsdorf. 2017. Recsys challenge 2017: Offline and online evaluation. In Proceedings of the 11th ACM Conference on Recommender Systems. 372–373.

Digital Library

[2]

Gediminas Adomavicius and YoungOk Kwon. 2011. Improving aggregate recommendation diversity using ranking-based techniques. IEEE Trans. Knowl. Data Eng. 24, 5 (2011), 896–911.

Digital Library

[3]

Mohammad Aliannejadi, Hamed Zamani, Fabio Crestani, and W. Bruce Croft. 2021. Context-aware target apps selection and recommendation for enhancing personal mobile assistants. ACM Trans. Inf. Syst. 39, 3 (2021), 1–30.

Digital Library

[4]

Elizabeth S. Anderson. 1999. What is the point of equality? Ethics 109, 2 (1999), 287–337.

[5]

Ashwathy Ashokan and Christian Haas. 2021. Fairness metrics and bias mitigation strategies for rating predictions. Inf. Process. Manage. 58, 5 (2021), 102646.

Digital Library

[6]

Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Li Wei, Yi Wu, Lukasz Heldt, Zhe Zhao, Lichan Hong, Ed H. Chi, et al. 2019. Fairness in recommendation ranking through pairwise comparisons. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2212–2220.

Digital Library

[7]

Asia J. Biega, Krishna P Gummadi, and Gerhard Weikum. 2018. Equity of attention: Amortizing individual fairness in rankings. In Proceedings of the 41st International ACM Sigir Conference on Research & Development in Information Retrieval. 405–414.

Digital Library

[8]

Donald Black. 1993. Sociological Justice. Oxford University Press USA.

[9]

Jesús Bobadilla, Raúl Lara-Cabrera, Ángel González-Prieto, and Fernando Ortega. 2020. DeepFair: Deep learning for improving fairness in recommender systems. CoRR abs/2006.05255, (2020). Retrieved from https://arxiv.org/abs/2006.05255

[10]

Hamed Bonab, Mohammad Aliannejadi, Ali Vardasbi, Evangelos Kanoulas, and James Allan. 2021. Cross-market product recommendation. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 110–119.

Digital Library

[11]

Ludovico Boratto, Gianni Fenu, and Mirko Marras. 2021. Connecting user and item perspectives in popularity debiasing for collaborative recommendation. Inf. Process. Manage. 58, 1 (2021), 102387.

[12]

Ludovico Boratto, Gianni Fenu, and Mirko Marras. 2021. Interplay between upsampling and regularization for provider fairness in recommender systems. User Model. User-Adapt. Interact. 31, 3 (2021), 421–455.

Digital Library

[13]

Rodrigo Borges and Kostas Stefanidis. 2019. Enhancing long term fairness in recommendations with variational autoencoders. In Proceedings of the 11th International Conference on Management of Digital EcoSystems. 95–102.

Digital Library

[14]

Keith Bradley and Barry Smyth. 2001. Improving recommendation diversity. In Proceedings of the 12th Irish Conference on Artificial Intelligence and Cognitive Science, Vol. 85. Citeseer, 141–152.

[15]

Robin Burke. 2017. Multisided Fairness for Recommendation. CoRR abs/1707.00093, (2017). Retrieved from http://arxiv.org/abs/1707.00093

[16]

Robin Burke, Nasim Sonboli, and Aldo Ordonez-Gauger. 2018. Balanced neighborhoods for multi-sided fairness in recommendation. In Proceedings of the Conference on Fairness, Accountability and Transparency. PMLR, 202–214.

[17]

Iván Cantador, Peter Brusilovsky, and Tsvi Kuflik. 2011. Second workshop on information heterogeneity and fusion in recommender systems (HetRec2011). In Proceedings of the 5th ACM Conference on Recommender systems. 387–388.

Digital Library

[18]

Simon Caton and Christian Haas. 2020. Fairness in machine learning: A Survey. CoRR abs/2010.04053, (2020). Retrieved from https://arxiv.org/abs/2010.04053

[19]

Abhijnan Chakraborty, Gourab K. Patro, Niloy Ganguly, Krishna P. Gummadi, and Patrick Loiseau. 2019. Equality of voice: Towards fair representation in crowdsourced top-k recommendations. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 129–138.

Digital Library

[20]

Chong Chen, Weizhi Ma, Min Zhang, Zhaowei Wang, Xiuqiang He, Chenyang Wang, Yiqun Liu, and Shaoping Ma. 2021. Graph heterogeneous multi-relational recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 3958–3966.

[21]

Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2023. Bias and debias in recommender system: A survey and future directions. ACM Transactions on Information Systems 41, 3 (2023), 1–39.

[22]

Eunjoon Cho, Seth A. Myers, and Jure Leskovec. 2011. Friendship and mobility: User movement in location-based social networks. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1082–1090.

Digital Library

[23]

Zhendong Chu, Hongning Wang, Yun Xiao, Bo Long, and Lingfei Wu. 2023. Meta Policy Learning for Cold-Start Conversational Recommendation. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining (WSDM’23), Association for Computing Machinery, Singapore, 222–230.

[24]

Sam Corbett-Davies and Sharad Goel. 2018. The measure and mismeasure of fairness: A critical review of fair machine learning. CoRR abs/1808.00023, (2018). Retrieved from http://arxiv.org/abs/1808.00023

[25]

Paolo Cremonesi, Franca Garzotto, Sara Negro, Alessandro Papadopoulos, and Roberto Turrin. 2011. Comparative evaluation of recommender system quality. In CHI’11 Extended Abstracts on Human Factors in Computing Systems. 1927–1932.

Digital Library

[26]

Gabriel de Souza Pereira Moreira, Felipe Ferreira, and Adilson Marques da Cunha. 2018. News session-based recommendations using deep neural networks. In Proceedings of the 3rd Workshop on Deep Learning for Recommender Systems. 15–23.

Digital Library

[27]

Miguel Delgado-Rodriguez and Javier Llorca. 2004. Bias. J. Epidemiol. Commun. Health 58, 8 (2004), 635–641.

[28]

Danilo Dessì, Gianni Fenu, Mirko Marras, and Diego Reforgiato Recupero. 2018. Coco: Semantic-enriched collection of online courses at scale with experimental use cases. In Trends and Advances in Information Systems and Technologies: Volume 2 6. Springer, Berlin, 1386–1396.

[29]

Virginie Do, Sam Corbett-Davies, Jamal Atif, and Nicolas Usunier. 2022. Online certification of preference-based fairness for personalized recommender systems. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 6532–6540.

[30]

Yushun Dong, Jian Kang, Hanghang Tong, and Jundong Li. 2021. Individual fairness for graph neural networks: A ranking based approach. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 300–310.

Digital Library

[31]

Yvonne Durham, Jack Hirshleifer, and Vernon L. Smith. 1998. Do the rich get richer and the poor poorer? Experimental tests of a model of power. Am. Econ. Rev. 88, 4 (1998), 970–983.

[32]

Bora Edizel, Francesco Bonchi, Sara Hajian, André Panisson, and Tamir Tassa. 2020. FaiRecSys: Mitigating algorithmic bias in recommender systems. Int. J. Data Sci. Anal. 9, 2 (2020), 197–213.

[33]

Michael D. Ekstrand, Mucun Tian, Mohammed R. Imran Kazi, Hoda Mehrpouyan, and Daniel Kluver. 2018. Exploring author gender in book rating and recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems. 242–250.

[34]

Michael D. Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D. Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2018. All the cool kids, how do they fit in?: Popularity and demographic biases in recommender evaluation and effectiveness. In Conference on Fairness, Accountability and Transparency. PMLR, 172–186.

[35]

Andres Ferraro, Dmitry Bogdanov, Xavier Serra, and Jason Yoon. 2019. Artist and style exposure bias in collaborative filtering based music recommendations. CoRR abs/1911.04827, (2019). Retrieved from http://arxiv.org/abs/1911.04827

[36]

Zuohui Fu, Yikun Xian, Ruoyuan Gao, Jieyu Zhao, Qiaoying Huang, Yingqiang Ge, Shuyuan Xu, Shijie Geng, Chirag Shah, Yongfeng Zhang, et al. 2020. Fairness-aware explainable recommendation over knowledge graphs. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 69–78.

Digital Library

[37]

Kun Gai, Xiaoqiang Zhu, Han Li, Kai Liu, and Zhe Wang. 2017. Learning piece-wise linear models from large scale data for ad click prediction. arXiv:1704.05194. Retrieved from https://arxiv.org/abs/1704.05194

[38]

David García-Soriano and Francesco Bonchi. 2021. Maxmin-fair ranking: Individual fairness under group-fairness constraints. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 436–446.

Digital Library

[39]

Yingqiang Ge, Shuchang Liu, Ruoyuan Gao, Yikun Xian, Yunqi Li, Xiangyu Zhao, Changhua Pei, Fei Sun, Junfeng Ge, Wenwu Ou, et al. 2021. Towards long-term fairness in recommendation. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 445–453.

Digital Library

[40]

Yingqiang Ge, Juntao Tan, Yan Zhu, Yinglong Xia, Jiebo Luo, Shuchang Liu, Zuohui Fu, Shijie Geng, Zelong Li, and Yongfeng Zhang. 2022. Explainable fairness in recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 681–691.

Digital Library

[41]

Yingqiang Ge, Xiaoting Zhao, Lucia Yu, Saurabh Paul, Diane Hu, Chu-Cheng Hsieh, and Yongfeng Zhang. 2022. Toward pareto efficient fairness-utility trade-off in recommendation through reinforcement learning. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining. 316–324.

Digital Library

[42]

Sahin Cem Geyik, Stuart Ambler, and Krishnaram Kenthapadi. 2019. Fairness-aware ranking in search & recommendation systems with application to linkedin talent search. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2221–2231.

Digital Library

[43]

David Goldberg, David Nichols, Brian M. Oki, and Douglas Terry. 1992. Using collaborative filtering to weave an information tapestry. Commun. ACM 35, 12 (1992), 61–70.

Digital Library

[44]

Jon Atle Gulla, Lemei Zhang, Peng Liu, Özlem Özgöbek, and Xiaomeng Su. 2017. The adressa dataset for news recommendation. In Proceedings of the International Conference on Web Intelligence. 1042–1048.

Digital Library

[45]

Asela Gunawardana and Guy Shani. 2009. A survey of accuracy evaluation metrics of recommendation tasks. J. Mach. Learn. Res. 10, 12 (2009).

[46]

Ananya Gupta, Eric Johnson, Justin Payan, Aditya Kumar Roy, Ari Kobren, Swetasudha Panda, Jean-Baptiste Tristan, and Michael Wick. 2021. Online post-processing in rankings for fair utility maximization. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 454–462.

Digital Library

[47]

Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of opportunity in supervised learning. Adv. Neural Inf. Process. Syst. 29 (2016), 3315–3323.

[48]

F. Maxwell Harper and Joseph A. Konstan. 2015. The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst. 5, 4 (2015), 1–19.

Digital Library

[49]

Ruining He, Wang-Cheng Kang, and Julian McAuley. 2017. Translation-based recommendation. In Proceedings of the 11th ACM Conference on Recommender Systems. 161–169.

Digital Library

[50]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web. 173–182.

Digital Library

[51]

Jonathan L. Herlocker, Joseph A. Konstan, and John Riedl. 2000. Explaining collaborative filtering recommendations. In Proceedings of the ACM Conference on Computer Supported Cooperative Work. 241–250.

Digital Library

[52]

Neil Hurley and Mi Zhang. 2011. Novelty and diversity in top-n recommendation–analysis and evaluation. ACM Trans. Internet Technol. 10, 4 (2011), 1–30.

Digital Library

[53]

Rashidul Islam, Kamrun Naher Keya, Ziqian Zeng, Shimei Pan, and James Foulds. 2021. Debiasing career recommendations with neural fair collaborative filtering. In Proceedings of the Web Conference. 3779–3790.

Digital Library

[54]

Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. 2016. Field-aware factorization machines for CTR prediction. In Proceedings of the 10th ACM Conference on Recommender Systems. 43–50.

Digital Library

[55]

Faisal Kamiran and Toon Calders. 2012. Data preprocessing techniques for classification without discrimination. Knowl. Inf. Syst. 33, 1 (2012), 1–33.

Digital Library

[56]

Chen Karako and Putra Manggala. 2018. Using image fairness representations in diversity-based re-ranking for recommendations. In Adjunct Publication of the 26th Conference on User Modeling, Adaptation and Personalization. 23–28.

Digital Library

[57]

Mesut Kaya, Derek Bridge, and Nava Tintarev. 2020. Ensuring fairness in group recommendations by rank-sensitive balancing of relevance. In Proceedings of the 14th ACM Conference on Recommender Systems. 101–110.

Digital Library

[58]

Muhammad Murad Khan, Roliana Ibrahim, and Imran Ghani. 2017. Cross domain recommender systems: A systematic literature review. ACM Comput. Surv. 50, 3 (2017), 1–34.

Digital Library

[59]

Ömer Kırnap, Fernando Diaz, Asia Biega, Michael Ekstrand, Ben Carterette, and Emine Yilmaz. 2021. Estimation of fair ranking metrics with incomplete judgments. In Proceedings of the Web Conference 2021. 1065–1075.

Digital Library

[60]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37.

Digital Library

[61]

Anja Lambrecht and Catherine Tucker. 2019. Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads. Manage. Sci. 65, 7 (2019), 2966–2981.

Digital Library

[62]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521, 7553 (2015), 436–444.

[63]

Jurek Leonhardt, Avishek Anand, and Megha Khosla. 2018. User fairness in recommender systems. In Companion Proceedings of the The Web Conference 2018. 101–102.

Digital Library

[64]

Jure Leskovec and Julian Mcauley. 2012. Learning to discover social circles in ego networks. In Advances in Neural Information Processing Systems, Curran Associates, Inc.

[65]

Mark Levy and Klaas Bosteels. 2010. Music recommendation and the long tail. In Proceedings of the Workshop on Music Recommendation and Discovery (WOMRAD’10). 55–58.

[66]

Cheng-Te Li, Cheng Hsu, and Yang Zhang. 2022. FairSR: Fairness-aware sequential recommendation through multi-task learning with preference graph embeddings. ACM Trans. Intell. Syst. Technol. 13, 1 (2022), 1–21.

Digital Library

[67]

Fenghua Li, Hui Li, Ben Niu, and Jinjun Chen. 2019. Privacy computing: Concept, computing framework, and future development trends. Eng. 5, 6 (2019), 1179–1192.

[68]

Jie Li, Yongli Ren, and Ke Deng. 2022. FairGAN: GANs-based fairness-aware learning for recommendations with implicit feedback. In Proceedings of the ACM Web Conference 2022. 297–307.

Digital Library

[69]

Yunqi Li, Hanxiong Chen, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2021. User-oriented fairness in recommendation. In Proceedings of the Web Conference 2021. 624–632.

Digital Library

[70]

Yunqi Li, Hanxiong Chen, Shuyuan Xu, Yingqiang Ge, and Yongfeng Zhang. 2021. Towards personalized fairness based on causal notion. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1054–1063.

Digital Library

[71]

Yunqi Li, Dingxian Wang, Hanxiong Chen, and Yongfeng Zhang. 2023. Transferable fairness for cold-start recommendation. arXiv:2301.10665. Retrieved from https://arxiv.org/abs/2301.10665

[72]

Leah A. Lievrouw and Sharon E. Farb. 2003. Information and equity. Annu. Rev. Inf. Sci. Technol. 37, 1 (2003), 499–540.

[73]

Xiao Lin, Hongjie Chen, Changhua Pei, Fei Sun, Xuanji Xiao, Hanxiao Sun, Yongfeng Zhang, Wenwu Ou, and Peng Jiang. 2019. A pareto-efficient algorithm for multiple objective optimization in e-commerce recommendation. In Proceedings of the 13th ACM Conference on Recommender Systems. 20–28.

Digital Library

[74]

Greg Linden, Brent Smith, and Jeremy York. 2003. Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet Comput. 7, 1 (2003), 76–80.

Digital Library

[75]

Qi Liu, Enhong Chen, Hui Xiong, Yong Ge, Zhongmou Li, and Xiang Wu. 2012. A cocktail approach for travel package recommendation. IEEE Trans. Knowl. Data Eng. 26, 2 (2012), 278–293.

Digital Library

[76]

Weiwen Liu, Jun Guo, Nasim Sonboli, Robin Burke, and Shengyu Zhang. 2019. Personalized fairness-aware re-ranking for microlending. In Proceedings of the 13th ACM Conference on Recommender Systems. 467–471.

Digital Library

[77]

Daniel Lowd and Christopher Meek. 2005. Adversarial learning. In Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data Mining. 641–647.

Digital Library

[78]

Lucas Machado and Kostas Stefanidis. 2019. Fair team recommendations for multidisciplinary projects. In Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence (WI’19). IEEE, 293–297.

Digital Library

[79]

Karima Makhlouf, Sami Zhioua, and Catuscia Palamidessi. 2021. Machine learning fairness notions: Bridging the gap with real-world applications. Inf. Process. Manage. 58, 5 (2021), 102642.

Digital Library

[80]

Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015. Image-based recommendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. 43–52.

Digital Library

[81]

Brian McFee, Thierry Bertin-Mahieux, Daniel P. W. Ellis, and Gert R. G. Lanckriet. 2012. The million song dataset challenge. In Proceedings of the 21st International Conference on World Wide Web. 909–916.

Digital Library

[82]

Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A survey on bias and fairness in machine learning. ACM Comput. Surv. 54, 6 (2021), 1–35.

Digital Library

[83]

Rishabh Mehrotra, James McInerney, Hugues Bouchard, Mounia Lalmas, and Fernando Diaz. 2018. Towards a fair marketplace: Counterfactual evaluation of the trade-off between relevance, fairness & satisfaction in recommendation systems. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 2243–2251.

Digital Library

[84]

Alessandro B. Melchiorre, Navid Rekabsaz, Emilia Parada-Cabaleiro, Stefan Brandl, Oleg Lesota, and Markus Schedl. 2021. Investigating gender fairness of recommendation algorithms in the music domain. Inf. Process. Manage. 58, 5 (2021), 102666.

Digital Library

[85]

Marco Morik, Ashudeep Singh, Jessica Hong, and Thorsten Joachims. 2020. Controlling fairness and bias in dynamic learning-to-rank. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 429–438.

Digital Library

[86]

Mohammadmehdi Naghiaei, Hossein A. Rahmani, and Yashar Deldjoo. 2022. Cpfair: Personalized consumer and producer fairness re-ranking for recommender systems. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 770–779.

Digital Library

[87]

Eirini Ntoutsi, Pavlos Fafalios, Ujwal Gadiraju, Vasileios Iosifidis, Wolfgang Nejdl, Maria-Esther Vidal, Salvatore Ruggieri, Franco Turini, Symeon Papadopoulos, Emmanouil Krasanakis, et al. 2020. Bias in data-driven artificial intelligence systems–An introductory survey. Data Min. Knowl. Discov. 10, 3 (2020), e1356.

[88]

Martin Oelz. 2014. The ILO’s domestic workers convention and recommendation: A window of opportunity for social justice. Int. Labour Rev. 153, 1 (2014), 143–172.

[89]

Yitong Pang, Lingfei Wu, Qi Shen, Yiming Zhang, Zhihua Wei, Fangli Xu, Ethan Chang, Bo Long, and Jian Pei. 2022. Heterogeneous global graph neural networks for personalized session-based recommendation. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining. 775–783.

Digital Library

[90]

Gourab K. Patro, Arpita Biswas, Niloy Ganguly, Krishna P. Gummadi, and Abhijnan Chakraborty. 2020. Fairrec: Two-sided fairness for personalized recommendations in two-sided platforms. In Proceedings of the Web Conference 2020. 1194–1204.

Digital Library

[91]

Gourab K. Patro, Abhijnan Chakraborty, Niloy Ganguly, and Krishna Gummadi. 2020. Incremental fairness in two-sided market platforms: On smoothly updating recommendations. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 181–188.

[92]

Chao-Ying Joanne Peng, Kuk Lida Lee, and Gary M. Ingersoll. 2002. An introduction to logistic regression analysis and reporting. J. Educ. Res. 96, 1 (2002), 3–14.

[93]

Evaggelia Pitoura, Kostas Stefanidis, and Georgia Koutrika. 2022. Fairness in Rankings and Recommendations: An Overview. The VLDB Journal 31, 3 (2022), 431–458.

[94]

Tao Qi, Fangzhao Wu, Chuhan Wu, Peijie Sun, Le Wu, Xiting Wang, Yongfeng Huang, and Xing Xie. 2022. ProFairRec: Provider fairness-aware news recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1164–1173.

Digital Library

[95]

Shiyou Qian, Jian Cao, Frédéric Le Mouël, Issam Sahel, and Minglu Li. 2015. SCRAM: A sharing considered route assignment mechanism for fair taxi route recommendations. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 955–964.

Digital Library

[96]

Hossein A. Rahmani, Mohammadmehdi Naghiaei, Mahdi Dehghan, and Mohammad Aliannejadi. 2022. Experiments on generalizability of user-oriented fairness in recommender systems. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2755–2764.

Digital Library

[97]

Naime Ranjbar Kermany, Jian Yang, Jia Wu, and Luiz Pizzato. 2022. Fair-SRS: A fair session-based recommendation system. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining. 1601–1604.

[98]

Bashir Rastegarpanah, Krishna P. Gummadi, and Mark Crovella. 2019. Fighting fire with fire: Using antidote data to improve polarization and fairness of recommender systems. In Proceedings of the 12th ACM International Conference on Web Search and Data Mining. 231–239.

Digital Library

[99]

Shaina Raza and Chen Ding. 2019. Progress in context-aware recommender systems–An overview. Comput. Sci. Rev. 31 (2019), 84–97.

Digital Library

[100]

Shaina Raza and Chen Ding. 2021. Deep neural network to tradeoff between accuracy and diversity in a news recommender system. In Proceedings of the IEEE International Conference on Big Data (Big Data’21). IEEE, 5246–5256.

[101]

Steffen Rendle. 2010. Factorization machines. In Proceedings of the IEEE International Conference on Data Mining. IEEE, 995–1000.

Digital Library

[102]

Marco Tulio Ribeiro, Nivio Ziviani, Edleno Silva De Moura, Itamar Hata, Anisio Lacerda, and Adriano Veloso. 2014. Multiobjective pareto-efficient approaches for recommender systems. ACM Trans. Intell. Syst. Technol. 5, 4 (2014), 1–20.

Digital Library

[103]

Dimitris Sacharidis. 2019. Top-N group recommendations with fairness. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing. 1663–1670.

Digital Library

[104]

Dimitris Sacharidis, Kyriakos Mouratidis, and Dimitrios Kleftogiannis. 2019. A common approach for consumer and provider fairness in recommendations. In ACM RecSys: Late-Breaking Results. 1–5.

[105]

Dimitris Sacharidis, Carine Pierrette Mukamakuza, and Hannes Werthner. 2020. Fairness and diversity in social-based recommender systems. In Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization. 83–88.

Digital Library

[106]

Ryoma Sato. 2022. Enumerating fair packages for group recommendations. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining. 870–878.

Digital Library

[107]

Laura Schelenz. 2021. Diversity-aware recommendations for social justice? Exploring user diversity and fairness in recommender systems. In Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization. 404–410.

Digital Library

[108]

Dimitris Serbos, Shuyao Qi, Nikos Mamoulis, Evaggelia Pitoura, and Panayiotis Tsaparas. 2017. Fairness in package-to-group recommendations. In Proceedings of the 26th International Conference on World Wide Web. 371–379.

Digital Library

[109]

Till Speicher, Hoda Heidari, Nina Grgic-Hlaca, Krishna P Gummadi, Adish Singla, Adrian Weller, and Muhammad Bilal Zafar. 2018. A unified approach to quantifying algorithmic unfairness: Measuring individual &group unfairness via inequality indices. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2239–2248.

Digital Library

[110]

Maria Stratigi, Jyrki Nummenmaa, Evaggelia Pitoura, and Kostas Stefanidis. 2020. Fair sequential group recommendations. In Proceedings of the 35th Annual ACM Symposium on Applied Computing. 1443–1452.

Digital Library

[111]

Xiaoyuan Su and Taghi M. Khoshgoftaar. 2009. A survey of collaborative filtering techniques. Advances in Artificial Intelligence 2009, (2009), 421425.

[112]

Tom Sühr, Asia J. Biega, Meike Zehlike, Krishna P. Gummadi, and Abhijnan Chakraborty. 2019. Two-sided fairness for repeated matchings in two-sided markets: A case study of a ride-hailing platform. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 3082–3092.

Digital Library

[113]

Zhoubao Sun, Lixin Han, Wenliang Huang, Xueting Wang, Xiaoqin Zeng, Min Wang, and Hong Yan. 2015. Recommender systems based on social networks. J. Syst. Softw. 99 (2015), 109–119.

Digital Library

[114]

Jiakai Tang, Xu Chen, and Xueyang Feng. 2023. Fairness-aware cross-domain recommendation. arXiv:2302.00158. Retrieved from https://arxiv.org/abs/2302.00158

[115]

Jiliang Tang, Huiji Gao, and Huan Liu. 2012. mTrust: Discerning multi-faceted trust in a connected world. In Proceedings of the 5th ACM International Conference on Web Search and Data Mining. 93–102.

Digital Library

[116]

Sahil Verma, Ruoyuan Gao, and Chirag Shah. 2020. Facets of fairness in search and recommendation. In International Workshop on Algorithmic Bias in Search and Recommendation. Springer, 1–11.

[117]

Mengting Wan, Jianmo Ni, Rishabh Misra, and Julian McAuley. 2020. Addressing marketing bias in product recommendations. In Proceedings of the 13th International Conference on Web Search and Data Mining. 618–626.

Digital Library

[118]

Shoujin Wang, Longbing Cao, Yan Wang, Quan Z. Sheng, Mehmet A. Orgun, and Defu Lian. 2021. A survey on session-based recommender systems. ACM Comput. Surv. 54, 7 (2021), 1–38.

Digital Library

[119]

Xuezhi Wang, Nithum Thain, Anu Sinha, Flavien Prost, Ed H. Chi, Jilin Chen, and Alex Beutel. 2021. Practical compositional fairness: Understanding fairness in multi-component recommender systems. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 436–444.

Digital Library

[120]

Zifeng Wang, Xi Chen, Rui Wen, and Shao-Lun Huang. 2020. On the fairness of randomized trials for recommendation with heterogeneous demographics and beyond. CoRR abs/2001.09328, (2020). Retrieved from https://arxiv.org/abs/2001.09328

[121]

Tianxin Wei, Fuli Feng, Jiawei Chen, Ziwei Wu, Jinfeng Yi, and Xiangnan He. 2021. Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1791–1800.

Digital Library

[122]

Tianxin Wei and Jingrui He. 2022. Comprehensive fair meta-learned recommender system. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1989–1999.

Digital Library

[123]

Leonard Weydemann, Dimitris Sacharidis, and Hannes Werthner. 2019. Defining and measuring fairness in location recommendations. In Proceedings of the 3rd ACM SIGSPATIAL International Workshop on Location-based Recommendations, Geosocial Networks and Geoadvertising. 1–8.

Digital Library

[124]

Chuhan Wu, Fangzhao Wu, Xiting Wang, Yongfeng Huang, and Xing Xie. 2021. Fairness-aware news recommendation with decomposed adversarial learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, 4462–4469.

[125]

Fangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu, Tao Qi, Jianxun Lian, Danyang Liu, Xing Xie, Jianfeng Gao, Winnie Wu, et al. 2020. Mind: A large-scale dataset for news recommendation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 3597–3606.

[126]

Haolun Wu, Bhaskar Mitra, Chen Ma, Fernando Diaz, and Xue Liu. 2022. Joint multisided exposure fairness for recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 703–714.

Digital Library

[127]

Le Wu, Lei Chen, Pengyang Shao, Richang Hong, Xiting Wang, and Meng Wang. 2021. Learning fair representations for recommendation: A Graph-based perspective. In Proceedings of the Web Conference 2021. 2198–2208.

Digital Library

[128]

Libing Wu, Cong Quan, Chenliang Li, Qian Wang, Bolong Zheng, and Xiangyang Luo. 2019. A context-aware user-item representation learning for item recommendation. ACM Trans. Inf. Syst. 37, 2 (2019), 1–29.

Digital Library

[129]

Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019. Session-based recommendation with graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 346–353.

Digital Library

[130]

Yao Wu, Jian Cao, and Guandong Xu. 2020. FAST: A fairness assured service recommendation strategy considering service capacity constraint. In International Conference on Service-Oriented Computing. Springer, 287–303.

Digital Library

[131]

Yao Wu, Jian Cao, Guandong Xu, and Yudong Tan. 2021. Tfrom: A two-sided fairness-aware recommendation model for both customers and providers. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1013–1022.

Digital Library

[132]

Yiqing Wu, Ruobing Xie, Yongchun Zhu, Fuzhen Zhuang, Ao Xiang, Xu Zhang, Leyu Lin, and Qing He. 2022. Selective fairness in recommendation via prompts. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2657–2662.

Digital Library

[133]

Wenyi Xiao, Huan Zhao, Haojie Pan, Yangqiu Song, Vincent W. Zheng, and Qiang Yang. 2019. Beyond personalization: Social content recommendation for creator equality and consumer satisfaction. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 235–245.

Digital Library

[134]

Yang Xiao, Qingqi Pei, Lina Yao, Shui Yu, Lei Bai, and Xianzhi Wang. 2020. An enhanced probabilistic fairness-aware group recommendation by incorporating social activeness. J. Netw. Comput. Appl. 156 (2020), 102579.

Digital Library

[135]

Dingqi Yang, Daqing Zhang, and Bingqing Qu. 2016. Participatory cultural mapping based on collective behavior data in location-based social networks. ACM Trans. Intell. Syst. Technol. 7, 3 (2016), 1–23.

Digital Library

[136]

Jun Yang, Weizhi Ma, Min Zhang, Xin Zhou, Yiqun Liu, and Shaoping Ma. 2021. LegalGNN: Legal information enhanced graph neural network for recommendation. ACM Trans. Inf. Syst. 40, 2 (2021), 1–29.

Digital Library

[137]

Qiang Yang and Hong Cheng. 2002. Mining case bases for action recommendation. In Proceedings of the IEEE International Conference on Data Mining. IEEE, 522–529.

[138]

Sirui Yao and Bert Huang. 2017. Beyond Parity: Fairness Objectives for Collaborative Filtering. In Advances in Neural Information Processing Systems, Vol. 30. Curran Associates, Inc.

[139]

Guanghu Yuan, Fajie Yuan, Yudong Li, Beibei Kong, Shujie Li, Lei Chen, Min Yang, Chenyun YU, Bo Hu, Zang Li, Yu Xu, and Xiaohu Qie. 2022. Tenrec: A large-scale multipurpose benchmark dataset for recommender systems. In Advances in Neural Information Processing Systems, Curran Associates, Inc., 11480–11493.

[140]

Li Zhang, Han Zhang, and SuMin Hao. 2018. An equity fund recommendation system by combing transfer learning and the utility function of the prospect theory. J. Finan. Data Sci. 4, 4 (2018), 223–233.

[141]

Yiming Zhang, Lingfei Wu, Qi Shen, Yitong Pang, Zhihua Wei, Fangli Xu, Bo Long, and Jian Pei. 2022. Multiple choice questions based multi-interest policy learning for conversational recommendation. In Proceedings of the ACM Web Conference 2022. 2153–2162.

Digital Library

[142]

Cheng Zhao, Chenliang Li, Rong Xiao, Hongbo Deng, and Aixin Sun. 2020. CATN: Cross-domain recommendation for cold-start users via aspect transfer network. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 229–238.

Digital Library

[143]

Guanjie Zheng, Fuzheng Zhang, Zihan Zheng, Yang Xiang, Nicholas Jing Yuan, Xing Xie, and Zhenhui Li. 2018. DRN: A deep reinforcement learning framework for news recommendation. In Proceedings of the World Wide Web Conference. 167–176.

Digital Library

[144]

Chang Zhou, Jianxin Ma, Jianwei Zhang, Jingren Zhou, and Hongxia Yang. 2021. Contrastive learning for debiased candidate generation in large-scale recommender systems. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 3985–3995.

Digital Library

[145]

Fan Zhou, Ruiyang Yin, Kunpeng Zhang, Goce Trajcevski, Ting Zhong, and Jin Wu. 2019. Adversarial point-of-interest recommendation. In Proceedings of the World Wide Web Conference. 3462–34618.

Digital Library

[146]

Qiliang Zhu, Ao Zhou, Qibo Sun, Shangguang Wang, and Fangchun Yang. 2018. FMSR: A fairness-aware mobile service recommendation method. In Proceedings of the IEEE International Conference on Web Services (ICWS’18). IEEE, 171–178.

[147]

Yongchun Zhu, Ruobing Xie, Fuzhen Zhuang, Kaikai Ge, Ying Sun, Xu Zhang, Leyu Lin, and Juan Cao. 2021. Learning to warm up cold item embeddings for cold-start recommendation with meta scaling and shifting networks. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1167–1176.

Digital Library

[148]

Ziwei Zhu, Yun He, Xing Zhao, and James Caverlee. 2021. Popularity bias in dynamic recommendation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2439–2449.

Digital Library

[149]

Ziwei Zhu, Xia Hu, and James Caverlee. 2018. Fairness-aware tensor-based recommendation. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 1153–1162.

Digital Library

[150]

Ziwei Zhu, Jingu Kim, Trung Nguyen, Aish Fenton, and James Caverlee. 2021. Fairness among new items in cold start recommender systems. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 767–776.

Digital Library

[151]

Cai-Nicolas Ziegler, Sean M. McNee, Joseph A. Konstan, and Georg Lausen. 2005. Improving recommendation lists through topic diversification. In Proceedings of the 14th International Conference on World Wide Web. 22–32.

Digital Library

[152]

F. J. Zuiderveen Borgesius, Damian Trilling, Judith Möller, Balázs Bodó, C. H. de Vreese, and Natali Helberger. 2016. Should we worry about filter bubbles? Internet Pol. Rev. 5 1 (2016), 1–16.

Cited By

Kecki VSaid A(2024)Understanding Fairness in Recommender Systems: A Healthcare PerspectiveProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3691711(1125-1130)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3691711
Bojic L(2024)AI alignment: Assessing the global impact of recommender systemsFutures10.1016/j.futures.2024.103383160(103383)Online publication date: Jun-2024
https://doi.org/10.1016/j.futures.2024.103383
Singhal SPal K(2024)State of art and emerging trends on group recommender system: a comprehensive reviewInternational Journal of Multimedia Information Retrieval10.1007/s13735-024-00329-513:2Online publication date: 2-May-2024
https://doi.org/10.1007/s13735-024-00329-5
Show More Cited By

Index Terms

Fairness in Recommender Systems: Evaluation Approaches and Assurance Strategies
1. General and reference
  1. Document types
    1. Surveys and overviews
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

A Survey on the Fairness of Recommender Systems
Recommender systems are an essential tool to relieve the information overload challenge and play an important role in people’s daily lives. Since recommendations involve allocations of social resources (e.g., job recommendation), an important issue is ...
A survey on fairness-aware recommender systems
Abstract
As information filtering services, recommender systems have extremely enriched our daily life by providing personalized suggestions and facilitating people in decision-making, which makes them vital and indispensable to human society in the ...
Highlights
- This survey provides an in-depth, scientific taxonomy of fairness-aware recommender systems.
- This survey explores the relationship between the fairness and trustworthiness of recommender systems.
- Several future directions are ...
Airtime Fairness for IEEE 802.11 Multirate Networks

Under a multi rate network scenario, the IEEE 802.11 DCF MAC fails to provide air-time fairness for all competing stations since the protocol is designed for ensuring max-min throughput fairness and the maximum achievable throughput by any station gets ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 18, Issue 1

January 2024

854 pages

EISSN:1556-472X

DOI:10.1145/3613504

Editor:
Charu Aggarwal
IBM T. J. Watson Research, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 August 2023

Online AM: 12 June 2023

Accepted: 08 June 2023

Revised: 11 February 2023

Received: 28 June 2022

Published in TKDD Volume 18, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Tutorial

Funding Sources

Program of Technology Innovation of the Science and Technology Commission of Shanghai Municipality
China National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
1,153
Total Downloads

Downloads (Last 12 months)832
Downloads (Last 6 weeks)79

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kecki VSaid A(2024)Understanding Fairness in Recommender Systems: A Healthcare PerspectiveProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3691711(1125-1130)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3691711
Bojic L(2024)AI alignment: Assessing the global impact of recommender systemsFutures10.1016/j.futures.2024.103383160(103383)Online publication date: Jun-2024
https://doi.org/10.1016/j.futures.2024.103383
Singhal SPal K(2024)State of art and emerging trends on group recommender system: a comprehensive reviewInternational Journal of Multimedia Information Retrieval10.1007/s13735-024-00329-513:2Online publication date: 2-May-2024
https://doi.org/10.1007/s13735-024-00329-5
Escobedo GMoscati MMuellner PKopeinik SKowald DLex ESchedl M(2024)Making Alice Appear Like Bob: A Probabilistic Preference Obfuscation Method For Implicit Feedback Recommendation ModelsMachine Learning and Knowledge Discovery in Databases. Research Track10.1007/978-3-031-70368-3_21(349-365)Online publication date: 8-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-70368-3_21
Raza S(2023)Bias Reduction News Recommendation SystemDigital10.3390/digital40100034:1(92-103)Online publication date: 28-Dec-2023
https://doi.org/10.3390/digital4010003
Felfernig AWundara MTran TPolat-Erdeniz SLubos SEl Mansi MGarber DLe V(2023)Recommender systems for sustainability: overview and research issuesFrontiers in Big Data10.3389/fdata.2023.12845116Online publication date: 30-Oct-2023
https://doi.org/10.3389/fdata.2023.1284511
Raza SBashir SNaseem ULiu DReji D(2022)Incorporating Accuracy and Diversity in a News Recommender System2022 IEEE 9th International Conference on Data Science and Advanced Analytics (DSAA)10.1109/DSAA54385.2022.10032442(1-10)Online publication date: 13-Oct-2022
https://doi.org/10.1109/DSAA54385.2022.10032442

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents