review-article

Fairness in recommender systems: research landscape and future directions

Authors:

Yashar Deldjoo,

Dietmar Jannach,

Alejandro Bellogin,

Alessandro Difonzo,

Dario ZanzonelliAuthors Info & Claims

User Modeling and User-Adapted Interaction, Volume 34, Issue 1

Pages 59 - 108

https://doi.org/10.1007/s11257-023-09364-z

Published: 24 April 2023 Publication History

Abstract

Recommender systems can strongly influence which information we see online, e.g., on social media, and thus impact our beliefs, decisions, and actions. At the same time, these systems can create substantial business value for different stakeholders. Given the growing potential impact of such AI-based systems on individuals, organizations, and society, questions of fairness have gained increased attention in recent years. However, research on fairness in recommender systems is still a developing area. In this survey, we first review the fundamental concepts and notions of fairness that were put forward in the area in the recent past. Afterward, through a review of more than 160 scholarly publications, we present an overview of how research in this field is currently operationalized, e.g., in terms of general research methodology, fairness measures, and algorithmic approaches. Overall, our analysis of recent works points to certain research gaps. In particular, we find that in many research works in computer science, very abstract problem operationalizations are prevalent and questions of the underlying normative claims and what represents a fair recommendation in the context of a given application are often not discussed in depth. These observations call for more interdisciplinary research to address fairness in recommendation in a more comprehensive and impactful manner.

References

[1]

Abdollahpouri, H., Burke, R.: Multi-stakeholder recommendation and its connection to multi-sided fairness. In: Proceedings of the Workshop on Recommendation in Multi-stakeholder Environments co-located with the 13th ACM Conference on Recommender Systems (RecSys 2019), CEUR Workshop Proceedings, vol. 2440, (2019)

[2]

Abdollahpouri, H., Burke, R., Mobasher, B.: Managing popularity bias in recommender systems with personalized re-ranking. In: Proceedings of the Thirty-Second International Florida Artificial Intelligence Research Society Conference, pp. 413–418, (2019a)

[3]

Abdollahpouri, H., Mansoury, M., Burke, R., Mobasher, B.: The unfairness of popularity bias in recommendation. In: Proceedings of the Workshop on Recommendation in Multi-stakeholder Environments co-located with the 13th ACM Conference on Recommender Systems (RecSys 2019), vol. 2440, (2019b)

[4]

Abdollahpouri H, Adomavicius G, Burke R, Guy I, Jannach D, Kamishima T, Krasnodebski J, and Pizzato L Multistakeholder recommendation: Survey and research directions User Model. User-Adap. Inter. 2020 30 127-158

[5]

Abdollahpouri, H., Mansoury, M., Burke, R., Mobasher, B.: The connection between popularity bias, calibration, and fairness in recommendation. In: Fourteenth ACM Conference on Recommender Systems, pp. 726–731, (2020b)

[6]

Abdollahpouri, H., Mansoury, M., Burke, R., Mobasher, B., Malthouse, E.C.: User-centered evaluation of popularity bias in recommender systems. In: Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2021, ACM, pp. 119–129, (2021)

[7]

Adomavicius, G., Tuzhilin, A.: Context-aware recommender systems. In: Ricci F, Rokach L, Shapira B (eds) Recommender Systems Handbook, pp. 191–226, (2015)

[8]

Adomavicius, G., Jannach, D., Leitner, S., Zhang, J.: Understanding longitudinal dynamics of recommender systems with agent-based modeling and simulation. In: SimuRec Workshop at ACM RecSys 2021, (2021)

[9]

Amigó E, Deldjoo Y, Mizzaro S, and Bellogín A A unifying and general account of fairness measurement in recommender systems Inf. Process. Manag. 2023 60 1

Digital Library

[10]

Anelli, V.W., Belli, L., Deldjoo, Y., Di Noia, T., Ferrara, A., Narducci, F., Pomo, C.: Pursuing privacy in recommender systems: the view of users and researchers from regulations to applications. In: Fifteenth ACM Conference on Recommender Systems, pp. 838–841, (2021)

[11]

Anelli, V.W., Deldjoo, Y., Di Noia, T., Malitesta, D., Paparella, V., Pomo, C.: Auditing consumer- and producer-fairness in graph collaborative filtering. In: Proceedings ECIR ’23, (2023)

[12]

Angwin, J., Larson, J., Mattu, S., Kirchner, L.: Machine bias. In: Ethics of Data and Analytics, Auerbach Publications, pp. 254–264, (2016)

[13]

Ashokan A and Haas C Fairness metrics and bias mitigation strategies for rating predictions Inf. Process. Manag. 2021 58 5

Digital Library

[14]

Baeza-Yates R Bias on the web Commun. ACM 2018 61 6 54-61

Digital Library

[15]

Barocas, S., Hardt, M., Narayanan, A.: Fairness and Machine Learning. fairmlbook.org, (2019), http://www.fairmlbook.org

[16]

Bellogín A and Said A Improving accountability in recommender systems research through reproducibility User Model User Adapt Interact 2021 31 5 941-977

Digital Library

[17]

Beutel, A., Chen, J., Doshi, T., Qian, H., Wei, L., Wu, Y., Heldt, L., Zhao, Z., Hong, L., Chi, E.H., Goodrow, C.: Fairness in recommendation ranking through pairwise comparisons. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, pp. 2212–2220, (2019)

[18]

Binns, R.: On the apparent conflict between individual and group fairness. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAT* ’20, pp. 514–524, (2020)

[19]

Bobadilla J, Lara-Cabrera R, González-Prieto Á, and Ortega F Deepfair: deep learning for improving fairness in recommender systems Int. J. Interact Multim. Artif. Intell. 2021 6 6 86-94

[20]

Boratto L, Fenu G, and Marras M Connecting user and item perspectives in popularity debiasing for collaborative recommendation Inf. Process. Manag. 2021 58 1

[21]

Boratto L, Fenu G, and Marras M Interplay between upsampling and regularization for provider fairness in recommender systems User Model User Adapt Interact 2021 31 3 421-455

Digital Library

[22]

Borges, R., Stefanidis, K.: On mitigating popularity bias in recommendations via variational autoencoders. In: SAC ’21: The 36th ACM/SIGAPP Symposium on Applied Computing, pp. 1383–1389, (2021)

[23]

Buolamwini, J., Gebru, T.: Gender shades: Intersectional accuracy disparities in commercial gender classification. In: Conference on Fairness, pp. 77–91. Accountability and Transparency, PMLR (2018)

[24]

Burke, R.: Multisided fairness for recommendation. In: 4th Workshop on Fairness, Accountability, and Transparency in Machine Learning (FAT/ML 2017), (2017)

[25]

Burke, R., Sonboli, N., Mansoury, M., Ordoñez-Gauger, A.: Balanced neighborhoods for fairness-aware collaborative recommendation, (2017)

[26]

Burke, R., Sonboli, N., Ordonez-Gauger, A.: Balanced neighborhoods for multi-sided fairness in recommendation. In: Conference on Fairness, Accountability and Transparency, FAT 2018, Proceedings of Machine Learning Research, vol. 81, pp. 202–214, (2018)

[27]

Chakraborty, A., Messias, J., Benevenuto, F., Ghosh, S., Ganguly, N., Gummadi, K.P.: Who Makes Trends? Understanding Demographic Biases in Crowdsourced Recommendations. In: Proceedings of the Eleventh International Conference on Web and Social Media, ICWSM 2017, pp. 22–31 (2017)

[28]

Chakraborty, A., Patro, G.K., Ganguly, N., Gummadi, K.P., Loiseau, P.: Equality of voice: Towards fair representation in crowdsourced top-k recommendations. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* 2019, pp. 129–138, (2019)

[29]

Chaudhari, H.A., Lin, S., Linda, O.: A general framework for fairness in multistakeholder recommendations. (2020), arXiv:2009.02423

[30]

Chen, J., Dong, H., Wang, X., Feng, F., Wang, M., He, X.: Bias and debias in recommender system: a survey and future directions. ACM Trans. Inf. Syst. (2022)

[31]

Chouldechova A Fair prediction with disparate impact: a study of bias in recidivism prediction instruments Big Data 2017 5 2 153-163

[32]

Cooper, A.F.: Where is the normative proof? assumptions and contradictions in ML fairness research. (2020), CoRR arXiv:2010.10407

[33]

Corbett-Davies, S., Goel, S.: The measure and mismeasure of fairness: a critical review of fair machine learning. (2018), CoRR arXiv:1808.00023

[34]

Cornacchia, G., Narducci, F., Ragone, A.: A general model for fair and explainable recommendation in the loan domain. In: Joint Workshop Proceedings of the 3rd Edition of Knowledge-aware and Conversational Recommender Systems (KaRS) and the 5th Edition of Recommendation in Complex Environments (ComplexRec) co-located with 15th ACM Conference on Recommender Systems (RecSys 2021), (2021)

[35]

Council NR et al. Measuring Racial Discrimination 2004 London National Academies Press

[36]

Cremonesi P and Jannach D Progress in recommender systems research: crisis? What crisis? AI Mag. 2021 42 3 43-54

Digital Library

[37]

da Silva DC, Manzato MG, and Durão FA Exploiting personalized calibration and metrics for fairness recommendation Expert Syst. Appl. 2021 181

[38]

Dash, A., Chakraborty, A., Ghosh, S., Mukherjee, A., Gummadi, K.P.: When the umpire is also a player: Bias in private label product recommendations on e-commerce marketplaces. In: FAccT ’21: 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 873–884, (2021)

[39]

Deldjoo, Y., Anelli, V.W., Zamani, H., Kouki, A.B., Noia, T.D.: Recommender systems fairness evaluation via generalized cross entropy. In: Proceedings of the Workshop on Recommendation in Multi-stakeholder Environments co-located with the 13th ACM Conference on Recommender Systems (RecSys 2019), Copenhagen, Denmark, September 20, 2019, (2019)

[40]

Deldjoo, Y., Di Noia, T., Merra, F.A.: Adversarial machine learning in recommender systems (aml-recsys). In: Proceedings of the 13th International Conference on Web Search and Data Mining, pp. 869–872, (2020)

[41]

Deldjoo, Y., Anelli, V.W., Zamani, H., Bellogin, A., Di Noia, T.: A flexible framework for evaluating user and item fairness in recommender systems. User Modeling and User-Adapted Interaction pp. 1–47, (2021a)

[42]

Deldjoo Y, Bellogin A, and Di Noia T Explaining recommender systems fairness and accuracy through the lens of data characteristics Inf. Process. Manag. 2021 58 5

Digital Library

[43]

Deldjoo Y, Noia TD, and Merra FA A survey on adversarial recommender systems: from attack/defense strategies to generative adversarial networks ACM Comput. Surv. 2021 54 2 1-38

Digital Library

[44]

Deldjoo, Y., Schedl, M., Knees, P.: Content-driven music recommendation: evolution, state of the art, and challenges. (2021d), arXiv preprint arXiv:2107.11803

[45]

Deldjoo, Y., Nazary, F., Ramisa, A., McAuley, J., Pellegrini, G., Bellogin, A., Di Noia, T.: A review of modern fashion recommender systems. ACM Comput. Surv. (2023)

[46]

Delic A, Neidhardt J, Nguyen TN, and Ricci F An observational user study for group recommender systems in the tourism domain J. Inf. Technol. Tour. 2018 19 1–4 87-116

[47]

Di Noia T, Tintarev N, Fatourou P, and Schedl M Recommender systems under European AI regulations Commun. ACM 2022 65 4 69-73

Digital Library

[48]

Dong Q, Xie S, and Li W User-item matching for recommendation fairness IEEE Access 2021 9 130389-130398

[49]

Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.S.: Fairness through awareness. In: Goldwasser S (ed) Innovations in Theoretical Computer Science 2012, pp. 214–226, (2012)

[50]

Edizel B, Bonchi F, Hajian S, Panisson A, and Tassa T Fairecsys: mitigating algorithmic bias in recommender systems Int. J. Data Sci. Anal. 2020 9 2 197-213

[51]

Ekstrand MD, Das A, Burke R, and Diaz F Fairness in information access systems Found. Trends Inf. Retr. 2022 16 1–2 1-177

Digital Library

[52]

Elahi M, Jannach D, Skjærven L, Knudsen E, Sjøvaag H, Tolonen K, Holmstad Ø, Pipkin I, Throndsen E, Stenbom A, Fiskerud E, Oesch A, Vredenberg L, and Trattner C Towards responsible media recommendation AI Ethics 2022 2 103-114

[53]

Farnadi, G., Kouki, P., Thompson, S.K., Srinivasan, S., Getoor, L.: A fairness-aware hybrid recommender system, (2018). arXiv:1809.09030

[54]

Felfernig A, Boratto L, Stettinger M, and Tkali M Group Recommender Systems: An Introduction 2018 Berlin Springer

[55]

Ferraro, A.: Music cold-start and long-tail recommendation: bias in deep representations. In: Bogers T, Said A, Brusilovsky P, Tikk D (eds) Proceedings of the 13th ACM Conference on Recommender Systems, RecSys 2019, pp. 586–590, (2019)

[56]

Ferwerda, B., Ingesson, E., Berndl, M., Schedl, M.: I Don’t Care How Popular You Are! Investigating Popularity Bias From a User’s Perspective. In: Proceedings of the 8th ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR 2023), ACM, Austin, USA, (2023)

[57]

Friedler SA, Scheidegger C, and Venkatasubramanian S The (im)possibility of fairness: different value systems require different mechanisms for fair decision making Commun. ACM 2021 64 4 136-143

Digital Library

[58]

Friedman, A., Knijnenburg, B.P., Vanhecke, K., Martens, L., Berkovsky, S.: Privacy aspects of recommender systems. In: Recommender Systems Handbook, Springer, pp. 649–688, (2015)

[59]

Friedman B and Nissenbaum H Bias in computer systems ACM Trans. Inf. Syst. 1996 14 3 330-347

Digital Library

[60]

Fu, Z., Xian, Y., Gao, R., Zhao, J., Huang, Q., Ge, Y., Xu, S., Geng, S., Shah, C., Zhang, Y., de Melo, G.: Fairness-aware explainable recommendation over knowledge graphs. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2020, pp. 69–78, (2020)

[61]

Ge, Y., Liu, S., Gao, R., Xian, Y., Li, Y., Zhao, X., Pei, C., Sun, F., Ge, J., Ou, W., Zhang, Y.: Towards long-term fairness in recommendation. In: WSDM ’21, The Fourteenth ACM International Conference on Web Search and Data Mining, pp. 445–453, (2021)

[62]

Geyik, S.C., Ambler, S., Kenthapadi, K., Karypis, G.: Fairness-aware ranking in search & recommendation systems with application to linkedin talent search. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, pp. 2221–2231, (2019)

[63]

Ghanem, N., Leitner, S., Jannach, D.: Balancing consumer and business value of recommender systems: a simulation-based analysis. E-Commerce Research and Applications forthcoming, (2022)

[64]

Gharahighehi A, Vens C, and Pliakos K Fair multi-stakeholder news recommender system with hypergraph ranking Inf. Process. Manag. 2021 58 5

Digital Library

[65]

Ghosh, A., Dutt, R., Wilson, C.: When fair ranking meets uncertain inference. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1033–1043, (2021a)

[66]

Ghosh, A., Genuit, L., Reagan, M.: Characterizing intersectional group fairness with worst-case comparisons. In: Artificial Intelligence Diversity, Belonging, Equity, and Inclusion, PMLR, pp. 22–34, (2021b)

[67]

Giannakas, T., Sermpezis, P., Giovanidis, A., Spyropoulos, T., Arvanitakis, G.: Fairness in network-friendly recommendations. In: 22nd IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks, WoWMoM 2021, pp. 71–80, (2021)

[68]

Gómez, E., Zhang, C.S., Boratto, L., Salamó, M., Marras, M.: The winner takes it all: Geographic imbalance and provider (un)fairness in educational recommender systems. In: SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1808–1812, (2021)

[69]

Gorantla, S., Deshpande, A., Louis, A.: On the problem of underranking in group-fair ranking. In: Proceedings of the 38th International Conference on Machine Learning, ICML 2021, Proceedings of Machine Learning Research, vol. 139, pp. 3777–3787, (2021)

[70]

Grgic-Hlaca, N., Zafar, M.B., Gummadi, K.P., Weller, A.: The case for process fairness in learning: Feature selection for fair decision making. In: NIPS Symposium on Machine Learning and the Law, Barcelona, Spain, vol. 1, pp. 2, (2016)

[71]

Gunawardana A, Shani G, and Yogev S Rokach L, Shapira B, and Ricci F Evaluating recommender systems Recommender Systems Handbook 2022 Berlin Springer 547-601

[72]

Gundersen, O.E., Kjensmo, S.: State of the art: Reproducibility in artificial intelligence. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th Innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), pp. 1644–1651, (2018)

[73]

Gupta, A., Johnson, E., Payan, J., Roy, A.K., Kobren, A., Panda, S., Tristan, J.B., Wick, M.: Online post-processing in rankings for fair utility maximization. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining, WSDM ’21, pp. 454–462, (2021)

[74]

Hao, Q., Xu, Q., Yang, Z., Huang, Q.: Pareto optimality for fairness-constrained collaborative filtering. In: MM ’21: ACM Multimedia Conference, ACM, pp. 5619–5627, (2021)

[75]

Harper, F.M., Konstan, J.A.: The MovieLens datasets: history and context. ACM Trans. Interact Intell. Syst. 5(4), (2015)

[76]

Htun, N.N., Lecluse, E., Verbert, K.: Perception of fairness in group music recommender systems. In: 26th International Conference on Intelligent User Interfaces, pp. 302–306, (2021)

[77]

Jannach, D., Adomavicius, G.: Price and profit awareness in recommender systems. In: Proceedings of the ACM RecSys 2017 Workshop on Value-Aware and Multi-Stakeholder Recommendation, (2017)

[78]

Jannach D and Bauer C Escaping the McNamara fallacy: towards more impactful recommender systems research AI Mag. 2020 41 4 79-95

Digital Library

[79]

Jannach D and Jugovac M Measuring the business value of recommender systems ACM TMIS 2019 10 4 1-23

Digital Library

[80]

Jannach D, Zanker M, Felfernig A, and Friedrich G Recommender Systems-An Introduction 2010 Cambridge Cambridge University Press

[81]

Jannach, D., Zanker, M., Ge, M., Gröning, M.: Recommender systems in computer science and information systems-a landscape of research. In: 13th International Conference on Electronic Commerce and Web Technologies (EC-Web 2012), pp. 76–87, (2012)

[82]

Jannach D, Lerche L, Kamehkhosh I, and Jugovac M What recommenders recommend: an analysis of recommendation biases and possible countermeasures User Model. User-Adap. Inter. 2015 25 5 427-491

Digital Library

[83]

Jannach D, Resnick P, Tuzhilin A, and Zanker M Recommender systems-beyond matrix completion Commun. ACM 2016 59 11 94-102

Digital Library

[84]

Jannach D, Pu P, Ricci F, and Zanker M Recommender systems: past, present, future AI Mag. 2021 42 3 3-6

Digital Library

[85]

Jugovac M, Jannach D, and Lerche L Efficient optimization of multiple recommendation quality factors according to individual user tendencies Expert Syst. Appl. 2017 81 321-331

Digital Library

[86]

Kaya, M., Bridge, D., Tintarev, N.: Ensuring Fairness in Group Recommendations by Rank-Sensitive Balancing of Relevance, pp. 101–110, (2020)

[87]

Kirnap, Ö., Diaz, F., Biega, A., Ekstrand, M.D., Carterette, B., Yilmaz, E.: Estimation of fair ranking metrics with incomplete judgments. In: WWW ’21: The Web Conference 2021, pp. 1065–1075, (2021)

[88]

Kitchenham BA, Brereton P, Budgen D, Turner M, Bailey J, and Linkman SG Systematic literature reviews in software engineering-a systematic literature review Inf. Softw. Technol. 2009 51 1 7-15

Digital Library

[89]

Kleinberg, J.M., Mullainathan, S., Raghavan, M.: Inherent trade-offs in the fair determination of risk scores. In: 8th Innovations in Theoretical Computer Science Conference, ITCS, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, LIPIcs, vol. 67, pp. 43:1–43:23, (2017)

[90]

Koprinska, I., Yacef, K.: People-to-people reciprocal recommenders. In: Recommender Systems Handbook, Springer, pp. 545–567, (2015)

[91]

Koshiyama AS, Kazim E, and Treleaven PC Algorithm auditing: Managing the legal, ethical, and technological risks of artificial intelligence, machine learning, and associated algorithms Computer 2022 55 4 40-50

[92]

Koutsopoulos, I., Halkidi, M.: Efficient and fair item coverage in recommender systems. In: 2018 IEEE 16th Intl Conf on Dependable, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech), IEEE, pp. 912–918, (2018)

[93]

Krafft, T.D., Hauer, M.P., Zweig, K.A.: Why do we need to be bots? what prevents society from detecting biases in recommendation systems. In: Bias and Social Aspects in Search and Recommendation - First International Workshop, BIAS 2020, vol 1245, pp. 27–34, (2020)

[94]

Kusner, M.J., Loftus, J., Russell, C., Silva, R.: Counterfactual fairness. In: Advances in Neural Information Processing Systems, vol. 30, (2017)

[95]

Li, Y., Chen, H., Fu, Z., Ge, Y., Zhang, Y.: User-oriented fairness in recommendation. In: Proceedings of The Web Conference 2021, WWW ’21, pp. 624–632, (2021a)

[96]

Li, Y., Chen, H., Xu, S., Ge, Y., Zhang, Y.: Towards personalized fairness based on causal notion. In: 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21, pp. 1054–1063, (2021b)

[97]

Li, Y., Ge, Y., Zhang, Y.: Tutorial on fairness of machine learning in recommender systems. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2654–2657, (2021c)

[98]

Li, Y., Chen, H., Xu, S., Ge, Y., Tan, J., Liu, S., Zhang, Y.: Fairness in recommendation: a survey. (2022), CoRR arXiv:2205.13619

[99]

Lin, C., Liu, X., Xv, G., Li, H.: Mitigating sentiment bias for recommender systems. In: SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 31–40, (2021)

[100]

Lin, K., Sonboli, N., Mobasher, B., Burke, R.: Crank up the volume: Preference bias amplification in collaborative recommendation. In: Proceedings of the Workshop on Recommendation in Multi-stakeholder Environments co-located with the 13th ACM Conference on Recommender Systems (RecSys 2019), CEUR Workshop Proceedings, vol. 2440, (2019)

[101]

Lin, K., Sonboli, N., Mobasher, B., Burke, R.: Calibration in collaborative filtering recommender systems: a user-centered analysis. In: HT ’20: 31st ACM Conference on Hypertext and Social Media, pp. 197–206, (2020)

[102]

Liu, W., Liu, F., Tang, R., Liao, B., Chen, G., Heng, P.: Balancing between accuracy and fairness for interactive recommendation with reinforcement learning. In: Advances in Knowledge Discovery and Data Mining-24th Pacific-Asia Conference, PAKDD 2020, vol. 12084, pp. 155–167, (2020)

[103]

Malecek, L., Peska, L.: Fairness-preserving group recommendations with user weighting. In: Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization, pp. 4–9, (2021)

[104]

Mansoury, M., Mobasher, B., Burke, R., Pechenizkiy, M.: Bias disparity in collaborative recommendation: Algorithmic evaluation and comparison. In: Proceedings of the Workshop on Recommendation in Multi-stakeholder Environments co-located with the 13th ACM Conference on Recommender Systems (RecSys 2019), (2019)

[105]

Masthoff J and Delic A Rokach L, Shapira B, Kantor P, and Ricci F Group recommender systems: beyond preference aggregation Recommender Systems Handbook 2022 Berlin Springer

[106]

Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A survey on bias and fairness in machine learning. ACM Comput Surv 54(6), (2021)

[107]

Mehrotra, R., McInerney, J., Bouchard, H., Lalmas, M., Diaz, F.: Towards a fair marketplace: Counterfactual evaluation of the trade-off between relevance, fairness & satisfaction in recommendation systems. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM 2018, pp 2243–2251, (2018)

[108]

Melchiorre AB, Rekabsaz N, Parada-Cabaleiro E, Brandl S, Lesota O, and Schedl M Investigating gender fairness of recommendation algorithms in the music domain Inf. Process. Manag. 2021 58 5

Digital Library

[109]

Miller T Explanation in artificial intelligence: Insights from the social sciences Artif. Intell. 2019 267 1-38

[110]

Misztal-Radecka J and Indurkhya B Bias-aware hierarchical clustering for detecting the discriminated groups of users in recommendation systems Information Processing & Management 2021 58 3

Digital Library

[111]

Mladenov, M., Hsu, C., Jain, V., Ie, E., Colby, C., Mayoraz, N., Pham, H., Tran, D., Vendrov, I., Boutilier, C.: RecSim NG: toward principled uncertainty modeling for recommender ecosystems. (2021), CoRR arXiv:2103.08057

[112]

Morina, G., Oliinyk, V., Waton, J., Marusic, I., Georgatzis, K.: Auditing and achieving intersectional fairness in classification problems. (2019), arXiv preprint arXiv:1911.01468

[113]

Moscati, M., Parada-Cabaleiro, E., Deldjoo, Y., Zangerle, E., Schedl, M.: Music4all-onion. a large-scale multi-faceted content-centric music recommendation dataset. In: Proceedings of the 31th ACM International Conference on Information & Knowledge Management (CIKM’22), (2022)

[114]

Mulligan DK, Kroll JA, Kohli N, and Wong RY This thing called fairness: Disciplinary confusion realizing a value in technology Proc ACM Hum Comput Interact 2019 3 CSCW 119:1-119:36

Digital Library

[115]

Naghiaei, M., Rahmani, H.A., Deldjoo, Y.: CPFair: Personalized Consumer and Producer Fairness Re-ranking for Recommender Systems. In: SIGIR ’22SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, (2022)

[116]

Narayanan, A.: 21 definitions of fairness and their politics. Tutorial at FAT* 2018, (2018)

[117]

Ntoutsi, E., Fafalios, P., Gadiraju, U., Iosifidis, V., Nejdl, W., Vidal, M., Ruggieri, S., Turini, F., Papadopoulos, S., Krasanakis, E., Kompatsiaris, I., Kinder-Kurlanda, K., Wagner, C., Karimi, F., Fernández, M., Alani, H., Berendt, B., Kruegel, T., Heinze, C., Broelemann, K., Kasneci, G., Tiropanis, T., Staab, S.: Bias in data-driven artificial intelligence systems - an introductory survey. WIREs Data Mining Knowl Discov 10(3), (2020)

[118]

Nunes I and Jannach D A systematic review and taxonomy of explanations in decision support and recommender systems User-Modeling and User-Adapted Interaction 2017 27 3–5 393-444

Digital Library

[119]

Oh, J., Park, S., Yu, H., Song, M., Park, S.T.: Novel recommendation based on personal popularity tendency. In: ICDM ’11, pp 507–516, (2011)

[120]

Olteanu A, Castillo C, Diaz F, and Kiciman E Social data: Biases, methodological pitfalls, and ethical boundaries Frontiers Big Data 2019 2 13

[121]

Patro, G.K., Biswas, A., Ganguly, N., Gummadi, K.P., Chakraborty, A.: Fairrec: Two-sided fairness for personalized recommendations in two-sided platforms. In: WWW ’20: The Web Conference 2020, pp 1194–1204, (2020)

[122]

Pedreshi, D., Ruggieri, S., Turini, F.: Discrimination-aware data mining. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’08, p 560–568, (2008)

[123]

Pessach D and Shmueli E A review on fairness in machine learning ACM Computing Surveys (CSUR) 2022 55 3 1-44

Digital Library

[124]

Pitoura E, Stefanidis K, and Koutrika G Fairness in rankings and recommendations: an overview VLDB J. 2022 31 3 431-458

Digital Library

[125]

Qiu, R., Wang, S., Chen, Z., Yin, H., Huang, Z.: CausalRec: Causal Inference for Visual Debiasing in Visually-Aware Recommendation. In: MM ’21: ACM Multimedia Conference, pp 3844–3852, (2021)

[126]

Rahmani, H.A., Deldjoo, Y., di Noia, T.: The role of context fusion on accuracy, beyond-accuracy, and fairness of point-of-interest recommendation systems. Expert Systems with Applications p 117700, (2022a)

[127]

Rahmani, H.A., Deldjoo, Y., Tourani, A., Naghiaei, M.: The unfairness of active users and popularity bias in point-of-interest recommendation. In: Advances in Bias and Fairness in Information Retrieval - Third International Workshop, BIAS 2022, Springer, Communications in Computer and Information Science, vol 1610, pp 56–68, (2022b)

[128]

Rahmani, H.A., Naghiaei, M., Dehghan, M., Aliannejadi, M.: Experiments on generalizability of user-oriented fairness in recommender systems. In: SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 2755–2764, (2022c)

[129]

Rahmani, H.A., Naghiaei, M., Tourani, A., Deldjoo, Y.: Exploring the impact of temporal bias in point-of-interest recommendation. In: Proceedings of the 16th ACM Conference on Recommender Systems, (2022d)

[130]

Rastegarpanah, B., Gummadi, K.P., Crovella, M.: Fighting fire with fire: Using antidote data to improve polarization and fairness of recommender systems. In: Proceedings of the twelfth ACM International Conference on Web Search and Data Mining, pp 231–239, (2019)

[131]

Rawls, J.: Justice as fairness: A restatement. Harvard University Press (2001)

[132]

Riederer, C., Chaintreau, A.: The price of fairness in location based advertising. In: FATREC’17, (2017)

[133]

Rohde, D., Bonner, S., Dunlop, T., Vasile, F., Karatzoglou, A.: Recogym: A reinforcement learning environment for the problem of product recommendation in online advertising. arXiv preprint arXiv:1808.00720, (2018)

[134]

Schelenz, L.: Diversity-aware Recommendations for Social Justice? Exploring User Diversity and Fairness in Recommender Systems. In: Adjunct Publication of the 29th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2021, pp 404–410, (2021)

[135]

Selbst, A.D., Boyd, D., Friedler, S.A., Venkatasubramanian, S., Vertesi, J.: Fairness and abstraction in sociotechnical systems. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* ’19, p 59–68, (2019)

[136]

Serbos, D., Qi, S., Mamoulis, N., Pitoura, E., Tsaparas, P.: Fairness in package-to-group recommendations. In: Proceedings of the 26th International Conference on World Wide Web, WWW 2017, pp 371–379, (2017)

[137]

Seymen, S., Abdollahpouri, H., Malthouse, E.C.: A unified optimization toolbox for solving popularity bias, fairness, and diversity in recommender systems. In: Proceedings of the 1st Workshop on Multi-Objective Recommender Systems (MORS 2021) co-located with 15th ACM Conference on Recommender Systems (RecSys 2021), CEUR Workshop Proceedings, vol 2959, (2021)

[138]

Shakespeare, D., Porcaro, L., Gómez, E., Castillo, C.: Exploring artist gender bias in music recommendation. In: Proceedings of the Workshops on Recommendation in Complex Scenarios and the Impact of Recommender Systems co-located with 14th ACM Conference on Recommender Systems (RecSys 2020), CEUR Workshop Proceedings, vol 2697, (2020)

[139]

Shen, T., Li, J., Bouadjenek, M.R., Mai, Z., Sanner, S.: Towards understanding and mitigating unintended biases in language model-driven conversational recommendation. Information Processing and Management In press, (2023)

[140]

Shrestha YR and Yang Y Fairness in algorithmic decision-making: Applications in multi-winner voting, machine learning, and recommender systems Algorithms 2019 12 9 199

[141]

Slokom M, Hanjalic A, and Larson M Towards user-oriented privacy for recommender system data: A personalization-based approach to gender obfuscation for user profiles Information Processing & Management 2021 58 6

Digital Library

[142]

Sonboli, N., Burke, R., Mattei, N., Eskandanian, F., Gao, T.: “and the winner is...”: Dynamic lotteries for multi-group fairness-aware recommendation. In: FAccTRec Workshop: Responsible Recommendation (RecSys ’20), (2020)

[143]

Sonboli, N., Smith, J.J., Cabral Berenfus, F., Burke, R., Fiesler, C.: Fairness and transparency in recommendation: The users’ perspective. In: Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization, p 274–279, (2021)

[144]

Srivastava, M., Heidari, H., Krause, A.: Mathematical notions vs. human perception of fairness: A descriptive approach to fairness for machine learning. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, pp 2459–2468, (2019)

[145]

Steck, H.: Calibrated recommendations. In: Proceedings of the 12th ACM Conference on Recommender Systems, pp 154–162, (2018)

[146]

Stratigi, M., Kondylakis, H., Stefanidis, K.: Fairness in group recommendations in the health domain. In: 33rd IEEE International Conference on Data Engineering, ICDE 2017, pp 1481–1488, (2017)

[147]

Stratigi, M., Nummenmaa, J., Pitoura, E., Stefanidis, K.: Fair sequential group recommendations. In: Proceedings of the 35th Annual ACM Symposium on Applied Computing, pp 1443–1452, (2020)

[148]

Sühr, T., Hilgard, S., Lakkaraju, H.: Does Fair Ranking Improve Minority Outcomes? Understanding the Interplay of Human and Algorithmic Biases in Online Hiring, p 989–999, (2021)

[149]

Sun, W., Khenissi, S., Nasraoui, O., Shafto, P.: Debiasing the human-recommender system feedback loop in collaborative filtering. In: Companion of The 2019 World Wide Web Conference, WWW 2019, ACM, pp 645–651, (2019)

[150]

Tintarev, N., Masthoff, J.: Beyond explaining single item recommendations. In: Recommender Systems Handbook, Springer, pp 711–756, (2022)

[151]

Trattner C, Jannach D, Motta E, Meijer IC, Diakopoulos N, Elahi M, Opdahl AL, Tessem B, Borch N, Fjeld M, Øvrelid L, Smedt KD, and Moe H Responsible Media Technology and AI: Challenges and Research Directions AI and Ethics 2022 2 585-594

[152]

Tsintzou, V., Pitoura, E., Tsaparas, P.: Bias disparity in recommendation systems. In: Proceedings of the Workshop on Recommendation in Multi-stakeholder Environments co-located with the 13th ACM Conference on Recommender Systems (RecSys 2019), vol 2440, (2019)

[153]

Verma, S., Rubin, J.: Fairness definitions explained. In: Brun Y, Johnson B, Meliou A (eds) Proceedings of the International Workshop on Software Fairness, FairWare@ICSE 2018, pp 1–7, (2018)

[154]

Verma, S., Gao, R., Shah, C.: Facets of fairness in search and recommendation. In: Bias and Social Aspects in Search and Recommendation - First International Workshop, BIAS 2020, Communications in Computer and Information Science, vol 1245, pp 1–11, (2020)

[155]

Wan, M., Ni, J., Misra, R., McAuley, J.: Addressing marketing bias in product recommendations. In: Proceedings of the 13th International Conference on Web Search and Data Mining, pp 618–626, (2020)

[156]

Wang, C., Wang, K., Bian, A., Islam, R., Keya, K.N., Foulds, J.R., Pan, S.: Do humans prefer debiased AI algorithms? A case study in career recommendation. In: IUI 2022: 27th International Conference on Intelligent User Interfaces, pp 134–147, (2022a)

[157]

Wang, X., Thain, N., Sinha, A., Prost, F., Chi, E.H., Chen, J., Beutel, A.: Practical compositional fairness: Understanding fairness in multi-component recommender systems. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pp 436–444, (2021)

[158]

Wang, Y., Ma, W., Zhang, M., Liu, Y., Ma, S.: A survey on the fairness of recommender systems. ACM TOIS forthcoming, (2022b)

[159]

Weydemann, L., Sacharidis, D., Werthner, H.: Defining and measuring fairness in location recommendations. In: Proceedings of the 3rd ACM SIGSPATIAL International Workshop on Location-based Recommendations, Geosocial Networks and Geoadvertising, LocalRec@SIGSPATIAL 2019, pp 6:1–6:8, (2019)

[160]

Wu, C., Wu, F., Wang, X., Huang, Y., Xie. X.: Fairness-aware news recommendation with decomposed adversarial learning. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, pp 4462–4469, (2021a)

[161]

Wu, Y., Cao, J., Xu, G., Tan, Y.: TFROM: A two-sided fairness-aware recommendation model for both customers and providers. In: SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, pp 1013–1022, (2021b)

[162]

Wundervald BD Cluster-based quotas for fairness improvements in music recommendation systems Int J Multim Inf Retr 2021 10 1 25-32

[163]

Xia, B., Yin, J., Xu, J., Li, Y.: We-rec: A fairness-aware reciprocal recommendation based on walrasian equilibrium. Knowl Based Syst 182, (2019)

[164]

Xiao B and Benbasat I E-commerce product recommendation agents: Use, characteristics, and impact MIS Q. 2007 31 1 137-209

[165]

Xiao Y, Pei Q, Yao L, Yu S, Bai L, and Wang X An enhanced probabilistic fairness-aware group recommendation by incorporating social activeness J. Netw. Comput. Appl. 2020 156

Digital Library

[166]

Yadav, H., Du, Z., Joachims, T.: Policy-gradient training of fair and unbiased ranking functions. In: SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 1044–1053, (2021)

[167]

Yao, S., Huang, B.: Beyond parity: Fairness objectives for collaborative filtering. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS ’17, p 2925–2934, (2017)

[168]

Zafar, M.B., Valera, I., Gomez Rodriguez, M., Gummadi, K.P.: Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In: Proceedings of the 26th International Conference on World Wide Web, pp 1171–1180, (2017)

[169]

Zehlike, M., Yang, K., Stoyanovich, J.: Fairness in ranking, part i: Score-based ranking. ACM Comput Surv Just Accepted, (2022a)

[170]

Zehlike, M., Yang, K., Stoyanovich, J.: Fairness in ranking, part ii: Learning-to-rank and recommender systems. ACM Comput Surv forthcoming, (2022b)

[171]

Zhang, J., Bareinboim, E.: Fairness in decision-making—the causal explanation formula. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 32, (2018)

[172]

Zhao, Z., Chen, J., Zhou, S., He, X., Cao, X., Zhang, F., Wu, W.: Popularity bias is not always evil: Disentangling benign and harmful bias for recommendation. IEEE Transactions on Knowledge & Data Engineering (01):1–13, (2022)

[173]

Zheng, Y., Dave, T., Mishra, N., Kumar, H.: Fairness in reciprocal recommendations: A speed-dating study. In: Adjunct Publication of the 26th Conference on User Modeling, Adaptation and Personalization, UMAP 2018, pp 29–34, (2018)

[174]

Zhou, M., Zhang, J., Adomavicius, G.: Longitudinal impact of preference biases on recommender systems’ performance. Kelley School of Business (2021-10), (2021)

[175]

Zhu, Q., Zhou, A., Sun, Q., Wang, S., Yang, F.: FMSR: A fairness-aware mobile service recommendation method. In: 2018 IEEE International Conference on Web Services, ICWS 2018, San Francisco, CA, USA, July 2-7, 2018, IEEE, pp 171–178, (2018a)

[176]

Zhu Q, Sun Q, Li Z, and Wang S FARM: A fairness-aware recommendation method for high visibility and low visibility mobile apps IEEE Access 2020 8 122747-122756

[177]

Zhu, Z., Hu, X., Caverlee, J.: Fairness-aware tensor-based recommendation. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM 2018, pp 1153–1162, (2018b)

[178]

Zhu, Z., Wang, J., Zhang, Y., Caverlee, J.: Fairness-aware recommendation of information curators. (2018c), arXiv:1809.03040

[179]

Zhu, Z., Wang, J., Caverlee, J.: Measuring and mitigating item under-recommendation bias in personalized ranking systems. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, p 449–458, (2020b)

[180]

Zhu, Z., Kim, J., Nguyen, T., Fenton, A., Caverlee, J.: Fairness among new items in cold start recommender systems. In: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21, pp 767–776, (2021)

Cited By

He ZXie ZSteck HLiang DJha RKallus NMcAuley JNejdl WAuer SKarras OCha MMoens MNajork M(2025)Reindex-Then-Adapt: Improving Large Language Models for Conversational RecommendationProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining10.1145/3701551.3703573(866-875)Online publication date: 10-Mar-2025
https://dl.acm.org/doi/10.1145/3701551.3703573
Zhao YWang YLiu YCheng XAggarwal CDerr T(2025)Fairness and Diversity in Recommender Systems: A SurveyACM Transactions on Intelligent Systems and Technology10.1145/366492816:1(1-28)Online publication date: 3-Jan-2025
https://dl.acm.org/doi/10.1145/3664928
Yong JKim C(2024)Hybrid Inductive Graph Method for Matrix CompletionInternational Journal of Data Warehousing and Mining10.4018/IJDWM.34536120:1(1-16)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.4018/IJDWM.345361
Show More Cited By

Recommendations

Fairness in Recommender Systems: Evaluation Approaches and Assurance Strategies
With the wide application of recommender systems, the potential impacts of recommender systems on customers, item providers and other parties have attracted increasing attention. Fairness, which is the quality of treating people equally, is also becoming ...
A Survey on the Fairness of Recommender Systems
Recommender systems are an essential tool to relieve the information overload challenge and play an important role in people’s daily lives. Since recommendations involve allocations of social resources (e.g., job recommendation), an important issue is ...
A survey on fairness-aware recommender systems
Abstract
As information filtering services, recommender systems have extremely enriched our daily life by providing personalized suggestions and facilitating people in decision-making, which makes them vital and indispensable to human society in the ...
Highlights
- This survey provides an in-depth, scientific taxonomy of fairness-aware recommender systems.
- This survey explores the relationship between the fairness and trustworthiness of recommender systems.
- Several future directions are ...

Comments

Information & Contributors

Information

Published In

cover image User Modeling and User-Adapted Interaction

User Modeling and User-Adapted Interaction Volume 34, Issue 1

Mar 2024

255 pages

Issue’s Table of Contents

© The Author(s) 2023.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 24 April 2023

Accepted: 17 March 2023

Received: 27 June 2022

Author Tags

Qualifiers

Review-article

Funding Sources

Politecnico di Bari

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

He ZXie ZSteck HLiang DJha RKallus NMcAuley JNejdl WAuer SKarras OCha MMoens MNajork M(2025)Reindex-Then-Adapt: Improving Large Language Models for Conversational RecommendationProceedings of the Eighteenth ACM International Conference on Web Search and Data Mining10.1145/3701551.3703573(866-875)Online publication date: 10-Mar-2025
https://dl.acm.org/doi/10.1145/3701551.3703573
Zhao YWang YLiu YCheng XAggarwal CDerr T(2025)Fairness and Diversity in Recommender Systems: A SurveyACM Transactions on Intelligent Systems and Technology10.1145/366492816:1(1-28)Online publication date: 3-Jan-2025
https://dl.acm.org/doi/10.1145/3664928
Yong JKim C(2024)Hybrid Inductive Graph Method for Matrix CompletionInternational Journal of Data Warehousing and Mining10.4018/IJDWM.34536120:1(1-16)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.4018/IJDWM.345361
Aird AFarastu PSun JStefancová EAll CVoida AMattei NBurke R(2024)Dynamic Fairness-aware Recommendation Through Multi-agent Social ChoiceACM Transactions on Recommender Systems10.1145/36906533:2(1-35)Online publication date: 28-Sep-2024
https://dl.acm.org/doi/10.1145/3690653
Rodriguez JTommasel A(2024)Leveraging User History with Transformers for News Clicking: The DArgk ApproachProceedings of the Recommender Systems Challenge 202410.1145/3687151.3687161(48-52)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3687151.3687161
Tommasel A(2024)Fairness Matters: A look at LLM-generated group recommendationsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688182(993-998)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688182
Tomita YYokoyama T(2024)Fair Reciprocal Recommendation in Matching MarketsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688130(209-218)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688130
Gómez EContreras DBoratto LSalamo M(2024)AMBAR: A dataset for Assessing Multiple Beyond-Accuracy RecommendersProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688067(137-147)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688067
Kolb T(2024)Enhancing Cross-Domain Recommender Systems with LLMs: Evaluating Bias and Beyond-Accuracy MeasuresProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688027(1388-1394)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688027
Souza L(2024)Fairness Explanations in Recommender SystemsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688020(1353-1354)Online publication date: 8-Oct-2024
https://dl.acm.org/doi/10.1145/3640457.3688020
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents