research-article

Open access

Make Fairness More Fair: Fair Item Utility Estimation and Exposure Re-Distribution

Authors:

Shaoping MaAuthors Info & Claims

KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Pages 1868 - 1877

https://doi.org/10.1145/3534678.3539354

Published: 14 August 2022 Publication History

Abstract

The item fairness issue has become one of the significant concerns with the development of recommender systems in recent years, focusing on whether items' exposures are consistent with their utilities. So the measurement of item unfairness depends on the modeling of item utility, and most previous approaches estimated item utility simply based on user-item interaction logs in recommender systems. The Click-through rate (CTR) is the most popular one. However, we argue that these types of item utilities (named observed utility here) measurements may result in unfair exposures of items. The number of exposure for each item is uneven, and recommendation methods select the exposure audiences (users).

In this work, we propose the concept of items' fair utility, defined as the proportion of users who are interested in the item among all users. Firstly, we conduct a large-scale random exposure experiment to collect the fair utility in a real-world recommender application. Significant differences are observed between the fair utility and the widely used observed utility (CTR). Then, intending to obtain fair utility at a low cost, we propose an exploratory task for real-time estimations of fair utility with handy historical interaction logs. Encouraging results are achieved, validating the feasibility of fair utility projections. Furthermore, we present a fairness-aware re-distribution framework and conduct abundant simulation experiments, adopting fair utility to improve fairness and overall recommendation performance at the same time. Online and offline results show that both item fairness and recommendation quality can be improved simultaneously by introducing item fair utility.

Supplemental Material

MP4 File

Presentation video

Download
100.44 MB

References

[1]

2018. Herfindahl-Hirschman Index. (2018). https://www.justice.gov/atr/herfindahl-hirschman-index

[2]

Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Li Wei, Yi Wu, Lukasz Heldt, Zhe Zhao, Lichan Hong, Ed H. Chi, and Cristos Goodrow. 2019. Fairness in Recommendation Ranking through Pairwise Comparisons. In KDD 2019.

Digital Library

[3]

Asia J Biega, Krishna P Gummadi, and GerhardWeikum. 2018. Equity of attention: Amortizing individual fairness in rankings. In SIGIR '18. 405--414.

Digital Library

[4]

Robin Burke. 2017. Multisided fairness for recommendation. (2017).

[5]

Robin Burke, Nasim Sonboli, and Aldo Ordonez-Gauger. 2018. Balanced Neighborhoods for Multi-sided Fairness in Recommendation. In FAT 18 (Proceedings of Machine Learning Research), Vol. 81. PMLR, 202--214.

[6]

Jiawei Chen, Hande Dong, Yang Qiu, Xiangnan He, Xin Xin, Liang Chen, Guli Lin, and Keping Yang. 2021. AutoDebias: Learning to debias for recommendation. In SIGIR 21. 21--30.

[7]

J. Chen, H. Dong, X. Wang, F. Feng, M. Wang, and X. He. 2020. Bias and debias in recommender system: A survey and future directions. (2020).

[8]

Yashar Deldjoo, Vito Walter Anelli, Hamed Zamani, Alejandro Bellogín Kouki, and Tommaso Di Noia. 2019. Recommender Systems Fairness Evaluation via Generalized Cross Entropy. In RecSys 19 (CEUR Workshop Proceedings), Vol. 2440.

[9]

Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard S. Zemel. 2012. Fairness through awareness. In Innovations in Theoretical Computer Science 2012, Cambridge, MA, USA, January 8-10, 2012. ACM, 214--226.

Digital Library

[10]

Michael D. Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D. Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2018. All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness. In FAT 18 (Proceedings of Machine Learning Research), Vol. 81. PMLR, 172--186.

[11]

Michael D. Ekstrand, Mucun Tian, Mohammed R. Imran Kazi, Hoda Mehrpouyan, and Daniel Kluver. 2018. Exploring author gender in book rating and recommendation. In RecSys 18. ACM, 242--250.

[12]

Zuohui Fu, Yikun Xian, Ruoyuan Gao, Jieyu Zhao, Qiaoying Huang, Yingqiang Ge, Shuyuan Xu, Shijie Geng, Chirag Shah, Yongfeng Zhang, and Gerard de Melo. 2020. Fairness-Aware Explainable Recommendation over Knowledge Graphs. In SIGIR 20. ACM, 69--78. https://doi.org/10.1145/3397271.3401051

Digital Library

[13]

Yingqiang Ge, Shuchang Liu, Ruoyuan Gao, Yikun Xian, Yunqi Li, Xiangyu Zhao, Changhua Pei, Fei Sun, Junfeng Ge, Wenwu Ou, et al. 2021. Towards Long-term Fairness in Recommendation. In WSDM 21. 445--453.

[14]

Toshihiro Kamishima and Shotaro Akaho. 2017. Considerations on Recommendation Independence for a Find-Good-Items Task. In Proceedings of Workshop on Responsible Recommendation. 6. https://doi.org/10.18122/B2871W

[15]

Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. 2018. Recommendation Independence. In FAT 18, Vol. 81. 187--201.

[16]

Chen Karako and Putra Manggala. 2018. Using Image Fairness Representations in Diversity-Based Re-ranking for Recommendations. In UMAP 18.

[17]

D. Lee, S. Kang, H. Ju, C. Park, and H. Yu. 2021. Bootstrapping user and item representations for one-class collaborative filtering. In SIGIR 21. 317--326.

[18]

Jurek Leonhardt, Avishek Anand, and Megha Khosla. 2018. User Fairness in Recommender Systems. In WWW 18. ACM, 101--102.

[19]

Jiacheng Li, Yujie Wang, and Julian McAuley. 2020. Time interval aware selfattention for sequential recommendation. In WSDM 20. 322--330.

[20]

Roger Zhe Li, Julián Urbano, and Alan Hanjalic. 2021. Leave No User Behind: Towards Improving the Utility of Recommender Systems for Non-mainstream Users. In WSDM 21. 103--111.

Digital Library

[21]

Yunqi Li, Hanxiong Chen, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2021. User-oriented Fairness in Recommendation. In WWW 21. 624--632.

[22]

Yunqi Li, Hanxiong Chen, Shuyuan Xu, Yingqiang Ge, and Yongfeng Zhang. 2021. Towards Personalized Fairness based on Causal Notion. (2021).

[23]

Yunqi Li, Yingqiang Ge, and Yongfeng Zhang. 2021. Tutorial on Fairness of Machine Learning in Recommender Systems. SIGIR.

[24]

Dugang Liu, Pengxiang Cheng, Zhenhua Dong, Xiuqiang He, Weike Pan, and Zhong Ming. 2020. Ageneral knowledge distillation framework for counterfactual recommendation via uniform data. In SIGIR 20. 831--840.

[25]

Marco Morik, Ashudeep Singh, Jessica Hong, and Thorsten Joachims. [n. d.]. Controlling Fairness and Bias in Dynamic Learning-to-Rank. In SIGIR 20. 10.

[26]

Harrie Oosterhuis and Maarten de Rijke. 2021. Unifying online and counterfactual learning to rank: A novel counterfactual estimator that effectively utilizes online interventions. In KDD. 463--471.

[27]

Gourab K. Patro, Arpita Biswas, Niloy Ganguly, Krishna P. Gummadi, and Abhijnan Chakraborty. 2020. FairRec: Two-Sided Fairness for Personalized Recommendations in Two-Sided Platforms. In WWW 20. ACM / IW3C2, 1194--1204.

[28]

Dino Pedreschi, Salvatore Ruggieri, and Franco Turini. 2009. Measuring Discrimination in Socially-Sensitive Decision Records. In SDM 2009. SIAM, 581--592.

[29]

Dino Pedreshi, Salvatore Ruggieri, and Franco Turini. 2008. Discrimination-Aware Data Mining. In KDD 2008 (KDD '08). 560--568.

[30]

Bashir Rastegarpanah, Krishna P. Gummadi, and Mark Crovella. 2019. Fighting Fire with Fire: Using Antidote Data to Improve Polarization and Fairness of Recommender Systems. In WSDM 19. ACM, 231--239.

[31]

Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata. 2020. Unbiased recommender learning from missing-not-at-random implicit feedback. In WSDM 20. 501--509.

[32]

Nasim Sonboli, Farzad Eskandanian, Robin Burke, Weiwen Liu, and Bamshad Mobasher. 2020. Opportunistic Multi-aspect Fairness through Personalized Reranking. In UMAP 20. ACM, 239--247.

[33]

Maria Stratigi, Jyrki Nummenmaa, Evaggelia Pitoura, and Kostas Stefanidis. 2020. Fair sequential group recommendations. In Proceedings of the 35th Annual ACM Symposium on Applied Computing. 1443--1452.

Digital Library

[34]

Mengting Wan, Jianmo Ni, Rishabh Misra, and Julian McAuley. 2020. Addressing marketing bias in product recommendations. In WSDM 20. 618--626.

[35]

Chenyang Wang, Weizhi Ma, Min Zhang, Chong Chen, Yiqun Liu, and Shaoping Ma. 2020. Toward dynamic user intention: Temporal evolutionary effects of item relations in sequential recommendation. TOIS 20 39, 2 (2020), 1--33.

[36]

Chenyang Wang, Yi Ren, Weizhi Ma, Min Zhang, Yiqun Liu, and Shaoping Ma. 2021. ReChorus: A comprehensive, efficient, flexible lightweight recommendation algorithm framework. Software Journal 33, 4 (2021), 0--0.

[37]

Xiang Wang, Yaokun Xu, Xiangnan He, Yixin Cao, Meng Wang, and Tat-Seng Chua. 2020. Reinforced negative sampling over knowledge graph for recommendation. In WWW 20. 99--109.

[38]

Tianxin Wei, Fuli Feng, Jiawei Chen, Ziwei Wu, Jinfeng Yi, and Xiangnan He. 2021. Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In KDD 21. 1791--1800.

[39]

Chuhan Wu, Fangzhao Wu, Xiting Wang, Yongfeng Huang, and Xing Xie. 2021. Fairness-aware News Recommendation with Decomposed Adversarial Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 4462--4469.

[40]

Y. Wu, J. Cao, G. Xu, and Y. Tan. 2021. TFROM: A Two-sided Fairness-Aware Recommendation Model for Both Customers and Providers. (2021).

Digital Library

[41]

Longqi Yang, Yin Cui, Yuan Xuan, Chenyang Wang, Serge Belongie, and Deborah Estrin. 2018. Unbiased offline recommender evaluation for missing-not-atrandom implicit feedback. In RecSys 2018. 279--287.

Digital Library

[42]

Tao Yang and Qingyao Ai. 2021. Maximizing Marginal Fairness for Dynamic Learning to Rank. In WWW 21. 137--145.

[43]

Sirui Yao and Bert Huang. 2017. Beyond Parity: Fairness Objectives for Collaborative Filtering. In NIPS 17. 2921--2930.

[44]

Yang Zhang, Fuli Feng, Xiangnan He, Tianxin Wei, Chonggang Song, Guohui Ling, and Yongdong Zhang. 2021. Causal Intervention for Leveraging Popularity Bias in Recommendation. arXiv preprint arXiv:2105.06067 (2021).

[45]

Ziwei Zhu, Yun He, Xing Zhao, Yin Zhang, Jianling Wang, and James Caverlee. 2021. Popularity-Opportunity Bias in Collaborative Filtering. In WSDM 21. 85--93.

[46]

Ziwei Zhu, Xia Hu, and James Caverlee. 2018. Fairness-Aware Tensor-Based Recommendation. In CIKM 18. ACM, 1153--1162.

[47]

Ziwei Zhu, Jingu Kim, Trung Nguyen, Aish Fenton, and James Caverlee. 2021. Fairness among New Items in Cold Start Recommender Systems. (2021).

Cited By

Wu GFeng ZChen SWu HXue XXiao JFan GChen HLi J(2025)FairSort: Learning to Fair Rank for Personalized Recommendations in Two-Sided PlatformsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.350991237:2(641-654)Online publication date: Feb-2025
https://doi.org/10.1109/TKDE.2024.3509912
Zhang CChen WZhang WXu M(2024)Mitigating the Impact of Inaccurate Feedback in Dynamic Learning-to-Rank: A Study of Overlooked Interesting ItemsACM Transactions on Intelligent Systems and Technology10.1145/365398316:1(1-26)Online publication date: 26-Dec-2024
https://dl.acm.org/doi/10.1145/3653983
Dacrema MCastells PBasilico JCremonesi P(2024)Report on the Workshop on Learning and Evaluating Recommendations with Impressions (LERI) at RecSys 2023ACM SIGIR Forum10.1145/3642979.364300157:2(1-8)Online publication date: 22-Jan-2024
https://dl.acm.org/doi/10.1145/3642979.3643001
Show More Cited By

Index Terms

Make Fairness More Fair: Fair Item Utility Estimation and Exposure Re-Distribution
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Towards Personalized Fairness based on Causal Notion
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Recommender systems are gaining increasing and critical impacts on human and society since a growing number of users use them for information seeking and decision making. Therefore, it is crucial to address the potential unfairness problems in ...
Towards Long-term Fairness in Recommendation
WSDM '21: Proceedings of the 14th ACM International Conference on Web Search and Data Mining

As Recommender Systems (RS) influence more and more people in their daily life, the issue of fairness in recommendation is becoming more and more important. Most of the prior approaches to fairness-aware recommendation have been situated in a static or ...
Measuring Item Global Residual Value for Fair Recommendation
SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

In the era of information explosion, numerous items emerge every day, especially in feed scenarios. Due to the limited system display slots and user browsing attention, various recommendation systems are designed not only to satisfy users' personalized ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 2022

5033 pages

ISBN:9781450393850

DOI:10.1145/3534678

General Chairs:
Aidong Zhang
University of Virginia
,
Huzefa Rangwala
Amazon/George Mason University

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2022

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Science Foundation of China
Tsinghua University Guoqiang Research Institute

Conference

KDD '22

Sponsor:

KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 14 - 18, 2022

Washington DC, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
993
Total Downloads

Downloads (Last 12 months)363
Downloads (Last 6 weeks)51

Reflects downloads up to 23 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wu GFeng ZChen SWu HXue XXiao JFan GChen HLi J(2025)FairSort: Learning to Fair Rank for Personalized Recommendations in Two-Sided PlatformsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.350991237:2(641-654)Online publication date: Feb-2025
https://doi.org/10.1109/TKDE.2024.3509912
Zhang CChen WZhang WXu M(2024)Mitigating the Impact of Inaccurate Feedback in Dynamic Learning-to-Rank: A Study of Overlooked Interesting ItemsACM Transactions on Intelligent Systems and Technology10.1145/365398316:1(1-26)Online publication date: 26-Dec-2024
https://dl.acm.org/doi/10.1145/3653983
Dacrema MCastells PBasilico JCremonesi P(2024)Report on the Workshop on Learning and Evaluating Recommendations with Impressions (LERI) at RecSys 2023ACM SIGIR Forum10.1145/3642979.364300157:2(1-8)Online publication date: 22-Jan-2024
https://dl.acm.org/doi/10.1145/3642979.3643001
Xu CYe XWang WPang LXu JChua THui Yang GWang HHan SHauff CZuccon GZhang Y(2024)A Taxation Perspective for Fair Re-rankingProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657766(1494-1503)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657766
Shang YGao CChen JJin DLi YChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Improving Item-side Fairness of Multimodal Recommendation via Modality DebiasingProceedings of the ACM Web Conference 202410.1145/3589334.3648156(4697-4705)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3648156
Hu YLiao TChen JBian JZheng ZChen C(2024)Migrate demographic group for fair Graph Neural NetworksNeural Networks10.1016/j.neunet.2024.106264175:COnline publication date: 17-Jul-2024
https://dl.acm.org/doi/10.1016/j.neunet.2024.106264
Shi XLiu QXie HWu DPeng BShang MLian D(2023)Relieving Popularity Bias in Interactive Recommendation: A Diversity-Novelty-Aware Reinforcement Learning ApproachACM Transactions on Information Systems10.1145/361810742:2(1-30)Online publication date: 8-Nov-2023
https://dl.acm.org/doi/10.1145/3618107
Quan YDing JGao CLi NYi LJin DLi Y(2023)Alleviating Video-length Effect for Micro-video RecommendationACM Transactions on Information Systems10.1145/361782642:2(1-24)Online publication date: 8-Nov-2023
https://dl.acm.org/doi/10.1145/3617826
Ferrari Dacrema MCastells PBasilico JCremonesi P(2023)Workshop on Learning and Evaluating Recommendations with Impressions (LERI)Proceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608756(1248-1251)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608756
Cai QLiu SWang XZuo TXie WYang BZheng DJiang PGai K(2023)Reinforcing User Retention in a Billion Scale Short Video Recommender SystemCompanion Proceedings of the ACM Web Conference 202310.1145/3543873.3584640(421-426)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543873.3584640
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents