research-article

Free access

Debiasing Recommendation by Learning Identifiable Latent Confounders

Authors:

Xiaoying Zhang,

Ruocheng GuoAuthors Info & Claims

KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Pages 3353 - 3363

https://doi.org/10.1145/3580305.3599296

Published: 04 August 2023 Publication History

Abstract

Recommendation systems aim to predict users' feedback on items not exposed to them yet. Confounding bias arises due to the presence of unmeasured variables (e.g., the socio-economic status of a user) that can affect both a user's exposure and feedback. Existing methods either (1) make untenable assumptions about these unmeasured variables or (2) directly infer latent confounders from users' exposure. However, they cannot guarantee the identification of counterfactual feedback, which can lead to biased predictions. In this work, we propose a novel method, i.e., identifiable deconfounder (iDCF), which leverages a set of proxy variables (e.g., observed user features) to resolve the aforementioned non-identification issue. The proposed iDCF is a general deconfounded recommendation framework that applies proximal causal inference to infer the unmeasured confounders and identify the counterfactual feedback with theoretical guarantees. Extensive experiments on various real-world and synthetic datasets verify the proposed method's effectiveness and robustness.

Supplementary Material

MP4 File (rtfp0819-2min-promo.mp4)

Presentation video

Download
18.05 MB

References

[1]

Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. 2019. Invariant risk minimization. arXiv preprint arXiv:1907.02893 (2019).

[2]

Peter Bühlmann. 2020. Invariance, causality and robustness. Statist. Sci., Vol. 35, 3 (2020), 404--426.

[3]

Alexander D'Amour. 2019. On multi-cause causal inference with unobserved confounding: Counterexamples, impossibility, and alternatives. arXiv preprint arXiv:1902.10286 (2019).

[4]

Sihao Ding, Peng Wu, Fuli Feng, Yitong Wang, Xiangnan He, Yong Liao, and Yongdong Zhang. 2022. Addressing Unmeasured Confounder for Recommendation with Sensitivity Analysis. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Washington DC, USA) (KDD '22). Association for Computing Machinery, New York, NY, USA, 305--315. https://doi.org/10.1145/3534678.3539240

Digital Library

[5]

Chongming Gao, Shijun Li, Yuan Zhang, Jiawei Chen, Biao Li, Wenqiang Lei, Peng Jiang, and Xiangnan He. 2022. KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videos. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 3953--3957.

Digital Library

[6]

Justin Grimmer, Dean Knox, and Brandon M Stewart. 2020. Na\"ive regression requires weaker assumptions than factor models to adjust for multiple cause confounding. arXiv preprint arXiv:2007.12702 (2020).

[7]

Xiangnan He, Yang Zhang, Fuli Feng, Chonggang Song, Lingling Yi, Guohui Ling, and Yongdong Zhang. 2022. Addressing Confounding Feature Issue for Causal Recommendation. arXiv preprint arXiv:2205.06532 (2022).

[8]

Miguel A Hernán and James M Robins. 2010. Causal inference.

[9]

Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. 2020. Variational autoencoders and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence and Statistics. PMLR, 2207--2217.

[10]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[11]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).

[12]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer, Vol. 42, 8 (2009), 30--37.

Digital Library

[13]

J. B. Kruskal. 1989. Rank, Decomposition, and Uniqueness for 3-Way and n-Way Arrays. North-Holland Publishing Co., NLD, 7--18.

[14]

Manabu Kuroki and Judea Pearl. 2014. Measurement bias and effect restoration in causal inference. Biometrika, Vol. 101, 2 (2014), 423--437.

[15]

Haochen Liu, Da Tang, Ji Yang, Xiangyu Zhao, Hui Liu, Jiliang Tang, and Youlong Cheng. 2022. Rating Distribution Calibration for Selection Bias Mitigation in Recommendations. In Proceedings of the ACM Web Conference 2022 (Virtual Event, Lyon, France) (WWW '22). Association for Computing Machinery, New York, NY, USA, 2048--2057. https://doi.org/10.1145/3485447.3512078

Digital Library

[16]

Wang Miao, Zhi Geng, and Eric J Tchetgen Tchetgen. 2018a. Identifying causal effects with proxy variables of an unmeasured confounder. Biometrika, Vol. 105, 4 (2018), 987--993.

[17]

Wang Miao, Wenjie Hu, Elizabeth L Ogburn, and Xiao-Hua Zhou. 2022. Identifying effects of multiple treatments in the presence of unmeasured confounding. J. Amer. Statist. Assoc. (2022), 1--15.

[18]

Wang Miao, Xu Shi, and Eric Tchetgen Tchetgen. 2018b. A confounding bridge approach for double negative control inference on causal effects. arXiv preprint arXiv:1808.04945 (2018).

[19]

Abhirup Mondal, Anirban Majumder, and Vineet Chaoji. 2022. ASPIRE: Air Shipping Recommendation for E-Commerce Products via Causal Inference Framework. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Washington DC, USA) (KDD '22). Association for Computing Machinery, New York, NY, USA, 3584--3592. https://doi.org/10.1145/3534678.3539197

Digital Library

[20]

Judea Pearl. 2009. Causality: Models, Reasoning and Inference 2nd ed.). Cambridge University Press.

Digital Library

[21]

James Robins. 1986. A new approach to causal inference in mortality studies with a sustained exposure period-application to control of the healthy worker survivor effect. Mathematical Modelling, Vol. 7, 9 (1986), 1393--1512. https://doi.org/10.1016/0270-0255(86)90088-6

[22]

Donald B Rubin. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of educational Psychology, Vol. 66, 5 (1974), 688.

[23]

Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as treatments: Debiasing learning and evaluation. In international conference on machine learning. PMLR, 1670--1679.

[24]

Zihua Si, Xueran Han, Xiao Zhang, Jun Xu, Yue Yin, Yang Song, and Ji-Rong Wen. 2022. A Model-Agnostic Causal Learning Framework for Recommendation using Search Data. In Proceedings of the ACM Web Conference 2022. 224--233.

Digital Library

[25]

Gilbert Strang, Gilbert Strang, Gilbert Strang, and Gilbert Strang. 1993. Introduction to linear algebra. Vol. 3. Wellesley-Cambridge Press Wellesley, MA.

[26]

Eric J Tchetgen Tchetgen, Andrew Ying, Yifan Cui, Xu Shi, and Wang Miao. 2020. An introduction to proximal causal learning. arXiv preprint arXiv:2009.10982 (2020).

[27]

Wenjie Wang, Fuli Feng, Xiangnan He, Xiang Wang, and Tat-Seng Chua. 2021. Deconfounded recommendation for alleviating bias amplification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1717--1725.

Digital Library

[28]

Yixin Wang and David M Blei. 2019. The blessings of multiple causes. J. Amer. Statist. Assoc., Vol. 114, 528 (2019), 1574--1596.

[29]

Yixin Wang, Dawen Liang, Laurent Charlin, and David M Blei. 2020. Causal inference for recommender systems. In Fourteenth ACM Conference on Recommender Systems. 426--431.

Digital Library

[30]

Zimu Wang, Yue He, Jiashuo Liu, Wenchao Zou, Philip S Yu, and Peng Cui. 2022. Invariant Preference Learning for General Debiasing in Recommendation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1969--1978.

Digital Library

[31]

Tianxin Wei, Fuli Feng, Jiawei Chen, Ziwei Wu, Jinfeng Yi, and Xiangnan He. 2021. Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1791--1800.

Digital Library

[32]

Le Wu, Xiangnan He, Xiang Wang, Kun Zhang, and Meng Wang. 2022. A survey on accuracy-oriented neural recommendation: From collaborative filtering to information-rich recommendation. IEEE Transactions on Knowledge and Data Engineering (2022).

[33]

Shuyuan Xu, Juntao Tan, Shelby Heinecke, Jia Li, and Yongfeng Zhang. 2021. Deconfounded Causal Collaborative Filtering. arXiv preprint arXiv:2110.07122 (2021).

[34]

Ruohan Zhan, Changhua Pei, Qiang Su, Jianfeng Wen, Xueliang Wang, Guanyu Mu, Dong Zheng, Peng Jiang, and Kun Gai. 2022. Deconfounding Duration Bias in Watch-time Prediction for Video Recommendation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 4472--4481.

Digital Library

[35]

Yang Zhang, Fuli Feng, Xiangnan He, Tianxin Wei, Chonggang Song, Guohui Ling, and Yongdong Zhang. 2021. Causal intervention for leveraging popularity bias in recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 11--20.

Digital Library

[36]

Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 1059--1068.

Digital Library

[37]

Xinyuan Zhu, Yang Zhang, Fuli Feng, Xun Yang, Dingxian Wang, and Xiangnan He. 2022b. Mitigating Hidden Confounding Effects for Causal Recommendation. arXiv preprint arXiv:2205.07499 (2022).

[38]

Yaochen Zhu, Jing Yi, Jiayi Xie, and Zhenzhong Chen. 2022a. Deep causal reasoning for recommendations. arXiv preprint arXiv:2201.02088 (2022).

Cited By

Bai TChen WYang CShi C(2025)Invariant debiasing learning for recommendation via biased imputationInformation Processing & Management10.1016/j.ipm.2024.10402862:3(104028)Online publication date: May-2025
https://doi.org/10.1016/j.ipm.2024.104028
Ning WCheng RYan XKao BHuo NHaldar NTang BChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Debiasing Recommendation with Personal PopularityProceedings of the ACM Web Conference 202410.1145/3589334.3645421(3400-3409)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645421
Wang YSun JWang CZhang MYang M(2024)Navigate Beyond Shortcuts: Debiased Learning through the Lens of Neural Collapse2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01171(12322-12331)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01171
Show More Cited By

Index Terms

Debiasing Recommendation by Learning Identifiable Latent Confounders
1. Information systems
  1. World Wide Web

Recommendations

Addressing Unmeasured Confounder for Recommendation with Sensitivity Analysis
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Recommender systems should answer the intervention question "if recommending an item to a user, what would the feedback be", calling for estimating the causal effect of a recommendation on user feedback. Generally, this requires blocking the effect of ...
Debiasing Recommendation with Personal Popularity
WWW '24: Proceedings of the ACM Web Conference 2024

Global popularity (GP) bias is the phenomenon that popular items are recommended much more frequently than they should be, which goes against the goal of providing personalized recommendations and harms user experience and recommendation accuracy. Many ...
Using a trust network to improve top-N recommendation
RecSys '09: Proceedings of the third ACM conference on Recommender systems

Top-N item recommendation is one of the important tasks of recommenders. Collaborative filtering is the most popular approach to building recommender systems which can predict ratings for a given user and item. Collaborative filtering can be extended ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 2023

5996 pages

ISBN:9798400701030

DOI:10.1145/3580305

General Chairs:
Ambuj Singh
UC Santa Barbara, USA
,
Yizhou Sun
UC Los Angeles, USA
,
Program Chairs:
Leman Akoglu
Carnegie Mellon University, USA
,
Dimitrios Gunopulos
University of Athens, Greece
,
Xifeng Yan
UC Santa Barbara, USA
,
Ravi Kumar
Google, USA
,
Fatma Ozcan
Google, USA
,
Jieping Ye
Alibaba DAMO Academy

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 August 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD '23

Sponsor:

KDD '23: The 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 6 - 10, 2023

CA, Long Beach, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Sponsor:
sigkdd
sigkdd

The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 3 - 7, 2025

Toronto , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
406
Total Downloads

Downloads (Last 12 months)258
Downloads (Last 6 weeks)23

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bai TChen WYang CShi C(2025)Invariant debiasing learning for recommendation via biased imputationInformation Processing & Management10.1016/j.ipm.2024.10402862:3(104028)Online publication date: May-2025
https://doi.org/10.1016/j.ipm.2024.104028
Ning WCheng RYan XKao BHuo NHaldar NTang BChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Debiasing Recommendation with Personal PopularityProceedings of the ACM Web Conference 202410.1145/3589334.3645421(3400-3409)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645421
Wang YSun JWang CZhang MYang M(2024)Navigate Beyond Shortcuts: Debiased Learning through the Lens of Neural Collapse2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01171(12322-12331)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01171
Jui TRivas P(2024)Fairness issues, current approaches, and challenges in machine learning modelsInternational Journal of Machine Learning and Cybernetics10.1007/s13042-023-02083-215:8(3095-3125)Online publication date: 31-Jan-2024
https://doi.org/10.1007/s13042-023-02083-2

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents