research-article

Debiased Representation Learning in Recommendation via Information Bottleneck

Authors:

Pengxiang Cheng,

Zhong MingAuthors Info & Claims

ACM Transactions on Recommender Systems, Volume 1, Issue 1

Article No.: 5, Pages 1 - 27

https://doi.org/10.1145/3568030

Published: 27 January 2023 Publication History

Abstract

How to effectively mitigate the bias of feedback in recommender systems is an important research topic. In this article, we first describe the generation process of the biased and unbiased feedback in recommender systems via two respective causal diagrams, where the difference between them can be regarded as the source of system-induced biases. We then define this difference as a confounding bias and propose a new perspective on debiased representation learning to alleviate it. Specifically, for the case with biased feedback alone, we derive the conditions that need to be satisfied to obtain a debiased representation from the causal diagrams. Then, we propose a novel framework called debiased information bottleneck (DIB) to optimize these conditions and then find a tractable solution for it. The proposed framework constrains the model to learn a biased embedding vector with independent biased and unbiased components in the training phase, and uses only the unbiased component in the test phase to deliver more accurate recommendations. We further propose a variant of DIB by relaxing the independence between the biased and unbiased components. Finally, we conduct extensive experiments on a public dataset and a real product dataset to verify the effectiveness of the proposed framework.

References

[1]

Himan Abdollahpouri, Robin Burke, and Bamshad Mobasher. 2017. Controlling popularity bias in learning-to-rank recommendation. In Proceedings of the 11th ACM Conference on Recommender Systems. 42–46.

Digital Library

[2]

Alessandro Achille and Stefano Soatto. 2018. Information dropout: Learning optimal representations through noisy computation. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 12 (2018), 2897–2905.

Digital Library

[3]

Aman Agarwal, Ivan Zaitsev, Xuanhui Wang, Cheng Li, Marc Najork, and Thorsten Joachims. 2019. Estimating position bias without intrusive interventions. In Proceedings of the 12th ACM International Conference on Web Search and Data Mining. 474–482.

Digital Library

[4]

Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2623–2631.

Digital Library

[5]

Shaden Alshammari, Yu-Xiong Wang, Deva Ramanan, and Shu Kong. 2022. Long-tailed recognition via weight balancing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6897–6907.

[6]

Stephen Bonner and Flavian Vasile. 2018. Causal embeddings for recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems. 104–112.

Digital Library

[7]

Rocío Cañamares and Pablo Castells. 2018. Should I follow the crowd?: A probabilistic analysis of the effectiveness of popularity in recommender systems. In Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. 415–424.

[8]

Pengyu Cheng, Martin Renqiang Min, Dinghan Shen, Christopher Malon, Yizhe Zhang, Yitong Li, and Lawrence Carin. 2020. Improving disentangled text representation learning with information-theoretic guidance. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 7530–7541.

[9]

Bin Dai, Chen Zhu, Baining Guo, and David Wipf. 2018. Compressing neural networks using the variational information bottleneck. In Proceedings of the 35th International Conference on International Conference on Machine Learning. 1135–1144.

[10]

Marco Federici, Anjan Dutta, Patrick Forré, Nate Kushman, and Zeynep Akata. 2020. Learning robust representations via multi-view information bottleneck. In Proceedings of the 8th International Conference on Learning Representations.

[11]

Ruoyuan Gao and Chirag Shah. 2020. Counteracting bias and increasing fairness in search and recommender systems. In Proceedings of the 14th ACM Conference on Recommender Systems. 745–747.

Digital Library

[12]

Yingqiang Ge, Shuya Zhao, Honglu Zhou, Changhua Pei, Fei Sun, Wenwu Ou, and Yongfeng Zhang. 2020. Understanding echo chambers in e-commerce recommender systems. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 2261–2270.

Digital Library

[13]

Yingqiang Ge, Xiaoting Zhao, Lucia Yu, Saurabh Paul, Diane Hu, Chu-Cheng Hsieh, and Yongfeng Zhang. 2022. Toward Pareto efficient fairness-utility trade-off in recommendation through reinforcement learning. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining. 316–324.

Digital Library

[14]

Prem Gopalan, Jake M. Hofman, and David M. Blei. 2015. Scalable recommendation with hierarchical Poisson factorization. In Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence. 326–335.

[15]

Negar Hassanpour and Russell Greiner. 2020. Learning disentangled representations for counterfactual regression. In Proceedings of the 8th International Conference on Learning Representations.

[16]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 27th International Conference on World Wide Web. 173–182.

[17]

Ayush Jaiswal, Rob Brekelmans, Daniel Moyer, Greg Ver Steeg, Wael AbdAlmageed, and Premkumar Natarajan. 2019. Discovery and separation of features for invariant representation learning. https://arxiv.org/abs/1912.00646.

[18]

Ray Jiang, Silvia Chiappa, Tor Lattimore, András György, and Pushmeet Kohli. 2019. Degenerate feedback loops in recommender systems. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society. 383–390.

Digital Library

[19]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer8 (2009), 30–37.

Digital Library

[20]

Kun Kuang, Peng Cui, Bo Li, Meng Jiang, Shiqiang Yang, and Fei Wang. 2017. Treatment effect estimation with data-driven variable decomposition. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 140–146.

[21]

Dawen Liang, Laurent Charlin, and David M. Blei. 2016. Causal inference for recommendation. In Workshop on Causation: Foundation to Application Co-located with the 32nd Conference on Uncertainty in Artificial Intelligence.

[22]

Dawen Liang, Laurent Charlin, James McInerney, and David M. Blei. 2016. Modeling user exposure in recommendation. In Proceedings of the 25th International Conference on World Wide Web. 951–961.

Digital Library

[23]

Dawen Liang, Rahul G. Krishnan, Matthew D. Hoffman, and Tony Jebara. 2018. Variational autoencoders for collaborative filtering. In Proceedings of the 27th International Conference on World Wide Web. 689–698.

Digital Library

[24]

Chen Lin, Dugang Liu, Yanghua Xiao, and Hanghang Tong. 2020. Spiral of silence and its application in recommender systems. IEEE Transactions on Knowledge and Data Engineering 34, 6 (2020), 2934–2947.

[25]

Dugang Liu, Pengxiang Cheng, Zhenhua Dong, Xiuqiang He, Weike Pan, and Zhong Ming. 2020. A general knowledge distillation framework for counterfactual recommendation via uniform data. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 831–840.

Digital Library

[26]

Dugang Liu, Pengxiang Cheng, Hong Zhu, Zhenhua Dong, Xiuqiang He, Weike Pan, and Zhong Ming. 2021. Mitigating confounding bias in recommendation via information bottleneck. In Proceedings of the 15th ACM Conference on Recommender Systems. 351–360.

Digital Library

[27]

Dugang Liu, Chen Lin, Zhilin Zhang, Yanghua Xiao, and Hanghang Tong. 2019. Spiral of silence in recommender systems. In Proceedings of the 12th ACM International Conference on Web Search and Data Mining. 222–230.

[28]

Benjamin M. Marlin and Richard S. Zemel. 2009. Collaborative prediction and ranking with non-random missing data. In Proceedings of the 3rd ACM Conference on Recommender Systems. 5–12.

Digital Library

[29]

Daniel Moyer, Shuyang Gao, Rob Brekelmans, Greg Ver Steeg, and Aram Galstyan. 2018. Invariant representations without adversarial training. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 9102–9111.

Digital Library

[30]

Zohreh Ovaisi, Ragib Ahsan, Yifan Zhang, Kathryn Vasilaky, and Elena Zheleva. 2020. Correcting for selection bias in learning-to-rank systems. In Proceedings of the 29th International Conference on World Wide Web. 1863–1873.

[31]

Sonali Parbhoo, Mario Wieser, and Volker Roth. 2018. Causal deep information bottleneck. https://arxiv.org/abs/1807.02326v1.

[32]

Paul R. Rosenbaum and Donald B. Rubin. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika 70, 1 (1983), 41–55.

[33]

Yuta Saito. 2020. Asymmetric tri-training for debiasing missing-not-at-random explicit feedback. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 309–318.

Digital Library

[34]

Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata. 2020. Unbiased recommender learning from missing-not-at-random implicit feedback. In Proceedings of the 13th ACM International Conference on Web Search and Data Mining. 501–509.

Digital Library

[35]

Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as treatments: Debiasing learning and evaluation. In Proceedings of the 33rd International Conference on Machine Learning. 1670–1679.

[36]

Zihua Si, Xueran Han, Xiao Zhang, Jun Xu, Yue Yin, Yang Song, and Ji-Rong Wen. 2022. A model-agnostic causal learning framework for recommendation using search data. In Proceedings of the 31st International Conference on World Wide Web.

[37]

Yixin Su, Rui Zhang, Sarah Erfani, and Zhenghua Xu. 2021. Detecting beneficial feature interactions for recommender systems. In Proceedings of the 35th AAAI Conference on Artificial Intelligence. 4357–4365.

[38]

Adith Swaminathan and Thorsten Joachims. 2015. The self-normalized estimator for counterfactual learning. In Proceedings of the 29th International Conference on Neural Information Processing Systems. 3231–3239.

[39]

Thomas M. Cover and Joy A. Thomas. 2006. Elements of Information Theory. Wiley-Interscience.

Digital Library

[40]

Naftali Tishby, Fernando C Pereira, and William Bialek. 1999. The information bottleneck method. In Proceedings of the 37th Annual Allerton Conference on Communications, Control and Computing. 368–377.

[41]

Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 11 (2008), 2579–2605.

[42]

Qi Wan Wan, Xiangnan He, Xiang Wang, Jiancan Wu, Wei Guo, and Ruiming Tang. 2022. Cross pairwise ranking for unbiased item recommendation. In Proceedings of the 31st International Conference on World Wide Web.

[43]

Ruoxi Wang, Rakesh Shivanna, Derek Cheng, Sagar Jain, Dong Lin, Lichan Hong, and Ed Chi. 2021. DCN V2: Improved deep and cross network and practical lessons for web-scale learning to rank systems. In Proceedings of the 30th International Conference on World Wide Web. 1785–1797.

[44]

Wenjie Wang, Fuli Feng, Xiangnan He, Hanwang Zhang, and Tat-Seng Chua. 2021. Clicks can be cheating: Counterfactual recommendation for mitigating clickbait issue. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1288–1297.

Digital Library

[45]

Xuanhui Wang, Nadav Golbandi, Michael Bendersky, Donald Metzler, and Marc Najork. 2018. Position bias estimation for unbiased learning to rank in personal search. In Proceedings of the 11th ACM International Conference on Web Search and Data Mining. 610–618.

Digital Library

[46]

Xiaojie Wang, Rui Zhang, Yu Sun, and Jianzhong Qi. 2021. Combating selection biases in recommender systems with a few unbiased ratings. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 427–435.

Digital Library

[47]

Zifeng Wang, Xi Chen, Rui Wen, Shao-Lun Huang, Ercan E. Kuruoglu, and Yefeng Zheng. 2020. Information theoretic counterfactual learning from missing-not-at-random feedback. In Advances in Neural Information Processing Systems. 1854–1864.

[48]

Tianxin Wei, Fuli Feng, Jiawei Chen, Ziwei Wu, Jinfeng Yi, and Xiangnan He. 2021. Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1791–1800.

Digital Library

[49]

Teng Xiao and Suhang Wang. 2022. Towards unbiased and robust causal ranking for recommender systems. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining. 1158–1167.

Digital Library

[50]

Shuyuan Xu, Yingqiang Ge, Yunqi Li, Zuohui Fu, Xu Chen, and Yongfeng Zhang. 2021. Causal collaborative filtering. https://arxiv.org/abs/2102.01868.

[51]

Haiqin Yang, Guang Ling, Yuxin Su, Michael R. Lyu, and Irwin King. 2015. Boosting response aware model-based collaborative filtering. IEEE Transactions on Knowledge and Data Engineering 27, 8 (2015), 2064–2077.

Digital Library

[52]

Jiangxing Yu, Hong Zhu, Chih-Yao Chang, Xinhua Feng, Bowen Yuan, Xiuqiang He, and Zhenhua Dong. 2020. Influence function for unbiased recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1929–1932.

Digital Library

[53]

Bowen Yuan, Jui-Yang Hsia, Meng-Yuan Yang, Hong Zhu, Chih-Yao Chang, Zhenhua Dong, and Chih-Jen Lin. 2019. Improving ad click prediction by considering non-displayed events. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 329–338.

Digital Library

[54]

Shuxi Zeng, Murat Ali Bayir, Joel Pfeiffer, Denis Charles, and Emre Kiciman. 2021. Causal transfer random forest: Combining logged data and randomized experiments for robust prediction. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 211–219.

Digital Library

[55]

Ziwei Zhu and James Caverlee. 2022. Fighting mainstream bias in recommender systems via local fine tuning. In Proceedings of the 15th ACM International Conference on Web Search and Data Mining. 1497–1506.

Cited By

Wang XFukumoto FCui JSuzuki YLi JYu DHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)CaDRec: Contextualized and Debiased Recommender ModelProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657799(405-415)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657799
Liu DQiao YTang XChen LHe XMing ZEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Prior-Guided Accuracy-Bias Tradeoff Learning for CTR Prediction in Multimedia RecommendationProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3613801(995-1003)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3613801
Du XQian KMa YXiang X(undefined)Enhancing Item-level Bundle Representation for Bundle RecommendationACM Transactions on Recommender Systems10.1145/3637067
https://dl.acm.org/doi/10.1145/3637067

Index Terms

Debiased Representation Learning in Recommendation via Information Bottleneck
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Mitigating Confounding Bias in Recommendation via Information Bottleneck
RecSys '21: Proceedings of the 15th ACM Conference on Recommender Systems

How to effectively mitigate the bias of feedback in recommender systems is an important research topic. In this paper, we first describe the generation process of the biased and unbiased feedback in recommender systems via two respective causal ...
Read More
Causally Debiased Time-aware Recommendation
WWW '24: Proceedings of the ACM on Web Conference 2024

Time-aware recommendation has been widely studied for modeling the user dynamic preference and a lot of models have been proposed. However, these models often overlook the fact that users may not behave evenly on the timeline, and observed datasets can ...
Read More
Effects of Personalized and Aggregate Top-N Recommendation Lists on User Preference Ratings

Prior research has shown a robust effect of personalized product recommendations on user preference judgments for items. Specifically, the display of system-predicted preference ratings as item recommendations has been shown in multiple studies to bias ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Recommender Systems

ACM Transactions on Recommender Systems Volume 1, Issue 1

March 2023

163 pages

EISSN:2770-6699

DOI:10.1145/3581755

Editors:
Li Chen
Hong Kong Baptist University, China
,
Dietmar Jannach
University of Klagenfurt, Austria

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 January 2023

Online AM: 25 October 2022

Accepted: 25 September 2022

Revised: 10 August 2022

Received: 31 March 2022

Published in TORS Volume 1, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

National Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
1,017
Total Downloads

Downloads (Last 12 months)477
Downloads (Last 6 weeks)10

Other Metrics

View Author Metrics

Citations

Cited By

Wang XFukumoto FCui JSuzuki YLi JYu DHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)CaDRec: Contextualized and Debiased Recommender ModelProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657799(405-415)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657799
Liu DQiao YTang XChen LHe XMing ZEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Prior-Guided Accuracy-Bias Tradeoff Learning for CTR Prediction in Multimedia RecommendationProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3613801(995-1003)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3613801
Du XQian KMa YXiang X(undefined)Enhancing Item-level Bundle Representation for Bundle RecommendationACM Transactions on Recommender Systems10.1145/3637067
https://dl.acm.org/doi/10.1145/3637067

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents