research-article

Open access

Clicks can be Cheating: Counterfactual Recommendation for Mitigating Clickbait Issue

Authors:

Tat-Seng ChuaAuthors Info & Claims

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 1288 - 1297

https://doi.org/10.1145/3404835.3462962

Published: 11 July 2021 Publication History

Abstract

Recommendation is a prevalent and critical service in information systems. To provide personalized suggestions to users, industry players embrace machine learning, more specifically, building predictive models based on the click behavior data. This is known as the Click-Through Rate (CTR) prediction, which has become the gold standard for building personalized recommendation service. However, we argue that there is a significant gap between clicks and user satisfaction --- it is common that a user is "cheated" to click an item by the attractive title/cover of the item. This will severely hurt user's trust on the system if the user finds the actual content of the clicked item disappointing. What's even worse, optimizing CTR models on such flawed data will result in the Matthew Effect, making the seemingly attractive but actually low-quality items be more frequently recommended.

In this paper, we formulate the recommendation models as a causal graph that reflects the cause-effect factors in recommendation, and address the clickbait issue by performing counterfactual inference on the causal graph. We imagine a counterfactual world where each item has only exposure features (i.e., the features that the user can see before making a click decision). By estimating the click likelihood of a user in the counterfactual world, we are able to reduce the direct effect of exposure features and eliminate the clickbait issue. Experiments on real-world datasets demonstrate that our method significantly improves the post-click satisfaction of CTR models.

References

[1]

Himan Abdollahpouri, Robin Burke, and Bamshad Mobasher. 2019. Managing popularity bias in recommender systems with personalized re-ranking. In Proceedings of the International Flairs Conference. AAAI Press.

[2]

Qingyao Ai, Keping Bi, Cheng Luo, Jiafeng Guo, and W. Bruce Croft. 2018. Unbiased Learning to Rank with Unbiased Propensity Estimation. In SIGIR. ACM, 385--394.

[3]

Stephen Bonner and Flavian Vasile. 2018. Causal embeddings for recommendation. In RecSys. ACM, 104--112.

[4]

Remi Cadene, Corentin Dancette, Hedi Ben-younes, Matthieu Cord, Devi Parikh, et al. 2019. Rubi: Reducing unimodal biases for visual question answering. In NeuIPS. 841--852.

[5]

Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2020. Bias and Debias in Recommender System: A Survey and Future Directions. arXiv preprint arXiv:2010.03240 (2020).

[6]

Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. 2017. Attentive collaborative filtering: Multimedia recommendation with item-and component-level attention. In SIGIR. ACM, 335--344.

[7]

Xusong Chen, Dong Liu, Zheng-Jun Zha, Wengang Zhou, Zhiwei Xiong, and Yan Li. 2018. Temporal Hierarchical Attention at Category- and Item-Level for Micro-Video Click-Through Prediction. In MM. ACM, 1146--1153.

[8]

Konstantina Christakopoulou, Madeleine Traverse, Trevor Potter, Emma Marriott, Daniel Li, Chris Haulk, Ed H. Chi, and Minmin Chen. 2020. Deconfounding User Satisfaction Estimation from Response Rate Bias. In RecSys. ACM, 450--455.

[9]

Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey. 2008. An Experimental Comparison of Click Position-Bias Models. In WSDM. ACM, 87--94.

[10]

Balázs Csanád Csáji. 2001. Approximation with artificial neural networks. Faculty of Sciences, Etvs Lornd University, Hungary, Vol. 24, 48 (2001), 7.

[11]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. textBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In arXiv:1810.04805 .

[12]

Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph Neural Networks for Social Recommendation. WWW. ACM, 417--426.

[13]

Fuli Feng, Xiangnan He, Yiqun Liu, Liqiang Nie, and Tat-Seng Chua. 2018. Learning on partial-order hypergraphs. In WWW. ACM, 1523--1532.

[14]

Fuli Feng, Xiangnan He, Jie Tang, and Tat-Seng Chua. 2019. Graph adversarial training: Dynamically regularizing based on graph structure. TKDE (2019).

[15]

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning .MIT Press.

Digital Library

[16]

Jon Atle Gulla, Lemei Zhang, Peng Liu, Özlem Özgöbek, and Xiaomeng Su. 2017. The Adressa Dataset for News Recommendation. In WI. ACM, 1042--1048.

[17]

Ruining He and Julian McAuley. 2016. textVBPR: visual bayesian personalized ranking from implicit feedback. In AAAI. AAAI press.

[18]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, YongDong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. In SIGIR. ACM, 639--648.

[19]

Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural Collaborative Filtering. In WWW. ACM, 173--182.

[20]

Katja Hofmann, Fritz Behr, and Filip Radlinski. 2012. On Caption Bias in Interleaving Experiments. In CIKM. ACM, 115--124.

[21]

Richang Hong, Lei Li, Junjie Cai, Dapeng Tao, Meng Wang, and Qi Tian. 2017. Coherent semantic-visual indexing for large-scale image retrieval in the cloud. TIP, Vol. 26, 9 (2017), 4128--4138.

Digital Library

[22]

Richang Hong, Yang Yang, Meng Wang, and Xian-Sheng Hua. 2015. Learning visual semantic relationships for efficient visual retrieval. Transactions on Big Data, Vol. 1, 4 (2015), 152--161.

[23]

Hao Jiang, Wenjie Wang, Yinwei Wei, Zan Gao, Yinglong Wang, and Liqiang Nie. 2020. What Aspect Do You Like: Multi-scale Time-aware User Interest Modeling for Micro-video Recommendation. In MM. ACM, 3487--3495.

[24]

Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. 2017. Unbiased Learning-to-Rank with Biased Feedback. In WSDM. ACM, 781--789.

[25]

Youngho Kim, Ahmed Hassan, Ryen W White, and Imed Zitouni. 2014. Modeling dwell time to predict click-level satisfaction. In WSDM. ACM, 193--202.

[26]

Yongqi Li, Meng Liu, Jianhua Yin, Chaoran Cui, Xin-Shun Xu, and Liqiang Nie. 2019. Routing micro-videos via a temporal graph-guided recommendation system. In MM. ACM, 1464--1472.

[27]

Dawen Liang, Laurent Charlin, and David M Blei. 2016a. Causal inference for recommendation. In UAI. AUAI.

[28]

Dawen Liang, Laurent Charlin, James McInerney, and David M Blei. 2016b. Modeling user exposure in recommendation. In WWW. ACM, 951--961.

[29]

Chao Liu, Ryen W White, and Susan Dumais. 2010. Understanding web browsing behaviors through Weibull analysis of dwell time. In SIGIR. ACM, 379--386.

[30]

Dugang Liu, Pengxiang Cheng, Zhenhua Dong, Xiuqiang He, Weike Pan, and Zhong Ming. 2020 a. A General Knowledge Distillation Framework for Counterfactual Recommendation via Uniform Data. In SIGIR. ACM, 831--840.

[31]

Meng Liu, Leigang Qu, Liqiang Nie, Maofu Liu, Lingyu Duan, and Baoquan Chen. 2020 b. Iterative Local-Global Collaboration Learning Towards One-Shot Video Person Re-Identification. TIP, Vol. 29 (2020), 9360--9372.

[32]

Hongyu Lu, Min Zhang, and Shaoping Ma. 2018. Between Clicks and Satisfaction: Study on Multi-Phase User Preferences and Satisfaction for Online News Reading. In SIGIR. ACM, 435--444.

[33]

Hongyu Lu, Min Zhang, Weizhi Ma, Ce Wang, Feng xia, Yiqun Liu, Leyu Lin, and Shaoping Ma. 2019. Effects of User Negative Experience in Mobile News Streaming. In SIGIR. ACM, 705--714.

[34]

Marco Morik, Ashudeep Singh, Jessica Hong, and Thorsten Joachims. 2020. Controlling Fairness and Bias in Dynamic Learning-to-Rank. In SIGIR. ACM, 429--438.

[35]

Yulei Niu, Kaihua Tang, Hanwang Zhang, Zhiwu Lu, Xian-Sheng Hua, and Ji-Rong Wen. 2020. Counterfactual VQA: A Cause-Effect Look at Language Bias. In arXiv:2006.04315 .

[36]

Zohreh Ovaisi, Ragib Ahsan, Yifan Zhang, Kathryn Vasilaky, and Elena Zheleva. 2020. Correcting for Selection Bias in Learning-to-Rank Systems. In WWW. ACM, 1863--1873.

Digital Library

[37]

Judea Pearl. 2001. Direct and indirect effects. In UAI. Morgan Kaufmann Publishers Inc, 411--420.

[38]

Judea Pearl. 2009. Causality .Cambridge university press.

[39]

Judea Pearl and Dana Mackenzie. 2018. The Book of Why: The New Science of Cause and Effect 1st ed.). Basic Books, Inc.

[40]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In UAI. AUAI Press, 452--461.

Digital Library

[41]

Paul R. Rosenbaum and Donald B. Rubin. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika, Vol. 70, 1 (04 1983), 41--55.

[42]

Yuta Saito, Suguru Yaginuma, Yuta Nishino, Hayato Sakata, and Kazuhide Nakata. 2020. Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback. In WSDM. ACM, 501--509.

[43]

Kaihua Tang, Jianqiang Huang, and Hanwang Zhang. 2020 a. Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect. In NeurIPS .

[44]

Kaihua Tang, Yulei Niu, Jianqiang Huang, Jiaxin Shi, and Hanwang Zhang. 2020 b. Unbiased scene graph generation from biased training. In arXiv:2002.11949 .

[45]

Tyler J VanderWeele. 2013. A three-way decomposition of a total effect into direct, indirect, and interactive effects. Epidemiology (Cambridge, Mass.), Vol. 24, 2 (2013), 224.

[46]

Wenjie Wang, Ling-Yu Duan, Hao Jiang, Peiguang Jing, Xuemeng Song, and Liqiang Nie. 2021 a. Market2Dish: Health-Aware Food Recommendation. TOMM, Vol. 17 (April 2021).

Digital Library

[47]

Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. 2021 b. Denoising implicit feedback for recommendation. In WSDM. ACM, 373--381.

[48]

Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural Graph Collaborative Filtering. In SIGIR. ACM, 165--174.

[49]

Yinwei Wei, Xiang Wang, Liqiang Nie, Xiangnan He, and Tat-Seng Chua. 2020. Graph-Refined Convolutional Network for Multimedia Recommendation with Implicit Feedback. In MM. 3541--3549.

[50]

Yinwei Wei, Xiang Wang, Liqiang Nie, Xiangnan He, Richang Hong, and Tat-Seng Chua. 2019. textMMGCN: Multi-modal Graph Convolution Network for Personalized Recommendation of Micro-video. In MM. ACM, 1437--1445.

[51]

Hongyi Wen, Longqi Yang, and Deborah Estrin. 2019. Leveraging Post-click Feedback for Content Recommendations. In RecSys. ACM, 278--286.

[52]

Hong Wen, Jing Zhang, Yuan Wang, Fuyu Lv, Wentian Bao, Quan Lin, and Keping Yang. 2020. Entire space multi-task modeling via post-click behavior decomposition for conversion rate prediction. In SIGIR. ACM, 2377--2386.

[53]

Byoungju Yang, Sangkeun Lee, Sungchan Park, and Sang goo Lee. 2012. Exploiting Various Implicit Feedback for Collaborative Filtering. In WWW. ACM, 639--640.

[54]

Xing Yi, Liangjie Hong, Erheng Zhong, Nanthan Nan Liu, and Suju Rajan. 2014. Beyond clicks: dwell time for personalization. In RecSys. ACM, 113--120.

[55]

Peifeng Yin, Ping Luo, Wang-Chien Lee, and Min Wang. 2013. Silence is Also Evidence: Interpreting Dwell Time for Recommendation from Psychological Perspective. In KDD. ACM, 989--997.

[56]

Fajie Yuan, Xiangnan He, Alexandros Karatzoglou, and Liguang Zhang. 2020. Parameter-efficient transfer from sequential behaviors for user modeling and recommendation. In SIGIR. ACM, 1469--1478.

[57]

Yisong Yue, Rajan Patel, and Hein Roehrig. 2010. Beyond Position Bias: Examining Result Attractiveness as a Source of Presentation Bias in Clickthrough Data. In WWW. ACM, 1011--1018.

[58]

Savvas Zannettou, Sotirios Chatzis, Kostantinos Papadamou, and Michael Sirivianos. 2018. The good, the bad and the bait: Detecting and characterizing clickbait on YouTube. In SPW. IEEE, 63--69.

[59]

Wenhao Zhang, Wentian Bao, Xiao-Yang Liu, Keping Yang, Quan Lin, Hong Wen, and Ramin Ramezani. 2020. Large-Scale Causal Approaches to Debiasing Post-Click Conversion Rate Estimation with Multi-Task Learning. In WWW. ACM, 2775--2781.

[60]

Ziwei Zhu, Yun He, Yin Zhang, and James Caverlee. 2020. Unbiased Implicit Recommendation and Propensity Estimation via Combinational Joint Learning. In RecSys. ACM, 551--556.

Cited By

Cheng XPeng G(2024)Study on the Behavioral Motives of Algorithmic Avoidance in Intelligent Recommendation SystemsJournal of Global Information Management10.4018/JGIM.35285732:1(1-22)Online publication date: 23-Aug-2024
https://doi.org/10.4018/JGIM.352857
Song WXing X(2024)Dual-Tower Counterfactual Session-Aware Recommender SystemEntropy10.3390/e2606051626:6(516)Online publication date: 14-Jun-2024
https://doi.org/10.3390/e26060516
Möller LPadó S(2024)Explaining Neural News Recommendation with Attributions onto Reading HistoriesACM Transactions on Intelligent Systems and Technology10.1145/3673233Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3673233
Show More Cited By

Index Terms

Clicks can be Cheating: Counterfactual Recommendation for Mitigating Clickbait Issue
1. Computing methodologies
  1. Machine learning
    1. Learning settings
      1. Learning from implicit feedback
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Counterfactual Review-based Recommendation
CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management

Incorporating review information into the recommender system has been demonstrated to be an effective method for boosting the recommendation performance. Previous research mainly focus on designing advanced architectures to better profile the users and ...
Counterfactual Explainable Recommendation
CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management

By providing explanations for users and system designers to facilitate better understanding and decision making, explainable recommendation has been an important research problem. In this paper, we propose Counterfactual Explainable Recommendation (...
Do clicks measure recommendation relevancy?: an empirical user study
RecSys '10: Proceedings of the fourth ACM conference on Recommender systems

Evaluation have been an important subject since the early days of recommender systems. In online test, the click-through rate (CTR) is often adopted as the metric. However, recommended items with higher CTR does not imply higher relevance of two items ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2021

2998 pages

ISBN:9781450380379

DOI:10.1145/3404835

General Chairs:
Fernando Diaz
(Google)
,
Chirag Shah
University of Washington
,
Torsten Suel
New York University
,
Program Chairs:
Pablo Castells
Universidad Autónoma de Madrid, Amazon
,
Rosie Jones
Spotify
,
Tetsuya Sakai
Waseda University

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Key Research and Development Program of China
National Natural Science Foundation of China
the Sea-NExT Joint Lab

Conference

SIGIR '21

Sponsor:

SIGIR

SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2021

Virtual Event, Canada

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

104
Total Citations
View Citations
3,143
Total Downloads

Downloads (Last 12 months)905
Downloads (Last 6 weeks)113

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cheng XPeng G(2024)Study on the Behavioral Motives of Algorithmic Avoidance in Intelligent Recommendation SystemsJournal of Global Information Management10.4018/JGIM.35285732:1(1-22)Online publication date: 23-Aug-2024
https://doi.org/10.4018/JGIM.352857
Song WXing X(2024)Dual-Tower Counterfactual Session-Aware Recommender SystemEntropy10.3390/e2606051626:6(516)Online publication date: 14-Jun-2024
https://doi.org/10.3390/e26060516
Möller LPadó S(2024)Explaining Neural News Recommendation with Attributions onto Reading HistoriesACM Transactions on Intelligent Systems and Technology10.1145/3673233Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3673233
Liang SPan Zliu wYin Jde Rijke M(2024)A Survey on Variational Autoencoders in Recommender SystemsACM Computing Surveys10.1145/366336456:10(1-40)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3663364
Shao PWu LZhang KLian DHong RLi YWang M(2024)Average User-Side Counterfactual Fairness for Collaborative FilteringACM Transactions on Information Systems10.1145/365663942:5(1-26)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3656639
Jian MBai YFu XGuo JShi GWu L(2024)Counterfactual Graph Convolutional Learning for Personalized RecommendationACM Transactions on Intelligent Systems and Technology10.1145/365563215:4(1-20)Online publication date: 18-Jun-2024
https://dl.acm.org/doi/10.1145/3655632
Yu MZhou XZhao MXu TZhao YYu RLi XGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)A Causal View for Multi-Interest User Modeling in News RecommendationProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658093(433-441)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658093
Yi JChen Z(2024)Deconfounded Cross-modal Matching for Content-based Micro-video Background Music RecommendationACM Transactions on Intelligent Systems and Technology10.1145/365004215:3(1-25)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3650042
Zhang SMiao QNie PLi MChen ZFeng FKuang KWu F(2024)Transferring Causal Mechanism over Meta-representations for Target-Unknown Cross-domain RecommendationACM Transactions on Information Systems10.1145/364380742:4(1-27)Online publication date: 22-Mar-2024
https://dl.acm.org/doi/10.1145/3643807
Gao CZheng YWang WFeng FHe XLi Y(2024)Causal Inference in Recommender Systems: A Survey and Future DirectionsACM Transactions on Information Systems10.1145/363904842:4(1-32)Online publication date: 9-Feb-2024
https://dl.acm.org/doi/10.1145/3639048
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents