Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Modeling User Reviews through Bayesian Graph Attention Networks for Recommendation

Published: 25 April 2023 Publication History

Abstract

Recommender systems relieve users from cognitive overloading by predicting preferred items for users. Due to the complexity of interactions between users and items, graph neural networks (GNN) use graph structures to effectively model user–item interactions. However, existing GNN approaches have the following limitations: (1) User reviews are not adequately modeled in graphs. Therefore, user preferences and item properties that are described in user reviews are lost for modeling users and items; and (2) GNNs assume deterministic relations between users and items, which lack the stochastic modeling to estimate the uncertainties in neighbor relations. To mitigate the limitations, we build tripartite graphs to model user reviews as nodes that connect with users and items. We estimate neighbor relations with stochastic variables and propose a Bayesian graph attention network (i.e., ContGraph) to accurately predict user ratings. ContGraph incorporates the prior knowledge of user preferences to regularize the posterior inference of attention weights. Our experimental results show that ContGraph significantly outperforms 13 state-of-the-art models and improves the best performing baseline (i.e., ANR) by 5.23% on 25 datasets in the five-core version. Moreover, we show that correctly modeling the semantics of user reviews in graphs can help express the semantics of users and items.

References

[1]
Konstantin Bauman, Bing Liu, and Alexander Tuzhilin. 2017. Aspect based recommendations: Recommending items with the most valuable aspects based on user reviews. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 717–725.
[2]
Rianne van den Berg, Thomas N. Kipf, and Max Welling. 2017. Graph convolutional matrix completion. arXiv:1706.02263. Retrieved from https://arxiv.org.abs/1706.02263.
[3]
David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation. J. Mach. Learn. Res. 3 (January 2003), 993–1022.
[4]
Rose Catherine and William Cohen. 2017. Transnets: Learning to transform for recommendation. In Proceedings of the 11th ACM Conference on Recommender Systems. ACM, 288–296.
[5]
Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. 2018. Neural attentional rating regression with review-level explanations. In Proceedings of the World Wide Web Conference (WWW’18). 1583–1592.
[6]
Xu Chen, Zheng Qin, Yongfeng Zhang, and Tao Xu. 2016. Learning to rank features for recommendation over multiple categories. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. 305–314.
[7]
Zhiyong Cheng, Ying Ding, Lei Zhu, and Mohan Kankanhalli. 2018. Aspect-aware latent factor model: Rating prediction with ratings and reviews. In Proceedings of the World Wide Web Conference (WWW’18). 639–648.
[8]
Jin Yao Chin, Kaiqi Zhao, Shafiq Joty, and Gao Cong. 2018. ANR: Aspect-based neural recommender. In Proceedings of the Conference on Information and Knowledge Management (CIKM’18). 147–156.
[9]
Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C. Courville, and Yoshua Bengio. 2015. A recurrent latent variable model for sequential data. In Advances in Neural Information Processing Systems, Vol. 28.
[10]
Paolo Cremonesi, Yehuda Koren, and Roberto Turrin. 2010. Performance of recommender algorithms on top-n recommendation tasks. In Proceedings of the 4th ACM Conference on Recommender Systems. 39–46.
[11]
Yuntian Deng, Yoon Kim, Justin Chiu, Demi Guo, and Alexander Rush. 2018. Latent alignment and variational attention. In Advances in Neural Information Processing Systems, Vol. 31.
[12]
Qiming Diao, Minghui Qiu, Chao-Yuan Wu, Alexander J. Smola, Jing Jiang, and Chong Wang. 2014. Jointly modeling aspects, ratings and sentiments for movie recommendation (JMARS). In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 193–202.
[13]
Xin Dong, Lei Yu, Zhonghuo Wu, Yuxia Sun, Lingfeng Yuan, and Fangxi Zhang. 2017. A hybrid collaborative filtering model with deep structure for recommender systems. In Proceedings of the AAAI Annual Conference on Artificial Intelligence (AAAI’17). 1309–1315.
[14]
John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12 (July 2011), 2121–2159.
[15]
Shaohua Fan, Junxiong Zhu, Xiaotian Han, Chuan Shi, Linmei Hu, Biyu Ma, and Yongliang Li. 2019. Metapath-guided heterogeneous graph neural network for intent recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2478–2486.
[16]
Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph neural networks for social recommendation. In Proceedings of the World Wide Web Conference (WWW’19). 417–426.
[17]
Xinjie Fan, Shujian Zhang, Bo Chen, and Mingyuan Zhou. 2020. Bayesian attention modules. In Advances in Neural Information Processing Systems Vol 33.
[18]
Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King. 2020. Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding. In Proceedings of the Web Conference. 2331–2341.
[19]
Jingyue Gao, Yang Lin, Yasha Wang, Xiting Wang, Zhao Yang, Yuanduo He, and Xu Chu. 2020. Set-sequence-graph: A multi-view approach towards exploiting reviews for recommendation. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management. 395–404.
[20]
Zoubin Ghahramani and Matthew Beal. 2000. Propagation algorithms for variational Bayesian learning. In Advances in Neural Information Processing Systems, Vol. 13 (2000).
[21]
Xinyu Guan, Zhiyong Cheng, Xiangnan He, Yongfeng Zhang, Zhibo Zhu, Qinke Peng, and Tat-Seng Chua. 2019. Attentive aspect modeling for review-aware recommendation. ACM Trans. Inf. Syst. 37, 3 (2019), 1–27.
[22]
Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the World Wide Web Conference (WWW’16). 507–517.
[23]
Xiangnan He, Tao Chen, Min-Yen Kan, and Xiao Chen. 2015. Trirank: Review-aware explainable recommendation by modeling aspects. In Proceedings of the Conference on Information and Knowledge Management (CIKM’15). ACM, 1661–1670.
[24]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web. 173–182.
[25]
Antonio Hernando, Jesús Bobadilla, and Fernando Ortega. 2016. A non negative matrix factorization for collaborative filtering recommender systems based on a Bayesian probabilistic model. Knowl.-Bas. Syst. 97 (2016), 188–202.
[26]
Matthew D. Hoffman, David M. Blei, Chong Wang, and John Paisley. 2013. Stochastic variational inference.J. Mach. Learn. Res. 14, 5 (2013), 1303–1347.
[27]
Binbin Hu, Yuan Fang, and Chuan Shi. 2019. Adversarial Learning on Heterogeneous Information Networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 120–129.
[28]
Wentao Huang, Yuchen Li, Yuan Fang, Ju Fan, and Hongxia Yang. 2020. BiANE: Bipartite attributed network embedding. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 149–158.
[29]
Diederik P. Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv: 1312.6114. Retrieved from https://arxiv.org/abs/1312.6114.
[30]
Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37.
[31]
Ao Li, Zhou Qin, Runshi Liu, Yiqun Yang, and Dong Li. 2019. Spam review detection with graph convolutional networks. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2703–2711.
[32]
Chenliang Li, Cong Quan, Li Peng, Yunwei Qi, Yuming Deng, and Libing Wu. 2019. A capsule network for recommendation and explaining what you like and dislike. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 275–284.
[33]
Junyi Li, Siqing Li, Wayne Xin Zhao, Gaole He, Zhicheng Wei, Nicholas Jing Yuan, and Ji-Rong Wen. 2020. Knowledge-enhanced personalized review generation with capsule graph neural network. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management. 735–744.
[34]
Wenzhe Li, Sungjin Ahn, and Max Welling. 2016. Scalable MCMC for mixed membership stochastic blockmodels. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics. PMLR, 723–731.
[35]
Xiaopeng Li and James She. 2017. Collaborative variational autoencoder for recommender systems. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 305–314.
[36]
Donghua Liu, Jing Li, Bo Du, Jun Chang, and Rong Gao. 2019. Daml: Dual attention mutual learning between ratings and reviews for item recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 344–352.
[37]
Fan Liu, Zhiyong Cheng, Lei Zhu, Chenghao Liu, and Liqiang Nie. 2020. An attribute-aware attentive GCN model for attribute missing in recommendation. IEEE Trans. Knowl. Data Eng. 34, 9 (2020), 4077–4088.
[38]
Hongtao Liu, Wenjun Wang, Qiyao Peng, Nannan Wu, Fangzhao Wu, and Pengfei Jiao. 2021. Toward comprehensive user and item representations via three-tier attention network. ACM Trans. Inf. Syst. 39, 3 (2021), 1–22.
[39]
Ziqi Liu, Chaochao Chen, Xinxing Yang, Jun Zhou, Xiaolong Li, and Le Song. 2018. Heterogeneous graph neural networks for malicious account detection. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 2077–2085.
[40]
Yichao Lu, Ruihai Dong, and Barry Smyth. 2018. Coevolutionary recommendation model: Mutual learning between ratings and reviews. In Proceedings of the World Wide Web Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 773–782.
[41]
Julian McAuley and Jure Leskovec. 2013. Hidden factors and hidden topics: Understanding rating dimensions with review text. In Proceedings of the ACM Recommender Systems Conference (RecSys’13). ACM, 165–172.
[42]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS’13). 3111–3119.
[43]
Andriy Mnih and Ruslan R. Salakhutdinov. 2008. Probabilistic matrix factorization. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS’08). 1257–1264.
[44]
Soumyasundar Pal, Florence Regol, and Mark Coates. 2019. Bayesian graph convolutional neural networks using node copying. arXiv:1911.04965. Retrieved from https://arxiv.org/abs/1911.04965.
[45]
Steffen Rendle. 2010. Factorization machines. In Proceedings of the IEEE 10th International Conference on Data Mining (ICDM’10). IEEE, 995–1000.
[46]
Noveen Sachdeva and Julian McAuley. 2020. How useful are reviews for recommendation? a critical review and potential improvements. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1845–1848.
[47]
Sungyong Seo, Jing Huang, Hao Yang, and Yan Liu. 2017. Interpretable convolutional neural networks with dual local and global attention for review rating prediction. In Proceedings of the 11th ACM Conference on Recommender Systems. 297–305.
[48]
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1 (2014), 1929–1958.
[49]
Jianing Sun, Wei Guo, Dengcheng Zhang, Yingxue Zhang, Florence Regol, Yaochen Hu, Huifeng Guo, Ruiming Tang, Han Yuan, Xiuqiang He, et al. 2020. A framework for recommending accurate and diverse items using bayesian graph convolutional neural networks. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2030–2039.
[50]
Jianing Sun, Yingxue Zhang, Chen Ma, Mark Coates, Huifeng Guo, Ruiming Tang, and Xiuqiang He. 2019. Multi-graph convolution collaborative filtering. In Proceedings of the IEEE International Conference on Data Mining (ICDM’19). IEEE, 1306–1311.
[51]
Yizhou Sun, Jiawei Han, Xifeng Yan, Philip S. Yu, and Tianyi Wu. 2011. Pathsim: Meta path-based top-k similarity search in heterogeneous information networks. Proc. VLDB Endow. 4, 11 (2011), 992–1003.
[52]
Yi Tay, Anh Tuan Luu, and Siu Cheung Hui. 2018. Multi-pointer co-attention networks for recommendation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2309–2318.
[53]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE.J. Mach. Learn. Res. 9, 11 (2008), 2579–2605.
[54]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS’17). 5998–6008.
[55]
Suhang Wang, Jiliang Tang, Yilin Wang, and Huan Liu. 2018. Exploring hierarchical structures for recommender systems. IEEE Trans. Knowl. Data Eng. 30, 6 (2018), 1022–1035.
[56]
Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In Proceedings of the Annual Conference of the Association for Computing Machinery Special Interest Group in Information Retrieval (SIGIR’19). 165–174.
[57]
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. 2019. Heterogeneous graph attention network. In Proceedings of the World Wide Web Conference. 2022–2032.
[58]
Chuhan Wu, Fangzhao Wu, Tao Qi, Suyu Ge, Yongfeng Huang, and Xing Xie. 2019. Reviews meet graphs: Enhancing user and item representations for recommendation with hierarchical attentive graph neural network. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 4886–4895.
[59]
Libing Wu, Cong Quan, Chenliang Li, Qian Wang, Bolong Zheng, and Xiangyang Luo. 2019. A context-aware user-item representation learning for item recommendation. ACM Trans. Inf. Syst. 37, 2 (2019), 1–29.
[60]
Le Wu, Yonghui Yang, Kun Zhang, Richang Hong, Yanjie Fu, and Meng Wang. 2020. Joint item recommendation and attribute inference: An adaptive graph convolutional network approach. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 679–688.
[61]
Yao Wu, Christopher DuBois, Alice X. Zheng, and Martin Ester. 2016. Collaborative denoising auto-encoders for top-n recommender systems. In Proceedings of the 9th ACM International Conference on Web Search and Data Mining. 153–162.
[62]
Yao Wu and Martin Ester. 2015. Flame: A probabilistic model combining aspect based opinion mining and collaborative filtering. In Proceedings of the 8th ACM International Conference on Web Search and Data Mining. 199–208.
[63]
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 974–983.
[64]
Yongfeng Zhang, Guokun Lai, Min Zhang, Yi Zhang, Yiqun Liu, and Shaoping Ma. 2014. Explicit factor models for explainable recommendation based on phrase-level sentiment analysis. In Proceedings of the Annual Conference of the Association for Computing Machinery Special Interest Group in Information Retrieval (SIGIR’14). ACM, 83–92.
[65]
Yingxue Zhang, Soumyasundar Pal, Mark Coates, and Deniz Ustebay. 2019. Bayesian graph convolutional neural networks for semi-supervised classification. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 5829–5836.
[66]
Lei Zheng, Vahid Noroozi, and Philip S. Yu. 2017. Joint deep modeling of users and items using reviews for recommendation. In Proceedings of the International Conference on Web Search and Data Mining (WSDM’17). ACM, 425–434.

Cited By

View all
  • (2024)Skewness-aware Boosting Regression Trees for Customer Contribution Prediction in Financial Precision MarketingCompanion Proceedings of the ACM on Web Conference 202410.1145/3589335.3648346(461-470)Online publication date: 13-May-2024
  • (2024)The MMO Economist: AI Empowers Robust, Healthy, and Sustainable P2W MMO EconomiesCompanion Proceedings of the ACM on Web Conference 202410.1145/3589335.3648344(443-452)Online publication date: 13-May-2024
  • (2024)Collaborative-Enhanced Prediction of Spending on Newly Downloaded Mobile Games under Consumption UncertaintyCompanion Proceedings of the ACM on Web Conference 202410.1145/3589335.3648297(10-19)Online publication date: 13-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Information Systems
ACM Transactions on Information Systems  Volume 41, Issue 3
July 2023
890 pages
ISSN:1046-8188
EISSN:1558-2868
DOI:10.1145/3582880
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2023
Online AM: 09 November 2022
Accepted: 28 October 2022
Revised: 12 September 2022
Received: 04 April 2022
Published in TOIS Volume 41, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Recommender systems
  2. user reviews
  3. tripartite graph
  4. graph neural network
  5. Bayesian graph attention network

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)365
  • Downloads (Last 6 weeks)43
Reflects downloads up to 16 Oct 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Skewness-aware Boosting Regression Trees for Customer Contribution Prediction in Financial Precision MarketingCompanion Proceedings of the ACM on Web Conference 202410.1145/3589335.3648346(461-470)Online publication date: 13-May-2024
  • (2024)The MMO Economist: AI Empowers Robust, Healthy, and Sustainable P2W MMO EconomiesCompanion Proceedings of the ACM on Web Conference 202410.1145/3589335.3648344(443-452)Online publication date: 13-May-2024
  • (2024)Collaborative-Enhanced Prediction of Spending on Newly Downloaded Mobile Games under Consumption UncertaintyCompanion Proceedings of the ACM on Web Conference 202410.1145/3589335.3648297(10-19)Online publication date: 13-May-2024
  • (2024)A graph attention network with contrastive learning for temporal review-based recommendationsApplied Soft Computing10.1016/j.asoc.2024.111652159(111652)Online publication date: Jul-2024
  • (2023)Review-Enhanced Sequential Recommendation with Self-Attention and Graph Collaborative Features2023 IEEE International Conference on Data Mining Workshops (ICDMW)10.1109/ICDMW60847.2023.00190(1493-1499)Online publication date: 4-Dec-2023

View Options

Get Access

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media