research-article

Multi-level Recommendation Reasoning over Knowledge Graphs with Reinforcement Learning

Authors:

Xing XieAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 2098 - 2108

https://doi.org/10.1145/3485447.3512083

Published: 25 April 2022 Publication History

Abstract

Knowledge graphs (KGs) have been widely used to improve recommendation accuracy. The multi-hop paths on KGs also enable recommendation reasoning, which is considered a crystal type of explainability. In this paper, we propose a reinforcement learning framework for multi-level recommendation reasoning over KGs, which leverages both ontology-view and instance-view KGs to model multi-level user interests. This framework ensures convergence to a more satisfying solution by effectively transferring high-level knowledge to lower levels. Based on the framework, we propose a multi-level reasoning path extraction method, which automatically selects between high-level concepts and low-level ones to form reasoning paths that better reveal user interests. Experiments on three datasets demonstrate the effectiveness of our method.

Supplementary Material

ZIP File (supplementary.zip)

Supplement for reproducibility

Download
7.40 KB

References

[1]

Yixin Cao, Xiang Wang, Xiangnan He, Zikun Hu, and Tat-Seng Chua. 2019. Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences. In The Web Conference. 151–161.

Digital Library

[2]

Chong Chen, Min Zhang, Weizhi Ma, Yiqun Liu, and Shaoping Ma. 2020. Jointly non-sampling learning for knowledge graph enhanced recommendation. In SIGIR. 189–198.

[3]

Zhihong Cui, Hongxu Chen, Lizhen Cui, Shijun Liu, Xueyan Liu, Guandong Xu, and Hongzhi Yin. 2021. Reinforced KGs reasoning for explainable sequential recommendation. World Wide Web (2021), 1–24.

[4]

Chao Feng, Defu Lian, Xiting Wang, Zheng liu, Xing Xie, and Enhong Chen. 2022. Reinforcement Routing on Proximity Graph for Efficient Recommendation. arxiv:2201.09290 [cs.IR]

[5]

Zuohui Fu, Yikun Xian, Ruoyuan Gao, Jieyu Zhao, Qiaoying Huang, Yingqiang Ge, Shuyuan Xu, Shijie Geng, Chirag Shah, Yongfeng Zhang, 2020. Fairness-aware explainable recommendation over knowledge graphs. In SIGIR. 69–78.

[6]

Ivo Grondman, Lucian Busoniu, Gabriel AD Lopes, and Robert Babuska. 2012. A survey of actor-critic reinforcement learning: Standard and natural policy gradients. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 42, 6 (2012), 1291–1307.

Digital Library

[7]

William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In NeurIPS. 1025–1035.

[8]

Junheng Hao, Muhao Chen, Wenchao Yu, Yizhou Sun, and Wei Wang. 2019. Universal representation learning of knowledge bases by jointly embedding instances and ontological concepts. In KDD. 1709–1719.

[9]

Charles Heriot-Maitland. 2012. Multi-level models of information processing, and their application to psychosis. Journal of Experimental Psychopathology 3, 4 (2012), 552–571.

[10]

Jin Huang, Wayne Xin Zhao, Hong-Jian Dou, Ji-Rong Wen, and Edward Y. Chang. 2018. Improving Sequential Recommendation with Knowledge-Enhanced Memory Networks. In SIGIR. 505–514.

[11]

Wenqiang Lei, Gangyi Zhang, Xiangnan He, Yisong Miao, Xiang Wang, Liang Chen, and Tat-Seng Chua. 2020. Interactive path reasoning on graph for conversational recommendation. In KDD. 2073–2083.

[12]

Lihong Li, Thomas J Walsh, and Michael L Littman. 2006. Towards a Unified Theory of State Abstraction for MDPs. In ISAIM, Vol. 4. 5.

[13]

Danyang Liu, Jianxun Lian, Zheng Liu, Xiting Wang, Guangzhong Sun, and Xing Xie. 2021. Reinforced Anchor Knowledge Graph Generation for News Recommendation Reasoning. In KDD. 1055–1065.

[14]

Weizhi Ma, Min Zhang, Yue Cao, Woojeong Jin, Chenyang Wang, Yiqun Liu, Shaoping Ma, and Xiang Ren. 2019. Jointly Learning Explainable Rules for Recommendation with Knowledge Graph. In The Web Conference. 1210–1221.

[15]

Andrew Y Ng, Daishi Harada, and Stuart Russell. 1999. Policy invariance under reward transformations: Theory and application to reward shaping. In ICML, Vol. 99. 278–287.

[16]

Chien-Chun Ni, Kin Sum Liu, and Nicolas Torzec. 2020. Layered graph embedding for entity recommendation using wikipedia in the yahoo! knowledge graph. In Companion Proceedings of the Web Conference. 811–818.

Digital Library

[17]

Enrico Palumbo, Diego Monti, Giuseppe Rizzo, Raphaël Troncy, and Elena Baralis. 2020. entity2rec: Property-specific knowledge graph embeddings for item recommendation. Expert Systems with Applications 151 (2020), 113235.

[18]

Dean Pomerleau. 1991. Efficient Training of Artificial Neural Networks for Autonomous Navigation. Neural Computation 3, 1 (1991), 88–97.

[19]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback. In UAI. 452–461.

Digital Library

[20]

Ben Shneiderman. 2003. The eyes have it: A task by data type taxonomy for information visualizations. In The craft of information visualization. Elsevier, 364–371.

[21]

Xiaoli Tang, Tengyun Wang, Haizhi Yang, and Hengjie Song. 2019. AKUPM: Attention-enhanced knowledge-aware user preference model for recommendation. In KDD. 1891–1899.

Digital Library

[22]

Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2018. RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems. In CIKM. 417–426.

[23]

Hongwei Wang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018. DKN: Deep Knowledge-Aware Network for News Recommendation. In The Web Conference. 1835–1844.

Digital Library

[24]

Hongwei Wang, Fuzheng Zhang, Mengdi Zhang, Jure Leskovec, Miao Zhao, Wenjie Li, and Zhongyuan Wang. 2019. Knowledge-aware graph neural networks with label smoothness regularization for recommender systems. In KDD. 968–977.

[25]

Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019. Kgat: Knowledge graph attention network for recommendation. In KDD. 950–958.

Digital Library

[26]

Xiang Wang, Tinglin Huang, Dingxian Wang, Yancheng Yuan, Zhenguang Liu, Xiangnan He, and Tat-Seng Chua. 2021. Learning Intents behind Interactions with Knowledge Graph for Recommendation. In The Web Conference. 878–887.

Digital Library

[27]

Xiang Wang, Dingxian Wang, Canran Xu, Xiangnan He, Yixin Cao, and Tat-Seng Chua. 2019. Explainable Reasoning over Knowledge Graphs for Recommendation. In AAAI. 5329–5336.

[28]

Zhongyuan Wang, Haixun Wang, Ji-Rong Wen, and Yanghua Xiao. 2015. An inference approach to basic level of categorization. In CIKM. ACM, 653–662.

[29]

Wentao Wu, Hongsong Li, Haixun Wang, and Kenny Q Zhu. 2012. Probase: A probabilistic taxonomy for text understanding. In SIGMOD. ACM, 481–492.

Digital Library

[30]

Yikun Xian, Zuohui Fu, S. Muthukrishnan, Gerard de Melo, and Yongfeng Zhang. 2019. Reinforcement Knowledge Graph Reasoning for Explainable Recommendation. In SIGIR. 285–294.

[31]

Yikun Xian, Zuohui Fu, Handong Zhao, Yingqiang Ge, Xu Chen, Qiaoying Huang, Shijie Geng, Zhou Qin, Gerard De Melo, Shan Muthukrishnan, 2020. CAFE: Coarse-to-fine neural symbolic reasoning for explainable recommendation. In CIKM. 1645–1654.

Digital Library

[32]

Weikai Yang, Xiting Wang, Jie Lu, Wenwen Dou, and Shixia Liu. 2020. Interactive steering of hierarchical clustering. IEEE Transactions on Visualization and Computer Graphics (2020).

[33]

Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. 2016. Collaborative Knowledge Base Embedding for Recommender Systems. In KDD. 353–362.

[34]

Wei Zhang, Quan Yuan, Jiawei Han, and Jianyong Wang. 2016. Collaborative multi-Level embedding learning from reviews for rating prediction. In IJCAI. 2986–2992.

[35]

Jun Zhao, Zhou Zhou, Ziyu Guan, Wei Zhao, Wei Ning, Guang Qiu, and Xiaofei He. 2019. Intentgc: a scalable graph convolution framework fusing heterogeneous information for recommendation. In KDD. 2347–2357.

[36]

Kangzhi Zhao, Xiting Wang, Yuren Zhang, Li Zhao, Zheng Liu, Chunxiao Xing, and Xing Xie. 2020. Leveraging Demonstrations for Reinforcement Recommendation Reasoning over Knowledge Graphs. In SIGIR. 239–248.

[37]

Yuyue Zhao, Xiang Wang, Jiawei Chen, Wei Tang, Yashen Wang, Xiangnan He, and Haiyong Xie. 2021. Time-aware Path Reasoning on Knowledge Graph for Recommendation. arXiv preprint arXiv:2108.02634(2021).

[38]

Kun Zhou, Wayne Xin Zhao, Shuqing Bian, Yuanhang Zhou, Ji-Rong Wen, and Jingsong Yu. 2020. Improving conversational recommender systems via knowledge graph based semantic fusion. In KDD. 1006–1014.

[39]

Qiannan Zhu, Xiaofei Zhou, Jia Wu, Jianlong Tan, and Li Guo. 2020. A knowledge-aware attentional reasoning network for recommendation. In AAAI. 6999–7006.

[40]

Yaxin Zhu, Yikun Xian, Zuohui Fu, Gerard de Melo, and Yongfeng Zhang. 2021. Faithfully Explainable Recommendation via Neural Logic Reasoning. In NAACL.

Cited By

Sacilotto RMathur SGonsalves R(2024)Connecting Asset Stores with Graph DatabasesSMPTE Motion Imaging Journal10.5594/JMI.2024/XBZC2345133:2(28-36)Online publication date: Apr-2024
https://doi.org/10.5594/JMI.2024/XBZC2345
Cui YZhou PYu HSun PCao HYang P(2024)ASKAT: Aspect Sentiment Knowledge Graph Attention Network for RecommendationElectronics10.3390/electronics1301021613:1(216)Online publication date: 3-Jan-2024
https://doi.org/10.3390/electronics13010216
Luo TLiu YPan S(2024)Collaborative Sequential Recommendations via Multi-view GNN-transformersACM Transactions on Information Systems10.1145/364943642:6(1-27)Online publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1145/3649436
Show More Cited By

Index Terms

Multi-level Recommendation Reasoning over Knowledge Graphs with Reinforcement Learning
1. Computing methodologies
  1. Machine learning
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems
  2. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Reinforcement Knowledge Graph Reasoning for Explainable Recommendation
SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

Recent advances in personalized recommendation have sparked great interest in the exploitation of rich structured information provided by knowledge graphs. Unlike most existing approaches that only focus on leveraging knowledge graphs for more accurate ...
Reinforcement recommendation reasoning through knowledge graphs for explanation path quality
Abstract
Numerous Knowledge Graphs (KGs) are being created to make Recommender Systems (RSs) not only intelligent but also knowledgeable. Integrating a KG in the recommendation process allows the underlying model to extract reasoning paths ...
Reinforced Anchor Knowledge Graph Generation for News Recommendation Reasoning
KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining

News recommendation systems play a key role in online news reading service. Knowledge graphs (KG), which contain comprehensive structural knowledge, are well known for their potential to enhance both accuracy and explainability. While existing works ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

41
Total Citations
View Citations
1,233
Total Downloads

Downloads (Last 12 months)355
Downloads (Last 6 weeks)9

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sacilotto RMathur SGonsalves R(2024)Connecting Asset Stores with Graph DatabasesSMPTE Motion Imaging Journal10.5594/JMI.2024/XBZC2345133:2(28-36)Online publication date: Apr-2024
https://doi.org/10.5594/JMI.2024/XBZC2345
Cui YZhou PYu HSun PCao HYang P(2024)ASKAT: Aspect Sentiment Knowledge Graph Attention Network for RecommendationElectronics10.3390/electronics1301021613:1(216)Online publication date: 3-Jan-2024
https://doi.org/10.3390/electronics13010216
Luo TLiu YPan S(2024)Collaborative Sequential Recommendations via Multi-view GNN-transformersACM Transactions on Information Systems10.1145/364943642:6(1-27)Online publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1145/3649436
Rong HQian MMa TJin DSheng V(2024)CoBjeason: Reasoning Covered Object in Image by Multi-Agent Collaboration Based on Informed Knowledge GraphACM Transactions on Knowledge Discovery from Data10.1145/364356518:5(1-56)Online publication date: 28-Feb-2024
https://dl.acm.org/doi/10.1145/3643565
Xiao MWang DWu MLiu KXiong HZhou YFu Y(2024)Traceable Group-Wise Self-Optimizing Feature Transformation Learning: A Dual Optimization PerspectiveACM Transactions on Knowledge Discovery from Data10.1145/363805918:4(1-22)Online publication date: 13-Feb-2024
https://dl.acm.org/doi/10.1145/3638059
Frej JShah NKnezevic MNazaretsky TKäser T(2024)Finding Paths for Explainable MOOC Recommendation: A Learner PerspectiveProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636898(426-437)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3636555.3636898
Zhou EGuo SMa ZHong ZGuo TDong PChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Poisoning Attack on Federated Knowledge Graph EmbeddingProceedings of the ACM on Web Conference 202410.1145/3589334.3645422(1998-2008)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645422
Li YLiu KSatapathy RWang SCambria E(2024)Recent Developments in Recommender Systems: A Survey [Review Article]IEEE Computational Intelligence Magazine10.1109/MCI.2024.336398419:2(78-95)Online publication date: May-2024
https://doi.org/10.1109/MCI.2024.3363984
Wang HWang KZhang WZhang Y(2024)Bipartite Graph Analytics: Current Techniques and Future Trends2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00405(01-07)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00405
Tiwary NMohd Noah SFauzi FYee T(2024)A Review of Explainable Recommender Systems Utilizing Knowledge Graphs and Reinforcement LearningIEEE Access10.1109/ACCESS.2024.342241612(91999-92019)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3422416
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents