Improving Automatic Recommendation by Modeling Intrinsic and Extrinsic Motivation with Q-learning
Authors: 
Yue Zhao, Bo WangAuthors Info & Claims
Yue Zhao, Bo WangAuthors Info & ClaimsPages 1 - 7
Abstract
To understand users’ needs is an essential problem of automatic recommendation. Current researches often describe users' needs through their side attributes. However, motivation is the direct reason of users’ needs and choice behavior. In real social scenario, it is a challenge to measure the complex motivation of users. Inspired by psychological definitions, in recommendation, we propose to distinguish the Intrinsic Motivation driven by each individual's internal interests and the Extrinsic Motivation coming from the influence of social group. With latent factors of users and items, two motivations are separately modeled and integrated for recommendation with Q-learning (Off-policy), which improve the effectiveness and explainability of recommendation. Experiments on real-world datasets demonstrate the validity of our model compared with state-of-the-art methods.
References
[1]
S. Zhang, L. Yao, A. Sun, and Y. Tay, “Deep learning based recommender system: A survey and new perspectives,” ACM Comput. Surv., vol. 52, no. 1, pp. 5:1–5:38, 2019.
[2]
X. Dong, L. Yu, Z. Wu, Y. Sun, L. Yuan, and F. Zhang, “A hybrid collaborative filtering model with deep structure for recommender systems,” in Proceedings of the 21st AAAI Conference on Artificial Intelligence, 2017, pp. 1309–1315.
[3]
D. H. Kim, C. Park, J. Oh, S. Lee, and H. Yu, “Convolutional matrix factorization for document context-aware recommendation,” in Proceedings of the 10th ACM RecSys Conference on Recommender Systems, 2016, pp. 233–240.
[4]
F. Zhang, N. J. Yuan, D. Lian, X. Xie, and W. Ma, “Collaborative knowledge base embedding for recommender systems,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 353–362.
[5]
S. Sedhain, A. K. Menon, S. Sanner, and L. Xie, “Autorec : Autoencoders meet collaborative filtering,” in Proceedings of the 24th WWW International Conference on World Wide Web Companion, 2015, pp. 111–112.
[6]
Y. Shan, T. R. Hoens, J. Jiao, H. Wang, D. Yu, and J. C. Mao, “Deep crossing: Web-scale modeling without manually crafted combinatorial features,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 255– 262.
[7]
X. He, L. Liao, H. Zhang, L. Nie, X. Hu, and T. Chua, “Neural collaborative filtering,” in Proceedings of the 26th International Conference on World Wide Web, WWW 2017, 2017, pp. 173–182.
[8]
Y. Qu, H. Cai, K. Ren, W. Zhang, Y. Yu, Y. Wen, and J. Wang, “Product-based neural networks for user response prediction,” in Proceedings of the 16th ICDM IEEE International Conference on Data Mining, 2016, pp. 1149–1154.
[9]
X. He and T. Chua, “Neural factorization machines for sparse predictive analytics,” in Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2017, pp. 355–364.
[10]
W. Zhang, T. Du, and J. Wang, “Deep learning over multi-field categorical data : A case study on user response prediction,” in Proceedings of the 38th ECIR European Conference on IR Research, vol. 9626, 2016, pp. 45–57.
[11]
P. Covington, J. Adams, and E. Sargin, “Deep neural networks for YouTube recommendations,” in Proceedings of the 10th ACM Conference on Recommender Systems, 2016, pp. 191–198.
[12]
X. Wang, Y. Wang, and Y. Ling, “Attention-guide walk model in heterogeneous information network for multi-style recommendation explanation,” in Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020, pp. 6275–6282.
[13]
Q. Zhu, X. Zhou, J. Wu, J. Tan, and L. Guo, “A knowledge-aware attentional reasoning network for recommendation,” in Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020, pp. 6999– 7006.
[14]
L. Chen, Y. Liu, X. He, L. Gao, and Z. Zheng, “Matching user with item set: Collaborative bundle recommendation with deep attention network,” in Proceedings of the 28th IJCAI International Joint Conference on Artificial Intelligence, 2019, pp. 2095–2101.
[15]
J. Chen, H. Zhang, X. He, L. Nie, W. Liu, and T. Chua, “Attentive collaborative filtering: Multimedia recommendation with item- and component-level attention,” in Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2017, pp. 335–344.
[16]
G. Zhou, X. Zhu, C. Song, Y. Fan, H. Zhu, X. Ma, Y. Yan, J. Jin, H. Li, and K. Gai, “Deep interest network for click-through rate prediction,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Y. Guo and F. Farooq, Eds., 2018, pp. 1059–1068.
[17]
Y. Li and K. Mu, “Multi-attention item recommendation model based on social relations,” in Proceedings of the 12th KSEM International Conference on Knowledge Science, Engineering and Management, 2019, pp. 84–95.
[18]
N. Mu, D. Zha, Y. He, and Z. Tang, “Graph attention networks for neural social recommendation,” in 31st IEEE ICTAI International Conference on Tools with Artificial Intelligence, 2019, pp. 1320–1327.
[19]
J. Sanz-Cruzado, P. Castells, and E. Lo ́pez, “A simple multi-armed nearest-neighbor bandit for interactive recommendation,” in Proceedings of the 13th ACM RecSys Conference on Recommender Systems, 2019, pp. 358–362.
[20]
N. Gutowski, O. Camp, F. Chhel, T. Amghar, and P. Albers, “Improving bandit-based recommendations with spatial context reasoning: An online evaluation,” in 31st IEEE ICTAI International Conference on Tools with Artificial Intelligence, 2019, pp. 1366–1373.
[21]
N. Gutowski, T. Amghar, O. Camp, and F. Chhel, “Gorthaur: A portfolio approach for dynamic selection of multi-armed bandit algorithms for recommendation,” in 31st IEEE ICTAI International Conference on Tools with Artificial Intelligence, 2019, pp. 1164–1171.
[22]
H. Cheng, L. Koc, J. Harmsen, T. Shaked, T. Chandra, H. Aradhye, G. Anderson, G. Corrado, W. Chai, M. Ispir, R. Anil, Z. Haque, L. Hong, V. Jain, X. Liu, and H. Shah, “Wide & Deep learning for recommender systems,” in Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, DLRS@RecSys 2016, 2016, pp. 7–10.
[23]
X. Wang, X. He, F. Feng, L. Nie, and T. Chua, “TEM: tree-enhanced embedding model for explainable recommendation,” in Proceedings of the 27th WWW Conference on World Wide Web, 2018, pp. 1543–1552.
[24]
R. M. Ryan and E. L. Deci, “Intrinsic and extrinsic motivations: Classic definitions and new directions,” Contemporary educational psychology, vol. 25, no. 1, pp. 54–67, 2000.
[25]
R. S. Sutton and A. G. Barto, “Reinforcement learning: An introduction,” IEEE Trans. Neural Networks, vol. 9, no. 5, pp. 1054–1054, 1998.
[26]
B. M. Sarwar, G. Karypis, J. A. Konstan, and J. Riedl, “Item-based collaborative filtering recommendation algorithms,” in Proceedings of the 10th WWW International World Wide Web Conference, V. Y. Shen, N. Saito, M. R. Lyu, and M. E. Zurko, Eds., 2001, pp. 285–295.
[27]
S. Rendle, “Factorization machines,” in Proceedings of the 10th ICDM IEEE International Conference on Data Mining, 2010, pp. 995–1000.
[28]
Y. Koren, R. M. Bell, and C. Volinsky, “Matrix factorization techniques for recommender systems,” IEEE Computer, vol. 42, no. 8, pp. 30–37, 2009.
[29]
S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme, “BPR: bayesian personalized ranking from implicit feedback,” in Proceedings of the 29th UAI Conference on Uncertainty in Artificial Intelligence, 2009, pp. 452–461.
[30]
X. He, J. Pan, O. Jin, T. Xu, B. Liu, T. Xu, Y. Shi, A. Atallah, R. Herbrich, S. Bowers, and J. Q. Candela, “Practical lessons from predicting clicks on ads at facebook,” in Proceedings of the 8th ADKDD International Workshop on Data Mining for Online Advertising, 2014, pp. 5:1–5:9.
[31]
H. Zhang, F. Shen, W. Liu, X. He, H. Luan, and T. Chua, “Discrete collaborative filtering,” in Proceedings of the 39th SIGIR International ACM SIGIR conference on Research and Development in Information Retrieval, 2016, pp. 325–334.
[32]
H. Guo, R. Tang, Y. Ye, Z. Li, and X. He, “DeepFM: A factorization-machine based neural network for CTR prediction,” in Proceedings of the 26th IJCAI International Joint Conference on Artificial Intelligence, 2017, pp. 1725–1731.
[33]
G. Zheng, F. Zhang, Z. Zheng, Y. Xiang, N. J. Yuan, X. Xie, and Z. Li, “DRN: A deep reinforcement learning framework for news recommendation,” in Proceedings of the 27th WWW Conference on World Wide Web, 2018, pp. 167–176.
[34]
C. J. C. H. Watkins and P. Dayan, “Technical note Q-learning,” Mach. Learn., vol. 8, pp. 279–292, 1992.
[35]
R. Salakhutdinov and A. Mnih, “Probabilistic matrix factorization,” in Proceedings of the 21st NIPS Annual Conference on Neural Information Processing Systems, 2007, pp. 1257–1264.
[36]
H. Wang, N. Wang, and D. Yeung, “Collaborative deep learning for recommender systems,” in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015, pp. 1235–1244.
- Improving Automatic Recommendation by Modeling Intrinsic and Extrinsic Motivation with Q-learning
Recommendations
Learning Intrinsic and Extrinsic Intentions for Cold-start Recommendation with Neural Stochastic Processes
MM '22: Proceedings of the 30th ACM International Conference on MultimediaUser behavior data in recommendation are driven by the complex interactions of many intentions behind the user's decision making process. However, user behavior data tends to be sparse because of the limited user response and the vase combinations of ...
Improving Collaborative Recommendation via User-Item Subgroups
Collaborative filtering (CF) is out of question the most widely adopted and successful recommendation approach. A typical CF-based recommender system associates a user with a group of like-minded users based on their individual preferences over all the ...
Improving Cold Start Recommendation by Mapping Feature-Based Preferences to Item Comparisons
UMAP '17: Proceedings of the 25th Conference on User Modeling, Adaptation and PersonalizationMany Recommender Systems (RSs) rely on user preference data in the form of ratings or likes for items. Previous research has shown that item comparisons can also be effectively used to model user preferences and build RS. However, users often express ...
Comments
Information & Contributors
Information
Published In
May 2021
218 pages
ISBN:9781450389808
DOI:10.1145/3469968
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 06 October 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICBDC 2021
ICBDC 2021: 2021 6th International Conference on Big Data and Computing
May 22 - 24, 2021
Shenzhen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 59Total Downloads
- Downloads (Last 12 months)10
- Downloads (Last 6 weeks)0
Reflects downloads up to 15 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format