research-article

Neural Memory Streaming Recommender Networks with Adversarial Training

Authors:

Zi HuangAuthors Info & Claims

KDD '18: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Pages 2467 - 2475

https://doi.org/10.1145/3219819.3220004

Published: 19 July 2018 Publication History

Abstract

With the increasing popularity of various social media and E-commerce platforms, large volumes of user behaviour data (e.g., user transaction data, rating and review data) are being continually generated at unprecedented and ever-increasing scales. It is more realistic and practical to study recommender systems with inputs of streaming data. User-generated streaming data presents unique properties such as temporally ordered, continuous and high-velocity, which poses tremendous new challenges for the once very successful recommendation techniques. Although a few temporal or sequential recommender models have recently been developed based on recurrent neural models, most of them can only be applied to the session-based recommendation scenario, due to their short-term memories and the limited capability of capturing users' long-term stable interests. In this paper, we propose a streaming recommender model based on neural memory networks with external memories to capture and store both long-term stable interests and short-term dynamic interests in a unified way. An adaptive negative sampling framework based on Generative Adversarial Nets (GAN) is developed to optimize our proposed streaming recommender model, which effectively overcomes the limitations of classical negative sampling approaches and improves both effectiveness and efficiency of the model parameter inference. Extensive experiments have been conducted on two large-scale recommendation datasets, and the experimental results show the superiority of our proposed streaming recommender model in the streaming recommendation scenario.

References

[1]

Trapit Bansal, David Belanger, and Andrew McCallum . 2016. Ask the Gru: Multi-Task Learning for Deep Text Recommendations RecSys. 107--114.

Digital Library

[2]

Liwei Cai and William Yang Wang . 2017. KBGAN: Adversarial Learning for Knowledge Graph Embeddings. arXiv (2017).

[3]

Shiyu Chang, Yang Zhang, Jiliang Tang, Dawei Yin, Yi Chang, Mark A Hasegawa-Johnson, and Thomas S Huang . 2017. Streaming Recommender Systems. In WWW. 381--389.

Digital Library

[4]

Chen Chen, Hongzhi Yin, Junjie Yao, and Bin Cui . 2013. Terec: A Temporal Recommender System over Tweet Stream. VLDB, Vol. 6, 12 (2013), 1254--1257.

Digital Library

[5]

Ting Chen, Yizhou Sun, Yue Shi, and Liangjie Hong . 2017. On Sampling Strategies for Neural Network-based Collaborative Filtering SIGKDD. 767--776.

Digital Library

[6]

Paolo Cremonesi, Yehuda Koren, and Roberto Turrin . 2010. Performance of Recommender Algorithms on Top-N Recommendation Tasks RecSys. 39--46.

Digital Library

[7]

Peng Cui, Shaowei Liu, and Wenwu Zhu . 2018. General Knowledge Embedded Image Representation Learning. IEEE Transactions on Multimedia Vol. 20, 1 (2018), 198--207.

Digital Library

[8]

Peng Cui, Xiao Wang, Jian Pei, and Wenwu Zhu . 2017. A Survey on Network Embedding. arXiv (2017).

[9]

Abhinandan S Das, Mayur Datar, Ashutosh Garg, and Shyam Rajaram . 2007. Google News Personalization: Scalable Online Collaborative Filtering WWW. 271--280.

Digital Library

[10]

Robin Devooght, Nicolas Kourtellis, and Amin Mantrach . 2015. Dynamic Matrix Factorization with Priors on Unknown Values SIGKDD. 189--198.

Digital Library

[11]

Ernesto Diaz-Aviles, Lucas Drumond, Lars Schmidt-Thieme, and Wolfgang Nejdl . 2012. Real-Time Top-n Recommendation in Social Streams. RecSys. 59--66.

Digital Library

[12]

Jerome Friedman, Trevor Hastie, and Robert Tibshirani . 2001. The Elements of Statistical Learning. Vol. Vol. 1. Springer.

[13]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio . 2014. Generative Adversarial Nets. In NIPS. 2672--2680.

Digital Library

[14]

Alex Graves, Greg Wayne, and Ivo Danihelka . 2014. Neural Turing Machines. arXiv (2014).

[15]

Edward Grefenstette, Karl Moritz Hermann, Mustafa Suleyman, and Phil Blunsom . 2015. Learning to Transduce with Unbounded Memory. In NIPS. 1828--1836.

Digital Library

[16]

Xiangnan He, Hanwang Zhang, Min-Yen Kan, and Tat-Seng Chua . 2016. Fast Matrix Factorization for Online Recommendation with Implicit Feedback SiGIR. 549--558.

Digital Library

[17]

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk . 2015. Session-Based Recommendations with Recurrent Neural Networks. arXiv (2015).

[18]

Sepp Hochreiter and Jürgen Schmidhuber . 1997. Long Short-term Memory. Neural Computation, Vol. 9, 8 (1997), 1735--1780.

Digital Library

[19]

Cheng-Kang Hsieh, Longqi Yang, Yin Cui, Tsung-Yi Lin, Serge Belongie, and Deborah Estrin . 2017. Collaborative Metric Learning. In WWW. 193--201.

Digital Library

[20]

Bo Hu and Martin Ester . 2013. Spatial Topic Modeling in Online Social Media for Location Recommendation RecSys. 25--32.

Digital Library

[21]

Yanxiang Huang, Bin Cui, Wenyu Zhang, Jie Jiang, and Ying Xu . 2015. Tencentrec: Real-Time Stream Recommendation in Practice SIGMOD. 227--238.

Digital Library

[22]

Yehuda Koren . 2008. Factorization Meets the Neighborhood: a Multifaceted Collaborative Filtering Model SIGKDD. 426--434.

Digital Library

[23]

Minh-Thang Luong, Hieu Pham, and Christopher D Manning . 2015. Effective Approaches to Attention-Based Neural Machine Translation. arXiv (2015).

[24]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean . 2013 a. Efficient Estimation of Word Representations in Vector Space. arXiv (2013).

[25]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean . 2013 b. Distributed Representations of Words and Phrases and Their Compositionality NIPS. 3111--3119.

Digital Library

[26]

Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston . 2016. Key-value Memory Networks for Directly Reading Documents. arXiv (2016).

[27]

Volodymyr Mnih, Nicolas Heess, Alex Graves, and others . 2014. Recurrent Models of Visual Attention. In NIPS. 2204--2212.

Digital Library

[28]

David Moth . 2013. 94% of businesses say personalisation is critical to their success. (2013). https://econsultancy.com/blog/62583--94-of-businesses-say-personalisation-is-critical-to-their-success

[29]

Subhabrata Mukherjee, Hemank Lamba, and Gerhard Weikum . 2017. Item Recommendation with Evolving User Preferences and Experience. arXiv (2017).

[30]

Grace Noto . 2017. Alipay Dominates Alibaba Singles Day With 90% of Transactions. (2017). https://bankinnovation.net/2017/11/alipay-dominates-alibaba-singles-day-with-90-of-transactions/

[31]

Jan Peters and Stefan Schaal . 2008. Reinforcement Learning of Motor Skills with Policy Gradients. Neural Networks, Vol. 21, 4 (2008), 682--697.

Digital Library

[32]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme . 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback UAI. 452--461.

Digital Library

[33]

Steffen Rendle and Lars Schmidt-Thieme . 2008. Online-Updating Regularized Kernel Matrix Factorization Models for Large-Scale Recommender Systems. In RecSys. 251--258.

Digital Library

[34]

GTanmay Seth . 2017. M-Commerce Trends To Watch Out For In 2017. (2017). https://www.knowarth.com/m-commerce-trends-to-watch-out-for-in-2017/

[35]

Amit Sharma and Baoshi Yan . 2013. Pairwise Learning in Recommendation: Experiments with Community Recommendation on Linkedin RecSys. 193--200.

Digital Library

[36]

Karthik Subbian, Charu Aggarwal, and Kshiteesh Hegde . 2016. Recommendations for Streaming Data. In CIKM. 2185--2190.

Digital Library

[37]

Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, and others . 2015. End-To-End Memory Networks. In NIPS. 2440--2448.

Digital Library

[38]

Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei . 2015. Line: Large-Scale Information Network Embedding. WWW. 1067--1077.

Digital Library

[39]

Hao Wang, Yanmei Fu, Qinyong Wang, Hongzhi Yin, Changying Du, and Hui Xiong . 2017 a. A Location-Sentiment-Aware Recommender System for Both Home-Town and Out-of-Town Users KDD. 1135--1143.

Digital Library

[40]

Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, Peng Zhang, and Dell Zhang . 2017. Irgan: A minimax game for unifying generative and discriminative information retrieval models SIGIR. ACM, 515--524.

Digital Library

[41]

Sibo Wang and Yufei Tao . 2018. Efficient Algorithms for Finding Approximate Heavy Hitters in Personalized PageRanks SIGMOD. ACM.

Digital Library

[42]

Sibo Wang, Renchi Yang, Xiaokui Xiao, Zhewei Wei, and Yin Yang . 2017 b. FORA: Simple and Effective Approximate Single-Source Personalized PageRank SIGKDD. ACM, 505--514.

Digital Library

[43]

Weiqing Wang, Hongzhi Yin, Ling Chen, Yizhou Sun, Shazia Sadiq, and Xiaofang Zhou . 2015. Geo-Sage: A Geographical Sparse Additive Generative Model for Spatial Item Recommendation SIGKDD. 1255--1264.

Digital Library

[44]

Weiqing Wang, Hongzhi Yin, Qinyong Huang, Zi Wang, Xingzhong Du, and Nguyen Quoc Viet Hung . 2018. Streaming Ranking Based Recommender Systems. In SIGIR.

Digital Library

[45]

Weiqing Wang, Hongzhi Yin, Shazia Sadiq, Ling Chen, Min Xie, and Xiaofang Zhou . 2016. SPORE: A Sequential Personalized Spatial Item Recommender System ICDE. 954--965.

[46]

Jason Weston, Samy Bengio, and Nicolas Usunier . 2011. Wsabie: Scaling Up to Large Vocabulary Image Annotation IJCAI, Vol. Vol. 11. 2764--2770.

Digital Library

[47]

Jason Weston, Hector Yee, and Ron J Weiss . 2013. Learning to Rank Recommendations with the K-Order Statistic Loss RecSys. 245--248.

Digital Library

[48]

Ronald J Williams . 1992. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Machine Learning, Vol. 8, 3--4 (1992), 229--256.

Digital Library

[49]

Chao-Yuan Wu, Amr Ahmed, Alex Beutel, Alexander J Smola, and How Jing . 2017. Recurrent Recommender Networks. In WSDM. 495--503.

Digital Library

[50]

Min Xie, Hongzhi Yin, Hao Wang, Fanjiang Xu, Weitong Chen, and Sen Wang . 2016. Learning Graph-based POI Embedding for Location-Based Recommendation CIKM. 15--24.

Digital Library

[51]

Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio . 2015. Show, Attend And Tell: Neural Image Caption Generation with Visual Attention ICML. 2048--2057.

Digital Library

[52]

Hongzhi Yin, Bin Cui, Ling Chen, Zhiting Hu, and Zi Huang . 2014. A Temporal Context-Aware Model for User Behavior Modeling in Social Media Systems SIGMOD. 1543--1554.

Digital Library

[53]

Hongzhi Yin, Bin Cui, Ling Chen, Zhiting Hu, and Xiaofang Zhou . 2015. Dynamic User Modeling in Social Media Systems. TOIS, Vol. 33, 3 (2015), 10.

Digital Library

[54]

Hongzhi Yin, Bin Cui, Xiaofang Zhou, Weiqing Wang, Zi Huang, and Shazia Sadiq . 2016. Joint Modeling of User Check-In Behaviors for Real-Time Point-Of-Interest Recommendation. TOIS, Vol. 35, 2 (2016), 11.

Digital Library

[55]

Hongzhi Yin, Weiqing Wang, Hao Wang, Ling Chen, and Xiaofang Zhou . 2017. Spatial-Aware Hierarchical Collaborative Deep Learning for POI Recommendation. TKDE, Vol. 29, 11 (2017), 2537--2551.

Digital Library

[56]

Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu . 2017. SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. AAAI. 2852--2858.

[57]

Jiani Zhang, Xingjian Shi, Irwin King, and Dit-Yan Yeung . 2017. Dynamic Key-Value Memory Networks for Knowledge Tracing WWW. 765--774.

Digital Library

[58]

Yizhe Zhang, Zhe Gan, and Lawrence Carin . 2016. Generating Text via Adversarial Training. In NIPS workshop on Adversarial Training.

[59]

Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu . 2016. Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification ACL, Vol. Vol. 2. 207--212.

Cited By

Yang ZDing MHuang TCen YSong JXu BDong YTang J(2024)Does Negative Sampling Matter? a Review With Insights Into its Theory and ApplicationsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.337147346:8(5692-5711)Online publication date: Aug-2024
https://doi.org/10.1109/TPAMI.2024.3371473
Xia LHuang CXu YDai PBo L(2024)Multi-Behavior Graph Neural Networks for Recommender SystemIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.3204775(1-15)Online publication date: 2024
https://doi.org/10.1109/TNNLS.2022.3204775
Gao KLiu CWu JDu BHu W(2024)Towards a better negative sampling strategy for dynamic graphsNeural Networks10.1016/j.neunet.2024.106175(106175)Online publication date: Feb-2024
https://doi.org/10.1016/j.neunet.2024.106175
Show More Cited By

Index Terms

Neural Memory Streaming Recommender Networks with Adversarial Training
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information systems applications
    1. Data mining
      1. Collaborative filtering
  2. World Wide Web
    1. Web searching and information discovery
      1. Social recommendation

Recommendations

A Scalable, Accurate Hybrid Recommender System
WKDD '10: Proceedings of the 2010 Third International Conference on Knowledge Discovery and Data Mining

Recommender systems apply machine learning techniques for filtering unseen information and can predict whether a user would like a given resource. There are three main types of recommender systems: collaborative filtering, content-based filtering, and ...
Sequential Recommendation with User Memory Networks
WSDM '18: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining

User preferences are usually dynamic in real-world recommender systems, and a user»s historical behavior records may not be equally important when predicting his/her future interests. Existing recommendation algorithms -- including both shallow and deep ...
Improving Accuracy of Recommender System by Item Clustering

Recommender System (RS) predicts user's ratings towards items, and then recommends highly-predicted items to user. In recent years, RS has been playing more and more important role in the agent research field. There have been a great deal of researches ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

KDD '18: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

July 2018

2925 pages

ISBN:9781450355520

DOI:10.1145/3219819

General Chairs:
Yike Guo
Imperial College London
,
Faisal Farooq
IBM

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 July 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
New Staff Research Grant of The University of Queensland
ARC Discovery Early Career Researcher Award
ARC Discovery Project

Conference

KDD '18

Sponsor:

KDD '18: The 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 19 - 23, 2018

London, United Kingdom

Acceptance Rates

KDD '18 Paper Acceptance Rate 107 of 983 submissions, 11%;

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

104
Total Citations
View Citations
1,798
Total Downloads

Downloads (Last 12 months)63
Downloads (Last 6 weeks)1

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yang ZDing MHuang TCen YSong JXu BDong YTang J(2024)Does Negative Sampling Matter? a Review With Insights Into its Theory and ApplicationsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.337147346:8(5692-5711)Online publication date: Aug-2024
https://doi.org/10.1109/TPAMI.2024.3371473
Xia LHuang CXu YDai PBo L(2024)Multi-Behavior Graph Neural Networks for Recommender SystemIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.3204775(1-15)Online publication date: 2024
https://doi.org/10.1109/TNNLS.2022.3204775
Gao KLiu CWu JDu BHu W(2024)Towards a better negative sampling strategy for dynamic graphsNeural Networks10.1016/j.neunet.2024.106175(106175)Online publication date: Feb-2024
https://doi.org/10.1016/j.neunet.2024.106175
Kang JPoria SHerremans D(2024)Video2Music: Suitable music generation from videos using an Affective Multimodal Transformer modelExpert Systems with Applications10.1016/j.eswa.2024.123640249(123640)Online publication date: Sep-2024
https://doi.org/10.1016/j.eswa.2024.123640
Paul AWan YWu ZChen BGong S(2024)Deep recommendation with iteration directional adversarial trainingComputing10.1007/s00607-024-01326-6106:10(3151-3174)Online publication date: 17-Jul-2024
https://doi.org/10.1007/s00607-024-01326-6
Luo YZhang R(2024)MEOM: Memory-Efficient Online Meta-recommender for Cold-Start RecommendationWeb and Big Data10.1007/978-981-97-2390-4_23(331-346)Online publication date: 28-Apr-2024
https://doi.org/10.1007/978-981-97-2390-4_23
Nasser MYusof U(2023)Deep Learning Based Methods for Breast Cancer Diagnosis: A Systematic Review and Future DirectionDiagnostics10.3390/diagnostics1301016113:1(161)Online publication date: 3-Jan-2023
https://doi.org/10.3390/diagnostics13010161
Gao YChen JXiao LWang HPan LWen XYe ZWu X(2023)Adversarial Neural Collaborative Filtering with Embedding Dimension CorrelationsData Intelligence10.1162/dint_a_001515:3(786-806)Online publication date: 1-Aug-2023
https://doi.org/10.1162/dint_a_00151
Chen HLi XLai VYeh CFan YZheng YDas MYang H(2023)Adversarial Collaborative Filtering for FreeProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608771(245-255)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608771
Shen CZhang XWei WXu JFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)HyperBandit: Contextual Bandit with Hypernewtork for Time-Varying User Preferences in Streaming RecommendationProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614921(2239-2248)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614921
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents