research-article

Learning Temporal Interaction Graph Embedding via Coupled Memory Networks

Authors:

Jianfeng Zhang,

Can WangAuthors Info & Claims

WWW '20: Proceedings of The Web Conference 2020

Pages 3049 - 3055

https://doi.org/10.1145/3366423.3380076

Published: 20 April 2020 Publication History

Abstract

Graph embedding has become the research focus in both academic and industrial communities due to its powerful capabilities. The majority of existing work overwhelmingly learn node embeddings in the context of static, plain or attributed, homogeneous graphs. However, many real-world applications frequently involve bipartite graphs with temporal and attributed interaction edges, named temporal interaction graphs. The temporal interactions usually imply different facets of interest and might even evolve over time, thus putting forward huge challenges in learning effective node representations. In this paper, we propose a novel framework named TigeCMN to learn node representations from a sequence of temporal interactions. Specifically, we devise two coupled memory networks to store and update node embeddings in external matrices explicitly and dynamically, which forms deep matrix representations and could enhance the expressiveness of the node embeddings. We conduct experiments on two real-world datasets and the experimental results empirically demonstrate that TigeCMN can outperform the state-of-the-arts with different gains.

References

[1]

Smriti Bhagat, Graham Cormode, and S Muthukrishnan. 2011. Node classification in social networks. In Social network data analytics. Springer, 115–148.

[2]

Shaosheng Cao, Wei Lu, and Qiongkai Xu. 2015. Grarep: Learning graph representations with global structural information. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 891–900.

Digital Library

[3]

Shiyu Chang, Wei Han, Jiliang Tang, Guo-Jun Qi, Charu C Aggarwal, and Thomas S Huang. 2015. Heterogeneous network embedding via deep architectures. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 119–128.

Digital Library

[4]

Xu Chen, Hongteng Xu, Yongfeng Zhang, Jiaxi Tang, Yixin Cao, Zheng Qin, and Hongyuan Zha. 2018. Sequential recommendation with user memory networks. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. ACM, 108–116.

Digital Library

[5]

Yuxiao Dong, Nitesh V Chawla, and Ananthram Swami. 2017. metapath2vec: Scalable representation learning for heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 135–144.

Digital Library

[6]

Travis Ebesu, Bin Shen, and Yi Fang. 2018. Collaborative Memory Network for Recommendation Systems. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. ACM.

[7]

Hongchang Gao and Heng Huang. 2018. Self-paced network embedding. In SIGKDD. ACM, 1406–1415.

[8]

Ming Gao, Leihui Chen, Xiangnan He, and Aoying Zhou. 2018. BiNE: Bipartite Network Embedding. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. ACM, 715–724.

Digital Library

[9]

Palash Goyal, Nitin Kamra, Xinran He, and Yan Liu. 2018. DynGEM: Deep Embedding Method for Dynamic Graphs. arXiv preprint arXiv:1805.11273(2018).

[10]

Alex Graves, Greg Wayne, and Ivo Danihelka. 2014. Neural turing machines. arXiv preprint arXiv:1410.5401(2014).

[11]

Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwińska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, 2016. Hybrid computing using a neural network with dynamic external memory. Nature 538, 7626 (2016), 471.

[12]

Edward Grefenstette, Karl Moritz Hermann, Mustafa Suleyman, and Phil Blunsom. 2015. Learning to transduce with unbounded memory. In Advances in Neural Information Processing Systems. 1828–1836.

[13]

Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 855–864.

Digital Library

[14]

Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems. 1024–1034.

[15]

Di Jin, Mark Heimann, Ryan Rossi, and Danai Koutra. 2019. node2bits: Compact Time-and Attribute-aware Node Representations for User Stitching. arXiv preprint arXiv:1904.08572(2019).

[16]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980(2014).

[17]

Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907(2016).

[18]

Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong, Romain Paulus, and Richard Socher. 2016. Ask me anything: Dynamic memory networks for natural language processing. In International Conference on Machine Learning. 1378–1387.

[19]

Srijan Kumar, Xikun Zhang, and Jure Leskovec. 2019. Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 1269–1278.

Digital Library

[20]

Jundong Li, Harsh Dani, Xia Hu, Jiliang Tang, Yi Chang, and Huan Liu. 2017. Attributed network embedding for learning in a dynamic environment. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. ACM, 387–396.

Digital Library

[21]

Linyuan Lü and Tao Zhou. 2011. Link prediction in complex networks: A survey. Physica A: statistical mechanics and its applications 390, 6(2011), 1150–1170.

[22]

Yuanfu Lu, Xiao Wang, Chuan Shi, Philip S Yu, and Yanfang Ye. 2019. Temporal Network Embedding with Micro-and Macro-dynamics. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 469–478.

Digital Library

[23]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781(2013).

[24]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111–3119.

[25]

Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston. 2016. Key-value memory networks for directly reading documents. arXiv preprint arXiv:1606.03126(2016).

[26]

Giang Hoang Nguyen, John Boaz Lee, Ryan A Rossi, Nesreen K Ahmed, Eunyee Koh, and Sungchul Kim. 2018. Continuous-Time Dynamic Network Embeddings. In Companion of the The Web Conference 2018 on The Web Conference 2018. International World Wide Web Conferences Steering Committee, 969–976.

[27]

Shirui Pan, Jia Wu, Xingquan Zhu, Chengqi Zhang, and Yang Wang. 2016. Tri-party deep network representation. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. AAAI Press, 1895–1901.

Digital Library

[28]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 701–710.

Digital Library

[29]

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1 (2014), 1929–1958.

Digital Library

[30]

Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, 2015. End-to-end memory networks. In Advances in neural information processing systems. 2440–2448.

[31]

Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 1067–1077.

Digital Library

[32]

Rakshit Trivedi, Mehrdad Farajtbar, Prasenjeet Biswal, and Hongyuan Zha. 2018. Representation Learning over Dynamic Graphs. arXiv preprint arXiv:1803.04051(2018).

[33]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998–6008.

[34]

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 1, 2 (2017).

[35]

Daixin Wang, Peng Cui, and Wenwu Zhu. 2016. Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 1225–1234.

Digital Library

[36]

Qinyong Wang, Hongzhi Yin, Zhiting Hu, Defu Lian, Hao Wang, and Zi Huang. 2018. Neural Memory Streaming Recommender Networks with Adversarial Training. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2467–2475.

Digital Library

[37]

Jason Weston, Sumit Chopra, and Antoine Bordes. 2014. Memory Networks. CoRR abs/1410.3916(2014).

[38]

Caiming Xiong, Stephen Merity, and Richard Socher. 2016. Dynamic memory networks for visual and textual question answering. In International conference on machine learning. 2397–2406.

Digital Library

[39]

Zhilin Yang, William W Cohen, and Ruslan Salakhutdinov. 2016. Revisiting semi-supervised learning with graph embeddings. arXiv preprint arXiv:1603.08861(2016).

[40]

Jiani Zhang, Xingjian Shi, Irwin King, and Dit-Yan Yeung. 2017. Dynamic key-value memory networks for knowledge tracing. In Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 765–774.

Digital Library

[41]

Shuai Zhang, Yi Tay, Lina Yao, and Aixin Sun. 2018. Next Item Recommendation with Self-Attention. arXiv:1808.06414 (2018).

[42]

Yao Zhang, Yun Xiong, Xiangnan Kong, and Yangyong Zhu. 2017. Learning Node Embeddings in Interaction Graphs. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. ACM, 397–406.

Digital Library

[43]

Zhen Zhang, Hongxia Yang, Jiajun Bu, Sheng Zhou, Pinggang Yu, Jianwei Zhang, Martin Ester, and Can Wang. 2018. ANRL: Attributed Network Representation Learning via Deep Neural Networks. In IJCAI, Vol. 18. 3155–3161.

[44]

Le-kui Zhou, Yang Yang, Xiang Ren, Fei Wu, and Yueting Zhuang. 2018. Dynamic Network Embedding by Modeling Triadic Closure Process. In AAAI Conference on Artificial Intelligence. 571–578.

[45]

Yang Zhou, Hong Cheng, and Jeffrey Xu Yu. 2009. Graph clustering based on structural/attribute similarities. Proceedings of the VLDB Endowment 2, 1 (2009), 718–729.

Digital Library

[46]

Yuan Zuo, Guannan Liu, Hao Lin, Jia Guo, Xiaoqian Hu, and Junjie Wu. 2018. Embedding Temporal Network via Neighborhood Formation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2857–2866.

Digital Library

Cited By

Zhou SXu HZheng ZChen JLi ZBu JWu JWang XZhu WEster M(2024)A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future DirectionsACM Computing Surveys10.1145/3689036Online publication date: 16-Oct-2024
https://doi.org/10.1145/3689036
Sheng GSu JHuang CWu CBaeza-Yates RBonchi F(2024)MSPipe: Efficient Temporal GNN Training via Staleness-Aware PipelineProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671844(2651-2662)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671844
Gupta SBedathur S(2024)Robust Training of Temporal GNNs using Nearest Neighbour based Hard NegativesProceedings of the 7th Joint International Conference on Data Science & Management of Data (11th ACM IKDD CODS and 29th COMAD)10.1145/3632410.3632464(54-63)Online publication date: 4-Jan-2024
https://dl.acm.org/doi/10.1145/3632410.3632464
Show More Cited By

Index Terms

Learning Temporal Interaction Graph Embedding via Coupled Memory Networks
1. Theory of computation

Index terms have been assigned to the content through auto-classification.

Recommendations

tdGraphEmbed: Temporal Dynamic Graph-Level Embedding
CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management

Temporal dynamic graphs are graphs whose topology evolves over time, with nodes and edges added and removed between different time snapshots. Embedding such graphs in a low-dimensional space is important for a variety of tasks, including graphs' ...
GTEA: Inductive Representation Learning on Temporal Interaction Graphs via Temporal Edge Aggregation
Advances in Knowledge Discovery and Data Mining
Abstract
In this paper, we propose the Graph Temporal Edge Aggregation (GTEA) framework for inductive learning on Temporal Interaction Graphs (TIGs). Different from previous works, GTEA models the temporal dynamics of interaction sequences in the ...
Multi-view Clustering with Graph Embedding for Connectome Analysis
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

Multi-view clustering has become a widely studied problem in the area of unsupervised learning. It aims to integrate multiple views by taking advantages of the consensus and complimentary information from multiple views. Most of the existing works in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '20: Proceedings of The Web Conference 2020

April 2020

3143 pages

ISBN:9781450370233

DOI:10.1145/3366423

Editors:
Yennun Huang
Acadmica sinica, Taiwan
,
Irwin King
The Chinese University of Hong Kong, Hong Kong
,
Tie-Yan Liu
Microsoft Research Asia, China
,
Maarten van Steen
University of Twente, Netherlands

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '20

Sponsor:

SIGWEB

WWW '20: The Web Conference 2020

April 20 - 24, 2020

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
991
Total Downloads

Downloads (Last 12 months)42
Downloads (Last 6 weeks)1

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhou SXu HZheng ZChen JLi ZBu JWu JWang XZhu WEster M(2024)A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future DirectionsACM Computing Surveys10.1145/3689036Online publication date: 16-Oct-2024
https://doi.org/10.1145/3689036
Sheng GSu JHuang CWu CBaeza-Yates RBonchi F(2024)MSPipe: Efficient Temporal GNN Training via Staleness-Aware PipelineProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671844(2651-2662)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671844
Gupta SBedathur S(2024)Robust Training of Temporal GNNs using Nearest Neighbour based Hard NegativesProceedings of the 7th Joint International Conference on Data Science & Management of Data (11th ACM IKDD CODS and 29th COMAD)10.1145/3632410.3632464(54-63)Online publication date: 4-Jan-2024
https://dl.acm.org/doi/10.1145/3632410.3632464
Wang PWu DChen CLiu KFu YHuang JZhou YZhan JHua X(2024)Deep Adaptive Graph Clustering via von Mises-Fisher DistributionsACM Transactions on the Web10.1145/358052118:2(1-21)Online publication date: 8-Jan-2024
https://dl.acm.org/doi/10.1145/3580521
Zheng TFeng ZZhang THao YSong MWang XWang XZhao JChen C(2024)Transition Propagation Graph Neural Networks for Temporal NetworksIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.3220548(1-13)Online publication date: 2024
https://doi.org/10.1109/TNNLS.2022.3220548
Tang XChen L(2024)GTRL: An Entity Group-Aware Temporal Knowledge Graph Representation Learning MethodIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.333416536:9(4707-4721)Online publication date: Sep-2024
https://doi.org/10.1109/TKDE.2023.3334165
Zhu XWu YWang LSu HLi Z(2024)Continuous-Time Dynamic Interaction Network Learning Based on Evolutionary ExpectationIEEE Transactions on Cognitive and Developmental Systems10.1109/TCDS.2023.330528516:3(840-849)Online publication date: Jun-2024
https://doi.org/10.1109/TCDS.2023.3305285
Zhao JNi HJin CZheng TShi LWang X(2024)TrajGraph: A Dual-View Graph Transformer Model for Effective Next Location Recommendation2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650071(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650071
Jin YWang MXiong YRen ZHuo CZhu FZhang JWang GChen H(2024)Bridging distribution gaps: invariant pattern discovery for dynamic graph learningWorld Wide Web10.1007/s11280-024-01283-227:4Online publication date: 2-Jul-2024
https://doi.org/10.1007/s11280-024-01283-2
Fang LFeng KGui JFeng SHu A(2023)Anonymous Edge Representation for Inductive Anomaly Detection in Dynamic Bipartite GraphProceedings of the VLDB Endowment10.14778/3579075.357908816:5(1154-1167)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.14778/3579075.3579088
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents