research-article

TREND: TempoRal Event and Node Dynamics for Graph Representation Learning

Authors:

Yuan FangAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 1159 - 1169

https://doi.org/10.1145/3485447.3512164

Published: 25 April 2022 Publication History

Abstract

Temporal graph representation learning has drawn significant attention for the prevalence of temporal graphs in the real world. However, most existing works resort to taking discrete snapshots of the temporal graph, or are not inductive to deal with new nodes, or do not model the exciting effects which is the ability of events to influence the occurrence of another event. In this work, We propose TREND, a novel framework for temporal graph representation learning, driven by TempoRal Event and Node Dynamics and built upon a Hawkes process-based graph neural network (GNN). TREND presents a few major advantages: (1) it is inductive due to its GNN architecture; (2) it captures the exciting effects between events by the adoption of the Hawkes process; (3) as our main novelty, it captures the individual and collective characteristics of events by integrating both event and node dynamics, driving a more precise modeling of the temporal process. Extensive experiments on four real-world datasets demonstrate the effectiveness of our proposed model.

References

[1]

Hongyun Cai, Vincent W Zheng, and Kevin Chen-Chuan Chang. 2018. A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Transactions on Knowledge and Data Engineering 30, 9(2018), 1616–1637.

Digital Library

[2]

Lun Du, Yun Wang, Guojie Song, Zhicong Lu, and Junshan Wang. 2018. Dynamic network embedding: an extended approach for skip-gram based network embedding. In Proceedings of the International Joint Conference on Artificial Intelligence, Vol. 2018. 2086–2092.

[3]

Zhengxiao Du, Xiaowei Wang, Hongxia Yang, Jingren Zhou, and Jie Tang. 2019. Sequential scenario-specific meta learner for online recommendation. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2895–2904.

Digital Library

[4]

Ross Girshick. 2015. Fast R-CNN. In Proceedings of the IEEE International Conference on Computer Vision. 1440–1448.

[5]

Palash Goyal, Sujit Rokka Chhetri, and Arquimedes Canedo. 2020. dyngraph2vec: Capturing network dynamics using dynamic graph representation learning. Knowledge-Based Systems 187 (2020), 104816.

[6]

Palash Goyal, Nitin Kamra, Xinran He, and Yan Liu. 2018. DynGEM: Deep embedding method for dynamic graphs. arXiv preprint arXiv:1805.11273(2018).

[7]

Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 855–864.

Digital Library

[8]

David Ha, Andrew Dai, and Quoc V Le. 2017. Hypernetworks. In Proceedings of the International Conference on Learning Representations.

[9]

Ehsan Hajiramezanali, Arman Hasanzadeh, Krishna Narayanan, Nick Duffield, Mingyuan Zhou, and Xiaoning Qian. 2019. Variational Graph Recurrent Neural Networks. In Proceedings of the International Conference on Neural Information Processing Systems. 10701–10711.

[10]

William L Hamilton, Rex Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Proceedings of the International Conference on Neural Information Processing Systems. 1025–1035.

[11]

Alan G Hawkes. 1971. Spectra of some self-exciting and mutually exciting point processes. Biometrika 58, 1 (1971), 83–90.

[12]

Petter Holme and Jari Saramäki. 2012. Temporal networks. Physics reports 519, 3 (2012), 97–125.

[13]

Yugang Ji, Tianrui Jia, Yuan Fang, and Chuan Shi. 2021. Dynamic Heterogeneous Graph Embedding via Heterogeneous Hawkes Process. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, Part I. 388–403.

Digital Library

[14]

Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In Proceedings of the International Conference on Learning Representations.

[15]

Thomas N Kipf and Max Welling. 2016. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308(2016).

[16]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In Proceedings of the International Conference on Learning Representations.

[17]

Srijan Kumar, Xikun Zhang, and Jure Leskovec. 2019. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1269–1278.

Digital Library

[18]

Jure Leskovec, Jon Kleinberg, and Christos Faloutsos. 2005. Graphs over time: densification laws, shrinking diameters and possible explanations. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 177–187.

Digital Library

[19]

Aming Li, Sean P Cornelius, Y-Y Liu, Long Wang, and A-L Barabási. 2017. The fundamental advantages of temporal networks. Science 358, 6366 (2017), 1042–1046.

[20]

Jundong Li, Harsh Dani, Xia Hu, Jiliang Tang, Yi Chang, and Huan Liu. 2017. Attributed network embedding for learning in a dynamic environment. In Proceedings of the ACM Conference on Information and Knowledge Management. 387–396.

Digital Library

[21]

Taisong Li, Jiawei Zhang, S Yu Philip, Yan Zhang, and Yonghong Yan. 2018. Deep dynamic network embedding for link prediction. IEEE Access 6(2018), 29219–29230.

[22]

Yuanfu Lu, Xiao Wang, Chuan Shi, Philip S Yu, and Yanfang Ye. 2019. Temporal network embedding with micro-and macro-dynamics. In Proceedings of the ACM International Conference on Information and Knowledge Management. 469–478.

Digital Library

[23]

Hongyuan Mei and Jason M Eisner. 2017. The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 6754–6764.

[24]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the International Conference on Neural Information Processing Systems. 3111–3119.

[25]

Giang Hoang Nguyen, John Boaz Lee, Ryan A Rossi, Nesreen K Ahmed, Eunyee Koh, and Sungchul Kim. 2018. Continuous-time dynamic network embeddings. In Companion Proceedings of the The Web Conference. 969–976.

Digital Library

[26]

Pietro Panzarasa, Tore Opsahl, and Kathleen M Carley. 2009. Patterns and dynamics of users’ behavior and interaction: Network analysis of an online community. Journal of the American Society for Information Science and Technology 60, 5 (2009), 911–932.

Digital Library

[27]

Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, Tao Schardl, and Charles Leiserson. 2020. Evolvegcn: Evolving graph convolutional networks for dynamic graphs. In Proceedings of the AAAI Conference on Artificial Intelligence. 5363–5370.

[28]

James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates.

[29]

Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. 2018. FiLM: Visual reasoning with a general conditioning layer. In Proceedings of the AAAI Conference on Artificial Intelligence.

[30]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. DeepWalk: Online learning of social representations. In Proceedings of the ACM SIGKDD international conference on Knowledge discovery and data mining. 701–710.

Digital Library

[31]

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of the International Conference on Machine Learning. 1278–1286.

[32]

Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and Hao Yang. 2020. DySAT: Deep neural representation learning on dynamic graphs via self-attention networks. In Proceedings of the ACM International Conference on Web Search and Data Mining. 519–527.

Digital Library

[33]

Purnamrita Sarkar, Deepayan Chakrabarti, and Michael I Jordan. 2012. Nonparametric Link Prediction in Dynamic Networks. In Proceedings of the International Conference on Machine Learning. 1897–1904.

[34]

Uriel Singer, Ido Guy, and Kira Radinsky. 2019. Node Embedding over Temporal Graphs. In Proceedings of the International Joint Conference on Artificial Intelligence. 4605–4612.

[35]

Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. LINE: Large-scale information network embedding. In Proceedings of the International Conference on World Wide Web. 1067–1077.

Digital Library

[36]

Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. 2019. DyRep: Learning representations over dynamic graphs. In Proceedings of the International Conference on Learning Representations.

[37]

Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In Proceedings of the International Conference on Learning Representations.

[38]

Yanbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. 2021. Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks. In Proceedings of the International Conference on Learning Representations.

[39]

Zhihao Wen, Yuan Fang, and Zemin Liu. 2021. Meta-Inductive Node Classification across Graphs. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. 1219–1228.

Digital Library

[40]

Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. 2019. Simplifying graph convolutional networks. In Proceedings of the International Conference on Machine Learning. 6861–6871.

[41]

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems 32, 1(2020), 4–24.

[42]

Da Xu, Chuanwei Ruan, Evren Körpeoglu, Sushant Kumar, and Kannan Achan. 2020. Inductive representation learning on temporal graphs. In Proceedings of the International Conference on Learning Representations.

[43]

Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the International Conference on Machine learning. PMLR, 2048–2057.

Digital Library

[44]

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How Powerful are Graph Neural Networks?. In Proceedings of the International Conference on Learning Representations.

[45]

Lekui Zhou, Yang Yang, Xiang Ren, Fei Wu, and Yueting Zhuang. 2018. Dynamic network embedding by modeling triadic closure process. In Proceedings of the AAAI Conference on Artificial Intelligence.

[46]

Dingyuan Zhu, Peng Cui, Ziwei Zhang, Jian Pei, and Wenwu Zhu. 2018. High-order proximity preserved embedding for dynamic networks. IEEE Transactions on Knowledge and Data Engineering 30, 11(2018), 2134–2144.

Digital Library

[47]

Yuan Zuo, Guannan Liu, Hao Lin, Jia Guo, Xiaoqian Hu, and Junjie Wu. 2018. Embedding temporal network via neighborhood formation. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2857–2866.

Digital Library

Cited By

Zhu YGuo JLi HLiu SLi Y(2024)TSAGNNIntelligent Data Analysis10.3233/IDA-23736728:1(77-97)Online publication date: 3-Feb-2024
https://dl.acm.org/doi/10.3233/IDA-237367
Xia JLi DGu HLu TZhang PShang LGu NAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)Neural Kalman Filtering for Robust Temporal RecommendationProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635837(836-845)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635837
Wu YFang YLiao LChua TNgo CKa-Wei Lee RKumar RLauw H(2024)On the Feasibility of Simple Transformer for Dynamic Graph ModelingProceedings of the ACM on Web Conference 202410.1145/3589334.3645622(870-880)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645622
Show More Cited By

Index Terms

TREND: TempoRal Event and Node Dynamics for Graph Representation Learning

Index terms have been assigned to the content through auto-classification.

Recommendations

Time-aware Graph Structure Learning via Sequence Prediction on Temporal Graphs
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

Temporal Graph Learning, which aims to model the time-evolving nature of graphs, has gained increasing attention and achieved remarkable performance recently. However, in reality, graph structures are often incomplete and noisy, which hinders temporal ...
Temporal Graph Representation Learning with Adaptive Augmentation Contrastive
Machine Learning and Knowledge Discovery in Databases: Research Track
Abstract
Temporal graph representation learning aims to generate low-dimensional dynamic node embeddings to capture temporal information as well as structural and property information. Current representation learning methods for temporal networks often ...
Node Classification in Temporal Graphs Through Stochastic Sparsification and Temporal Structural Convolution
Machine Learning and Knowledge Discovery in Databases
Abstract
Node classification in temporal graphs aims to predict node labels based on historical observations. In real-world applications, temporal graphs are complex with both graph topology and node attributes evolving rapidly, which poses a high ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

AME Programmatic Funds of the Agency for Science, Technology and Research (A*STAR)

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
915
Total Downloads

Downloads (Last 12 months)284
Downloads (Last 6 weeks)10

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhu YGuo JLi HLiu SLi Y(2024)TSAGNNIntelligent Data Analysis10.3233/IDA-23736728:1(77-97)Online publication date: 3-Feb-2024
https://dl.acm.org/doi/10.3233/IDA-237367
Xia JLi DGu HLu TZhang PShang LGu NAngélica LLattanzi SMuñoz Medina AAkoglu LGionis AVassilvitskii S(2024)Neural Kalman Filtering for Robust Temporal RecommendationProceedings of the 17th ACM International Conference on Web Search and Data Mining10.1145/3616855.3635837(836-845)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3616855.3635837
Wu YFang YLiao LChua TNgo CKa-Wei Lee RKumar RLauw H(2024)On the Feasibility of Simple Transformer for Dynamic Graph ModelingProceedings of the ACM on Web Conference 202410.1145/3589334.3645622(870-880)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645622
Yao HYu YZhang CLin YLi S(2024)Fuzzy Representation Learning on Dynamic GraphsIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2023.332074954:2(878-890)Online publication date: Mar-2024
https://doi.org/10.1109/TSMC.2023.3320749
Liu JLiu JZhao KTang YChen W(2024)TP-GNN: Continuous Dynamic Graph Neural Network for Graph Classification2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00215(2848-2861)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00215
Bao PLi JYan RLiu Z(2024)Dynamic Graph Contrastive Learning via Maximize Temporal ConsistencyPattern Recognition10.1016/j.patcog.2023.110144148:COnline publication date: 17-Apr-2024
https://dl.acm.org/doi/10.1016/j.patcog.2023.110144
Tao HCao JChen LSun HShi YZhu X(2024)Black-box attacks on dynamic graphs via adversarial topology perturbationsNeural Networks10.1016/j.neunet.2023.11.060171:C(308-319)Online publication date: 17-Apr-2024
https://dl.acm.org/doi/10.1016/j.neunet.2023.11.060
Mu ZZhuang YTang S(2024)Contrastive Hawkes graph neural networks with dynamic sampling for event predictionNeurocomputing10.1016/j.neucom.2024.127265575:COnline publication date: 16-May-2024
https://dl.acm.org/doi/10.1016/j.neucom.2024.127265
Chen CCai FChen WZheng JZhang XLuo A(2024)BP-MoE: Behavior Pattern-aware Mixture-of-Experts for Temporal Graph Representation LearningKnowledge-Based Systems10.1016/j.knosys.2024.112056299(112056)Online publication date: Sep-2024
https://doi.org/10.1016/j.knosys.2024.112056
Guo YZang ZGao HXu XWang RLiu LLi J(2024)Unsupervised social event detection via hybrid graph contrastive learning and reinforced incremental clusteringKnowledge-Based Systems10.1016/j.knosys.2023.111225284:COnline publication date: 17-Apr-2024
https://dl.acm.org/doi/10.1016/j.knosys.2023.111225
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents