Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Dynamic Neighborhood Selection for Context Aware Temporal Evolution Using Graph Neural Networks

  • Research
  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Graph neural networks (GNN) have seen significant growth recently for modeling temporal evolution in dynamic networks. Representation of complex networks in the form of graph data structures has enabled researchers to study how entities within these networks interact with each other. These interactions evolve over time. Developing a generic methodology for modeling this temporal evolution in complex networks for tracking evolving relationships has been a significant challenge. Most of the existing methods fail to extract contextual representations of historical neighborhood interactions for future link prediction. To address these challenges, this paper presents a novel method for modeling temporal evolution in complex networks using GNNs. A Context-Aware Graph Temporal Neural Network (CATGNN) method that uses dynamic neighborhood selection based on common neighbors for a given node is presented. The method uses dynamic neighborhood selection using contextual embeddings extracted from the historical interactions of the down-sampled set of neighbors of a central node based on a common neighborhood. Fixed-sized contextual memory modules are constructed for each node that store the historical interactions of its neighbors and are updated based on the recency and significance of interactions. The proposed method has been evaluated using six real-world datasets and has comparable performance against state-of-the-art methods, both in terms of accuracy and efficiency. It shows an improvement of 7.52 to 0.05% over the baselines in terms of average precision. The results demonstrate that the proposed CATGNN model can capture complex patterns of change that are difficult to identify using traditional techniques by propagating information over the graph structure. The model can be applied in various fields involving complex systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The datasets used in this study are publicly available and can be accessed through the links and references provided in relevant sections in the paper. The project number HU24K14859 Hosei University Japan is acknowledged.

Notes

  1. https://zenodo.org/records/7213796#.Y1cO6y8r30o

  2. https://zenodo.org/records/7213796#.Y1cO6y8r30o

References

  1. Chen Q, Jiang F, Guo X, Chen J, Sha K, Wang Y. Combine temporal information in session-based recommendation with graph neural networks. Expert Syst Appl. 2024;238:121969. https://doi.org/10.1016/j.eswa.2023.121969.

    Article  Google Scholar 

  2. del Mondo G, Stell JG, Claramunt C, Thibaud R. A graph model for spatio-temporal evolution. J Univers Comput Sci. 2010;16(11):1452–77.

    MathSciNet  Google Scholar 

  3. Li Z, Wang C, Wang X, Chen Z, Li J. HJE: joint convolutional representation learning for knowledge hypergraph completion. IEEE Trans Knowl Data Eng. 2024;36(8):3879–92. https://doi.org/10.1109/TKDE.2024.3365727.

    Article  Google Scholar 

  4. Gong Y, Lv X, Yuan Z, You X, Hu F, Chen Y. GNN-based multimodal named entity recognition. Comput J. 2024;67(8):2622–32. https://doi.org/10.1093/comjnl/bxae030.

    Article  Google Scholar 

  5. Li H, Li C, Feng K, Yuan Y, Wang G, Zha H. Robust knowledge adaptation for dynamic graph neural networks. IEEE Trans Knowl Data Eng. 2024;36(11):6920–33. https://doi.org/10.1109/TKDE.2024.3388453.

    Article  Google Scholar 

  6. Shabani N, et al. A comprehensive survey on graph summarization with graph neural networks. IEEE Trans Artif Intell. 2024;5(8):3780–800. https://doi.org/10.1109/TAI.2024.3350545.

    Article  Google Scholar 

  7. Cai H, Zheng VW, Chang KC-C. A comprehensive survey of graph embedding: problems, techniques, and applications. IEEE Trans Knowl Data Eng. 2018;30(9):1616–37. https://doi.org/10.1109/TKDE.2018.2807452.

    Article  Google Scholar 

  8. Liang F, Qian C, Yu W, Griffith D, Golmie N. Survey of graph neural networks and applications. Wirel Commun Mob Comput. 2022;2022:1–18. https://doi.org/10.1155/2022/9261537.

    Article  Google Scholar 

  9. Barros CDT, Mendonça MRF, Vieira AB, Ziviani A. A survey on embedding dynamic graphs. ACM Comput Surv. 2021;55(1):1–37. https://doi.org/10.1145/3483595.

    Article  Google Scholar 

  10. Thomas JM, Moallemy-Oureh A, Beddar-Wiesing S, Holzhüter C. Graph neural networks designed for different graph types: A survey. Trans Mach Learn Res 2023. https://doi.org/10.48550/arXiv.2204.03080

  11. AlBadani B, Shi R, Dong J, Al-Sabri R, Moctard OB. Transformer-based graph convolutional network for sentiment analysis. Appl Sci. 2022;12(3):1316. https://doi.org/10.3390/app12031316.

    Article  Google Scholar 

  12. Zhang F, Zheng W, Yang Y. Graph convolutional network with syntactic dependency for aspect-based sentiment analysis. Int J Comput Intell Syst. 2024;17(1):37. https://doi.org/10.1007/s44196-024-00419-6.

    Article  Google Scholar 

  13. Allamanis M. Graph neural networks in program analysis. In: Wu L, Cui P, Pei J, Zhao L, editors. Graph neural networks: foundations, frontiers, and applications. Singapore: Springer Nature Singapore; 2022. p. 483–497. https://doi.org/10.1007/978-981-16-6054-2_22.

  14. Carbonell M, Riba P, Villegas M, Fornes A, Llados J. Named entity recognition and relation extraction with graph neural networks in semi structured documents. In 25th International Conference on Pattern Recognition (ICPR). Milan, Italy: IEEE; 2021, p. 9622–9627. https://doi.org/10.1109/ICPR48806.2021.9412669.

  15. Fan W, et al. Graph neural networks for social recommendation,” in The World Wide Web Conference. In WWW ’19. New York, NY, USA: Association for Computing Machinery; 2019, p. 417–426. https://doi.org/10.1145/3308558.3313488.

  16. Rossi E, Chamberlain B, Frasca F, Eynard D, Monti F, Bronstein M. Temporal graph networks for deep learning on dynamic graphs. In 37th International Conference on Machine Learning Workshop on Graph Representation Learning and Beyond. Vienna, Austria; 2020.

  17. Chakraborty A. Aspect based sentiment analysis using spectral temporal graph neural network. In Proceedings of the 18th International Conference on Natural Language Processing (ICON), National Institute of Technology Silchar. Silchar, India: NLP Association of India (NLPAI); 2021, p. 508–518.

  18. Lu J, Shi L, Liu G, Zhan X. Dual-channel edge-featured graph attention networks for aspect-based sentiment analysis. Electronics. 2023;12(3):624. https://doi.org/10.3390/electronics12030624.

    Article  Google Scholar 

  19. Diao Z, Wang X, Zhang D, Liu Y, Xie K, He S. Dynamic spatial-temporal graph convolutional neural networks for traffic forecasting. Proc AAAI Conf Artif Intell. 2019;33(01):890–7. https://doi.org/10.1609/aaai.v33i01.3301890.

    Article  Google Scholar 

  20. Li J, et al. Evaluating graph neural networks for link prediction: current pitfalls and new benchmarking. In: Oh A, Naumann T, Globerson A, Saenko A, Hardt M, Levine S, editors. Advances in neural information processing systems. Curran Associates, Inc.; 2023, p. 3853–3866. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2023/file/0be50b4590f1c5fdf4c8feddd63c4f67-Paper-Datasets_and_Benchmarks.pdf

  21. Bai T, Zhang Y, Wu B, Nie JY. Temporal graph neural networks for social recommendation. In IEEE International Conference on Big Data. Atlanta, GA, USA: IEEE; 2020, p. 898–903. https://doi.org/10.1109/BigData50022.2020.9378444.

  22. Fan Z, Liu Z, Zhang J, Xiong J, Zheng L, Yu PS. Continuous-time sequential recommendation with temporal graph collaborative transformer. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. Virtual Event Queensland Australia: ACM; 2021, p. 433–442. https://doi.org/10.1145/3459637.3482242.

  23. Guo J, Han Z, Zhou S, Li J, Tresp V, Wang Y. Continuous temporal graph networks for event-based graph data. In Proceedings of the 2nd Workshop on Deep Learning on Graphs for Natural Language Processing. Seattle, Washington: Association for Computational Linguistics; 2022, p. 22–29. https://doi.org/10.18653/v1/2022.dlg4nlp-1.3.

  24. Bhattacharya U, Mittal T, Chandra R, Randhavane T, Bera A, Manocha D. STEP: spatial temporal graph convolutional networks for emotion perception from gaits. Proc AAAI Conf Artif Intell. Apr.2020;34(02):1342–50. https://doi.org/10.1609/aaai.v34i02.5490.

    Article  Google Scholar 

  25. Liu Z, Wang Y, Wang S, Zhao X, Wang H, Yin H. Heterogeneous graphs neural networks based on neighbor relationship filtering. Expert Syst Appl. 2024;239:122489. https://doi.org/10.1016/j.eswa.2023.122489.

    Article  Google Scholar 

  26. Cao D, et al. Spectral temporal graph neural network for multivariate time-series forecasting. In Proceedings of the 34th International Conference on Neural Information Processing Systems, in NIPS’20. Red Hook, NY, USA: Curran Associates Inc.; 2020.

  27. Peng H, Zhang R, Dou Y, Yang R, Zhang J, Yu PS. Reinforced neighborhood selection guided multi-relational graph neural networks. ACM Trans Inf Syst. 2021;40(4):1–46. https://doi.org/10.1145/3490181.

    Article  Google Scholar 

  28. Zang X, Tang B. Self-supervised Dynamic Graph Embedding with evolutionary neighborhood and community. Expert Syst Appl. 2023;228:120409. https://doi.org/10.1016/j.eswa.2023.120409.

    Article  Google Scholar 

  29. Zhu W, Ruan K, Huang J, Xiao J, Yu W. Dynamic graph representation based on temporal and contextual contrasting. In Proceedings of the 2022 5th International Conference on Algorithms, Computing and Artificial Intelligence. Sanya China: ACM; 2022, p. 1–8. https://doi.org/10.1145/3579654.3579771.

  30. Zhang M, Wu S, Gao M, Jiang X, Xu K, Wang L. Personalized graph neural networks with attention mechanism for session-aware recommendation. IEEE Trans Knowl Data Eng. 2022;34(8):3946–57. https://doi.org/10.1109/TKDE.2020.3031329.

    Article  Google Scholar 

  31. Li Y, Wu Y, Sun M, Yang B, Wang Y. Learning continuous dynamic network representation with transformer-based temporal graph neural network. Inf Sci. 2023;649:119596. https://doi.org/10.1016/j.ins.2023.119596.

    Article  Google Scholar 

  32. Wang F, Hou D, Yan H. Detecting community evolution by utilizing individual temporal semantics in social networks. IEEE Access. 2023;11:120362–71. https://doi.org/10.1109/ACCESS.2023.3328616.

    Article  Google Scholar 

  33. Mu Z, Zhuang Y, Tang S. Contrastive Hawkes graph neural networks with dynamic sampling for event prediction. Neurocomputing. 2024;575:127265. https://doi.org/10.1016/j.neucom.2024.127265.

    Article  Google Scholar 

  34. Gao S, Li Y, Shen Y, Shao Y, Chen L. ETC: efficient training of temporal graph neural networks over large-scale dynamic graphs. Proc VLDB Endow. 2024;17(5):1060–72. https://doi.org/10.14778/3641204.3641215.

    Article  Google Scholar 

  35. Gao S, Li Y, Zhang X, Shen Y, Shao Y, Chen L. SIMPLE: efficient temporal graph neural network training at scale with dynamic data placement. Proc ACM Manag Data. 2024;2(3):1–25. https://doi.org/10.1145/3654977.

    Article  Google Scholar 

  36. Zhang H, Han X, Xiao X, Bai J. Time-aware graph structure learning via sequence prediction on temporal graphs. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. Birmingham United Kingdom: ACM; 2023, p. 3288–3297. https://doi.org/10.1145/3583780.3615081.

  37. Wu Y, Fang Y, Liao L. On the feasibility of simple transformer for dynamic graph modeling. In Proceedings of the ACM Web Conference 2024. Singapore Singapore: ACM; 2024, p. 870–880. https://doi.org/10.1145/3589334.3645622.

  38. Yu L, Sun L, Du B, Lv W. Towards better dynamic graph learning: new architecture and unified library. In Proceedings of the 37th International Conference on Neural Information Processing Systems, in NIPS ’23. Red Hook, NY, USA: Curran Associates Inc., 2024.

  39. Cong W, et al. DyFormer : a scalable dynamic graph transformer with provable benefits on generalization ability. In Proceedings of the 2023 SIAM International Conference on Data Mining (SDM). 2023, p. 442–450. https://doi.org/10.1137/1.9781611977653.ch50.

  40. Peng H, Zhang R, Dou Y, Yang R, Zhang J, Yu PS. Reinforced neighborhood selection guided multi-relational graph neural networks. ACM Trans Inf Syst. 2022;40(4):1–46. https://doi.org/10.1145/3490181.

    Article  Google Scholar 

  41. Xie Y, Li S, Yang C, Wong RCW, Han J. When do GNNs work: understanding and improving neighborhood aggregation. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, Yokohama, Japan: International Joint Conferences on Artificial Intelligence Organization. 2020, p. 1303–1309. https://doi.org/10.24963/ijcai.2020/181.

  42. Li Y, Shen Y, Chen L, Yuan M. Zebra: when temporal graph neural networks meet temporal personalized PageRank. Proc VLDB Endow. 2023;16(6):1332–45. https://doi.org/10.14778/3583140.3583150.

    Article  Google Scholar 

  43. Ghafouri S, Khasteh SH, Azarkasb SO. Influence maximization (IM) in complex networks with limited visibility using statistical methods. J Supercomput. 2024;80(5):6809–54. https://doi.org/10.1007/s11227-023-05695-1.

    Article  Google Scholar 

  44. Finkelshtein B, Huang X, Bronstein M, Ceylan II. Cooperative graph neural networks. In: Proceedings of fourty-first International Conference on Machine Learning (ICML). 2024. [Online]. Available: https://arxiv.org/abs/2310.01267. Accessed 10 Oct 2024.

  45. Kovanen L, Karsai M, Kaski K, Kertész J, Saramäki J. Temporal motifs in time-dependent networks. J Stat Mech Theory Exp. 2011;2011(11):P11005. https://doi.org/10.1088/1742-5468/2011/11/P11005.

    Article  Google Scholar 

  46. Longa A, et al. Graph neural networks for temporal graphs: state of the art, open challenges, and opportunities. Trans Mach Learn Res. 2023.

  47. Kumar S, Zhang X, Leskovec J. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Anchorage AK USA: ACM; 2019, p. 1269–1278. https://doi.org/10.1145/3292500.3330895.

  48. Poursafaei F, Huang S, Pelrine K, Rabbany R. Towards better evaluation for dynamic link prediction. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track. 2022, p. 32928–32941.

  49. Shetty J, Adibi J. The Enron email dataset database schema and brief statistical report. Information sciences institute technical report. University of Southern California; 2004. [Online]. Available: http://www.cs.cmu.edu/~enron/. Accessed 26 Aug 2024.

  50. Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y. Graph attention networks. In: Internation Conference on Learning and Representation. 2018. [Online]. Available: http://arxiv.org/abs/1710.10903. Accessed 17 Nov 2023.

  51. Hamilton WL, Ying R, Leskovec J. Inductive representation learning on large graphs. In 31st Conference on Neural Information Processing Systems. CA, USA: Long Beach; 2017.

  52. Xu D, Ruan C, Korpeoglu E, Kumar S, Achan K. Inductive representation learning on temporal graphs,” Feb. 18, 2020, arXiv: arXiv:2002.07962. [Online]. Available: http://arxiv.org/abs/2002.07962. Accessed 14 May 2023.

  53. Wang L, et al. TCL: Transformer-based dynamic graph modelling via contrastive learning. May 17, 2021, arXiv: arXiv:2105.07944. [Online]. Available: http://arxiv.org/abs/2105.07944.  Accessed 30 Aug 2024.

  54. Cong W, et al. Do we really need complicated model architectures for temporal networks? In: The Eleventh International Conference on Learning Representations. 2023. [Online]. Available: https://openreview.net/forum?id=ayPPc0SyLv1.  Accessed 30 Aug 2024.

Download references

Author information

Authors and Affiliations

  1. Institute of Computing, Kohat University of Science and Technology, Kohat, 26000, Pakistan

    Muhammad Ali Zeb & M. Irfan Uddin

  2. College of Computer Science and Engineering, University of Jeddah, 21959, Jeddah, Saudi Arabia

    Ala Abdulsalam Alarood

  3. Institute of Numerical Sciences, Kohat University of Science and Technology, Kohat, 26000, Pakistan

    Muhammad Shafiq

  4. Department of Information Systems and Technology, College of Computer Science and Engineering, University of Jeddah, Jeddah, Saudi Arabia

    Safa Habibullah

  5. Department of Information Technology et al-Kamil, University of Jeddah, Jeddah, Saudi Arabia

    Abdulkream A. Alsulami

Authors
  1. Muhammad Ali Zeb
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Irfan Uddin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ala Abdulsalam Alarood
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Muhammad Shafiq
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Safa Habibullah
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Abdulkream A. Alsulami
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

M. Ali.Zeb; conceptualization, implementation, experimentation, writing the manuscript. M. Irfan Uddin; conceptualization, visualization, critical analysis, mathematical formulation, writing manuscript. A. Abdulsalam Alarood; Mathematical analysis, supervision, reviewing manuscript. M. Shafiq.; Experimentation, resources, software, supervision, reviewing manuscript. S. Habibullah.; Visualization, Analysis, Resources, supervision, manuscript correction. A. A. Alsulami.; Conceptualization, visualization, Mathematical Analysis, Review manuscript.

Corresponding author

Correspondence to M. Irfan Uddin.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zeb, M.A., Uddin, M.I., Alarood, A.A. et al. Dynamic Neighborhood Selection for Context Aware Temporal Evolution Using Graph Neural Networks. Cogn Comput 17, 22 (2025). https://doi.org/10.1007/s12559-024-10359-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12559-024-10359-0

Keywords

Search

Navigation