research-article

Collaborative representation learning for nodes and relations via heterogeneous graph neural network

Authors:

Can WangAuthors Info & Claims

Volume 255, Issue C

https://doi.org/10.1016/j.knosys.2022.109673

Published: 14 November 2022 Publication History

Abstract

Heterogeneous graphs, which consist of multiple types of nodes and edges, are highly suitable for characterizing real-world complex systems. In recent years, due to their strong capability of capturing rich semantics, heterogeneous graph neural networks (HGNNs) have proven to be a powerful technique for representation learning on heterogeneous graphs. However, most of the existing HGNNs only focus on learning node representations and ignore the learning of relation representations, which are complementary to node representations. To address this limitation, we propose a new HGNN model with Collaborative Representation Learning for Nodes and Relations (named CoNR) for link prediction task in this paper. Collaborative learning means that node representations and relation representations participate in and affect each other’s learning process. Specifically, node representations are obtained through a delicate two-step attention mechanism incorporating relation representations that can hierarchically aggregate information within one relation and across different relations. For relation representations, a relation encoder based on node information is designed to encode node representations into relation representations. Therefore, in this framework, node representations and relation representations are mutually updated in a layer-wise manner and work together to facilitate the downstream tasks better. Extensive experimental results on different datasets show the excellent performance of the proposed CoNR.

References

[1]

Li W., Zhu H., Li S., Wang H., Dai H., Wang C., Jin Q., Evolutionary community discovery in dynamic social networks via resistance distance, Expert Syst. Appl. 171 (2021) URL https://www.sciencedirect.com/science/article/pii/S0957417420311805.

[2]

Li W., Jiang S., Jin Q., Overlap community detection using spectral algorithm based on node convergence degree, Future Gener. Comput. Syst. 79 (2018) 408–416. URL https://doi.org/10.1016/j.future.2017.08.028.

[3]

Park C., Kim D., Zhu Q., Han J., Yu H., Task-guided pair embedding in heterogeneous network, in: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019, pp. 489–498. URL https://doi.org/10.1145/3357384.3357982.

[4]

Shi C., Hu B., Zhao W.X., Philip S.Y., Heterogeneous information network embedding for recommendation, IEEE Trans. Knowl. Data Eng. 31 (2) (2018) 357–370. URL https://doi.org/10.1109/TKDE.2018.2833443.

Digital Library

[5]

Xu F., Lian J., Han Z., Li Y., Xu Y., Xie X., Relation-aware graph convolutional networks for agent-initiated social e-commerce recommendation, in: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, 2019, pp. 529–538. URL https://doi.org/10.1145/3357384.3357924.

[6]

Zhong Q., Liu Y., Ao X., Hu B., Feng J., Tang J., He Q., Financial defaulter detection on online credit payment via multi-view attributed heterogeneous information network, in: Proceedings of the Web Conference 2020, 2020, pp. 785–795. URL https://doi.org/10.1145/3366423.3380159.

[7]

Hong H., Guo H., Lin Y., Yang X., Li Z., Ye J., An attention-based graph neural network for heterogeneous structural learning, in: Proceedings of the AAAI Conference on Artificial Intelligence, 34, 2020, pp. 4132–4139. URL https://doi.org/10.1609/aaai.v34i04.5833.

[8]

Chen H., Yin H., Wang W., Wang H., Nguyen Q.V.H., Li X., PME: projected metric embedding on heterogeneous networks for link prediction, in: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018, pp. 1177–1186. URL https://doi.org/10.1145/3219819.3219986.

[9]

Kipf T.N., Welling M., Semi-supervised classification with graph convolutional networks, in: 5th International Conference on Learning Representations, LCLR, 2017, URL https://openreview.net/forum?id=SJU4ayYgl.

[10]

Velickovic P., Cucurull G., Casanova A., Romero A., Liò P., Bengio Y., Graph attention networks, in: 6th International Conference on Learning Representations, ICLR, 2018, URL https://openreview.net/forum?id=rJXMpikCZ.

[11]

Wang X., Ji H., Shi C., Wang B., Ye Y., Cui P., Yu P.S., Heterogeneous graph attention network, in: The World Wide Web Conference, 2019, pp. 2022–2032. URL https://doi.org/10.1145/3308558.3313562.

Digital Library

[12]

Zhang C., Song D., Huang C., Swami A., Chawla N.V., Heterogeneous graph neural network, in: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 793–803. URL https://doi.org/10.1145/3292500.3330961.

Digital Library

[13]

Sun Y., Han J., Yan X., Yu P.S., Wu T., Pathsim: Meta path-based top-k similarity search in heterogeneous information networks, Proc. VLDB Endow. 4 (11) (2011) 992–1003. URL http://www.vldb.org/pvldb/vol4/p992-sun.pdf.

Digital Library

[14]

Hu Z., Dong Y., Wang K., Sun Y., Heterogeneous graph transformer, in: Proceedings of the Web Conference 2020, 2020, pp. 2704–2710. URL https://doi.org/10.1145/3366423.3380027.

Digital Library

[15]

Schlichtkrull M., Kipf T.N., Bloem P., Berg R.v.d., Titov I., Welling M., Modeling relational data with graph convolutional networks, in: European Semantic Web Conference, Springer, 2018, pp. 593–607.

[16]

Vashishth S., Sanyal S., Nitin V., Talukdar P.P., Composition-based multi-relational graph convolutional networks, in: 8th International Conference on Learning Representations, ICLR, OpenReview.net, 2020, URL https://openreview.net/forum?id=BylA_C4tPr.

[17]

Jiang J.-L., Fang H., Li S.-Q., Li W.-M., Identifying important nodes for temporal networks based on the asam model, Physica A 586 (2022) URL https://doi.org/10.1016/j.physa.2021.126455.

[18]

Perozzi B., Al-Rfou R., Skiena S., Deepwalk: Online learning of social representations, in: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2014, pp. 701–710. URL https://doi.org/10.1145/2623330.2623732.

Digital Library

[19]

Grover A., Leskovec J., Node2vec: Scalable feature learning for networks, in: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 855–864. URL https://doi.org/10.1145/2939672.2939754.

Digital Library

[20]

Mikolov T., Sutskever I., Chen K., Corrado G.S., Dean J., Distributed representations of words and phrases and their compositionality, Adv. Neural Inf. Process. Syst. 26 (2013) URL https://proceedings.neurips.cc/paper/2013/hash/9aa42b31882ec039965f3c4923ce901b-Abstract.html.

[21]

Tang J., Qu M., Wang M., Zhang M., Yan J., Mei Q., Line: Large-scale information network embedding, in: Proceedings of the 24th International Conference on World Wide Web, 2015, pp. 1067–1077. URL https://doi.org/10.1145/2736277.2741093.

Digital Library

[22]

Huang X., Li J., Hu X., Label informed attributed network embedding, in: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, 2017, pp. 731–739. URL https://doi.org/10.1145/3018661.3018667.

[23]

Li Z., Wang X., Li J., Zhang Q., Deep attributed network representation learning of complex coupling and interaction, Knowl.-Based Syst. 212 (2021) URL https://doi.org/10.1016/j.knosys.2020.106618.

[24]

Yang S., Verma S., Cai B., Jiang J., Yu K., Chen F., Yu S., Variational co-embedding learning for attributed network clustering, 2021, arXiv preprint arXiv:2104.07295.

[25]

Yin H., Yang S., Song X., Liu W., Li J., Deep fusion of multimodal features for social media retweet time prediction, World Wide Web 24 (4) (2021) 1027–1044. URL https://doi.org/10.1007/s11280-020-00850-7.

Digital Library

[26]

Gilmer J., Schoenholz S.S., Riley P.F., Vinyals O., Dahl G.E., Neural message passing for quantum chemistry, in: International Conference on Machine Learning, PMLR, 2017, pp. 1263–1272. URL http://proceedings.mlr.press/v70/gilmer17a.html.

[27]

Hamilton W., Ying Z., Leskovec J., Inductive representation learning on large graphs, Adv. Neural Inf. Process. Syst. 30 (2017) URL https://proceedings.neurips.cc/paper/2017/hash/5dd9db5e033da9c6fb5ba83c7a7ebea9-Abstract.html.

[28]

Hochreiter S., Schmidhuber J., Long short-term memory, Neural Comput. 9 (8) (1997) 1735–1780. URL https://doi.org/10.1162/neco.1997.9.8.1735.

Digital Library

[29]

He X., Deng K., Wang X., Li Y., Zhang Y., Wang M., Lightgcn: Simplifying and powering graph convolution network for recommendation, in: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 2020, pp. 639–648. URL https://doi.org/10.1145/3397271.3401063.

Digital Library

[30]

Song X., Li J., Tang Y., Zhao T., Chen Y., Guan Z., Jkt: A joint graph convolutional network based deep knowledge tracing, Inform. Sci. 580 (2021) 510–523. URL https://doi.org/10.1016/j.ins.2021.08.100.

Digital Library

[31]

Song X., Li J., Lei Q., Zhao W., Chen Y., Mian A., Bi-CLKT: Bi-graph contrastive learning based knowledge tracing, Knowl.-Based Syst. 241 (2022) URL https://doi.org/10.1016/j.knosys.2022.108274.

Digital Library

[32]

Dong Y., Chawla N.V., Swami A., Metapath2vec: Scalable representation learning for heterogeneous networks, in: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017, pp. 135–144. URL https://doi.org/10.1145/3097983.3098036.

Digital Library

[33]

Fu T.-y., Lee W.-C., Lei Z., Hin2vec: Explore meta-paths in heterogeneous information networks for representation learning, in: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, 2017, pp. 1797–1806. URL https://doi.org/10.1145/3132847.3132953.

[34]

Lu Y., Shi C., Hu L., Liu Z., Relation structure-aware heterogeneous information network embedding, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 2019, pp. 4456–4463. URL https://doi.org/10.1609/aaai.v33i01.33014456.

[35]

Fu X., Zhang J., Meng Z., King I., Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding, in: Proceedings of the Web Conference 2020, 2020, pp. 2331–2341.

Digital Library

[36]

Ma S., Liu J.-W., Zuo X., Li W.-M., Heterogeneous graph gated attention network, in: 2021 International Joint Conference on Neural Networks (IJCNN), 2021, pp. 1–6. URL https://doi.org/10.1109/IJCNN52387.2021.9533711.

[37]

Cen Y., Zou X., Zhang J., Yang H., Zhou J., Tang J., Representation learning for attributed multiplex heterogeneous network, in: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 1358–1368. URL https://doi.org/10.1145/3292500.3330964.

[38]

Yang Y., Guan Z., Li J., Zhao W., Cui J., Wang Q., Interpretable and efficient heterogeneous graph convolutional network, IEEE Trans. Knowl. Data Eng. (2021),.

[39]

Cai L., Yan B., Mai G., Janowicz K., Zhu R., Transgcn: Coupling transformation assumptions with graph convolutional networks for link prediction, in: Proceedings of the 10th International Conference on Knowledge Capture, 2019, pp. 131–138. URL https://doi.org/10.1145/3360901.3364441.

[40]

Chen C., Ma W., Zhang M., Wang Z., He X., Wang C., Liu Y., Ma S., Graph heterogeneous multi-relational recommendation, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, 2021, pp. 3958–3966. URL https://ojs.aaai.org/index.php/AAAI/article/view/16515.

[41]

Luo L., Fang Y., Cao X., Zhang X., Zhang W., Detecting communities from heterogeneous graphs: A context path-based graph neural network model, in: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 1170–1180. URL https://doi.org/10.1145/3459637.3482250.

[42]

Srivastava N., Hinton G., Krizhevsky A., Sutskever I., Salakhutdinov R., Dropout: a simple way to prevent neural networks from overfitting, J. Mach. Learn. Res. 15 (1) (2014) 1929–1958. URL http://dl.acm.org/citation.cfm?id=2670313.

Digital Library

[43]

Yang B., Yih W.-t., He X., Gao J., Deng L., Embedding entities and relations for learning and inference in knowledge bases, 2014, arXiv preprint arXiv:1412.6575.

[44]

Cantador I., Brusilovsky P., Kuflik T., Second workshop on information heterogeneity and fusion in recommender systems (HetRec2011), in: Proceedings of the Fifth ACM Conference on Recommender Systems, 2011, pp. 387–388. URL https://doi.org/10.1145/2043932.2044016.

[45]

Kingma D.P., Ba J., Adam: A method for stochastic optimization, 2014, arXiv preprint arXiv:1412.6980.

Cited By

Gu HDuan ZXie PZhou DChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Modeling Balanced Explicit and Implicit Relations with Contrastive Learning for Knowledge Concept Recommendation in MOOCsProceedings of the ACM Web Conference 202410.1145/3589334.3645559(3712-3722)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645559
Guo CLi WLiu FZhong KWu XZhao YJin Q(2024)Influence maximization algorithm based on group trust and local topology structureNeurocomputing10.1016/j.neucom.2023.126936564:COnline publication date: 7-Jan-2024
https://dl.acm.org/doi/10.1016/j.neucom.2023.126936
Luvembe ALi WLi SLiu FWu X(2024)CAF-ODNNInformation Processing and Management: an International Journal10.1016/j.ipm.2024.10365361:3Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.ipm.2024.103653
Show More Cited By

Index Terms

Collaborative representation learning for nodes and relations via heterogeneous graph neural network

Index terms have been assigned to the content through auto-classification.

Recommendations

Centrality-based Relation aware Heterogeneous Graph Neural Network
Abstract
The representation of heterogeneous graph nodes has become a hot research topic due to its diverse applications. However, extant approaches can only give consideration partly to three aspects: node structure, semantics and features. To better ...
Seq-HGNN: Learning Sequential Node Representation on Heterogeneous Graph
SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Recent years have witnessed the rapid development of heterogeneous graph neural networks (HGNNs) in information retrieval (IR) applications. Many existing HGNNs design a variety of tailor-made graph convolutions to capture structural and semantic ...
DDGK: Learning Graph Representations for Deep Divergence Graph Kernels
WWW '19: The World Wide Web Conference

Can neural networks learn to compare graphs without feature engineering? In this paper, we show that it is possible to learn representations for graph similarity with neither domain knowledge nor supervision (i.e. feature engineering or labeled graphs). ...

Comments

Information & Contributors

Information

Published In

cover image Knowledge-Based Systems

Knowledge-Based Systems Volume 255, Issue C

Nov 2022

756 pages

ISSN:0950-7051

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 14 November 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gu HDuan ZXie PZhou DChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Modeling Balanced Explicit and Implicit Relations with Contrastive Learning for Knowledge Concept Recommendation in MOOCsProceedings of the ACM Web Conference 202410.1145/3589334.3645559(3712-3722)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645559
Guo CLi WLiu FZhong KWu XZhao YJin Q(2024)Influence maximization algorithm based on group trust and local topology structureNeurocomputing10.1016/j.neucom.2023.126936564:COnline publication date: 7-Jan-2024
https://dl.acm.org/doi/10.1016/j.neucom.2023.126936
Luvembe ALi WLi SLiu FWu X(2024)CAF-ODNNInformation Processing and Management: an International Journal10.1016/j.ipm.2024.10365361:3Online publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.ipm.2024.103653
Cao YZhao XChen DHuang H(2024)Multiplex heterogeneous network representation learning with unipath based global awareness neural networkFuture Generation Computer Systems10.1016/j.future.2023.09.007150:C(317-325)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.future.2023.09.007
Wang JLi WLiu FWang ZLuvembe AJin QPan QLiu F(2024)ConeEExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.123116246:COnline publication date: 15-Jul-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.123116
Zhao ZLiu ZWang YYang DChe W(2024)RA-HGNNExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122945243:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122945
Zhao YLi WLiu FWang JLuvembe A(2024)Integrating heterogeneous structures and community semantics for unsupervised community detection in heterogeneous networksExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.121821238:PDOnline publication date: 15-Mar-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.121821
Amouzad ADehghanian ZSaravani SAmirmazlaghani MRoshanfekr B(2024)Graph isomorphism U-NetExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.121280236:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.121280
Du HMa CLu DLiu J(2024)HHSE: heterogeneous graph neural network via higher-order semantic enhancementComputing10.1007/s00607-023-01246-x106:3(865-887)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s00607-023-01246-x
Wang JLi WLiu FSheng BLiu WJin Q(2023)Hic-KGQAKnowledge-Based Systems10.1016/j.knosys.2023.110810277:COnline publication date: 9-Oct-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110810
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents