research-article

Multi-Hot Compact Network Embedding

Authors:

Zhoujun LiAuthors Info & Claims

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

Pages 459 - 468

https://doi.org/10.1145/3357384.3357903

Published: 03 November 2019 Publication History

Abstract

Network embedding, as a promising way of the network representation learning, is capable of supporting various subsequent network mining and analysis tasks, and has attracted growing research interests recently. Traditional approaches assign each node with an independent continuous vector, which will cause memory overhead for large networks. In this paper we propose a novel multi-hot compact network embedding framework to effectively reduce memory cost by learning partially shared embeddings. The insight is that a node embedding vector is composed of several basis vectors according to a multi-hot index vector. The basis vectors are shared by different nodes, which can significantly reduce the number of continuous vectors while maintain similar data representation ability. Specifically, we propose a MCNE$_p $ model to learn compact embeddings from pre-learned node features. A novel component named compressor is integrated into MCNE$_p $ to tackle the challenge that popular back-propagation optimization cannot propagate loss through discrete samples. We further propose an end-to-end model MCNE$_t $ to learn compact embeddings from the input network directly. Empirically, we evaluate the proposed models over four real network datasets, and the results demonstrate that our proposals can save about 90% of memory cost of network embeddings without significantly performance decline.

References

[1]

Mikhail Belkin and Partha Niyogi. 2003. Laplacian eigenmaps for dimensionality reduction and data representation. Neural computation, Vol. 15, 6 (2003), 1373--1396.

[2]

Shaosheng Cao, Wei Lu, and Qiongkai Xu. 2015. Grarep: Learning graph representations with global structural information. In CIKM. ACM, 891--900.

Digital Library

[3]

Haochen Chen, Bryan Perozzi, Yifan Hu, and Steven Skiena. 2018b. Harp: Hierarchical representation learning for networks. In AAAI .

[4]

Ting Chen, Martin Renqiang Min, and Yizhou Sun. 2018a. Learning K-way D-dimensional Discrete Codes for Compact Embedding Representations. arXiv preprint arXiv:1806.09464 (2018).

[5]

Peng Cui, Xiao Wang, Jian Pei, and Wenwu Zhu. 2018. A survey on network embedding. TKDE (2018).

[6]

Quanyu Dai, Qiang Li, Jian Tang, and Dan Wang. 2017. Adversarial Network Embedding. arXiv preprint arXiv:1711.07838 (2017).

[7]

Hongchang Gao and Heng Huang. 2018. Deep Attributed Network Embedding. In IJCAI. 3364--3370.

[8]

Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In KDD. ACM, 855--864.

[9]

Qingbo Hu, Sihong Xie, Shuyang Lin, Senzhang Wang, and S Yu Philip. 2016. Clustering Embedded Approaches for Efficient Information Network Inference. Data Science and Engineering, Vol. 1, 1 (2016), 29--40.

[10]

Eric Jang, Shixiang Gu, and Ben Poole. 2016. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 (2016).

[11]

Junghwan Kim, Haekyu Park, Ji-Eun Lee, and U Kang. 2018. Side: Representation learning in signed directed networks. In WWW. 509--518.

[12]

Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).

[13]

Yann A LeCun, Léon Bottou, Genevieve B Orr, and Klaus-Robert Müller. 2012. Efficient backprop. In Neural networks: Tricks of the trade . Springer, 9--48.

Digital Library

[14]

Chaozhuo Li, Senzhang Wang, Lifang He, Philip S Yu, and Zhoujun Li. 2018. SSDMV: Semi-supervised Deep Social Spammer Detection by Multi-View Data Fusion. In ICDM .

[15]

Chaozhuo Li, Senzhang Wang, Dejian Yang, Zhoujun Li, Yang Yang, and Xiaoming Zhang. 2017. Semi-Supervised Network Embedding. In DASFAA. Springer, 131--147.

[16]

Ping Li, Trevor J Hastie, and Kenneth W Church. 2006. Very sparse random projections. In KDD. ACM, 287--296.

[17]

Jie Liu, Zhicheng He, Lai Wei, and Yalou Huang. 2018. Content to node: Self-translation network embedding. In KDD. ACM, 1794--1802.

[18]

Jianxin Ma, Peng Cui, Xiao Wang, and Wenwu Zhu. 2018. Hierarchical taxonomy aware network embedding. In KDD. ACM, 1920--1929.

Digital Library

[19]

Chris J Maddison, Andriy Mnih, and Yee Whye Teh. 2016. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint arXiv:1611.00712 (2016).

[20]

Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, Lina Yao, and Chengqi Zhang. 2018. Adversarially regularized graph autoencoder for graph embedding. arXiv preprint arXiv:1802.04407 (2018).

[21]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In KDD. ACM, 701--710.

Digital Library

[22]

Leonardo FR Ribeiro, Pedro HP Saverese, and Daniel R Figueiredo. 2017. struc2vec: Learning node representations from structural identity. In KDD. ACM, 385--394.

[23]

Ryan A Rossi, Rong Zhou, and Nesreen K Ahmed. 2018. Deep inductive network representation learning. In WWW. 953--960.

[24]

Sam T Roweis and Lawrence K Saul. 2000. Nonlinear dimensionality reduction by locally linear embedding. science, Vol. 290, 5500 (2000), 2323--2326.

[25]

Xiaobo Shen, Shirui Pan, Weiwei Liu, Yew-Soon Ong, and Quan-Sen Sun. 2018. Discrete Network Embedding. In IJCAI. 3549--3555.

[26]

Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line: Large-scale information network embedding. In WWW. ACM.

Digital Library

[27]

Joshua B Tenenbaum, Vin De Silva, and John C Langford. 2000. A global geometric framework for nonlinear dimensionality reduction. science, Vol. 290, 5500 (2000), 2319--2323.

[28]

Cunchao Tu, Weicheng Zhang, Zhiyuan Liu, and Maosong Sun. 2016. Max-margin DeepWalk: discriminative learning of network representation. In IJCAI . 3889--3895.

[29]

Curtis R Vogel. 2002. Computational methods for inverse problems. Vol. 23. Siam.

[30]

Daixin Wang, Peng Cui, and Wenwu Zhu. 2016. Structural deep network embedding. In KDD. ACM, 1225--1234.

[31]

Hongwei Wang, Jia Wang, Jialin Wang, Miao Zhao, Weinan Zhang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2017. GraphGAN: Graph Representation Learning with Generative Adversarial Nets. arXiv preprint arXiv:1711.08267 (2017).

[32]

Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S Yu. 2019. A comprehensive survey on graph neural networks. arXiv preprint arXiv:1901.00596 (2019).

[33]

Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y Chang. 2015. Network Representation Learning with Rich Text Information. In IJCAI . 2111--2117.

[34]

Daokun Zhang, Jie Yin, Xingquan Zhu, and Chengqi Zhang. 2018b. Network representation learning: A survey. IEEE transactions on Big Data (2018).

[35]

Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Maosong Sun, Zhichong Fang, Bo Zhang, and Leyu Lin. 2018a. COSINE: Compressive Network Embedding on Large-scale Information Networks. arXiv preprint arXiv:1812.08972 (2018).

Cited By

Shen LWang YLi CChen Z(2025)A Cloud API Personalized Recommendation Method Based on Multiple Attribute Features and Mashup Requirement AttentionIEEE Access10.1109/ACCESS.2024.350594313(13285-13299)Online publication date: 2025
https://doi.org/10.1109/ACCESS.2024.3505943
Yang HBao LSun XPeng Y(2024)Dual Sparse Variational Autoencoder-driven Interactive Estimation of Distribution Algorithm with User Generated Contents2024 International Conference on New Trends in Computational Intelligence (NTCI)10.1109/NTCI64025.2024.10776443(172-177)Online publication date: 18-Oct-2024
https://doi.org/10.1109/NTCI64025.2024.10776443
Lee JShin JOck C(2022)The Multi-Hot Representation-Based Language Model to Maintain Morpheme UnitsApplied Sciences10.3390/app12201061212:20(10612)Online publication date: 20-Oct-2022
https://doi.org/10.3390/app122010612
Show More Cited By

Index Terms

Multi-Hot Compact Network Embedding
1. Information systems
  1. Information systems applications
    1. Data mining

Recommendations

Adversarial learning for multi-view network embedding on incomplete graphs
Abstract
Network embedding, as a promising way of node representation learning, is capable of supporting various downstream network mining tasks, and has attracted growing research interests recently. Existing approaches mostly focus on learning the low-...
Highlights
- We investigate the novel problem of network embedding on incomplete graphs.
- We propose an adversarial learning based network embedding model MVANE.
- The experimental results demonstrate the superiority of the proposed model.
Multi-view Heterogeneous Network Embedding
Knowledge Science, Engineering and Management
Abstract
In the real world, the complex and diverse relations among different objects can be described in the form of networks. At the same time, with the emergence and development of network embedding, it has become an effective tool for processing ...
Cross-Network Embedding for Multi-Network Alignment
WWW '19: The World Wide Web Conference

Recently, data mining through analyzing the complex structure and diverse relationships on multi-network has attracted much attention in both academia and industry. One crucial prerequisite for this kind of multi-network mining is to map the nodes ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '19: Proceedings of the 28th ACM International Conference on Information and Knowledge Management

November 2019

3373 pages

ISBN:9781450369763

DOI:10.1145/3357384

General Chairs:
Wenwu Zhu
Tsinghua University, China
,
Dacheng Tao
University of Massachusetts, USA
,
Xueqi Cheng
Institute of Computing Technology, CAS, China
,
Program Chairs:
Peng Cui
Tsinghua University, China
,
Elke Rundensteiner
Worcester Polytechnic Institute, USA
,
David Carmel
Amazon Research, USA
,
Qi He
LinkedIn, USA
,
Jeffrey Xu Yu
Chinese University of Hong Kong, China

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Science Foundation of China
Beijing Advanced Innovation Center for Imaging Technology
National Key R&D Program of China

Conference

CIKM '19

Sponsor:

CIKM '19: The 28th ACM International Conference on Information and Knowledge Management

November 3 - 7, 2019

Beijing, China

Acceptance Rates

CIKM '19 Paper Acceptance Rate 202 of 1,031 submissions, 20%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
223
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shen LWang YLi CChen Z(2025)A Cloud API Personalized Recommendation Method Based on Multiple Attribute Features and Mashup Requirement AttentionIEEE Access10.1109/ACCESS.2024.350594313(13285-13299)Online publication date: 2025
https://doi.org/10.1109/ACCESS.2024.3505943
Yang HBao LSun XPeng Y(2024)Dual Sparse Variational Autoencoder-driven Interactive Estimation of Distribution Algorithm with User Generated Contents2024 International Conference on New Trends in Computational Intelligence (NTCI)10.1109/NTCI64025.2024.10776443(172-177)Online publication date: 18-Oct-2024
https://doi.org/10.1109/NTCI64025.2024.10776443
Lee JShin JOck C(2022)The Multi-Hot Representation-Based Language Model to Maintain Morpheme UnitsApplied Sciences10.3390/app12201061212:20(10612)Online publication date: 20-Oct-2022
https://doi.org/10.3390/app122010612
Pang BLi CLiu YLian JZhao JSun HDeng WXie XZhang QZhang ARangwala H(2022)Improving Relevance Modeling via Heterogeneous Behavior Graph Learning in Bing AdsProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3534678.3539128(3713-3721)Online publication date: 14-Aug-2022
https://dl.acm.org/doi/10.1145/3534678.3539128
Yan BWang PLiu JLin WLee KXu JZheng BDemartini GZuccon GCulpepper JHuang ZTong H(2021)Binary Code based Hash Embedding for Web-scale ApplicationsProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482065(3563-3567)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482065
Huang FLi CGao BLiu YAlotaibi SChen H(2021)Deep Attentive Multimodal Network Representation Learning for Social Media ImagesACM Transactions on Internet Technology10.1145/341729521:3(1-17)Online publication date: 16-Jun-2021
https://dl.acm.org/doi/10.1145/3417295
Huang FYang YZheng ZWu GMumtaz S(2021)Recognizing Influential Nodes in Social Networks With Controllability and ObservabilityIEEE Internet of Things Journal10.1109/JIOT.2020.30404878:8(6197-6204)Online publication date: 15-Apr-2021
https://doi.org/10.1109/JIOT.2020.3040487
Liu YGu RYang ZJi Y(2021)Hierarchical community discovery for multi-stage IP bearer network upgradationJournal of Network and Computer Applications10.1016/j.jnca.2021.103151189:COnline publication date: 1-Sep-2021
https://dl.acm.org/doi/10.1016/j.jnca.2021.103151

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten