research-article

Elastic Embedding through Graph Convolution-based Regression for Semi-supervised Classification

Author:

F. DornaikaAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data (TKDD), Volume 15, Issue 4

Article No.: 56, Pages 1 - 11

https://doi.org/10.1145/3441456

Published: 26 March 2021 Publication History

Abstract

This article introduces a scheme for semi-supervised learning by estimating a flexible non-linear data representation that exploits Spectral Graph Convolutions structure. Structured data are exploited in order to determine non-linear and linear models. The introduced scheme takes advantage of data-driven graphs at two levels. First, it incorporates manifold smoothness that is naturally encoded by the graph itself. Second, the regression model is built on the convolved data samples that are derived from the data and their associated graph. The proposed semi-supervised embedding can tackle the problem of over-fitting on neighborhood structures for image data. The proposed Graph Convolution-based Semi-supervised Embedding paves the way to new theoretical and application perspectives related to the non-linear embedding. Indeed, building flexible models that adopt convolved data samples can enhance both the data representation and the final performance of the learning system. Several experiments are conducted on six image datasets for comparing the introduced scheme with some state-of-the-art semi-supervised approaches. This empirical evaluation shows the effectiveness of the proposed embedding scheme.

References

[1]

Mikhail Belkin and Partha Niyogi. 2001. Laplacian eigenmaps and spectral techniques for embedding and clustering. In Advances in Neural Information Processing Systems. 585--591.

[2]

Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. 2006. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. The Journal of Machine Learning Research 7, 85 (2006), 2399--2434.

Digital Library

[3]

Deng Cai, Xiaofei He, and Jiawei Han. 2007. Semi-supervised discriminant analysis. In IEEE International Conference on Computer Vision. 1--7.

[4]

Hwann Tzong Chen, Huang Wei Chang, and Tyng Luh Liu. 2005. Local discriminant embedding and its variants. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 2. 846--853.

[5]

Fadi Dornaika and Youssof El Traboulsi. 2016. Learning flexible graph-based semi-supervised embedding. IEEE Transactions on Cybernetics 46, 1 (2016), 206--218.

[6]

F. Dornaika, M. Kejani, and A. Bosaghzadeh. 2017. Graph construction using adaptive local hybrid coding scheme. Neural Networks 95, C (2017), 91--101.

[7]

F. Dornaika and Y. El Traboulsi. 2019. Joint sparse graph and flexible embedding for graph-based semi-supervised learning. Neural Networks 114 (2019), 91--95.

Digital Library

[8]

Youssof El Traboulsi, Fadi Dornaika, and Ammar Assoum. 2015. Kernel flexible manifold embedding for pattern classification. Neurocomputing 167 (2015), 517--527.

Digital Library

[9]

X. Fang, Y. Xu, X. Li, Z. Lai, and W. K. Wong. 2015. Learning a nonnegative sparse graph for linear regression. IEEE Transactions on Image Processing 24, 9 (2015), 2760--2771.

Digital Library

[10]

Luca Franceschi, Mathias Niepert, Massimiliano Pontil, and Xiao He. 2019. Learning discrete structures for graph neural networks. In International Conference on Machine Learning. 1972--1982.

[11]

Chenping Hou, Feiping Nie, Xuelong Li, Dongyun Yi, and Yi Wu. 2014. Joint embedding learning and sparse regression: A framework for unsupervised feature selection. IEEE Transactions on Cybernetics 44, 6 (2014), 793--804.

[12]

Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations.

[13]

S. Lazebnik, C. Schmid, and J. Ponce. 2006. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In IEEE Conference on Computer Vision and Pattern Recognition. 2169--2178.

[14]

Li Jia Li and Fei Fei Li. 2007. What, where and who? Classifying events by scene and object recognition. In IEEE International Conference on Computer Vision. 1--8.

[15]

Wei Liu and Shih-Fu Chang. 2009. Robust multi-class transductive learning with graphs. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’09). IEEE, 381--388.

[16]

A. M. Martinez and A. C. Kak. 2001. PCA versus LDA. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 2 (2001), 228--233.

Digital Library

[17]

Sameer A. Nene, Shree Nayar, and Hiroshi Murase. 1996. Columbia object image library (COIL-20). Technical Report CUCS-005-96.

[18]

Feiping Nie, Dong Xu, Ivor Wai-Hung Tsang, and Changshui Zhang. 2010. Flexible manifold embedding: A framework for semi-supervised and unsupervised dimension reduction. IEEE Transactions on Image Processing 19, 7 (2010), 1921--1932.

Digital Library

[19]

Timo Ojala, Matti Pietikäinen, and David Harwood. 1996. A comparative study of texture measures with classification based on featured distributions. Pattern Recognition 29, 1 (1996), 51--59.

[20]

Lishan Qiao, Songcan Chen, and Xiaoyang Tan. 2010. Sparsity preserving discriminant analysis for single training image face recognition. Pattern Recognition Letters 31, 5 (2010), 422--429.

Digital Library

[21]

Meng Qu, Yoshua Bengio, and Jian Tang. 2019. GMNN: Graph Markov neural networks. In International Conference on Machine Learning. 5241--5250.

[22]

Bogdan Raducanu and Fadi Dornaika. 2012. A supervised non-linear dimensionality reduction approach for manifold learning. Pattern Recognition 45, 6 (2012), 2432--2444.

Digital Library

[23]

Sam T. Roweis and Lawrence K. Saul. 2000. Nonlinear dimensionality reduction by locally linear embedding. Science 290, 5500 (2000), 2323--2326.

[24]

Joshua B. Tenenbaum, Vin De Silva, and John C. Langford. 2000. A global geometric framework for nonlinear dimensionality reduction. Science 290, 5500 (2000), 2319--2323.

[25]

Fei Wang, Xin Wang, Daoqiang Zhang, Changshui Zhang, and Tao Li. 2009. MarginFace: A novel face recognition method by average neighborhood margin maximization. Pattern Recognition 42, 11 (2009), 2863--2875.

Digital Library

[26]

Hongwei Wang and Jure Leskovec. 2020. Unifying graph convolutional neural networks and label propagation. In arXiv preprint arXiv:2002.06755.

[27]

J. Wen, Y. Xu, and H. Liu. 2020. Incomplete multiview spectral clustering with adaptive graph learning. IEEE Transactions on Cybernetics 50, 4 (2020), 1418--1429.

[28]

Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. 2019. Simplifying graph convolutional networks. In International Conference on Machine Learning. 6861--6871.

[29]

Shuicheng Yan, Dong Xu, Benyu Zhang, HongJiang Zhang, Qiang Yang, and Stephen Lin. 2007. Graph embedding and extensions: A general framework for dimensionality reduction. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 1 (2007), 40--51.

Digital Library

[30]

Guoxian Yu, Guoji Zhang, Carlotta Domeniconi, Zhiwen Yu, and Jane You. 2012. Semi-supervised classification based on random subspace dimensionality reduction. Pattern Recognition 45, 3 (2012), 1119--1135.

Digital Library

[31]

Yuan Yuan, Lichao Mou, and Xiaoqiang Lu. 2015. Scene recognition by manifold regularized deep learning architecture. IEEE Transactions on Neural Networks and Learning Systems 26, 10 (2015), 2222--2233.

[32]

Ruifeng Zhu, Fadi Dornaika, and Yassine Ruichek. 2019. Joint graph based embedding and feature weighting for image classification. Pattern Recognition 93 (2019), 458--469.

Digital Library

[33]

Ruifeng Zhu, Fadi Dornaika, and Yassine Ruichek. 2019. Learning a discriminant graph-based embedding with feature selection for image categorization. Neural Networks 111 (2019), 35--46.

Digital Library

[34]

R. Zhu, F. Dornaika, and Y. Ruichek. 2020. Semi-supervised elastic manifold embedding with deep learning architecture. Pattern Recognition 107 (2020), 107425.

[35]

Xiaojin Zhu, Zoubin Ghahramani, and John Lafferty. 2003. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the 20th International Conference on Machine Learning. Vol. 3. 912--919.

Cited By

Wu LZhao HLi ZHuang ZLiu QChen E(2023)Learning the Explainable Semantic Relations via Unified Graph Topic-Disentangled Neural NetworksACM Transactions on Knowledge Discovery from Data10.1145/358996417:8(1-23)Online publication date: 12-May-2023
https://dl.acm.org/doi/10.1145/3589964
Jiang BChen YWang BXu HLuo B(2023)DropAGGNeural Networks10.1016/j.neunet.2023.03.022163:C(65-74)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1016/j.neunet.2023.03.022
Yang YSun YJu FWang SGao JYin B(2023)Multi-graph Fusion Graph Convolutional Networks with pseudo-label supervisionNeural Networks10.1016/j.neunet.2022.11.027158:C(305-317)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.neunet.2022.11.027
Show More Cited By

Index Terms

Elastic Embedding through Graph Convolution-based Regression for Semi-supervised Classification
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Machine learning

Recommendations

Semi-supervised multi-label classification using incomplete label information
Highlights
- An inductive semi-supervised method called Smile is proposed for multi-label classification using incomplete label information.
Abstract
Classifying multi-label instances using incompletely labeled instances is one of the fundamental tasks in multi-label learning. Most existing methods regard this task as supervised weak-label learning problem and assume sufficient ...
Flexible data representation with feature convolution for semi-supervised learning
Abstract
Data representation plays a crucial role in semi-supervised learning. This paper proposes a framework for semi-supervised data representation. It introduces a flexible nonlinear embedding model that integrates graph-based data convolutions. The ...
Discriminant Manifold Learning with Graph Convolution Based Regression for Image Classification
Graph-Based Representations in Pattern Recognition
Abstract
Many learning problems can be cast into learning from data-driven graphs. This paper introduces a framework for supervised and semi-supervised learning by estimating a non-linear embedding that incorporates Spectral Graph Convolutions structure. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 15, Issue 4

August 2021

486 pages

ISSN:1556-4681

EISSN:1556-472X

DOI:10.1145/3458847

Editor:
Charu Aggarwal
IBM T. J. Watson Research, USA

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 March 2021

Accepted: 01 December 2020

Received: 01 June 2020

Published in TKDD Volume 15, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
144
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)1

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wu LZhao HLi ZHuang ZLiu QChen E(2023)Learning the Explainable Semantic Relations via Unified Graph Topic-Disentangled Neural NetworksACM Transactions on Knowledge Discovery from Data10.1145/358996417:8(1-23)Online publication date: 12-May-2023
https://dl.acm.org/doi/10.1145/3589964
Jiang BChen YWang BXu HLuo B(2023)DropAGGNeural Networks10.1016/j.neunet.2023.03.022163:C(65-74)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1016/j.neunet.2023.03.022
Yang YSun YJu FWang SGao JYin B(2023)Multi-graph Fusion Graph Convolutional Networks with pseudo-label supervisionNeural Networks10.1016/j.neunet.2022.11.027158:C(305-317)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.neunet.2022.11.027
Zhao RShi F(2022)I2DKPCN: an unsupervised deep learning networkApplied Intelligence10.1007/s10489-021-03007-952:9(9938-9951)Online publication date: 10-Jan-2022
https://dl.acm.org/doi/10.1007/s10489-021-03007-9

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents