research-article

Subspace clustering guided unsupervised feature selection

Authors:

Changqing Zhang,

Wangmeng ZuoAuthors Info & Claims

Pattern Recognition, Volume 66, Issue C

Pages 364 - 374

https://doi.org/10.1016/j.patcog.2017.01.016

Published: 01 June 2017 Publication History

Abstract

Unsupervised feature selection (UFS) aims to reduce the time complexity and storage burden, improve the generalization ability of learning machines by removing the redundant, irrelevant and noisy features. Due to the lack of training labels, most existing UFS methods generate the pseudo labels by spectral clustering, matrix factorization or dictionary learning, and convert UFS to a supervised problem. The learned clustering labels reflect the data distribution with respect to classes and therefore are vital to the UFS performance. In this paper, we proposed a novel subspace clustering guided unsupervised feature selection (SCUFS) method. The clustering labels of the training samples are learned by representation based subspace clustering, and features that can well preserve the cluster labels are selected. SCUFS can well learn the data distribution in that it uncovers the underlying multi-subspace structure of the data and iteratively learns the similarity matrix and clustering labels. Experimental results on benchmark datasets for unsupervised feature selection show that SCUFS outperforms the state-of-the-art UFS methods. HighlightsA novel subspace clustering guided unsupervised feature selection (SCUFS) model is proposed.SCUFS learns a similarity graph by self-representation of samples and can uncover the underlying multi-subspace structure of data.The iterative updating of similarity graph and pseudo label matrix can learn a more accurate data distribution.

References

[1]

Y. Zhai, Y.S. Ong, I.W. Tsang, The emerging big dimensionality, Comput. Intell. Mag. IEEE, 9 (2014) 14-26.

Digital Library

[2]

V.D. Mil'Man, New proof of the theorem of a. dvoretzky on intersections of convex bodies, Funct. Anal. Appl., 5 (1971) 288-295.

[3]

X. Lu, X. Li, Multiresolution imaging, IEEE Trans. Cybern., 44 (2014) 149-160.

[4]

X. Lu, Y. Wang, Y. Yuan, Sparse coding from a bayesian perspective, IEEE Trans. Neural Netw. Learn. Syst., 24 (2013) 929-939.

[5]

L. Shao, L. Liu, X. Li, Feature learning for image classification via multiobjective genetic programming, IEEE Trans. Neural Netw. Learn. Syst., 25 (2014) 1359-1371.

[6]

X. Zhu, S. Zhang, Z. Jin, Z. Zhang, Z. Xu, Missing value estimation for mixed-attribute data sets, IEEE Trans. Knowl. Data Eng., 23 (2011) 110-121.

Digital Library

[7]

Y. Yang, Z. Ma, F. Nie, X. Chang, A.G. Hauptmann, Multi-class active learning by uncertainty sampling with diversity maximization, Int. J. Comput. Vis., 113 (2015) 113-127.

Digital Library

[8]

X. He, D. Cai, P. Niyogi, Laplacian score for feature selection, in: Advances in neural information processing systems, pp. 507514.

Digital Library

[9]

S. Wang, J. Tang, H. Liu, Embedded unsupervised feature selection, in: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, pp. 470476.

Digital Library

[10]

L. Du, Y.-D. Shen, Unsupervised feature selection with adaptive structure learning, in: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 15, ACM, New York, NY, USA, 2015, pp. 209218.

Digital Library

[11]

X. Zhu, X. Li, S. Zhang, C. Ju, X. Wu, Robust joint graph sparse coding for unsupervised spectral feature selection, IEEE Trans. Neural Netw. Learn. Syst. (2016) 1-13.

[12]

X. Chang, F. Nie, Y. Yang, H. Huang, A convex formulation for semisupervised multi-label feature selection, in: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence.

Digital Library

[13]

F. Nie, H. Huang, X. Cai, C. H. Ding, Efficient and robust feature selection via joint l2,1-norms minimization, in: Advances in neural information processing systems, pp. 18131821.

Digital Library

[14]

R. He, T. Tan, L. Wang, W.-S. Zheng, l2,1 regularized correntropy for robust feature selection, in: Computer Vision and Pattern Recognition (CVPR), pp. 25042511.

Digital Library

[15]

Y. Han, Y. Yang, X. Zhou, Co-regularized ensemble for feature selection, in: Proceedings of the 23rd International Joint Conference on Artificial Intelligence, IJCAI 2013, pp. 13801386.

Digital Library

[16]

H. Gao, F. Nie, X. Li, H. Huang, Multi-view subspace clustering, in: Proceedings of the IEEE International Conference on Computer Vision, pp. 42384246.

Digital Library

[17]

M. Qian, C. Zhai, Unsupervised feature selection for multi-view clustering on text-image web news data, in:Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 19631966.

Digital Library

[18]

Y. Feng, J. Xiao, Y. Zhuang, X. Liu, Adaptive unsupervised multi-view feature selection for visual concept recognition, in: Computer Vision-ACCV 2012, Springer, 2012, pp. 343357.

Digital Library

[19]

J. Tang, X. Hu, H. Gao, H. Liu, Unsupervised feature selection for multi-view data in social media., in: SDM, SIAM, pp. 270278.

[20]

Y. Lei, L. Jun, Y. Jieping, Efficient methods for overlapping group lasso, IEEE Trans. Pattern Anal. Mach. Intell., 35 (2013) 2104-2116.

Digital Library

[21]

J. Wang, J. Ye, Multi-layer feature reduction for tree structured group lasso via hierarchical projection, in: Advances in Neural Information Processing Systems, pp. 12791287.

Digital Library

[22]

T. Gao, Z. Wang, Q. Ji, Structured feature selection, in: Proceedings of the IEEE International Conference on Computer Vision, pp. 42564264.

Digital Library

[23]

X. Wu, K. Yu, W. Ding, H. Wang, X. Zhu, Online feature selection with streaming features, IEEE Trans. Pattern Anal. Mach. Intell., 35 (2013) 1178-1192.

Digital Library

[24]

J. Wang, M. Wang, P. Li, L. Liu, Z. Zhao, X. Hu, X. Wu, Online feature selection with group structure analysis, IEEE Trans. Knowl. Data Eng., 27 (2015) 3029-3041.

Digital Library

[25]

K. Yu, W. Ding, X. Wu, Lofs: Library of online streaming feature selection, arXiv:1603.00531(2016)

[26]

J. Tang, H. Liu, Unsupervised feature selection for linked social media data, in:Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 904912.

Digital Library

[27]

F. Nie, S. Xiang, Y. Jia, C. Zhang, S. Yan, Trace ratio criterion for feature selection., in: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence.

Digital Library

[28]

R. Kohavi, G.H. John, Wrappers for feature subset selection, Artif. Intell., 97 (1997) 273-324.

Digital Library

[29]

J. Weston, S. Mukherjee, O. Chapelle, M. Pontil, T. Poggio, V. Vapnik, Feature selection for svms, in: NIPS, volume 12, Citeseer, pp. 668674.

Digital Library

[30]

D. Cai, C. Zhang, X. He, Unsupervised feature selection for multi-cluster data, in:Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 333342.

Digital Library

[31]

Z. Zhao, L. Wang, H. Liu, Efficient spectral feature selection with minimum redundancy, in: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence.

Digital Library

[32]

Z. Li, Y. Yang, J. Liu, X. Zhou, H. Lu, Unsupervised feature selection using nonnegative spectral analysis., in: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence.

Digital Library

[33]

M. Qian, C. Zhai, Robust unsupervised feature selection, in:Proceedings of the Twenty-Third international joint conference on Artificial Intelligence, pp. 16211627.

Digital Library

[34]

P. Zhu, Q. Hu, C. Zhang, W. Zuo, Coupled dictionary learning for unsupervised feature selection, in: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence.

Digital Library

[35]

H. Liu, M. Shao, Y. Fu, Consensus guided unsupervised feature selection, in: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence.

Digital Library

[36]

D. Cai, X. He, J. Han, T.S. Huang, Graph regularized nonnegative matrix factorization for data representation, IEEE Trans. Pattern Anal. Mach. Intell., 33 (2011) 1548-1560.

Digital Library

[37]

J. Wu, H. Liu, H. Xiong, J. Cao, J. Chen, K-means-based consensus clustering: a unified view, IEEE Trans. Knowl. Data Eng., 27 (2015) 155-169.

[38]

E. Elhamifar, R. Vidal, Sparse subspace clustering: algorithm, theory, and applications, IEEE Trans. Pattern Anal. Mach. Intell., 35 (2013) 2765-2781.

Digital Library

[39]

G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, Y. Ma, Robust recovery of subspace structures by low-rank representation, IEEE Trans. Pattern Anal. Mach. Intell., 35 (2013) 171-184.

Digital Library

[40]

T. Zhang, A. Szlam, Y. Wang, G. Lerman, Hybrid linear modeling via local best-fit flats, Int. J. Comput. Vis., 100 (2012) 217-240.

Digital Library

[41]

A.M. Martinez, The ar face database, CVC Technical Report 24, 1998.

[42]

S. Boyd, L. Vandenberghe, Convex optimization, Cambridge university press, 2004.

Digital Library

[43]

Y. Yang, H. T. Shen, Z. Ma, Z. Huang, X. Zhou, l2,1-norm regularized discriminative feature selection for unsupervised learning, in: IJCAI Proceedings-International Joint Conference on Artificial Intelligence, pp. 15891594.

Digital Library

[44]

Z. Zhao, H. Liu, Spectral feature selection for supervised and unsupervised learning, in:Proceedings of the 24th international conference on Machine learning, pp. 11511157.

Digital Library

Cited By

Chen L(2024)A Conditionally Positive Define Kernel Low-rank Subspace ClusteringProceedings of the International Conference on Computer Vision and Deep Learning10.1145/3653781.3653802(1-5)Online publication date: 19-Jan-2024
https://dl.acm.org/doi/10.1145/3653781.3653802
Jiang KCao TZhu LSun Q(2024)Adaptive and flexible -norm graph embedding for unsupervised feature selectionApplied Intelligence10.1007/s10489-024-05760-z54:22(11732-11751)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10489-024-05760-z
Li YZhang BYuan ZLiu YLei STan X(2024)Unsupervised attribute reduction based on neighborhood dependencyApplied Intelligence10.1007/s10489-024-05604-w54:21(10653-10670)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10489-024-05604-w
Show More Cited By

Subspace clustering guided unsupervised feature selection
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
    2. Machine learning algorithms

Recommendations

Nonnegative Laplacian embedding guided subspace learning for unsupervised feature selection
Abstract
Unsupervised feature selection plays an important role in machine learning and data mining, which is very challenging because of unavailable class labels. We propose an unsupervised feature selection framework by combining the ...
Joint clustering and feature selection
WAIM'13: Proceedings of the 14th international conference on Web-Age Information Management

Due to the absence of class labels, unsupervised feature selection is much more difficult than supervised feature selection. Traditional unsupervised feature selection algorithms usually select features to preserve the structure of the data set. ...
Unsupervised Feature Selection with Feature Clustering
WI-IAT '12: Proceedings of the The 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology - Volume 01

As an effective technique for dimensionality reduction, feature selection has a broad application in different research areas. In this paper, we present a feature selection method based on a novel feature clustering procedure, which aims at partitioning ...

Comments

Information & Contributors

Information

Published In

cover image Pattern Recognition

Pattern Recognition Volume 66, Issue C

June 2017

422 pages

ISSN:0031-3203

Issue’s Table of Contents

Copyright © Elsevier Ltd.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 June 2017

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen L(2024)A Conditionally Positive Define Kernel Low-rank Subspace ClusteringProceedings of the International Conference on Computer Vision and Deep Learning10.1145/3653781.3653802(1-5)Online publication date: 19-Jan-2024
https://dl.acm.org/doi/10.1145/3653781.3653802
Jiang KCao TZhu LSun Q(2024)Adaptive and flexible -norm graph embedding for unsupervised feature selectionApplied Intelligence10.1007/s10489-024-05760-z54:22(11732-11751)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10489-024-05760-z
Li YZhang BYuan ZLiu YLei STan X(2024)Unsupervised attribute reduction based on neighborhood dependencyApplied Intelligence10.1007/s10489-024-05604-w54:21(10653-10670)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10489-024-05604-w
Kumar MSharma ASaxena SKumar SKrause ABrunskill ECho KEngelhardt BSabato SScarlett J(2023)Featured graph coarsening with similarity guaranteesProceedings of the 40th International Conference on Machine Learning10.5555/3618408.3619148(17953-17975)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.5555/3618408.3619148
Xie XCao ZSun F(2023)Joint learning of graph and latent representation for unsupervised feature selectionApplied Intelligence10.1007/s10489-023-04893-x53:21(25282-25295)Online publication date: 7-Aug-2023
https://dl.acm.org/doi/10.1007/s10489-023-04893-x
Francis JBaburaj MGeorge S(2022)An l½ and Graph Regularized Subspace Clustering Method for Robust Image SegmentationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/347651418:2(1-24)Online publication date: 16-Feb-2022
https://dl.acm.org/doi/10.1145/3476514
Chen YYang XLi JWang PQian Y(2022)Fusing attribute reduction acceleratorsInformation Sciences: an International Journal10.1016/j.ins.2021.12.047587:C(354-370)Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1016/j.ins.2021.12.047
Oliveira MQueiroz SCarvalho F(2022)Unsupervised feature selection method based on iterative similarity graph factorization and clustering by modularityExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.118092208:COnline publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1016/j.eswa.2022.118092
Xu RWang H(2022)Multi-view learning with privileged weighted twin support vector machineExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.117787206:COnline publication date: 15-Nov-2022
https://dl.acm.org/doi/10.1016/j.eswa.2022.117787
Guo LZhang XWang QXue XLiu ZMu Y(2022)Joint enhanced low-rank constraint and kernel rank-order distance metric for low level vision processingExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.116976201:COnline publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1016/j.eswa.2022.116976
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents