research-article

Unsupervised feature selection via maximum projection and minimum redundancy

Knowledge-Based Systems, Volume 75, Issue C

Pages 19 - 29

https://doi.org/10.1016/j.knosys.2014.11.008

Published: 01 February 2015 Publication History

Abstract

Dimensionality reduction is an important and challenging task in machine learning and data mining. It can facilitate data clustering, classification and information retrieval. As an efficient technique for dimensionality reduction, feature selection is about finding a small feature subset preserving the most relevant information. In this paper, we propose a new criterion, called maximum projection and minimum redundancy feature selection, to address unsupervised learning scenarios. First, the feature selection is formalized with the use of the projection matrices and then characterized equivalently as a matrix factorization problem. Second, an iterative update algorithm and a greedy algorithm are proposed to tackle this problem. Third, kernel techniques are considered and the corresponding algorithm is also put forward. Finally, the proposed algorithms are compared with four state-of-the-art feature selection methods. Experimental results reported for six publicly datasets demonstrate the superiority of the proposed algorithms.

References

[1]

I. Guyon, A. Elisseeff (Eds.), Feature Extraction: Foundations and Applications, vol. 207, Springer, 2006.

Digital Library

[2]

I. Guyon, A. Elisseeff, An introduction to variable and feature selection, J. Mach. Learn. Res. 3 (2003) 1157-1182.

Digital Library

[3]

Y. Yang, J.O. Pedersen, A comparative study on feature selection in text categorization, in: International Conference on Machine Learning, 1997, pp. 412-420.

[4]

R.O. Duda, P.E. Hart, D.G. Stork (Eds.), Pattern Classification, Wiley-Interscience, 2000.

[5]

Q. Gu, Z. Li, J. Han, Generalized fisher score for feature selection, in: Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence, Barcelona, Spain, 2011, pp. 1-8.

[6]

H. Peng, F. Long, C. Ding, Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy, IEEE Trans. Pattern Anal. Mach. Intell. 27 (8) (2005) 1226-1238.

Digital Library

[7]

J.-B. Yang, C.-J. Ong, An effective feature selection method via mutual information estimation, IEEE Trans. Syst. Sci. Cybernet. - Part B: Cybernet. 42 (6) (2012) 1550-1559.

Digital Library

[8]

J.L. Rodgers, W.A. Nicewander, Thirteen ways to look at the correlation coefficient, Am. Statist. 42 (1) (1988) 59-66.

[9]

J.G. Dy, C.E. Brodley, Feature selection for unsupervised learning, J. Mach. Learn. Res. 5 (12) (2004) 845-889.

Digital Library

[10]

H. Liu, H. Motoda (Eds.), Feature Selection for Knowledge Discovery and Data Mining, Springer, 1998.

[11]

Z. Li, Y. Yang, J. Liu, X. Zhou, H. Lu, Unsupervised feature selection using nonnegative spectral analysis, in: Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, 2012, pp. 1026-1032.

Digital Library

[12]

F. Nie, H. Huang, X. Cai, C. Ding, Efficient and robust feature selection via joint l_2,1-norms minimization, in: Advances in Neural Information Processing Systems, 2010, pp. 1813-1821.

Digital Library

[13]

M. Qian, C. Zhai, Robust unsupervised feature selection, in: Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, 2013, pp. 1621-1627.

Digital Library

[14]

Y. Sun, S. Todorovic, S. Goodison, Local-learning-based feature selection for high-dimensional data analysis, IEEE Trans. Pattern Anal. Mach. Intell. 32 (9) (2010) 1610-1626.

Digital Library

[15]

H. Liu, L. Yu, Toward integrating feature selection algorithms for classification and clustering, IEEE Trans. Knowl. Data Eng. 17 (4) (2005) 491-502.

Digital Library

[16]

D. Mladenic, M. Grobelnik, Feature selection for unbalanced class distribution and Naive Bayes, in: International Conference on Machine Learning, 1999, pp. 258-267.

[17]

J. Weston, S. Mukherjee, O. Chapelle, M. Pontil, T. Poggio, V. Vapnik, Feature selection for SVMS, in: Advances in Neural Information Processing Systems, vol. 12, 2000, pp. 668-674.

[18]

M.H. Law, M.A. Figueiredo, A.K. Jain, Simultaneous feature selection and clustering using mixture models, IEEE Trans. Pattern Anal. Mach. Intell. 26 (9) (2004) 1154-1166.

Digital Library

[19]

R. Kohavi, G.H. John, Wrappers for feature subset selection, Artif. Intell. 97 (1- 2) (1997) 273-324.

Digital Library

[20]

L. Yu, H. Liu, Feature selection for high-dimensional data: a fast correlation-based filter solution, in: Proceedings of the Twentieth International Conference on Machine Learning, 2003, pp. 856-863.

[21]

A.K. Farahat, A. Ghodsi, M.S. Kamel, Efficient greedy feature selection for unsupervised learning, Knowl. Inform. Syst. 35 (2013) 285-310.

[22]

H. Yang, M.R. Lyu, I. King, Efficient online learning for multitask feature selection, ACM Trans. Knowl. Discov. Data 7 (2) (2013) 1-27.

Digital Library

[23]

L. Song, A. Smola, A. Gretton, J. Bedo, K. Borgwardt, Feature selection via dependence maximization, J. Mach. Learn. Res. 13 (2012) 1393-1434.

Digital Library

[24]

B. Liu, B. Fang, X. Liu, J. Chen, Z. Huang, X. He, Large margin subspace learning for feature selection, Pattern Recog. 46 (10) (2013) 2798-2806.

Digital Library

[25]

Z. Zhao, L. Wang, H. Liu, J. Ye, On similarity preserving feature selection, IEEE Trans. Knowl. Data Eng. 25 (3) (2013) 619-632.

Digital Library

[26]

P. Mitra, C. Murthy, S.K. Pal, Unsupervised feature selection using feature similarity, IEEE Trans. Pattern Anal. Mach. Intell. 24 (4) (2002) 1-13.

[27]

X. He, D. Cai, P. Niyogi, Laplacian score for feature selection, in: Advances in Neural Information Processing Systems, 2005, pp. 507-514.

Digital Library

[28]

D. Cai, C. Zhang, X. He, Unsupervised feature selection for multi-cluster data, in: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2010, pp. 333-342.

Digital Library

[29]

Y. Yang, H. Shen, Z. Ma, Z. Huang, X. Zhou, l_2,1 norm regularized discriminative feature selection for unsupervised learning, in: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, 2011, pp. 1589-1594.

Cited By

Ceylan CGhoorchian KKragic DSerra ESpezzano F(2024)Scalable Unsupervised Feature Selection with Reconstruction Error Guarantees via QMR DecompositionProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679994(3658-3662)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679994
Solorio-Fernández SCarrasco-Ochoa JMartínez-Trinidad J(2024)Filter unsupervised spectral feature selection method for mixed data based on a new feature correlation measureNeurocomputing10.1016/j.neucom.2023.127111571:COnline publication date: 28-Feb-2024
https://dl.acm.org/doi/10.1016/j.neucom.2023.127111
Moslemi AAhmadian A(2024)Subspace learning for feature selection via rank revealing QR factorizationExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.124919256:COnline publication date: 5-Dec-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.124919
Show More Cited By

Unsupervised feature selection via maximum projection and minimum redundancy
1. Computing methodologies
  1. Machine learning
    1. Machine learning algorithms

Recommendations

Subspace learning for unsupervised feature selection via matrix factorization

Dimensionality reduction is an important and challenging task in machine learning and data mining. Feature selection and feature extraction are two commonly used techniques for decreasing dimensionality of the data and increasing efficiency of learning ...
Minimum-maximum local structure information for feature selection

Feature selection methods have been extensively applied in machine learning tasks, such as computer vision, pattern recognition, and data mining. These methods aim to identify a subset of the original features with high discriminating power. Among them, ...
Semi-supervised minimum redundancy maximum relevance feature selection for audio classification

It is still a changing problem of choosing the most relevant ones from multiple features for their specific machine learning tasks. However, feature selection provides an effective solution to it, which aims to choose the most relevant and least ...

Comments

Information & Contributors

Information

Published In

cover image Knowledge-Based Systems

Knowledge-Based Systems Volume 75, Issue C

February 2015

239 pages

ISSN:0950-7051

Issue’s Table of Contents

Copyright © Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 February 2015

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ceylan CGhoorchian KKragic DSerra ESpezzano F(2024)Scalable Unsupervised Feature Selection with Reconstruction Error Guarantees via QMR DecompositionProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679994(3658-3662)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679994
Solorio-Fernández SCarrasco-Ochoa JMartínez-Trinidad J(2024)Filter unsupervised spectral feature selection method for mixed data based on a new feature correlation measureNeurocomputing10.1016/j.neucom.2023.127111571:COnline publication date: 28-Feb-2024
https://dl.acm.org/doi/10.1016/j.neucom.2023.127111
Moslemi AAhmadian A(2024)Subspace learning for feature selection via rank revealing QR factorizationExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.124919256:COnline publication date: 5-Dec-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.124919
Wang YHuang MZhou LChe HJiang B(2024)Multi-cluster nonlinear unsupervised feature selection via joint manifold learning and generalized LassoExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.124502255:PAOnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.124502
Moslemi AAhmadian A(2024)Dual regularized subspace learning using adaptive graph learning and rank constraintComputers in Biology and Medicine10.1016/j.compbiomed.2023.107659167:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.compbiomed.2023.107659
Dwivedi RTiwari ABharill NRatnaparkhe MTiwari A(2024)A taxonomy of unsupervised feature selection methods including their pros, cons, and challengesThe Journal of Supercomputing10.1007/s11227-024-06368-380:16(24212-24240)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s11227-024-06368-3
Niu FZhao XGuo JShi MLiu XLiu B(2023)Fast and Robust Unsupervised Dimensionality Reduction with Adaptive Bipartite GraphsKnowledge-Based Systems10.1016/j.knosys.2023.110680276:COnline publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1016/j.knosys.2023.110680
Zeng CChen HLi TWan J(2022)Robust unsupervised feature selection via sparse and minimum-redundant subspace learning with dual regularizationNeurocomputing10.1016/j.neucom.2022.09.074511:C(1-21)Online publication date: 28-Oct-2022
https://dl.acm.org/doi/10.1016/j.neucom.2022.09.074
Saberi-Movahed FRostami MBerahmand KKarami STiwari POussalah MBand S(2022)Dual Regularized Unsupervised Feature Selection Based on Matrix Factorization and Minimum Redundancy with application in gene selectionKnowledge-Based Systems10.1016/j.knosys.2022.109884256:COnline publication date: 28-Nov-2022
https://dl.acm.org/doi/10.1016/j.knosys.2022.109884
Beiranvand FMehrdad VDowlatshahi M(2022)Unsupervised feature selection for image classificationKnowledge-Based Systems10.1016/j.knosys.2022.109085250:COnline publication date: 17-Aug-2022
https://dl.acm.org/doi/10.1016/j.knosys.2022.109085
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents