Article

l_2,1-norm regularized discriminative feature selection for unsupervised learning

Authors:

Xiaofang ZhouAuthors Info & Claims

IJCAI'11: Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Two

July 2011

Pages 1589 - 1594

Published: 16 July 2011 Publication History

Abstract

Compared with supervised learning for feature selection, it is much more difficult to select the discriminative features in unsupervised learning due to the lack of label information. Traditional unsupervised feature selection algorithms usually select the features which best preserve the data distribution, e.g., manifold structure, of the whole feature set. Under the assumption that the class label of input data can be predicted by a linear classifier, we incorporate discriminative analysis and l_2,1-norm minimization into a joint framework for unsupervised feature selection. Different from existing unsupervised feature selection algorithms, our algorithm selects the most discriminative feature subset from the whole feature set in batch mode. Extensive experiment on different data types demonstrates the effectiveness of our algorithm.

References

[1]

Andreas Argyriou, Theodoros Evgeniou, Massimiliano Pontil, Andreas Argyriou, Theodoros Evgeniou, and Massimiliano Pontil. Convex multitask feature learning. In Machine Learning, 2008.

[2]

Deng Cai, Chiyuan Zhang, and Xiaofei He. Unsupervised feature selection for multi-cluster data. KDD, 2010.

[3]

R.O. Duda, P.E. Hart, and D.G. Stork. Pattern Classification (2nd Edition). John Wiley & Sons, New York, USA, 2001.

[4]

K. Fukunaga. Introduction to statistical pattern recognition (2nd Edition). Academic Press Professional, Inc, San Diego, USA, 1990.

[5]

A. Georghiades, P. Belhumeur, and D. Kriegman. From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE TPAMI, page 23(6):643660, 2001.

[6]

Xiaofei He, Deng Cai, and Partha Niyogi. Laplacian score for feature selection. NIPS, 2005.

[7]

J.J. Hull. A database for handwritten text recognition research. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 16(5):550-554, 1994.

[8]

Jun Liu, Shuiwang Ji, and Jieping Ye. Multi-task feature learning via efficient l _2,1-norm minimization. In UAI, 2009.

[9]

Feiping Nie, Shiming Xiang, Yangqing Jia, Changshui Zhang, and Shuicheng Yan. Trace ratio criterion for feature selection. In AAAI, 2008.

[10]

Feiping Nie, Heng Huang, Xiao Cai, and Chris Ding. Efficient and robust feature selection via joint l _2,1-norms minimization. In NIPS, 2010.

[11]

G. Obozinski, M. J. Wainwright, andM. I. Jordan. Highdimensional union support recovery in multivariate regression. In NIPS, 2008.

[12]

S. Sarkar, P.J. Phillips, Z. Liu, I.R. Vega, P. Grother, and K.W. Bowyer. The humanid gait challenge problem: data sets, performance, and analysis. IEEE TPAMI, pages 27(2):162-177, 2005.

[13]

A. Strehl and J. Ghosh. Cluster ensembles-a knowledge reuse framework for combining multiple partitions. Journal of Machine Learning Research, 3:583-617, 2002.

[14]

Masashi Sugiyama. Local fisher discriminant analysis for supervised dimensionality reduction. In ICML, 2006.

[15]

Yi Yang, Feiping Nie, Shiming Xiang, Yueting Zhuang, and Wenhua Wang. Local and global regressive mapping for manifold learning with out-of-sample extrapolation. In AAAI, 2010.

[16]

Yi Yang, Dong Xu, Feiping Nie, Shuicheng Yan, and Yueting Zhuang. Image clustering using local discriminant models and global integration. IEEE TIP, pages 19(10)2761-2773, 2010.

[17]

Yang Yang, Yi Yang, Zi Huang, Heng Tao Shen, and Feiping Nie. Tag localization with spatial correlations and joint group sparsity. In CVPR, pages 881-888, 2011.

[18]

Zheng Zhao and Huan Liu. Spectral feature selection for supervised and unsupervised learning. In ICML, 2007.

[19]

Z. Zhao, L. Wang, and H. Liu. Efficient spectral feature selection with minimum redundancy. In AAAI, 2010.

Cited By

Xing XLiu HChen CLi JDemartini GZuccon GCulpepper JHuang ZTong H(2021)Fairness-Aware Unsupervised Feature SelectionProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482106(3548-3552)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482106
Bai ZNguyen HDavidson IGupta RLiu YShah MRajan STang JPrakash B(2020)Block Model Guided Unsupervised Feature SelectionProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3394486.3403173(1201-1211)Online publication date: 23-Aug-2020
https://dl.acm.org/doi/10.1145/3394486.3403173
Ma JZhang YZhang LDu BTao D(2019)Pseudo supervised matrix factorization in discriminative subspaceProceedings of the 28th International Joint Conference on Artificial Intelligence10.5555/3367471.3367676(4554-4560)Online publication date: 10-Aug-2019
https://dl.acm.org/doi/10.5555/3367471.3367676
Show More Cited By

Index Terms

l_2,1-norm regularized discriminative feature selection for unsupervised learning
1. Computing methodologies
  1. Machine learning
    1. Machine learning algorithms
      1. Feature selection

Index terms have been assigned to the content through auto-classification.

Recommendations

Spectral feature selection for supervised and unsupervised learning
ICML '07: Proceedings of the 24th international conference on Machine learning

Feature selection aims to reduce dimensionality for building comprehensible learning models with good generalization performance. Feature selection algorithms are largely studied separately according to the type of learning: supervised or unsupervised. ...
Read More
Manifold regularized discriminative feature selection for multi-label learning
Highlights
- Label correlations are incorporated into the framework via manifold regularization.
Abstract
In multi-label learning, objects are essentially related to multiple semantic meanings, and the type of data is confronted with the impact of high feature dimensionality simultaneously, such as the bioinformatics and text mining ...
Read More
Discriminative semi-supervised feature selection via manifold regularization

Feature selection has attracted a huge amount of interest in both research and application communities of data mining. We consider the problem of semi-supervised feature selection, where we are given a small amount of labeled examples and a large amount ...
Read More

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

IJCAI'11: Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Two

July 2011

1756 pages

ISBN:9781577355144

Editor:
Toby Walsh
NICTA and University of NSW

Sponsors

The International Joint Conferences on Artificial Intelligence, Inc. (IJCAI)

Publisher

AAAI Press

Publication History

Published: 16 July 2011

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

89
Total Citations
View Citations
191
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Xing XLiu HChen CLi JDemartini GZuccon GCulpepper JHuang ZTong H(2021)Fairness-Aware Unsupervised Feature SelectionProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482106(3548-3552)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3482106
Bai ZNguyen HDavidson IGupta RLiu YShah MRajan STang JPrakash B(2020)Block Model Guided Unsupervised Feature SelectionProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3394486.3403173(1201-1211)Online publication date: 23-Aug-2020
https://dl.acm.org/doi/10.1145/3394486.3403173
Ma JZhang YZhang LDu BTao D(2019)Pseudo supervised matrix factorization in discriminative subspaceProceedings of the 28th International Joint Conference on Artificial Intelligence10.5555/3367471.3367676(4554-4560)Online publication date: 10-Aug-2019
https://dl.acm.org/doi/10.5555/3367471.3367676
Liu CWang YHan KXu CXu C(2019)Learning instance-wise sparsity for accelerating deep modelsProceedings of the 28th International Joint Conference on Artificial Intelligence10.5555/3367243.3367456(3001-3007)Online publication date: 10-Aug-2019
https://dl.acm.org/doi/10.5555/3367243.3367456
Zohrizadeh FKheirandishfard MKamangar FMadani R(2019)Non-smooth optimization over stiefel manifolds with applications to dimensionality reduction and graph clusteringProceedings of the 28th International Joint Conference on Artificial Intelligence10.5555/3367032.3367219(1319-1326)Online publication date: 10-Aug-2019
https://dl.acm.org/doi/10.5555/3367032.3367219
Valem LPedronette DEl Saddik ADel Bimbo AZhang ZHauptmann ACandan KBertini MXie LWei X(2019)An Unsupervised Genetic Algorithm Framework for Rank Selection and Fusion on Image RetrievalProceedings of the 2019 on International Conference on Multimedia Retrieval10.1145/3323873.3325022(58-62)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3323873.3325022
Li JGuo RLiu CLiu HTeredesai AKumar VLi YRosales RTerzi EKarypis G(2019)Adaptive Unsupervised Feature Selection on Attributed NetworksProceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3292500.3330856(92-100)Online publication date: 25-Jul-2019
https://dl.acm.org/doi/10.1145/3292500.3330856
Zhou NXu YCheng HYuan ZChen B(2019)Maximum Correntropy Criterion-Based Sparse Subspace Learning for Unsupervised Feature SelectionIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2017.278336429:2(404-417)Online publication date: 1-Feb-2019
https://dl.acm.org/doi/10.1109/TCSVT.2017.2783364
Zheng XGuo YGuo JKong X(2019)$$\hbox {U}^2\hbox {F}^2\hbox {S}^2$$U2F2S2Neural Processing Letters10.1007/s11063-018-9886-549:3(1071-1091)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s11063-018-9886-5
Hoseini EMansoori E(2019)Unsupervised feature selection in linked biological dataPattern Analysis & Applications10.1007/s10044-018-0707-222:3(999-1013)Online publication date: 1-Aug-2019
https://dl.acm.org/doi/10.1007/s10044-018-0707-2
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents