research-article

Free access

Single feature ranking and binary particle swarm optimisation based feature subset ranking for feature selection

Authors:

Will N. BrowneAuthors Info & Claims

ACSC '12: Proceedings of the Thirty-fifth Australasian Computer Science Conference - Volume 122

Pages 27 - 36

Published: 30 January 2012 Publication History

Abstract

This paper proposes two wrapper based feature selection approaches, which are single feature ranking and binary particle swarm optimisation (BPSO) based feature subset ranking. In the first approach, individual features are ranked according to the classification accuracy so that feature selection can be accomplished by using only a few top-ranked features for classification. In the second approach, BPSO is applied to feature subset ranking to search different feature subsets. K-nearest neighbour (KNN) with n-fold cross-validation is employed to evaluate the classification accuracy on eight datasets in the experiments. Experimental results show that using a relatively small number of the top-ranked features obtained from the first approach or one of the top-ranked feature subsets obtained from the second approach can achieve better classification performance than using all features. BPSO could efficiently search for subsets of complementary features to avoid redundancy and noise. Compared with linear forward selection (LFS) and greedy stepwise backward selection (GSBS), in almost all cases, the two proposed approaches could achieve better performance in terms of classification accuracy and the number of features. The BPSO based approach outperforms single feature ranking approach for all the datasets.

References

[1]

Abeel, T., de Peer, Y. V. & Saeys, Y. (2009), 'Java-ML: A Machine Learning Library', Journal of Machine Learning Research 10, 931--934.

Digital Library

[2]

Alba, E., Garcia-Nieto, J., Jourdan, L. & Talbi, E. G. (2007), Gene selection in cancer classification using PSO/SVM and GA/SVM hybrid algorithms, in 'IEEE Congress on Evolutionary Computation', pp. 284--290.

[3]

Azevedo, G. L. F., Cavalcanti, G. D. C. & Filho, E. C. B. (2007), An approach to feature selection for keystroke dynamics systems based on PSO and feature weighting, in 'IEEE Congress on Evolutionary Computation', pp. 3577--3584.

[4]

Caruana, R. & Freitag, D. (1994), Greedy Attribute Selection, in 'In Proceedings of International Conference on Machine Learning', pp. 28--36.

[5]

Chakraborty, B. (2008), Feature subset selection by particle swarm optimization with fuzzy fitness function, in '3rd International Conference on Intelligent System and Knowledge Engineering', Vol. 1, pp. 1038--1042.

[6]

Chuang, L. Y., Chang, H. W., Tu, C. J.& Yang, C. H. (2008), 'Improved binary PSO for feature selection using gene expression data', Computational Biology and Chemistry 32(29), 29--38.

Digital Library

[7]

Clerc, M. & Kennedy, J. (2002), The particle swarm - explosion, stability, and convergence in a multidimensional complex space, in 'IEEE Congress on Evolutionary Computation', Vol. 6, pp. 58--73.

Digital Library

[8]

Dash, M. & Liu, H. (1997), 'Feature selection for classification', Intelligent Data Analysis 1, 131--156.

[9]

Frank, A. & Asuncion, A. (2010), UCI Machine Learning Repository, {http://archive.ics.uci.edu/ml}. Irvine, CA: University of California, School of Information and Computer Science.

[10]

Gutlein, M., Frank, E., Hall, M. & Karwath, A. (2009), Large-scale attribute selection using wrappers, in 'IEEE Symposium on Computational Intelligence and Data Mining', pp. 332--339.

[11]

Guyon, I., Elisseeff, A. & Liu, H. (2003), 'An introduction to variable and feature selection', The Journal of Machine Learning Research 3, 1157--1182.

Digital Library

[12]

Huang, C. J. & Dun, J. F. (2008), 'A distributed PSO-SVM hybrid system with feature selection and parameter optimization', Applied Soft Computing 8(4), 1381--1391.

Digital Library

[13]

Kennedy, J. & Eberhart, R. (1995), Particle swarm optimization, in 'IEEE International Conference on Neural Networks', Vol. 4, pp. 1942--1948.

[14]

Kennedy, J. & Eberhart, R. (1997), A discrete binary version of the particle swarm algorithm, in 'IEEE International Conference on Systems, Man, and Cybernetics', Vol. 5, pp. 4104--4108.

[15]

Kennedy, J. & Spears, W. M. (1998), Matching algorithms to problems: an experimental test of the particle swarm and some genetic algorithms on the multimodal problem generator, in 'IEEE Congress on Evolutionary Computation', pp. 78--83.

[16]

Kohavi, R. & John, G. H. (1997), 'Wrappers for feature subset selection', Artificial Intelligence 97, 315--333.

Digital Library

[17]

Langley, P. (1994), Selection of relevant features in machine learning, in 'Proceedings of the AAAI Fall symposium on relevance', pp. 127--131.

[18]

Liu, Y. N., Wang, G., Chen, H. L., & Dong, H. (2011), 'An Improved Particle Swarm Optimization for Feature Selection', Journal of Bionic Engineering 8(2), 191--200.

[19]

Marill, T., & Green, D. M. (1963), 'On the effectiveness of receptors in recognition systems', IEEE Transactions on Information Theory 9(1), 11--17.

Digital Library

[20]

Neshatian, K. & Zhang, M. J. (2009), Genetic Programming for Feature Subset Ranking in Binary Classification Problems, in 'European Conference on Genetic Programming', pp. 121--132.

Digital Library

[21]

Talbi, E. G., Jourdan, L., Garcia-Nieto, J. & Alba, E. (2008), Comparison of population based meta-heuristics for feature selection: Application to microarray data classification, in 'ACS/IEEE International Conference on Computer Systems and Applications', pp. 45--52.

Digital Library

[22]

Unler, A. & Murat, A. (2010), 'A discrete particle swarm optimization method for feature selection in binary classification problems', European Journal of Operational Research 206, 528--539.

[23]

Van Den Bergh, F. (2002), An analysis of particle swarm optimizers, Ph.D., University of Pretoria, South Africa.

Digital Library

[24]

Wang, X. Y., Yang, J., Teng, X. L. & Xia, W. J. (2007), 'Feature selection based on rough sets and particle swarm optimization', Pattern Recognition Letters 28(4), 459--471.

Digital Library

[25]

Whitney, A. W. (2007), 'A direct method of nonparametric measurement selection', IEEE Transactions on Computers 20(4), 1100--1103.

Digital Library

[26]

Witten I. H. & Frank E. (2005), Data Mining: Practical Machine Learning Tools and Techniques., 2nd Edition, Morgan Kaufmann, San Francisco.

Digital Library

[27]

Yang, C. S., Chuang, L. Y. & Ke, C. H. (2008), Boolean binary particle swarm optimization for feature selection, in 'IEEE Congress on Evolutionary Computation', pp. 2093--2098.

[28]

Yang, C. S., Chuang, L. Y. & Li, J. C. (2008), Chaotic maps in binary particle swarm optimization for feature selection, in 'IEEE Conference on Soft Computing in Industrial Applications', pp. 107--112.

Cited By

Nguyen HXue BZhang M(2016)A Subset Similarity Guided Method for Multi-objective Feature SelectionProceedings of the Second Australasian Conference on Artificial Life and Computational Intelligence - Volume 959210.1007/978-3-319-28270-1_25(298-310)Online publication date: 2-Feb-2016
https://dl.acm.org/doi/10.1007/978-3-319-28270-1_25
Hancer EXue BKaraboga DZhang M(2015)A binary ABC algorithm based on advanced similarity scheme for feature selectionApplied Soft Computing10.1016/j.asoc.2015.07.02336:C(334-348)Online publication date: 1-Nov-2015
https://dl.acm.org/doi/10.1016/j.asoc.2015.07.023
Selvaraj GS. J(2013)Improved Feature Selection Based on Particle Swarm Optimization for Liver Disease DiagnosisProceedings of the 4th International Conference on Swarm, Evolutionary, and Memetic Computing - Volume 829810.1007/978-3-319-03756-1_19(214-225)Online publication date: 19-Dec-2013
https://dl.acm.org/doi/10.1007/978-3-319-03756-1_19

Index Terms

Single feature ranking and binary particle swarm optimisation based feature subset ranking for feature selection

Recommendations

Rank based binary particle swarm optimisation for feature selection in classification
ICFNDS '18: Proceedings of the 2nd International Conference on Future Networks and Distributed Systems

Feature selection (FS) is an important and challenging task in machine learning. FS can be defined as the process of finding the best informative subset of features in order to avoid the curse of dimensionality and maximise the classification accuracy. ...
Particle Swarm Optimisation and Statistical Clustering for Feature Selection
Proceedings of the 26th Australasian Joint Conference on AI 2013: Advances in Artificial Intelligence - Volume 8272

Feature selection is an important issue in classification, but it is a difficult task due to the large search space and feature interaction. Statistical clustering methods, which consider feature interaction, group features into different feature ...
Particle swarm optimisation for feature selection in classification

In classification, feature selection is an important data pre-processing technique, but it is a difficult problem due mainly to the large search space. Particle swarm optimisation (PSO) is an efficient evolutionary computation technique. However, the ...

Comments

Information & Contributors

Information

Published In

cover image DL Hosted proceedings

ACSC '12: Proceedings of the Thirty-fifth Australasian Computer Science Conference - Volume 122

January 2012

140 pages

ISBN:9781921770036

Editors:
Mark Reynolds
The University of Western Australia, Crawley, WA, Australia
,
Bruce Thomas
University of South Australia, Adelaide, SA, Australia

Publisher

Australian Computer Society, Inc.

Australia

Publication History

Published: 30 January 2012

Author Tags

Qualifiers

Research-article

Acceptance Rates

ACSC '12 Paper Acceptance Rate 14 of 38 submissions, 37%;

Overall Acceptance Rate 136 of 379 submissions, 36%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
185
Total Downloads

Downloads (Last 12 months)53
Downloads (Last 6 weeks)17

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nguyen HXue BZhang M(2016)A Subset Similarity Guided Method for Multi-objective Feature SelectionProceedings of the Second Australasian Conference on Artificial Life and Computational Intelligence - Volume 959210.1007/978-3-319-28270-1_25(298-310)Online publication date: 2-Feb-2016
https://dl.acm.org/doi/10.1007/978-3-319-28270-1_25
Hancer EXue BKaraboga DZhang M(2015)A binary ABC algorithm based on advanced similarity scheme for feature selectionApplied Soft Computing10.1016/j.asoc.2015.07.02336:C(334-348)Online publication date: 1-Nov-2015
https://dl.acm.org/doi/10.1016/j.asoc.2015.07.023
Selvaraj GS. J(2013)Improved Feature Selection Based on Particle Swarm Optimization for Liver Disease DiagnosisProceedings of the 4th International Conference on Swarm, Evolutionary, and Memetic Computing - Volume 829810.1007/978-3-319-03756-1_19(214-225)Online publication date: 19-Dec-2013
https://dl.acm.org/doi/10.1007/978-3-319-03756-1_19

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents