research-article

Feature selection under a complexity constraint

Authors:

Jan H. Plasberg,

W. Bastiaan KleijnAuthors Info & Claims

IEEE Transactions on Multimedia, Volume 11, Issue 3

Pages 565 - 571

https://doi.org/10.1109/TMM.2009.2012944

Published: 01 April 2009 Publication History

Abstract

Classification on mobile devices is often done in an uninterrupted fashion. This requires algorithms with gentle demands on the computational complexity. The performance of a classifier depends heavily on the set of features used as input variables. Existing feature selection strategies for classification aim at finding a "best" set of features that performs well in terms of classification accuracy, but are not designed to handle constraints on the computational complexity. We demonstrate that an extension of the performance measures used in state-of-the-art feature selection algorithms with a penalty on the feature extraction complexity leads to superior feature sets if the allowed computational complexity is limited. Our solution is independent of a particular classification algorithm.

References

[1]

A. J. Eronen, V. T. Peltonen, J. T. Tuomi, A. P. Klapuri, S. Fagerlund, T. Sorsa, G. Lorho, and J. Huopaniemis, "Audio-based context recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 1, pp. 321-329, Jan. 2006.

Digital Library

[2]

G. H. John, R. Kohavi, and K. Pfleger, "Irrelevant features and the subset selection problem," in Machine Learning: Proceedings of the Eleventh International Conference, W.W. Cohen and H. Hirsh, Eds. San Francisco, CA: Morgan Kaufmann, 1994, pp. 121-129.

[3]

P. Pudil, J. Novovicova, and J. Kittler, "Floating search methods in feature selection," Pattern Recognit. Lett., vol. 15, pp. 1119-1125, Nov. 1994.

Digital Library

[4]

F. J. Ferri, P. Pudil, M. Hatef, and J. Kittler, "Comparative study of techniques for large-scale feature selection," in Pattern Recognition in Practice IV, Multiple Paradigms, Comparative Studies and Hybrid Systems, E. S. Gelsema and L. S. Kanal, Eds. New York: Elsevier, 1994, pp. 403-413.

[5]

A. Jain and D. Zongker, "Feature selection: Evaluation, application and small sample performance," IEEE Trans. Pattern Anal. Machine Intell., vol. 19, no. 1, pp. 153-158, Jan. 1997.

Digital Library

[6]

H. Liu and L. Yu, "Toward integrating feature selection algorithms for classification and clustering," IEEE Trans. Knowl. Data Eng., vol. 17, no. 4, pp. 491-502, Apr. 2005.

Digital Library

[7]

G. D. Nelson and D. M. Levy, "A dynamic programming approach to the selection of pattern features," IEEE Trans. Syst. Sci. Cybern., vol. SSC-4, no. 2, pp. 145-151, 1968.

[8]

W. Siedlecki and J. Sklansky, "A note on genetic algorithms for largescale feature selection," Pattern Recognit. Lett., vol. 10, pp. 335-347, 1989.

Digital Library

[9]

P. Pudil, J. Novovicova, and S. Blaha, "Multistage pattern recognition with reject option," in Proc. 11th IAPR Int. Conf. Pattern Recognition, Conf. B: Pattern Recognition Methodology and Systems, 1992, vol. 2, pp. 92-95.

[10]

J. Yang and V. Honavar, "Feature subset selection using a genetic algorithm," in Feature Extraction, Construction and Selection, H. Liu and H. Motoda, Eds. Norwell, MA: Kluwer, 1998, pp. 117-136.

[11]

M. Egmont-Petersen, W. R. M. Dassen, and J. H. C. Reiber, "Sequential selection of discrete features for neural networks--A Bayesian approach to building a cascade," Pattern Recognit. Lett., vol. 20, pp. 1439-1448, 1999.

Digital Library

[12]

P. Paclik, R. P. W. Duin, G. M. P. van Kempen, and R. Kohlus, "On feature selection with measurement cost and grouped features," in Lecture Notes in Computer Science, Structural, Syntactic, and Statistical Pattern Recognition. New York: Springer, 2002, vol. 2396/2002, pp. 491-515.

[13]

K. Iswandy and A. Koenig, "Feature selection with acquisition cost for optimizing sensor system design," Adv. Radio Sci., vol. 4, pp. 135-141, 2006.

[14]

S. Ji and L. Carin, "Cost-sensitive feature acquisition and classification," Pattern Recognit., vol. 40, pp. 1474-1485, 2007.

Digital Library

[15]

J. H. Plasberg and W. B. Kleijn, "Complexity-constrained feature selection for classification," in Proc. IEEE Int. Conf. Consumer Electronics, Las Vegas, NV, 2007, pp. 1-2.

[16]

T. M. Cover and J. van Campenhout, "On the possible orderings in the measurement selection problem," IEEE Trans. Syst., Man, Cybern., vol. SMC-7, no. 9, pp. 657-661, Sep. 1997.

[17]

R. Battiti, "Using mutual information for selecting features in supervised neural net learning," IEEE Trans. Neural Netw., vol. 5, no. 4, pp. 537-550, Jul. 1994.

Digital Library

[18]

H. Yang and J. Moody, "Feature selection based on joint mutual information," in Proc. Advances Intelligent Data Analysis (AIDA), Computational Intelligence Methods Applications (CIMA), Int. Computer Science Conv., Rochester, NY, Jun. 1999.

[19]

G. D. Tourassi, E. D. Frederick, M. M. Markey, and J. C. E. Floyd, "Application of the mutual information criterion for feature selection in computer-aided diagnosis," Med. Phys., vol. 28, no. 12, pp. 2394-2402, Dec. 2001.

[20]

N. Kwak and C. H. Choi, "Input feature selection for classification problems," IEEE Trans. Neural Netw., vol. 13, no. 1, pp. 143-159, Jan. 2002.

Digital Library

[21]

F. Fleuret, "Fast binary feature selection with conditional mutual information," J. Mach. Learn. Res., vol. 5, pp. 1531-1555, 2004.

Digital Library

[22]

H. Peng, F. Long, and C. Ding, "Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 8, pp. 1226-1238, Aug. 2005.

Digital Library

[23]

T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.

Digital Library

[24]

E. Parzen, "On estimation of a probability density function and mode," Ann. Math. Stat., vol. 33, no. 2, pp. 1065-1076, 1962.

[25]

A. Fraser, "Information and entropy in strange attractors," IEEE Trans. Inf. Theory, vol. 35, no. 2, pp. 245-262, Mar. 1989.

[26]

M. Nilsson and W. B. Kleijn, "On the estimation of differential entropy from data located on embedded manifolds," IEEE Trans. Inf. Theory, vol. 53, no. 7, pp. 2330-2341, Jul. 2007.

Digital Library

[27]

P. Hedelin and J. Skoglund, "Vector quantization based on Gaussian mixture models," IEEE Trans. Speech Audio Process., vol. 8, no. 4, pp. 385-401, Jul. 2000.

[28]

M. Nilsson, H. Gustafsson, S. V. Andersen, and W. B. Kleijn, "Gaussian mixture model based mutual information estimation between frequency bands in speech," in Proc. IEEE Int. Conf. Acoustics Speech Signal Processing (ICASSP), 2002, pp. 525-528.

[29]

W. B. Kleijn, A Basis for Source Coding. Stockholm, Sweden: Royal Inst. Technol. (KTH), 2007, Lecture Notes.

[30]

JTC-1, ISO/IEC 15938-4 Information Technology--Multimedia Content Description Interface--Part 4: Audio, ISO/IEC, 2001, Tech. Rep.

[31]

V. Grancharov, D. Y. Zhao, J. Lindblom, and W. B. Kleijn, "Low complexity, nonintrusive speech quality assessment," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 6, pp. 1948-1956, Nov. 2006.

Digital Library

[32]

K. Murphy, Bayes Net Toolbox for MATLAB. {Online}. Available: http://www.bnt.sourceforge.net.

[33]

A. Dempster, N. Laird, and D. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. R. Statist. Soc. Ser. B (Meth.), vol. 39, no. 1, pp. 1-38, 1977.

[34]

V. Peltonen, J. Tuomi, A. Klapuri, J. Huopaniemi, and T. Sorsa, "Computational auditory scene recognition," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, Orlando, FL, May 2002, vol. 1, pp. 1941-1944.

Cited By

Wang YLiu SLi LZhou WLi H(2024)SwinShadow: Shifted Window for Ambiguous Adjacent Shadow DetectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3688803Online publication date: 27-Aug-2024
https://doi.org/10.1145/3688803
Bach JZoller KTrittenbach HSchulz KBöhm K(2022)An Empirical Evaluation of Constrained Feature SelectionSN Computer Science10.1007/s42979-022-01338-z3:6Online publication date: 6-Oct-2022
https://dl.acm.org/doi/10.1007/s42979-022-01338-z
Yang YPouyanfar STian HChen MChen SShyu M(2018)IF-MCAIEEE Transactions on Multimedia10.1109/TMM.2017.276062320:4(1024-1032)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1109/TMM.2017.2760623

Feature selection under a complexity constraint
1. Computing methodologies
  1. Machine learning

Recommendations

A Cosine-Similarity Mutual-Information Approach for Feature Selection on High Dimensional Datasets

A novel hybrid method based on Cosine Similarity and Mutual Information is presented to find out relevant feature subset. Initially, the supervised Cosine Similarity of each feature is calculated with respect to the class vector and then features are ...
Wide-ranging approach-based feature selection for classification
Abstract
Feature selection methods have been issued in the context of data classification due to redundant and irrelevant features. The above features slow the overall system performance, and wrong decisions are more likely to be made with extensive data ...
Feature selection considering weighted relevancy

Feature selection plays an important role in pattern recognition and machine learning. Feature selection based on information theory intends to preserve the feature relevancy between features and class labels while eliminating irrelevant and redundant ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Multimedia

IEEE Transactions on Multimedia Volume 11, Issue 3

Special section on communities and media computing

April 2009

236 pages

ISSN:1520-9210

Issue’s Table of Contents

Copyright © 2009.

Publisher

IEEE Press

Publication History

Published: 01 April 2009

Revised: 05 November 2008

Received: 28 December 2007

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang YLiu SLi LZhou WLi H(2024)SwinShadow: Shifted Window for Ambiguous Adjacent Shadow DetectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3688803Online publication date: 27-Aug-2024
https://doi.org/10.1145/3688803
Bach JZoller KTrittenbach HSchulz KBöhm K(2022)An Empirical Evaluation of Constrained Feature SelectionSN Computer Science10.1007/s42979-022-01338-z3:6Online publication date: 6-Oct-2022
https://dl.acm.org/doi/10.1007/s42979-022-01338-z
Yang YPouyanfar STian HChen MChen SShyu M(2018)IF-MCAIEEE Transactions on Multimedia10.1109/TMM.2017.276062320:4(1024-1032)Online publication date: 1-Apr-2018
https://dl.acm.org/doi/10.1109/TMM.2017.2760623

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents