Abstract
All the traditional feature selection methods assume that the entire input feature set is available from the beginning. However, online streaming features (OSF) are integral part of many real-world applications. In OSF, the number of training examples is fixed while the number of features grows with time as new features stream in. A critical challenge for online streaming feature selection (OSFS) is the unavailability of the entire feature set before learning starts. OS-NRRSAR-SA is a successful OSFS algorithm that controls the unknown feature space in OSF by means of the rough sets-based significance analysis. This paper presents an extension to the OS-NRRSAR-SA algorithm. In the proposed extension, the redundant features are filtered out before significance analysis. In this regard, a redundancy analysis method based on functional dependency concept is proposed. The result is a general OSFS framework containing two major steps, (1) online redundancy analysis that discards redundant features, and (2) online significance analysis, which eliminates non-significant features. The proposed algorithm is compared with OS-NRRSAR-SA algorithm, in terms of compactness, running time and classification accuracy during the features streaming. The experiments demonstrate that the proposed algorithm achieves better results than OS-NRRSAR-SA algorithm, in every way.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Beaubouef T, Petry F (2013) Incorporating rough data in database design for imprecise information representation. In: Skowron A, Suraj Z (eds) Rough sets and intelligent systems—Professor Zdzisław Pawlak in Memoriam, intelligent systems reference library, vol 43. Springer, Berlin, pp 137–155
Blake C, Merz CJ (1998) UCI Repository of machine learning databases. http://www.ics.uci.edu/mlearn/MLRepository.html. Accessed 06 March 2015
Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):27:1–27:27
Chao S, Cai J, Yang S, Wang S (2016) A clustering based feature selection method using feature information distance for text data. Springer, Cham, pp 122–132
Clopinet: feature selection challenge, NIPS 2003. http://clopinet.com/isabelle/Projects/NIPS2003/ (2003). Accessed 6 March 2015
Clopinet: performance prediction challenge, WCCI 2006. http://clopinet.com/isabelle/Projects/modelselect/ (2006). Accessed 6 March 2015
Clopinet: causation and prediction challenge, WCCI 2008. http://www.causality.inf.ethz.ch (2008). Accessed 6 March 2015
Dubois D, Prade H (1992) Putting rough sets and fuzzy sets together. In: SÅowiÅki R (ed) Intelligent decision support, theory and decision library, vol 11. Springer, Dordrecht, pp 203–232
Eskandari S, Akbas E (2017) Supervised infinite feature selection. CoRR abs/1704.02665. http://arxiv.org/abs/1704.02665
Eskandari S, Javidi M (2016) Online streaming feature selection using rough sets. Int J Approx Reason 69:35–57
Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results. http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html
Glocer K, Eads D, Theiler J (2005) Online feature selection for pixel classification. In: Proceedings of the 22nd international conference on machine learning, Bonn, Germany
Gosztolya G, Tóth L (2017) A feature selection-based speaker clustering method for paralinguistic tasks. Pattern Anal Appl 1–12
Herbert JP, Yao J (2011) Game-theoretic rough sets. Fundam Inform 108(3–4):267–286
Javidi MM, Eskandari S (2016) Streamwise feature selection: a rough set method. Int J Mach Learn Cybern 1–10
Javidi MM, Eskandari S (2017) A noise resistant dependency measure for rough set-based feature selection. J Intell Fuzzy Syst 33(3):1–14
Jensen R, Shen Q (2001) A rough set-aided system for sorting www bookmarks. In: Proceedings of the first Asia-Pacific conference on web intelligence: research and development, WI’01, London, UK
Jensen R, Shen Q (2005) Fuzzy-rough data reduction with ant colony optimization. Fuzzy Sets Syst 149(1):5–20
Jensen R, Shen Q (2008) Computational intelligence and feature selection: rough and fuzzy approaches. Wiley, London
Jensen R (2004) Qiang Shen: semantics-preserving dimensionality reduction: rough and fuzzy-rough based approaches. IEEE Trans Knowl Data Eng 16(16):1457–1471
Jensen R, Tuson A, Shen Q (2014) Finding rough and fuzzy-rough set reducts with SAT. Inf Sci 255:100–120
Liu W, Zha ZJ, Wang Y, Lu K, Tao D (2016) p-Laplacian regularized sparse coding for human activity recognition. IEEE Trans Ind Electron 63(8):5120–5129
Li T, Ruan D, Geert W, Song J, Xu Y (2007) A rough sets based characteristic relation approach for dynamic attribute generalization in data mining. Knowl Based Syst 20(5):485–494
Li Z, Liu J, Tang J, Lu H (2015) Robust structured subspace learning for data representation. IEEE Trans Pattern Anal Mach Intell 37(10):2085–2098
Li Z, Liu J, Yang Y, Zhou X, Lu H (2014) Clustering-guided sparse structural learning for unsupervised feature selection. IEEE Trans Knowl Data Eng 26(9):2138–2150
Li Z, Tang J (2015) Unsupervised feature selection via nonnegative spectral analysis and redundancy control. IEEE Trans Image Process 24(12):5343–5355
Parthalain N, Shen Q, Jensen R (2010) A distance measure approach to exploring the rough set boundary region for attribute reduction. IEEE Trans Knowl Data Eng 22(3):305–317
Pawlak Z (1982) Rough sets. Int J Comput Inform Sci 11(5):341–356
Perkins S, Theiler J (2003) Online feature selection using grafting. In: International conference on machine learning. ACM Press, pp 592–599
Pudil P, Novovičová J, Kittler J (1994) Floating search methods in feature selection. Pattern Recogn Lett 15(11):1119–1125
Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Preprint arXiv:1409.1556
Skowron A, Stepaniuk J (1996) Tolerance approximation spaces. Fundam Inf 27(2–3):245–253
Swiniarski RW, Skowron A (2003) Rough set methods in feature selection and recognition. Pattern Recogn Lett 24(6):833–849
Ungar L, Zhou J, Foster D, Stine B (2005) Streaming feature selection using IIC. In: Proceedings of the 10th international conference on artificial intelligence and statistics
Walczak B, Massart D (1999) Rough sets theory. Chemometr Intell Lab Syst 47(1):1–16
Wang F, Liang J, Qian Y (2013) Attribute reduction: a dimension incremental strategy. Knowl Based Syst 39:95–108
Wang J, Zhao P, Hoi S, Jin R (2014) Online feature selection and its applications. IEEE Trans Knowl Data Eng 26(3):698–710. https://doi.org/10.1109/TKDE.2013.32
Witten IH, Frank E (2005) Data mining: practical machine learning tools and techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems). Morgan Kaufmann Publishers Inc., San Francisco
Wu X, Yu K, Ding W, Wang H, Zhu X (2013) Online feature selection with streaming features. IEEE Trans Pattern Anal Mach Intell 35:1178–1192
Yang X, Liu W, Tao D, Cheng J (2017) Canonical correlation analysis networks for two-view image recognition. Inf Sci 385–386:338–352
Yao Y (2007) Decision-theoretic rough set models. Springer, Berlin, pp 1–12
Yu K, Ding W, Simovici DA, Wang H, Pei J, Wu X (2015) Classification with streaming features: an emerging-pattern mining approach. ACM Trans Knowl Discov Data (TKDD) 9(4):30
Yu K, Ding W, Wu X (2016) Lofs: a library of online streaming feature selection. Knowl Based Syst 113:1–3
Yu K, Wang D, Ding W, Pei J, Small DL, Islam S, Wu X (2015) Tornado forecasting with multiple markov boundaries. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 2237–2246
Yu K, Wu X, Ding W, Pei J (2014) Towards scalable and accurate online feature selection for big data. In: IEEE international conference on data mining (ICDM). IEEE, pp 660–669
Zhou J, Foster D, Stine R, Ungar L (2005) Streaming feature selection using alpha-investing. In: Proceedings of the 11th ACM SIGKDD international conference on knowledge discovery in data mining, Chicago, IL, USA
Ziarko W (1993) Variable precision rough set model. J Comput Syst Sci 46(1):39–59
Acknowledgements
The authors would like to thank professor Mahbano Tata for her comments that greatly improved the manuscript.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Javidi, M.M., Eskandari, S. Online streaming feature selection: a minimum redundancy, maximum significance approach. Pattern Anal Applic 22, 949–963 (2019). https://doi.org/10.1007/s10044-018-0690-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10044-018-0690-7