article

Ensemble classification for imbalanced data based on feature space partitioning and hybrid metaheuristics

Authors:

Pedro Lopez-Garcia,

Antonio D. Masegosa,

Enrique Onieva,

Asier PerallosAuthors Info & Claims

Applied Intelligence, Volume 49, Issue 8

Pages 2807 - 2822

https://doi.org/10.1007/s10489-019-01423-6

Published: 01 August 2019 Publication History

Abstract

One of the most challenging issues when facing a classification problem is to deal with imbalanced datasets. Recently, ensemble classification techniques have proven to be very successful in addressing this problem. We present an ensemble classification approach based on feature space partitioning for imbalanced classification. A hybrid metaheuristic called GACE is used to optimize the different parameters related to the feature space partitioning. To assess the performance of the proposal, an extensive experimentation over imbalanced and real-world datasets compares different configurations and base classifiers. Its performance is competitive with that of reference techniques in the literature.

References

[1]

Alcala-Fdez J, Alcala R, Herrera F (2011) A fuzzy association rule-based classification model for high-dimensional problems with genetic rule selection and lateral tuning. IEEE Trans Fuzzy Syst 19(5):857---872

Digital Library

[2]

Alcalá-Fdez J, Fernández A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2011) KEEL data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J Multiple-Valued Logic Soft Comput 17(2---3):255---287

[3]

Amami R, Ben Ayed D, Ellouze N (2013) Adaboost with SVM using GMM supervector for imbalanced phoneme data. In: 2013 The 6th international conference on human system interaction (HSI), pp 328---333

[4]

Bäck T, Schwefel H (1993) An overview of evolutionary algorithms for parameter optimization. Evol Comput 1(1):1--- 23

Digital Library

[5]

Bi Y, Guan J, Bell D (2008) The combination of multiple classifiers using an evidential reasoning approach. Artif Intell 172(15):1731---1751

Digital Library

[6]

Bian J, Peng XG, Wang Y, Zhang H (2016) An efficient cost-sensitive feature selection using chaos genetic algorithm for class imbalance problem. Math Probl Eng, 2016

[7]

Burges C (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2 (2):121---167

Digital Library

[8]

Cervantes J, Huang DS, García-Lamont F, Chau A (2014) A hybrid algorithm to improve the accuracy of support vector machines on skewed data-sets. In: International conference on intelligent computing, pp 782---788

[9]

Chawla NV, Lazarevic A, Hall LO, Bowyer KW (2003) SMOTEBoost: improving prediction of the minority class in boosting. In: European conference on principles of data mining and knowledge discovery. Springer, pp 107---119

[10]

Cover T, Hart P (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21---27

Digital Library

[11]

Danesh A, Moshiri B, Fatemi O (2007) Improve text classification accuracy based on classifier fusion methods. In: 10th International conference on information fusion, pp 1---6

[12]

Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3---18

[13]

Díez-Pastor JF, Rodríguez GOCJ, Kuncheva LIJ (2015) Random balance: ensembles of variable priors classifiers for imbalanced data. Knowl-Based Syst 85:96---111

Digital Library

[14]

Dorigo M, Gambardella LM (1997) Ant colony system: a cooperative learning approach to the traveling salesman problem. IEEE Trans Evol Comput 1(1):53---66

Digital Library

[15]

Duin RP (2002) The combining classifier: to train or not to train? In: Proceedings 16th international conference patter recognition, vol 2. IEEE, pp 765---770

[16]

Eshelman LJ, Schaffer JD (1992) Real-coded genetic algorithms and interval-schemata. Found Gen Algor 2:187---202

[17]

Fattahi S, Othman Z, Othman Z (2015) New approach with ensemble method to address class imbalance problem. J Theor Appl Inf Technol 72:1

[18]

Finner H (1993) On a monotonicity problem in step-down multiple test procedures. J Am Stat Assoc 88 (423):920---923

[19]

Giacinto G, Roli F (2001) Dynamic classifier selection based on multiple classifier behaviour. Pattern Recogn 34(9):1879--- 1881

[20]

Goldberg DE, Deb K (1991) A comparative analysis of selection schemes used in genetic algorithms. Found Gen Algor 1:69---93

[21]

Haixiang G, Xiuwu L, Kejun Z, Chang D, Yanhui G (2011) Optimizing reservoir features in oil exploration management based on fusion of soft computing. Appl Soft Comput 11(1):1144---1155

Digital Library

[22]

Hashem S (1997) Optimal linear combinations of neural networks. Neural Netw 10(4):599---614

Digital Library

[23]

Herrera F, Lozano M, Verdegay JL (1998) Tackling real-coded genetic algorithms: operators and tools for behavioural analysis. Artif Intell Rev 12(4):265---319

Digital Library

[24]

Ho D, Drake T, Bentley R, Valea F, Wax A (2015) Evaluation of hybrid algorithm for analysis of scattered light using ex vivo nuclear morphology measurements of cervical epithelium. Biom Opt Express 6 (8):2755---2765

[25]

Holland JH (1992) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT Press

Digital Library

[26]

Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6:65---70

[27]

Hopfield J (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA 79(8):2554---2558

[28]

Jackowski K, Wozniak M (2009) Algorithm of designing compound recognition system on the basis of combining classifiers with simultaneous splitting feature space into competence areas. Pattern Anal Applic 12(4):415---425

Digital Library

[29]

Jackowski K, Krawczyk B, Wo?niak M (2014) Improved adaptive splitting and selection: the hybrid training method of a classifier based on a feature space partitioning. Int J Neural Syst 24(03):1430007

[30]

Jackowski K (2015) Adaptive splitting and selection algorithm for regression. N Gener Comput 33(4):425---448

[31]

del Jesus M, Hoffmann F, Junco L, Sánchez L (2004) Induction of fuzzy-rule-based classifiers with evolutionary boosting algorithms. IEEE Trans Fuzzy Syst 12(3):296---308

Digital Library

[32]

Jurek A, Bi Y, Wu S, Nugent C (2011) Classification by cluster analysis: a new meta-learning based approach. Multiple Classif Syst, 259---268

Digital Library

[33]

Jurek A, Bi Y, Wu S, Nugent C (2014) A survey of commonly used ensemble-based classification techniques. Knowl Eng Rev 29(5):551---581

[34]

Kennedy J (2011) Particle swarm optimization. Encyclopedia of machine learning. Springer, pp 760---766

[35]

Krawczyk B, Cyganek B (2017) Selecting locally specialised classifiers for one-class classification ensembles. Pattern Anal Appl 20(2):427---439

Digital Library

[36]

Krawczyk B, McInnes BT (2018) Local ensemble learning from imbalanced and noisy data for word sense disambiguation. Pattern Recogn 78:103---119

Digital Library

[37]

Kuncheva LI (2004) Combining pattern classifiers: methods and algorithms. Wiley

Digital Library

[38]

Kuncheva LI, Jain LC (2000) Designing classifier fusion systems by genetic algorithms. IEEE Trans Evol Comput 4(4):327---336

Digital Library

[39]

Kuncheva LI, Whitaker CJ, Shipp CA, Duin RP (2003) Limits on the majority vote accuracy in classifier fusion. Pattern Anal Appl 6(1):22---31

[40]

Lavanya S, Palaniswami S, Divyabharathi M (2015) Resampling ensemble algorithm for class imbalance problem using optimization algorithm. Int J Appl Eng Res 10(13):11520---11526

[41]

Liu X, Lin J, Deng K (2011) Scheduling optimization in re-entrant lines based on a GA and PSO hybrid algorithm. Tongji Daxue Xuebao/J Tongji Univ 39:726---729

[42]

Lopez-Garcia P, Onieva E, Osaba E, Masegosa A, Perallos A (2016) Gace: a meta-heuristic based in the hybridization of genetic algorithms and cross entropy methods for continuous optimization. Expert Syst Appl 55:508---519

Digital Library

[43]

Lopez-Garcia P, Onieva E, Osaba E, Masegosa AD, Perallos A (2016) A hybrid method for short-term traffic congestion forecasting using genetic algorithms and cross entropy. IEEE Trans Intell Transp Syst 17(2):557---569

Digital Library

[44]

Lopez-Garcia P, Wo?niak M, Onieva E, Perallos A (2016c) Hybrid optimization method applied to adaptive splitting and selection algorithm. Lecture notes in computer science, vol 9648. Springer, pp 742---750

[45]

Mauša G, Galinac Grbac T (2017) Co-evolutionary multi-population genetic programming for classification in software defect prediction: an empirical case study. Appl Soft Comput J 55:331---351

Digital Library

[46]

Mokeddem D, Belbachir H (2009) A survey of distributed classification based ensemble data mining methods. J Appl Sci 9(20):3739---3745

[47]

Opitz DW, Maclin R (1999) Popular ensemble methods: an empirical study. J Artif Intell Res 11:169---198

[48]

Paredes R, Vidal E (2006) Learning weighted metrics to minimize nearest-neighbor classification error. IEEE Trans Pattern Anal Mach Intell 28(7):1100---1110

Digital Library

[49]

Qian Y, Liang Y, Li M, Feng G, Shi X (2014) A resampling ensemble algorithm for classification of imbalance problems. Neurocomputing 143:57---67

Digital Library

[50]

Rokach L (2010) Ensemble-based classifiers. Artif Intell Rev 33(1):1---39

Digital Library

[51]

Ruta D, Gabrys B (2005) Classifier selection for majority voting. Inform Fus 6(1):63---81

[52]

Seiffert C, Khoshgoftaar TM, Van Hulse J, Napolitano A (2010) RUSBoost: a hybrid approach to alleviating class imbalance. IEEE Trans Syst Man Cybern-Part A: Syst Humans 40(1):185---197

Digital Library

[53]

Sentinella M, Casalino L (2009) Cooperative evolutionary algorithm for space trajectory optimization. Celest Mech Dyn Astron 105(1-3):211

[54]

Stanciu S, Tranca D, Stanciu G, Hristu R, Bueno J (2016) Perspectives on combining nonlinear laser scanning microscopy and bag-of-features data classification strategies for automated disease diagnostics. Opt Quant Electron 48(6):320

[55]

Vorraboot P, Rasmequan S, Chinnasarn K, Lursinsap C (2015) Improving classification rate constrained to imbalanced data between overlapped and non-overlapped regions by hybrid algorithms. Neurocomputing 152:429---443

Digital Library

[56]

Wang S, Yao X (2009) Diversity analysis on imbalanced data sets by using ensemble models. In: Proceedings of IEEE symposium in computational intelligence and data mining, 2009, CIDM'09, pp 324---331

[57]

Wang S, Yao X (2012) Multiclass imbalance problems: analysis and potential solutions. IEEE Trans Syst Man Cybern Part B (Cybern) 42(4):1119---1130

Digital Library

[58]

Wang S, Minku L, Yao X (2015) Resampling-based ensemble methods for online class imbalance learning. IEEE Trans Knowl Data Eng 27(5):1356---1368

Digital Library

[59]

Xu L, Krzyzak A, Suen CY (1992) Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans Syst Man Cybern 22(3):418---435

[60]

Yang J, Ji Z, Xie W, Zhu Z (2016) Model selection based on particle swarm optimization for omics data classification. Shenzhen Daxue Xuebao (Ligong Ban)/J Shenzhen Univ Sci Eng 33(3):264---271

[61]

Yang P, Xu L, Zhou B, Zhang Z, Zomaya A (2009) A particle swarm based hybrid system for imbalanced medical data sampling. BMC Genomics 10:Suppl. 3.

[62]

Yang XS (2010) A new metaheuristic bat-inspired algorithm. Stud Comput Intell 284:65---74

[63]

Yu H, Ni J, Zhao J (2013) ACOSampling: an ant colony optimization-based undersampling method for classifying imbalanced DNA microarray data. Neurocomputing 101:309---318

Digital Library

[64]

Zhou ZH, Liu XY (2006) Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Trans Knowl Data Eng 18(1):63---77

Digital Library

[65]

Krawczyk B (2016) Learning from imbalanced data: open challenges and future directions. Progress Artif Intell 5(4):221---232

[66]

Cano A, Zafra A, Ventura S (2013) Weighted data gravitation classification for standard and imbalanced data. IEEE Trans Cybern 43(6):1672---1687

[67]

Mahdizadehaghdam S, Dai L, Krim H, Skau E, Wang H (2017) Image classification: a hierarchical dictionary learning approach. In: IEEE International conference in acoustics, speech and signal processing (ICASSP), 2017, pp 2597---2601

[68]

Khari M, Kumar P, Burgos D, Crespo RG (2017) Optimized test suites for automated testing using different optimization techniques. Soft Comput, 1---12

Digital Library

[69]

Fernández A, García S, Herrera F (2011) Addressing the classification with imbalanced data: open problems and new challenges on class distribution. Hybrid Artif Intell Syst, 1---10

[70]

Sun Y, Wong AK, Kamel MS (2009) Classification of imbalanced data: a review. Int J Pattern Recognit Artif Intell 23(04):687---719

[71]

Krawczyk B, Cano A, Wo?niak M (2018) Selecting local ensembles for multi-class imbalanced data classification, In: 2018 International joint conference on neural networks (IJCNN) 1---8

[72]

Fernandez A, Garcia S, Galar M, Prati RC, Krawczyk B, Herrera F (2018) Learning from imbalanced data sets. Springer

Cited By

Singh RKumar V(2024)Ensemble multi-view feature set partitioning method for effective multi-view learningKnowledge and Information Systems10.1007/s10115-024-02114-666:8(4957-5001)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s10115-024-02114-6
Uzun E(2023)Scraping Relevant Images from Web Pages without DownloadACM Transactions on the Web10.1145/361684918:1(1-27)Online publication date: 11-Oct-2023
https://dl.acm.org/doi/10.1145/3616849
Majd AVahidi-Asl MKhalilian ABagheri B(2022)ConsilientSFL: using preferential voting system to generate combinatorial ranking metrics for spectrum-based fault localizationApplied Intelligence10.1007/s10489-021-02954-752:10(11068-11088)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1007/s10489-021-02954-7
Show More Cited By

Index Terms

Ensemble classification for imbalanced data based on feature space partitioning and hybrid metaheuristics
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Classification and regression trees

Index terms have been assigned to the content through auto-classification.

Recommendations

A hybrid cost-sensitive ensemble for imbalanced breast thermogram classification

Graphical abstractDisplay Omitted HighlightsNovel cost-sensitive hybrid ensemble for efficient breast thermogram classification.Evolutionary algorithm for simultaneous feature selection and classifier weighting.A combined criterion for the ensemble ...
Towards Ensemble-Based Imbalanced Text Classification Using Metric Learning
Database and Expert Systems Applications
Abstract
This paper reports s series of ensemble approaches for imbalance text classification. All the approaches utilize a metric learning technique for obtaining better representations of texts to train weak classifiers. Each approach deals with the ...
Cost-sensitive decision tree ensembles for effective imbalanced classification

Real-life datasets are often imbalanced, that is, there are significantly more training samples available for some classes than for others, and consequently the conventional aim of reducing overall classification accuracy is not appropriate when dealing ...

Comments

Information & Contributors

Information

Published In

cover image Applied Intelligence

Applied Intelligence Volume 49, Issue 8

August 2019

351 pages

ISSN:0924-669X

Issue’s Table of Contents

Copyright © Copyright © 2019 Springer Science+Business Media, LLC, part of Springer Nature.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 August 2019

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Singh RKumar V(2024)Ensemble multi-view feature set partitioning method for effective multi-view learningKnowledge and Information Systems10.1007/s10115-024-02114-666:8(4957-5001)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s10115-024-02114-6
Uzun E(2023)Scraping Relevant Images from Web Pages without DownloadACM Transactions on the Web10.1145/361684918:1(1-27)Online publication date: 11-Oct-2023
https://dl.acm.org/doi/10.1145/3616849
Majd AVahidi-Asl MKhalilian ABagheri B(2022)ConsilientSFL: using preferential voting system to generate combinatorial ranking metrics for spectrum-based fault localizationApplied Intelligence10.1007/s10489-021-02954-752:10(11068-11088)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1007/s10489-021-02954-7
Hu JZhang HLiu YSutcliffe RFeng J(2022)BBW: a batch balance wrapper for training deep neural networks on extremely imbalanced datasets with few minority samplesApplied Intelligence10.1007/s10489-021-02623-952:6(6723-6738)Online publication date: 1-Apr-2022
https://dl.acm.org/doi/10.1007/s10489-021-02623-9
Mirzaei BRahmati FNezamabadi-pour H(2022)A score-based preprocessing technique for class imbalance problemsPattern Analysis & Applications10.1007/s10044-022-01084-125:4(913-931)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1007/s10044-022-01084-1
Srivastava VBiswas B(2022)LM-MFP: large-scale morphology and multi-criteria-based feature pooling for image parsingSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-022-07138-026:13(6201-6218)Online publication date: 1-Jul-2022
https://dl.acm.org/doi/10.1007/s00500-022-07138-0
Elden RGhoneim VHadhoud MAl-Atabany W(2022)Computational Microarray Gene Selection Model Using Metaheuristic Optimization Algorithm for Imbalanced Microarrays Based on Bagging and Boosting TechniquesModel and Data Engineering10.1007/978-3-031-21595-7_5(58-71)Online publication date: 21-Nov-2022
https://dl.acm.org/doi/10.1007/978-3-031-21595-7_5
Jedrzejowicz JJedrzejowicz P(2022)Collective of Base Classifiers for Mining Imbalanced DataComputational Science – ICCS 202210.1007/978-3-031-08754-7_62(571-585)Online publication date: 21-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-08754-7_62
Keswani VLease MKenthapadi KFourcade MKuipers BLazar SMulligan D(2021)Towards Unbiased and Accurate Deferral to Multiple ExpertsProceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society10.1145/3461702.3462516(154-165)Online publication date: 21-Jul-2021
https://dl.acm.org/doi/10.1145/3461702.3462516
Baliarsingh SMuhammad KBakshi S(2021)SARAApplied Soft Computing10.1016/j.asoc.2020.107009101:COnline publication date: 1-Mar-2021
https://dl.acm.org/doi/10.1016/j.asoc.2020.107009
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents