research-article

An automated ensemble learning framework using genetic programming for image classification

Authors:

Mengjie ZhangAuthors Info & Claims

GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference

Pages 365 - 373

https://doi.org/10.1145/3321707.3321750

Published: 13 July 2019 Publication History

Abstract

An ensemble consists of multiple learners and can achieve a better generalisation performance than a single learner. Genetic programming (GP) has been applied to construct ensembles using different strategies such as bagging and boosting. However, no GP-based ensemble methods focus on dealing with image classification, which is a challenging task in computer vision and machine learning. This paper proposes an automated ensemble learning framework using GP (EGP) for image classification. The new method integrates feature learning, classification function selection, classifier training, and combination into a single program tree. To achieve this, a novel program structure, a new function set and a new terminal set are developed in EGP. The performance of EGP is examined on nine different image classification data sets of varying difficulty and compared with a large number of commonly used methods including recently published methods. The results demonstrate that EGP achieves better performance than most competitive methods. Further analysis reveals that EGP evolves good ensembles simultaneously balancing diversity and accuracy. To the best of our knowledge, this study is the first work using GP to automatically generate ensembles for image classification.

Supplementary Material

ZIP File (p365-bi_suppl.zip)

Supplemental material.

Download
7.45 KB

References

[1]

Harith Al-Sahaf, Andy Song, Kourosh Neshatian, and Mengjie Zhang. 2012. Two-tier genetic programming: Towards raw pixel-based image classification. Expert Systems with Applications 39, 16 (2012), 12291--12301.

Digital Library

[2]

Harith Al-Sahaf, Mengjie Zhang, Ausama Al-Sahaf, and Mark Johnston. 2017. Keypoints detection and feature extraction: A dynamic genetic programming approach for evolving rotation-invariant texture image descriptors. IEEE Transactions on Evolutionary Computation 21, 6 (2017), 825 -- 844.

[3]

Ying Bi, Bing Xue, and Mengjie Zhang. 2018. A Gaussian Filter-Based Feature Learning Approach Using Genetic Programming to Image Classification. In Australasian Joint Conference on Artificial Intelligence. Springer, 251--257.

[4]

Ying Bi, Bing Xue, and Mengjie Zhang. 2018. A survey on genetic programming to image analysis. Journal of Zhengzhou University (Engineering Science) 39, 06 (2018), 3--13.

[5]

Joan Bruna and Stéphane Mallat. 2013. Invariant scattering convolution networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 8 (2013), 1872--1886.

Digital Library

[6]

Tsung-Han Chan, Kui Jia, Shenghua Gao, Jiwen Lu, Zinan Zeng, and Yi Ma. 2015. PCANet: A simple deep learning baseline for image classification? IEEE Transactions on Image Processing 24, 12 (2015), 5017--5032.

Digital Library

[7]

Boyuan Chen, Harvey Wu, Warren Mo, Ishanu Chattopadhyay, and Hod Lipson. 2018. Autostacker: A compositional evolutionary learning system. In Proceedings of the Genetic and Evolutionary Computation Conference. ACM, 402--409.

Digital Library

[8]

Grant Dick, Caitlin A Owen, and Peter A Whigham. 2018. Evolving bagging ensembles using a spatially-structured niching method. In Proceedings of the Genetic and Evolutionary Computation Conference. ACM, 418--425.

Digital Library

[9]

Pedro G Espejo, Sebastián Ventura, and Francisco Herrera. 2010. A survey on the application of genetic programming to classification. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 40, 2 (2010), 121--144.

Digital Library

[10]

Li Fei-Fei and Pietro Perona. 2005. A bayesian hierarchical model for learning natural scene categories. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2. 524--531.

Digital Library

[11]

Félix-Antoine Fortin, François-Michel De Rainville, Marc-André Gardner, Marc Parizeau, and Christian Gagné. 2012. DEAP: Evolutionary algorithms made easy. Journal of Machine Learning Research 13 (jul 2012), 2171--2175.

Digital Library

[12]

Muhammad Iqbal, Bing Xue, Harith Al-Sahaf, and Mengjie Zhang. 2017. Crossdomain reuse of extracted knowledge in genetic programming for image classification. IEEE Transactions on Evolutionary Computation 21, 4 (2017), 569--587.

[13]

Lukas Kammerer and Michael Affenzeller. 2018. Confidence-based ensemble modeling in medical data mining. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. ACM, 163--164.

Digital Library

[14]

Sašo Karakatič and Vili Podgorelec. 2018. Building boosted classification tree ensemble with genetic programming. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. ACM, 165--166.

Digital Library

[15]

John R Koza. {n. d.}. Genetic programming: on the programming of computers by means of natural selection. MIT press, Cambridge.

Digital Library

[16]

Hugo Larochelle, Dumitru Erhan, Aaron Courville, James Bergstra, and Yoshua Bengio. 2007. An empirical evaluation of deep architectures on problems with many factors of variation. In Proceedings of the 24th International Conference on Machine learning. ACM, 473--480.

Digital Library

[17]

Andrew Lensen, Harith Al-Sahaf, Mengjie Zhang, and Bing Xue. 2016. Genetic programming for region detection, feature extraction, feature construction and classification in image data. In European Conference on Genetic Programming. Springer, 51--67.

[18]

Hao Li and Maoguo Gong. 2017. Self-paced convolutional neural networks. In Proceedings of the International Joint Conference on Artificial Intelligence. 2110--2116.

Digital Library

[19]

Chengjun Liu and Harry Wechsler. 2001. A Gabor feature classifier for face recognition. In Eighth IEEE International Conference on Computer Vision, Vol. 2. 270--275.

[20]

Michael Lyons, Shigeru Akamatsu, Miyuki Kamachi, and Jiro Gyoba. 1998. Coding facial expressions with gabor wavelets. In Third IEEE International Conference on Automatic Face and Gesture Recognition. 200--205.

Digital Library

[21]

P Mallikarjuna, Alireza Tavakoli Targhi, Mario Fritz, Eric Hayman, Barbara Caputo, and Jan-Olof Eklundh. 2006. The kth-tips2 database. Computational Vision and Active Perception Laboratory, Stockholm, Sweden (2006), 1--10.

[22]

David J Montana. 1995. Strongly typed genetic programming. Evolutionary Computation 3, 2 (1995), 199--230.

Digital Library

[23]

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12 (2011), 2825--2830.

Digital Library

[24]

Salah Rifai, Pascal Vincent, Xavier Muller, Xavier Glorot, and Yoshua Bengio. 2011. Contractive auto-encoders: Explicit invariance during feature extraction. In Proceedings of the 28th International Conference on International Conference on Machine Learning. Omnipress, 833--840.

Digital Library

[25]

Mark E Roberts. 2003. The effectiveness of cost based subtree caching mechanisms in typed genetic programming for image segmentation. In Workshops on Applications of Evolutionary Computation. Springer, 444--454.

Digital Library

[26]

Ferdinando S Samaria and Andy C Harter. 1994. Parameterisation of a stochastic model for human face identification. In Proceedings of the Second IEEE Workshop on Applications of Computer Vision. 138--142.

[27]

Ling Shao, Li Liu, and Xuelong Li. 2014. Feature learning for image classification via multiobjective genetic programming. IEEE Transactions on Neural Networks and Learning Systems 25, 7 (2014), 1359--1371.

[28]

Carlos Eduardo Thomaz. 2012. FEI face database. online: http://fei.edu.br/~cet/facedatabase.html (2012).

[29]

Cao Truong Tran, Mengjie Zhang, Bing Xue, and Peter Andreae. 2018. Genetic Programming with Interval Functions and Ensemble Learning for Classification with Incomplete Data. In Australasian Joint Conference on Artificial Intelligence. Springer, 577--589.

[30]

Steven Young, Tamer Abdou, and Ayse Bener. 2018. Deep Super Learner: A Deep Ensemble for Classification Problems. In Advances in Artificial Intelligence: 31st Canadian Conference on Artificial Intelligence. Springer, 84--95.

[31]

Tingzhao Yu, Chaoxu Guo, Lingfeng Wang, Shiming Xiang, and Chunhong Pan. 2018. Self-paced autoencoder. IEEE Signal Processing Letters (2018).

[32]

Zhi-Hua Zhou. 2012. Ensemble methods: foundations and algorithms. Chapman and Hall/CRC.

Digital Library

[33]

Zhi-Hua Zhou and Ji Feng. 2018. Deep forest. National Science Review 6, 1 (2018), 74--86.

Cited By

Herrera-Sánchez DAcosta-Mesa HMezura-Montes EMárquez-Grajales ALi XHandl J(2024)Shifting Color Space for Image Classification using Genetic ProgrammingProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3654430(283-286)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3654430
Wang SHuang SLiu SBi Y(2023) Not just select samples, but exploration Genetic programming aided remote sensing target detection under deep learning Applied Soft Computing10.1016/j.asoc.2023.110570145(110570)Online publication date: Sep-2023
https://doi.org/10.1016/j.asoc.2023.110570
Li NMa LXing TYu GWang CWen YCheng SGao S(2023)Automatic design of machine learning via evolutionary computation: A surveyApplied Soft Computing10.1016/j.asoc.2023.110412143(110412)Online publication date: Aug-2023
https://doi.org/10.1016/j.asoc.2023.110412
Show More Cited By

Index Terms

An automated ensemble learning framework using genetic programming for image classification
1. Computing methodologies
  1. Machine learning

Recommendations

Building boosted classification tree ensemble with genetic programming
GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Adaptive boosting (AdaBoost) is a method for building classification ensemble, which combines multiple classifiers built in an iterative process of reweighting instances. This method proves to be a very effective classification method, therefore it was ...
Ensemble Genetic Programming
Genetic Programming
Abstract
Ensemble learning is a powerful paradigm that has been used in the top state-of-the-art machine learning methods like Random Forests and XGBoost. Inspired by the success of such methods, we have developed a new Genetic Programming method called ...
Ensemble classifiers using multi-objective Genetic Programming for unbalanced data
Abstract
Genetic Programming (GP) can be used to design effective classifiers due to its built-in feature selection and feature construction characteristics. Unbalanced data distributions affect the classification performance of GP classifiers. Some ...
Highlights
- Proposed multi-objective GP can significantly improve the classification performance compared with single-objective GP.
- Proposed ensemble classifiers can further improve the classification performance.
- Proposed approaches can ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference

July 2019

1545 pages

ISBN:9781450361118

DOI:10.1145/3321707

Editor:
Manuel López-Ibáñez
University of Manchester, UK
,
General Chairs:
Anne Auger
Inria and Ecole Polytechnique, France
,
Thomas Stützle
IRIDIA, Université libre de Bruxelles Belgium

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '19

Sponsor:

SIGEVO

GECCO '19: Genetic and Evolutionary Computation Conference

July 13 - 17, 2019

Prague, Czech Republic

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
438
Total Downloads

Downloads (Last 12 months)62
Downloads (Last 6 weeks)2

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Herrera-Sánchez DAcosta-Mesa HMezura-Montes EMárquez-Grajales ALi XHandl J(2024)Shifting Color Space for Image Classification using Genetic ProgrammingProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3654430(283-286)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3654430
Wang SHuang SLiu SBi Y(2023) Not just select samples, but exploration Genetic programming aided remote sensing target detection under deep learning Applied Soft Computing10.1016/j.asoc.2023.110570145(110570)Online publication date: Sep-2023
https://doi.org/10.1016/j.asoc.2023.110570
Li NMa LXing TYu GWang CWen YCheng SGao S(2023)Automatic design of machine learning via evolutionary computation: A surveyApplied Soft Computing10.1016/j.asoc.2023.110412143(110412)Online publication date: Aug-2023
https://doi.org/10.1016/j.asoc.2023.110412
Nguyen BXue BBrowne WZhang M(2023)Evolutionary ClassificationHandbook of Evolutionary Machine Learning10.1007/978-981-99-3814-8_7(171-204)Online publication date: 2-Nov-2023
https://doi.org/10.1007/978-981-99-3814-8_7
Shomanov AKuchenchirekova DKurenkov ALee M(2022)CT Image Classification Based on Stacked Ensemble Of Convolutional Neural Networks2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC53654.2022.9945565(1021-1025)Online publication date: 9-Oct-2022
https://doi.org/10.1109/SMC53654.2022.9945565
Yang LHe FDai LZhang L(2022)An Automatical And Efficient Image Classification Based On Improved Genetic Programming2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD)10.1109/CSCWD54268.2022.9776145(477-483)Online publication date: 4-May-2022
https://doi.org/10.1109/CSCWD54268.2022.9776145
Liu QWang XWang YSong X(2022)Evolutionary convolutional neural network for image classification based on multi-objective genetic programming with leader–follower mechanismComplex & Intelligent Systems10.1007/s40747-022-00919-y9:3(3211-3228)Online publication date: 29-Nov-2022
https://doi.org/10.1007/s40747-022-00919-y
Zhang TMa LLiu QLi NLiu Y(2022)Genetic Programming for Ensemble Learning in Face RecognitionAdvances in Swarm Intelligence10.1007/978-3-031-09726-3_19(209-218)Online publication date: 26-Jun-2022
https://doi.org/10.1007/978-3-031-09726-3_19
Li NMa LZhang THe M(2022)Multi-objective Evolutionary Ensemble Learning for Disease ClassificationAdvances in Swarm Intelligence10.1007/978-3-031-09677-8_41(491-500)Online publication date: 26-Jun-2022
https://doi.org/10.1007/978-3-031-09677-8_41
Shafqat WMalik SLee KKim D(2021)PSO Based Optimized Ensemble Learning and Feature Selection Approach for Efficient Energy ForecastElectronics10.3390/electronics1018218810:18(2188)Online publication date: 7-Sep-2021
https://doi.org/10.3390/electronics10182188
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents