technical-note

Performance evaluation of evolutionary algorithms in classification of biomedical datasets

Authors:

Ajay Kumar Tanwani,

Muddassar FarooqAuthors Info & Claims

GECCO '09: Proceedings of the 11th Annual Conference Companion on Genetic and Evolutionary Computation Conference: Late Breaking Papers

Pages 2617 - 2624

https://doi.org/10.1145/1570256.1570371

Published: 08 July 2009 Publication History

Abstract

Biomedical datasets pose a unique challenge for machine learning and data mining techniques to extract accurate, comprehensible and hidden knowledge from them. In this paper, we comprehensively investigate the role of a biomedical dataset on the classification accuracy of an algorithm. To this end, we quantify the complexity of a biomedical dataset in terms of its missing values, imbalance ratio, noise and information gain. We have performed our experiments using six well-known evolutionary rule learning algorithms: XCS, UCS, GAssist, cAnt-Miner, SLAVE and Ishibuchi, on 31 publicly available biomedical datasets. The results of our experiments show that GAssist gives better classification accuracy among the compared schemes. However, the nature of a biomedical dataset -- not the selection of evolutionary algorithm -- plays a major role in determining the classification accuracy of a dataset. We further show that noise is a dominating factor in determining the complexity of a dataset and it is inversely proportional to the classification accuracy of all the algorithms. The complexity of biomedical dataset will prove useful to researchers in evaluating the classification potential of their dataset for automatic knowledge extraction.

References

[1]

]]C.A. Pena-Reyes, M. Sipper, "Evolutionary computation in medicine: an overview", Journal of Artificial Intelligence in Medicine, 19(1), pp. 1--23, 2000.

[2]

]]R.S. Parpinelli, H.S. Lopes, A.A. Freitas, "Data mining with an ant colony optimization algorithm", IEEE Transactions on Evolutionary Computation, 6(4), pp. 321--332, 2002.

Digital Library

[3]

]]M.L. Wong, W. Lam, K.S. Leung, P.S. Ngan, J.C.V. Cheng, "Discovering knowledge from medical databases using evolutionary algorithms", IEEE Engineering in Medicine and Biology, 19(4), pp. 45--55, 2000.

[4]

]]J.H. Homes, "Learning classifier systems applied to knowledge discovery in clinical research databases", International Workshop on Learning Classifier Systems (IWLCS), pp. 243--261, 2000.

Digital Library

[5]

]]E. Bernado Mansilla, "Domain of competence of XCS classifier system in complexity measurement space", IEEE Transactions on Evolutionary Computation, 9(1), pp. 82--104, 2005.

Digital Library

[6]

]]F. Kharbat, L. Bull, M. Odeh, "Mining breast cancer data with XCS", Genetic and Evolutionary Computation Conference (GECCO), pp. 2066--2073, UK, 2007.

Digital Library

[7]

]]A.O.-Puig, E.B.-Mansilla, "Evolutionary rule-based systems for imbalanced data sets", Soft Computing -- A Fusion of Foundations, Methodologies and Applications, 13(3), pp. 213--225, Springer, 2009.

Digital Library

[8]

]]J. Bacardit, M.V. Butz, "Data mining in learning classifier systems: comparing XCS with GAssist", International Workshop on Learning Classifier Systems (IWLCS), pp. 282--290, UK, 2007.

Digital Library

[9]

]]E. Bernado, X. Llora, J.M. Garrell, "XCS and GALE: a comparative study of two learning classifier systems with six other learning algorithms on classification tasks", International Workshop on Learning Classifier Systems (IWLCS), pp. 115--132, Spain, 2002.

[10]

]]R.S. Parpinelli, H.S. Lopes, A.A. Freitas, "An ant colony based system for data mining: applications to medical data", Int. Conf. on Knowledge Discovery and Data mining, pp. 55--62, Boston, 2000.

[11]

]]M. Galea, Q. Shen, J. Levine, "Evolutionary approaches to fuzzy modelling for classification", Knowledge Engineering Review, 19(1), pp. 27--59, 2004.

Digital Library

[12]

]]UCI repository of machine learning databases, University of California-Irvine, Department of Information and Computer Science. www.ics.uci.edu/~mlearn/MLRepository.html

[13]

]]I.H. Witten, E. Frank, "Data mining: practical machine learning tools and techniques", Morgan Kaufmann, 2nd edition, 2005.

Digital Library

[14]

]]X. Zhu, X. Wu, "Class noise vs. attribute noise: a quantitative study of their impacts", Artificial Intelligence Review, 22(3), pp. 177--210, 2004.

Digital Library

[15]

]]C.E. Brodley, M.A. Friedl, "Identifying mislabeled training data", Journal of Artificial Intelligence Research, 11, pp. 131--167, 1999.

[16]

]]F.E.B. Otero, Ant Colony Optimization Framework, MYRA. http://sourceforge.net/projects/myra/

[17]

]]J. Alcala-Fdez, L. Sanchez, S. Garcia, M.J. del Jesus, S. Ventura, J.M. Garrell, J. Otero, C. Romero, J. Bacardit, V.M. Rivas, J.C. Fernandez, F. Herrera, "KEEL: a software tool to assess evolutionary algorithms for data mining problems", Soft Computing, 13, pp. 307--318, Springer, 2008.

Digital Library

[18]

]]M.V. Butz, T. Kovacs, P.L. Lanzi, S.W. Wilson, "Toward a theory of generalization and learning in XCS", IEEE Transactions on Evolutionary Computation, 8(1), pp. 28--46, 2004.

Digital Library

[19]

]]E. Bernado-Mansilla, J.M. Garrell-Guiu, "Accuracy-based learning classifier systems: models, analysis and applications to classification tasks", Evolutionary Computation, 11(3), pp. 209--238, MIT Press, 2006.

Digital Library

[20]

]]J. Bacardit, J.M. Garrell, "Bloat control and generalization pressure using the minimum description length principle for a Pittsburgh approach Learning Classifier System", International Workshop on Learning Classifier Systems (IWLCS), pp. 59--79, UK, 2007.

Digital Library

[21]

]]F.E.B. Otero, A.A. Freitas, C.J. Johnson, "cAnt-Miner: an ant colony classification algorithm to cope with continuous attributes", Ant Colony Optimization and Swarm Intelligence, pp. 48--59, Belgium, 2008.

Digital Library

[22]

]]A. Gonzalez, R. Perez, "SLAVE: a genetic learning system based on an iterative approach", IEEE Transaction on Fuzzy Systems, 7(2), pp. 176--191, 1999.

Digital Library

[23]

]]H. Ishibuchi, T. Nakashima, T. Murata, "Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems", IEEE Transactions on Systems, Man, and Cybernetics, 29(5), pp. 601--618, 1999.

Digital Library

[24]

]]A. Orriols-Puig, E. Bernado-Mansilla, "Revisiting UCS: description, fitness sharing and comparison with XCS", International Workshop on Learning Classifier Systems (IWLCS), pp. 96--116, 2008.

Digital Library

[25]

]]T. Jo, N. Japkowicz, "Class imbalances versus small disjuncts", SIGKDD Explor. Newsl., 6(1), pp. 40--49, 2004.

Digital Library

Cited By

Djahafi FGafour A(2022)Neuro-Immune Model Based on Bio-Inspired Methods for Medical DiagnosisInternational Journal of Ambient Computing and Intelligence10.4018/IJACI.29317613:1(1-18)Online publication date: 18-Mar-2022
https://dl.acm.org/doi/10.4018/IJACI.293176
Datta SPal Choudhury J(2020)A Comparative Study on the Performance of Fuzzy Logic, Particle Swarm Optimization, Firefly Algorithm and Cuckoo Search Algorithm Using Residual AnalysisIntelligent Techniques and Applications in Science and Technology10.1007/978-3-030-42363-6_106(923-930)Online publication date: 3-Mar-2020
https://doi.org/10.1007/978-3-030-42363-6_106
Gorzałczany MRudziński F(2017)Classification of Splice-Junction DNA Sequences Using Multi-objective Genetic-Fuzzy Optimization TechniquesArtificial Intelligence and Soft Computing10.1007/978-3-319-59063-9_57(638-648)Online publication date: 27-May-2017
https://doi.org/10.1007/978-3-319-59063-9_57
Show More Cited By

Index Terms

Recommendations

Guidelines to Select Machine Learning Scheme for Classification of Biomedical Datasets
EvoBIO '09: Proceedings of the 7th European Conference on Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics

Biomedical datasets pose a unique challenge to machine learning and data mining algorithms for classification because of their high dimensionality, multiple classes, noisy data and missing values. This paper provides a comprehensive evaluation of a set ...
The Role of Biomedical Dataset in Classification
AIME '09: Proceedings of the 12th Conference on Artificial Intelligence in Medicine: Artificial Intelligence in Medicine

In this paper, we investigate the role of a biomedical dataset on the classification accuracy of an algorithm. We quantify the complexity of a biomedical dataset using five complexity measures: correlation-based feature selection subset merit, noise, ...
A binary Bi-phase mutation-based hybrid Equilibrium Optimizer for feature selection in medical datasets classification
Highlights
- A Bi-phase mutation-based Hybrid EO (BMHEO) method is proposed for feature selection.
- The approach combines the EO method with a Sine operator and a mutation scheme.
- Effects of two transfer functions (TF) in BMHEO (S-shape and V-...
Abstract
With the rapid expansion in Biological Sciences, biomedical data classification has become challenging. These datasets generally consist of missing values, redundant features and irrelevant information. A novel hybrid wrapper-based feature ...
Graphical abstract

Display Omitted

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '09: Proceedings of the 11th Annual Conference Companion on Genetic and Evolutionary Computation Conference: Late Breaking Papers

July 2009

1760 pages

ISBN:9781605585055

DOI:10.1145/1570256

General Chair:
Franz Rothlauf
University of Mainz, Germany

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 July 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Technical-note

Conference

GECCO09

Sponsor:

GECCO09: Genetic and Evolutionary Computation Conference

July 8 - 12, 2009

Québec, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
262
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)1

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Djahafi FGafour A(2022)Neuro-Immune Model Based on Bio-Inspired Methods for Medical DiagnosisInternational Journal of Ambient Computing and Intelligence10.4018/IJACI.29317613:1(1-18)Online publication date: 18-Mar-2022
https://dl.acm.org/doi/10.4018/IJACI.293176
Datta SPal Choudhury J(2020)A Comparative Study on the Performance of Fuzzy Logic, Particle Swarm Optimization, Firefly Algorithm and Cuckoo Search Algorithm Using Residual AnalysisIntelligent Techniques and Applications in Science and Technology10.1007/978-3-030-42363-6_106(923-930)Online publication date: 3-Mar-2020
https://doi.org/10.1007/978-3-030-42363-6_106
Gorzałczany MRudziński F(2017)Classification of Splice-Junction DNA Sequences Using Multi-objective Genetic-Fuzzy Optimization TechniquesArtificial Intelligence and Soft Computing10.1007/978-3-319-59063-9_57(638-648)Online publication date: 27-May-2017
https://doi.org/10.1007/978-3-319-59063-9_57
AlMuhaideb SMenai M(2016)An Individualized Preprocessing for Medical Data ClassificationProcedia Computer Science10.1016/j.procs.2016.04.00682(35-42)Online publication date: 2016
https://doi.org/10.1016/j.procs.2016.04.006
Almuhaideb SMenai M(2016)Impact of preprocessing on medical data classificationFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-016-5203-510:6(1082-1102)Online publication date: 1-Dec-2016
https://dl.acm.org/doi/10.1007/s11704-016-5203-5
Guzzo AMoccia LSaccà DSerra E(2013)Solving inverse frequent itemset mining with infrequency constraints via large-scale linear programsACM Transactions on Knowledge Discovery from Data10.1145/2541268.25412717:4(1-39)Online publication date: 25-Dec-2013
https://dl.acm.org/doi/10.1145/2541268.2541271
Al-Muhaideb SEl Bachir Menai M(2013)Hybrid Metaheuristics for Medical Data ClassificationHybrid Metaheuristics10.1007/978-3-642-30671-6_7(187-217)Online publication date: 2013
https://doi.org/10.1007/978-3-642-30671-6_7
Afridi MFarooq M(2011)OG-MinerProceedings of the 2011 44th Hawaii International Conference on System Sciences10.1109/HICSS.2011.320(1-10)Online publication date: 4-Jan-2011
https://dl.acm.org/doi/10.1109/HICSS.2011.320
Sharma ASharma D(2011)Clonal Selection Algorithm for ClassificationArtificial Immune Systems10.1007/978-3-642-22371-6_31(361-370)Online publication date: 2011
https://doi.org/10.1007/978-3-642-22371-6_31

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents