research-article

In search of targeted-complexity problems

Authors:

Núria Macià,

Albert Orriols-Puig,

Ester Bernadó-MansillaAuthors Info & Claims

GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computation

Pages 1055 - 1062

https://doi.org/10.1145/1830483.1830674

Published: 07 July 2010 Publication History

Get Access

Abstract

Currently available real-world problems do not cover the whole complexity space and, therefore, do not allow us to thoroughly test learner behavior on the border of its domain of competence. Thus, the necessity of developing a more suitable testing scenario arises. With this in mind, data complexity analysis has shown promise in characterizing difficulty of classification problems through a set of complexity descriptors which used in artificial data sets generation could supply the required framework to refine and design learners. This paper, then, proposes the use of instance selection based on an evolutionary multiobjective technique to generate data sets that meet specific characteristics established by such complexity descriptors. These artificial targeted-complexity problems, which capture the essence of real-world structures, may help to define a set of benchmarks that contributes to test the properties of learners and to improve them.

References

[1]

A. Asuncion and D. J. Newman. UCI Machine Learning Repository. {http://www.ics.uci.edu/~mlearn/MLRepository.html}. University of California, 2007.

Google Scholar

[2]

M. Basu and T. K. Ho, editors. Data Complexity in Pattern Recognition. Springer-Verlag, 2006.

Digital Library

Google Scholar

[3]

E. Bernadó-Mansilla, T. K. Ho, and A. Orriols-Puig. Data complexity and evolutionary learning: Classifier's behavior and domain of competence, pages 115--134. Springer, 2006.

Google Scholar

[4]

C. A. Coello, G. B. Lamont, and D. A. V. Veldhuizen. Evolutionary algorithms for solving multi-objective problems. Springer, New York, 2nd edition, 2007.

Digital Library

Google Scholar

[5]

K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE TEC, 6(2):182--197, 2002.

Digital Library

Google Scholar

[6]

D. E. Goldberg. The design of innovation: Lessons from and for competent genetic algorithms. Kluwer Academic Publishers, 1 edition, 2002.

Digital Library

Google Scholar

[7]

T. K. Ho. Data complexity analysis for classifier combination. In Proceedings of the 2nd Int. Workshop on MCS, pages 53--67. Springer-Verlag, 2001.

Digital Library

Google Scholar

[8]

T. K. Ho and M. Basu. Measuring the complexity of classification problems. In Proceedings of the 15th Int. Conference on PR, pages 43--47, 2000.

Google Scholar

[9]

T. K. Ho and M. Basu. Complexity measures of supervised classification problems. IEEE Transactions on PAMI, 24(3):289--300, 2002.

Digital Library

Google Scholar

[10]

E. J. Hughes. Evolutionary many-objective optimisation: many once or one many? In Congress on Evolutionary Computation, pages 222--227, 2005.

Crossref

Google Scholar

[11]

H. Ishibuchi, N. Tsukamoto, and Y. Nojima. Evolutionary many-objective optimization: A short review. In IEEE CEC, pages 2419--2426, 2008.

Crossref

Google Scholar

[12]

N. Macià, A. Orriols-Puig, and E. Bernadó-Mansilla. EMO shines a light on the holes of complexity space. In GECCO, pages 1907--1908, 2009.

Digital Library

Google Scholar

[13]

I. H. Witten and E. Frank. Data mining: Practical machine learning tools and techniques. Morgan Kaufmann, San Francisco, 2nd edition, 2005.

Digital Library

Google Scholar

Cited By

View all

Pereira SMiranda PFranca TBastos-Filho CSi T(2022)A Many-Objective Optimization Approach to Generate Synthetic Datasets based on Real-World Classification Problems2022 IEEE Latin American Conference on Computational Intelligence (LA-CCI)10.1109/LA-CCI54402.2022.9981848(1-6)Online publication date: 23-Nov-2022
https://doi.org/10.1109/LA-CCI54402.2022.9981848
Schmeing EBrun ASilva R(2022)Dynamic selection of classifiers based on complexity measures2022 IEEE 34th International Conference on Tools with Artificial Intelligence (ICTAI)10.1109/ICTAI56018.2022.00021(82-89)Online publication date: Oct-2022
https://doi.org/10.1109/ICTAI56018.2022.00021
Fraca TMiranda PPrudencio RLorenaz ANascimento A(2020)A Many-Objective optimization Approach for Complexity-based Data set Generation2020 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC48606.2020.9185543(1-8)Online publication date: Jul-2020
https://doi.org/10.1109/CEC48606.2020.9185543
Show More Cited By

Index Terms

In search of targeted-complexity problems
1. Computing methodologies
  1. Machine learning
2. Theory of computation
  1. Design and analysis of algorithms

Recommendations

Meta-level multi-objective formulations of set optimization for multi-objective optimization problems: multi-reference point approach to hypervolume maximization
GECCO Comp '14: Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation

Hypervolume has been frequently used as an indicator to evaluate a solution set in indicator-based evolutionary algorithms (IBEAs). One important issue in such an IBEA is the choice of a reference point. A different solution set is often obtained from a ...
Dynamic multiobjective optimization problems: test cases, approximations, and applications

After demonstrating adequately the usefulness of evolutionary multiobjective optimization (EMO) algorithms in finding multiple Pareto-optimal solutions for static multiobjective optimization problems, there is now a growing need for solving dynamic ...
EMO shines a light on the holes of complexity space
GECCO '09: Proceedings of the 11th Annual conference on Genetic and evolutionary computation

Typical domains used in machine learning analyses only cover the complexity space partially, remaining a large proportion of problem difficulties that are not tested. Since the acquisition of new real-world problems is costly, the machine learning ...

Comments

Information & Contributors

Information

Published In

GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computation

July 2010

1520 pages

ISBN:9781450300728

DOI:10.1145/1830483

General Chair:
Martin Pelikan
University of Missouri, USA
,
Program Chair:
Jürgen Branke
University of Warwick, Coventry, UK

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '10

Sponsor:

SIGEVO

GECCO '10: Genetic and Evolutionary Computation Conference

July 7 - 11, 2010

Oregon, Portland, USA

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
155
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)1

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Pereira SMiranda PFranca TBastos-Filho CSi T(2022)A Many-Objective Optimization Approach to Generate Synthetic Datasets based on Real-World Classification Problems2022 IEEE Latin American Conference on Computational Intelligence (LA-CCI)10.1109/LA-CCI54402.2022.9981848(1-6)Online publication date: 23-Nov-2022
https://doi.org/10.1109/LA-CCI54402.2022.9981848
Schmeing EBrun ASilva R(2022)Dynamic selection of classifiers based on complexity measures2022 IEEE 34th International Conference on Tools with Artificial Intelligence (ICTAI)10.1109/ICTAI56018.2022.00021(82-89)Online publication date: Oct-2022
https://doi.org/10.1109/ICTAI56018.2022.00021
Fraca TMiranda PPrudencio RLorenaz ANascimento A(2020)A Many-Objective optimization Approach for Complexity-based Data set Generation2020 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC48606.2020.9185543(1-8)Online publication date: Jul-2020
https://doi.org/10.1109/CEC48606.2020.9185543
Lorena AGarcia LLehmann JSouto MHo T(2019)How Complex Is Your Classification Problem?ACM Computing Surveys10.1145/334771152:5(1-34)Online publication date: 13-Sep-2019
https://dl.acm.org/doi/10.1145/3347711
Shand CAllmendinger RHandl JWebb AKeane JAuger AStützle T(2019)Evolving controllably difficult datasets for clusteringProceedings of the Genetic and Evolutionary Computation Conference10.1145/3321707.3321761(463-471)Online publication date: 13-Jul-2019
https://dl.acm.org/doi/10.1145/3321707.3321761
Muñoz MVillanova LBaatar DSmith-Miles K(2018)Instance spaces for machine learning classificationMachine Language10.1007/s10994-017-5629-5107:1(109-147)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1007/s10994-017-5629-5
Macií NBernadó-Mansilla EOrriols-Puig AKam Ho T(2013)Learner excellence biased by data set selectionPattern Recognition10.1016/j.patcog.2012.09.02246:3(1054-1066)Online publication date: 1-Mar-2013
https://dl.acm.org/doi/10.1016/j.patcog.2012.09.022
Ishibuchi HNojima Y(2011)Toward quantitative definition of explanation ability of fuzzy rule-based classifiers2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011)10.1109/FUZZY.2011.6007738(549-556)Online publication date: Jun-2011
https://doi.org/10.1109/FUZZY.2011.6007738
Macià NHo TOrriols-Puig ABernadó-Mansilla E(2010)The landscape contest at ICPR 2010Proceedings of the 20th International conference on Recognizing patterns in signals, speech, images, and videos10.5555/1939170.1939177(29-45)Online publication date: 23-Aug-2010
https://dl.acm.org/doi/10.5555/1939170.1939177
Macià NHo TOrriols-Puig ABernadó-Mansilla E(2010)The Landscape Contest at ICPR 2010Recognizing Patterns in Signals, Speech, Images and Videos10.1007/978-3-642-17711-8_4(29-45)Online publication date: 2010
https://doi.org/10.1007/978-3-642-17711-8_4

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Meta-level multi-objective formulations of set optimization for multi-objective optimization problems: multi-reference point approach to hypervolume maximization

Dynamic multiobjective optimization problems: test cases, approximations, and applications

EMO shines a light on the holes of complexity space