Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Multi-objective Learning Classifier Systems

  • Chapter
Multi-Objective Machine Learning

Part of the book series: Studies in Computational Intelligence ((SCI,volume 16))

Abstract

Learning concept descriptions from data is a complex multiobjective task. The model induced by the learner should be accurate so that it can represent precisely the data instances, complete, which means it can be generalizable to new instances, and minimum, or easily readable. Learning Classifier Systems (LCSs) are a family of learners whose primary search mechanism is a genetic algorithm. Along the intense history of the field, the efforts of the community have been centered on the design of LCSs that solved these goals efficiently, resulting in the proposal of multiple systems. This paper revises the main LCS approaches and focuses on the analysis of the different mechanisms designed to fulfill the learning goals. Some of these mechanisms include implicit multiobjective learning mechanisms, while others use explicit multiobjective evolutionary algorithms. The paper analyses the advantages of using multiobjective evolutionary algorithms, especially in Pittsburgh LCSs, such as controlling the so-called bloat effect, and offering the human expert a set of concept description alternatives.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. D. Aha and D. Kibler. Instance-based learning algorithms. Machine Learning, 6:37–66, 1991.

    Google Scholar 

  2. J. Bacardit and M. V. Butz. Data Mining in Learning classifier Systems: Comparing XCS with GAssist. In Seventh International Workshop on Learning Classifier Systems (IWLCS-2004). LNAI, Springer (in press), 2004.

    Google Scholar 

  3. J. Bacardit and J. M. Garrell. Métodos de generalización para sistemas clasificadores de Pittsburgh. In Primer Congreso Espanol de Algoritmos Evolutivos y Bioinspirados (AEB’02), pages 486–493, 2002.

    Google Scholar 

  4. J. Bacardit and J. M. Garrell. Bloat Control and Generalization Pressure using the Minimum Description Length Principle for a Pittsburgh Approach Learning Classifier System. In Proceedings of the 6th International Workshop on Learning Classifier Systems. LNAI, Springer (in press), 2003.

    Google Scholar 

  5. J. Bacardit and J. M. Garrell. Evolving Multiple Discretizations with Adaptive Intervals for a Pittsburgh Rule-Based Learning Classifier System. In Proceedings of the Genetic and Evolutionary Computation Conference, volume 2724 of LNCS, pages 1818–1831. Springer, 2003.

    Google Scholar 

  6. T. Bäck. Generalized convergence models for tournament-and (μ, λ)-selection. Proceedings of the Sixth International Conference on Genetic Algorithms, pages 2–8, 1995.

    Google Scholar 

  7. J. E. Baker. Reducing bias and inefficieny in the selection algorithm. In Genetic Algorithms and their Applications: Proceedings of the Second International Conference on Genetic Algorithms, pages 14–21, 1987.

    Google Scholar 

  8. W. Banzhaf and W. B. Langdon. Some Considerations on the Reason for Bloat. Genetic Programming and Evolvable Hardware, 3(1):81–91, 2002.

    Article  MATH  Google Scholar 

  9. J. K. Bassett and K. A. De Jong. Evolving Behaviors for Cooperating Agents. In Foundations of Intelligent Systems: 12th International Symposium, volume 1932 of LNAI, pages 157–165. Springer-Verlag Berlin Heidelberg, 2000.

    Google Scholar 

  10. E. Bernadó-Mansilla and J. M. Garrell. MOLeCS: Using Multiobjective Evolutionary Algorithms for Learning. In Evolutionary Multi-Criterion Optimization, First International Conference, EMO 2001, volume 1993 of LNCS, pages 696–710. Springer Verlag, 2001.

    Google Scholar 

  11. E. Bernadó-Mansilla and J. M. Garrell. Accuracy-Based Learning Classifier Systems: Models, Analysis and Applications to Classification Tasks. Evolutionary Computation, 11(3):209–238, 2003.

    Article  Google Scholar 

  12. E. Bernadó-Mansilla and T. K. Ho. Domain of Competence of XCS Classifier System in Complexity Measurement Space. IEEE Transactions on Evolutionary Computation, 9(1):82–104, 2005.

    Article  Google Scholar 

  13. E. Bernadó-Mansilla, X. Llorà, and J. M. Garrell. XCS and GALE: a Comparative Study of Two Learning Classifier Systems on Data Mining. In Advances in Learning Classifier Systems, 4th International Workshop, volume 2321 of LNAI, pages 115–132. Springer, 2002.

    Google Scholar 

  14. E. Bernadó-Mansilla, A. Mekaouche, and J. M. Garrell. A Study of a Genetic Classifier System Based on the Pittsburgh Approach on a Medical Domain. In 12th International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems, IEA/AIE-99, pages 175–184, 1999.

    Google Scholar 

  15. S. Bleuler, M. Brack, L. Thiele, and E. Zitzler. Multiobjective genetic programming: Reducing bloat using SPEA2. In Proceedings of the 2001 Congress on Evolutionary Computation CEC 2001, pages 536–543. IEEE Press, 2001.

    Google Scholar 

  16. P. Bonelli, A. Parodi, S. Sen, and S. W. Wilson. NEWBOOLE: A fast GBML System. In Seventh International Conference on Machine Learning, pages 153–159. Morgan Kaufmann, 1990.

    Google Scholar 

  17. L. Breiman. Bagging predictors. Machine Learning, 24(2):123–140, 1996.

    MATH  MathSciNet  Google Scholar 

  18. L. Breiman, J. Friedman, R. Olshen, and C. Stone. Classification and Regression Trees. Wadsworth International Group, 1984.

    Google Scholar 

  19. M. V. Butz. Rule-based Evolutionary Online Learning Systems: Learning Bounds, Classification, and Prediction. PhD thesis, University of Illinois, 2004.

    Google Scholar 

  20. M. V. Butz, P. L. Lanzi, X. Llorà, and D. E. Goldberg. Knowledge Extraction and Problem Structure Identification in XCS. In Parallel Problem Solving from Nature (PPSN-2004), volume 3242 of LNCS, pages 1051–1060. Springer, 2004.

    Google Scholar 

  21. M. V. Butz and M. Pelikan. Analyzing the Evolutionary Pressures in XCS. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’2001), pages 935–942. San Francisco, CA: Morgan Kaufmann, 2001.

    Google Scholar 

  22. P Clark and T. Niblett. The CN2 induction algorithm. Machine Learning, 3(4):261–283, 1989.

    Google Scholar 

  23. K. A. De Jong and W. M. Spears. Learning Concept Classification Rules Using Genetic Algorithms. In Proceedings of the International Joint Conference on Artificial Intelligence, pages 651–656. Sidney, Australia, 1991.

    Google Scholar 

  24. P. W. Dixon, D. W. Corne, and M. J. Oates. A Preliminary Investigation of Modified XCS as a Generic Data Mining Tool. In Advances in Learning Classifier Systems, 4th International Workshop, volume 2321 of LNCS, pages 133–150. Springer, 2002.

    Google Scholar 

  25. I. W. Flockhart. GA-MINER: Parallel Data Mining with Hierarchical Genetic Algorithms. Technical Report EPCC-AIKMS-GA-MINER-REPORT 1.0, University of Edinburgh, 1995.

    Google Scholar 

  26. E. Frank and I. H. Witten. Generating Accurate Rule Sets Without Global Optimization. In Machine Learning: Proceedings of the Fifteenth International Conference, pages 144–151. Morgan Kaufmann, 1998.

    Google Scholar 

  27. J. M Garrell, E. Golobardes, E. Bernadó-Mansilla, and X. Llorà. Automatic Diagnosis with Genetic Algorithms and Case-Based Reasoning. Artificial Intelligence in Engineering, 13:367–372, 1999.

    Article  Google Scholar 

  28. A. Giordana and F. Neri. Search-Intensive Concept Induction. Evolutionary Computation, 3(4):375–416, 1995.

    Google Scholar 

  29. D. E. Goldberg. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Publishing Company, Inc., 1989.

    Google Scholar 

  30. D. E. Goldberg and J. Richardson. Genetic algorithms with sharing for multimodal function optimization. In Proceedings of the Second International Conference on Genetic Algorithms, pages 41–49, 1987.

    Google Scholar 

  31. D. P. Greene and S. F. Smith. Competition-based induction of decision models from examples. Machine Learning, 13:229–257, 1993.

    Article  Google Scholar 

  32. J. H. Holland. Processing and processors for schemata. In Associative Information Processing, pages 127–146, 1971.

    Google Scholar 

  33. J. H. Holland. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. MIT Press/ Bradford Books edition, 1975.

    Google Scholar 

  34. J. H. Holland and J. S. Reitman. Cognitive systems based on adaptive algorithms. Pattern Directed Inference Systems, pages 313–329, 1978.

    Google Scholar 

  35. J. H. Holmes. Discovering Risk of Disease with a Learning Classifier System. In Proceedings of the Seventh International Conference of Genetic Algorithms (ICGA97), pages 426–433. Morgan Kaufmann, 1997.

    Google Scholar 

  36. C. Janikow. Inductive Learning of Decision Rules in Attribute-Based Examples: a Knowledge-Intensive Genetic Algorithm Approach. PhD thesis, University of North Carolina at Chapel Hill, July 1991.

    Google Scholar 

  37. T. Kovacs. XCS Classifier System Reliably Evolves Accurate, Complete and Minimal Representations for Boolean Functions. In Soft Computing in Engineering Design and Manufacturing, pages 59–68. Springer-Verlag, 1997.

    Google Scholar 

  38. W. B. Langdon and R. Poli. Fitness causes bloat: Mutation. Genetic Programming: First European Conference, pages 37–48, 1998.

    Google Scholar 

  39. X. Llorà. Genetic Based Machine Learning using Fine-grained Parallelism for Data Mining. PhD thesis, Enginyeria i Arquitectura La Salle. Ramon Llull University, Barcelona, Catalonia, European Union, February, 2002.

    Google Scholar 

  40. X. Llorà, J. Bacardit, I. Traus, and E. Bernadó-Mansilla. Where to go once you evolved a bunch of promising hypotheses? In Learning Classifier Systems, 6th International Workshop, IWLCS 2003, LNAI. Springer (in press), 2005.

    Google Scholar 

  41. X. Llorà and J. M. Garrell. Automatic Classification and Artificial Life Models. In Proceedings of Learning00 Workshop. IEEE and Univesidad Carlos III, 2000.

    Google Scholar 

  42. X. Llorà and J. M. Garrell. Evolving Partially-Defined Instances with Evolutionary Algorithms. In Proceedings of the 18th International Conference on Machine Learning (ICML’2001), pages 337–344. Morgan Kaufmann Publishers, 2001.

    Google Scholar 

  43. X. Llorà and J. M. Garrell. Knowledge-Independent Data Mining with Fine-Grained Parallel Evolutionary Algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’2001), pages 461–468. Morgan Kaufmann Publishers, 2001.

    Google Scholar 

  44. X. Llorà and D. E. Goldberg. Bounding the effect of noise in Multiobjective Learning Classifier Systems. Evolutionary Computation, 11(3):279–298, 2003.

    Article  Google Scholar 

  45. X. Llorà, D. E. Goldberg, I. Traus, and E. Bernadó-Mansilla. Accuracy, Parsimony, and Generality in Evolutionary Learning Systems via Multiobjective Selection. In Learning Classifier Systems, 5th International Workshop, IWLCS 2002, volume 2661 of LNAI, pages 118–142. Springer, 2003.

    Google Scholar 

  46. X. Llorà, K. Sastry, and D. E. Goldberg. The Compact Classifier System: Scalability Analysis and First Results. In Proceedings of the IEEE Conference on Evolutionary Computation. IEEE press (in press), 2005.

    Google Scholar 

  47. C. J. Merz and P. M. Murphy. UCI Repository for Machine Learning Data-Bases [http://www.ics.uci.edu/~mlearn/MLRepository.html]. Irvine, CA: University of California, Department of Information and Computer Science, 1998.

    Google Scholar 

  48. T. M. Mitchell. Machine Learning. McGraw-Hill, 1997.

    Google Scholar 

  49. P. Nordin and W. Banzhaf. Complexity Compression and Evolution. In Proceedings of the Sixth International Conference on Genetic Algorithms, 1995.

    Google Scholar 

  50. C. K. Oei, D. E. Goldberg, and S. J. Chang. Tournament selection, niching, and the preservation of diversity. IlliGAL Report No. 91011, University of Illinois at Urbana-Champaign, Urbana, IL, 1991.

    Google Scholar 

  51. A. Orriols Puig and E. Bernadó-Mansilla. Analysis of Reduction Algorithms in XCS Classifier System. In Recent Advances in Artificial Intelligence Research and Development, volume 113 of Frontiers in Artificial Intelligence and Applications, pages 383–390. IOS Press, 2004.

    Google Scholar 

  52. A. Orriols Puig and E. Bernadó-Mansilla. The Class Imbalance Problem in Learning Classifier Systems: A Preliminary Study. In Learning Classifier Systems, 7th International Workshop, IWLCS 2005, LNAI. Springer (in press), 2005.

    Google Scholar 

  53. V. Pareto. Cours d'Economie Politique, volume I and II. F. Rouge, Lausanne, 1896.

    Google Scholar 

  54. R. Quinlan. Induction of decision trees. Machine Learning, 1(1):81–106, 1986.

    Google Scholar 

  55. R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, 1993.

    Google Scholar 

  56. J. Rissanen. Modeling by shortest data description. Automatica, vol. 14:465–471, 1978.

    Article  MATH  Google Scholar 

  57. D.E. Rumelhart, J.L. McClelland, and the PDP Research Group. Parallel Distributed Processing, Vol. I, II. The MIT Press, 1986.

    Google Scholar 

  58. S. F. Smith. Flexible Learning of Problem Solving Heuristics through Adaptive Search. In Proceedings of the 8th International Joint Conference on Artificial Intelligence, pages 422–425, 1983.

    Google Scholar 

  59. T. Soule and J. Foster. Effects of code growth and parsimony pressure on populations in genetic programming. Evolutionary Computation, 6(4):293–309, Winter 1998.

    Google Scholar 

  60. W. A. Tackett. Recombination, selection, and the genetic construction of computer programs. Unpublished doctoral dissertation, University of Southern California, 1994.

    Google Scholar 

  61. D. A. Van Veldhuizen and G. B. Lamont. Multiobjective evolutionary algorithms: Analyzing the state-of-the-art. Evolutionary Computation, 8(2):125–147, 2000.

    Article  Google Scholar 

  62. S. W. Wilson. Classifier System Learning of a Boolean Function. Technical Report RIS 27r, The Rowland Institute for Science, 1986.

    Google Scholar 

  63. S. W. Wilson. Classifier Fitness Based on Accuracy. Evolutionary Computation, 3(2):149–175, 1995.

    Google Scholar 

  64. S. W. Wilson. Generalization in the XCS Classifier System. In Genetic Programming: Proceedings of the Third Annual Conference, pages 665–674. San Francisco, CA: Morgan Kaufmann, 1998.

    Google Scholar 

  65. S. W. Wilson. Mining Oblique Data with XCS. In Advances in Learning Classifier Systems: Proceedings of the Third International Workshop, volume 1996 of LNAI, pages 158–176. Springer-Verlag Berlin Heidelberg, 2001.

    Google Scholar 

  66. S. W. Wilson. Compact Rulesets from XCSI. In Advances in Learning Classifier Systems, 4th International Workshop, volume 2321 of LNAI, pages 197–210. Springer, 2002.

    Google Scholar 

  67. I. H. Witten and F. Eibe. Data Mining. Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, 2003.

    Google Scholar 

  68. E. Zitzler. Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications. PhD thesis, Swiss Federal Institute of Technology (ETH) Zurich, 1999.

    Google Scholar 

  69. E. Zitzler. SPEA2: Improving the Strength Pareto Evolutionary Algorithm. Technical report 103, Swiss Federal Institute of Technology (ETH) Zurich, Gloriastrasse 35, CH-8092 Zurich, May, 2001.

    Google Scholar 

  70. E. Zitzler, K. Deb, and L. Thiele. Comparison of Multiobjective Evolutionary Algorithms: Empirical Results. Evolutionary Computation, 8(2):173–195, 2000.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Enginyeria i Arquitectura La Salle, Universitat Ramon Llull. Quatre Camins, Barcelona, 2.08022, Spain

    Ester Bernadó-Mansilla

  2. Illinois Genetic Algorithms Lab, University of Illinois at Urbana-Champaign, 104 S. Mathews Ave., Urbana, IL, 61801, USA

    Xavier Llorà

  3. Conducive Corp., 55 Broad Street, 3rd Floor, New York, NY, 10004, USA

    Ivan Traus

Authors
  1. Ester Bernadó-Mansilla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xavier Llorà
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ivan Traus
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Honda Research Institute Europe GmbH, Carl-Legien-Str. 30, Offenbach, 63073, Germany

    Yaochu Jin Dr.

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer

About this chapter

Cite this chapter

Bernadó-Mansilla, E., Llorà, X., Traus, I. (2006). Multi-objective Learning Classifier Systems. In: Jin, Y. (eds) Multi-Objective Machine Learning. Studies in Computational Intelligence, vol 16. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-33019-4_12

Download citation

  • DOI: https://doi.org/10.1007/3-540-33019-4_12

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-30676-4

  • Online ISBN: 978-3-540-33019-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation