article

Multi-Objective Methods for Tree Size Control

Authors:

Edwin D. De Jong,

Jordan B. PollackAuthors Info & Claims

Genetic Programming and Evolvable Machines, Volume 4, Issue 3

Pages 211 - 233

https://doi.org/10.1023/A:1025122906870

Published: 01 September 2003 Publication History

Abstract

Variable length methods for evolutionary computation can lead to a progressive and mainly unnecessary growth of individuals, known as bloat. First, we propose to measure performance in genetic programming as a function of the number of nodes, rather than trees, that have been evaluated. Evolutionary Multi-Objective Optimization (EMOO) constitutes a principled way to optimize both size and fitness and may provide parameterless size control. Reportedly, its use can also lead to minimization of size at the expense of fitness. We replicate this problem, and an empirical analysis suggests that multi-objective size control particularly requires diversity maintenance. Experiments support this explanation.

The multi-objective approach is compared to genetic programming without size control on the 11-multiplexer, 6-parity, and a symbolic regression problem. On all three test problems, the method greatly reduces bloat and significantly improves fitness as a function of computational expense. Using the FOCUS algorithm, multi-objective size control is combined with active pursuit of diversity, and hypothesized minimum-size solutions to 3-, 4- and 5-parity are found. The solutions thus found are furthermore easily interpretable. When combined with diversity maintenance, EMOO can provide an adequate and parameterless approach to size control in variable length evolution.

References

[1]

1. P. J. Angeline, "Genetic programming and emergent intelligence," in Advances in Genetic Programming, K. E. Kinnear Jr. (ed.), MIT Press: Cambridge, MA, 1994, pp. 75-98.]]

Digital Library

[2]

2. P. J. Angeline and J. B. Pollack, "The evolutionary induction of subroutines," in Proceedings of the Fourteenth Annual Conference of the Cognitive Science Society, Lawrence Erlbaum: Bloomington, Indiana, USA, 1992, pp. 236-241.]]

[3]

3. W. Banzhaf, D. Banscherus and P. Dittrich, "Hierarchical genetic programming using local modules," Interjournal Complex Systems (228), 1998, URL: http://www.interjournal.org.]]

[4]

4. W. Banzhaf and W. B. Langdon, "Some considerations on the reason for bloat," Genetic Programming and Evolvable Machines, vol. 3, no. 1, pp. 81-91, 2002.]]

Digital Library

[5]

5. W. Banzhaf, P. Nordin, R. E. Keller, and F. D. Francone, Genetic Programming--An Introduction: On the Automatic Evolution of Computer Programs and its Applications, dpunkt--Verlag für digitale Technologie GbmH and Morgan Kaufmann Publishers Inc.: Heidelberg and San Francisco, CA, 1998.]]

Digital Library

[6]

6. T. Blickle and L. Thiele, "Genetic programming and redundancy," in Genetic Algorithms within the Framework of Evolutionary Computation. Workshop at KI-94, J. Hopf (ed.), Max-Planck-Institut für Informatik (MPI-I-94-241): Saarbrücken, Germany, 1994, pp. 33-38.]]

[7]

7. A. Brindle, Genetic Algorithms for Function Optimization. Unpublished doctoral dissertation, (Computer Science Department, Technical Report TR81-2), University of Alberta, Canada.]]

[8]

8. C. A. C. Coello, "An updated survey of GA-Based multiobjective optimization techniques," ACM Computing Surveys, vol. 32, no. 2, pp. 109-143, 2000.]]

Digital Library

[9]

9. W. Conover, Practical Nonparametric Statistics, Wiley: New York, NY, 1980.]]

[10]

10. D. W. Corne, N. R. Jerram, J. D. Knowles, and M. J. Oates, "PESA-II: Region-based selection in evolutionary multiobjective optimization," in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO-2001, L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke (eds.), Morgan Kaufmann: San Francisco, CA, 2001, pp. 283-290.]]

[11]

11. N. L. Cramer, "A representation for the adaptive generation of simple sequential programs," in Proceedings of an International Conference on Genetic Algorithms and their Applications, J. J. Grefenstette (ed.), Pittsburgh, PA, 1985, pp. 183-187.]]

Digital Library

[12]

12. E. D. De Jong, R. A. Watson, and J. B. Pollack, "Reducing bloat and promoting diversity using multi-objective methods," in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO-2001, L. Spector, E. Goodman, A. Wu, W. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. Garzon, and E. Burke (eds.), Morgan Kaufmann Publishers: San Francisco, CA, 2001, pp. 11-18.]]

[13]

13. K. Deb, Multi-objective Optimization using Evolutionary Algorithms, Wiley: Chichester, UK, 2001.]]

Digital Library

[14]

14. K. Deb, S. Agrawal, A. Pratab, and T. Meyarivan, "A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II," in Parallel Problem Solving from Nature-- PPSN VI (Vol. 1917), M. Schoenauer, K. Deb, G. Rudolph, X. Yao, E. Lutton, J. J. Merelo, and H.-P. Schwefel (eds.), Springer Verlag, 2000, pp. 849-858.]]

Digital Library

[15]

15. A. Ekárt and S. Németh, "Selection based on the Pareto nondomination criterion for controlling code growth in genetic programming," Genetic Programming and Evolvable Machines, vol. 2, pp. 61-73, 2001.]]

Digital Library

[16]

16. C. M. Fonseca and P. J. Fleming, "Genetic algorithms for multiobjective optimization: formulation, discussion and generalization," in Proceedings of the Fifth International Conference on Genetic Algorithms, ICGA-93, S. Forrest (ed.), Morgan Kauffman Publishers: San Mateo, CA, 1993, pp. 416-423.]]

Digital Library

[17]

17. C. M. Fonseca and P. J. Fleming, "An overview of evolutionary algorithms in multiobjective optimization," Evolutionary Computation, vol. 3, no. 1, pp. 1-16, 1995.]]

Digital Library

[18]

18. M. P. Fourman, "Compaction of symbolic layout using genetic algorithms," in Proceedings of the First International Conference on Genetic Algorithms and their Applications, J. J. Grefenstette (ed.), Lawrence Erlbaum Associates: Hillsdale, New Jersey, 1985, pp. 141-153.]]

Digital Library

[19]

19. M. Keijzer, Scientific Discovery using Genetic Programming, Unpublished doctoral dissertation, Danish Technical University, Lyngby.]]

[20]

20. K. E. Kinnear, Jr, "Generality and difficulty in genetic programming: Evolving a sort," in Proceedings of the 5th International Conference on Genetic Algorithms, ICGA-93, S. Forrest (ed.), Morgan Kaufmann: San Mateo, CA, 1993, pp. 287-294.]]

Digital Library

[21]

21. J. R. Koza, Genetic Programming, MIT Press: Cambridge, MA, 1992.]]

[22]

22. J. R. Koza, Genetic Programming II: Automatic Discovery of Reusable Programs, MIT Press: Cambridge, MA, 1994.]]

Digital Library

[23]

23. W. B. Langdon, "Data structures and genetic programming," in Advances in Genetic Programming 2, P. J. Angeline and K. Kinnear (eds.), MIT Press: Cambridge, MA, 1996, pp. 395-414.]]

Digital Library

[24]

24. W. B. Langdon, "The evolution of size in variable length representations," in 1998 IEEE International Conference on Evolutionary Computation, IEEE Press: Anchorage, Alaska, USA, 1998, pp. 633-638.]]

[25]

25. W. B. Langdon and J. P. Nordin, "Seeding GP populations," in Genetic Programming, Proceedings of EuroGP-2000 (Vol. 1802), R. Poli, W. Banzhaf, W. B. Langdon, J. F. Miller, P. Nordin, and T. C. Fogarty (eds.), Springer-Verlag: Edinburgh, 2000, pp. 304-315.]]

Digital Library

[26]

26. W. B. Langdon and R. Poli, "Fitness causes bloat: Mutation," in Proceedings of the First European Workshop on Genetic Programming (Vol. 1391), W. Banzhaf, R. Poli, M. Schoenauer, and T. C. Fogarty (eds.), Springer-Verlag: Paris, 1998, pp. 37-48.]]

Digital Library

[27]

27. W. B. Langdon and R. Poli, Foundations of Genetic Programming, Springer-Verlag, 2002.]]

Digital Library

[28]

28. W. B. Langdon, T. Soule, R. Poli, and J. A. Foster, "The evolution of size and shape," in Advances in Genetic Programming 3, L. Spector, W. B. Langdon, U.-M. O'Reilly, P. J. Angeline (eds.), MIT Press: Cambridge, MA, 1999, pp. 163-190.]]

Digital Library

[29]

29. M. Laumanns, L. Thiele, K. Deb, and E. Zitzler, Combining convergence and diversity in evolutionary multi-objective optimization, Evolutionary Computation, vol. 10, no. 3, pp. 263-282, 2002.]]

Digital Library

[30]

30. S. Luke and L. Panait, "Fighting bloat with nonparametric parsimony pressure," in Parallel problem solving from nature--PPSN VII. (p. 411 ff.), H.-P. Schwefel, J.-J. Merelo Guervós, P. Adamidis, H.-G. Beyer, and J.-L. Fernández-Villacañas (eds.), Springer-Verlag: 2002.]]

Digital Library

[31]

31. S. W. Mahfoud, Niching Methods for Genetic Algorithms, Unpublished doctoral dissertation, Urbana, IL, (IlliGAL Report 95001).]]

Digital Library

[32]

32. N. F. McPhee and J. D. Miller, "Accurate replication in genetic programming," in Genetic Algorithms: Proceedings of the Sixth International Conference, ICGA-95, L. Eshelman (ed.), Morgan Kaufmann: Pittsburgh, PA, 1995, pp. 303-309.]]

Digital Library

[33]

33. P. Nordin and W. Banzhaf, "Complexity compression and evolution," in Genetic Algorithms: Proceedings of the Sixth International Conference, ICGA-95, L. Eshelman (ed.), Morgan Kaufmann: Pittsburgh, PA, 1995, pp. 310-317.]]

Digital Library

[34]

34. P. Nordin, F. Francone and W. Banzhaf, "Explicitly defined introns and destructive crossover in genetic programming," in Advances in Genetic Programming 2, P. J. Angeline and K. E. Kinnear Jr. (eds.), MIT Press: Cambridge, MA, 1996, pp. 111-134.]]

Digital Library

[35]

35. R. Olsson, "Inductive functional programming using incremental program transformation," Artificial Intelligence, vol. 74, no. 1, pp. 55-81, 1995.]]

Digital Library

[36]

36. R. Poli, "Discovery of symbolic, neuro-symbolic and neural networks with parallel distributed genetic programming. In 3rd International Conference on Artifical Neural Networks and Genetic Algorithms," in ICANNGA'97, Springer: Berlin, 1997.]]

[37]

37. R. Poli and N. F. McPhee, "Exact schema theorems for gp with one-point and standard crossover operating on linear structures and their application to the study of the evolution of size," in Genetic Programming. 4th European Conference, EuroGP 2001, J. Miller, M. Tomassini, P. Lanzi, C. Ryan, A. Tettamanzi, and W. Langdon (eds.), Springer: Berlin, 2001, pp. 126-142.]]

Digital Library

[38]

38. K. Rodríguez-Vázquez, C. M. Fonseca and P. J. Fleming, "Multi-objective genetic programming: A nonlinear system identification application," in Late Breaking Papers at the 1997 Genetic Programming Conference, J. R. Koza (ed.), Stanford Bookstore: Stanford University, CA, 1997, pp. 207-212.]]

[39]

39. J. Rosca, "Generality versus size in genetic programming," in Genetic Programming 1996: Proceedings of the First Annual Conference, J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo (eds.), MIT Press: Cambridge, MA, 1996, pp. 381-387.]]

[40]

40. J. D. Schaffer, "Multiple objective optimization with vector evaluated genetic algorithms," in Proceedings of the First International Conference on Genetic Algorithms and their Applications, J. J. Grefenstette (ed.), Lawrence Erlbaum Associates: Hillsdale, New Jersey, 1985, pp. 93-100.]]

Digital Library

[41]

41. S. Smith, A Learning System Based on Genetic Adaptive Algorithms, Unpublished doctoral dissertation, University of Pittsburgh.]]

Digital Library

[42]

42. T. Soule, Code Growth in Genetic Programming, Unpublished doctoral dissertation, University of Idaho.]]

Digital Library

[43]

43. T. Soule and J. A. Foster, "Effects of code growth and parsimony presure on populations in genetic programming," Evolutionary Computation, vol. 6, no. 4, pp. 293-309, 1999.]]

Digital Library

[44]

44. T. Soule and R. B. Heckendorn, "An analysis of the causes of code growth in genetic programming," Genetic Programming and Evolvable Machines, vol. 3, pp. 283-309, 2002.]]

Digital Library

[45]

45. B.-T. Zhang and H. Mühlenbein, "Balancing accuracy and parsimony in genetic programming," Evolutionary Computation, vol. 3, no. 1, pp. 17-38, 1995.]]

Digital Library

[46]

46. E. Zitzler and L. Thiele, "Multiobjective evolutionary algorithms: A comparative case study and the strength pareto approach," IEEE Transactions on Evolutionary Computation, vol. 3, no. 4, pp. 257-271, 1999.]]

Digital Library

Cited By

Zhang HChen QXue BBanzhaf WZhang M(2024)A Semantic-Based Hoist Mutation Operator for Evolutionary Feature Construction in RegressionIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.333123428:6(1689-1703)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TEVC.2023.3331234
Zhang HChen QXue BBanzhaf WZhang M(2024)Modular Multitree Genetic Programming for Evolutionary Feature Construction for RegressionIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.331863828:5(1455-1469)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TEVC.2023.3318638
Langdon W(2023)Jaws 30Genetic Programming and Evolvable Machines10.1007/s10710-023-09467-x24:2Online publication date: 22-Nov-2023
https://dl.acm.org/doi/10.1007/s10710-023-09467-x
Show More Cited By

Recommendations

Reducing bloat and promoting diversity using multi-objective methods
GECCO'01: Proceedings of the 3rd Annual Conference on Genetic and Evolutionary Computation

Two important problems in genetic programming (GP) are its tendency to find unnecessarily large trees (bloat), and the general evolutionary algorithms problem that diversity in the population can be lost prematurely. The prevention of these problems is ...
Genetic diversity as an objective in multi-objective evolutionary algorithms

A key feature of an efficient and reliable multi-objective evolutionary algorithm is the ability to maintain genetic diversity within a population of solutions. In this paper, we present a new diversity-preserving mechanism, the Genetic Diversity ...
Evolutionary Algorithms for Multi-Objective Optimization: Performance Assessments and Comparisons

Evolutionary techniques for multi-objective (MO) optimization are currently gaining significant attention from researchers in various fields due to their effectiveness and robustness in searching for a set of trade-off solutions. Unlike conventional methods ...

Comments

Information & Contributors

Information

Published In

cover image Genetic Programming and Evolvable Machines

Genetic Programming and Evolvable Machines Volume 4, Issue 3

September 2003

81 pages

ISSN:1389-2576

Issue’s Table of Contents

Copyright © Copyright © 2003 Kluwer Academic Publishers.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 September 2003

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

30
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang HChen QXue BBanzhaf WZhang M(2024)A Semantic-Based Hoist Mutation Operator for Evolutionary Feature Construction in RegressionIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.333123428:6(1689-1703)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TEVC.2023.3331234
Zhang HChen QXue BBanzhaf WZhang M(2024)Modular Multitree Genetic Programming for Evolutionary Feature Construction for RegressionIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.331863828:5(1455-1469)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TEVC.2023.3318638
Langdon W(2023)Jaws 30Genetic Programming and Evolvable Machines10.1007/s10710-023-09467-x24:2Online publication date: 22-Nov-2023
https://dl.acm.org/doi/10.1007/s10710-023-09467-x
de França F(2023)Alleviating overfitting in transformation-interaction-rational symbolic regression with multi-objective optimizationGenetic Programming and Evolvable Machines10.1007/s10710-023-09461-324:2Online publication date: 20-Oct-2023
https://dl.acm.org/doi/10.1007/s10710-023-09461-3
Liu DVirgolin MAlderliesten TBosman PWagner M(2022)Evolvability degeneration in multi-objective genetic programming for symbolic regressionProceedings of the Genetic and Evolutionary Computation Conference10.1145/3512290.3528787(973-981)Online publication date: 8-Jul-2022
https://dl.acm.org/doi/10.1145/3512290.3528787
Mei YChen QLensen AXue BZhang M(2022)Explainable Artificial Intelligence by Genetic Programming: A SurveyIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.322550927:3(621-641)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.1109/TEVC.2022.3225509
Nieto-Fuentes RSegura C(2022)GP-DMD: a genetic programming variant with dynamic management of diversityGenetic Programming and Evolvable Machines10.1007/s10710-021-09426-423:2(279-304)Online publication date: 21-Jan-2022
https://dl.acm.org/doi/10.1007/s10710-021-09426-4
Liang JXue Y(2022)Bloat-aware GP-based methods with bloat quantificationApplied Intelligence10.1007/s10489-021-02245-152:4(4211-4225)Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1007/s10489-021-02245-1
Wang SMei YZhang M(2020)A Multi-Objective Genetic Programming Hyper-Heuristic Approach to Uncertain Capacitated Arc Routing Problems2020 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC48606.2020.9185890(1-8)Online publication date: 19-Jul-2020
https://dl.acm.org/doi/10.1109/CEC48606.2020.9185890
Amir Haeri MEbadzadeh MFolino G(2017)Statistical genetic programming for symbolic regressionApplied Soft Computing10.1016/j.asoc.2017.06.05060:C(447-469)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1016/j.asoc.2017.06.050
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents