Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
article

General schema theory for genetic programming with subtree-swapping crossover: Part II

Published: 01 May 2003 Publication History

Abstract

This paper is the second part of a two-part paper which introduces a general schema theory for genetic programming (GP) with subtree-swapping crossover (Part I (Poli and McPhee, 2003)). Like other recent GP schema theory results, the theory gives an exact formulation (rather than a lower bound) for the expected number of instances of a schema at the next generation. The theory is based on a Cartesian node reference system, introduced in Part I, and on the notion of a variable-arity hyperschema, introduced here, which generalises previous definitions of a schema. The theory indudes two main theorems describing the propagation of GP schemata: a microscopic and a macroscopic schema theorem. The microscopic version is applicable to crossover operators which replace a subtree in one parent with a subtree from the other parent to produce the offspring. Therefore, this theorem is applicable to Koza's GP crossover with and without uniform selection of the crossover points, as well as one-point crossover, size-fair crossover, strongly-typed GP crossover, context-preserving crossover and many others. The macroscopic version is applicable to crossover operators in which the probability of selecting any two crossover points in the parents depends only on the parents' size and shape. In the paper we provide examples, we show how the theory can be specialised to specific crossover operators and we illustrate how it can be used to derive other general results. These include an exact definition of effective fitness and a size-evolution equation for GP with subtree-swapping crossover.

References

[1]
Altenberg, L. (1994). Emergent phenomena in genetic programming. In Sebald, A. V. and Fogel, L. J., editors, Evolutionary Programming-- Proceedings of the Third Annual Conference, pages 233-241, San Diego, CA, USA. World Scientific Publishing.]]
[2]
Altenberg, L. (1995). The Schema Theorem and Price's Theorem. In Whitley, L. D. and Vose, M. D., editors, Foundations of Genetic Algorithms 3, pages 23-49, Estes Park, Colorado, USA. Morgan Kaufmann.]]
[3]
Blickle, T. and Thiele, L. (1994). Genetic programming and redundancy. In Hopf, J., editor, Genetic Algorithms within the Framework of Evolutionary Computation (Workshop at KI-94, Saarbrücken), pages 33-38, Im Stadtwald, Building 44, D-66123 Saarbrücken, Germany. Max-Planck-Institut für Informatik (MPI-I-94-241).]]
[4]
Chung, S. W. and Perez, R. A. (1994). The schema theorem considered insufficient. In Proceedings of the Sixth IEEE International Conference on Tools with Artificial Intelligence, pages 748-751, New Orleans.]]
[5]
Davis, T. E. and Principe, J. C. (1993). A Markov chain framework for the simple genetic algorithm. Evolutionary Computation, 1(3):269-288.]]
[6]
D'haeseleer, P. (1994). Context preserving crossover in genetic programming. In Proceedings of the 1994 IEEE World Congress on Computational Intelligence, volume 1, pages 256-261, Orlando, Florida, USA. IEEE Press.]]
[7]
Fogel, D. B. and Ghozeil, A. (1997). Schema processing under proportional selection in the presence of random effects. IEEE Transactions on Evolutionary Computation, 1 (4):290-293.]]
[8]
Fogel, D. B. and Ghozeil, A. (1998). The schema theorem and the misallocation of trials in the presence of stochastic effects. In Porto, V. W., Saravanan, N., Waagen, D., and Eiben, A. E., editors, Evolutionary Programming VII: Proc. of the 7th Ann. Conf. on Evolutionary Programming, pages 313-321, Berlin. Springer.]]
[9]
Geiringer, H. (1944). On the probability theory of linkage in Mendelian heredity. Annals of Mathematical Statistics, 15(1):25-57.]]
[10]
Goldberg, D. E. (1989a). Genetic algorithms and Walsh functions: II. Deception and its analysis. Complex Systems, 3(2):153-171.]]
[11]
Goldberg, D. E. (1989b). Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Reading, Massachusetts.]]
[12]
Holland, J. (1975). Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor, USA.]]
[13]
Koza, J. R. (1992). Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA, USA.]]
[14]
Langdon, W. B. (1999a). Scaling of program tree fitness spaces. Evolutionary Computation, 7(4):399-428.]]
[15]
Langdon, W. B. (1999b). Size fair and homologous tree genetic programming crossovers. In Banzhaf, W., Daida, J., Eiben, A. E., Garzon, M. H., Honavar, V., Jakiela, M., and Smith, R. E., editors, Proceedings of the Genetic and Evolutionary Computation Conference, volume 2, pages 1092-1097, Orlando, Florida, USA. Morgan Kaufmann.]]
[16]
Langdon, W. B. (2000a). Quadratic bloat in genetic programming. In Whitley, D., Goldberg, D., Cantu-Paz, E., Spector, L., Parmee, I., and Beyer, H.-G., editors, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000), pages 451-458, Las Vegas, Nevada, USA. Morgan Kaufmann.]]
[17]
Langdon, W. B. (2000b). Size fair and homologous tree genetic programming crossovers. Genetic Programming and Evolvable Machines, 1(1/2):95-119.]]
[18]
Langdon, W. B. and Banzhaf, W. (2000). Genetic programming bloat without semantics. In Schoenauer, M., Deb, K., Rudolph, G., Yao, X., Lutton, E., Merelo, J. J., and Schwefel, H.-P., editors, Parallel Problem Solving from Nature - PPSN VI 6th International Conference, volume 1917 of LNCS, pages 201-210, Paris, France. Springer Verlag.]]
[19]
Langdon, W. B. and Poli, R. (2002). Foundations of Genetic Programming. Springer-Verlag.]]
[20]
Langdon, W. B., Soule, T., Poli, R., and Foster, J. A. (1999). The evolution of size and shape. In Spector, L., Langdon, W. B., O'Reilly, U.-M., and Angeline, P. J., editors, Advances in Genetic Programming 3, chapter 8, pages 163-190. MIT Press, Cambridge, MA, USA.]]
[21]
McPhee, N. F. and Miller, J. D. (1995). Accurate replication in genetic programming. In Eshelman, L., editor, Genetic Algorithms: Proceedings of the Sixth International Conference (ICGA95), pages 303-309, Pittsburgh, PA, USA. Morgan Kaufmann.]]
[22]
McPhee, N. F. and Poli, R. (2001). A schema theory analysis of the evolution of size in genetic programming with linear representations. In Miller, J. F., Tomassini, M., Lanzi, P. L., Ryan, C., Tettamanzi, A. G. B., and Langdon, W. B., editors, Genetic Programming, Proceedings of EuroGP'2001, volume 2038 of LNCS, pages 108-125, Lake Como, Italy. Springer-Verlag.]]
[23]
McPhee, N. F. and Poli, R. (2002). Using schema theory to explore interactions of multiple operators. In Langdon, W. B., Cantú-Paz, E., Mathias, K., Roy, R., Davis, D., Poli, R., Balakrishnan, K., Honavar, V., Rudolph, G., Wegener, J., Bull, L., Potter, M. A., Schultz, A. C., Miller, J. F., Burke, E., and Jonoska, N., editors, GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, pages 853-860, New York. Morgan Kaufmann Publishers.]]
[24]
McPhee, N. F., Poli, R., and Rowe, J. E. (2001). A schema theory analysis of mutation size biases in genetic programming with linear representations. In Proceedings of the 2001 Congress on Evolutionary Computation CEC2001, pages 1078-1085, COEX, World Trade Center, 159 Samseong-dong, Gangnam-gu, Seoul, Korea. IEEE Press.]]
[25]
Montana, D. J. (1995). Strongly typed genetic programming. Evolutionary Computation, 3(2):199-230.]]
[26]
Nix, A. E. and Vose, M. D. (1992). Modeling genetic algorithms with Markov chains. Annals of Mathematics and Artificial Intelligence, 5:79-88.]]
[27]
Nordin, P. (1994). A compiling genetic programming system that directly manipulates the machine code. In Kinnear, Jr., K. E., editor, Advances in Genetic Programming, chapter 14, pages 311-331. MIT Press.]]
[28]
Nordin, P. and Banzhaf, W. (1995). Complexity compression and evolution. In Eshelman, L., editor, Genetic Algorithms: Proceedings of the Sixth International Conference (ICGA95), pages 310-317, Pittsburgh, PA, USA. Morgan Kaufmann.]]
[29]
Nordin, P., Francone, F., and Banzhaf, W. (1995). Explicitly defined introns and destructive crossover in genetic programming. In Rosca, J. P., editor, Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications, pages 6-22, Tahoe City, California, USA.]]
[30]
O'Neill, M. and Ryan, C. (2001). Grammatical evolution. IEEE Transaction on Evolutionary Compuation, 5(4):349-358.]]
[31]
O'Reilly, U.-M. and Oppacher, F. (1995). The troubling aspects of a building block hypothesis for genetic programming. In Whitley, L. D. and Vose, M. D., editors, Foundations of Genetic Algorithms 3, pages 73-88, Estes Park, Colorado, USA. Morgan Kaufmann.]]
[32]
Pohlheim, H. (1999). Visualization of evolutionary algorithms - set of standard techniques and multidimensional visualization. In Banzhaf, W., Daida, J., Eiben, A. E., Garzon, M. H., Honavar, V., Jakiela, M., and Smith, R. E., editors, Proceedings of the Genetic and Evolutionary Computation Conference, volume 1, pages 533-540, Orlando, Florida, USA. Morgan Kaufmann.]]
[33]
Poli, R. (1999). Schema theorems without expectations. In Banzhaf, W., Daida, J., Eiben, A. E., Garzon, M. H., Honavar, V., Jakiela, M., and Smith, R. E., editors, Proceedings of the Genetic and Evolutionary Computation Conference, volume 1, page 806, Orlando, Florida, USA. Morgan Kaufmann.]]
[34]
Poli, R. (2000a). Exact schema theorem and effective fitness for GP with one-point crossover. In Whitley, D., Goldberg, D., Cantu-Paz, E., Spector, L., Parmee, I., and Beyer, H.-G., editors, Proceedings of the Genetic and Evolutionary Computation Conference, pages 469-476, Las Vegas. Morgan Kaufmann.]]
[35]
Poli, R. (2000b). Hyperschema theory for GP with one-point crossover, building blocks, and some new results in GA theory. In Poli, R., Banzhaf, W., Langdon, W. B., Miller, J. F., Nordin, P., and Fogarty, T. C., editors, Genetic Programming, Proceedings of EuroGP'2000, volume 1802 of LNCS, pages 163-180, Edinburgh. Springer-Verlag.]]
[36]
Poli, R. (2000c). Recursive conditional schema theorem, convergence and population sizing in genetic algorithms. In Spears, W. M. and Martin, W., editors, Proceedings of the Foundations of Genetic Algorithms Workshop (FOGA 6), pages 143-163, Charlottesville, VA, USA.]]
[37]
Poll, R. (2000d). Why the schema theorem is correct also in the presence of stochastic effects. In Proceedings of the Congress on Evolutionary Computation (CEC 2000), pages 487-492, San Diego, USA.]]
[38]
Poli, R. (2001a). Exact schema theory for genetic programming and variable-length genetic algorithms with one-point crossover. Genetic Programming and Evolvable Machines, 2(2):123-163.]]
[39]
Poli, R. (2001b). General schema theory for genetic programming with subtree-swapping crossover. In Miller, J. E., Tomassini, M., Lanzi, P. L., Ryan, C., Tettamanzi, A. G. B., and Langdon, W. B., editors, Genetic Programming, Proceedings of EuroGP'2001, volume 2038 of LNCS, pages 143-159, Lake Como, Italy. Springer-Verlag.]]
[40]
Poli, R. (2003). A simple but theoretically-motivated method to control bloat in genetic programming. In Ryan, C., Soule, T., Keijzer, M., Tsang, E., Poli, R., and Costa, E., editors, Genetic Programming, Proceedings of the 6th European Conference, EuroGP 2003, LNCS, pages 211-223, Essex, UK. Springer-Verlag.]]
[41]
Poli, R. and Langdon, W. B. (1997a). Genetic programming with one-point crossover. In Chawdhry, P. K., Roy, R., and Pant, R. K., editors, Soft Computing in Engineering Design and Manufacturing, pages 180-189. Springer-Verlag London.]]
[42]
Poli, R. and Langdon, W. B. (1997b). A new schema theory for genetic programming with one-point crossover and point mutation. In Koza, J. R., Deb, K., Dorigo, M., Fogel, D. B., Garzon, M., Iba, H., and Riolo, R. L., editors, Genetic Programming 1997: Proceedings of the Second Annual Conference, pages 278-285, Stanford University, CA, USA. Morgan Kaufmann.]]
[43]
Poli, R. and Langdon, W. B. (1998). Schema theory for genetic programming with one-point crossover and point mutation. Evolutionary Computation, 6(3):231-252]]
[44]
Poli, R., Langdon, W. B., and O'Reilly, U,-M. (1998). Analysis of schema variance and short term extinction likelihoods. In Koza, J. R., Banzhaf, W., Chellapilla, K., Deb, K., Dorigo, M., Fogel, D. B., Garzon, M. H., Goldberg, D. E., Iba, H, and Riolo, R., editors, Genetic Programming 1998: Proceedings of the Third Annual Conference, pages 284-292, University of Wisconsin, Madison, Wisconsin, USA. Morgan Kaufmann.]]
[45]
Poli, R. and McPhee, N. F. (2001a). Exact GP schema theory for headless chicken crossover and subtree mutation. In Proceedings of the 2001 Congress on Evolutionary Computation CEC2001, pages 1062-1069, COEX, World Trade Center, 159 Samseong-dong, Gangnam-gu, Seoul, Korea. IEEE Press.]]
[46]
Poli, R. and McPhee, N. F. (2001b). Exact schema theorems for GP with one-point and standard crossover operating on linear structures and their application to the study of the evolution of size. In Miller, J. F., Tomassini, M., Lanzi, P. L., Ryan, C., Tettamanzi, A. G. B., and Langdon, W. B., editors, Genetic Programming, Proceedings of EuroGP'2001, volume 2038 of LNCS, pages 126-142, Lake Como, Italy. Springer-Verlag.]]
[47]
Poli, R. and McPhee, N. F. (2001c). Exact schema theory for GP and variable-length GAs with homologous crossover. In Spector, L., Goodman, E. D., Wu, A., Langdon, W. B., Voigt, H.-M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M. H., and Burke, E., editors, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pages 104-111, San Francisco, California, USA. Morgan Kaufmann.]]
[48]
Poli, R. and McPhee, N. F. (2003). General schema theory for genetic programming with subtree-swapping crossover: Part I. Evolutionary Computation, 11(1):53-66.]]
[49]
Poli, R., Rowe, J. E., and McPhee, N. F. (2001). Markov chain models for GP and variable-length GAs with homologous crossover. In Spector, L., Goodman, E. D., Wu, A., Langdon, W. B., Voigt, H.-M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M. H., and Burke, E., editors, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pages 112-119, San Francisco, California, USA. Morgan Kaufmann.]]
[50]
Poli, R., Rowe, J. E., Stephens, C. R., and Wright, A. H. (2002). Allele diffusion in linear genetic programming and variable-length genetic algorithms with subtree crossover. In Foster, J. A., Lutton, E., Miller, J., Ryan, C., and Tettamanzi, A. G. B., editors, Genetic Programming, Proceedings of the 5th European Conference, EuroGP 2002, volume 2278 of LNCS, pages 212-227, Kinsale, Ireland: Springer-Verlag.]]
[51]
Prügel-Bennett, A. and Shapiro, J. L. (1994). An analysis of genetic algorithms using statistical mechanics. Physical Review Letters, 72:1305-1309.]]
[52]
Radcliffe, N. J. (1997). Schema processing. In Baeck, T., Fogel, D. B., and Michalewicz, Z., editors, Handbook of Evolutionary Computation, pages B2.5-1-10. Oxford University Press.]]
[53]
Rosca, J. P. (1997). Analysis of complexity drift in genetic programming. In Koza, J. R., Deb, K., Dorigo, M., Fogel, D. B., Garzon, M., Iba, H., and Riolo, R. L., editors, Genetic Programming 1997: Proceedings of the Second Annual Conference, pages 286-294, Stanford University, CA, USA. Morgan Kaufmann.]]
[54]
Soule, T. (1998). Code Growth in Genetic Programming. PhD thesis, University of Idaho, Moscow, Idaho, USA.]]
[55]
Stephens, C. R. and Waelbroeck, H. (1997). Effective degrees of freedom in genetic algorithms and the block hypothesis. In Bäck, T., editor, Proceedings of the Seventh International Conference on Genetic Algorithms (ICGA97), pages 34-40, East Lansing. Morgan Kaufmann.]]
[56]
Stephens, C. R. and Waelbroeck, H. (1999). Schemata evolution and building blocks. Evolutionary Computation, 7(2):109-124.]]
[57]
Vose, M. D. (1999). The simple genetic algorithm: Foundations and theory. MIT Press, Cambridge, MA.]]
[58]
Whigham, P. A. (1995). A schema theorem for context-free grammars. In 1995 IEEE Conference on Evolutionary Computation, volume 1, pages 178-181, Perth, Australia. IEEE Press.]]
[59]
Whigham, P. A. (1996). Search bias, language bias, and genetic programming. In Koza, J. R., Goldberg, D. E., Fogel, D. B., and Riolo, R. L., editors, Genetic Programming 1996: Proceedings of the First Annual Conference, pages 230-237, Stanford University, CA, USA. MIT Press.]]
[60]
Whitley, L. D. (1994). A Genetic Algorithm Tutorial. Statistics and Computing, 4:65-85.]]
[61]
Zhang, B.-T. and Mühlenbein, H. (1995). Balancing accuracy and parsimony in genetic programming. Evolutionary Computation, 3(1):17-38.]]

Cited By

View all
  • (2024)Survey on Genetic Programming and Machine Learning Techniques for Heuristic Design in Job Shop SchedulingIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.325524628:1(147-167)Online publication date: 1-Feb-2024
  • (2024)Population diversity and inheritance in genetic programming for symbolic regressionNatural Computing: an international journal10.1007/s11047-022-09934-x23:3(531-566)Online publication date: 1-Sep-2024
  • (2023)A Double Lexicase Selection Operator for Bloat Control in Evolutionary Feature Construction for RegressionProceedings of the Genetic and Evolutionary Computation Conference10.1145/3583131.3590365(1194-1202)Online publication date: 15-Jul-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Evolutionary Computation
Evolutionary Computation  Volume 11, Issue 2
Summer 2003
94 pages
ISSN:1063-6560
EISSN:1530-9304
Issue’s Table of Contents

Publisher

MIT Press

Cambridge, MA, United States

Publication History

Published: 01 May 2003
Published in EVOL Volume 11, Issue 2

Author Tags

  1. genetic programming
  2. schema theory
  3. standard crossover

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 09 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Survey on Genetic Programming and Machine Learning Techniques for Heuristic Design in Job Shop SchedulingIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.325524628:1(147-167)Online publication date: 1-Feb-2024
  • (2024)Population diversity and inheritance in genetic programming for symbolic regressionNatural Computing: an international journal10.1007/s11047-022-09934-x23:3(531-566)Online publication date: 1-Sep-2024
  • (2023)A Double Lexicase Selection Operator for Bloat Control in Evolutionary Feature Construction for RegressionProceedings of the Genetic and Evolutionary Computation Conference10.1145/3583131.3590365(1194-1202)Online publication date: 15-Jul-2023
  • (2022)A Novel Fitness Function for Genetic Programming in Dynamic Flexible Job Shop Scheduling2022 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC55065.2022.9870235(1-8)Online publication date: 18-Jul-2022
  • (2022)On sampling error in genetic programmingNatural Computing: an international journal10.1007/s11047-020-09828-w21:2(173-186)Online publication date: 1-Jun-2022
  • (2022)Genetic programming convergenceGenetic Programming and Evolvable Machines10.1007/s10710-021-09405-923:1(71-104)Online publication date: 1-Mar-2022
  • (2021)Correlation Coefficient-Based Recombinative Guidance for Genetic Programming Hyperheuristics in Dynamic Flexible Job Shop SchedulingIEEE Transactions on Evolutionary Computation10.1109/TEVC.2021.305614325:3(552-566)Online publication date: 1-Jun-2021
  • (2018)Schema Theory-Based Data Engineering in Gene Expression Programming for Big Data AnalyticsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2017.277144522:5(792-804)Online publication date: 1-Oct-2018
  • (2018)An improved semantic schema modeling for genetic programmingSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-017-2781-622:10(3237-3260)Online publication date: 1-May-2018
  • (2017)Statistical genetic programming for symbolic regressionApplied Soft Computing10.1016/j.asoc.2017.06.05060:C(447-469)Online publication date: 1-Nov-2017
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media