research-article

Compositional pattern producing networks: A novel abstraction of development

Author:

Kenneth O. StanleyAuthors Info & Claims

Genetic Programming and Evolvable Machines, Volume 8, Issue 2

Pages 131 - 162

https://doi.org/10.1007/s10710-007-9028-8

Published: 01 June 2007 Publication History

Abstract

Natural DNA can encode complexity on an enormous scale. Researchers are attempting to achieve the same representational efficiency in computers by implementing developmental encodings, i.e. encodings that map the genotype to the phenotype through a process of growth from a small starting point to a mature form. A major challenge in in this effort is to find the right level of abstraction of biological development to capture its essential properties without introducing unnecessary inefficiencies. In this paper, a novel abstraction of natural development, called Compositional Pattern Producing Networks (CPPNs), is proposed. Unlike currently accepted abstractions such as iterative rewrite systems and cellular growth simulations, CPPNs map to the phenotype without local interaction, that is, each individual component of the phenotype is determined independently of every other component. Results produced with CPPNs through interactive evolution of two-dimensional images show that such an encoding can nevertheless produce structural motifs often attributed to more conventional developmental abstractions, suggesting that local interaction may not be essential to the desirable properties of natural encoding in the way that is usually assumed.

References

[1]

Dellaert, F.: Toward a biologically defensible model of development. Master’s thesis, Case Western Reserve University, Clevekand, OH (1995)

[2]

Kandel E.R., Schwartz J.H., and Jessell T.M. Principles of Neural Science 1991 3 Amsterdam Elsevier

[3]

Zigmond M.J., Bloom F.E., Landis S.C., Roberts J.L., and Squire L.R. Fundamental Neuroscience 1999 London Academic Press

[4]

Deloukas P., Schuler G.D., Gyapay G., Beasley E.M., et al. A physical map of 30,000 human genes Science 1998 282 744-746

[5]

Angeline P.J. McDonnell J.R., Reynolds R.G., and Fogel D.B. Morphogenic evolutionary computations: Introduction, issues and examples Evolutionary Programming IV: The Fourth Annual Conference on Evolutionary Programming 1995 Cambridge MIT Press 387-401

[6]

Bentley, P.J., Kumar, S.: The ways to grow designs: A comparison of embryogenies for an evolutionary design problem. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-1999), pp. 35–43. Kaufmann, San Francisco (1999)

[7]

Hornby G.S. and Pollack J.B. Creating high-level components with a generative representation for body-brain evolution Artif. Life 2002 8 223-246

[8]

Stanley K.O. and Miikkulainen R. A taxonomy for artificial embryogeny Artif. Life 2003 9 93-130

[9]

Stanley K.O., Bryant B.D., and Miikkulainen R. Real-time neuroevolution in the NERO video game IEEE T. Evolut. Comput. Special Issue Evolut. Comput. Games 2005 9 653-668

[10]

Stanley K.O. and Miikkulainen R. Evolving neural networks through augmenting topologies Evolutionary Comput. 2002 10 99-127

[11]

Stanley K.O. and Miikkulainen R. Competitive coevolution through evolutionary complexification J. Artif. Intell. Res. 2004 21 63-100

[12]

Astor J.S. and Adami C. A developmental model for the evolution of artificial neural networks Artif. Life 2000 6 189-218

[13]

Boers, E.J.W., Kuiper, H.: Biological metaphors and the design of modular artificial neural networks. Master’s thesis, Departments of Computer Science and Experimental and Theoretical Psychology at Leiden University, The Netherlands (1992)

[14]

Bongard, J.C.: Evolving modular genetic regulatory networks. In: Proceedings of the 2002 Congress on Evolutionary Computation (2002)

[15]

Bongard, J.C., Pfeifer, R.: Repeated structure and dissociation of genotypic and phenotypic complexity in artificial ontogeny. In: Spector, L. et al. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference. Kaufmann, San Francisco, pp. 829–836 (2001)

[16]

Dellaert F. and Beer R.D. Maes P., Mataric M.J., Meyer J.-A., Pollack J., and Wilson S.W. A developmental model for the evolution of complete autonomous agents From Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior 1996 Cambridge, MA MIT Press

[17]

Eggenberger P. Husbands P. and Harvey I. Evolving morphologies of simulated 3D organisms based on differential gene expression Proceedings of the Fourth European Conference on Artificial Life 1997 Cambridge, MA MIT Press 205-213

[18]

Federici D. Schaal S., Ijspeert A.J., Billard A., Vijayakumar S., Hallam J., and Jean-Arcady Evolving a neurocontroller through a process of embryogeny Proceedings of the Eighth International Conference on Simulation and Adaptive Behavior (SAB-2004) 2004 Cambridge, MA MIT Press 373-384

[19]

Federici, D.: Using embryonic stages to increase the evolvability of development. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004) Workshop Program. Springer-Verlag, Berlin (2004b)

[20]

Fleischer K. and Barr A.H. Langton C.G. A simulation testbed for the study of multicellular development: The multiple mechanisms of morphogenesis Artificial Life III 1993 Reading Addison-Wesley 389-416

[21]

Gruau F., Whitley D., and Pyeatt L. Koza J.R., Goldberg D.E., Fogel D.B., and Riolo R.L. A comparison between cellular encoding and direct encoding for genetic neural networks Genetic Programming 1996: Proceedings of the First Annual Conference 1996 Cambridge, MA MIT Press 81-89

[22]

Hornby, G.S., Pollack, J.B.: The advantages of generative grammatical encodings for physical design. In: Proceedings of the 2002 Congress on Evolutionary Computation (2001a)

[23]

Hornby, G.S., Pollack, J.B.: Body-brain co-evolution using L-systems as a generative encoding. In: Spector, L. et al. (ed.) Proceedings of the Genetic and Evolutionary Computation Conference. Kaufmann, San Francisco (2001b)

[24]

Jakobi, N.: Harnessing morphogenesis. In: Proceedings of Information Processing in Cells and Tissues, pp. 29–41. University of Liverpool (1995)

[25]

Kaneko K. and Furusawa C. Emergence of multicellular organisms with dynamic differentiation and spatial pattern Artif. Life 1998 4 79-93

[26]

Kauffman S.A. The Origins of Order 1993 New York Oxford University Press

[27]

Kitano H. Designing neural networks using genetic algorithms with graph generation system Complex Syst. 1990 4 461-476

[28]

Komosinski M. and Rotaru-Varga A. Comparison of different genotype encodings for simulated 3D agents Artif. Life 2001 7 395-418

[29]

Lindenmayer A. Mathematical models for cellular interaction in development parts I and II J. Theor. Biol. 1968 18 280-299 300–315

[30]

Lindenmayer, A.: Adding continuous components to L-systems. In: Rozenberg, G., Salomaa, A. (eds.) L Systems, Lecture Notes in Computer Science 15, pp. 53–68. Springer-Verlag, Heidelberg, Germany (1974)

[31]

Miller, J.F.: Evolving a self-repairing, self-regulating, French flag organism. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004). Springer Verlag, Berlin (2004)

[32]

Mjolsness E., Sharp D.H., and Reinitz J. A connectionist model of development J. Theor. Biol. 1991 152 429-453

[33]

Nolfi S. and Parisi D. Growing neural networks Technical Report PCIA-91- 15 1991 Rome Institute of Psychology C.N.R.

[34]

Prusinkiewicz P. and Lindenmayer A. The Algorithmic Beauty of Plants 1990 Heidelberg, Germany Springer-Verlag

[35]

Schnier, T.: Evolved representations and their use in computational creativity. PhD thesis, University of Sydney Department of Architectural and Design Science, Australia (1998)

[36]

Sims, K.: Evolving 3D morphology and behavior by competition. In: Brooks, R.A., Maes, P. (eds.) Proceedings of the Fourth International Workshop on the Synthesis and Simulation of Living Systems (Artificial Life IV), pp. 28–39. MIT Press, Cambridge, MA (1994)

[37]

Turing A. The chemical basis of morphogenesis Phil. T. Roy. Soc. B 1952 237 37-72

[38]

Gilbert C.D. and Wiesel T.N. Receptive field dynamics in adult primary visual cortex Nature 1992 356 150-152

[39]

Hubel D.H. and Wiesel T.N. Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat J. Neurophysiol. 1965 28 229-289

[40]

Behravan, R., Bentley, P.J.: Exploring reaction-diffusion and pattern formation. In: Proceedings of the First Australian Conference on Artificial Life (ACAL 2003) (2003)

[41]

Cangelosi, A., Parisi, D., Nolfi, S.: Cell division and migration in a genotype for neural networks. Technical Report PCIA-93, Institute of Psychology, C.N.R. Rome (1993)

[42]

Belew, R.K., Kammeyer, T.E.: In: Forrest, S. (ed.) Proceedings of the Fifth International Conference on Genetic Algorithms. Kaufmann, San Francisco (1993)

[43]

Stanley, K.O., Reisinger, J., Miikkulainen, R.: Exploiting morphological conventions for genetic reuse. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004) Workshop Program. Springer Verlag, Berlin (2004)

[44]

Raff R.A. The Shape of Life: Genes, Development, and the Evolution of Animal Form 1996 Chicago The University of Chicago Press

[45]

Amores A., Force A., Yan Y.-L., Joly L., Amemiya C., Fritz A., Ho R.K., Langeland J., Prince V., Wang Y.-L., Westerfield M., Ekker M., and Postlethwait J.H. Zebrafish HOX clusters and vertebrate genome evolution Science 1998 282 1711-1784

[46]

Darnell J.E. and Doolittle W.F. Speculations on the early course of evolution Proc. Natl. Acad. Sci. USA 1986 83 1271-1275

[47]

Force A., Lynch M., Pickett F.B., Amores A., Lin Yan Y., and Postlethwait J. Preservation of duplicate genes by complementary, degenerative mutations Genetics 1999 151 1531-1545

[48]

Martin A.P. Increasing genomic complexity by gene duplication and the origin of vertebrates Am. Nat. 1999 154 111-128

[49]

Watson J.D., Hopkins N.H., Roberts J.W., Steitz J.A., and Weiner A.M. Molecular Biology of the Gene, 4th Edition 1987 Menlo Park, CA The Benjamin Cummings Publishing Company, Inc

[50]

Altenberg, L.: Evolving better representations through selective genome growth. In: Proceedings of the IEEE World Congress on Computational Intelligence, pp. 182–187. IEEE Press, Piscataway, NJ (1994)

[51]

Carroll S.B. Homeotic genes and the evolution of arthropods and chordates Nature 1995 376 479-485

[52]

Cybenko G. Approximation by superpositions of a sigmoidal function Math. Control Signals Syst. 1989 2 303-314

[53]

Meinhardt H. Models of Biological Pattern Formation 1982 London Academic Press

[54]

Curtis D., Apfeld J., and Lehmann R. Nanos is an evolutionarily conserved organizer of anterior-posterior polarity Development 1995 121 1899-1910

[55]

Lall S. and Patel N. Conservation and divergence in molecular mechanisms of axis formation Annu. Rev. Genet. 2001 35 407-447

[56]

Angeline P.J., Saunders G.M., and Pollack J.B. An evolutionary algorithm that constructs recurrent neural networks IEEE Trans. Neural Networks 1993 5 54-65

[57]

Opitz D.W. and Shavlik J.W. Connectionist theory refinement: Genetically searching the space of network topologies J. Artif. Intell. Res. 1997 6 177-209

[58]

Pujol, J.C.F., Poli, R.: Evolution of the topology and the weights of neural networks using genetic programming with a dual representation. Technical Report CSRP-97-7, School of Computer Science, The University of Birmingham, Birmingham B15 2TT, UK (1997)

[59]

Yao X. and Liu Y. Towards designing artificial neural networks by evolution Appl. Math. Comput. 1996 91 83-90

[60]

Zhang B.-T. and Muhlenbein H. Evolving optimal neural networks using genetic algorithms with Occam’s razor Complex Syst. 1993 7 199-220

[61]

Miller, J.F.P.T.: Cartesian genetic programming. In: Proceedings of the Third European Conference on Genetic Programming Published as Lecture Notes in Computer Science, vol. 1802, pp. 121–132 (2000)

[62]

Koza J.R. Genetic Programming: On the Programming of Computers by Means of Natural Slection 1992 Cambridge, MA MIT Press

[63]

Gomez, F., Miikkulainen, R.: Solving non-Markovian control tasks with neuroevolution. In: Proceedings of the 16th International Joint Conference on Artificial Intelligence, pp. 1356–1361. Kaufmann, San Francisco (1999)

[64]

Saravanan N. and Fogel D.B. Evolving neural control systems IEEE Expert 1995 10 23-27

[65]

Yao X. Evolving artificial neural networks Proc. IEEE 1999 87 1423-1447

[66]

Radcliffe N.J. Genetic set recombination and its application to neural network topology optimization Neural Comput. Appl. 1993 1 67-90

[67]

Roggen, D., Federici, D.: Multi-cellular development: Is there scalability and robustness to gain. In: Proceedings of the 8th International Conference on Parallel Problem Solving from Nature (PPSN VIII), pp. 391–400 (2004)

[68]

Reisinger, J., Stanley, K.O., Miikkulainen, R.: Towards an empirical measure of evolvability. In: Genetic and Evolutionary Computation Conference (GECCO2005) Workshop Program, pp. 257–264. ACM Press, Washington, D.C. (2005)

[69]

Parmee I.C. Improving problem definition through interactive evolutionary computation J. Artif. Intell. Eng. Design Analy. Manuf. Special Issue: Human-computer Interact. Eng. 2002 16 185-202

[70]

Takagi H. Interactive evolutionary computation: Fusion of the capacities of EC optimization and human evaluation Proc. IEEE 2001 89 1275-1296

[71]

Baluja S., Pomerleau D., and Jochem T. Towards automated artificial evolution for computer-generated images Connect. Sci. 1994 6 325-354

[72]

Dawkins R. Langton C.G. The evolution of evolvability Artificial Life 1989 Reading, MA Addison-Wesley 201-220

[73]

Greenfield G.R. Evolving expressions and art by choice Leonard 2000 33 93-99

[74]

Lund, H.H., Pagliarini, L., Miglino, P.: Artistic design with GA and NN. In: Proceedings of the 1st Nordic Workshop on Genetic Algorithms and Their Applications (1NWGA), pp. 97–105 (1995)

[75]

McCormack J.P. Green D.G. and Bossomaier T. Interactive evolution of L-system grammars for computer graphics modelling Complex Systems: From Biology to Computation 1993 Amsterdam IOS Press 118-130

[76]

Sims, K.: Artificial evolution for computer graphics. In: Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’91), pp. 319–328. ACM Press, New York, NY (1991)

[77]

Smith J.R. Belew R.K. and Booker L.B. Designing biomorphs with an interactive genetic algorithm Proceedings of the 4th International Conference on Genetic Algorithms (ICGA-91) 1991 San Mateo, CA Morgan Kaufmann 535-538

[78]

Todd S. and Latham W. Evolutionary Art and Computers 1992 London Academic Press

[79]

Dawkins R. The Blind Watchmaker 1986 Essex, UK Longman

[80]

Nishino, H., Takagi, H., Cho, S.-B., Utsumiya, K.: A 3D modeling system for creative design. In: 15th International Conference on Information Networking, pp. 479–486 (2001)

[81]

Todd, S., Latham, W.: The Mutation and Growth of Art by Computers, chapter 9, pp. 221–250. Morgan Kaufmann (1999)

[82]

Husbands P., Germy G., McIlhagga M., and Ives R. Two applications of genetic algorithms to component design Evolutionary Computing LNCS 1996 1143 50-61

[83]

Andersen T., Otter C., Petschulat C., Eoff U., Menten T., Davis R., Crowley B., et al. Westwood J. et al. A biologically-derived approach to tissue modeling Technology and Informatics 2005 Amsterdam IOS Press 15-21

[84]

Miller J.F. Banzhaf W., Christaller T., Dittrich P., Kim J.T., and Ziegler J. Evolving developmental programs for adaptation, morphogenesis, and self-repair Advances in Artificial Life. 7th European Conference on Artificial Life, vol. 2801 of Lecture Notes in Artificial Intelligence 2003 Dortmund, Germany Springer 256-265

Cited By

Eguiarte-Morett LAguilar W(2024)Premature convergence in morphology and control co-evolutionAdaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems10.1177/1059712323119849732:2(137-165)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1177/10597123231198497
Meyerson ENelson MBradley HGaier AMoradi AHoover ALehman J(2024)Language Model Crossover: Variation through Few-Shot PromptingACM Transactions on Evolutionary Learning and Optimization10.1145/36947914:4(1-40)Online publication date: 5-Sep-2024
https://dl.acm.org/doi/10.1145/3694791
Xie YPinskier JWang XHoward D(2024)Evolutionary Seeding of Diverse Structural Design Solutions via Topology OptimizationACM Transactions on Evolutionary Learning and Optimization10.1145/36706934:4(1-23)Online publication date: 5-Jun-2024
https://dl.acm.org/doi/10.1145/3670693
Show More Cited By

Index Terms

Compositional pattern producing networks: A novel abstraction of development
1. Computing methodologies
  1. Artificial intelligence
    1. Search methodologies
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Theory of computation

Index terms have been assigned to the content through auto-classification.

Recommendations

A hypercube-based encoding for evolving large-scale neural networks

Research in neuroevolution---that is, evolving artificial neural networks (ANNs) through evolutionary algorithms---is inspired by the evolution of biological brains, which can contain trillions of connections. Yet while neuroevolution has produced ...
A Taxonomy for artificial embryogeny

A major challenge for evolutionary computation is to evolve phenotypes such as neural networks, sensory systems, or motor controllers at the same level of complexity as found in biological organisms. In order to meet this challenge, many researchers are ...
Generative and developmental systems tutorial
GECCO Comp '14: Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation

Comments

Information & Contributors

Information

Published In

cover image Genetic Programming and Evolvable Machines

Genetic Programming and Evolvable Machines Volume 8, Issue 2

Jun 2007

108 pages

ISSN:1389-2576

Issue’s Table of Contents

© Springer Science+Business Media, LLC 2007.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 June 2007

Revision received: 21 January 2007

Received: 13 April 2006

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

214
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Eguiarte-Morett LAguilar W(2024)Premature convergence in morphology and control co-evolutionAdaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems10.1177/1059712323119849732:2(137-165)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1177/10597123231198497
Meyerson ENelson MBradley HGaier AMoradi AHoover ALehman J(2024)Language Model Crossover: Variation through Few-Shot PromptingACM Transactions on Evolutionary Learning and Optimization10.1145/36947914:4(1-40)Online publication date: 5-Sep-2024
https://dl.acm.org/doi/10.1145/3694791
Xie YPinskier JWang XHoward D(2024)Evolutionary Seeding of Diverse Structural Design Solutions via Topology OptimizationACM Transactions on Evolutionary Learning and Optimization10.1145/36706934:4(1-23)Online publication date: 5-Jun-2024
https://dl.acm.org/doi/10.1145/3670693
Mujkanovic FNsampi NTheobalt CSeidel HLeimkühler T(2024)Neural Gaussian Scale-Space FieldsACM Transactions on Graphics10.1145/365816343:4(1-15)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658163
Baeta FCorreia JMartins TMachado PLi XHandl J(2024)Exploring Evolutionary Generators within Generative Adversarial NetworksProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3654348(251-254)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3654348
Machado PCorreia JLi XHandl J(2024)Evolutionary Art and Design in the Machine Learning EraProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3648408(1460-1501)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3648408
Wang LZhao MLiu ESun KCheng RLi XHandl J(2024)Tensorized NeuroEvolution of Augmenting Topologies for GPU AccelerationProceedings of the Genetic and Evolutionary Computation Conference10.1145/3638529.3654210(1156-1164)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638529.3654210
Bishop JJooste JHoward DLi XHandl J(2024)Evolutionary Exploration of Triply Periodic Minimal Surfaces via Quality DiversityProceedings of the Genetic and Evolutionary Computation Conference10.1145/3638529.3654039(1165-1173)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638529.3654039
Liu SYao WWang HPeng WYang Y(2024)Rapidly Evolving Soft Robots via Action InheritanceIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.332745928:6(1674-1688)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TEVC.2023.3327459
Li WBuchanan EGoff LHart EHale MWei BCarlo MAngus MWoolley RGan ZWinfield ATimmis JEiben ATyrrell A(2024)Evaluation of Frameworks That Combine Evolution and Learning to Design Robots in Complex Morphological SpacesIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.331636328:6(1561-1574)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TEVC.2023.3316363
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents