research-article

Evolving the placement and density of neurons in the hyperneat substrate

Authors:

Sebastian Risi,

Kenneth O. StanleyAuthors Info & Claims

GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computation

Pages 563 - 570

https://doi.org/10.1145/1830483.1830589

Published: 07 July 2010 Publication History

Abstract

The Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT) approach demonstrated that the pattern of weights across the connectivity of an artificial neural network (ANN) can be generated as a function of its geometry, thereby allowing large ANNs to be evolved for high-dimensional problems. Yet it left to the user the question of where hidden nodes should be placed in a geometry that is potentially infinitely dense. To relieve the user from this decision, this paper introduces an extension called evolvable-substrate HyperNEAT (ES-HyperNEAT) that determines the placement and density of the hidden nodes based on a quadtree-like decomposition of the hypercube of weights and a novel insight about the relationship between connectivity and node placement. The idea is that the representation in HyperNEAT that encodes the pattern of connectivity across the ANN contains implicit information on where the nodes should be placed and can therefore be exploited to avoid the need to evolve explicit placement. In this paper, as a proof of concept, ES-HyperNEAT discovers working placements of hidden nodes for a simple navigation domain on its own, thereby eliminating the need to configure the HyperNEAT substrate by hand and suggesting the potential power of the new approach.

References

[1]

T. Aaltonen et al. Measurement of the top quark mass with dilepton events selected using neuroevolution at CDF. Physical Review Letters, 2009.

[2]

P. J. Bentley and S. Kumar. Three ways to grow designs: A comparison of embryogenies for an evolutionary design problem. In Proceedings of the Genetic and Evol. Comp. Conf. (GECCO-1999), pages 35--43, San Francisco, 1999. Kaufmann.

[3]

J. C. Bongard. Evolving modular genetic regulatory networks. In Proceedings of the 2002 Congress on Evolutionary Computation, 2002.

Digital Library

[4]

J. Clune, B. E. Beckmann, C. Ofria, and R. T. Pennock. Evolving coordinated quadruped gaits with the HyperNEAT generative encoding. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC-2009) Special Section on Evolutionary Robotics, Piscataway, NJ, USA, 2009. IEEE Press.

Digital Library

[5]

D. D'Ambrosio and K. O. Stanley. A novel generative encoding for exploiting neural network sensor and output geometry. In Proc. of the Genetic and Evol. Comp. Conf. (GECCO 2007), NY, 2007. ACM Press.

Digital Library

[6]

J. Drchal, J. Koutnik, and M. Šnorek. HyperNEAT controlled robots learn to drive on roads in simulated environment. In Proc. of the IEEE Congress on Evol. Comp. (CEC 2009). IEEE Press, 2009.

Digital Library

[7]

R. Finkel and J. Bentley. Quad trees: A data structure for retrieval on composite keys. Acta informatica, 4(1):1--9, 1974.

Digital Library

[8]

D. Floreano, P. Dürr, and C. Mattiussi. Neuroevolution: from architectures to learning. Evolutionary Intelligence, 1(1):47--62, 2008.

[9]

J. Gauci and K. O. Stanley. A case study on the critical role of geometric regularity in machine learning. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (AAAI-2008), Menlo Park, CA, 2008. AAAI Press.

Digital Library

[10]

J. Gauci and K. O. Stanley. Autonomous evolution of topographic regularities in artificial neural networks. Neural Computation, 2010. To appear.

Digital Library

[11]

C. Green. SharpNEAT homepage. http://sharpneat.sourceforge.net/, 2003{2006.

[12]

G. S. Hornby and J. B. Pollack. Creating high-level components with a generative representation for body-brain evolution. Artificial Life, 8(3), 2002.

Digital Library

[13]

E. R. Kandel, J. H. Schwartz, and T. M. Jessell. Principles of Neural Science. McGraw-Hill, New York, fourth edition, 2000.

[14]

A. Rosenfeld. Quadtrees and pyramids for pattern recognition and image processing. In Proceedings of the 5th International Conference on Pattern Recognition, pages 802--809. IEEE Press, 1980.

[15]

J. Secretan, N. Beato, D. B. D'Ambrosio, A. Rodriguez, A. Campbell, and K. O. Stanley. Picbreeder: Evolving pictures collaboratively online. In CHI '08: Proc. of the twenty-sixth annual SIGCHI conf. on Human factors in computing systems, pages 1759--1768, New York, NY, USA, 2008. ACM.

Digital Library

[16]

K. O. Stanley. Compositional pattern producing networks: A novel abstraction of development. Genetic Programming and Evolvable Machines Special Issue on Developmental Systems, 8(2):131--162, 2007.

Digital Library

[17]

K. O. Stanley, D. B. D'Ambrosio, and J. Gauci. A hypercube-based indirect encoding for evolving large-scale neural networks. Artificial Life, 15(2):185--212, 2009.

Digital Library

[18]

K. O. Stanley and R. Miikkulainen. Evolving neural networks through augmenting topologies. Evolutionary Computation, 10:99--127, 2002.

Digital Library

[19]

K. O. Stanley and R. Miikkulainen. A taxonomy for artificial embryogeny. Art. Life, 9(2):93--130, 2003.

Digital Library

[20]

K. O. Stanley and R. Miikkulainen. Competitive coevolution through evolutionary complexification. Journal of Art. Int. Research, 21:63--100, 2004.

Digital Library

[21]

P. Strobach. Quadtree-structured recursive plane decomposition coding of images. Signal Processing, 39:1380--1397, 1991.

Digital Library

Cited By

Xue MPeng KGong XZhang QChen YLi R(2023)EchoProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108747:3(1-24)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610874
Behjat AMaurer NChidambaran SChowdhury S(2023)Adaptive Neuroevolution With Genetic Operator Control and Two-Way Complexity VariationIEEE Transactions on Artificial Intelligence10.1109/TAI.2022.32141814:6(1627-1641)Online publication date: Dec-2023
https://doi.org/10.1109/TAI.2022.3214181
Wang MYang XQian YLei YCai JHuan ZLin XDong H(2022)Adaptive Neural Network Structure Optimization Algorithm Based on Dynamic NodesCurrent Issues in Molecular Biology10.3390/cimb4402005644:2(817-832)Online publication date: 7-Feb-2022
https://doi.org/10.3390/cimb44020056
Show More Cited By

Index Terms

Evolving the placement and density of neurons in the hyperneat substrate
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

Investigating whether hyperNEAT produces modular neural networks
GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computation

HyperNEAT represents a class of neuroevolutionary algorithms that captures some of the power of natural development with a computationally efficient high-level abstraction of development. This class of algorithms is intended to provide many of the ...
The sensitivity of HyperNEAT to different geometric representations of a problem
GECCO '09: Proceedings of the 11th Annual conference on Genetic and evolutionary computation

HyperNEAT, a generative encoding for evolving artificial neural networks (ANNs), has the unique and powerful ability to exploit the geometry of a problem (e.g., symmetries) by encoding ANNs as a function of a problem's geometry. This paper provides the ...
Constraining connectivity to encourage modularity in HyperNEAT
GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computation

A challenging goal of generative and developmental systems (GDS) is to effectively evolve neural networks as complex and capable as those found in nature. Two key properties of neural structures in nature are regularity and modularity. While HyperNEAT ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computation

July 2010

1520 pages

ISBN:9781450300728

DOI:10.1145/1830483

General Chair:
Martin Pelikan
University of Missouri, USA
,
Program Chair:
Jürgen Branke
University of Warwick, Coventry, UK

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '10

Sponsor:

SIGEVO

GECCO '10: Genetic and Evolutionary Computation Conference

July 7 - 11, 2010

Oregon, Portland, USA

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
229
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)1

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Xue MPeng KGong XZhang QChen YLi R(2023)EchoProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108747:3(1-24)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610874
Behjat AMaurer NChidambaran SChowdhury S(2023)Adaptive Neuroevolution With Genetic Operator Control and Two-Way Complexity VariationIEEE Transactions on Artificial Intelligence10.1109/TAI.2022.32141814:6(1627-1641)Online publication date: Dec-2023
https://doi.org/10.1109/TAI.2022.3214181
Wang MYang XQian YLei YCai JHuan ZLin XDong H(2022)Adaptive Neural Network Structure Optimization Algorithm Based on Dynamic NodesCurrent Issues in Molecular Biology10.3390/cimb4402005644:2(817-832)Online publication date: 7-Feb-2022
https://doi.org/10.3390/cimb44020056
Templier PRachelson EWilson DKrawiec K(2021)A geometric encoding for neural network evolutionProceedings of the Genetic and Evolutionary Computation Conference10.1145/3449639.3459361(919-927)Online publication date: 26-Jun-2021
https://dl.acm.org/doi/10.1145/3449639.3459361
Martinez ADel Ser JVillar-Rodriguez EOsaba EPoyatos JTabik SMolina DHerrera F(2021)Lights and shadows in Evolutionary Deep Learning: Taxonomy, critical methodological analysis, cases of study, learned lessons, recommendations and challengesInformation Fusion10.1016/j.inffus.2020.10.01467(161-194)Online publication date: Mar-2021
https://doi.org/10.1016/j.inffus.2020.10.014
Ünal HBaşçiftçi F(2021)Evolutionary design of neural network architectures: a review of three decades of researchArtificial Intelligence Review10.1007/s10462-021-10049-555:3(1723-1802)Online publication date: 27-Jul-2021
https://doi.org/10.1007/s10462-021-10049-5
Miller JWilson DCussat-Blanc S(2019)Evolving Programs to Build Artificial Neural NetworksFrom Astrophysics to Unconventional Computation10.1007/978-3-030-15792-0_2(23-71)Online publication date: 17-Apr-2019
https://doi.org/10.1007/978-3-030-15792-0_2
Miller JWilson DCussat-Blanc S(2019)Evolving Developmental Programs That Build Neural Networks for Solving Multiple ProblemsGenetic Programming Theory and Practice XVI10.1007/978-3-030-04735-1_8(137-178)Online publication date: 24-Jan-2019
https://doi.org/10.1007/978-3-030-04735-1_8
Hagg AMensing MAsteroth ABosman P(2017)Evolving parsimonious networks by mixing activation functionsProceedings of the Genetic and Evolutionary Computation Conference10.1145/3071178.3071275(425-432)Online publication date: 1-Jul-2017
https://dl.acm.org/doi/10.1145/3071178.3071275
Gaier AEsparcia-Alcázar A(2015)Evolutionary Design via Indirect Encoding of Non-Uniform Rational Basis SplinesProceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation10.1145/2739482.2768478(1197-1200)Online publication date: 11-Jul-2015
https://dl.acm.org/doi/10.1145/2739482.2768478
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents