research-article

Efficiently evolving programs through the search for novelty

Authors:

Kenneth O. StanleyAuthors Info & Claims

GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computation

Pages 837 - 844

https://doi.org/10.1145/1830483.1830638

Published: 07 July 2010 Publication History

Abstract

A significant challenge in genetic programming is premature convergence to local optima, which often prevents evolution from solving problems. This paper introduces to genetic programming a method that originated in neuroevolution (i.e. the evolution of artificial neural networks) that circumvents the problem of deceptive local optima. The main idea is to search only for behavioral novelty instead of for higher fitness values. Although such novelty search abandons following the gradient of the fitness function, if such gradients are deceptive they may actually occlude paths through the search space towards the objective. Because there are only so many ways to behave, the search for behavioral novelty is often computationally feasible and differs significantly from random search. Counterintuitively, in both a deceptive maze navigation task and the artificial ant benchmark task, genetic programming with novelty search, which ignores the objective, outperforms traditional genetic programming that directly searches for optimal behavior. Additionally, novelty search evolves smaller program trees in every variation of the test domains. Novelty search thus appears less susceptible to bloat, another significant problem in genetic programming. The conclusion is that novelty search is a viable new tool for efficiently solving some deceptive problems in genetic programming.

References

[1]

Wolfgang Banzhaf, Frank D. Francone, and Peter Nordin. The effect of extensive use of the mutation operator on generalization in genetic programming using sparse data sets. In PPSN IV: Proceedings of the 4th International Conference on Parallel Problem Solving from Nature, pages 300--309, London, UK, 1996. Springer-Verlag.

Digital Library

[2]

Edmund K. Burke, Steven Gustafson, Graham Kendall, and Natalio Krasnogor. Is increased diversity in genetic programming beneficial? An analysis of lineage selection. In Congress on Evolutionary Computation, pages 1398--1405. IEEE Press, 2003.

[3]

Sean B. Carroll. Chance and necessity: the evolution of morphological complexity and diversity. Nature, 409 (6823): 1102--1109, February 2001.

[4]

Steffen Christensen and Franz Oppacher. Solving the artificial ant on the santa fe trail problem in 20, 696 fitness evaluations. In GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computation, pages 1574--1579, New York, NY, USA, 2007. ACM.

Digital Library

[5]

Sevan Ficici and Jordan B. Pollack. Challenges in coevolutionary learning: Arms-race dynamics, open-endedness, and mediocre stable states. In Proceedings of the Sixth International Conference on Artificial Life, pages 238--247. MIT Press, 1998.

Digital Library

[6]

Chris Gathercole and Peter Ross. An adverse interaction between crossover and restricted tree depth in genetic programming. In GECCO '96: Proceedings of the First Annual Conference on Genetic Programming, pages 291--296, Cambridge, MA, USA, 1996. MIT Press.

Digital Library

[7]

David E. Goldberg. Simple genetic algorithms and the minimal deceptive problem. In L. D. Davis, editor, Genetic Algorithms and Simulated Annealing, Research Notes in Artificial Intelligence. Morgan Kaufmann, 1987.

[8]

John H. Holland. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. University of Michigan Press, Ann Arbor, MI, 1975.

Digital Library

[9]

J. R. Koza. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA, 1992.

Digital Library

[10]

John R. Koza. Genetic Programming II: Automatic Discovery of Reusable Programs. MIT Press, Cambridge Massachusetts, May 1994.

Digital Library

[11]

W. B. Langdon and R. Poli. Fitness causes bloat. In Soft Computing in Engineering Design and Manufacturing, pages 23--27. Springer-Verlag, 1997.

[12]

W. B. Langdon and R. Poli. Why ants are hard. In Genetic Programming 1998: Proceedings of the Third Annual Conference, pages 193--201, University of Wisconsin, Madison, Wisconsin, USA, 22-25 1998. Morgan Kaufmann.

[13]

Joel Lehman and Kenneth O. Stanley. Exploiting open-endedness to solve problems through the search for novelty. In Proceedings of the Eleventh International Conference on Artificial Life (ALIFE XI), Cambridge, MA, 2008. MIT Press.

[14]

Sean Luke. Genetic programming produced competitive soccer softbot teams for RoboCup97. In Genetic Programming 1998: Proceedings of the Third Annual Conference, pages 214--222, University of Wisconsin, Madison, Wisconsin, USA, 1998. Morgan Kaufmann.

[15]

Dylan Mawhinney and Vic Ciesielski. Preventing early convergence in genetic programming by replacing similar programs. In Proceedings of the 2002 Congress on Evolutionary Computation, pages 67--72. IEEE, 2000.

[16]

Thomas Miconi. Evolution and complexity: The double-edged sword. Artificial Life: Special Issue on the Evolution of Complexity, 2007.

Digital Library

[17]

John Stuart Mill. A System of Logic, Ratiocinative and Inductive. 1846.

[18]

Tom M. Mitchell. Machine Learning. McGraw-Hill, New York, 1997.

Digital Library

[19]

Jean-Baptiste Mouret. Novelty-based multiobjectivization. In Proceedings of the Workshop on Exploring New Horizons in Evolutionary Design of Robots, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009.

[20]

Esteban Ricalde and Katya Rodríguez Vázquez. A GP neutral function for the artificial ant problem. In GECCO '07: Proceedings of the 9th conference companion on Genetic and evolutionary computation, pages 2565--2571, New York, NY, USA, 2007. ACM.

Digital Library

[21]

Sebastian Risi, Sandy D. Vanderbleek, Charles E. Hughes, and Kenneth O. Stanley. How novelty search escapes the deceptive trap of learning to learn. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2009), New York, NY, 2009. ACM.

Digital Library

[22]

Conor Ryan. Pygmies and civil servants. In Kenneth E. Kinnear, Jr., editor, Advances in Genetic Programming, chapter 11, pages 243--263. MIT Press, 1994.

Digital Library

[23]

M. D. Schmidt and H. Lipson. Coevolution of fitness predictors. Evolutionary Computation, IEEE Transactions on, 12(6): 736--749, Dec. 2008.

Digital Library

[24]

Karl Sigmund. Games of Life: Explorations in Ecology, Evolution and Behaviour. Oxford University Press, Inc., New York, NY, USA, 1993.

Digital Library

[25]

Andrea Soltoggio and Ben Jones. Novelty of behaviour as a basis for the neuro-evolution of operant reward learning. In Proceedings of the Genetic and Evolutionary Computation Conference(GECCO-2009), New York, NY, 2009. ACM.

Digital Library

[26]

Terence Soule. Code Growth in Genetic Programming. PhD thesis, University of Idaho, Moscow, Idaho, USA, 15 May 1998.

Digital Library

[27]

Russell Standish. Open-ended artificial evolution. International Journal of Computational Intelligence and Applications, 3(167), 2003.

[28]

Wei Yan and Christopher D. Clack. Behavioural GP diversity for adaptive stock selection. In GECCO '09: Proceedings of the 11th Annual conference on Genetic and evolutionary computation, pages 1641--1648, New York, NY, USA, 2009. ACM.

Digital Library

[29]

N. Zaera, D. Cliff, and J. Bruten. (Not)evolving collective behaviours in synthetic fish. In From Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior. MIT Press Bradford Books., 1996.

[30]

Douglas Zongker and Bill Punch. lilgp 1. 01 user's manual. Technical report, Michigan State University, USA, 26 March 1996.

Cited By

Hufkens LVos TMarín B(2024)Novelty-Driven Evolutionary Scriptless TestingResearch Challenges in Information Science10.1007/978-3-031-59468-7_12(100-108)Online publication date: 4-May-2024
https://doi.org/10.1007/978-3-031-59468-7_12
Ely BKoh WHo EHassan TPham AQiu W(2023)Novelty Search Promotes Antigenic Diversity in Microbial PathogensPathogens10.3390/pathogens1203038812:3(388)Online publication date: 28-Feb-2023
https://doi.org/10.3390/pathogens12030388
Naredo ESansores CGodinez FLópez FUrbano PTrujillo LRyan C(2023)Comprehensive Analysis of Learning Cases in an Autonomous Navigation Task for the Evolution of General ControllersMathematical and Computational Applications10.3390/mca2802003528:2(35)Online publication date: 2-Mar-2023
https://doi.org/10.3390/mca28020035
Show More Cited By

Index Terms

Efficiently evolving programs through the search for novelty
1. Computing methodologies
  1. Machine learning

Recommendations

Devising Effective Novelty Search Algorithms: A Comprehensive Empirical Study
GECCO '15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation

Novelty search is a state-of-the-art evolutionary approach that promotes behavioural novelty instead of pursuing a static objective. Along with a large number of successful applications, many different variants of novelty search have been proposed. It ...
Critical factors in the performance of novelty search
GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computation

Novelty search is a recently proposed method for evolutionary computation designed to avoid the problem of deception, in which the fitness function guides the search process away from global optima. Novelty search replaces fitness-based selection with ...
Evolving a diversity of virtual creatures through novelty search and local competition
GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computation

An ambitious challenge in artificial life is to craft an evolutionary process that discovers a wide diversity of well-adapted virtual creatures within a single run. Unlike in nature, evolving creatures in virtual worlds tend to converge to a single ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computation

July 2010

1520 pages

ISBN:9781450300728

DOI:10.1145/1830483

General Chair:
Martin Pelikan
University of Missouri, USA
,
Program Chair:
Jürgen Branke
University of Warwick, Coventry, UK

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '10

Sponsor:

SIGEVO

GECCO '10: Genetic and Evolutionary Computation Conference

July 7 - 11, 2010

Oregon, Portland, USA

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

63
Total Citations
View Citations
414
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)3

Reflects downloads up to 01 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hufkens LVos TMarín B(2024)Novelty-Driven Evolutionary Scriptless TestingResearch Challenges in Information Science10.1007/978-3-031-59468-7_12(100-108)Online publication date: 4-May-2024
https://doi.org/10.1007/978-3-031-59468-7_12
Ely BKoh WHo EHassan TPham AQiu W(2023)Novelty Search Promotes Antigenic Diversity in Microbial PathogensPathogens10.3390/pathogens1203038812:3(388)Online publication date: 28-Feb-2023
https://doi.org/10.3390/pathogens12030388
Naredo ESansores CGodinez FLópez FUrbano PTrujillo LRyan C(2023)Comprehensive Analysis of Learning Cases in an Autonomous Navigation Task for the Evolution of General ControllersMathematical and Computational Applications10.3390/mca2802003528:2(35)Online publication date: 2-Mar-2023
https://doi.org/10.3390/mca28020035
Bach BKundu AAcharya TSarkar A(2023)Visualizing Quantum Circuit Probability: Estimating Quantum State Complexity for Quantum Program SynthesisEntropy10.3390/e2505076325:5(763)Online publication date: 7-May-2023
https://doi.org/10.3390/e25050763
Xia XSong HZhang YGui LXu XLi KLi Y(2023)A Particle Swarm Optimization With Adaptive Learning Weights Tuned by a Multiple-Input Multiple-Output Fuzzy Logic ControllerIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2022.322746431:7(2464-2478)Online publication date: Jul-2023
https://doi.org/10.1109/TFUZZ.2022.3227464
Weissl OEiben A(2023)Morphological-Novelty in Modular Robot Evolution2023 IEEE Symposium Series on Computational Intelligence (SSCI)10.1109/SSCI52147.2023.10372045(1066-1071)Online publication date: 5-Dec-2023
https://doi.org/10.1109/SSCI52147.2023.10372045
Surana SLim BCully A(2023)Efficient Learning of Locomotion Skills through the Discovery of Diverse Environmental Trajectory Generator Priors2023 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA48891.2023.10161267(12134-12141)Online publication date: 29-May-2023
https://doi.org/10.1109/ICRA48891.2023.10161267
Chen SChiu WWu C(2023)Quality Diversity Optimization Method for Bilinear Matrix Inequality Problems in Control System DesignIEEE Access10.1109/ACCESS.2023.329455911(77371-77384)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3294559
Allwood DEllis MGriffin DHayward TManneschi LMusameh MO'Keefe SStepney SSwindells CTrefzer MVasilaki EVenkat GVidamour IWringe C(2023)A perspective on physical reservoir computing with nanomagnetic devicesApplied Physics Letters10.1063/5.0119040122:4(040501)Online publication date: 23-Jan-2023
https://doi.org/10.1063/5.0119040
Nogueira ARibeiro JFernández de Vega FZenha-Rela M(2022)Evolutionary Approaches to Test Data Generation for Object-Oriented SoftwareResearch Anthology on Agile Software, Software Development, and Testing10.4018/978-1-6684-3702-5.ch044(884-909)Online publication date: 2022
https://doi.org/10.4018/978-1-6684-3702-5.ch044
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents