research-article

Crossing the reality gap in evolutionary robotics by promoting transferable controllers

Authors:

Jean-Baptiste Mouret,

Stéphane DoncieuxAuthors Info & Claims

GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computation

Pages 119 - 126

https://doi.org/10.1145/1830483.1830505

Published: 07 July 2010 Publication History

Abstract

The reality gap, that often makes controllers evolved in simulation inefficient once transferred onto the real system, remains a critical issue in Evolutionary Robotics (ER); it prevents ER application to real-world problems. We hypothesize that this gap mainly stems from a conflict between the efficiency of the solutions in simulation and their transferability from simulation to reality: best solutions in simulation often rely on bad simulated phenomena (e.g. the most dynamic ones). This hypothesis leads to a multi-objective formulation of ER in which two main objectives are optimized via a Pareto-based Multi-Objective Evolutionary Algorithm: (1) the fitness and (2) the transferability. To evaluate this second objective, a simulation-to-reality disparity value is approximated for each controller. The proposed method is applied to the evolution of walking controllers for a real 8-DOF quadrupedal robot. It successfully finds efficient and well-transferable controllers with only a few experiments in reality.

References

[1]

A. Boeing and T. Bräunl. Evaluation of real-time physics simulation systems. In Proc. of the 5th international conference on Computer graphics and interactive techniques in Australia and Southeast Asia, pages 281--288. ACM New York, NY, USA, 2007.

Digital Library

[2]

J. Bongard and H. Lipson. Once more unto the breach: Co-evolving a robot and its simulator. In Proc. of Artificial life IX, page 57. MIT Press, 2004.

[3]

C. Coello. Constraint-handling using an evolutionary multiobjective optimization technique. Civil Engineering and Environmental Systems, 17(4):319--346, 2000.

[4]

K. Deb. Multi-objective optimization using evolutionary algorithms. Wiley, 2001.

Digital Library

[5]

K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation, 6(2):182--197, 2002.

Digital Library

[6]

D. Floreano and F. Mondada. Evolutionary neurocontrollers for autonomous mobile robots. Neural Networks, 11(7-8):1461--1478, 1998.

Digital Library

[7]

D. Floreano and J. Urzelai. Evolution of plastic control networks. Autonomous Robots, 11(3):311--317, 2001.

Digital Library

[8]

C. Grand, F. BenAmar, F. Plumet, and P. Bidaud. Decoupled control of posture and trajectory of the hybrid wheel-legged robot Hylos. In IEEE International Conference on Robotics and Automation, volume 5, pages 5111--5116, 2004.

[9]

S. Hanford, L. Long, and J. Horn. A small semi-autonomous rotary-wing unmanned air vehicle (UAV). In 2003 AIAA Atmospheric Flight Mechanics Conference & Exhibit, 2003.

[10]

C. Hartland and N. Bredèche. Evolutionary Robotics, Anticipation and the Reality Gap. In Proc. of ROBIO-06, pages 1640--1645, 2006.

[11]

N. Jakobi. Minimal Simulations for Evolutionary Robotics. PhD thesis, University of Sussex, 1998.

[12]

N. Jakobi. Running across the reality gap: Octopod locomotion evolved in a minimal simulation. Lecture Notes in Computer Science, 1468:39--58, 1998.

Digital Library

[13]

S. Koos, J. Mouret, and S. Doncieux. Automatic system identification based on coevolution of models and tests. In Proc. of IEEE CEC, volume 2009, 2009.

Digital Library

[14]

J. Lehman and K. Stanley. Exploiting open-endedness to solve problems through the search for novelty. In Proc. of ALIFE XI, volume 54, 2008.

[15]

O. Miglino, H. Lund, and S. Nolfi. Evolving Mobile Robots in Simulated and Real Environments. Artificial Life, 2(4):417--434, 1995.

Digital Library

[16]

J.-B. Mouret and S. Doncieux. Using behavioral exploration objectives to solve deceptive problems in neuro-evolution. In Proc. of GECCO-09, pages 627--634. ACM, 2009.

Digital Library

[17]

J.-B. Mouret and S. Doncieux. Sferes v2: Evolvin- in the multicore world. In Proc. of IEEE CEC-2010, 2010.

[18]

S. Nolfi and D. Floreano. Evolutionary robotics. MIT Press, 2000.

[19]

M. Palmer, D. Miller, and T. Blackwell. An Evolved Neural Controller for Bipedal Walking: Transitioning from Simulator to Hardware. In Proc. of IROS 2009 Workshop on Exploring new horizons in Evolutionary Design of Robots, 2009.

[20]

J. Pollack, H. Lipson, S. Ficici, P. Funes, G. Hornby, and R. Watson. Evolutionary techniques in physical robotics. In Evolvable systems: from biology to hardware: third international conference, ICES 2000, Proceedings, pages 175--186. Springer Verlag, 2000.

Digital Library

[21]

A. Thompson, P. Layzell, and R. Zebulum. Explorations in design space: Unconventional electronics design through artificial evolution. IEEE Transactions on Evolutionary Computation, 3(3):167--196, 1999.

Digital Library

[22]

J. Zagal and J. Ruiz-del Solar. Combining simulation and reality in evolutionary robotics. Journal of Intelligent and Robotic Systems, 50(1):19--39, 2007.

Digital Library

Cited By

Kim MShin UKim J(2024)Learning Quadrupedal Locomotion with Impaired Joints Using Random Joint Masking2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10610088(9751-9757)Online publication date: 13-May-2024
https://doi.org/10.1109/ICRA57147.2024.10610088
Li MMatthews DKriegman S(2024)Reinforcement learning for freeform robot design2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10610048(8799-8806)Online publication date: 13-May-2024
https://doi.org/10.1109/ICRA57147.2024.10610048
Aoshima KServin M(2024)Examining the simulation-to-reality gap of a wheel loader digging in deformable terrainMultibody System Dynamics10.1007/s11044-024-10005-5Online publication date: 19-Jul-2024
https://doi.org/10.1007/s11044-024-10005-5
Show More Cited By

Index Terms

Crossing the reality gap in evolutionary robotics by promoting transferable controllers
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Robotics
2. Computing methodologies
  1. Artificial intelligence
    1. Control methods
      1. Robotic planning
    2. Planning and scheduling
      1. Robotic planning

Recommendations

Some distance measures for morphological diversification in generative evolutionary robotics
GECCO '14: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation

Evolutionary robotics often involves optimization in large, complex search spaces, requiring good population diversity. Recently, measures to actively increase diversity or novelty have been employed in order to get sufficient exploration of the search ...
On self-adaptive rate restarts for evolutionary robotics with real rotorcraft
GECCO '17: Proceedings of the Genetic and Evolutionary Computation Conference

Self-adaptive parameters are increasingly used in the field of Evolutionary Robotics, as they allow key evolutionary rates to vary autonomously in a context-sensitive manner throughout the optimisation process. A significant limitation to self-adaptive ...
Multiobjective Optimization and Hybrid Evolutionary Algorithm to Solve Constrained Optimization Problems

This paper presents a novel evolutionary algorithm (EA) for constrained optimization problems, i.e., the hybrid constrained optimization EA (HCOEA). This algorithm effectively combines multiobjective optimization with global and local search models. In ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computation

July 2010

1520 pages

ISBN:9781450300728

DOI:10.1145/1830483

General Chair:
Martin Pelikan
University of Missouri, USA
,
Program Chair:
Jürgen Branke
University of Warwick, Coventry, UK

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGEVO: ACM Special Interest Group on Genetic and Evolutionary Computation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO '10

Sponsor:

SIGEVO

GECCO '10: Genetic and Evolutionary Computation Conference

July 7 - 11, 2010

Oregon, Portland, USA

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

63
Total Citations
View Citations
485
Total Downloads

Downloads (Last 12 months)39
Downloads (Last 6 weeks)2

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kim MShin UKim J(2024)Learning Quadrupedal Locomotion with Impaired Joints Using Random Joint Masking2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10610088(9751-9757)Online publication date: 13-May-2024
https://doi.org/10.1109/ICRA57147.2024.10610088
Li MMatthews DKriegman S(2024)Reinforcement learning for freeform robot design2024 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA57147.2024.10610048(8799-8806)Online publication date: 13-May-2024
https://doi.org/10.1109/ICRA57147.2024.10610048
Aoshima KServin M(2024)Examining the simulation-to-reality gap of a wheel loader digging in deformable terrainMultibody System Dynamics10.1007/s11044-024-10005-5Online publication date: 19-Jul-2024
https://doi.org/10.1007/s11044-024-10005-5
Li MGuo BZhao KXu RLiu SMao SZhou SXu QYu Z(2024)A Novel Framework for Adaptive Quadruped Robot Locomotion Learning in Uncertain EnvironmentsGreen, Pervasive, and Cloud Computing10.1007/978-981-99-9896-8_10(139-154)Online publication date: 23-Jan-2024
https://doi.org/10.1007/978-981-99-9896-8_10
Peng ZHan CLiu YZhou Z(2024)Energy-Based Policy Constraint for Offline Reinforcement LearningArtificial Intelligence10.1007/978-981-99-9119-8_30(335-346)Online publication date: 3-Feb-2024
https://doi.org/10.1007/978-981-99-9119-8_30
Feng HXue YLi HTang ZWang WWei ZZeng GLi MDai J(2023)Deformable Morphing and Multivariable Stiffness in the Evolutionary RoboticsInternational Journal of Automotive Manufacturing and Materials10.53941/ijamm.2023.100013(1)Online publication date: 24-Oct-2023
https://doi.org/10.53941/ijamm.2023.100013
Wu XDong WLai HYu YWen Y(2023)Adaptive Control Strategy for Quadruped Robots in Actuator Degradation ScenariosProceedings of the Fifth International Conference on Distributed Artificial Intelligence10.1145/3627676.3627686(1-13)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3627676.3627686
Salih AMoshaiov A(2023)Promoting Transfer of Robot Neuro-Motion-Controllers by Many-Objective Topology and Weight EvolutionIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.317229427:2(385-395)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TEVC.2022.3172294
Annaswamy AGuha ACui YTang SFisher PGaudio J(2023)Integration of Adaptive Control and Reinforcement Learning for Real-Time Control and LearningIEEE Transactions on Automatic Control10.1109/TAC.2023.329003768:12(7740-7755)Online publication date: Dec-2023
https://doi.org/10.1109/TAC.2023.3290037
Huang ZLin XHeld D(2023)Self-supervised Cloth Reconstruction via Action-conditioned Cloth Tracking2023 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA48891.2023.10160653(7111-7118)Online publication date: 29-May-2023
https://doi.org/10.1109/ICRA48891.2023.10160653
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents