research-article

Optimizing walking controllers for uncertain inputs and environments

Authors:

David J. Fleet, and

Aaron HertzmannAuthors Info & Claims

SIGGRAPH '10: ACM SIGGRAPH 2010 papers

July 2010

Article No.: 73, Pages 1 - 8

https://doi.org/10.1145/1833349.1778810

Published: 26 July 2010 Publication History

Abstract

We introduce methods for optimizing physics-based walking controllers for robustness to uncertainty. Many unknown factors, such as external forces, control torques, and user control inputs, cannot be known in advance and must be treated as uncertain. These variables are represented with probability distributions, and a return function scores the desirability of a single motion. Controller optimization entails maximizing the expected value of the return, which is computed by Monte Carlo methods. We demonstrate examples with different sources of uncertainty and task constraints. Optimizing control strategies under uncertainty increases robustness and produces natural variations in style.

Supplementary Material

Supplemental material. (073.zip)

Download
34.65 MB

MP4 File (tp097-10.mp4)

Download
61.97 MB

References

[1]

Abbeel, P., Coates, A., Quigley, M., and Ng, A. Y. 2007. An application of reinforcement learning to aerobatic helicopter flight. In Adv. NIPS 19. MIT Press, 1--8.

[2]

Byl, K., and Tedrake, R. 2009. Metastable walking machines. Int. J. Rob. Res. 28, 8, 1040--1064.

Digital Library

[3]

Coros, S., Beaudoin, P., and van de Panne, M. 2009. Robust task-based control policies for physics-based characters. ACM Trans. Graphics 28, 5, 170.

Digital Library

[4]

da Silva, M., Abe, Y., and Popović, J. 2008. Interactive simulation of stylized human locomotion. ACM Trans. Graphics 27, 3, 82.

Digital Library

[5]

Faisal, A. A., Selen, L. P. J., and Wolpert, D. M. 2008. Noise in the nervous system. Nature Rev. Neurosci. 9, 292--303.

[6]

Faloutsos, P., van de Panne, M., and Terzopoulos, D. 2001. Composable controllers for physics-based character animation. In Proc. SIGGRAPH, ACM, 251--260.

Digital Library

[7]

Grzeszczuk, R., and Terzopoulos, D. 1995. Automated learning of muscle-actuated locomotion through control abstraction. In Proc. SIGGRAPH, ACM, 63--70.

Digital Library

[8]

Grzeszczuk, R., Terzopoulos, D., and Hinton, G. 1998. NeuroAnimator: Fast neural network emulation and control of physics-based models. In Proc. SIGGRAPH, ACM, 9--20.

Digital Library

[9]

Hamilton, A. F. d. C., Jones, K. E., and Wolpert, D. M. 2004. The scaling of motor noise with muscle strength and motor unit in number in humans. Exp. Brain Res. 157, 4, 417--430.

[10]

Hansen, N. 2006. The CMA evolution strategy: A comparing review. In Towards a New Evolutionary Computation. Advances on Estimation of Distribution Algorithms. Springer, 75--102.

[11]

Harris, C. M., and Wolpert, D. M. 1998. Signal-dependent noise determines motor planning. Nature 394, 780--784.

[12]

Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H.-P. 2009. A statistical model of human pose and body shape. Computer Graphics Forum 28, 2, 337--346.

[13]

Körding, K. 2007. Decision Theory: What "Should" the Nervous System Do? Science 318, 606--610.

[14]

Lee, J., and Lee, K. H. 2006. Precomputing avatar behavior from human motion data. Graphical Models 68, 2, 158--174.

Digital Library

[15]

Lee, Y., Lee, S. J., and Popović, Z. 2009. Compact character controllers. ACM Trans. Graphics 28, 5, 169.

Digital Library

[16]

Lo, W.-Y., and Zwicker, M. 2008. Real-time planning for parameterized human motion. In Proc. Symposium on Computer Animation, ACM SIGGRAPH/Eurographics, 29--38.

Digital Library

[17]

McCann, J., and Pollard, N. S. 2007. Responsive characters from motion fragments. ACM Trans. Graphics 26, 3, 6.

Digital Library

[18]

Muico, U., Lee, Y., Popović, J., and Popović, Z. 2009. Contact-aware nonlinear control of dynamic characters. ACM Trans. Graphics 28, 3, 81.

Digital Library

[19]

Ng, A. Y., and Jordan, M. I. 2000. PEGASUS: A policy search method for large MDPs and POMDPs. In Proc. UAI, AUAI, 406--415.

Digital Library

[20]

Sharon, D., and van de Panne, M. 2005. Synthesis of controllers for stylized planar bipedal walking. In Proc. ICRA, IEEE, 2387--2392.

[21]

Sims, K. 1994. Evolving virtual creatures. In Proc. SIGGRAPH, ACM, 15--22.

Digital Library

[22]

Sok, K. W., Kim, M., and Lee, J. 2007. Simulating biped behaviors from human motion data. ACM Trans. Graphics 26, 3, 107.

Digital Library

[23]

Spall, J. C. 2003. Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control. Wiley.

Digital Library

[24]

Sutton, R. S., and Barto, A. G. 1998. Reinforcement Learning: An Introduction. MIT Press.

Digital Library

[25]

Tedrake, R., Zhang, T. W., and Seung, H. S. 2004. Stochastic policy gradient reinforcement learning on a simple 3D biped. In Proc. IROS, vol. 3, IEEE/RSJ, 2849--2854.

[26]

Todorov, E. 2004. Optimality principles in sensorimotor control. Nature Neuroscience 7, 9, 907--915.

[27]

Treuille, A., Lee, Y., and Popović, Z. 2007. Near-optimal character animation with continuous control. ACM Trans. Graphics 26, 3, 7.

Digital Library

[28]

van de Panne, M., and Fiume, E. 1993. Sensor-actuator networks. In Proc. SIGGRAPH, ACM, 335--342.

Digital Library

[29]

van de Panne, M., and Lamouret, A. 1995. Guided optimization for balanced locomotion. In Proc. CAS, EG, 165--177.

[30]

Wang, J. M., Fleet, D. J., and Hertzmann, A. 2009. Optimizing walking controllers. ACM Trans. Graphics 28, 5, 168.

Digital Library

[31]

Yin, K., Loken, K., and van de Panne, M. 2007. SIMBICON: Simple biped locomotion control. ACM Trans. Graphics 26, 3, 105.

Digital Library

Cited By

Alvarado ERohmer DCani M(2023)Generating Upper‐Body Motion for Real‐Time Characters Making their Way through Dynamic EnvironmentsComputer Graphics Forum10.1111/cgf.1463341:8(169-181)Online publication date: 20-Mar-2023
https://doi.org/10.1111/cgf.14633
Alvarado EPaliard CRohmer DCani M(2022)Real-Time Locomotion on Soft Grounds With Dynamic FootprintsFrontiers in Virtual Reality10.3389/frvir.2022.8018563Online publication date: 28-Mar-2022
https://doi.org/10.3389/frvir.2022.801856
Graur DMüller IProffitt MFourny GWatts GAlonso G(2022)Evaluating query languages and systems for high-energy physics dataProceedings of the VLDB Endowment10.14778/3489496.348949815:2(154-168)Online publication date: 4-Feb-2022
https://dl.acm.org/doi/10.14778/3489496.3489498
Show More Cited By

Recommendations

Optimizing walking controllers for uncertain inputs and environments

We introduce methods for optimizing physics-based walking controllers for robustness to uncertainty. Many unknown factors, such as external forces, control torques, and user control inputs, cannot be known in advance and must be treated as uncertain. ...
Read More
Optimizing walking controllers

This paper describes a method for optimizing the parameters of a physics-based controller for full-body, 3D walking. A modified version of the SIMBICON controller [Yin et al. 2007] is optimized for characters of varying body shape, walking speed and ...
Read More
Optimizing walking controllers
SIGGRAPH Asia '09: ACM SIGGRAPH Asia 2009 papers

This paper describes a method for optimizing the parameters of a physics-based controller for full-body, 3D walking. A modified version of the SIMBICON controller [Yin et al. 2007] is optimized for characters of varying body shape, walking speed and ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGGRAPH '10: ACM SIGGRAPH 2010 papers

July 2010

984 pages

ISBN:9781450302104

DOI:10.1145/1833349

Conference Chair:
Tony DeRose,
Editor:
Hugues Hoppe
ACM Transactions on Graphics

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 July 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGGRAPH '10

Sponsor:

SIGGRAPH

SIGGRAPH '10: Special Interest Group on Computer Graphics and Interactive Techniques Conference

July 26 - 30, 2010

California, Los Angeles

Acceptance Rates

SIGGRAPH '10 Paper Acceptance Rate 103 of 390 submissions, 26%;

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Upcoming Conference

SIGGRAPH '24

Sponsor:
siggraph

Special Interest Group on Computer Graphics and Interactive Techniques Conference

July 27 - August 1, 2024

Denver , CO , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
1,090
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)1

Other Metrics

View Author Metrics

Citations

Cited By

Alvarado ERohmer DCani M(2023)Generating Upper‐Body Motion for Real‐Time Characters Making their Way through Dynamic EnvironmentsComputer Graphics Forum10.1111/cgf.1463341:8(169-181)Online publication date: 20-Mar-2023
https://doi.org/10.1111/cgf.14633
Alvarado EPaliard CRohmer DCani M(2022)Real-Time Locomotion on Soft Grounds With Dynamic FootprintsFrontiers in Virtual Reality10.3389/frvir.2022.8018563Online publication date: 28-Mar-2022
https://doi.org/10.3389/frvir.2022.801856
Graur DMüller IProffitt MFourny GWatts GAlonso G(2022)Evaluating query languages and systems for high-energy physics dataProceedings of the VLDB Endowment10.14778/3489496.348949815:2(154-168)Online publication date: 4-Feb-2022
https://dl.acm.org/doi/10.14778/3489496.3489498
Etoh HOmura YKohei Kaminishi Chiba RTakakusaki KOta J(2022)Investigation of a Method to Extend a 2-Dimensional Gait to 3-Dimensions in a Human Musculoskeletal Model with 70 Muscles2022 International Symposium on Micro-NanoMehatronics and Human Science (MHS)10.1109/MHS56725.2022.10092017(1-5)Online publication date: 27-Nov-2022
https://doi.org/10.1109/MHS56725.2022.10092017
Fang JShen YWang YChen L(2021)ETOProceedings of the VLDB Endowment10.14778/3489496.348950015:2(183-195)Online publication date: 1-Oct-2021
https://dl.acm.org/doi/10.14778/3489496.3489500
Kang GLee Y(2021)Finite State Machine-Based Motion-Free Learning of Biped WalkingIEEE Access10.1109/ACCESS.2021.30552419(20662-20672)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3055241
Lee JLee Y(2020)Luxo character control using deep reinforcement learningJournal of the Korea Computer Graphics Society10.15701/kcgs.2020.26.4.126:4(1-8)Online publication date: 1-Sep-2020
https://doi.org/10.15701/kcgs.2020.26.4.1
Rajamaki JHamalainen P(2019)Continuous Control Monte Carlo Tree Search Informed by Multiple ExpertsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2018.284938625:8(2540-2553)Online publication date: 1-Aug-2019
https://doi.org/10.1109/TVCG.2018.2849386
Chentanez NMüller MMacklin MMakoviychuk VJeschke SCharalambous PChrysanthou YJones BLee J(2018)Physics-based motion capture imitation with deep reinforcement learningProceedings of the 11th ACM SIGGRAPH Conference on Motion, Interaction and Games10.1145/3274247.3274506(1-10)Online publication date: 8-Nov-2018
https://dl.acm.org/doi/10.1145/3274247.3274506
Baek SJeon DTong XKim M(2018)Simultaneous acquisition of polarimetric SVBRDF and normalsACM Transactions on Graphics10.1145/3272127.327501837:6(1-15)Online publication date: 4-Dec-2018
https://dl.acm.org/doi/10.1145/3272127.3275018
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents