Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Learning Virtual Chimeras by Dynamic Motion Reassembly

Published: 30 November 2022 Publication History

Abstract

The Chimera is a mythological hybrid creature composed of different animal parts. The chimera's movements are highly dependent on the spatial and temporal alignments of its composing parts. In this paper, we present a novel algorithm that creates and animates chimeras by dynamically reassembling source characters and their movements. Our algorithm exploits a two-network architecture: part assembler and dynamic controller. The part assembler is a supervised learning layer that searches for the spatial alignment among body parts, assuming that the temporal alignment is provided. The dynamic controller is a reinforcement learning layer that learns robust control policy for a wide variety of potential temporal alignments. These two layers are tightly intertwined and learned simultaneously. The chimera animation generated by our algorithm is energy efficient and expressive in terms of describing weight shifting, balancing, and full-body coordination. We demonstrate the versatility of our algorithm by generating the motor skills of a large variety of chimeras from limited source characters.

Supplemental Material

MP4 File
presentation

References

[1]
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. Tensorflow: A system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16). 265--283.
[2]
Kfir Aberman, Rundi Wu, Dani Lischinski, Baoquan Chen, and Daniel Cohen-Or. 2019. Learning Character-Agnostic Motion for Motion Retargeting in 2D. ACM Trans. Graph. 38, 4, Article 75 (2019).
[3]
Kevin Bergamin, Simon Claver, Daniel Holden, and James Richard Forbes. 2019. DReCon: Data-Driven Responsive Control of Physics-Based Characters. ACM Trans. Graph. 38, 6, Article 1 (2019).
[4]
James M. Bern, Kai-Hung Chang, and Stelian Coros. 2017. Interactive Design of Animated Plushies. ACM Trans. Graph. 36, 4, Article 80 (2017).
[5]
Stelian Coros, Sebastian Martin, Bernhard Thomaszewski, Christian Schumacher, Robert Sumner, and Markus Gross. 2012. Deformable Objects Alive! ACM Trans. Graph. 31, 4, Article 69 (2012).
[6]
Claire T Farley and C Richard Taylor. 1991. A mechanical trigger for the trot-gallop transition in horses. Science 253, 5017 (1991).
[7]
Thomas Geijtenbeek, Michiel van de Panne, and A. Frank van der Stappen. 2013. Flexible Muscle-based Locomotion for Bipedal Creatures. ACM Trans. Graph. 32, 6, Article 206 (2013).
[8]
Michael Gleicher. 1998. Retargetting Motion to New Characters. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '98). 33--42.
[9]
N. Hansen and A. Ostermeier. 1996. Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation. In Proceedings of IEEE International Conference on Evolutionary Computation. 312--317.
[10]
Rachel Heck, Lucas Kovar, and Michael Gleicher. 2006. Splicing upper-body actions with locomotion. 25, 3 (2006), 459--466.
[11]
Chris Hecker, Bernd Raabe, Ryan W. Enslow, John DeWeese, Jordan Maynard, and Kees van Prooijen. 2008. Real-time Motion Retargeting to Highly Varied User-Created Morphologies. In Proceedings of ACM SIGGRAPH '08.
[12]
Edmond S. L. Ho, Taku Komura, and Chiew-Lan Tai. 2010. Spatial Relationship Preserving Character Motion Adaptation. ACM Trans. Graph. 29, 4 (2010), 33:1--33:8.
[13]
Jessica K Hodgins and Nancy S Pollard. 1997. Adapting simulated behaviors for new characters. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques. ACM, 153--162.
[14]
Daniel Holden, Taku Komura, and Jun Saito. 2017. Phase-Functioned Neural Networks for Character Control. ACM Trans. Graph. 36, 4, Article 42 (2017).
[15]
Wenlong Huang, Igor Mordatch, and Deepak Pathak. 2020. One policy to control them all: Shared modular policies for agent-agnostic control. In International Conference on Machine Learning. 4455--4464.
[16]
Leslie Ikemoto and David A Forsyth. 2004. Enriching a motion collection by transplanting limbs. In Proceedings of the 2004 ACM SIGGRAPH/Eurographics symposium on Computer animation. 99--108.
[17]
Deok-Kyeong Jang, Soomin Park, and Sung-Hee Lee. 2022. Motion Puzzle: Arbitrary Motion Style Transfer by Body Part. ACM Trans. Graph. (2022).
[18]
Won-Seob Jang, Won-Kyu Lee, In-Kwon Lee, and Jehee Lee. 2008. Enriching a motion database by analogous combination of partial human motions. The Visual Computer 24, 4 (2008), 271--280.
[19]
Taeil Jin, Meekyoung Kim, and Sung-Hee Lee. 2018. Aura Mesh: Motion Retargeting to Preserve the Spatial Relationships between Skinned Characters. Computer Graphics Forum 37, 2 (2018), 311--320.
[20]
Manmyung Kim, Kyunglyul Hyun, Jongmin Kim, and Jehee Lee. 2009. Synchronized multi-character motion editing. ACM Trans. Graph. 28, 3, Article 79 (2009).
[21]
Paul G Kry, Lionel Revéret, François Faure, and M-P Cani. 2009. Modal locomotion: Animating virtual characters with natural vibrations. In Computer Graphics Forum. 289--298.
[22]
Jehee Lee. 2008. Representing Rotations and Orientations in Geometric Computing. IEEE Computer Graphics and Applications 28, 2 (2008), 75--83.
[23]
Jeongseok Lee, Michael X Grey, Sehoon Ha, Tobias Kunz, Sumit Jain, Yuting Ye, Siddhartha S Srinivasa, Mike Stilman, and C Karen Liu. 2018. DART: Dynamic animation and robotics toolkit. The Journal of Open Source Software 3, 22 (2018).
[24]
Jehee Lee and Sung Yong Shin. 1999. A hierarchical approach to interactive motion editing for human-like figures. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques. 39--48.
[25]
Kyungho Lee, Sehee Min, Sunmin Lee, and Jehee Lee. 2021b. Learning Time-Critical Responses for Interactive Character Control. ACM Trans. Graph. 40, 4, Article 147 (2021).
[26]
Seyoung Lee, Sunmin Lee, Yongwoo Lee, and Jehee Lee. 2021a. Learning a Family of Motor Skills from a Single Motion Clip. ACM Trans. Graph. 40, 4, Article 93 (2021).
[27]
Seunghwan Lee, Moonseok Park, Kyoungmin Lee, and Jehee Lee. 2019. Scalable Muscle-actuated Human Simulation and Control. ACM Trans. Graph. 38, 4, Article 73 (2019).
[28]
Yoonsang Lee, Sungeun Kim, and Jehee Lee. 2010. Data-driven Biped Control. ACM Trans. Graph. 29, 4, Article 129 (2010).
[29]
Hung Yu Ling, Fabio Zinno, George Cheng, and Michiel Van De Panne. 2020. Character controllers using motion vaes. ACM Trans. Graph. 39, 4, Article 40 (2020).
[30]
Libin Liu, Michiel Van De Panne, and Kangkang Yin. 2016. Guided Learning of Control Graphs for Physics-Based Characters. ACM Trans. Graph. 35, 3, Article 29 (2016).
[31]
Libin Liu, KangKang Yin, Bin Wang, and Baining Guo. 2013. Simulation and Control of Skeleton-driven Soft Body Characters. ACM Trans. Graph. 32, 6, Article 215 (2013).
[32]
Ying-Sheng Luo, Jonathan Hans Soeseno, Trista Pei-Chun Chen, and Wei-Chao Chen. 2020. CARL: Controllable Agent with Reinforcement Learning for Quadruped Locomotion. ACM Trans. Graph. 39, 4, Article 38 (2020), 10 pages.
[33]
Josh Merel, Saran Tunyasuvunakool, Arun Ahuja, Yuval Tassa, Leonard Hasenclever, Vu Pham, Tom Erez, Greg Wayne, and Nicolas Heess. 2020. Catch Carry: Reusable Neural Controllers for Vision-Guided Whole-Body Tasks. ACM Trans. Graph. 39, 4, Article 39 (2020).
[34]
Sehee Min, Jungdam Won, Seunghwan Lee, Jungnam Park, and Jehee Lee. 2019. SoftCon: Simulation and Control of Soft-Bodied Animals with Biomimetic Actuators. ACM Trans. Graph. 38, 6, Article 208 (2019).
[35]
Igor Mordatch, Kendall Lowrey, Galen Andrew, Zoran Popovic, and Emanuel V Todorov. 2015. Interactive control of diverse complex characters with neural networks. Advances in neural information processing systems 28 (2015).
[36]
Uldarico Muico, Jovan Popović, and Zoran Popović. 2011. Composite Control of Physically Simulated Characters. ACM Trans. Graph. 30, 3, Article 16 (2011).
[37]
Jungnam Park, Sehee Min, Phil Sik Chang, Jaedong Lee, Moon Seok Park, and Jehee Lee. 2022. Generative GaitNet. In Proceedings of ACM SIGGRAPH '22.
[38]
Soohwan Park, Hoseok Ryu, Seyoung Lee, Sunmin Lee, and J. Lee. 2019. Learning predict-and-simulate policies from unorganized human motion data. ACM Trans. Graph. 38, 6, Article 205 (2019).
[39]
Deepak Pathak, Christopher Lu, Trevor Darrell, Phillip Isola, and Alexei A Efros. 2019. Learning to control self-assembling morphologies: a study of generalization via modularity. Advances in Neural Information Processing Systems 32 (2019).
[40]
Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018. Deepmimic: Example-guided deep reinforcement learning of physics-based character skills. ACM Trans. Graph. 37, 4, Article 143 (2018).
[41]
Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa. 2021. AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control. ACM Trans. Graph. 40, 4, Article 144 (2021).
[42]
Daniele Reda, Hung Yu Ling, and Michiel Van De Panne. 2022. Learning to Brachiate via Simplified Model Imitation. In Proceedings of ACM SIGGRAPH '22.
[43]
Christian Schulz, Christoph von Tycowicz, Hans-Peter Seidel, and Klaus Hildebrandt. 2014. Animating Deformable Objects Using Sparse Spacetime Constraints. ACM Trans. Graph. 33, 4, Article 109 (2014).
[44]
Yeongho Seol, Carol O'Sullivan, and Jehee Lee. 2013. Creature features: online motion puppetry for non-human characters. In Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 213--221.
[45]
Hyun Joon Shin, Jehee Lee, Sung Yong Shin, and Michael Gleicher. 2001. Computer puppetry: An importance-based approach. ACM Trans. Graph. 20, 2 (2001), 67--94.
[46]
Karl Sims. 1994. Evolving virtual creatures. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques. 15--22.
[47]
Kwang Won Sok, Manmyung Kim, and Jehee Lee. 2007. Simulating biped behaviors from human motion data. ACM Trans. Graph. 26, 3, Article 107 (2007).
[48]
Sebastian Starke, Ian Mason, and Taku Komura. 2022. DeepPhase: Periodic Autoencoders for Learning Motion Phase Manifolds. ACM Trans. Graph. 41, 4 (2022).
[49]
Sebastian Starke, Yiwei Zhao, Fabio Zinno, and Taku Komura. 2021. Neural Animation Layering for Synthesizing Martial Arts Movements. ACM Trans. Graph. 40, 4, Article 92 (2021).
[50]
Jie Tan, Greg Turk, and C. Karen Liu. 2012. Soft Body Locomotion. ACM Trans. Graph. 28, 3, Article 26 (2012).
[51]
Tianxin Tao, Matthew Wilson, Ruiyu Gou, and Michiel Van De Panne. 2022. Learning to Get Up. In Proceedings of ACM SIGGRAPH '22.
[52]
C Richard Taylor, Knut Schmidt-Nielsen, and Jacob L Raab. 1970. Scaling of energetic cost of running to body size in mammals. American Journal of Physiology-Legacy Content 219, 4 (1970).
[53]
Vance A. Tucker. 1970. Energetic cost of locomotion in animals. Comparative Biochemistry and Physiology 34, 4 (1970).
[54]
Kevin Wampler and Zoran Popović. 2009. Optimal gait and form for animal locomotion. ACM Trans. Graph. 28, 3, Article 60 (2009).
[55]
Kevin Wampler, Zoran Popović, and Jovan Popović. 2014. Generalizing Locomotion Style to New Animals with Inverse Optimal Regression. ACM Trans. Graph. 33, 4, Article 49 (2014).
[56]
Tingwu Wang, Renjie Liao, Jimmy Ba, and Sanja Fidler. 2018. Nervenet: Learning structured policy with graph neural networks. In International conference on learning representations.
[57]
Jungdam Won, Deepak Gopinath, and Jessica Hodgins. 2020. A Scalable Approach to Control Diverse Behaviors for Physically Simulated Characters. ACM Trans. Graph. 39, 4, Article 33 (2020).
[58]
Jungdam Won and Jehee Lee. 2019. Learning body shape variation in physics-based characters. ACM Trans. Graph. 38, 6, Article 207 (2019).
[59]
Jungdam Won, Jongho Park, Kwanyu Kim, and Jehee Lee. 2017. How to Train Your Dragon: Example-Guided Control of Flapping Flight. ACM Trans. Graph. 36, 6, Article 198 (2017).
[60]
KangKang Yin, Kevin Loken, and Michiel Van de Panne. 2007. Simbicon: Simple biped locomotion control. ACM Trans. Graph. 26, 3, Article 105 (2007).
[61]
Zhiqi Yin, Zeshi Yang, Michiel Van De Panne, and Kangkang Yin. 2021. Discovering Diverse Athletic Jumping Strategies. ACM Trans. Graph. 40, 4, Article 91 (2021).
[62]
Zhiqi Yin and KangKang Yin. 2020. Linear Time Stable PD Controllers for Physics-Based Character Animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 191--200.
[63]
Wenhao Yu, Greg Turk, and C. Karen Liu. 2018. Learning Symmetric and Low-Energy Locomotion. ACM Trans. Graph. 37, 4, Article 144 (2018).
[64]
He Zhang, Sebastian Starke, Taku Komura, and Jun Saito. 2018. Mode-Adaptive Neural Networks for Quadruped Motion Control. ACM Trans. Graph. 37, 4, Article 145 (2018).
[65]
Allan Zhao, Jie Xu, Mina Konaković Luković, Josephine Hughes, Andrew Speilberg, Daniela Rus, and Wojciech Matusik. 2020. RoboGrammar: Graph Grammar for Terrain-Optimized Robot Design. ACM Trans. Graph. 39, 6 (2020).

Cited By

View all
  • (2024)3D-Layers: Bringing Layer-Based Color Editing to VR PaintingACM Transactions on Graphics10.1145/365818343:4(1-15)Online publication date: 19-Jul-2024
  • (2024)MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete RepresentationsACM Transactions on Graphics10.1145/365813743:4(1-21)Online publication date: 19-Jul-2024
  • (2024)LGTM: Local-to-Global Text-Driven Human Motion Diffusion ModelACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657422(1-9)Online publication date: 13-Jul-2024
  • Show More Cited By

Index Terms

  1. Learning Virtual Chimeras by Dynamic Motion Reassembly

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Graphics
      ACM Transactions on Graphics  Volume 41, Issue 6
      December 2022
      1428 pages
      ISSN:0730-0301
      EISSN:1557-7368
      DOI:10.1145/3550454
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 30 November 2022
      Published in TOG Volume 41, Issue 6

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. data-driven animation
      2. deep reinforcement learning
      3. physics-based simulation

      Qualifiers

      • Research-article

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)60
      • Downloads (Last 6 weeks)6
      Reflects downloads up to 25 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)3D-Layers: Bringing Layer-Based Color Editing to VR PaintingACM Transactions on Graphics10.1145/365818343:4(1-15)Online publication date: 19-Jul-2024
      • (2024)MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete RepresentationsACM Transactions on Graphics10.1145/365813743:4(1-21)Online publication date: 19-Jul-2024
      • (2024)LGTM: Local-to-Global Text-Driven Human Motion Diffusion ModelACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657422(1-9)Online publication date: 13-Jul-2024
      • (2024)Evolution-Based Shape and Behavior Co-Design of Virtual AgentsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.335574530:12(7579-7591)Online publication date: Dec-2024
      • (2023)Adaptive Tracking of a Single-Rigid-Body Character in Various EnvironmentsSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618187(1-11)Online publication date: 10-Dec-2023
      • (2023)Example-based Motion Synthesis via Generative Motion MatchingACM Transactions on Graphics10.1145/359239542:4(1-12)Online publication date: 26-Jul-2023
      • (2023)Bidirectional GaitNet: A Bidirectional Prediction Model of Human Gait and Anatomical ConditionsACM SIGGRAPH 2023 Conference Proceedings10.1145/3588432.3591492(1-9)Online publication date: 23-Jul-2023
      • (2023)PMP: Learning to Physically Interact with Environments using Part-wise Motion PriorsACM SIGGRAPH 2023 Conference Proceedings10.1145/3588432.3591487(1-10)Online publication date: 23-Jul-2023
      • (2023)Pose-Aware Attention Network for Flexible Motion Retargeting by Body PartIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.327791830:8(4792-4808)Online publication date: 19-May-2023

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media