Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Dexterous manipulation and control with volumetric muscles

Published: 30 July 2018 Publication History

Abstract

We propose a framework for simulation and control of the human musculoskeletal system, capable of reproducing realistic animations of dexterous activities with high-level coordination. We present the first controllable system in this class that incorporates volumetric muscle actuators, tightly coupled with the motion controller, in enhancement of line-segment approximations that prior art is overwhelmingly restricted to. The theoretical framework put forth by our methodology computes all the necessary Jacobians for control, even with the drastically increased dimensionality of the state descriptors associated with three-dimensional, volumetric muscles. The direct coupling of volumetric actuators in the controller allows us to model muscular deficiencies that manifest in shape and geometry, in ways that cannot be captured with line-segment approximations. Our controller is coupled with a trajectory optimization framework, and its efficacy is demonstrated in complex motion tasks such as juggling, and weightlifting sequences with variable anatomic parameters and interaction constraints.

Supplementary Material

ZIP File (057-301.zip)
Supplemental files.
MP4 File (057-301.mp4)
MP4 File (a57-lee.mp4)

References

[1]
Sofien Bouaziz, Sebastian Martin, Tiantian Liu, Ladislav Kavan, and Mark Pauly. 2014. Projective dynamics: fusing constraint projections for fast simulation. ACM Transactions on Graphics 33, 4, Article 154 (2014).
[2]
Robert Bridson, Sebastian. Marino, and Ron Fedkiw. 2003. Simulation of Clothing with Folds and Wrinkles. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 28--36.
[3]
Steve Capell, Seth Green, Brian Curless, Tom Duchamp, and Zoran Popović. 2002. Interactive Skeleton-driven Dynamic Deformations. ACM Transactions on Graphics 21, 3 (2002), 586--593.
[4]
Stelian Coros, Sebastian Martin, Bernhard Thomaszewski, Christian Schumacher, Robert Sumner, and Markus Gross. 2012. Deformable objects alive! ACM Transactions on Graphics 31, 4, Article 69 (2012).
[5]
Ana Lucia Cruz Ruiz, Charles Pontonnier, Nicolas Pronost, and Georges Dumont. 2017. Muscle-Based Control for Character Animation. Computer Graphics Forum 36, 6 (2017), 122--147.
[6]
Michael Damsgaard, John Rasmussen, SÃÿren T. Christensen, Egidijus Surma, and Mark D. Zee. 2006. Analysis of musculoskeletal systems in the AnyBody Modeling System. Simulation Modelling Practice and Theory 14, 8 (2006), 1100 -- 1111.
[7]
Scott L. Delp, Frank C. Anderson, Anderson S. Arnold, Peter Loan, Ayman Habib, T. John, Eran Guendelman, and Darryl G. Thelen. 2007. OpenSim: Open-Source Software to Create and Analyze Dynamic Simulations of Movement. IEEE Transactions on Biomedical Engineering 54, 11 (2007), 1940--1950.
[8]
Ye Fan, Joshua Litven, and Dinesh K Pai. 2014. Active volumetric musculoskeletal systems. ACM Transactions on Graphics 33, 4, Article 152 (2014).
[9]
François Faure, Christian Duriez, Hervé Delingette, Jérémie Allard, Benjamin Gilles, Stéphanie Marchesseau, Hugo Talbot, Hadrien Courtecuisse, Guillaume Bousquet, Igor Peterlik, and Stéphane Cotin. 2012. SOFA: A Multi-Model Framework for Interactive Physical Simulation. In Soft Tissue Biomechanical Modeling for Computer Assisted Surgery. Studies in Mechanobiology, Tissue Engineering and Biomaterials, Vol. 11. 283--321.
[10]
Thomas Geijtenbeek, Michiel van de Panne, and A. Frank van der Stappen. 2013. Flexible Muscle-based Locomotion for Bipedal Creatures. ACM Transactions on Graphics 32, 6, Article 206 (2013).
[11]
Alexandru-Eugen Ichim, Petr Kadleček, Ladislav Kavan, and Mark Pauly. 2017. Phace: physics-based face modeling and animation. ACM Transactions on Graphics 36, 4, Article 153 (2017).
[12]
Petr Kadleček, Alexandru-Eugen Ichim, Tiantian Liu, Jaroslav Křivánek, and Ladislav Kavan. 2016. Reconstructing personalized anatomical models for physics-based body animation. ACM Transactions on Graphics 35, 6, Article 213 (2016).
[13]
Meekyoung Kim, Gerard Pons-Moll, Sergi Pujades, Seungbae Bang, Jinwook Kim, Michael J. Black, and Sung-Hee Lee. 2017. Data-driven Physics for Human Soft Tissue Animation. ACM Transactions on Graphics 36, 4, Article 54 (2017), 54:1--54:12 pages.
[14]
Yeara Kozlov, Derek Bradley, Moritz Bächer, Bernhard Thomaszewski, Thabo Beeler, and Markus Gross. 2017. Enriching Facial Blendshape Rigs with Physical Simulation. Computer Graphics Forum 36, 2 (2017), 75--84.
[15]
Jeongseok Lee, Michael X. Grey, Sehoon Ha, Tobias Kunz, Sumit Jain, Yuting Ye, Siddhartha S. Srinivasa, Mike Stilman, and C. Karen Liu. 2018. DART: Dynamic Animation and Robotics Toolkit. The Journal of Open Source Software 3, 22 (2018), 500.
[16]
Sunghee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2009. Comprehensive biomechanical modeling and simulation of the upper body. ACM Transactions on Graphics 28, 4, Article 99 (2009).
[17]
Sung Hee Lee and Demetri Terzopoulos. 2006. Heads Up!: Biomechanical Modeling and Neuromuscular Control of the Neck. ACM Transactions on Graphics 25, 3 (2006), 1188--1198.
[18]
Yoonsang Lee, Kyungho Lee, Soon-Sun Kwon, Jiwon Jeong, Carol O'Sullivan, Moon Seok Park, and Jehee Lee. 2015. Push-Recovery Stability of Biped Locomotion. ACM Transactions on Graphics 34, 6 (2015).
[19]
Yoonsang Lee, Moon Seok Park, Taesoo Kwon, and Jehee Lee. 2014. Locomotion control for many-muscle humanoids. ACM Transactions on Graphics 33, 6, Article 218 (2014).
[20]
C. Karen Liu. 2008. Synthesis of Interactive Hand Manipulation. In Proceedings of the ACMSIGGRAPH/Eurographics Symposium on Computer Animation. 163--171.
[21]
Libin Liu, KangKang Yin, Bin Wang, and Baining Guo. 2013. Simulation and Control of Skeleton-driven Soft Body Characters. ACM Transactions on Graphics 32, 6, Article 215 (2013), 8 pages.
[22]
Tiantian Liu, Sofien Bouaziz, and Ladislav Kavan. 2017. Quasi-Newton Methods for Real-Time Simulation of Hyperplastic Materials. ACM Transactions on Graphics 36, 3, Article 23 (2017).
[23]
John E. Lloyd, Ian Stavness, and Sidney Fels. 2012. ARTISYNTH: a fast interactive biomechanical modeling toolkit combining multibody and finite element simulation. (2012).
[24]
Richard Malgat, Benjamin Gilles, David IW Levin, Matthieu Nesme, and François Faure. 2015. Multifarious hierarchies of mechanical models for artist assigned levels-of-detail. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 27--36.
[25]
Nathan Mitchell, Mridul Aanjaneya, Rajsekhar Setaluri, and Eftychios Sifakis. 2015a. Non-manifold level sets: A multivalued implicit surface representation with applications to self-collision processing. ACM Transactions on Graphics 34, 6, Article 247 (2015).
[26]
Nathan Mitchell, Eftychios Sifakis, et al. 2015b. GRIDiron: An interactive authoring and cognitive training foundation for reconstructive plastic surgery procedures. ACM Transactions on Graphics 34, 4, Article 43 (2015).
[27]
Akihiko Murai, Q Youn Hong, Katsu Yamane, and Jessica K Hodgins. 2017. Dynamic skin deformation simulation using musculoskeletal model and soft tissue dynamics. Computational Visual Media 3, 1 (2017), 49--60.
[28]
Taylor Patterson, Nathan Mitchell, and Eftychios Sifakis. 2012. Simulation of complex nonlinear elastic bodies using lattice deformers. ACM Transactions on Graphics 31, 6, Article 197 (2012).
[29]
Xue Bin Peng, Glen Berseth, KangKang Yin, and Michiel Van De Panne. 2017. Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning. ACM Transactions on Graphics 36, 4, Article 41 (2017).
[30]
Burkard Polster. 2003. The Mathematics of Juggling.
[31]
Gerard Pons-Moll, Javier Romero, Naureen Mahmood, and Michael J Black. 2015. Dyna: A model of dynamic human shape in motion. ACM Transactions on Graphics 34, 4, Article 120 (2015).
[32]
HJ Ralston. 1976. Energetics of human walking. In Neural control of locomotion. 77--98.
[33]
Prashant Sachdeva, Shinjiro Sueda, Susanne Bradley, Mikhail Fain, and Dinesh K Pai. 2015. Biomechanical simulation and control of hands and tendinous systems. ACM Transactions on Graphics 34, 4, Article 42 (2015).
[34]
Shunsuke Saito, Zi-Ye Zhou, and Ladislav Kavan. 2015. Computational bodybuilding: Anatomically-based modeling of human bodies. ACM Transactions on Graphics 34, 4, Article 41 (2015).
[35]
Weiguang Si, Sung-Hee Lee, Eftychios Sifakis, and Demetri Terzopoulos. 2014. Realistic biomechanical simulation and control of human swimming. ACM Transactions on Graphics 34, 1, Article 10 (2014).
[36]
Eftychios Sifakis and Jernej Barbic. 2012. FEM simulation of3D deformable solids: a practitioner's guide to theory, discretization and model reduction. In ACM SIGGRAPH 2012 Courses. 20.
[37]
Eftychios Sifakis, Igor Neverov, and Ronald Fedkiw. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Transactions on Graphics 24, 3 (2005).
[38]
Ian Stavness, Alan G Hannam, John E Lloyd, and Sidney Fels. 2010. Predicting muscle patterns for hemimandibulectomy models. (2010), 483--91.
[39]
Julien Stelletta, Raphaël Dumas, and Yoann Lafon. 2017. Modeling of the thigh: a 3D deformable approach considering muscle interactions. In Biomechanics of Living Organs. 497--521.
[40]
Shinjiro Sueda, Andrew Kaufman, and Dinesh K. Pai. 2008. Musculotendon Simulation for Hand Animation. ACM Transactions on Graphics 27, 3, Article 83 (2008), 8 pages.
[41]
Jie Tan, Greg Turk, and C Karen Liu. 2012. Soft body locomotion. ACM Transactions on Graphics 31, 4, Article 26 (2012).
[42]
Joseph Teran, Sylvia S Blemker, Victor Ng-Thow-Hing, and Ronald Fedkiw. 2003. Finite volume methods for the simulation of skeletal muscle. In Proceedings of the ACM SIGGRAPH/Eurographics symposium on Computer animation. 68--74.
[43]
Joseph Teran, Eftychios Sifakis, Silvia S Blemker, Victor Ng-Thow-Hing, Cynthia Lau, and Ronald Fedkiw. 2005a. Creating and simulating skeletal muscle from the visible human data set. IEEE Transactions on Visualization and Computer Graphics 11, 3 (2005), 317--328.
[44]
Joseph Teran, Eftychios Sifakis, Geoffrey Irving, and Ronald Fedkiw. 2005b. Robust quasistatic finite elements and flesh simulation. In Proceedings of the ACM SIGGRAPH/Eurographics symposium on Computer animation. 181--190.
[45]
Darryl G Thelen et al. 2003. Adjustment of muscle mechanics model parameters to simulate dynamic contractions in older adults. Transactions-American Society Of Mechanical Engineers Journal Of Biomechanical Engineering 125, 1 (2003), 70--77.
[46]
Andreas Wachter and Lorenz T. Biegler. 2006. On the Implementation of an Interior-point Filter Line-search Algorithm for Large-scale Nonlinear Programming. Math. Program. 106, 1 (2006), 25--57.
[47]
Jack M. Wang, Samuel R. Hamner, Scott L. Delp, and Vladlen Koltun. 2012. Optimizing Locomotion Controllers Using Biologically-based Actuators and Objectives. ACM Transactions on Graphics 31, 4, Article 25 (2012).
[48]
Jungdam Won, Jongho Park, Kwanyu Kim, and Jehee Lee. 2017. How to train your dragon: example-guided control of flapping flight. ACM Transactions on Graphics 36, 6, Article 198 (2017).
[49]
Hongyi Xu and Jernej Barbič. 2016. Pose-space Subspace Dynamics. ACM Transactions on Graphics 35, 4, Article 35 (2016), 14 pages.
[50]
Felix E Zajac. 1989. Muscle and tendon Properties models scaling and application to biomechanics and motor. Critical reviews in biomedical engineering 17, 4 (1989), 359--411.
[51]
Lifeng Zhu, Xiaoyan Hu, and Ladislav Kavan. 2015. Adaptable anatomical models for realistic bone motion reconstruction. Computer Graphics Forum 34, 2 (2015), 459--471.

Cited By

View all
  • (2024)Analytic rotation-invariant modelling of anisotropic finite elementsACM Transactions on Graphics10.1145/366608643:5(1-20)Online publication date: 9-Aug-2024
  • (2024)A Neural Network Model for Efficient Musculoskeletal-Driven Skin DeformationACM Transactions on Graphics10.1145/365813543:4(1-12)Online publication date: 19-Jul-2024
  • (2024)MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00231(2382-2392)Online publication date: 16-Jun-2024
  • Show More Cited By

Index Terms

  1. Dexterous manipulation and control with volumetric muscles

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Graphics
      ACM Transactions on Graphics  Volume 37, Issue 4
      August 2018
      1670 pages
      ISSN:0730-0301
      EISSN:1557-7368
      DOI:10.1145/3197517
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 30 July 2018
      Published in TOG Volume 37, Issue 4

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. motion control
      2. musculoskeletal simulation
      3. trajectory optimization
      4. volumetric muscles

      Qualifiers

      • Research-article

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)87
      • Downloads (Last 6 weeks)11
      Reflects downloads up to 25 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Analytic rotation-invariant modelling of anisotropic finite elementsACM Transactions on Graphics10.1145/366608643:5(1-20)Online publication date: 9-Aug-2024
      • (2024)A Neural Network Model for Efficient Musculoskeletal-Driven Skin DeformationACM Transactions on Graphics10.1145/365813543:4(1-12)Online publication date: 19-Jul-2024
      • (2024)MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00231(2382-2392)Online publication date: 16-Jun-2024
      • (2023)MyoDexProceedings of the 40th International Conference on Machine Learning10.5555/3618408.3618543(3327-3346)Online publication date: 23-Jul-2023
      • (2023)Reinforcement Learning of Bipedal Walking with Musculoskeletal Models and Reference MotionsJournal of the Korea Computer Graphics Society10.15701/kcgs.2022.29.1.2329:1(23-29)Online publication date: 1-Mar-2023
      • (2023)Discovering Fatigued Movements for Virtual Character AnimationSIGGRAPH Asia 2023 Conference Papers10.1145/3610548.3618176(1-12)Online publication date: 10-Dec-2023
      • (2023)A Generalized Constitutive Model for Versatile MPM Simulation and Inverse Learning with Differentiable PhysicsProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/36069256:3(1-20)Online publication date: 24-Aug-2023
      • (2023)Bidirectional GaitNet: A Bidirectional Prediction Model of Human Gait and Anatomical ConditionsACM SIGGRAPH 2023 Conference Proceedings10.1145/3588432.3591492(1-9)Online publication date: 23-Jul-2023
      • (2023)Supervised Contrastive Learning for Interpretable Long-Form Document MatchingACM Transactions on Knowledge Discovery from Data10.1145/354282217:2(1-17)Online publication date: 20-Feb-2023
      • (2023)Numerical instability of Hill-type muscle modelsJournal of The Royal Society Interface10.1098/rsif.2022.043020:199Online publication date: Feb-2023
      • Show More Cited By

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media