MeshGraphNetRP: Improving Generalization of GNN-based Cloth Simulation
Article No.: 5, Pages 1 - 7
Abstract
Deep learning-based cloth simulation approaches have potential in achieving real-time simulation of complex cloth by directly learning a mapping from control input to resulting cloth movement, bypassing the need for time-consuming dynamic solving and collision processing. Recent advancements have demonstrated the effectiveness of Graph Neural Networks (GNN) in learning cloth dynamics. However, existing GNN-based models have limitations in predicting scenarios involving complex cloth movement. To overcome this limitation, we propose a novel GNN-based model that incorporates several components, including RNN-based state encoding and physics-informed features. Our model significantly improves the accuracy of cloth dynamics prediction in various scenarios, including those with complex cloth movement driven by control handles. Furthermore, our model demonstrates generalization capabilities for cloth mesh topology and control handle configurations. We validate the effectiveness of our approach through ablation studies and comparisons with a baseline model.
Supplementary Material
References
[1]
Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, and Razvan Pascanu. 2018. Relational inductive biases, deep learning, and graph networks. arxiv:1806.01261 [cs.LG]
[2]
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks(NIPS’15). MIT Press, Cambridge, MA, USA, 1171–1179.
[3]
Hugo Bertiche, Meysam Madadi, and Sergio Escalera. 2020a. CLOTH3D: Clothed 3D Humans. arxiv:1912.02792 [cs.CV]
[4]
Hugo Bertiche, Meysam Madadi, and Sergio Escalera. 2020b. PBNS: physically based neural simulator for unsupervised garment pose space deformation. arXiv preprint arXiv:2012.11310 (2020).
[5]
Hugo Bertiche, Meysam Madadi, and Sergio Escalera. 2021. PBNS: Physically Based Neural Simulator for Unsupervised Garment Pose Space Deformation. arxiv:2012.11310 [cs.CV]
[6]
Hugo Bertiche, Meysam Madadi, and Sergio Escalera. 2022. Neural Cloth Simulation. ACM Transactions on Graphics 41, 6 (nov 2022), 1–14. https://doi.org/10.1145/3550454.3555491
[7]
Robert Bridson, Sebastian Marino, and Ronald Fedkiw. 2005. Simulation of clothing with folds and wrinkles. In ACM SIGGRAPH 2005 Courses. 3–es.
[8]
Artur Grigorev, Bernhard Thomaszewski, Michael J Black, and Otmar Hilliges. 2022. HOOD: Hierarchical Graphs for Generalized Modelling of Clothing Dynamics. arXiv preprint arXiv:2212.07242 (2022).
[9]
Erhan Gundogdu, Victor Constantin, Amrollah Seifoddini, Minh Dang, Mathieu Salzmann, and Pascal Fua. 2019. Garnet: A two-stream network for fast and accurate 3d cloth draping. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 8739–8748.
[10]
Doyub Kim, Woojong Koh, Rahul Narain, Kayvon Fatahalian, Adrien Treuille, and James F. O’Brien. 2013. Near-Exhaustive Precomputation of Secondary Cloth Effects. ACM Trans. Graph. 32, 4, Article 87 (jul 2013), 8 pages. https://doi.org/10.1145/2461912.2462020
[11]
Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. arxiv:1609.02907 [cs.LG]
[12]
Zorah Lahner, Daniel Cremers, and Tony Tung. 2018. Deepwrinkles: Accurate and realistic clothing modeling. In Proceedings of the European conference on computer vision (ECCV). 667–684.
[13]
Alberta Longhini, Marco Moletta, Alfredo Reichlin, Michael C Welle, David Held, Zackory Erickson, and Danica Kragic. 2022. EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamics. arXiv preprint arXiv:2209.08996 (2022).
[14]
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. 2015. SMPL: A Skinned Multi-Person Linear Model. ACM Trans. Graph. 34, 6, Article 248 (nov 2015), 16 pages. https://doi.org/10.1145/2816795.2818013
[15]
E. Miguel, D. Bradley, B. Thomaszewski, B. Bickel, W. Matusik, M. A. Otaduy, and S. Marschner. 2012. Data-Driven Estimation of Cloth Simulation Models. Comput. Graph. Forum 31, 2pt2 (may 2012), 519–528.
[16]
Rahul Narain, Armin Samii, and James F O’brien. 2012. Adaptive anisotropic remeshing for cloth simulation. ACM transactions on graphics (TOG) 31, 6 (2012), 1–10.
[17]
Xiaoyu Pan, Jiaming Mai, Xinwei Jiang, Dongxue Tang, Jingxiang Li, Tianjia Shao, Kun Zhou, Xiaogang Jin, and Dinesh Manocha. 2022. Predicting Loose-Fitting Garment Deformations Using Bone-Driven Motion Networks. In Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings. ACM. https://doi.org/10.1145/3528233.3530709
[18]
Chaitanya Patel, Zhouyingcheng Liao, and Gerard Pons-Moll. 2020. TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style. arxiv:2003.04583 [cs.CV]
[19]
Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W. Battaglia. 2021. Learning Mesh-Based Simulation with Graph Networks. arxiv:2010.03409 [cs.LG]
[20]
Igor Santesteban, Miguel A. Otaduy, and Dan Casas. 2022. SNUG: Self-Supervised Neural Dynamic Garments. arxiv:2204.02219 [cs.CV]
[21]
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2008. The graph neural network model. IEEE transactions on neural networks 20, 1 (2008), 61–80.
[22]
An Ping Song, Xin Yi Di, Xiao Kang Xu, and Zi Heng Song. 2020. MeshGraphNet: An effective 3D polygon mesh recognition With topology reconstruction. IEEE Access 8 (2020), 205181–205189.
[23]
Demetri Terzopoulos, John Platt, Alan Barr, and Kurt Fleischer. 1987. Elastically Deformable Models. In Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH ’87). Association for Computing Machinery, New York, NY, USA, 205–214. https://doi.org/10.1145/37401.37427
[24]
Lokender Tiwari and Brojeshwar Bhowmick. 2023. GarSim: Particle Based Neural Garment Simulator. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 4472–4481.
[25]
Yunjin Tong, Shiying Xiong, Xingzhe He, Guanghan Pan, and Bo Zhu. 2021. Symplectic neural networks in Taylor series form for Hamiltonian systems. J. Comput. Phys. 437 (2021), 110325.
[26]
Pascal Volino, Nadia Magnenat-Thalmann, and Francois Faure. 2009. A simple approach to nonlinear tensile stiffness for accurate cloth simulation. ACM Transactions on Graphics 28, 4 (2009), Article–No.
[27]
Huamin Wang, James F O’Brien, and Ravi Ramamoorthi. 2011. Data-driven elastic models for cloth: modeling and measurement. ACM transactions on graphics (TOG) 30, 4 (2011), 1–12.
[28]
Tuanfeng Y. Wang, Tianjia Shao, Kai Fu, and Niloy J. Mitra. 2019. Learning an Intrinsic Garment Space for Interactive Authoring of Garment Animation. ACM Trans. Graph. 38, 6, Article 220 (nov 2019), 12 pages. https://doi.org/10.1145/3355089.3356512
[29]
Martin Wicke, Daniel Ritchie, Bryan M Klingner, Sebastian Burke, Jonathan R Shewchuk, and James F O’Brien. 2010. Dynamic local remeshing for elastoplastic simulation. ACM Transactions on graphics (TOG) 29, 4 (2010), 1–11.
[30]
Meng Zhang, Duygu Ceylan, and Niloy J Mitra. 2022. Motion guided deep dynamic 3d garments. ACM Transactions on Graphics (TOG) 41, 6 (2022), 1–12.
[31]
Meng Zhang, Tuanfeng Wang, Duygu Ceylan, and Niloy J. Mitra. 2021. Deep Detail Enhancement for Any Garment. Computer Graphics Forum 40, 2 (2021), 399–411. https://doi.org/10.1111/cgf.142642 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.142642
Index Terms
- MeshGraphNetRP: Improving Generalization of GNN-based Cloth Simulation
Recommendations
Faking dynamics of cloth animation for animated films
AMDO'10: Proceedings of the 6th international conference on Articulated motion and deformable objectsIn this paper we argue for the concept of fake dynamics to allow animators to interactively create visually pleasing animations of cloth models while keeping him/her in full control of the animation process. Existing animation and simulation techniques ...
Faking Dynamics of Cloth Animation for Animated Films
6th International Conference on Articulated Motion and Deformable Objects - Volume 6169In this paper we argue for the concept of fake dynamics to allow animators to interactively create visually pleasing animations of cloth models while keeping him/her in full control of the animation process. Existing animation and simulation techniques ...
GPU Cloth Simulation Pipeline in Lightchaser Animation Studio
SA '21: SIGGRAPH Asia 2021 Technical CommunicationsWe present the simulation pipeline of character effects in Lightchaser Animation Studio and how we utilize GPU resources to accelerate clothing simulations. Many characters from ancient Chinese tales in our films are in complex costumes.(Figure 1) Such ...
Comments
Information & Contributors
Information
Published In
November 2023
224 pages
ISBN:9798400703935
DOI:10.1145/3623264
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 15 November 2023
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Funding Sources
- NCSoft
- NST
Conference
MIG '23
Sponsor:
MIG '23: The 16th ACM SIGGRAPH Conference on Motion, Interaction and Games
November 15 - 17, 2023
Rennes, France
Acceptance Rates
Overall Acceptance Rate -9 of -9 submissions, 100%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 264Total Downloads
- Downloads (Last 12 months)264
- Downloads (Last 6 weeks)12
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
Cited By
View allView Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format