Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Physics-Inspired Garment Recovery from a Single-View Image

Published: 19 November 2018 Publication History

Abstract

Most recent garment capturing techniques rely on acquiring multiple views of clothing, which may not always be readily available, especially in the case of pre-existing photographs from the web. As an alternative, we propose a method that is able to compute a 3D model of a human body and its outfit from a single photograph with little human interaction. Our algorithm is not only able to capture the global shape and overall geometry of the clothing, it can also extract the physical properties (i.e., material parameters needed for simulation) of cloth. Unlike previous methods using full 3D information (i.e., depth, multi-view images, or sampled 3D geometry), our approach achieves garment recovery from a single-view image by using physical, statistical, and geometric priors and a combination of parameter estimation, semantic parsing, shape/pose recovery, and physics-based cloth simulation. We demonstrate the effectiveness of our algorithm by re-purposing the reconstructed garments for virtual try-on and garment transfer applications and for cloth animation on digital characters.

References

[1]
Ankur Agarwal and Bill Triggs. 2006. Recovering 3D human pose from monocular images. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 1 (2006), 44--58.
[2]
AliExpress. 2015. Homepage. Retrieved from http://www.aliexpress.com.
[3]
Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis. 2005. SCAPE: Shape completion and animation of people. ACM Transactions on Graphics (TOG) 24, 3 (2005), 408--416.
[4]
Anthropologie. 2015. Homepage. Retrieved from http://www.anthropologie.com.
[5]
Alexandru O. Balan, Leonid Sigal, Michael J. Black, James E. Davis, and Horst W. Haussecker. 2007. Detailed human shape and pose from images. In Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1--8.
[6]
David Baraff and Andrew Witkin. 1998. Large steps in cloth simulation. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques. ACM, 43--54.
[7]
Jo Barnfield. 2012. The Pattern Making Primer: All You Need to Know About Designing, Adapting and Customizing Sewing Patterns. Barron’s Educational Series, Inc, Hauppauge, NY.
[8]
Aric Bartle, Alla Sheffer, Vladimir G. Kim, Danny M. Kaufman, Nicholas Vining, and Floraine Berthouzoz. 2016. Physics-driven pattern adjustment for direct 3D garment editing. ACM Transactions on Graphics (TOG) 35, 4 (2016), 50--1.
[9]
Pascal Bérard, Derek Bradley, Maurizio Nitti, Thabo Beeler, and Markus Gross. 2014. High-quality capture of eyes. ACM Transactions on Graphics (TOG) 33, 6 (2014), 223.
[10]
Floraine Berthouzoz, Akash Garg, Danny M. Kaufman, Eitan Grinspun, and Maneesh Agrawala. 2013. Parsing sewing patterns into 3D garments. ACM Transactions on Graphics (TOG) 32, 4 (2013), 85.
[11]
Boden. 2015. Homepage. Retrieved from http://www.bodenusa.com.
[12]
Robert Bridson, Ronald Fedkiw, and John Anderson. 2002. Robust treatment of collisions, contact and friction for cloth animation. ACM Transactions on Graphics (TOG) 21, 3 (2002), 594--603.
[13]
Robert Bridson, Sebastian Marino, and Ronald Fedkiw. 2003. Simulation of clothing with folds and wrinkles. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Eurographics Association, 28--36.
[14]
Alexander M. Bronstein, Michael M Bronstein, and Ron Kimmel. 2006. Efficient computation of isometry-invariant distances between surfaces. SIAM Journal on Scientific Computing 28, 5 (2006), 1812--1836.
[15]
Alexander M. Bronstein, Michael M. Bronstein, and Ron Kimmel. 2008. Numerical Geometry of Non-rigid Shapes. Springer Science 8 Business Media.
[16]
Remi Brouet, Alla Sheffer, Laurence Boissieux, and Marie-Paule Cani. 2012. Design preserving garment transfer. ACM Transactions on Graphics 31, 4 (2012), 36.
[17]
Chen Cao, Yanlin Weng, Stephen Lin, and Kun Zhou. 2013. 3D shape regression for real-time facial animation. ACM Transactions on Graphics (TOG) 32, 4 (2013), 41.
[18]
Romain Casati, Gilles Daviet, and Florence Bertails-Descoubes. 2016. Inverse Elastic Cloth Design with Contact and Friction. Ph.D. Dissertation. Inria Grenoble Rhône-Alpes, Université de Grenoble.
[19]
Menglei Chai, Lvdi Wang, Yanlin Weng, Yizhou Yu, Baining Guo, and Kun Zhou. 2012. Single-view hair modeling for portrait manipulation. ACM Transactions on Graphics 31, 4 (July 2012), Article 116, 8 pages.
[20]
Xiaowu Chen, Yu Guo, Bin Zhou, and Qinping Zhao. 2013. Deformable model for estimating clothed and naked human shapes from a single image. The Visual Computer 29, 11 (2013), 1187--1196.
[21]
Xiaowu Chen, Bin Zhou, Feixiang Lu, Lin Wang, Lang Bi, and Ping Tan. 2015. Garment modeling with a depth camera. ACM Transactions on Graphics (TOG) 34, 6 (2015), 203.
[22]
Sean Curtis, Rasmus Tamstorf, and Dinesh Manocha. 2008. Fast collision detection for deformable models using representative-triangles. In Proceedings of the 2008 Symposium on Interactive 3D Graphics and Games. ACM, 61--69.
[23]
Philippe Decaudin, Dan Julius, Jamie Wither, Laurence Boissieux, Alla Sheffer, and Marie-Paule Cani. 2006. Virtual garments: A fully geometric approach for clothing design. Computer Graphics Forum 25, 625--634.
[24]
Elliot English and Robert Bridson. 2008. Animating developable surfaces using nonconforming elements. ACM Transactions on Graphics (TOG) 27, 3 (2008), 66.
[25]
C. Farabet, C. Couprie, L. Najman, and Y. LeCun. 2013. Learning hierarchical features for scene labeling. Pattern Analysis and Machine Intelligence 35, 8 (2013), 1915--1929.
[26]
FashionableShoes. 2013. Homepage. Retrieved from http://bestfashionableshoess.blogspot.com.
[27]
FashionUnited. 2016. Global Fashion Industry Statistics—International Apparel. https://fashionunited.com/global-fashion-industry-statistics.
[28]
Rony Goldenthal, David Harmon, Raanan Fattal, Michel Bercovier, and Eitan Grinspun. 2007. Efficient simulation of inextensible cloth. ACM Transactions on Graphics (TOG) 26, 3 (2007), 49.
[29]
Naga K. Govindaraju, Ilknur Kabul, Ming C. Lin, and Dinesh Manocha. 2007. Fast continuous collision detection among deformable models using graphics processors. Computers 8 Graphics 31, 1 (2007), 5--14.
[30]
Nils Hasler, Mark Asbach, Bodo Rosenhahn, Jens-Rainer Ohm, and Hans-Peter Seidel. 2006. Physically based tracking of cloth. In Proceedings of the International Workshop on Vision, Modeling, and Visualization, VMV. 49--56.
[31]
Nils Hasler, Carsten Stoll, Martin Sunkel, Bodo Rosenhahn, and H.-P. Seidel. 2009. A statistical model of human pose and body shape. Computer Graphics Forum 28, 337--346.
[32]
Hillsweddingdress. 2015. Homepage. Retrieved from http://hillsweddingdress.xyz.
[33]
Donald H. House and David E. Breen. 2000. Cloth Modeling and Animation. AK Peters.
[34]
Takeo Igarashi, Tomer Moscovich, and John F. Hughes. 2005. As-rigid-as-possible shape manipulation. ACM Transactions on Graphics (TOG) 24, 3 (2005), 1134--1141.
[35]
Michal Jancosek and Tomás Pajdla. 2011. Multi-view reconstruction preserving weakly-supported surfaces. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 3121--3128.
[36]
Moon-Hwan Jeong, Dong-Hoon Han, and Hyeong-Seok Ko. 2015. Garment capture from a photograph. Computer Animation and Virtual Worlds 26, 3–4 (2015), 291--300.
[37]
Ladislav Kavan, P.-P. Sloan, and Carol O’Sullivan. 2010. Fast and efficient skinning of animated meshes. Computer Graphics Forum 29, 327--336.
[38]
James Kennedy. 2010. Particle swarm optimization. In Encyclopedia of Machine Learning. Springer, 760--766.
[39]
Anat Levin, Dani Lischinski, and Yair Weiss. 2008. A closed-form solution to natural image matting. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 2 (2008), 228--242.
[40]
Hao Li, Linjie Luo, Daniel Vlasic, Pieter Peers, Jovan Popović, Mark Pauly, and Szymon Rusinkiewicz. 2012. Temporally coherent completion of dynamic shapes. ACM Transactions on Graphics 31, 1 (January 2012), 2.
[41]
Yin Li, Jian Sun, Chi-Keung Tang, and Heung-Yeung Shum. 2004. Lazy snapping. In ACM Transactions on Graphics (TOG) 23, 3 (2004), 303--308.
[42]
Matt Lindner. 2015. Global E-commerce Sales Set to Grow 25% in 2015. Retrieved from https://www.internetretailer.com/2015/07/29/global-e-commerce-set-grow-25-2015.
[43]
Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. CVPR (to Appear) (Nov. 2015). arxiv:cs.CV/1411.4038
[44]
Yuwei Meng, Charlie C. L. Wang, and Xiaogang Jin. 2012. Flexible shape control for automatic resizing of apparel products. Computer-Aided Design 44, 1 (2012), 68--76.
[45]
ModCloth. 2015. Homepage. Retrieved from http://www.modcloth.com.
[46]
Thomas B. Moeslund, Adrian Hilton, and Volker Krüger. 2006. A survey of advances in vision-based human motion capture and analysis. Computer Vision and Image Understanding 104, 2 (2006), 90--126.
[47]
Koki Nagano, Graham Fyffe, Oleg Alexander, Jernej Barbiç, Hao Li, Abhijeet Ghosh, and Paul Debevec. 2015. Skin microstructure deformation with displacement map convolution. ACM Transactions on Graphics 34, 4, Article 109 (July 2015), 10 pages.
[48]
Rahul Narain, Armin Samii, and James F. O’Brien. 2012. Adaptive anisotropic remeshing for cloth simulation. ACM Transactions on Graphics (TOG) 31, 6 (2012), 152.
[49]
Hing N. Ng and Richard L. Grimsdale. 1996. Computer graphics techniques for modeling cloth. IEEE Computer Graphics and Applications 16, 5 (1996), 28--41.
[50]
P. H. Pinheiro and R. Collobert. 2014. Recurrent convolutional neural networks for scene labeling. In ICML.
[51]
Tiberiu Popa, Quan Zhou, Derek Bradley, Vladislav Kraevoy, Hongbo Fu, Alla Sheffer, and Wolfgang Heidrich. 2009. Wrinkling captured garments using space-time data-driven deformation. Computer Graphics Forum 28, 427--435.
[52]
Dimitris Protopsaltou, Christiane Luible, Marlene Arevalo, and Nadia Magnenat-Thalmann. 2002. A Body and Garment Creation Method for an Internet Based Virtual Fitting Room. Springer.
[53]
RedBubble. 2015. Homepage. Retrieved from http://www.redbubble.com.
[54]
Cody Robson, Ron Maharik, Alla Sheffer, and Nathan Carr. 2011. Context-aware garment modeling from sketches. Computers 8 Graphics 35, 3 (2011), 604--613.
[55]
Damien Rohmer, Tiberiu Popa, Marie-Paule Cani, Stefanie Hahmann, and Alla Sheffer. 2010. Animation wrinkling: Augmenting coarse cloth simulations with realistic-looking wrinkles. In ACM Transactions on Graphics (TOG) 29, 6 (2010), 157.
[56]
Saaclothes. 2015. Retrieved from http://www.saaclothes.com.
[57]
Volker Scholz and Marcus Magnor. 2006. Texture replacement of garments in monocular video sequences. In Proceedings of the 17th Eurographics Conference on Rendering Techniques. Eurographics Association, 305--312.
[58]
Volker Scholz, Timo Stich, Michael Keckeisen, Markus Wacker, and Marcus Magnor. 2005. Garment motion capture using color-coded patterns. Computer Graphics Forum 24, 439--447.
[59]
Hyewon Seo and Nadia Magnenat-Thalmann. 2003. An automatic modeling of human bodies from sizing parameters. In Proceedings of the 2003 Symposium on Interactive 3D Graphics. ACM, 19--26.
[60]
Leonid Sigal, Moshe Mahler, Spencer Diaz, Kyna McIntosh, Elizabeth Carter, Timothy Richards, and Jessica Hodgins. 2015. A perceptual control space for garment simulation. ACM Transactions on Graphics (TOG) 34, 4 (2015), 117.
[61]
Robert W. Sumner and Jovan Popović. 2004. Deformation transfer for triangle meshes. ACM Transactions on Graphics (TOG) 23, 3 (2004), 399--405.
[62]
Min Tang, Sean Curtis, Sung-Eui Yoon, and Dinesh Manocha. 2009. ICCD: Interactive continuous collision detection between deformable models using connectivity-based culling. IEEE Transactions on Visualization and Computer Graphics 15, 4 (2009), 544--557.
[63]
Hiroaki Tanie, Katsu Yamane, and Yoshihiko Nakamura. 2005. High marker density motion capture by retroreflective mesh suit. In Proceedings of the 2005 IEEE International Conference on Robotics and Automation. ICRA 2005. IEEE, 2884--2889.
[64]
Gabriel Taubin. 1995. A signal processing approach to fair surface design. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’95). ACM, New York, NY, 351--358.
[65]
Camillo J. Taylor. 2000. Reconstruction of articulated objects from point correspondences in a single uncalibrated image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1. IEEE, 677--684.
[66]
Bernhard Thomaszewski, Simon Pabst, and Wolfgang Strasser. 2009. Continuum-based strain limiting. Computer Graphics Forum 28, 569--576.
[67]
Emmanuel Turquin, Jamie Wither, Laurence Boissieux, Marie-Paule Cani, and John F Hughes. 2007. A sketch-based interface for clothing virtual characters. IEEE Computer Graphics and Applications1 (2007), 72--81.
[68]
Frans Van den Bergh and Andries Petrus Engelbrecht. 2006. A study of particle swarm optimization particle trajectories. Information sciences 176, 8 (2006), 937--971.
[69]
Pascal Volino and Nadia Magnenat-Thalmann. 1999. Fast geometrical wrinkles on animated surfaces. In Proceedings of the 7th International Conference in Central Europe on Computer Graphics and Visualization (Winter School on Computer Graphics).
[70]
Charlie CL Wang, Yu Wang, and Matthew MF Yuen. 2005. Design automation for customized apparel products. Computer-Aided Design 37, 7 (2005), 675--691.
[71]
Huamin Wang, James O’Brien, and Ravi Ramamoorthi. 2010. Multi-resolution isotropic strain limiting. ACM Transactions on Graphics (TOG) 29, 156.
[72]
Huamin Wang, James F. O’Brien, and Ravi Ramamoorthi. 2011. Data-driven elastic models for cloth: Modeling and measurement. ACM Transactions on Graphics (TOG) 30, 4 (2011), 71.
[73]
Jerry Weil. 1986. The synthesis of cloth objects. ACM Siggraph Computer Graphics 20, 4 (1986), 49--54.
[74]
Ryan White, Keenan Crane, and David A. Forsyth. 2007. Capturing and animating occluded cloth. ACM Transactions on Graphics (TOG), Vol. 26. ACM, 34.
[75]
Changchang Wu. 2011. VisualSFM: A visual structure from motion system. Retreived from http://homes.cs.washington.edu/ccwu/vsfm 9 (2011).
[76]
Changchang Wu. 2013. Towards linear-time incremental structure from motion. In Proceedings of the 2013 International Conference on 3D Vision. 127--134.
[77]
Saining Xie and Zhuowen Tu. 2015. Holistically-nested edge detection. In Proceedings of the IEEE International Conference on Computer Vision. 1395--1403.
[78]
Kazuhiro Yamaguchi, Mohammad Hadi Kiapour, and Tamara Berg. 2013. Paper doll parsing: Retrieving similar styles to parse clothing items. In Proceedings of the 2013 IEEE International Conference on Computer Vision. IEEE, 3519--3526.
[79]
Yipin Yang, Yao Yu, Yu Zhou, Sidan Du, James Davis, and Ruigang Yang. 2014. Semantic parametric reshaping of human body models. In Proceedings of the 2014 2nd International Conference on 3D Vision, Vol. 2. IEEE, 41--48.
[80]
Mao Ye, Huamin Wang, Nianchen Deng, Xubo Yang, and Ruigang Yang. 2014. Real-time human pose and shape estimation for virtual try-on using a single commodity depth camera. IEEE Transactions on Visualization and Computer Graphics 20, 4 (2014), 550--559.
[81]
S. D. Young, B. D. Adelstein, and S. R. Ellis. 2007. Calculus of nonrigid surfaces for geometry and texture manipulation. IEEE Transactions on Visualization and Computer Graphics 13, 5 (2007), 902--913.
[82]
Wenyi Zhao, Rama Chellappa, P. Jonathon Phillips, and Azriel Rosenfeld. 2003. Face recognition: A literature survey. ACM Computing Surveys (CSUR) 35, 4 (2003), 399--458.
[83]
Bin Zhou, Xiaowu Chen, Qiang Fu, Kan Guo, and Ping Tan. 2013. Garment modeling from a single image. Computer Graphics Forum 32, 85--91.
[84]
Shizhe Zhou, Hongbo Fu, Ligang Liu, Daniel Cohen-Or, and Xiaoguang Han. 2010. Parametric reshaping of human bodies in images. ACM Transactions on Graphics (TOG), Vol. 29. ACM, 126.

Cited By

View all
  • (2024)Development of an emotional response model for hospital gown design using structural equation modelingAUTEX Research Journal10.1515/aut-2023-001024:1Online publication date: 16-Jan-2024
  • (2024)GarVerseLOD: High-Fidelity 3D Garment Reconstruction from a Single In-the-Wild Image using a Dataset with Levels of DetailsACM Transactions on Graphics10.1145/368792143:6(1-12)Online publication date: 19-Dec-2024
  • (2024)DressCode: Autoregressively Sewing and Generating Garments from Text GuidanceACM Transactions on Graphics10.1145/365814743:4(1-13)Online publication date: 19-Jul-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 37, Issue 5
October 2018
140 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/3278329
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 November 2018
Accepted: 01 March 2018
Revised: 01 December 2017
Received: 01 September 2016
Published in TOG Volume 37, Issue 5

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Garment reconstruction
  2. image-based modelling
  3. physiologically-based modelling

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

  • National Science Foundation CyberManufacturing Program

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)171
  • Downloads (Last 6 weeks)16
Reflects downloads up to 25 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Development of an emotional response model for hospital gown design using structural equation modelingAUTEX Research Journal10.1515/aut-2023-001024:1Online publication date: 16-Jan-2024
  • (2024)GarVerseLOD: High-Fidelity 3D Garment Reconstruction from a Single In-the-Wild Image using a Dataset with Levels of DetailsACM Transactions on Graphics10.1145/368792143:6(1-12)Online publication date: 19-Dec-2024
  • (2024)DressCode: Autoregressively Sewing and Generating Garments from Text GuidanceACM Transactions on Graphics10.1145/365814743:4(1-13)Online publication date: 19-Jul-2024
  • (2024)Automatic Digital Garment Initialization from Sewing PatternsACM Transactions on Graphics10.1145/365812843:4(1-12)Online publication date: 19-Jul-2024
  • (2024)Efficient Deformation Learning of Varied Garments with a Structure-Preserving Multilevel FrameworkProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/36512867:1(1-19)Online publication date: 13-May-2024
  • (2024)Inverse Garment and Pattern Modeling with a Differentiable SimulatorComputer Graphics Forum10.1111/cgf.1524943:7Online publication date: 7-Nov-2024
  • (2024)Practical Methods to Estimate Fabric Mechanics from MetadataComputer Graphics Forum10.1111/cgf.1502943:2Online publication date: 30-Apr-2024
  • (2024)Estimating Cloth Simulation Parameters From Tag Information and Cusick Drape TestComputer Graphics Forum10.1111/cgf.1502743:2Online publication date: 30-Apr-2024
  • (2024)Panelformer: Sewing Pattern Reconstruction from 2D Garment Images2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00051(443-452)Online publication date: 3-Jan-2024
  • (2024)Garment Recovery with Shape and Deformation Priors2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00157(1586-1595)Online publication date: 16-Jun-2024
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media