Image-based Virtual Try-on via Channel Attention and Appearance Flow
Authors: 
Chao He, Rong Liu, Jinxuan E, Ming LiuAuthors Info & Claims
Chao He, Rong Liu, Jinxuan E, Ming LiuAuthors Info & ClaimsPages 198 - 203
Abstract
Virtual try-on is an image generation task for changing characters' clothes while preserving the characters' and the cloth's original attributes. Existing methods usually apply the traditional appearance flow method, which is susceptible to complex body postures or occlusions, leading to unclear texture of target clothing or distorted limbs of characters. We apply a StyleGAN-based(generative adversarial network) flow generator to estimate appearance flow, which provides more global information to overcome the issue. Additionally, more local information is captured to refine the appearance flow by adopting the channel attention mechanism. Qualitative and quantitative experiments demonstrate that our model can generate more realistic images.
References
[1]
Peng Guan, Loretta Reiss, David A Hirshberg, Alexander Weiss, and Michael J Black. DRAPE: Dressing Any Person. ACM Transactions on Graphics, 31(4):1–10, 2012.
[2]
Chaitanya Patel, Zhouyingcheng Liao, and Gerard Pons Moll. TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style. In CVPR, 2020.
[3]
Gerard Pons-Moll, Sergi Pujades, Sonny Hu, and Michael Black. ClothCap: Seamless 4D Clothing Capture and Retargeting. ACM Transactions on Graphics, 36(4):1–15, 2017.
[4]
Igor Santesteban, Miguel A Otaduy, and Dan Casas. Learning-based Animation of Clothing for Virtual Try-on.Computer Graphics Forum, 38(2):355–366, 2019.
[5]
Masahiro Sekine, Kaoru Sugita, Frank Perbet, Bjorn Stenger, and Masashi Nishiyama. Virtual fitting by single-shot body shape estimation. In Int. Conf. on 3D Body Scanning Technologies, pages 406–413. Citeseer, 2014.
[6]
Xintong Han, Zuxuan Wu, Zhe Wu, Ruichi Y u, and Larry S Davis. Viton: An image-based virtual try-on network. In CVPR, 2018.
[7]
Xintong Han, Xiaojun Hu, Weilin Huang, and Matthew R Scott. Clothflow: A flow-based model for clothed person generation. In ICCV, 2019.
[8]
Bochao Wang, Huabin Zheng, Xiaodan Liang, Yimin Chen, Liang Lin, and Meng Y ang. Toward characteristic preserving image-based virtual try-on network. In ECCV,2018.
[9]
Ruiyun Y u, Xiaoqi Wang, and Xiaohui Xie. Vtnfp: An image-based virtual try-on network with body and clothing feature preservation. In ICCV, 2019.
[10]
Han Yang, Ruimao Zhang, Xiaobao Guo, Wei Liu, Wangmeng Zuo, and Ping Luo. Towards photo-realistic virtual try-on by adaptively generating-preserving image content. In CVPR, 2020.
[11]
Matiur Rahman Minar, Thai Thanh Tuan, Heejune Ahn, Paul Rosin, and Y u-Kun Lai. Cp-vton+: Clothing shape and texture preserving image-based virtual try-on. In CVPR Workshops, June 2020.
[12]
Thibaut Issenhuth, J. Mary, and Clément Calauzènes. Donot mask what you do not need to mask: a parser-free virtual try-on. In ECCV, 2020.
[13]
Yuying Ge, Yibing Song, Ruimao Zhang, Chongjian Ge, Wei Liu, and Ping Luo. Parser-free virtual try-on via distilling appearance flows. In CVPR, 2021.
[14]
Jean Duchon. Splines minimizing rotation-invariant seminorms in so-bolev spaces. In Constructive theory of functions of several variables, pages 85–100. Springer, 1977.
[15]
Chongjian Ge, Yibing Song, Y uying Ge, Han Yang, Wei Liu, and Ping Luo. Disentangled cycle consistency for highly-realistic virtual try-on. In CVPR, 2021.
[16]
Thibaut Issenhuth, Jérémie Mary, and Clément Calauzenes.Do not mask what you do not need to mask: a parser-free virtual try-on. In ECCV, 2020.
[17]
Ayush Chopra, Rishabh Jain, Mayur Hemani, and Balaji Krishnamurthy. Zflow: Gated appearance flow-based virtual try-on with 3d priors. In ICCV, 2021.
[18]
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR,2015.
[19]
Sen He, Wentong Liao, Michael Ying Yang, Yi-Zhe Song,Bodo Rosenhahn, and Tao Xiang. Disentangled lifespan face synthesis. In ICCV, 2021.
[20]
Roy Or-El, Soumyadip Sengupta, Ohad Fried, Eli Shechtman, and Ira Kemelmacher-Shlizerman. Lifespan age transformation synthesis. In ECCV, pages 739–755, 2020.
[21]
Y ujun Shen, Ceyuan Yang, Xiaoou Tang, and Bolei Zhou. Interfacegan: Interpreting the disentangled face representation learned by gans. TPAMI, 2020.
[22]
Y ujun Shen and Bolei Zhou. Closed-form factorization of latent semantics in gans. In CVPR, 2021.
[23]
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In CVPR, 2019.
[24]
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner,Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NeurIPS, 2017.
[25]
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. TIP, 13(4):600–612, 2004.
[26]
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In NeurIPS, 2016.
[27]
He, Sen and Song, Yi-Zhe and Xiang, Tao.Style-Based Global Appearance Flow for Virtual Try-On. In CVPR,2022
[28]
Foivos I Diakogiannis, Franc ¸ois Waldner, Peter Caccetta,and Chen Wu. Resunet-a: a deep learning framework for semantic segmentation of remotely sensed data. In ISPRS, 162,2020.
Recommendations
Virtual Try-On through Image-Based Rendering
Virtual try-on applications have become popular because they allow users to watch themselves wearing different clothes without the effort of changing them physically. This helps users to make quick buying decisions and, thus, improves the sales ...
Single Stage Virtual Try-On Via Deformable Attention Flows
Computer Vision – ECCV 2022AbstractVirtual try-on aims to generate a photo-realistic fitting result given an in-shop garment and a reference person image. Existing methods usually build up multi-stage frameworks to deal with clothes warping and body blending respectively, or rely ...
Three stages of 3D virtual try-on network with appearance flow and shape field
AbstractThe virtual try-on technology can satisfy the demands for online shopping and help consumers experience online clothes through image generation technology. Compared with image-based try-on, the 3D virtual try-on methods can realize the multi-...
Comments
Information & Contributors
Information
Published In
May 2024
668 pages
ISBN:9798400716751
DOI:10.1145/3670105
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 29 July 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
CNIOT 2024
CNIOT 2024: 2024 5th International Conference on Computing, Networks and Internet of Things
May 24 - 26, 2024
Tokyo, Japan
Acceptance Rates
Overall Acceptance Rate 39 of 82 submissions, 48%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 9Total Downloads
- Downloads (Last 12 months)9
- Downloads (Last 6 weeks)3
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format