Reference-based Face Editing with Spherical Harmonic Illumination and Geometry Improvement
Article No.: 45, Pages 1 - 7
Abstract
Face editing is a topic in computer vision. To accurately meet the demands of users, many researchers focus on the reference-based face editing task which aims to manipulate an original image from reference attributes. However, existing methods tend to have illumination information mistakes and geometry structure missing issues. In this paper, we present an illumination and geometry-improved framework. The framework consists of two vital parts: the illumination adjustment module and the geometry improvement part. The illumination adjustment module is introduced to predict spherical harmonic lighting code concatenating with the original style code to combine more illumination details. And the geometry improvement part serves as an extra disentangled geometry feature provider and a combiner to achieve a fine-grained blended image. Extensive results demonstrate that our method has significant improvements in illumination and geometry preservation.
References
[1]
Abdal R, Zhu P, Mitra N J, Styleflow: Attribute-conditioned exploration of stylegan-generated images using conditional continuous normalizing flows[J]. ACM Transactions on Graphics (TOG), 2021, 40(3): 1-21.
[2]
Chen S Y, Liu F L, Lai Y K, DeepFaceEditing: deep face generation and editing with disentangled geometry and appearance control[J]. arXiv preprint arXiv:2105.08935, 2021.
[3]
Portenier T, Hu Q, Szabo A, Faceshop: Deep sketch-based face image editing[J]. arXiv preprint arXiv:1804.08972, 2018.
[4]
Patashnik O, Wu Z, Shechtman E, Styleclip: Text-driven manipulation of stylegan imagery[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 2085-2094.
[5]
Wang H, Lin G, Hoi S C H, Cycle-Consistent Inverse GAN for Text-to-Image Synthesis[J]. arXiv preprint arXiv:2108.01361, 2021.
[6]
Li X, Zhang S, Hu J, Image-to-image translation via hierarchical style disentanglement[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 8639-8648.
[7]
Karras T, Laine S, Aila T. A style-based generator architecture for generative adversarial networks[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 4401-4410.
[8]
Karras T, Laine S, Aittala M, Analyzing and improving the image quality of stylegan[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 8110-8119.
[9]
Yu J, Lin Z, Yang J, Free-form image inpainting with gated convolution[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 4471-4480.
[10]
Jo Y, Park J. Sc-fegan: Face editing generative adversarial network with user's sketch and color[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2019: 1745-1753.
[11]
Xiao T, Hong J, Ma J. Elegant: Exchanging latent encodings with gan for transferring multiple face attributes[C]//Proceedings of the European conference on computer vision (ECCV). 2018: 168-184.
[12]
He K, Zhang X, Ren S, Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778. https://doi.org/10.1109/cvpr.2016.90
[13]
Zhu J Y, Park T, Isola P, Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE international conference on computer vision. 2017: 2223-2232.
[14]
Zhou H, Hadap S, Sunkavalli K, Deep single-image portrait relighting[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 7194-7202.
[15]
Sengupta S, Kanazawa A, Castillo C D, Sfsnet: Learning shape, reflectance and illuminance of faces in the wild[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 6296-6305.
[16]
Salimans T, Goodfellow I, Zaremba W, Improved techniques for training gans[J]. Advances in neural information processing systems, 2016, 29.
[17]
Heusel M, Ramsauer H, Unterthiner T, Gans trained by a two time-scale update rule converge to a local nash equilibrium[J]. Advances in neural information processing systems, 2017, 30: 6629–6640.
[18]
Zhou W. Image quality assessment: from error measurement to structural similarity[J]. IEEE transactions on image processing, 2004, 13: 600-612.
Index Terms
- Reference-based Face Editing with Spherical Harmonic Illumination and Geometry Improvement
Recommendations
Acquiring reflectance and shape from continuous spherical harmonic illumination
We present a novel technique for acquiring the geometry and spatially-varying reflectance properties of 3D objects by observing them under continuous spherical harmonic illumination conditions. The technique is general enough to characterize either ...
Analytic spherical harmonic coefficients for polygonal area lights
Spherical Harmonic (SH) lighting is widely used for real-time rendering within Precomputed Radiance Transfer (PRT) systems. SH coefficients are precomputed and stored at object vertices, and combined interactively with SH lighting coefficients to enable ...
Virtual Spherical Gaussian Lights for Real-time Glossy Indirect Illumination
Virtual point lights VPLs are well established for real-time global illumination. However, this method suffers from spiky artifacts and flickering caused by singularities of VPLs, highly glossy materials, high-frequency textures, and discontinuous ...
Comments
Information & Contributors
Information
Published In
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 December 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
CSAE 2022
CSAE 2022: The 6th International Conference on Computer Science and Application Engineering
October 21 - 23, 2022
Virtual Event, China
Acceptance Rates
Overall Acceptance Rate 368 of 770 submissions, 48%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 43Total Downloads
- Downloads (Last 12 months)5
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format