short-paper

Open access

SLGAN: Style- and Latent-Guided Generative Adversarial Network for Desirable Makeup Transfer and Removal

Authors:

Kiyoharu AizawaAuthors Info & Claims

MMAsia '22: Proceedings of the 4th ACM International Conference on Multimedia in Asia

Article No.: 29, Pages 1 - 5

https://doi.org/10.1145/3551626.3564967

Published: 13 December 2022 Publication History

Abstract

There are five features to consider when using generative adversarial networks to apply makeup to photos of the human face. These features include (1) facial components, (2) interactive color adjustments, (3) makeup variations, (4) robustness to poses and expressions, and the (5) use of multiple reference images. To tackle the key features, we propose a novel style- and latent-guided makeup generative adversarial network for makeup transfer and removal. We provide a novel, perceptual makeup loss and a style-invariant decoder that can transfer makeup styles based on histogram matching to avoid the identity-shift problem. In our experiments, we show that our SLGAN is better than or comparable to state-of-the-art methods. Furthermore, we show that our proposal can interpolate facial makeup images to determine the unique features, compare existing methods, and help users find desirable makeup configurations.

References

[1]

Huiwen Chang, Jingwan Lu, Fisher Yu, and Adam Finkelstein. 2018. PairedCycleGAN: Asymmetric Style Transfer for Applying and Removing Makeup. In CVPR.

[2]

Hung-Jen Chen, Ka-Ming Hui, Szu-Yu Wang, Li-Wu Tsao, Hong-Han Shuai, and Wen-Huang Cheng. 2019. BeautyGlow: On-Demand Makeup Transfer Framework With Reversible Generative Network. In CVPR.

[3]

Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016. InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets. arXiv:1606.03657

[4]

Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha. 2020. StarGAN v2: Diverse Image Synthesis for Multiple Domains. In CVPR.

[5]

Qiao Gu, Guanzhi Wang, Mang Tik Chiu, Yu-Wing Tai, and Chi-Keung Tang. 2019. LADN: Local Adversarial Disentangling Network for Facial Makeup and De-Makeup. In ICCV.

[6]

Xun Huang and Serge Belongie. 2017. Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization. In ICCV.

[7]

Xun Huang, Ming-Yu Liu, Serge Belongie, and Jan Kautz. 2018. Multimodal Unsupervised Image-to-Image Translation. In ECCV.

[8]

Wentao Jiang, Si Liu, Chen Gao, Jie Cao, Ran He, Jiashi Feng, and Shuicheng Yan. 2020. PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer. In CVPR.

[9]

Hsin-Ying Lee, Hung-Yu Tseng, Jia-Bin Huang, Maneesh Kumar Singh, and Ming-Hsuan Yang. 2018. Diverse Image-to-Image Translation via Disentangled Representations. In ECCV.

[10]

Tingting Li, Ruihe Qian, Chao Dong, Si Liu, Qiong Yan, Wenwu Zhu, and Liang Lin. 2018. BeautyGAN: Instance-Level Facial Makeup Transfer with Deep Generative Adversarial Network. In ACMMM.

[11]

Jing Liao, Yuan Yao, Lu Yuan, Gang Hua, and Sing Bing Kang. 2017. Visual Attribute Transfer through Deep Image Analogy. ACM Trans. Graph. 36, 4 (2017).

Digital Library

[12]

Qi Mao, Hsin-Ying Lee, Hung-Yu Tseng, Siwei Ma, and Ming-Hsuan Yang. 2019. Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis. In CVPR.

[13]

Dingdong Yang, Seunghoon Hong, Yunseok Jang, Tianchen Zhao, and Honglak Lee. 2019. Diversity-Sensitive Conditional Generative Adversarial Networks. In ICLR.

[14]

Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkss. In ICCV.

[15]

Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A. Efros, Oliver Wang, and Eli Shechtman. 2017. Toward Multimodal Image-to-Image Translation. In NeurIPS.

Cited By

Liu JZheng ZYang ZYang Y(2024)High Fidelity Makeup via 2D and 3D Identity Preservation NetACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365647520:8(1-24)Online publication date: 13-Jun-2024
https://dl.acm.org/doi/10.1145/3656475
Ahmad ZJaffri ZChen MBao S(2024)Understanding GANs: fundamentals, variants, training challenges, applications, and open problemsMultimedia Tools and Applications10.1007/s11042-024-19361-yOnline publication date: 14-May-2024
https://doi.org/10.1007/s11042-024-19361-y
Guo JLi XYang PBai M(2024)HR-CycleGAN: Face highlight reduction based on improved cycle-consistent adversarial networksMultimedia Tools and Applications10.1007/s11042-024-18188-xOnline publication date: 24-Jan-2024
https://doi.org/10.1007/s11042-024-18188-x
Show More Cited By

Index Terms

SLGAN: Style- and Latent-Guided Generative Adversarial Network for Desirable Makeup Transfer and Removal
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Image representations

Recommendations

Beautify As You Like
MM '20: Proceedings of the 28th ACM International Conference on Multimedia

Customizable makeup transfer, which aims to transfer the makeup from an arbitrary reference face to a source face, is widely demanded in many applications such as short video platforms and online meeting applications. However, existing methods are ...
TSEV-GAN: Generative Adversarial Networks with Target-aware Style Encoding and Verification for facial makeup transfer
Abstract
Generative Adversarial Networks (GANs) have brought great progress in image-to-image translation. The problem that we focus on is how to accurately extract and transfer the makeup style from a reference facial image to a target face. ...
TC-GAN: Triangle Cycle-Consistent GANs for Face Frontalization with Facial Features Preserved
MM '19: Proceedings of the 27th ACM International Conference on Multimedia

Face frontalization has always been an important field. Recently, with the introduction of generative adversarial networks (GANs), face frontalization has achieved remarkable success. A critical challenge during face frontalization is to ensure the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MMAsia '22: Proceedings of the 4th ACM International Conference on Multimedia in Asia

December 2022

296 pages

ISBN:9781450394789

DOI:10.1145/3551626

Conference Chair:
Shuqiang Jiang
CASROLE@GENERAL CHAIR
,
General Chairs:
Kiyoharu Aizawa
The University of Tokyo
,
Phoebe Chen
La Trobe
,
Keiji Yanai
The University of Electro-Communications

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 December 2022

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

MMAsia '22

Sponsor:

SIGMM

MMAsia '22: ACM Multimedia Asia

December 13 - 16, 2022

Tokyo, Japan

Acceptance Rates

Overall Acceptance Rate 59 of 204 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
304
Total Downloads

Downloads (Last 12 months)112
Downloads (Last 6 weeks)21

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu JZheng ZYang ZYang Y(2024)High Fidelity Makeup via 2D and 3D Identity Preservation NetACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365647520:8(1-24)Online publication date: 13-Jun-2024
https://dl.acm.org/doi/10.1145/3656475
Ahmad ZJaffri ZChen MBao S(2024)Understanding GANs: fundamentals, variants, training challenges, applications, and open problemsMultimedia Tools and Applications10.1007/s11042-024-19361-yOnline publication date: 14-May-2024
https://doi.org/10.1007/s11042-024-19361-y
Guo JLi XYang PBai M(2024)HR-CycleGAN: Face highlight reduction based on improved cycle-consistent adversarial networksMultimedia Tools and Applications10.1007/s11042-024-18188-xOnline publication date: 24-Jan-2024
https://doi.org/10.1007/s11042-024-18188-x
Raju BSriram MGanesan V(2023)Aspect Based Text Summarization Model for the E-Commerce Recommendation System2023 3rd International Conference on Mobile Networks and Wireless Communications (ICMNWC)10.1109/ICMNWC60182.2023.10435796(1-6)Online publication date: 4-Dec-2023
https://doi.org/10.1109/ICMNWC60182.2023.10435796

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten