Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
survey

Generative Adversarial Networks for Face Generation: A Survey

Published: 03 December 2022 Publication History

Abstract

Recently, generative adversarial networks (GANs) have progressed enormously, which makes them able to learn complex data distributions in particular faces. More and more efficient GAN architectures have been designed and proposed to learn the different variations of faces, such as cross pose, age, expression, and style. These GAN-based approaches need to be reviewed, discussed, and categorized in terms of architectures, applications, and metrics. Several reviews that focus on the use and advances of GAN in general have been proposed. However, to the best of our knowledge, the GAN models applied to the face, which we call facial GANs, have never been addressed. In this article, we review facial GANs and their different applications. We mainly focus on architectures, problems, and performance evaluation with respect to each application and used datasets. More precisely, we review the progress of architectures and discuss the contributions and limits of each. Then, we expose the encountered problems of facial GANs and propose solutions to handle them. Additionally, as GAN evaluation has become a notable current defiance, we investigate the state-of-the-art quantitative and qualitative evaluation metrics and their applications. We conclude this work with a discussion on the face generation challenges and propose open research issues.

Supplementary Material

3527850.supp (3527850.supp.pdf)
Supplementary material

References

[1]
Mahla Abdolahnejad and Peter Xiaoping Liu. 2020. Deep learning for face image synthesis and semantic manipulations: A review and future perspectives. Artificial Intelligence Review 53, 8 (2020), 5847–5880.
[2]
Alankrita Aggarwal, Mamta Mittal, and Gopi Battineni. 2021. Generative adversarial network: An overview of theory and applications. International Journal of Information Management Data Insights 1, 1 (2021), 100004.
[3]
Hamed Alqahtani and Manolya Kavakli-Thorne. 2019. Adversarial disentanglement using latent classifier for pose-independent representation. In Proceedings of the International Conference on Image Analysis and Processing (ICIAP’19).
[4]
Hamed Alqahtani, Manolya Kavakli-Thorne, and Gulshan Kumar. 2019. Applications of generative adversarial networks (GANs): An updated review. Archives of Computational Methods in Engineering 28, 1 (2019), 1–28.
[5]
Grigory Antipov, Moez Baccouche, and Jean-Luc Dugelay. 2017. Face aging with conditional generative adversarial networks. In Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP’17). IEEE Los Alamitos, CA, 2089–2093.
[6]
Martin Arjovsky and Léon Bottou. 2017. Towards principled methods for training generative adversarial networks. arXiv preprint arXiv:1701.04862 (2017).
[7]
Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein generative adversarial networks. In Proceedings of the International Conference on Machine Learning. 214–223.
[8]
Sanjeev Arora, Rong Ge, Yingyu Liang, Tengyu Ma, and Yi Zhang. 2017. Generalization and equilibrium in generative adversarial nets (GANs). In Proceedings of the International Conference on Machine Learning. 224–232.
[9]
Sandipan Banerjee, Walter Scheirer, Kevin Bowyer, and Patrick Flynn. 2020. On hallucinating context and background pixels from a face mask using multi-scale GANs. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 300–309.
[10]
Aayush Bansal, Shugao Ma, Deva Ramanan, and Yaser Sheikh. 2018. Recycle-GAN: Unsupervised video retargeting. In Proceedings of the European Conference on Computer Vision (ECCV’18). 119–135.
[11]
Jianmin Bao, Dong Chen, Fang Wen, Houqiang Li, and Gang Hua. 2018. Towards open-set identity preserving face synthesis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6713–6722.
[12]
Samuel A. Barnett. 2018. Convergence problems with generative adversarial networks (GANs). arXiv preprint arXiv:1806.11382 (2018).
[13]
Shane Barratt and Rishi Sharma. 2018. A note on the inception score. arXiv preprint arXiv:1801.01973 (2018).
[14]
David Berthelot, Thomas Schumm, and Luke Metz. 2017. BEGAN: Boundary equilibrium generative adversarial networks. arXiv preprint arXiv:1703.10717 (2017).
[15]
Binod Bhattarai and Tae-Kyun Kim. 2020. Inducing optimal attribute representations for conditional GANs. In Proceedings of the European Conference on Computer Vision. 69–85.
[16]
Mikołaj Bińkowski, Danica J. Sutherland, Michael Arbel, and Arthur Gretton. 2018. Demystifying MMD GANs. arXiv preprint arXiv:1801.01401 (2018).
[17]
Navaneeth Bodla, Gang Hua, and Rama Chellappa. 2018. Semi-supervised FusedGAN for conditional image generation. In Proceedings of the European Conference on Computer Vision (ECCV’18). 669–683.
[18]
Ali Borji. 2019. Pros and cons of GAN evaluation measures. Computer Vision and Image Understanding 179 (2019), 41–65.
[19]
Andrew Brock, Jeff Donahue, and Karen Simonyan. 2018. Large scale GAN training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096 (2018).
[20]
Anpei Chen, Ruiyang Liu, Ling Xie, Zhang Chen, Hao Su, and Jingyi Yu. 2020. SofGAN: A portrait image generator with dynamic styling. arXiv preprint arXiv:2007.03780 (2020).
[21]
Chaofeng Chen, Xiaoming Li, Lingbo Yang, Xianhui Lin, Lei Zhang, and Kwan-Yee K. Wong. 2020. Progressive semantic-aware style transformation for blind face restoration. arXiv preprint arXiv:2009.08709 (2020).
[22]
Jiawei Chen, Janusz Konrad, and Prakash Ishwar. 2018. VGAN-based image representation learning for privacy-preserving facial expression recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 1570–1579.
[23]
Qifeng Chen and Vladlen Koltun. 2017. Photographic image synthesis with cascaded refinement networks. In Proceedings of the IEEE International Conference on Computer Vision. 1511–1520.
[24]
WenTing Chen, Xinpeng Xie, Xi Jia, and Linlin Shen. 2018. Texture deformation based generative adversarial networks for face editing. arXiv preprint arXiv:1812.09832 (2018).
[25]
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016. InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets. arXiv preprint arXiv:1606.03657 (2016).
[26]
Yizhen Chen and Haifeng Hu. 2019. An improved method for semantic image inpainting with GANs: Progressive inpainting. Neural Processing Letters 49, 3 (2019), 1355–1367.
[27]
Yu-An Chen, Wei-Che Chen, Chia-Po Wei, and Yu-Chiang Frank Wang. 2017. Occlusion-aware face inpainting via generative adversarial networks. In Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP’17). IEEE, Los Alamitos, CA, 1202–1206.
[28]
Shiyang Cheng, Michael Bronstein, Yuxiang Zhou, Irene Kotsia, Maja Pantic, and Stefanos Zafeiriou. 2019. MeshGAN: Non-linear 3D morphable models of faces. arXiv preprint arXiv:1903.10384 (2019).
[29]
Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. 2018. StarGAN: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8789–8797.
[30]
Grigorios G. Chrysos, Paolo Favaro, and Stefanos Zafeiriou. 2019. Motion deblurring of faces. International Journal of Computer Vision 127, 6 (2019), 801–823.
[31]
Edo Collins, Raja Bala, Bob Price, and Sabine Susstrunk. 2020. Editing in style: Uncovering the local semantics of gans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5771–5780.
[32]
Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sengupta, and Anil A. Bharath. 2018. Generative adversarial networks: An overview. IEEE Signal Processing Magazine 35, 1 (2018), 53–65.
[33]
Joachim D. Curtó, Irene C. Zarza, Fernando De La Torre, Irwin King, and Michael R. Lyu. 2017. High-resolution deep convolutional generative adversarial networks. arXiv preprint arXiv:1711.06491 (2017).
[34]
Jiankang Deng, Shiyang Cheng, Niannan Xue, Yuxiang Zhou, and Stefanos Zafeiriou. 2018. UV-GAN: Adversarial facial UV map completion for pose-invariant face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7093–7102.
[35]
Jia Deng, Gaoyang Pang, Zhiyu Zhang, Zhibo Pang, Huayong Yang, and Geng Yang. 2019. cGAN based facial expression recognition for human-robot interaction. IEEE Access 7 (2019), 9848–9859.
[36]
Yu Deng, Jiaolong Yang, Dong Chen, Fang Wen, and Xin Tong. 2020. Disentangled and controllable face image generation via 3D imitative-contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5154–5163.
[37]
Sourabh Dhere, Suresh B. Rathod, Sanket Aarankalle, Yash Lad, and Megh Gandhi. 2020. A review on face reenactment techniques. In Proceedings of the 2020 International Conference on Industry 4.0 Technology (I4Tech’20). IEEE, Los Alamitos, CA, 191–194.
[38]
Chris Donahue, Zachary C. Lipton, Akshay Balsubramani, and Julian McAuley. 2017. Semantically decomposing the latent spaces of generative adversarial networks. arXiv preprint arXiv:1705.07904 (2017).
[39]
Shiqing Fan and Ye Luo. 2021. Deblurring processor for motion-blurred faces based on generative adversarial networks. arXiv preprint arXiv:2103.02121 (2021).
[40]
Charlie Frogner, Chiyuan Zhang, Hossein Mobahi, Mauricio Araya-Polo, and Tomaso Poggio. 2015. Learning with a Wasserstein loss. arXiv preprint arXiv:1506.05439 (2015).
[41]
Ruohan Gao and Kristen Grauman. 2017. On-demand learning for deep image restoration. In Proceedings of the IEEE International Conference on Computer Vision. 1086–1095.
[42]
Zhongpai Gao, Juyong Zhang, Yudong Guo, Chao Ma, Guangtao Zhai, and Xiaokang Yang. 2020. Semi-supervised 3D face representation learning from unconstrained photo collections. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 348–349.
[43]
Jon Gauthier. 2014. Conditional generative adversarial nets for convolutional face generation. Class Project for Stanford CS231N: Convolutional Neural Networks for Visual Recognition, Winter Semester 2014, 5 (2014), 2.
[44]
Shiming Ge, Chenyu Li, Shengwei Zhao, and Dan Zeng. 2020. Occluded face recognition in the wild by identity-diversity inpainting. IEEE Transactions on Circuits and Systems for Video Technology 30, 10 (2020), 3387–3397.
[45]
Baris Gecer, Binod Bhattarai, Josef Kittler, and Tae-Kyun Kim. 2018. Semi-supervised adversarial learning to generate photorealistic face images of new identities from 3D morphable model. In Proceedings of the European Conference on Computer Vision (ECCV’18). 217–234.
[46]
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial networks. arXiv preprint arXiv:1406.2661 (2014).
[47]
Shuyang Gu, Jianmin Bao, Hao Yang, Dong Chen, Fang Wen, and Lu Yuan. 2019. Mask-guided portrait editing with conditional GANs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3436–3445.
[48]
Shanyan Guan, Ying Tai, Bingbing Ni, Feida Zhu, Feiyue Huang, and Xiaokang Yang. 2020. Collaborative learning for faster StyleGAN embedding. arXiv preprint arXiv:2007.01758 (2020).
[49]
Jie Gui, Zhenan Sun, Yonggang Wen, Dacheng Tao, and Jieping Ye. 2020. A review on generative adversarial networks: Algorithms, theory, and applications. arXiv preprint arXiv:2001.06937 (2020).
[50]
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. 2017. Improved training of Wasserstein GANs. arXiv preprint arXiv:1704.00028 (2017).
[51]
Guodong Guo and Na Zhang. 2019. A survey on deep learning based face recognition. Computer Vision and Image Understanding 189 (2019), 102805.
[52]
Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, and Sylvain Paris. 2020. GANSpace: Discovering interpretable gan controls. arXiv preprint arXiv:2004.02546 (2020).
[53]
Zhenliang He, Wangmeng Zuo, Meina Kan, Shiguang Shan, and Xilin Chen. 2019. AttGAN: Facial attribute editing by only changing what you want. IEEE Transactions on Image Processing 28, 11 (2019), 5464–5478.
[54]
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. arXiv preprint arXiv:1706.08500 (2017).
[55]
Saifuddin Hitawala. 2018. Comparative study on generative adversarial networks. arXiv preprint arXiv:1801.04271 (2018).
[56]
Yongjun Hong, Uiwon Hwang, Jaeyoon Yoo, and Sungroh Yoon. 2019. How generative adversarial networks and their variants work: An overview. ACM Computing Surveys 52, 1 (2019), 1–43.
[57]
Jiahui Hu, Bing Yu, Yun Yang, and Bailan Feng. 2019. Towards facial de-expression and expression recognition in the wild. In Proceedings of the 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII’19). IEEE, Los Alamitos, CA, 157–163.
[58]
Yibo Hu, Xiang Wu, Bing Yu, Ran He, and Zhenan Sun. 2018. Pose-guided photorealistic face rotation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8398–8406.
[59]
Rui Huang, Shu Zhang, Tianyu Li, and Ran He. 2017. Beyond face rotation: Global and local perception GAN for photorealistic and identity preserving frontal view synthesis. In Proceedings of the IEEE International Conference on Computer Vision. 2439–2448.
[60]
Yihao Huang, Felix Juefei-Xu, Run Wang, Xiaofei Xie, Lei Ma, Jianwen Li, Weikai Miao, Yang Liu, and Geguang Pu. 2020. FakeLocator: Robust localization of GAN-based face manipulations via semantic segmentation networks with bells and whistles. arXiv preprint arXiv:2001.09598 (2020).
[61]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1125–1134.
[62]
Abdul Jabbar, Xi Li, and Bourahla Omar. 2020. A survey on generative adversarial networks: Variants, applications, and training. arXiv preprint arXiv:2006.05132 (2020).
[63]
Xiaohan Jin, Ye Qi, and Shangxuan Wu. 2017. CycleGAN face-off. arXiv preprint arXiv:1712.03451 (2017).
[64]
Animesh Karnewar and Oliver Wang. 2020. MSG-GAN: Multi-scale gradients for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7799–7808.
[65]
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2017. Progressive growing of GANs for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196 (2017).
[66]
Tero Karras, Samuli Laine, and Timo Aila. 2019. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4401–4410.
[67]
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. 2020. Analyzing and improving the image quality of StyleGAN. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8110–8119.
[68]
Hadi Kazemi, Fariborz Taherkhani, and Nasser M. Nasrabadi. 2018. Unsupervised facial geometry learning for sketch to photo synthesis. In Proceedings of the 2018 International Conference of the Biometrics Special Interest Group (BIOSIG’18). IEEE, Los Alamitos, CA, 1–5.
[69]
Youngsung Kim, ByungIn Yoo, Youngjun Kwak, Changkyu Choi, and Junmo Kim. 2017. Deep generative-contrastive networks for facial expression recognition. arXiv preprint arXiv:1703.07140 (2017).
[70]
Diederik P. Kingma and Prafulla Dhariwal. 2018. Glow: Generative flow with invertible 1x1 convolutions. arXiv preprint arXiv:1807.03039 (2018).
[71]
Diederik P. Kingma and Max Welling. 2013. Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 (2013).
[72]
Ivan Kobyzev, Simon Prince, and Marcus Brubaker. 2021. Normalizing flows: An introduction and review of current methods. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 11 (2021), 3964–3979.
[73]
Naveen Kodali, Jacob Abernethy, James Hays, and Zsolt Kira. 2017. On convergence and stability of GANs. arXiv preprint arXiv:1705.07215 (2017).
[74]
Karol Kurach, Mario Lučić, Xiaohua Zhai, Marcin Michalski, and Sylvain Gelly. 2019. A large-scale study on regularization and normalization in GANs. In Proceedings of the International Conference on Machine Learning. 3581–3590.
[75]
Ying-Hsiu Lai and Shang-Hong Lai. 2018. Emotion-preserving representation learning via generative adversarial network for multi-view facial expression recognition. In Proceedings of the 2018 13th IEEE International Conference on Automatic Face and Gesture Recognition (FG’18). IEEE, Los Alamitos, CA, 263–270.
[76]
Alexandros Lattas, Stylianos Moschoglou, Baris Gecer, Stylianos Ploumpis, Vasileios Triantafyllou, Abhijeet Ghosh, and Stefanos Zafeiriou. 2020. AvatarMe: Realistically renderable 3D facial reconstruction “in-the-wild.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 760–769.
[77]
Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, et al. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4681–4690.
[78]
Soochan Lee, Junsoo Ha, and Gunhee Kim. 2019. Harmonizing maximum likelihood with GANs for multimodal conditional generation. arXiv preprint arXiv:1902.09225 (2019).
[79]
Shan Li and Weihong Deng. 2020. Deep facial expression recognition: A survey. IEEE Transactions on Affective Computing. Early access, March 17, 2020.
[80]
Xiaoyi Li and Xiaowen Yu. n.d. Generating Cartoon Style Facial Expressions with StackGAN. Stanford University.
[81]
Jingjun Liang, Shizhe Chen, and Qin Jin. 2019. Semi-supervised multimodal emotion recognition with improved Wasserstein GANs. In Proceedings of the 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC’19). IEEE, Los Alamitos, CA, 695–703.
[82]
David Liu and Nathan Hu. 2020. GAN-Based Image Data Augmentation. Stanford University.
[83]
Jinjin Liu, Qingbao Li, Ming Liu, and Tongxin Wei. 2020. CP-GAN: A cross-pose profile face frontalization boosting pose-invariant face recognition. IEEE Access 8 (2020), 198659–198667.
[84]
Longfei Liu, Sheng Li, Yisong Chen, and Guoping Wang. 2018. X-GANs: Image reconstruction made easy for extreme cases. arXiv preprint arXiv:1808.04432 (2018).
[85]
Ming Liu, Yukang Ding, Min Xia, Xiao Liu, Errui Ding, Wangmeng Zuo, and Shilei Wen. 2019. STGAN: A unified selective transfer network for arbitrary image attribute editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3673–3682.
[86]
Ming-Yu Liu, Xun Huang, Jiahui Yu, Ting-Chun Wang, and Arun Mallya. 2021. Generative adversarial networks for image and video synthesis: Algorithms and applications. Proceedings of the IEEE 109, 5 (2021), 839–862.
[87]
Ming-Yu Liu and Oncel Tuzel. 2016. Coupled generative adversarial networks. arXiv preprint arXiv:1606.07536 (2016).
[88]
Yanfei Liu and Junhua Chen. 2021. Unsupervised face frontalization for pose-invariant face recognition. Image and Vision Computing 106 (2021), 104093.
[89]
Yunfan Liu, Qi Li, and Zhenan Sun. 2019. Attribute-aware face aging with wavelet-based generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11877–11886.
[90]
Boyu Lu, Jun-Cheng Chen, and Rama Chellappa. 2019. Unsupervised domain-specific deblurring via disentangled representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10225–10234.
[91]
Yongyi Lu, Yu-Wing Tai, and Chi-Keung Tang. 2018. Attribute-guided face generation using conditional cycleGAN. In Proceedings of the European Conference on Computer Vision (ECCV’18). 282–297.
[92]
Mario Lucic, Karol Kurach, Marcin Michalski, Sylvain Gelly, and Olivier Bousquet. 2017. Are GANs created equal? A large-scale study. arXiv preprint arXiv:1711.10337 (2017).
[93]
Sebastian Lutz, Konstantinos Amplianitis, and Aljosa Smolic. 2018. AlphaGAN: Generative adversarial networks for natural image matting. arXiv preprint arXiv:1807.10088 (2018).
[94]
Kushal Mahalingaiah and Bruce Matichuk. 2019. ProDeblurGAN: Progressive growing of GANs for blind motion deblurring in face recognition. In Proceedings of the International Conference on Smart Multimedia. 439–450.
[95]
Farkhod Makhmudkhujaev, Sungeun Hong, and In Kyu Park. 2021. Re-aging GAN: Toward personalized face age transformation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 3908–3917.
[96]
Dejan Malesevic, Christoph Mayer, Shuhang Gu, and Radu Timofte. 2019. Photo-realistic and robust inpainting of faces using refinement GANs. In Inpainting and Denoising Challenges. Springer, 129–144.
[97]
Xudong Mao and Qing Li. 2018. Unpaired multi-domain image generation via regularized conditional GANs. arXiv preprint arXiv:1805.02456 (2018).
[98]
Richard T. Marriott, Sami Romdhani, and Liming Chen. 2020. A 3D GAN for improved large-pose facial recognition. arXiv preprint arXiv:2012.10545 (2020).
[99]
Iacopo Masi, Yue Wu, Tal Hassner, and Prem Natarajan. 2018. Deep face recognition: A survey. In Proceedings of the 2018 31st SIBGRAPI Conference on Graphics, Patterns, and Images (SIBGRAPI’18). IEEE, Los Alamitos, CA, 471–478.
[100]
Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014).
[101]
Joel Ruben Antony Moniz, Christopher Beckham, Simon Rajotte, Sina Honari, and Christopher Pal. 2018. Unsupervised depth estimation, 3D face rotation and replacement. arXiv preprint arXiv:1803.09202 (2018).
[102]
Stylianos Moschoglou, Stylianos Ploumpis, Mihalis A. Nicolaou, Athanasios Papaioannou, and Stefanos Zafeiriou. 2020. 3DFaceGAN: Adversarial nets for 3D face representation, generation, and translation. International Journal of Computer Vision 128, 10 (2020), 2534–2551.
[103]
Thu Nguyen-Phuoc, Chuan Li, Lucas Theis, Christian Richardt, and Yong-Liang Yang. 2019. HoloGAN: Unsupervised learning of 3D representations from natural images. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 7588–7597.
[104]
Thekke Madam Nimisha, Kumar Sunil, and A. N. Rajagopalan. 2018. Unsupervised class-specific deblurring. In Proceedings of the European Conference on Computer Vision (ECCV’18). 353–369.
[105]
Augustus Odena, Christopher Olah, and Jonathon Shlens. 2017. Conditional image synthesis with auxiliary classifier gans. In Proceedings of the International Conference on Machine Learning. 2642–2651.
[106]
Frans A. Oliehoek, Rahul Savani, Jose Gallego-Posada, Elise Van der Pol, Edwin D. De Jong, and Roderich Groß. 2017. GANGs: Generative adversarial network games. arXiv preprint arXiv:1712.00679 (2017).
[107]
Greg Olmschenk, Zhigang Zhu, and Hao Tang. 2019. Generalizing semi-supervised generative adversarial networks to regression using feature contrasting. Computer Vision and Image Understanding 186 (2019), 1–12.
[108]
Zhaoqing Pan, Weijie Yu, Bosi Wang, Haoran Xie, Victor S. Sheng, Jianjun Lei, and Sam Kwong. 2020. Loss functions of generative adversarial networks (GANs): Opportunities and challenges. IEEE Transactions on Emerging Topics in Computational Intelligence 4, 4 (2020), 500–522.
[109]
Zhaoqing Pan, Weijie Yu, Xiaokai Yi, Asifullah Khan, Feng Yuan, and Yuhui Zheng. 2019. Recent progress on generative adversarial networks (GANs): A survey. IEEE Access 7 (2019), 36322–36333.
[110]
Guim Perarnau, Joost Van De Weijer, Bogdan Raducanu, and Jose M. Álvarez. 2016. Invertible conditional GANs for image editing. arXiv preprint arXiv:1611.06355 (2016).
[111]
Shengju Qian, Kwan-Yee Lin, Wayne Wu, Yangxiaokang Liu, Quan Wang, Fumin Shen, Chen Qian, and Ran He. 2019. Make a face: Towards arbitrary high fidelity face manipulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 10033–10042.
[112]
Yichen Qian, Weihong Deng, and Jiani Hu. 2019. Unsupervised face normalization with extreme pose and expression in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9851–9858.
[113]
Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015).
[114]
Xuanchi Ren, Tao Yang, Yuwang Wang, and Wenjun Zeng. 2021. Do generative models know disentanglement? Contrastive learning is all you need. arXiv preprint arXiv:2102.10543 (2021).
[115]
Changle Rong, Xingming Zhang, and Yubei Lin. 2020. Feature-improving generative adversarial network for face frontalization. IEEE Access 8 (2020), 68842–68851.
[116]
Kevin Roth, Aurelien Lucchi, Sebastian Nowozin, and Thomas Hofmann. 2017. Stabilizing training of generative adversarial networks through regularization. arXiv preprint arXiv:1705.09367 (2017).
[117]
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training GANs. arXiv preprint arXiv:1606.03498 (2016).
[118]
Divya Saxena and Jiannong Cao. 2021. Generative adversarial networks (GANs): Challenges, solutions, and future directions. ACM Computing Surveys 54, 3 (2021), 1–42.
[119]
Pourya Shamsolmoali, Masoumeh Zareapoor, Eric Granger, Huiyu Zhou, Ruili Wang, M. Emre Celebi, and Jie Yang. 2021. Image synthesis with adversarial networks: A comprehensive survey and case studies. Information Fusion 72 (2021), 126–146.
[120]
Yujun Shen, Ceyuan Yang, Xiaoou Tang, and Bolei Zhou. 2022. InterFaceGAN: Interpreting the disentangled face representation learned by GANs. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 4 (2022), 2004–2018.
[121]
Ziyi Shen, Wei-Sheng Lai, Tingfa Xu, Jan Kautz, and Ming-Hsuan Yang. 2020. Exploiting semantics for face image deblurring. International Journal of Computer Vision 128, 7 (2020), 1829–1846.
[122]
Jingkuan Song, Jingqiu Zhang, Lianli Gao, Xianglong Liu, and Heng Tao Shen. 2018. Dual conditional GANs for face aging and rejuvenation. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI’18). 899–905.
[123]
Kumar Sricharan, Raja Bala, Matthew Shreve, Hui Ding, Kumar Saketh, and Jin Sun. 2017. Semi-supervised conditional GANs. arXiv preprint arXiv:1708.05789 (2017).
[124]
Jiaze Sun, Binod Bhattarai, and Tae-Kyun Kim. 2020. MatchGAN: A self-supervised semi-supervised conditional generative adversarial network. In Proceedings of the Asian Conference on Computer Vision.
[125]
Gino Tesei. 2019. Generating realistic facial expressions through conditional cycle-consistent generative adversarial networks (CCycleGAN). Project for CS230, Deep Learning.
[126]
Mukhiddin Toshpulatov, Wookey Lee, and Suan Lee. 2021. Generative adversarial networks and their application to 3D face generation: A survey. Image and Vision Computing 108 (2021), 104119.
[127]
Luan Tran, Xi Yin, and Xiaoming Liu. 2017. Disentangled representation learning GAN for pose-invariant face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1415–1424.
[128]
Luan Tran, Xi Yin, and Xiaoming Liu. 2018. Representation learning by rotating your faces. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 12 (2018), 3007–3021.
[129]
Soumya Tripathy, Juho Kannala, and Esa Rahtu. 2020. ICface: Interpretable and controllable face reenactment using gans. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 3385–3394.
[130]
Soumya Tripathy, Juho Kannala, and Esa Rahtu. 2021. FACEGAN: Facial attribute controllable rEenactment GAN. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 1329–1338.
[131]
Andrey Voynov and Artem Babenko. 2020. Unsupervised discovery of interpretable directions in the GAN latent space. In Proceedings of the International Conference on Machine Learning. 9786–9796.
[132]
De Wang and Liang-Jie Zhang. 2019. Artificial Intelligence and Mobile Services—AIMS 2019. Springer.
[133]
Kunfeng Wang, Chao Gou, Yanjie Duan, Yilun Lin, Xinhu Zheng, and Fei-Yue Wang. 2017. Generative adversarial networks: Introduction and outlook. IEEE/CAA Journal of Automatica Sinica 4, 4 (2017), 588–598.
[134]
Qiang Wang, Huijie Fan, Gan Sun, Yang Cong, and Yandong Tang. 2019. Laplacian pyramid adversarial network for face completion. Pattern Recognition 88 (2019), 493–505.
[135]
Qiang Wang, Huijie Fan, Gan Sun, Weihong Ren, and Yandong Tang. 2020. Recurrent generative adversarial network for face completion. IEEE Transactions on Multimedia 23 (2020), 429–442.
[136]
Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. 2018. High-resolution image synthesis and semantic manipulation with conditional GANs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8798–8807.
[137]
Xiang Wang, Kai Wang, and Shiguo Lian. 2019. A survey on face data augmentation. arXiv preprint arXiv:1904.11685 (2019).
[138]
Zhengwei Wang, Qi She, and Tomas E. Ward. 2019. Generative adversarial networks in computer vision: A survey and taxonomy. arXiv preprint arXiv:1906.01529 (2019).
[139]
Tongxin Wei, Qingbao Li, Jinjin Liu, Ping Zhang, and Zhifeng Chen. 2020. 3D face image inpainting with generative adversarial nets. Mathematical Problems in Engineering 2020, 6 (2020), 1–11.
[140]
Maciej Wiatrak, Stefano V. Albrecht, and Andrew Nystrom. 2019. Stabilizing generative adversarial networks: A survey. arXiv preprint arXiv:1910.00927 (2019).
[141]
Olivia Wiles, A. Koepke, and Andrew Zisserman. 2018. X2Face: A network for controlling face generation using images, audio, and pose codes. In Proceedings of the European Conference on Computer Vision (ECCV’18). 670–686.
[142]
Hongyi Xu, Eduard Gabriel Bazavan, Andrei Zanfir, William T. Freeman, Rahul Sukthankar, and Cristian Sminchisescu. 2020. GHUM & GHUML : Generative 3D human shape and articulated pose models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6184–6193.
[143]
Zhiliang Xu, Xiyu Yu, Zhibin Hong, Zhen Zhu, Junyu Han, Jingtuo Liu, Errui Ding, and Xiang Bai. 2021. FaceController: Controllable attribute editing for face in the wild. arXiv preprint arXiv:2102.11464 (2021).
[144]
Huiyuan Yang, Umur Ciftci, and Lijun Yin. 2018. Facial expression recognition by de-expression residue learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2168–2177.
[145]
Hongyu Yang, Di Huang, Yunhong Wang, and Anil K. Jain. 2018. Learning face age progression: A pyramid architecture of GANs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 31–39.
[146]
Huiyuan Yang, Zheng Zhang, and Lijun Yin. 2018. Identity-adaptive facial expression recognition through expression regeneration using conditional generative adversarial networks. In Proceedings of the 2018 13th IEEE International Conference on Automatic Face and Gesture Recognition (FG’18). IEEE, Los Alamitos, CA, 294–301.
[147]
Wei Yi, Yaoran Sun, and Sailing He. 2018. Data augmentation using conditional GANs for facial emotion recognition. In Proceedings of the 2018 Progress in Electromagnetics Research Symposium (PIERS-Toyama’18). IEEE, Los Alamitos, CA, 710–714.
[148]
Junho Yim, Heechul Jung, ByungIn Yoo, Changkyu Choi, Dusik Park, and Junmo Kim. 2015. Rotating your face using multi-task deep neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 676–684.
[149]
Xi Yin, Xiang Yu, Kihyuk Sohn, Xiaoming Liu, and Manmohan Chandraker. 2017. Towards large-pose face frontalization in the wild. In Proceedings of the IEEE International Conference on Computer Vision. 3990–3999.
[150]
Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S. Huang. 2019. Free-form image inpainting with gated convolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4471–4480.
[151]
Egor Zakharov, Aliaksandra Shysheya, Egor Burkov, and Victor Lempitsky. 2019. Few-shot adversarial learning of realistic neural talking head models. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 9459–9468.
[152]
Bassel Zeno, Ilya Kalinovskiy, and Yuri Matveev. 2021. PFA-GAN: Pose face augmentation based on generative adversarial network. Informatica 32, 2 (2021), 425–440.
[153]
Feifei Zhang, Tianzhu Zhang, Qirong Mao, and Changsheng Xu. 2018. Joint pose and expression modeling for facial expression recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3359–3368.
[154]
Guanghao Zhang, Enmei Tu, and Dongshun Cui. 2017. Stable and improved generative adversarial nets (GANS): A constructive survey. In Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP’17). IEEE, Los Alamitos, CA, 1871–1875.
[155]
Jiayuan Zhang, Ao Li, Yu Liu, and Minghui Wang. 2019. Adversarially regularized U-Net-based GANs for facial attribute modification and generation. IEEE Access 7 (2019), 86453–86462.
[156]
Jichao Zhang, Fan Zhong, Gongze Cao, and Xueying Qin. 2017. ST-GAN: Unsupervised facial image semantic transformation using generative adversarial networks. In Proceedings of the Asian Conference on Machine Learning. 248–263.
[157]
Ke-Yue Zhang, Taiping Yao, Jian Zhang, Ying Tai, Shouhong Ding, Jilin Li, Feiyue Huang, Haichuan Song, and Lizhuang Ma. 2020. Face anti-spoofing via disentangled representation learning. In Proceedings of the European Conference on Computer Vision. 641–657.
[158]
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 586–595.
[159]
Yinghui Zhang, Bo Sun, Yongkang Xiao, Rong Xiao, and YunGang Wei. 2019. Feature augmentation for imbalanced classification with conditional mixture WGANs. Signal Processing: Image Communication 75 (2019), 89–99.
[160]
Zhifei Zhang, Yang Song, and Hairong Qi. 2017. Age progression/regression by conditional adversarial autoencoder. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5810–5818.
[161]
Guoping Zhao, Mingyu Zhang, Jiajun Liu, and Ji-Rong Wen. 2019. Unsupervised adversarial attacks on deep feature-based retrieval with GAN. arXiv preprint arXiv:1907.05793 (2019).
[162]
Jian Zhao, Yu Cheng, Yi Cheng, Yang Yang, Fang Zhao, Jianshu Li, Hengzhu Liu, Shuicheng Yan, and Jiashi Feng. 2019. Look across elapse: Disentangled representation learning and photorealistic cross-age face synthesis for age-invariant face recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 9251–9258.
[163]
Jian Zhao, Lin Xiong, Jayashree Karlekar, Jianshu Li, Fang Zhao, Zhecan Wang, Sugiri Pranata, Shengmei Shen, Shuicheng Yan, and Jiashi Feng. 2017. Dual-agent GANs for photorealistic and identity preserving profile face synthesis. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17). Vol. 2, 65–75.
[164]
Jian Zhao, Lin Xiong, Jianshu Li, Junliang Xing, Shuicheng Yan, and Jiashi Feng. 2018. 3D-aided dual-agent GANs for unconstrained face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 10 (2018), 2380–2394.
[165]
Hang Zhou, Jihao Liu, Ziwei Liu, Yu Liu, and Xiaogang Wang. 2020. Rotate-and-render: Unsupervised photorealistic face rotation from single-view images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5911–5920.
[166]
Sharon Zhou, Mitchell L. Gordon, Ranjay Krishna, Austin Narcomey, Li Fei-Fei, and Michael S. Bernstein. 2019. Hype: A benchmark for human eye perceptual evaluation of generative models. arXiv preprint arXiv:1904.01121 (2019).
[167]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision. 2223–2232.
[168]
Xinyue Zhu, Yifan Liu, Jiahong Li, Tao Wan, and Zengchang Qin. 2018. Emotion classification with data augmentation using generative adversarial networks. In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining. 349–360.
[169]
Zhenyao Zhu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2014. Multi-view perceptron: A deep model for learning face identity and view representations. In Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS’14), Vol. 1. 217–225.

Cited By

View all
  • (2024)Safeguarding Identities with GAN-based Face AnonymizationEngineering, Technology & Applied Science Research10.48084/etasr.752714:4(15581-15589)Online publication date: 2-Aug-2024
  • (2024)Masked GANs for Face Completion: A Novel Deep Learning ApproachEAI Endorsed Transactions on Pervasive Health and Technology10.4108/eetpht.9.48509Online publication date: 15-Jan-2024
  • (2024)Toward Unbiased High-Quality Portraits through Latent-Space EvaluationJournal of Imaging10.3390/jimaging1007015710:7(157)Online publication date: 28-Jun-2024
  • Show More Cited By

Index Terms

  1. Generative Adversarial Networks for Face Generation: A Survey

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Computing Surveys
    ACM Computing Surveys  Volume 55, Issue 5
    May 2023
    810 pages
    ISSN:0360-0300
    EISSN:1557-7341
    DOI:10.1145/3567470
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 03 December 2022
    Accepted: 20 March 2022
    Revised: 17 March 2022
    Received: 11 July 2021
    Published in CSUR Volume 55, Issue 5

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Survey
    2. generative adversarial networks (GANs)
    3. face generation
    4. applications
    5. training
    6. loss functions
    7. benchmarks
    8. algorithms

    Qualifiers

    • Survey
    • Refereed

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)1,167
    • Downloads (Last 6 weeks)100
    Reflects downloads up to 10 Oct 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Safeguarding Identities with GAN-based Face AnonymizationEngineering, Technology & Applied Science Research10.48084/etasr.752714:4(15581-15589)Online publication date: 2-Aug-2024
    • (2024)Masked GANs for Face Completion: A Novel Deep Learning ApproachEAI Endorsed Transactions on Pervasive Health and Technology10.4108/eetpht.9.48509Online publication date: 15-Jan-2024
    • (2024)Toward Unbiased High-Quality Portraits through Latent-Space EvaluationJournal of Imaging10.3390/jimaging1007015710:7(157)Online publication date: 28-Jun-2024
    • (2024)Opportunities and Challenges of Generative AI in Construction Industry: Focusing on Adoption of Text-Based ModelsBuildings10.3390/buildings1401022014:1(220)Online publication date: 14-Jan-2024
    • (2024)Integration effect of artificial intelligence and traditional animation creation technologyJournal of Intelligent Systems10.1515/jisys-2023-030533:1Online publication date: 31-May-2024
    • (2024)Analysis of virtual standardized patients for assessing clinical fundamental skills of medical students: a prospective studyBMC Medical Education10.1186/s12909-024-05982-224:1Online publication date: 10-Sep-2024
    • (2024)Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture SearchACM Computing Surveys10.1145/366513856:11(1-36)Online publication date: 28-Jun-2024
    • (2024)Application of CNN in Visual Image Design: Image Feature Extraction and ProcessingProceedings of the 2024 International Conference on Machine Intelligence and Digital Applications10.1145/3662739.3672304(569-573)Online publication date: 30-May-2024
    • (2024)Metrics for Dataset Demographic Bias: A Case Study on Facial Expression RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.336197946:8(5209-5226)Online publication date: 1-Aug-2024
    • (2024)Dual-Constraint Autoencoder and Adaptive Weighted Similarity Spatial Attention for Unsupervised Anomaly DetectionIEEE Transactions on Industrial Informatics10.1109/TII.2024.338458320:7(9393-9403)Online publication date: Jul-2024
    • Show More Cited By

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media