DrawGAN: Multi-view Generative Model Inspired by the Artist’s Drawing Method
Pages 479 - 490
Abstract
We present a novel approach for modeling artists’ drawing processes using an architecture that combines an unconditional generative adversarial network (GAN) with a multi-view generator and multi-discriminator. Our method excels in synthesizing various types of picture drawing, including line drawing, shading, and color drawing, achieving high quality and robustness. Notably, our approach surpasses the existing state-of-the-art unconditional GANs. The key novelty of our approach lies in its architecture design, which closely resembles the typical sequence of an artist’s drawing process, leading to significantly enhanced image quality. Through experimental results on few-shot datasets, we demonstrate the potential of leveraging a multi-view generative model to enhance feature knowledge and modulate image generation processes. Our proposed method holds great promise for advancing AI in the visual arts field and opens up new avenues for research and creative practices.
References
[1]
Arjovsky, M., Bottou, L.: Towards principled methods for training generative adversarial networks. arXiv preprint arXiv:1701.04862 (2017)
[2]
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: International Conference on Machine Learning, pp. 214–223. PMLR (2017)
[3]
Cheema MN et al. Modified GAN-cAED to minimize risk of unintentional liver major vessels cutting by controlled segmentation using CTA/SPET-CT IEEE Trans. Ind. Inform. 2021 17 12 7991-8002
[4]
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., Choo, J.: StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8789–8797 (2018)
[5]
Choi, Y., Uh, Y., Yoo, J., Ha, J.-W.: StarGAN v2: diverse image synthesis for multiple domains. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8188–8197 (2020)
[6]
Elgammal, A., Liu, B., Elhoseiny, M., Mazzone, M.: CAN: creative adversarial networks, generating “art” by learning about styles and deviating from style norms. arXiv preprint arXiv:1706.07068 (2017)
[7]
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)
[8]
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of wasserstein GANs. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
[9]
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
[10]
Jeong, J., Shin, J.: Training GANs with stronger augmentations via contrastive discriminator. arXiv preprint arXiv:2103.09742 (2021)
[11]
Karnewar, A., Wang, O.: MSG-GAN: multi-scale gradients for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7799–7808 (2020)
[12]
Karras, T., Aittala, M., Hellsten, J., Laine, S., Lehtinen, J., Aila, T.: Training generative adversarial networks with limited data. In: Advances in Neural Information Processing Systems, vol. 33, pp. 12104–12114 (2020)
[13]
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4401–4410 (2019)
[14]
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of StyleGAN. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8110–8119 (2020)
[15]
Kwon, Y.-H., Park, M.-G.: Predicting future frames using retrospective cycle GAN. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1811–1820 (2019)
[16]
Li C Two-stage sketch colorization ACM Trans. Graph. 2018 37 6 1-14
[17]
Lim, J.H., Ye, J.C.: Geometric GAN. arXiv preprint arXiv:1705.02894 (2017)
[18]
Liu, B., Zhu, Y., Song, K., Elgammal, A.: Towards faster and stabilized GAN training for high-fidelity few-shot image synthesis. In: International Conference on Learning Representations (2020)
[19]
Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3730–3738 (2015)
[20]
Miyato, T., Kataoka, T., Koyama, M., Yoshida, Y.: Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957 (2018)
[21]
Nazeri, K., Ng, E., Joseph, T., Qureshi, F.Z., Ebrahimi, M.: EdgeConnect: generative image inpainting with adversarial edge learning. arXiv preprint arXiv:1901.00212 (2019)
[22]
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)
[23]
Royer A et al. Singh R, Vatsa M, Patel VM, Ratha N, et al. XGAN: unsupervised image-to-image translation for many-to-many mappings Domain Adaptation for Visual Understanding 2020 Cham Springer 33-49
[24]
Si Z and Zhu S-C Learning hybrid image templates (HIT) by information projection IEEE Trans. Pattern Anal. Mach. Intell. 2011 34 7 1354-1367
[25]
Tran, D., Ranganath, R., Blei, D.M.: Deep and hierarchical implicit models 7(3), 13 (2017). arXiv preprint arXiv:1702.08896
[26]
Tseng, H.-Y., Jiang, L., Liu, C., Yang, M.-H., Yang, W.: Regularizing generative adversarial networks under limited data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7921–7931 (2021)
[27]
Wang Z, Bovik AC, Sheikh HR, and Simoncelli EP Image quality assessment: from error visibility to structural similarity IEEE Trans. Image Process. 2004 13 4 600-612
[28]
Wen Y et al. Structure-aware motion deblurring using multi-adversarial optimized CycleGAN IEEE Trans. Image Process. 2021 30 6142-6155
[29]
Yi, R., Liu, Y.-J., Lai, Y.-K., Rosin, P.L.: APDrawingGAN: generating artistic portrait drawings from face photos with hierarchical GANs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10743–10752 (2019)
[30]
Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., Xiao, J.: LSUN: construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365 (2015)
[31]
Zhang, D., Khoreva, A.: PA-GAN: improving GAN training by progressive augmentation (2018)
[32]
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–595 (2018)
[33]
Zhang Y, Han S, Zhang Z, Wang J, and Bi H CF-GAN: cross-domain feature fusion generative adversarial network for text-to-image synthesis Vis. Comput. 2022 39 4 1283-1293
[34]
Zhao, S., Liu, Z., Lin, J., Zhu, J.-Y., Han, S.: Differentiable augmentation for data-efficient GAN training. In: Advances in Neural Information Processing Systems, vol. 33, pp. 7559–7570 (2020)
Index Terms
- DrawGAN: Multi-view Generative Model Inspired by the Artist’s Drawing Method
Index terms have been assigned to the content through auto-classification.
Recommendations
Does human–AI collaboration lead to more creative art? Aesthetic evaluation of human-made and AI-generated haiku poetry
AbstractWith the development of technology, the quality of AI-generated text has improved. This is relevant in the AI art field, where AI generates literature or poetry that is appreciated. This study compared human-made and AI-generated haiku ...
Highlights- Human-made and AI-generated poetry were examined using haiku poetry.
- The beauty ...
Unveiling New Artistic Dimensions in Calligraphic Arabic Script with Generative Adversarial Networks
We present an artistic exploration into calligraphic Arabic script, focusing on the nastaliq style predominant in Iran, by harnessing the affordances of Generative Adversarial Networks (GANs). Recognizing the unique challenges posed by Arabic script's ...
Predicting Artist Drawing Activity via Multi-camera Inputs for Co-creative Drawing
Towards Autonomous Robotic SystemsAbstractThis paper presents the results of computer vision experiments in the perception of an artist drawing with analog media (pen and paper), with the aim to contribute towards a human-robot co-creative drawing system. Using data gathered from user ...
Comments
Information & Contributors
Information
Published In
Aug 2023
517 pages
ISBN:978-3-031-50071-8
DOI:10.1007/978-3-031-50072-5
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 29 December 2023
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 02 Sep 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in