ArtDiff: Artwork Generation via Conditional Diffusion Models
Authors: 
Wenhui Liu, Bin Liu, Xiao Lin, Bo Li, Zhifen He, Kang WangAuthors Info & Claims
Wenhui Liu, Bin Liu, Xiao Lin, Bo Li, Zhifen He, Kang WangAuthors Info & ClaimsArticle No.: 88, Pages 1 - 6
Abstract
Artwork Generation is an important research area of computer vision. Recently, kinds of generative models have achieved great success in natural image generation. However, artwork generation has rarely been studied due to the unfixed structure of artworks. Combined with prevailing diffusion models, we propose a simple yet effective framework, named as ArtDiff, for artwork generation. Given a name of an artist, we can generate diverse and novel artworks which reflecting the style of the artist. To the best of our knowledge, ArtDiff is the first diffusion model that generate artworks with the guidance of artist's name. Experimental results demonstrate that ArtDiff is able to recognize the artist's preference and generate artworks with reasonable structures and fine-grained details.
References
[1]
Gatys, Leon A., Alexander S. Ecker, and Matthias Bethge. "Image style transfer using convolutional neural networks." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. https://doi.org/10.1109/cvpr.2016.265
[2]
Huang, Xun, and Serge Belongie. "Arbitrary style transfer in real-time with adaptive instance normalization." Proceedings of the IEEE international conference on computer vision. 2017. https://doi.org/10.1109/iccv.2017.167
[3]
Li, Yijun, "Universal style transfer via feature transforms." Advances in neural information processing systems 30 (2017).
[4]
Sheng, Lu, "Avatar-net: Multi-scale zero-shot style transfer by feature decoration." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. https://doi.org/10.1109/cvpr.2018.00860
[5]
Sanakoyeu, Artsiom, "A style-aware content loss for real-time hd style transfer." proceedings of the European conference on computer vision (ECCV). 2018. https://doi.org/10.1007/978-3-030-01237-3_43
[6]
Chen, Xuanhong, "Anisotropic stroke control for multiple artists style transfer." Proceedings of the 28th ACM International Conference on Multimedia. 2020. https://doi.org/10.1145/3394171.3413770
[7]
Xu, Wenju, "Drb-gan: A dynamic resblock generative adversarial network for artistic style transfer." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021. https://doi.org/10.1109/iccv48922.2021.00632
[8]
Goodfellow, Ian, "Generative adversarial networks." Communications of the ACM 63.11 (2020): 139-144.
[9]
Tan, Wei Ren, "ArtGAN: Artwork synthesis with conditional categorical GANs." 2017 IEEE International Conference on Image Processing (ICIP). IEEE, 2017. https://doi.org/10.1109/icip.2017.8296985
[10]
Mirza, Mehdi, and Simon Osindero. "Conditional generative adversarial nets." arXiv preprint arXiv:1411.1784 (2014).
[11]
Sohl-Dickstein, Jascha, "Deep unsupervised learning using nonequilibrium thermodynamics." International Conference on Machine Learning. PMLR, 2015.
[12]
Song, Yang, and Stefano Ermon. "Generative modeling by estimating gradients of the data distribution." Advances in neural information processing systems 32 (2019).
[13]
Ho, Jonathan, Ajay Jain, and Pieter Abbeel. "Denoising diffusion probabilistic models." Advances in Neural Information Processing Systems 33 (2020): 6840-6851.
[14]
Nichol, Alexander Quinn, and Prafulla Dhariwal. "Improved denoising diffusion probabilistic models." International Conference on Machine Learning. PMLR, 2021.
[15]
Amin Tavakoli and Ali Pourmohammad. "Image Denoising Based on Compressed Sensing." International Journal of Computer Theory and Engineering vol.4, no.2, pp.266-269, 2012. https://doi.org/10.7763/ijcte.2012.v4.463
[16]
Rakesh Kumar and B.S.Saini. "Improved Image Denoising Technique Using Neighboring Wavelet Coefficients of Optimal Wavelet with Adaptive Thresholding." International Journal of Computer Theory and Engineering vol.4, no.3, pp.395-400, 2012. https://doi.org/10.7763/ijcte.2012.v4.491
[17]
Jyoti Choudhary and Alka Choudhary. "Enhancement in Morphological Mean Filter for Image Denoising Using GLCM Algorithm." International Journal of Computer Theory and Engineering vol.13, no.4, pp.134-137, 2021. https://doi.org/10.7763/ijcte.2021.v13.1302
Index Terms
- ArtDiff: Artwork Generation via Conditional Diffusion Models
Recommendations
Creative and diverse artwork generation using adversarial networks
Existing style transfer methods have achieved great success in artwork generation by transferring artistic styles onto everyday photographs while keeping their contents unchanged. Despite this success, these methods have one inherent limitation: they ...
3D Printing and Camera Mapping - Artwork: Digital Buddha
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing SystemsThe new media artwork Digital Buddha combines 3D Printing and camera mapping. The creative plan for Digital Buddha applies industrial design concepts and methods that utilize multiple complex digital tools in order to achieve effects of precision ...
Artwork evolution
SIGGRAPH '10: ACM SIGGRAPH 2010 PostersCreating digital artwork requires a lot of time, talent, and effort from artists and programmers. It takes artists hours to design pleasing artwork and programmers even more time as they develop and debug complex graphics shaders. One way to aid in the ...
Comments
Information & Contributors
Information
Published In
May 2023
711 pages
ISBN:9798400708237
DOI:10.1145/3604078
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 26 October 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- PHD Start-up Fund of Nanchang Hangkong University
- Natural Science Foundation of Jiangxi Province
- Jiangxi Science Fund for Distinguished Young Scholars
- Science and Technology Research Project of Jiangxi Provincial Education Department
- National Natural Science Foundation of China
Conference
ICDIP 2023
ICDIP 2023: The 15th International Conference on Digital Image Processing
May 19 - 22, 2023
Nanjing, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 36Total Downloads
- Downloads (Last 12 months)36
- Downloads (Last 6 weeks)2
Reflects downloads up to 11 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format