Abstract
Style transfer is the process that aims to recreate a given image (target image) with the style of another image (style image). In this work, a new style transfer scheme is proposed that uses a single-image super resolution (SISR) network to increase the resolution of the given target image as well as the style image and perform the transformation process using the pre-trained VGG19 model. The Combination of perceptual loss and total variation loss is used which results in more photo-realistic output. With the change in content weight, the output image contains different semantic information and precise structure of the target image resulting in visually distinguishable results. The generated outputs can be altered accordingly by the user from artistic style to photo-realistic style by changing the weights. Detailed experimentation is done with different target image and style image pairs. The subjective quality of the stylised images is measured. Experimental results show that the quality of the generated image is better than the state of the art existing schemes. This proposed scheme preserves more information from the target image and creates less distortion for all combinations of different types of images. For more effective comparison, the contour of the stylizing images are extracted and also similarity is measured. This experiment shows that the result images have contour closer to the target images, also measured similarity is found maximum which indicates more preservation of semantic information than other existing schemes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
An, J., Xiong, H., Huan, J., Luo, J.: Ultrafast photorealistic style transfer via neural architecture search. In: AAAI, pp. 10443–10450 (2020)
Anokhin, I., et al.: High-resolution daytime translation without domain labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7488–7497 (2020)
Chang, J.R., Chen, Y.S.: Exploiting spatial relation for reducing distortion in style transfer. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1209–1217 (2021)
Chiu, T.-Y., Gurari, D.: Iterative feature transformation for fast and versatile universal style transfer. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12364, pp. 169–184. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58529-7_11
Dumoulin, V., Shlens, J., Kudlur, M.: A learned representation for artistic style. arXiv preprint arXiv:1610.07629 (2016)
Gao, W., Li, Y., Yin, Y., Yang, M.H.: Fast video multi-style transfer. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 3222–3230 (2020)
Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576 (2015)
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)
Gooch, B., Gooch, A.: Non-photorealistic Rendering. CRC Press, New York (2001)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Kolkin, N., Salavon, J., Shakhnarovich, G.: Style transfer by relaxed optimal transport and self-similarity. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10051–10060 (2019)
Kotovenko, D., Sanakoyeu, A., Ma, P., Lang, S., Ommer, B.: A content transformation block for image style transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10032–10041 (2019)
Li, X., Liu, S., Kautz, J., Yang, M.H.: Learning linear transformations for fast arbitrary style transfer. arXiv preprint arXiv:1808.04537 (2018)
Li, Z., Zhou, F., Yang, L., Li, X., Li, J.: Accelerate neural style transfer with super-resolution. Multimed. Tools Appl. 79(7), 4347–4364 (2020)
Liang, Y., Lee, D., Li, Y., Shin, B.S.: Unpaired medical image colorization using generative adversarial network. Multimed. Tools Appl. 1–15 (2021)
Park, D.Y., Lee, K.H.: Arbitrary style transfer with style-attentional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5880–5888 (2019)
Rosin, P., Collomosse, J.: Image and Video-based Artistic Stylisation, vol. 42. Springer, Heidelberg (2012). https://doi.org/10.1007/978-1-4471-4519-6
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2015)
Strothotte, T., Schlechtweg, S.: Non-photorealistic Computer Graphics: Modeling, Rendering, and Animation. Morgan Kaufmann, New York (2002)
Sun, K., Xiao, B., Liu, D., Wang, J.: Deep high-resolution representation learning for human pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5693–5703 (2019)
Wang, H., Li, Y., Wang, Y., Hu, H., Yang, M.H.: Collaborative distillation for ultra-resolution universal style transfer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1860–1869 (2020)
Yao, Y., Ren, J., Xie, X., Liu, W., Liu, Y.J., Wang, J.: Attention-aware multi-stroke style transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1467–1475 (2019)
Yoo, J., Uh, Y., Chun, S., Kang, B., Ha, J.W.: Photorealistic style transfer via wavelet transforms. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 9036–9045 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Das, A., Sen, P., Sahu, N. (2021). Adaptive Style Transfer Using SISR. In: Tsapatsoulis, N., Panayides, A., Theocharides, T., Lanitis, A., Pattichis, C., Vento, M. (eds) Computer Analysis of Images and Patterns. CAIP 2021. Lecture Notes in Computer Science(), vol 13052. Springer, Cham. https://doi.org/10.1007/978-3-030-89128-2_34
Download citation
DOI: https://doi.org/10.1007/978-3-030-89128-2_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-89127-5
Online ISBN: 978-3-030-89128-2
eBook Packages: Computer ScienceComputer Science (R0)