NNST-based Image Outpainting via SinGAN
Pages 26 - 31
Abstract
The restoration process can be further categorized into image inpainting and outpainting depending on the extent of the damage. Image inpainting is utilized when only a few parts require repair, while image outpainting is employed for more extensive restoration efforts. This study focuses on extensively damaged ancient fabric characterized by the presence of a repeated pattern. In order to complement the extensively damaged ancient fabric, this paper introduces a novel image outpainting algorithm based on Neural Neighbor Style Transfer (NNST) using SinGAN. SinGAN is employed to initially complement the damaged region, followed by the utilization of NNST to reconstruct the complemented region. The purpose is to enhance the clarity of the previously blurry region and adjust colors to align with the colors of the known region. The effectiveness of the proposed method is verified through both objective and subjective experiments, demonstrating its superiority over SinGAN.
References
[1]
Marcelo Bertalmio, Guillermo Sapirom, Vincent Caselles, and Coloma Ballester. July 2000. Image inpainting. In, Proceedings of the 27th annual conference on Computer graphics and interactive techniques. 417–424.
[2]
Keyan Ding, Yi Liu, Xueyi Zou, Shiqi Wang, and Kede Ma. 2021. Locally Adaptive Structure and Texture Similarity for Image Quality Assessment. In Proceedings of the 29th ACM International Conference on Multimedia(MM ’21). ACM. https://doi.org/10.1145/3474085.3475419
[3]
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Networks. arxiv:1406.2661 [stat.ML]
[4]
Basile Van Hoorick. 2020. Image Outpainting and Harmonization using Generative Adversarial Networks. arxiv:1912.10960 [cs.CV]
[5]
Diederik P Kingma and Max Welling. 2022. Auto-Encoding Variational Bayes. arxiv:1312.6114 [stat.ML]
[6]
Nicholas Kolkin, Michal Kucera, Sylvain Paris, Daniel Sykora, Eli Shechtman, and Greg Shakhnarovich. 2022. Neural Neighbor Style Transfer. arxiv:2203.13215 [cs.CV]
[7]
Mark Sabini and Gili Rusak. 2018. Painting Outside the Box: Image Outpainting with GANs. arxiv:1808.08483 [cs.CV]
[8]
Tamar Rott Shaham, Tali Dekel, and Tomer Michaeli. 2019. SinGAN: Learning a Generative Model from a Single Natural Image. arxiv:1905.01164 [cs.CV]
[9]
Qi Shan, Brian Curless, Yasutaka Furukawa, Carlos Hernandez, and Steven M. Seitz. 2014. Photo Uncrop. In Computer Vision – ECCV 2014, David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars (Eds.). Springer International Publishing, Cham, 16–31.
[10]
Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. arxiv:1409.1556 [cs.CV]
[11]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2023. Attention Is All You Need. arxiv:1706.03762 [cs.CL]
[12]
Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image Quality Assessment: From Error Visibility to Structural Similarity. In ”, IEEE Transactions on Image Processing Volume 13 Issue –– 4, pp.6004. 600–612.
[13]
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. arxiv:1801.03924 [cs.CV]
[14]
Yinda Zhang, Jianxiong Xiao, James Hays, and Ping Tan. 2013. FrameBreak: Dramatic Image Extrapolation by Guided Shift-Maps. In Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recongnition. 1171–1178.
Index Terms
- NNST-based Image Outpainting via SinGAN
Recommendations
Image outpainting guided by prior structure information
Highlights- The image-level and feature-level global structure information guides image outpainting.
AbstractDeep learning-based image outpainting infers the missing region by using the known parts of images. However, due to lack of fully exploring the known image structure or texture information, most existing methods always produce blurry ...
Cascaded refinement residual attention network for image outpainting
AbstractThe image outpainting based on deep learning shows good performance and has a wide range of applications in many fields. The previous image outpainting methods mostly used a single image as input. In this paper, we use the left and right images as ...
A three-stage GAN model based on edge and color prediction for image outpainting
Highlights- A novel three-stage image outpainting model is proposed.
- The model predicts ...
AbstractHuman beings have a natural ability to perceive invisible things around them based on materials. This paper examines the problems encountered during image outpainting, which is widely utilized in computational photography, image ...
Comments
Information & Contributors
Information
Published In
April 2024
491 pages
ISBN:9798400717055
DOI:10.1145/3669754
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 30 August 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICCAI 2024
ICCAI 2024: 2024 10th International Conference on Computing and Artificial Intelligence
April 26 - 29, 2024
Bali Island, Indonesia
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 30 Aug 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format