Cited By
View all- Long ZLi W(2024)Landscape Information Sketching Integrating Image Structural FeaturesInventive Communication and Computational Technologies10.1007/978-981-97-7710-5_68(875-885)Online publication date: 15-Dec-2024
Sketch-to-image Synthesis with Image-level Encoding Aggregation and Patch-level Semantic Refinement
Article No.: 4, Pages 1 - 6
Abstract
Sketch-to-image synthesis is a hotspot in computer vision, which can be applied to criminal investigation and recreation industry. To better satisfy the requirements of users, many researchers concentrate on the reference-based sketch-to-image task which aims to convert a sketch-like image into a photo-like image with a given appearance reference image. However, existing methods tend to produce broken geometry and incomplete appearance. In this paper, we develop a novel network with image-level encoding aggregation and patch-level semantic refinement to generate high-quality images from sketches with reference. The image-level part is developed with an aggregated encoder and serves as a feature amplifier to extract detailed features and enhance geometry reservation. And the patch-level semantic part works with the CIELAB space and scan line seed-filling algorithm to focus on local appearance transfer. Extensive results demonstrate significant enhancements in both geometry and appearance.
References
[1]
Kazemi H, Taherkhani F and Nasrabadi N M (2018). Unsupervised facial geometry learning for sketch to photo synthesis[C]. 2018 international conference of the biometrics special interest group (BIOSIG). IEEE, 1-5.
[2]
Isola P, Zhu J Y, Zhou T, (2017). Image-to-image translation with conditional adversarial networks[C]. Proceedings of the IEEE conference on computer vision and pattern recognition, 1125-1134.
[3]
Zhang M, Wang R, Gao X, (2018). Dual-transfer face sketch–photo synthesis[J]. IEEE Transactions on Image Processing, 28(2): 642-657.
[4]
Osahor U, Kazemi H, Dabouei A, (2020). Quality guided sketch-to-photo image synthesis[C]. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 820-821.
[5]
Wang T C, Liu M Y, Zhu J Y, (2018). High-resolution image synthesis and semantic manipulation with conditional gans[C]. Proceedings of the IEEE conference on computer vision and pattern recognition, 8798-8807.
[6]
Yu J, Lin Z, Yang J, (2019). Free-form image inpainting with gated convolution[C]. Proceedings of the IEEE/CVF International Conference on Computer Vision, 4471-4480.
[7]
Li Y, Chen X, Wu F, (2019). Linestofacephoto: Face photo generation from lines with conditional self-attention generative adversarial networks[C]. Proceedings of the 27th ACM International Conference on Multimedia, 2323-2331.
[8]
Xiang X, Liu D, Yang X, (2022). Adversarial open domain adaptation for sketch-to-photo synthesis[C]. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 1434-1444.
[9]
Jo Y and Park J (2019). Sc-fegan: Face editing generative adversarial network with user's sketch and color[C]. Proceedings of the IEEE/CVF international conference on computer vision, 1745-1753.
[10]
Lee J, Kim E, Lee Y, (2020). Reference-based sketch image colorization using augmented-self reference and dense semantic correspondence[C]. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 5801-5810.
[11]
Yang S, Wang Z, Liu J, (2021). Controllable sketch-to-image translation for robust face synthesis[J]. IEEE Transactions on Image Processing, 30: 8797-8810.
[12]
Chen S Y, Liu F L, Lai Y K, (2021). DeepFaceEditing: deep face generation and editing with disentangled geometry and appearance control[J]. ACM Transactions on Graphics (TOG), 40.
[13]
Huang X, Mallya A, Wang T C, (2022). Multimodal conditional image synthesis with product-of-experts gans[C]. Computer Vision–ECCV 2022: 17th European Conference, 91-109.
[14]
Yu F, Wang D, Shelhamer E, (2018). Deep layer aggregation[C]. Proceedings of the IEEE conference on computer vision and pattern recognition, 2403-2412.
[15]
Li C and Wand M (2016). Combining markov random fields and convolutional neural networks for image synthesis[C]. Proceedings of the IEEE conference on computer vision and pattern recognition, 2479-2486.
[16]
Liu B, Zhu Y, Song K, (2021). Self-supervised sketch-to-image synthesis[C]. Proceedings of the AAAI Conference on Artificial Intelligence, 35(3): 2073-2081.
[17]
He K, Fan H, Wu Y, (2020). Momentum contrast for unsupervised visual representation learning[C]. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 9729-9738.
[18]
Karras T, Aila T, Laine S, (2017). Progressive growing of gans for improved quality, stability, and variation[J]. arXiv preprint arXiv:1710.10196.
Index Terms
- Sketch-to-image Synthesis with Image-level Encoding Aggregation and Patch-level Semantic Refinement
Recommendations
Histogram refinement for texture descriptor based image retrieval
Texture descriptors such as local binary patterns (LBP) have been successfully employed for feature extraction in image retrieval algorithms because of their high discriminating ability and computational efficiency. In this paper, we propose histogram ...
Image Texture Feature Extraction Method Based on Regional Average Binary Gray Level Difference Co-occurrence Matrix
ICVRV '11: Proceedings of the 2011 International Conference on Virtual Reality and VisualizationTexture feature is a measure method about relationship among the pixels in local area, reflecting the changes of image space gray levels. This paper presents a texture feature extraction method based on regional average binary gray level difference co-...
Comments
Information & Contributors
Information
Published In
October 2023
358 pages
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 21 December 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
CSAE 2023
CSAE 2023: The 7th International Conference on Computer Science and Application Engineering
October 17 - 19, 2023
Virtual Event, China
Acceptance Rates
Overall Acceptance Rate 368 of 770 submissions, 48%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 31Total Downloads
- Downloads (Last 12 months)29
- Downloads (Last 6 weeks)0
Reflects downloads up to 27 Jan 2025
Other Metrics
Citations
Cited By
View all- Long ZLi W(2024)Landscape Information Sketching Integrating Image Structural FeaturesInventive Communication and Computational Technologies10.1007/978-981-97-7710-5_68(875-885)Online publication date: 15-Dec-2024
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format