A Model-Guided Unfolding Network for Single Image Reflection Removal
Article No.: 27, Pages 1 - 7
Abstract
Removing undesirable reflections from a single image captured through a glass surface is of broad application to various image processing and computer vision tasks, but it is an ill-posed and challenging problem. Existing traditional single image reflection removal(SIRR) methods are often less efficient to remove reflection due to the limited description ability of handcrafted priors. State-of-the-art learning based methods often cause instability problems because they are designed as unexplainable black boxes. In this paper, we present an explainable approach for SIRR named model-guided unfolding network(MoG-SIRR), which is unfolded from our proposed reflection removal model with non-local autoregressive prior and dereflection prior. In order to complement the transmission layer and the reflection layer in a single image, we construct a deep learning framework with two streams by integrating reflection removal and non-local regularization into trainable modules. Extensive experiments on public benchmark datasets demonstrate that our method achieves superior performance for single image reflection removal.
References
[1]
Nikolaos Arvanitopoulos, Radhakrishna Achanta, and Sabine Süsstrunk. 2017. Single Image Reflection Suppression. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1752–1760. https://doi.org/10.1109/CVPR.2017.190
[2]
Weisheng Dong, Peiyao Wang, Wotao Yin, Guangming Shi, Fangfang Wu, and Xiaotong Lu. 2019. Denoising Prior Driven Deep Neural Network for Image Restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 10(2019), 2305–2318. https://doi.org/10.1109/TPAMI.2018.2873610
[3]
Qingnan Fan, Jiaolong Yang, Gang Hua, Baoquan Chen, and David Wipf. 2017. A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing. In 2017 IEEE International Conference on Computer Vision (ICCV). 3258–3267. https://doi.org/10.1109/ICCV.2017.351
[4]
Kun Gai, Zhenwei Shi, and Changshui Zhang. 2012. Blind Separation of Superimposed Moving Images Using Image Statistics. IEEE Transactions on Pattern Analysis and Machine Intelligence 34, 1(2012), 19–32. https://doi.org/10.1109/TPAMI.2011.87
[5]
Xiaojie Guo, Xiaochun Cao, and Yi Ma. 2014. Robust Separation of Reflection from Multiple Images. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2195–2202. https://doi.org/10.1109/CVPR.2014.281
[6]
Ran He, Wei-Shi Zheng, Tieniu Tan, and Zhenan Sun. 2014. Half-Quadratic-Based Iterative Minimization for Robust Sparse Representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 2(2014), 261–275. https://doi.org/10.1109/TPAMI.2013.102
[7]
Soomin Kim, Yuchi Huo, and Sung-Eui Yoon. 2020. Single Image Reflection Removal With Physically-Based Training Images. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5163–5172. https://doi.org/10.1109/CVPR42600.2020.00521
[8]
Diederik. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. arXiv e-prints (2014).
[9]
A. Levin, A. Zomet, and Y. Weiss. 2004. Separating reflections from a single image using local features. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004., Vol. 1. I–I. https://doi.org/10.1109/CVPR.2004.1315047
[10]
Chao Li, Yixiao Yang, Kun He, Stephen Lin, and John E. Hopcroft. 2020. Single Image Reflection Removal Through Cascaded Refinement. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 3562–3571. https://doi.org/10.1109/CVPR42600.2020.00362
[11]
Yu Li and Michael S. Brown. 2013. Exploiting Reflection Change for Automatic Reflection Removal. In 2013 IEEE International Conference on Computer Vision. 2432–2439. https://doi.org/10.1109/ICCV.2013.302
[12]
Yu Li and Michael S. Brown. 2014. Single Image Layer Separation Using Relative Smoothness. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2752–2759. https://doi.org/10.1109/CVPR.2014.346
[13]
Qian Ning, Weisheng Dong, Guangming Shi, Leida Li, and Xin Li. 2021. Accurate and Lightweight Image Super-Resolution With Model-Guided Deep Unfolding Network. IEEE Journal of Selected Topics in Signal Processing 15, 2(2021), 240–252. https://doi.org/10.1109/JSTSP.2020.3037516
[14]
YiChang Shih, Dilip Krishnan, Frédo Durand, and William T. Freeman. 2015. Reflection removal using ghosting cues. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 3193–3201. https://doi.org/10.1109/CVPR.2015.7298939
[15]
Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. Computer Science (2014).
[16]
R. Szeliski, S. Avidan, and P. Anandan. 2000. Layer extraction from multiple images containing reflections and transparency. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662), Vol. 1. 246–253 vol.1. https://doi.org/10.1109/CVPR.2000.855826
[17]
Renjie Wan, Boxin Shi, Ling-Yu Duan, Ah-Hwee Tan, and Alex C. Kot. 2017. Benchmarking Single-Image Reflection Removal Algorithms. In 2017 IEEE International Conference on Computer Vision (ICCV). 3942–3950. https://doi.org/10.1109/ICCV.2017.423
[18]
Renjie Wan, Boxin Shi, Tan Ah Hwee, and Alex C. Kot. 2016. Depth of field guided reflection removal. In 2016 IEEE International Conference on Image Processing (ICIP). 21–25. https://doi.org/10.1109/ICIP.2016.7532311
[19]
Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. 2018. Non-local Neural Networks. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7794–7803. https://doi.org/10.1109/CVPR.2018.00813
[20]
Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600–612. https://doi.org/10.1109/TIP.2003.819861
[21]
Kaixuan Wei, Jiaolong Yang, Ying Fu, David Wipf, and Hua Huang. 2019. Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8170–8179. https://doi.org/10.1109/CVPR.2019.00837
[22]
Qiang Wen, Yinjie Tan, Jing Qin, Wenxi Liu, Guoqiang Han, and Shengfeng He. 2019. Single Image Reflection Removal Beyond Linearity. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 3766–3774. https://doi.org/10.1109/CVPR.2019.00389
[23]
Jian Zhang and Bernard Ghanem. 2018. ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1828–1837. https://doi.org/10.1109/CVPR.2018.00196
[24]
Xuaner Zhang, Ren Ng, and Qifeng Chen. 2018. Single Image Reflection Separation with Perceptual Losses. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4786–4794. https://doi.org/10.1109/CVPR.2018.00503
Index Terms
- A Model-Guided Unfolding Network for Single Image Reflection Removal
Index terms have been assigned to the content through auto-classification.
Recommendations
Personalized Single Image Reflection Removal Network through Adaptive Cascade Refinement
MM '23: Proceedings of the 31st ACM International Conference on MultimediaIn this paper, we aim to restore a reflection-free image from a single reflection-contaminated image captured through the glass. Many deep-learning-based methods attempt to solve the challenging problem by utilizing a uniform model obtained from training ...
Semantic Guided Single Image Reflection Removal
Reflection is common when we see through a glass window, which not only is a visual disturbance but also influences the performance of computer vision algorithms. Removing the reflection from a single image, however, is highly ill-posed since the color at ...
R2SFD: Improving Single Image Reflection Removal using Semantic Feature Dictionary
MM '24: Proceedings of the 32nd ACM International Conference on MultimediaSingle image reflection removal is a severely ill-posed problem and it is very hard to separate the desirable transmission and undesirable reflection layers. Most of the existing single image reflection removal methods try to recover the transmission ...
Comments
Information & Contributors
Information
Published In
December 2021
508 pages
ISBN:9781450386074
DOI:10.1145/3469877
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 10 January 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- Natural Science Foundation of China
Conference
MMAsia '21
Sponsor:
Acceptance Rates
Overall Acceptance Rate 59 of 204 submissions, 29%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 91Total Downloads
- Downloads (Last 12 months)16
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format