A Survey of Single Image Rain Removal Based on Deep Learning
Authors: 
Zhipeng Su, Yixiong Zhang, Jianghong Shi, and Xiao-Ping ZhangAuthors Info & Claims
Zhipeng Su, Yixiong Zhang, Jianghong Shi, and Xiao-Ping ZhangAuthors Info & ClaimsArticle No.: 103, Pages 1 - 35
Abstract
The rain removal task is to restore a clean image from the contaminated image by separating the background. Since the rise of deep learning in 2016, the task of image deraining has also stepped into the era of deep learning. Numerous researchers have devoted themselves to the field of computer vision and pattern recognition. However, there is still a lack of comprehensive review papers focused on using deep learning to perform rain removal tasks. In this paper, we present a comprehensive review of single image deraining based on deep learning over the past ten years. Two categories of deraining methods are discussed: the data-driven approach and the data-model-based approach. For the first type, we compare the existing network structures and loss functions. For the second type, we analyze the combination of different deraining models with deep learning, and each branch method is introduced in detail. Additionally, we quantitatively investigate the performances of the existing state-of-the-art methods on both publicly synthetic and real datasets. The trend of image deraining is also discussed.
References
[1]
Linwen Cai, Si-Yao Li, Dongwei Ren, and Ping Wang. 2019. Dual recursive network for fast image deraining. In 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2756–2760.
[2]
Bo-Hao Chen, Shih-Chia Huang, and Sy-Yen Kuo. 2017. Error-optimized sparse representation for single image rain removal. IEEE Transactions on Industrial Electronics 64, 8 (2017), 6573–6581. DOI:
[3]
Chenghao Chen and Hao Li. 2021. Robust representation learning with feedback for single image deraining. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 7742–7751.
[4]
Dongdong Chen, Mingming He, Qingnan Fan, Jing Liao, Liheng Zhang, Dongdong Hou, Lu Yuan, and Gang Hua. 2019. Gated context aggregation network for image dehazing and deraining. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1375–1383.
[5]
Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao. 2021. Pre-trained image processing transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12299–12310.
[6]
Liangyu Chen, Xin Lu, Jie Zhang, Xiaojie Chu, and Chengpeng Chen. 2021. HINet: Half instance normalization network for image restoration. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.
[7]
Xiang Chen, Jinshan Pan, Kui Jiang, Yufeng Huang, Caihua Kong, Longgang Dai, and Yufeng Li. 2022. Unpaired deep image deraining using dual contrastive learning. In The IEEE Conference on Computer Vision and Pattern Recognition.
[8]
Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V. Le. 2019. AutoAugment: Learning augmentation strategies from data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 113–123.
[9]
Sen Deng, Yidan Feng, Mingqiang Wei, Haoran Xie, Yiping Chen, Jonathan Li, Xiao-Ping Zhang, and Jing Qin. 2021. Direction-aware feature-level frequency decomposition for single image deraining. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, Zhi-Hua Zhou (Ed.). International Joint Conferences on Artificial Intelligence Organization, 650–656. Main Track.
[10]
Sen Deng, Mingqiang Wei, Jun Wang, Yidan Feng, Luming Liang, Haoran Xie, Fu Lee Wang, and Meng Wang. 2020. Detail-recovery image deraining via context aggregation networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14560–14569.
[11]
Yuyang Ding, Mingyue Li, Tao Yan, Fan Zhang, Yuan Liu, and Rynson W. H. Lau. 2022. Rain streak removal from light field images. IEEE Transactions on Circuits and Systems for Video Technology 32, 2 (2022), 467–482. DOI:
[12]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Machine Learning. ICML.
[13]
Yingjun Du, Jun Xu, Xiantong Zhen, Ming-Ming Cheng, and Ling Shao. 2020. Conditional variational image deraining. IEEE Transactions on Image Processing 29 (2020), 6288–6301.
[14]
David Eigen, Dilip Krishnan, and Rob Fergus. 2014. Restoring an image taken through a window covered with dirt or rain. In IEEE International Conference on Computer Vision.
[15]
Zhiwen Fan, Huafeng Wu, Xueyang Fu, Yue Huang, and Xinghao Ding. 2018. Residual-guide network for single image deraining. In 2018 ACM Multimedia Conference on Multimedia Conference. ACM, 1751–1759.
[16]
Xueyang Fu, Jiabin Huang, Xinghao Ding, Yinghao Liao, and John Paisley. 2017. Clearing the skies: A deep network architecture for single-image rain removal. IEEE Transactions on Image Processing 26, 6 (2017), 2944–2956.
[17]
Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley. 2017. Removing rain from single images via a deep detail network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3855–3863.
[18]
Xueyang Fu, Borong Liang, Yue Huang, Xinghao Ding, and John Paisley. 2019. Lightweight pyramid networks for image deraining. IEEE Transactions on Neural Networks and Learning Systems (2019).
[19]
Xueyang Fu, Qi Qi, Zheng-Jun Zha, Xinghao Ding, Feng Wu, and John Paisley. 2021. Successive graph convolutional network for image de-raining. International Journal of Computer Vision 129, 5 (2021), 1691–1711.
[20]
Xueyang Fu, Qi Qi, Zheng-Jun Zha, Yurui Zhu, and Xinghao Ding. 2021. Rain streak removal via dual graph convolutional network. In Proc. AAAI Conf. Artif. Intell. 1–9.
[21]
Zilin Gao, Jiangtao Xie, Qilong Wang, and Peihua Li. 2019. Global second-order pooling convolutional networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3024–3033.
[22]
Qing Guo, Jingyang Sun, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Wei Feng, and Yang Liu. 2021. EfficientDeRain: Learning pixel-wise dilation filtering for high-efficiency single-image deraining. In AAAI.
[23]
Kewen Han and Xinguang Xiang. 2020. Decomposed CycleGAN for single image deraining with unpaired data. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1828–1832.
[24]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.
[25]
Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7132–7141.
[26]
Xiaowei Hu, Chi-Wing Fu, Lei Zhu, and Pheng-Ann Heng. 2019. Depth-attentional features for single-image rain removal. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8022–8031.
[27]
Xiaowei Hu, Lei Zhu, Tianyu Wang, Chi-Wing Fu, and Pheng-Ann Heng. 2021. Single-image real-time rain removal based on depth-guided non-local features. IEEE Transactions on Image Processing 30 (2021), 1759–1770. DOI:
[28]
Huaibo Huang, Aijing Yu, and Ran He. 2021. Memory oriented transfer learning for semi-supervised image deraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7732–7741.
[29]
Kamran Javed, Ghulam Hussain, Furqan Shoukat, and Seong Oun Hwang. 2018. RRGAN: Removing rain using generative neural network. (2018).
[30]
Kui Jiang, Zhongyuan Wang, Zheng Wang, Peng Yi, Junjun Jiang, Jinsheng Xiao, and Chia-Wen Lin. 2022. DANet: Image deraining via dynamic association learning. In Proceedings of the Thirty-first International Joint Conference on Artificial Intelligence, IJCAI-22, Lud De Raedt (Ed.). International Joint Conferences on Artificial Intelligence Organization, 980–986. Main Track.
[31]
Kui Jiang, Zhongyuan Wang, Peng Yi, Chen Chen, Zhen Han, Tao Lu, Baojin Huang, and Junjun Jiang. 2021. Decomposition makes better rain removal: An improved attention-guided deraining network. IEEE Transactions on Circuits and Systems for Video Technology 31, 10 (2021), 3981–3995. DOI:
[32]
Kui Jiang, Zhongyuan Wang, Peng Yi, Chen Chen, Baojin Huang, Yimin Luo, Jiayi Ma, and Junjun Jiang. 2020. Multi-scale progressive fusion network for single image deraining. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
[33]
Kui Jiang, Zhongyuan Wang, Peng Yi, Chen Chen, Xiaofen Wang, Junjun Jiang, and Zixiang Xiong. 2021. Multi-level memory compensation network for rain removal via divide-and-conquer strategy. IEEE Journal of Selected Topics in Signal Processing 15, 2 (2021), 216–228. DOI:
[34]
Kui Jiang, Zhongyuan Wang, Peng Yi, Chen Chen, Yuhong Yang, Xin Tian, and Junjun Jiang. 2020. Attention-guided deraining network via stage-wise learning. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2618–2622. DOI:
[35]
Nanfeng Jiang, Weiling Chen, Liqun Lin, and Tiesong Zhao. 2021. Single image rain removal via multi-module deep grid network. Computer Vision and Image Understanding 202 (2021), 103106.
[36]
Xin Jin, Zhibo Chen, Jianxin Lin, Zhikai Chen, and Wei Zhou. 2019. Unsupervised single image deraining with self-supervised constraints. (2019), 2761–2765.
[37]
Li-Wei Kang, Chia-Wen Lin, and Yu-Hsiang Fu. 2011. Automatic single-image-based rain streaks removal via image decomposition. IEEE Transactions on Image Processing 21, 4 (2011), 1742–1755.
[38]
Sotiris Karavarsamis, Ioanna Gkika, Vasileios Gkitsas, Konstantinos Konstantoudakis, and Dimitrios Zarpalas. 2022. A survey of deep learning-based image restoration methods for enhancing situational awareness at disaster sites: The cases of rain, snow and haze. Sensors 22, 13 (2022), 4707.
[39]
Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang. 2018. Fast and accurate image super-resolution with deep Laplacian pyramid networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 11 (2018), 2599–2613.
[40]
Rushi Lan, Xipu Hu, Cheng Pang, Zhenbing Liu, and Xiaonan Luo. 2020. Multi-scale single image rain removal using a squeeze-and-excitation residual network. Applied Soft Computing 92 (2020), 106296.
[41]
Chen-Yu Lee, Saining Xie, Patrick Gallagher, Zhengyou Zhang, and Zhuowen Tu. 2014. Deeply-Supervised Nets. arxiv:1409.5185 [stat.ML]
[42]
Guanbin Li, Xiang He, Wei Zhang, Huiyou Chang, Le Dong, and Liang Lin. 2018. Non-locally enhanced encoder-decoder network for single image de-raining. In Proceedings of the 26th ACM International Conference on Multimedia. 1056–1064.
[43]
Ruotent Li, Loong Fah Cheong, and Robby T. Tan. 2019. Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning. arxiv:1904.05050 [cs.CV]
[44]
Siyuan Li, Iago Breno Araujo, Wenqi Ren, Zhangyang Wang, Eric K. Tokuda, Roberto Hirata Junior, Roberto Cesar-Junior, Jiawan Zhang, Xiaojie Guo, and Xiaochun Cao. 2019. Single image deraining: A comprehensive benchmark analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3838–3847.
[45]
Siyuan Li, Wenqi Ren, Jiawan Zhang, Jinke Yu, and Xiaojie Guo. 2019. Single image rain removal via a deep decomposition–composition network. Computer Vision and Image Understanding 186 (2019), 48–57.
[46]
Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, and Hongbin Zha. 2018. Recurrent squeeze-and-excitation context aggregation net for single image deraining. In Proceedings of the European Conference on Computer Vision (ECCV). 254–269.
[47]
Yi Li, Yi Chang, Changfeng Yu, and Luxin Yan. 2022. Close the loop: A unified bottom-up and top-down paradigm for joint image deraining and segmentation. In AAAI. AAAI Press, 1438–1446.
[48]
Yu Li, Robby T. Tan, Xiaojie Guo, Jiangbo Lu, and Michael S. Brown. 2016. Rain streak removal using layer priors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2736–2744.
[49]
Yu Li, Robby T. Tan, Xiaojie Guo, Jiangbo Lu, and Michael S. Brown. 2017. Single image rain streak decomposition using layer priors. IEEE Transactions on Image Processing 26, 8 (2017), 3874–3885.
[50]
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. 2021. SwinIR: Image restoration using Swin transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 1833–1844.
[51]
Tingting Liang, Xiaojie Chu, Yudong Liu, Yongtao Wang, Zhi Tang, Wei Chu, Jingdong Chen, and Haibing Ling. 2021. CBNetV2: A composite backbone network architecture for object detection. arXiv preprint arXiv:2107.00420 (2021).
[52]
Xiangguo Liang, Bin Qiu, Zhuo Su, Chengying Gao, X. Shi, and Ruomei Wang. 2019. Rain wiper: An incremental randomly wired network for single image deraining. Comput. Graph. Forum 38, 7 (2019), 159–169. DOI:
[53]
Chih-Yang Lin, Zhuang Tao, Ai-Sheng Xu, Li-Wei Kang, and Fityanul Akhyar. 2020. Sequential dual attention network for rain streak removal in a single image. IEEE Transactions on Image Processing 29 (2020), 9250–9265.
[54]
Hong Liu, Hanrong Ye, Xia Li, Wei Shi, Mengyuan Liu, and Qianru Sun. 2019. Self-refining deep symmetry enhanced network for rain removal. In 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2786–2790.
[55]
Lin Liu, Jianzhuang Liu, Shanxin Yuan, Gregory Slabaugh, Aleš Leonardis, Wengang Zhou, and Qi Tian. 2020. Wavelet-based dual-branch network for image demoiréing. In European Conference on Computer Vision. Springer, 86–102.
[56]
Risheng Liu, Zhiying Jiang, Xin Fan, and Zhongxuan Luo. 2019. Knowledge-driven deep unrolling for robust image layer separation. IEEE Transactions on Neural Networks and Learning Systems 31, 5 (2019), 1653–1666.
[57]
Risheng Liu, Zhiying Jiang, Long Ma, Xin Fan, Haojie Li, and Zhongxuan Luo. 2018. Deep layer prior optimization for single image rain streaks removal. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1408–1412.
[58]
Xing Liu, Masanori Suganuma, Zhun Sun, and Takayuki Okatani. 2019. Dual residual networks leveraging the potential of paired operations for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7007–7016.
[59]
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).
[60]
Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. 2022. A ConvNet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022).
[61]
Ilya Loshchilov and Frank Hutter. 2016. SGDR: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 (2016).
[62]
Yu Luo, Yong Xu, and Hui Ji. 2015. Removing rain from a single image via discriminative sparse coding. In Proceedings of the IEEE International Conference on Computer Vision. 3397–3405.
[63]
Siqi Ni, Xueyun Cao, Tao Yue, and Xuemei Hu. 2021. Controlling the rain: From removal to rendering. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 6324–6333. DOI:
[64]
Jinshan Pan, Jiangxin Dong, Yang Liu, Jiawei Zhang, Jimmy Ren, Jinhui Tang, Yu-Wing Tai, and Ming-Hsuan Yang. 2020. Physics-based generative adversarial models for image restoration and beyond. arxiv:1808.00605 [cs.CV]
[65]
Jinshan Pan, Sifei Liu, Deqing Sun, Jiawei Zhang, Yang Liu, Jimmy Ren, Zechao Li, Jinhui Tang, Huchuan Lu, Yu-Wing Tai, and Ming-Hsuan Yang. 2018. Learning dual convolutional neural networks for low-level vision. In Proceedings of the IEEE International Conference on Computer Vision.
[66]
Zaiyu Pan, Jun Wang, Zhengwen Shen, Shuyu Han, and Jihong Zhu. 2023. Cross-domain collaborative learning for single image deraining. Expert Systems with Applications 211 (2023), 118611.
[67]
Yeachan Park, Myeongho Jeon, Junho Lee, and Myungjoo Kang. 2020. MARA-Net: Single Image Deraining Network with Multi-level connections and Adaptive Regional Attentions. arxiv:2009.13990 [cs.CV]
[68]
Rui Qian, Robby T. Tan, Wenhan Yang, Jiajun Su, and Jiaying Liu. 2018. Attentive generative adversarial network for raindrop removal from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2482–2491.
[69]
Ruijie Quan, Xin Yu, Yuanzhi Liang, and Yi Yang. 2021. Removing raindrops and rain streaks in one go. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 9143–9152. DOI:
[70]
Yue Que, Suli Li, and Hyo Jong Lee. 2021. Attentive composite residual network for robust rain removal from single images. IEEE Transactions on Multimedia 23 (2021), 3059–3072. DOI:
[71]
Wu Ran, Youzhao Yang, and Hong Lu. 2020. Single image rain removal boosting via directional gradient. In 2020 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1–6.
[72]
Dongwei Ren, Wangmeng Zuo, Qinghua Hu, Pengfei Zhu, and Deyu Meng. 2019. Progressive image deraining networks: A better and simpler baseline. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3937–3946.
[73]
Wei Shang, Pengfei Zhu, Dongwei Ren, and Hong Shi. 2020. Bilateral recurrent network for single image deraining. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2503–2507. DOI:
[74]
Mingwen Shao, Le Li, Hong Wang, and Deyu Meng. 2021. Selective generative adversarial network for raindrop removal from a single image. Neurocomputing 426 (2021), 265–273.
[75]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
[76]
Chang-Hwan Son and Xiao-Ping Zhang. 2020. Rain detection and removal via shrinkage-based sparse coding and learned rain dictionary. Journal of Imaging Science and Technology 64, 3 (2020), 30501–1.
[77]
Zhipeng Su, Yixiong Zhang, Xiao-Ping Zhang, and Feng Qi. 2022. Non-local channel aggregation network for single image rain removal. Neurocomputing 469 (2022), 261–272.
[78]
Maitreya Suin, Kuldeep Purohit, and A. N. Rajagopalan. 2020. Degradation aware approach to image restoration using knowledge distillation. IEEE Journal of Selected Topics in Signal Processing 15, 2 (2020), 162–173.
[79]
Hao Sun, Marcelo H. Ang, and Daniela Rus. 2019. A convolutional network for joint deraining and dehazing from a single image for autonomous driving in rain. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 962–969.
[80]
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1–9.
[81]
Jing Tan, Yu Zhang, Huiyuan Fu, Huadong Ma, and Ning Gao. 2019. Multi-scale attentive residual network for single image deraining. In International Conference on Human Centered Computing. Springer, 351–362.
[82]
Chiat-Pin Tay, Sharmili Roy, and Kim-Hui Yap. 2019. AANet: Attribute attention network for person re-identifications. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7134–7143.
[83]
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. 2021. Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning. ICML, 10347–10357.
[84]
Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li. 2022. MAXIM: Multi-axis MLP for image processing. (2022), 5769–5780.
[85]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. arXiv preprint arXiv:1706.03762 (2017).
[86]
Cong Wang, Jinshan Pan, and Xiao-Ming Wu. 2022. Online-updated high-order collaborative networks for single image deraining. (2022), 2406–2413.
[87]
Cong Wang, Yutong Wu, Zhixun Su, and Junyang Chen. 2020. Joint self-attention and scale-aggregation for self-calibrated deraining network. In Proceedings of the 28th ACM International Conference on Multimedia. 2517–2525.
[88]
Cong Wang, Man Zhang, Zhixun Su, Yutong Wu, Guangle Yao, and Hongyan Wang. 2019. Learning a multi-level guided residual network for single image deraining. Signal Processing: Image Communication 78 (2019), 206–215.
[89]
Cong Wang, Man Zhang, Zhixun Su, Guangle Yao, Yan Wang, Xiyan Sun, and Xiaonan Luo. 2019. From coarse to fine: A stage-wise deraining net. IEEE Access 7 (2019), 84420–84428. DOI:
[90]
Guoqing Wang, Changming Sun, and Arcot Sowmya. 2021. Attentive feature refinement network for single rainy image restoration. IEEE Transactions on Image Processing 30 (2021), 3734–3747. DOI:
[91]
Hong Wang, Yichen Wu, Minghan Li, Qian Zhao, and Deyu Meng. 2022. Survey on rain removal from videos or a single image. Science China Information Sciences 65, 1 (2022), 111101.
[92]
Hong Wang, Yichen Wu, Qi Xie, Qian Zhao, Yong Liang, Shijun Zhang, and Deyu Meng. 2021. Structural residual learning for single image rain removal. Knowledge-Based Systems 213 (2021), 106595.
[93]
Hong Wang, Qi Xie, Yichen Wu, Qian Zhao, and Deyu Meng. 2020. Single image rain streaks removal: A review and an exploration. International Journal of Machine Learning and Cybernetics 11 (2020), 853–872.
[94]
Hong Wang, Qi Xie, Qian Zhao, and Deyu Meng. 2020. A model-driven deep neural network for single image rain removal. 3103–3112 pages.
[95]
Hong Wang, Zongsheng Yue, Qi Xie, Qian Zhao, Yefeng Zheng, and Deyu Meng. 2021. From rain generation to rain removal. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14791–14801.
[96]
Junsheng Wang, Shan Gai, Xiang Huang, and Hai Zhang. 2021. From coarse to fine: A two stage conditional generative adversarial network for single image rain removal. Digital Signal Processing 111 (2021), 102985.
[97]
Qiang Wang, Gan Sun, Jiahua Dong, and Yulun Zhang. 2022. PFDN: Pyramid feature decoupling network for single image deraining. IEEE Transactions on Image Processing 31 (2022), 7091–7101. DOI:
[98]
Tianyu Wang, Xin Yang, Ke Xu, Shaozhe Chen, Qiang Zhang, and Rynson W. H. Lau. 2019. Spatial attentive single-image deraining with a high quality real rain dataset. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 12270–12279.
[99]
Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. 2021. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 568–578.
[100]
Yinglong Wang, Chao Ma, and Bing Zeng. 2021. Multi-decoding deraining network and quasi-sparsity based training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13375–13384.
[101]
Ye-Tao Wang, Xi-Le Zhao, Tai-Xiang Jiang, Liang-Jian Deng, Yi Chang, and Ting-Zhu Huang. 2020. Rain streaks removal for single image via kernel-guided convolutional neural network. IEEE Transactions on Neural Networks and Learning Systems 32, 8 (2020), 3664–3676.
[102]
Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li. 2022. Uformer: A general u-shaped transformer for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 17683–17693.
[103]
Zheng Wang, Jianwu Li, and Ge Song. 2019. DTDN: Dual-task de-raining network. In Proceedings of the 27th ACM International Conference on Multimedia. 1833–1841.
[104]
Zhen Wang, Cong Wang, Zhixun Su, and Junyang Chen. 2021. Dense feature pyramid grids network for single image deraining. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2025–2029. DOI:
[105]
Wei Wei, Deyu Meng, Qian Zhao, Zongben Xu, and Ying Wu. 2019. Semi-supervised transfer learning for image rain removal. (2019), 3877–3886.
[106]
Yanyan Wei, Zhao Zhang, Yang Wang, Mingliang Xu, Yi Yang, Shuicheng Yan, and Meng Wang. 2021. DerainCycleGAN: Rain attentive cycleGAN for single image deraining and rainmaking. IEEE Transactions on Image Processing 30 (2021), 4788–4801.
[107]
Yanyan Wei, Zhao Zhang, Haijun Zhang, Richang Hong, and Meng Wang. 2019. A coarse-to-fine multi-stream hybrid deraining network for single image deraining. In 2019 IEEE International Conference on Data Mining (ICDM). IEEE, 628–637.
[108]
Xiao Wu, Ting-Zhu Huang, Liang-Jian Deng, and Tian-Jing Zhang. 2022. A decoder-free transformer-like architecture for high-efficiency single image deraining. In Proceedings of the Thirty-first International Joint Conference on Artificial Intelligence, IJCAI-22. International Joint Conferences on Artificial Intelligence Organization, 1474–1480.
[109]
Jie Xiao, Xueyang Fu, Aiping Liu, Feng Wu, and Zheng-Jun Zha. 2022. Image de-raining transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence (2022), 1–18. DOI:
[110]
Jie Xiao, Man Zhou, Xueyang Fu, Aiping Liu, and Zheng-Jun Zha. 2021. Improving de-raining generalization via neural reorganization. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 4987–4996.
[111]
Wenhan Yang, Jiaying Liu, Shuai Yang, and Zongming Guo. 2019. Scale-free single image deraining via visibility-enhanced recurrent wavelet learning. IEEE Transactions on Image Processing 28, 6 (2019), 2948–2961. DOI:
[112]
Wenhan Yang, Robby T. Tan, Jiashi Feng, Zongming Guo, Shuicheng Yan, and Jiaying Liu. 2020. Joint rain detection and removal from a single image with contextualized deep networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 42, 6 (2020), 1377–1393. DOI:
[113]
Wenhan Yang, Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan. 2017. Deep joint rain detection and removal from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1357–1366.
[114]
Wenhan Yang, Robby T. Tan, Shiqi Wang, Yuming Fang, and Jiaying Liu. 2020. Single image deraining: From model-based to data-driven and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 11 (2020), 4059–4077.
[115]
Youzhao Yang and Hong Lu. 2019. Single image deraining using a recurrent multi-scale aggregation and enhancement network. In 2019 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1378–1383.
[116]
Youzhao Yang and Hong Lu. 2019. Single image deraining via recurrent hierarchy enhancement network. In Proceedings of the 27th ACM International Conference on Multimedia. 1814–1822.
[117]
Youzhao Yang and Hong Lu. 2021. A fast and efficient network for single image deraining. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2030–2034.
[118]
Rajeev Yasarla and Vishal M. Patel. 2019. Uncertainty guided multi-scale residual learning-using a cycle spinning CNN for single image de-raining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8405–8414.
[119]
Rajeev Yasarla and Vishal M. Patel. 2020. Confidence measure guided single image de-raining. IEEE Transactions on Image Processing 29 (2020), 4544–4555.
[120]
Rajeev Yasarla, Vishwanath A. Sindagi, and Vishal M. Patel. 2020. Syn2Real transfer learning for image deraining using Gaussian processes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2726–2736.
[121]
Yuntong Ye, Yi Chang, Hanyu Zhou, and Luxin Yan. 2021. Closing the loop: Joint rain generation and removal via disentangled image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2053–2062.
[122]
Weijiang Yu, Zhe Huang, Wayne Zhang, Litong Feng, and Nong Xiao. 2019. Gradual network for single image de-raining. In Proceedings of the 27th ACM International Conference on Multimedia. 1795–1804.
[123]
Ye Yuntong, Yu Changfeng, Chang Yi, Zhu Lin, Zhao Xile, Yan Luxin, and Tian Yonghong. 2022. Unsupervised deraining: Where contrastive learning meets self-similarity. In The IEEE Conference on Computer Vision and Pattern Recognition.
[124]
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. 2022. Restormer: Efficient transformer for high-resolution image restoration. (2022), 5728–5739.
[125]
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. 2021. Multi-stage progressive image restoration. 14821–14831 pages.
[126]
He Zhang and Vishal M. Patel. 2018. Density-aware single image de-raining using a multi-stream dense network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 695–704.
[127]
He Zhang, Vishwanath Sindagi, and Vishal M. Patel. 2019. Image de-raining using a conditional generative adversarial network. IEEE Transactions on Circuits and Systems for Video Technology 30, 11 (2019), 3943–3956.
[128]
Kaihao Zhang, Wenhan Luo, Yanjiang Yu, Wenqi Ren, Fang Zhao, Changsheng Li, Lin Ma, Wei Liu, and Hongdong Li. 2022. Beyond monocular deraining: Parallel stereo deraining network via semantic prior. International Journal of Computer Vision (2022), 1–16.
[129]
Zheyu Zhang, Yurui Zhu, Xueyang Fu, Zhiwei Xiong, Zheng-Jun Zha, and Feng Wu. 2021. Multifocal attention-based cross-scale network for image de-raining. In Proceedings of the 29th ACM International Conference on Multimedia. 3673–3681.
[130]
Jing Zhao, Jiyu Xie, Ruiqin Xiong, Siwei Ma, Tiejun Huang, and Wen Gao. 2019. Pyramid convolutional network for single image deraining. In CVPR Workshops. 9–16.
[131]
Yupei Zheng, Xin Yu, Miaomiao Liu, and Shunli Zhang. 2019. Residual multiscale based single image deraining. In BMVC. 147.
[132]
Yupei Zheng, Xin Yu, Miaomiao Liu, and Shunli Zhang. 2022. Single-image deraining via recurrent residual multiscale networks. IEEE Transactions on Neural Networks and Learning Systems 33, 3 (2022), 1310–1323. DOI:
[133]
Man Zhou, Jie Xiao, Yifan Chang, Xueyang Fu, Aiping Liu, Jinshan Pan, and Zheng-Jun Zha. 2021. Image de-raining via continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4907–4916.
[134]
Hongyuan Zhu, Xi Peng, Joey Tianyi Zhou, Songfan Yang, Vijay Chanderasekh, Liyuan Li, and Joo-Hwee Lim. 2019. Single image rain removal with unpaired information: A differentiable programming perspective. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 9332–9339.
[135]
Honghe Zhu, Cong Wang, Yajie Zhang, Zhixun Su, and Guohui Zhao. 2020. Physical model guided deep image deraining. In 2020 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1–6.
[136]
Weiqi Zou, Yang Wang, Xueyang Fu, and Yang Cao. 2022. Dreaming to prune image deraining networks. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 6013–6022. DOI:
Index Terms
- A Survey of Single Image Rain Removal Based on Deep Learning
Recommendations
Single image deraining using deep convolutional networks
A deep learning-based single image deraining algorithm is proposed in this work. Instead of modeling a rain layer as a linear function between the rain image and its clear version as previous works do, we directly formulate the clear image as the result ...
Rain Removal from a Single Image Using Refined Inception ResNet v2
AbstractThe atmospheric conditions like rain degrade visibility, creating problems for computer vision applications. In single-image de-raining, the lack of temporal information creates challenges. Rain removal requires a high-frequency layer extraction, ...
Image inpainting based on deep learning: A review
AbstractImage inpainting is an important research direction in the study of computer vision, and is widely used in image editing and photo inpainting etc. Traditional image inpainting algorithms are often difficult to deal with large-scale ...
Highlights- Reviewing the image inpainting on deep learning of the past 15 years.
- ...
Comments
Information & Contributors
Information
Published In
April 2024
1026 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 10 November 2023
Online AM: 02 October 2023
Accepted: 18 September 2023
Revised: 15 August 2023
Received: 24 March 2022
Published in CSUR Volume 56, Issue 4
Check for updates
Author Tags
Qualifiers
- Survey
Funding Sources
- Science and Technology Innovation Project of Xiongan New Area
- Science and Technology Key Project of Fujian Province
- National Natural Science Foundation of China
- President’s Fund of Xiamen University for Undergraduate
- Open Project of State Key Laboratory of Matamaterial Electromagnetic Modulation Technology
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 719Total Downloads
- Downloads (Last 12 months)719
- Downloads (Last 6 weeks)106
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in