research-article

DAWN: Direction-aware Attention Wavelet Network for Image Deraining

Authors:

Chia-Wen LinAuthors Info & Claims

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Pages 7065 - 7074

https://doi.org/10.1145/3581783.3611697

Published: 27 October 2023 Publication History

Abstract

Single image deraining aims to remove rain perturbation while restoring the clean background scene from a rain image. However, existing methods tend to produce blurry and over-smooth outputs, lacking some textural details. Wavelet transform can depict the contextual and textural information of an image at different levels, showing impressive capability of learning structural information in the images to avoid artifacts, and thus has been recently explored to consider the inherent overlap of background and rain perturbation in both the pixel domain and the frequency embedding space. However, the existing wavelet-based methods ignore the heterogeneous degradation for different coefficients due to the inherent directional characteristics of rain streaks, leading to inter-frequency conflicts and compromised deraining results. To address this issue, we propose a novel Direction-aware Attention Wavelet Network (DAWN) for rain streaks removal. DAWN has several key distinctions from existing wavelet transform-based methods: 1) introducing the vector decomposition to parameterize the learning procedure, where the rain streaks are derived into the vertical (V) and horizontal (H) components to learn the specific representation; 2) a novel direction-aware attention module (DAM) to fit the projection and transformation parameters to characterize the direction-specific rain components, which helps accurate texture restoration; 3) exploring practical composite constraints on the structure, details, and chrominance aspects for high-quality background restoration. Our proposed DAWN delivers significant performance gains on nine datasets across image deraining and object detection tasks, exceeding the state-of-the-art method MPRNet by 0.88 dB in PSNR on the Test1200 dataset with only 35.5% computation cost.

References

[1]

Peter C Barnum, Srinivasa Narasimhan, and Takeo Kanade. 2010. Analysis of rain and snow in frequency space. IJCV, Vol. 86, 2--3 (2010), 256.

Digital Library

[2]

Xiang Chen, Jinshan Pan, Kui Jiang, Yufeng Li, Yufeng Huang, Caihua Kong, Longgang Dai, and Zhentao Fan. 2022. Unpaired deep image deraining using dual contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2017--2026.

[3]

Yi-Lei Chen and Chiou-Ting Hsu. 2013. A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In CVPR. 1968--1975.

[4]

S. Deng, M. Wei, J. Wang, Y. Feng, L. Liang, H. Xie, F. L. Wang, and M. Wang. 2020. Detail-recovery Image Deraining via Context Aggregation Networks. In CVPR. 14548--14557. https://doi.org/10.1109/CVPR42600.2020.01457

[5]

Xueyang Fu, Jiabin Huang, Xinghao Ding, Yinghao Liao, and John Paisley. 2017a. Clearing the skies: A deep network architecture for single-image rain removal. IEEE Trans. Image Process., Vol. 26, 6 (2017), 2944--2956.

Digital Library

[6]

Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley. 2017b. Removing rain from single images via a deep detail network. In CVPR. 3855--3863.

[7]

X. Fu, B. Liang, Y. Huang, X. Ding, and J. Paisley. 2020. Lightweight Pyramid Networks for Image Deraining. IEEE Transactions on Neural Networks and Learning Systems, Vol. 31, 6 (2020), 1794--1807.

[8]

Xueyang Fu, Jie Xiao, Yurui Zhu, Aiping Liu, Feng Wu, and Zheng-Jun Zha. 2023. Continual image deraining with hypergraph convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).

Digital Library

[9]

Kshitiz Garg and Shree K Nayar. 2005. When does a camera see rain?. In ICCV, Vol. 2. IEEE, 1067--1074.

Digital Library

[10]

Qibin Hou, Daquan Zhou, and Jiashi Feng. 2021. Coordinate attention for efficient mobile network design. In CVPR. 13713--13722.

[11]

X. Hu, L. Zhu, T. Wang, C. W. Fu, and P. A. Heng. 2021. Single-Image Real-Time Rain Removal Based on Depth-Guided Non-Local Features. IEEE Trans. Image Process., Vol. 30 (2021), 1--1.

[12]

Huaibo Huang, Ran He, Zhenan Sun, and Tieniu Tan. 2017. Wavelet-srnet: A wavelet-based cnn for multi-scale face super resolution. In ICCV. 1689--1697.

[13]

Huaibo Huang, Aijing Yu, Zhenhua Chai, Ran He, and Tieniu Tan. 2021. Selective Wavelet Attention Learning for Single Image Deraining. International Journal of Computer Vision, Vol. 129, 4 (2021), 1282--1300.

Digital Library

[14]

Kui Jiang, Zhongyuan Wang, Chen Chen, Zheng Wang, Laizhong Cui, and Chia-Wen Lin. 2022a. Magic ELF: Image deraining meets association learning and transformer. arXiv preprint arXiv:2207.10455 (2022).

[15]

Kui Jiang, Zhongyuan Wang, Zheng Wang, Peng Yi, Junjun Jiang, Jinsheng Xiao, and Chia-Wen Lin. 2022b. Danet: Image deraining via dynamic association learning. In Proc. 31st Int. Joint Conf. Artif. Intell.

[16]

K. Jiang, Z. Wang, P. Yi, C. Chen, Z. Han, T. Lu, B. Huang, and J. Jiang. 2020a. Decomposition Makes Better Rain Removal: An Improved Attention-guided Deraining Network. IEEE Trans. Circuits Syst. Video Technol. (2020), 1-1. https://doi.org/10.1109/TCSVT.2020.3044887

[17]

K. Jiang, Z. Wang, P. Yi, C. Chen, B. Huang, Y. Luo, J. Ma, and J. Jiang. 2020b. Multi-Scale Progressive Fusion Network for Single Image Deraining. In CVPR. 8343--8352.

[18]

Kui Jiang, Zhongyuan Wang, Peng Yi, Chen Chen, Guangcheng Wang, Zhen Han, Junjun Jiang, and Zixiang Xiong. 2021a. Multi-Scale Hybrid Fusion Network for Single Image Deraining. IEEE Transactions on Neural Networks and Learning Systems (2021), 1--15.

[19]

Kui Jiang, Zhongyuan Wang, Peng Yi, Chen Chen, Xiaofen Wang, Junjun Jiang, and Zixiang Xiong. 2021b. Multi-level memory compensation network for rain removal via divide-and-conquer strategy. IEEE Journal of Selected Topics in Signal Processing, Vol. 15, 2 (2021), 216--228.

[20]

Kui Jiang, Zhongyuan Wang, Peng Yi, Chen Chen, Zheng Wang, Xiao Wang, Junjun Jiang, and Chia-Wen Lin. 2021c. Rain-free and residue hand-in-hand: A progressive coupled network for real-time image deraining. IEEE Trans. Image Process., Vol. 30 (2021), 7404--7418.

Digital Library

[21]

Kui Jiang, Zhongyuan Wang, Peng Yi, Guangcheng Wang, Ke Gu, and Junjun Jiang. 2019. ATMFN: Adaptive-threshold-based multi-model fusion network for compressed face hallucination. IEEE Transactions on Multimedia, Vol. 22, 10 (2019), 2734--2747.

[22]

Li-Wei Kang, Chia-Wen Lin, and Yu-Hsiang Fu. 2012. Automatic single-frame-based rain streaks removal via image decomposition. IEEE Trans. Image Process., Vol. 21, 4 (2012), 3888--3901.

[23]

Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang. 2017. Deep laplacian pyramid networks for fast and accurate super-resolution. In CVPR. 624--632.

[24]

Pengpeng Li, Jiyu Jin, Guiyue Jin, Lei Fan, Xiao Gao, Tianyu Song, and Xiang Chen. 2022. Deep Scale-Space Mining Network for Single Image Deraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4276--4285.

[25]

Ruoteng Li, Loong-Fah Cheong, and Robby T Tan. 2017. Single image deraining using scale-aware multi-stage recurrent network. arXiv preprint arXiv:1712.06830 (2017).

[26]

Siyuan Li, Iago Breno Araujo, Wenqi Ren, Zhangyang Wang, Eric K Tokuda, Roberto Hirata Junior, Roberto Cesar-Junior, Jiawan Zhang, Xiaojie Guo, and Xiaochun Cao. 2019a. Single image deraining: A comprehensive benchmark analysis. In CVPR. 3838--3847.

[27]

Siyuan Li, Wenqi Ren, Jiawan Zhang, Jinke Yu, and Xiaojie Guo. 2019b. Single image rain removal via a deep decomposition--composition network. Computer Vision and Image Understanding, Vol. 186 (2019), 48--57.

Digital Library

[28]

Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, and Hongbin Zha. 2018. Recurrent squeeze-and-excitation context aggregation net for single image deraining. In ECCV. 254--269.

[29]

Yu Li, Robby T Tan, Xiaojie Guo, Jiangbo Lu, and Michael S Brown. 2016. Rain streak removal using layer priors. In CVPR. 2736--2744.

[30]

G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, and Y. Ma. 2013. Robust Recovery of Subspace Structures by Low-Rank Representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, 1 (2013), 171--184.

Digital Library

[31]

Lixiong Liu, Bao Liu, Hua Huang, and Alan Conrad Bovik. 2014. No-reference image quality assessment based on spatial and spectral entropies. Signal Processing: Image Communication, Vol. 29, 8 (2014), 856--863.

[32]

Pengju Liu, Hongzhi Zhang, Wei Lian, and Wangmeng Zuo. 2019. Multi-level wavelet convolutional neural networks. IEEE Access, Vol. 7 (2019), 74973--74985.

[33]

Yang Liu, Ziyu Yue, Jinshan Pan, and Zhixun Su. 2021. Unpaired Learning for Deep Image Deraining With Rain Direction Regularizer. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4753--4761.

[34]

Y. Luo, Y. Xu, and H. Ji. 2015. Removing Rain from a Single Image via Discriminative Sparse Coding. In ICCV. 3397--3405.

[35]

Stephane Mallat. 1996. Wavelets for a vision. Proc. IEEE, Vol. 84, 4 (1996), 604--614.

[36]

Stephane G Mallat. 1989. A theory for multiresolution signal decomposition: The wavelet representation. IEEE transactions on pattern analysis and machine intelligence, Vol. 11, 7 (1989), 674--693.

[37]

Sachin Mehta, Mohammad Rastegari, Linda Shapiro, and Hannaneh Hajishirzi. 2019. Espnetv2: A light-weight, power efficient, and general purpose convolutional neural network. In CVPR. 9190--9200.

[38]

Anish Mittal, Rajiv Soundararajan, and Alan C Bovik. 2012. Making a ?completely blind" image quality analyzer. IEEE Signal Process. Lett., Vol. 20, 3 (2012), 209--212.

[39]

Yuuto Nanba, Hikaru Miyata, and Xian-Hua Han. 2022. Dual heterogeneous complementary networks for single image deraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 568--577.

[40]

Rui Qian, Robby T Tan, Wenhan Yang, Jiajun Su, and Jiaying Liu. 2018. Attentive generative adversarial network for raindrop removal from a single image. In CVPR. 2482--2491.

[41]

Joseph Redmon and Ali Farhadi. 2018. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 (2018).

[42]

Dongwei Ren, Wangmeng Zuo, Qinghua Hu, Pengfei Zhu, and Deyu Meng. 2019. Progressive image deraining networks: a better and simpler baseline. In CVPR. 3937--3946.

[43]

Harold H Szu, Brian A Telfer, and Shubha L Kadambe. 1992. Neural network adaptive wavelets for signal representation and classification. Optical Engineering, Vol. 31, 9 (1992), 1907--1916.

[44]

Ye-Tao Wang, Xi-Le Zhao, Tai-Xiang Jiang, Liang-Jian Deng, Yi Chang, and Ting-Zhu Huang. 2020. Rain streaks removal for single image via kernel-guided convolutional neural network. IEEE Transactions on Neural Networks and Learning Systems, Vol. 32, 8 (2020), 3664--3676.

[45]

Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, et al. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., Vol. 13, 4 (2004), 600--612.

Digital Library

[46]

Pengxu Wei, Ziwei Xie, Hannan Lu, Zongyuan Zhan, Qixiang Ye, Wangmeng Zuo, and Liang Lin. 2020. Component divide-and-conquer for real-world image super-resolution. In ECCV. Springer, 101--117.

[47]

Yi Xiao, Xin Su, Qiangqiang Yuan, Denghong Liu, Huanfeng Shen, and Liangpei Zhang. 2022. Satellite Video Super-Resolution via Multiscale Deformable Convolution Alignment and Temporal Grouping Projection. IEEE Trans. Geosci. Remote. Sens., Vol. 60 (2022), 1--19.

[48]

Hao-Hsiang Yang, Chao-Han Huck Yang, and Yu-Chiang Frank Wang. 2020. Wavelet channel attention module with a fusion network for single image deraining. In ICIP. IEEE, 883--887.

[49]

Wenhan Yang, Jiaying Liu, Shuai Yang, and Zongming Guo. 2019. Scale-free single image deraining via visibility-enhanced recurrent wavelet learning. IEEE Trans. Image Process., Vol. 28, 6 (2019), 2948--2961.

[50]

Wenhan Yang, Robby T Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan. 2017. Deep joint rain detection and removal from a single image. In CVPR. 1357--1366.

[51]

Xi Yang, Wangmeng Xiang, Hui Zeng, and Lei Zhang. 2021. Real-world Video Super-resolution: A Benchmark Dataset and A Decomposition based Learning Scheme. In ICCV. 4781--4790.

[52]

Rajeev Yasarla and Vishal M Patel. 2019. Uncertainty Guided Multi-Scale Residual Learning-Using a Cycle Spinning CNN for Single Image De-Raining. In CVPR. 8405--8414.

[53]

Changfeng Yu, Yi Chang, Yi Li, Xile Zhao, and Luxin Yan. 2021a. Unsupervised image deraining: Optimization model driven deep cnn. In Proceedings of the 29th ACM International Conference on Multimedia. 2634--2642.

Digital Library

[54]

Yingchen Yu, Fangneng Zhan, Shijian Lu, Jianxiong Pan, Feiying Ma, Xuansong Xie, and Chunyan Miao. 2021b. WaveFill: A Wavelet-based Generation Network for Image Inpainting. In CVPR. 14114--14123.

[55]

Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. 2021. Multi-Stage Progressive Image Restoration. In CVPR.

[56]

He Zhang and Vishal M Patel. 2017. Convolutional sparse and low-rank coding-based rain streak removal. In WACV. IEEE, 1259--1267.

[57]

He Zhang and Vishal M Patel. 2018. Density-aware single image de-raining using a multi-stream dense network. In CVPR. 695--704.

[58]

He Zhang, Vishwanath Sindagi, and Vishal M Patel. 2020. Image de-raining using a conditional generative adversarial network. IEEE Trans. Circuits Syst. Video Technol., Vol. 30, 11 (2020), 3943--3956.

[59]

Junhao Zhuang, Yisi Luo, Xile Zhao, Taixiang Jiang, and Bichuan Guo. 2022. UConNet: Unsupervised Controllable Network for Image and Video Deraining. In Proceedings of the 30th ACM International Conference on Multimedia. 5436--5445.

Digital Library

[60]

Weiqi Zou, Yang Wang, Xueyang Fu, and Yang Cao. 2022. Dreaming to prune image deraining networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6023--6032.

Cited By

Wang CYan THuang WChen XXu KChang X(2025)APANet: Asymmetrical Parallax Attention Network for Efficient Stereo Image DerainingIEEE Transactions on Computational Imaging10.1109/TCI.2025.352714211(101-115)Online publication date: 2025
https://doi.org/10.1109/TCI.2025.3527142
Wang XZhang YXu JGao J(2024)A review on learning based image reflection removal algorithmsIntelligent Data Analysis10.3233/IDA-230904(1-27)Online publication date: 9-Jun-2024
https://doi.org/10.3233/IDA-230904
Tao WYan XWang YWei M(2024)DWTN: deep wavelet transform network for lightweight single image derainingAdvances in Continuous and Discrete Models10.1186/s13662-024-03843-22024:1Online publication date: 30-Sep-2024
https://doi.org/10.1186/s13662-024-03843-2
Show More Cited By

Index Terms

DAWN: Direction-aware Attention Wavelet Network for Image Deraining
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Machine learning
    1. Machine learning algorithms

Recommendations

Single image deraining with dual U-Net generative adversarial network
Abstract
Most of the existing deraining methods cannot preserve the details of the image while removing the rain streaks. To solve this problem, we propose a single image de-raining method with dual U-Net generative adversarial network (DU-GAN). By using ...
Single image deraining using deep convolutional networks

A deep learning-based single image deraining algorithm is proposed in this work. Instead of modeling a rain layer as a linear function between the rain image and its clear version as previous works do, we directly formulate the clear image as the result ...
Multi-level wavelet network based on CNN-Transformer hybrid attention for single image deraining
Abstract
Removing rain streaks from rainy images can improve the accuracy of computer vision applications such as object detection. In order to make full use of the frequency domain analysis characteristics of wavelet and combine the advantages of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

October 2023

9913 pages

ISBN:9798400701085

DOI:10.1145/3581783

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada & MBZUAI, UAE
,
Tao Mei
HiDream.ai, China
,
Rita Cucchiara
University of Modena and Reggio Emilia, Italy
,
Program Chairs:
Marco Bertini
University of Florence, Italy
,
Diana Patricia Tobon Vallejo
Unversidad de Medellin, Colombia
,
Pradeep K. Atrey
University at Albany, State University of New York, USA
,
M. Shamim Hossain
M. Shamim Hossain (King Saud University, KSA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
Hubei Key R&D

Conference

MM '23

Sponsor:

SIGMM

MM '23: The 31st ACM International Conference on Multimedia

October 29 - November 3, 2023

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
397
Total Downloads

Downloads (Last 12 months)303
Downloads (Last 6 weeks)17

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang CYan THuang WChen XXu KChang X(2025)APANet: Asymmetrical Parallax Attention Network for Efficient Stereo Image DerainingIEEE Transactions on Computational Imaging10.1109/TCI.2025.352714211(101-115)Online publication date: 2025
https://doi.org/10.1109/TCI.2025.3527142
Wang XZhang YXu JGao J(2024)A review on learning based image reflection removal algorithmsIntelligent Data Analysis10.3233/IDA-230904(1-27)Online publication date: 9-Jun-2024
https://doi.org/10.3233/IDA-230904
Tao WYan XWang YWei M(2024)DWTN: deep wavelet transform network for lightweight single image derainingAdvances in Continuous and Discrete Models10.1186/s13662-024-03843-22024:1Online publication date: 30-Sep-2024
https://doi.org/10.1186/s13662-024-03843-2
Keh ZWong LLoh YGu KLin W(2024)KBY-Net: A Dual Learning Framework for Improving Object Detection in Rainy Weather ConditionsProceedings of the 6th ACM International Conference on Multimedia in Asia10.1145/3696409.3700246(1-7)Online publication date: 3-Dec-2024
https://dl.acm.org/doi/10.1145/3696409.3700246
Wan XYan X(2024)Transformer-based Multi-scale Feature Aggregation Network for Battlefield Image DerainingProceedings of the 2024 International Conference on Image Processing, Intelligent Control and Computer Engineering10.1145/3691016.3691036(109-115)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3691016.3691036
Cheng DLi YZhang DWang NSun JGao X(2024)Progressive Negative Enhancing Contrastive Learning for Image Dehazing and BeyondIEEE Transactions on Multimedia10.1109/TMM.2024.338249326(8783-8798)Online publication date: 27-Mar-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3382493
Zhang RYu JChen JLi GLin LWang D(2024)A Prior Guided Wavelet-Spatial Dual Attention Transformer Framework for Heavy Rain Image RestorationIEEE Transactions on Multimedia10.1109/TMM.2024.335948026(7043-7057)Online publication date: 29-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3359480
Duan WJi LChen SZhu SYe M(2024)Triple-Domain Feature Learning With Frequency-Aware Memory Enhancement for Moving Infrared Small Target DetectionIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.345217562(1-14)Online publication date: 2024
https://doi.org/10.1109/TGRS.2024.3452175
Xiang DChen SLiu MShi FCheng X(2024)Multilevel Signal Decomposition Layer-Specific Residual Network for Blade Icing PredictionIEEE Signal Processing Letters10.1109/LSP.2024.344669231(2160-2164)Online publication date: 2024
https://doi.org/10.1109/LSP.2024.3446692
Jiang KWang RXiao YJiang JXu XLu T(2024)Image Enhancement via Associated Perturbation Removal and Texture Reconstruction LearningIEEE/CAA Journal of Automatica Sinica10.1109/JAS.2024.12452111:11(2253-2269)Online publication date: Nov-2024
https://doi.org/10.1109/JAS.2024.124521
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten