Transformer-based Multi-scale Feature Aggregation Network for Battlefield Image Deraining
Authors: 
Xiaotian Wan, Xuefeng YanAuthors Info & Claims
Xiaotian Wan, Xuefeng YanAuthors Info & ClaimsPages 109 - 115
Abstract
In rainy scenarios, military target images captured by sensors are occluded by rain, leading to local information loss, which hampers the accurate reception and judgment of battlefield situation information. To tackle this issue, we propose a Transformer-based multi-scale feature aggregation algorithm to eliminate rain from battlefield images. The algorithm utilizes a U-Net architecture with a Transformer module to perform image deraining, introducing a spatial adaptive attention mechanism to enhance feature extraction capability. By aggregating multi-scale stage features of rain images, the model generates more complete and detailed feature representations, thus improving the model's rain removal performance. For simulating real battlefield scenarios with rain, a battlefield rainy day image dataset was constructed for the first time. Experimental results on multiple public datasets show that our proposed algorithm effectively removes rain from both synthetic and real scene images while preserving original details.
References
[1]
Li-Wei Kang, Chia-Wen Lin, and Yu-Hsiang Fu. 2012. Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition. IEEE Transactions on Image Processing, 21, 4 (April 2012), 1742–1755. https://doi.org/10.1109/tip.2011.2179057.
[2]
Yu Li, Robby T. Tan, Xiaojie Guo, Jiangbo Lu, and Michael S. Brown. 2016. Rain Streak Removal Using Layer Priors. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR '16), Las Vegas, NV, USA, 2736-2744. https://doi.org/10.1109/cvpr.2016.299.
[3]
Yu Luo, Yong Xu, and Hui Ji. 2015. Removing Rain from a Single Image via Discriminative Sparse Coding. In 2015 IEEE International Conference on Computer Vision (ICCV '15), Santiago, Chile, 3397-3405. https://doi.org/10.1109/iccv.2015.388.
[4]
Kui Jiang, Zhongyuan Wang, Peng Yi, Chen Chen, Zheng Wang, and Chia-Wen Lin. 2021. PCNET: Progressive Coupled Network for Real-Time Image Deraining. In 2021 IEEE International Conference on Image Processing (ICIP '21), Anchorage, AK, USA, 1759-1763. https://doi.org/10.1109/icip42928.2021.9506318.
[5]
Chenghao Chen and Hao Li. 2021. Robust Representation Learning with Feedback for Single Image Deraining. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR '21), Nashville, TN, USA, 7738-7747. https://doi.org/10.1109/cvpr46437.2021.00765.
[6]
Pengpeng Li, Jiyu Jin, Guiyue Jin, Lei Fan, Xiao Gao, Tianyu Song, and Xiang Chen. 2022. Deep Scale-space Mining Network for Single Image Deraining. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW '22), New Orleans, LA, USA, 4275-4284. https://doi.org/ 10.1109/CVPRW56347.2022.00473.
[7]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17). Curran Associates Inc., Red Hook, NY, USA, 6000–6010.
[8]
Jeya Maria Jose Valanarasu, Rajeev Yasarla, and Vishal M. Patel. 2022. TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR '22), New Orleans, LA, USA, 2343-2353. https://doi.org/10.1109/cvpr52688.2022.00239.
[9]
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV '21), Montreal, QC, Canada, 9992-10002. https://doi.org/10.1109/iccv48922.2021.00986.
[10]
Zheng Chen, Yulun Zhang, Jinjin Gu, Linghe Kong, Xiaokang Yang, and Fisher Yu. 2023. Dual Aggregation Transformer for Image Super-Resolution. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV '23), Paris, France, 12278-12287. https://doi.org/10.1109/ICCV51070.2023.01131.
[11]
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. 2022. Restormer: Efficient Transformer for High-Resolution Image Restoration. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR '22), New Orleans, LA, USA, 5718-5729. https://doi.org/10.1109/cvpr52688.2022.00564.
[12]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 9(September 2015), 1904–1916.
[13]
Wenhan Yang, Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan. 2017. Deep Joint Rain Detection and Removal from a Single Image. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR '17), Honolulu, HI, USA, 1685-1694. https://doi.org/10.1109/cvpr.2017.183.
[14]
Xueyang Fu, Jiabin Huang, Xinghao Ding, Yinghao Liao, and John Paisley. 2017. Clearing the Skies: A deep network architecture for single-image rain removal. IEEE Transactions on Image Processing, 26, 6 (June 2017), 2944–2956. https://doi.org/10.1109/tip.2017.2691802.
[15]
Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, and Hongbin Zha. 2018. Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining. In Computer Vision – ECCV 2018, Lecture Notes in Computer Science. 262–277.
[16]
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. 2021. Multi-Stage Progressive Image Restoration. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR '21), Nashville, TN, 14816-14826. https://doi.org/10.1109/cvpr46437.2021.01458.
[17]
Yuhong He, Long Peng, Lu Wang, and Jun Cheng. 2023. Latent Degradation Representation Constraint for Single Image Deraining. In 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '23), Seoul, Korea, 3155-3159. https://doi.org/10.1109/ICASSP48485.2024.10447510.
[18]
He Zhang, Vishwanath Sindagi, and Vishal M. Patel. 2020. Image De-raining Using a Conditional Generative Adversarial Network. IEEE Transactions on Circuits and Systems for Video Technology, 30, 11 (November 2020), 3943–3956. https://doi.org/10.1109/tcsvt.2019.2920407.
[19]
Tianyu Wang, Xin Yang, Ke Xu, Shaozhe Chen, Qiang Zhang, and Rynson W.H. Lau. 2019. Spatial Attentive Single-Image Deraining With a High Quality Real Rain Dataset. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR '19), Long Beach, CA, 12262-12271. https://doi.org/10.1109/cvpr.2019.01255.
[20]
Chun Ren, Danfeng Yan, Yuanqiang Cai, and Yangchun Li. 2023. Semi-swinderain: Semi-supervised image deraining network using swin transformer. In 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP '23), Rhodes Island, Greece, 1-5, https://doi.org/10.1109/ICASSP49357.2023.10095214.
[21]
Kui Jiang, Wenxuan Liu, Zheng Wang, Xian Zhong, Junjun Jiang, and Chia-Wen Lin. 2023. DAWN: Direction-aware Attention Wavelet Network for Image Deraining. In Proceedings of the 31st ACM International Conference on Multimedia (MM '23). Association for Computing Machinery, New York, NY, USA, 7065–7074. https://doi.org/10.1145/3581783.3611697.
Index Terms
- Transformer-based Multi-scale Feature Aggregation Network for Battlefield Image Deraining
Recommendations
Context-detail-aware United Network for Single Image Deraining
Images captured outdoors are often affected by rainy days, resulting in a severe deterioration in the visual quality of the captured images and a decrease in the performance of related applications. Therefore, single image deraining has attracted ...
Edge-preserving image deraining network using cumulative feature aggregation
AbstractThis study proposes an edge-preserving image deraining network using a wavelet feature aggregation method. Wavelet subbands re correlated with each other, and the high-frequency subband in the horizontal direction is the least affected by rain ...
Highlights- This paper presents a single image deraining network that cumulatively aggregates wavelet subband features.
- The wavelet feature aggregation block integrates long short-term memory-based multiscale attentive rain streak removal blocks.
Multi-receptive Field Aggregation Network for single image deraining
AbstractImage deraining is a significant problem that ensures the visual quality of images to prompt computer vision systems. However, due to the insufficiency of captured rain streaks features and global information, current image deraining ...
Highlights- Propose a Multi-receptive Field Aggregation Network for single image deraining.
Comments
Information & Contributors
Information
Published In
July 2024
335 pages
ISBN:9798400710285
DOI:10.1145/3691016
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 03 October 2024
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
IPICE 2024
IPICE 2024: 2024 International Conference on Image Processing, Intelligent Control and Computer Engineering
July 19 - 21, 2024
Qingdao, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 16Total Downloads
- Downloads (Last 12 months)16
- Downloads (Last 6 weeks)3
Reflects downloads up to 03 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format