Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Lightweight vision Mamba for weather-degraded remote sensing image restoration

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In the field of remote sensing, the acquired images are often severely degraded due to adverse weather conditions, such as haze and raindrops, posing significant challenges for subsequent visual tasks. Although CNN and Transformer have been widely applied to address these issues, they struggle to balance the relationship between global scene recovery, local detail preservation, and computational efficiency, leading to an imbalance between model performance and efficiency. To this end, we propose a lightweight and efficient visual state space model for remote sensing image restoration. Specifically, we propose the Efficient Vision Mamba Block as the core component of the model, incorporating the State Space Model to leverage its linear complexity for modeling long-range dependencies. Furthermore, we design a multi-router scanning strategy to perform global modeling of remote sensing images, capturing large spatial features from different routes and directions. Compared with existing methods that employ fixed-direction scanning, our approach avoids information redundancy caused by repeated scanning, making the model better adaptable to the complex and changeable weather conditions. Extensive experiments validate the superiority of our proposed model, outperforming state-of-the-art methods on both the StateHaze1k and UAV-Rain1k datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

We do not generate any datasets and the data results of all studies generated during this study are reflected in the article.

References

  1. Chang, W., Chen, H., He, X., Chen, X., Shen, L.: Uav-rain1k: A benchmark for raindrop removal from uav aerial imagery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15–22 (2024)

  2. Chen, Xiang, Li, Yufeng, Dai, Longgang, Kong, Caihua: Hybrid high-resolution learning for single remote sensing satellite image dehazing. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2021)

    MATH  Google Scholar 

  3. Rasti, Behnood, Chang, Yi., Dalsasso, Emanuele, Denis, Loic, Ghamisi, Pedram: Image restoration for remote sensing: overview and toolbox. IEEE Geosci. Remote Sens. Mag. 10(2), 201–230 (2021)

    Article  Google Scholar 

  4. He, Kaiming, Sun, Jian, Tang, Xiaoou: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010)

    MATH  Google Scholar 

  5. Luo, Y., Yong, X., Ji, H.: Removing rain from a single image via discriminative sparse coding. In Proceedings of the IEEE International Conference on Computer Vision, 3397–3405 (2015)

  6. Song, T., Fan, S., Li, P., Jin, J., Jin, G., Fan, L.: Learning an effective transformer for remote sensing satellite image dehazing. IEEE Geosci. Remote Sens. Lett. (2023)

  7. Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., Li, H.: Uformer: A general u-shaped transformer for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 17683–17693 (2022)

  8. Liu, Y., Tian, Y., Zhao, Y., Hongtian, Y., Xie, L., Wang, Y., Ye, Q., Yunfan L.: Visual State Space Model, Vmamba (2024)

  9. Zhu, L., Liao, B., Zhang, Q., Wang, X., Liu, W., Xinggang W.: Efficient Visual Representation Learning with Bidirectional State Space Model, Vision mamba (2024)

  10. Chen, K., Chen, B., Liu, C., Li, W., Zou, Z., Shi, Z.: Rsmamba: Remote sensing image classification with state space model. IEEE Geosci. Remote Sens. Lett. (2024)

  11. Ma, X., Zhang, X., Pun, M.-On.: Rs 3 mamba: Visual state space model for remote sensing image semantic segmentation. IEEE Geosci. Remote Sens. Lett. (2024)

  12. Zhao, S., Chen, H., Zhang, X., Xiao, P., Bai, L., Ouyang, W.: Rs-mamba for large remote sensing image dense prediction. arXiv preprint arXiv:2404.02668 (2024)

  13. Berman, D., Avidan, S., et al.: Non-local image dehazing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1674–1682 (2016)

  14. Yanyun, Q., Chen, Y., Huang, J., Xie, Y.: Enhanced pix2pix dehazing network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8160–8168 (2019)

  15. Li, Yufeng, Chen, Xiang: A coarse-to-fine two-stage attentive network for haze removal of remote sensing images. IEEE Geosci. Remote Sens. Lett. 18(10), 1751–1755 (2020)

    Article  MATH  Google Scholar 

  16. Li, B., Peng, X., Wang, Z., Jizheng, X., Feng, D.: Aod-net: All-in-one dehazing network. In Proceedings of the IEEE International Conference on Computer Vision, 4770–4778 (2017)

  17. Qin, Xu., Wang, Zhilin, Bai, Yuanchao, Xie, Xiaodong, Jia, Huizhu: Ffa-net: feature fusion attention network for single image dehazing. Proc. AAAI Conf. Artif. Intell. 34, 11908–11915 (2020)

    Google Scholar 

  18. Dong, H., Pan, J., Xiang, L., Zhe, H., Zhang, X., Wang, F., Yang, M.-H..: Multi-scale boosted dehazing network with dense feature fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2157–2167 (2020)

  19. Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., Timofte, R.: Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 1833–1844 (2021)

  20. Syed Waqas, Z., Aditya, A., Salman, K., Munawar, H., Fahad Shahbaz, K., Ming-Hsuan, Y.: Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5728–5739, 2022

  21. Chen, L., Chu, X., Zhang, X., Sun J.: Simple baselines for image restoration. In European Conference on Computer Vision, 17–33. Springer, (2022)

  22. Guo, C.-L., Yan, Q., Anwar, S., Cong, R., Ren, W., Li, C.: Image dehazing transformer with transmission-aware 3d position embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5812–5820 (2022)

  23. Guo, H., Li, J., Dai, T., Ouyang, Z., Ren, X, Xia, S.-T.: Mambair: A simple baseline for image restoration with state-space model. arXiv preprint arXiv:2402.15648, (2024)

  24. Yang, C., Chen, Z., Espinosa, M., Ericsson, L., Wang, Z., Liu, J., Crowley, E. J.: Plainmamba: Improving non-hierarchical mamba in visual recognition, (2024)

  25. Li, B., Peng, X., Wang, Z., Jizheng, X., Feng, D.: Aod-net: All-in-one dehazing network. In Proceedings of the IEEE International Conference on Computer Vision, 4770–4778 (2017)

  26. Ullah, H., Muhammad, K., Irfan, M., Anwar, S., Sajjad, M., Imran, A., de Victor Hugo, C., Albuquerque.: Light-dehazenet: a novel lightweight cnn architecture for single image dehazing. IEEE Trans. Image Process., 30, 8968–8982 (2021)

  27. Chen, D., He, M., Fan, Q., Liao, J., Zhang, L., Hou, D., Lu, Y., Hua, G.: Gated context aggregation network for image dehazing and deraining. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV)

  28. Liu, X., Ma, Y., Shi, Z., Chen, J.: Griddehazenet: Attention-based multi-scale network for image dehazing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 7314–7323 (2019)

  29. Li, Shuoshi, Zhou, Yuan, Xiang, Wei: M2scn: multi-model self-correcting network for satellite remote sensing single-image dehazing. IEEE Geosci. Remote Sens. Lett. 20, 1–5 (2022)

    MATH  Google Scholar 

  30. Song, Yuda, He, Zhuqing, Qian, Hui, Xin, Du.: Vision transformers for single image dehazing. IEEE Trans. Image Process. 32, 1927–1941 (2023)

  31. Kulkarni, A., Murala, S.: Aerial image dehazing with attentive deformable transformers. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision pages 6305–6314 (2023)

  32. Wang, H., Xie, Q., Zhao, Q., Meng, D.: A model-driven deep neural network for single image rain removal. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3103–3112 (2020)

  33. Yi, Qiaosi, Li, Juncheng, Dai, Qinyan, Fang, Faming, Zhang, Guixu, Zeng, Tieyong: Structure-preserving deraining with residue channel prior guidance. In Proceedings of the IEEE/CVF International Conference on Computer vision, 4238–4247 (2021)

  34. Xiao, Jie, Xueyang, Fu., Liu, Aiping, Feng, Wu., Zha, Zheng-Jun.: Image de-raining transformer. IEEE Trans. Pattern Anal. Mach. Intell. 45(11), 12978–12995 (2022)

    Article  Google Scholar 

  35. Chen, X., Li, H., Li, M., Pan, J.: Learning a sparse transformer network for effective image deraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5896–5905 (2023)

  36. Huang, B., Zhi, L., Yang, C., Sun, F., Song, Y.: Single satellite optical imagery dehazing using sar image prior based on conditional generative adversarial networks. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 1806–1813 (2020)

  37. Chen, X., Pan, J., Dong, J., Jinhui, T.: A survey and a new benchmark, Towards unified deep image deraining (2023)

Download references

Funding

This work has been supported in part by the Shenyang Science and Technology Project under Grant 23-503-6-18 and the Fundamental Research Funds for the Universities of Liaoning Province LJ232410143060.

Author information

Authors and Affiliations

  1. College of Electronic Information Engineering, Shenyang Aerospace University, Shenyang, 110136, China

    Yufeng Li & Shuang Wu

  2. College of Information Science and Technology, Dalian Maritime University, Dalian, 116026, China

    Hongming Chen

Authors
  1. Yufeng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongming Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yufeng Li and Shuang Wu wrote the main manuscript text, and Hongming Chen prepared all figures and tables. All authors reviewed the manuscript.

Corresponding author

Correspondence to Shuang Wu.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Wu, S. & Chen, H. Lightweight vision Mamba for weather-degraded remote sensing image restoration. SIViP 19, 180 (2025). https://doi.org/10.1007/s11760-024-03767-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11760-024-03767-0

Keywords

Search

Navigation