Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Single image deraining with dual U-Net generative adversarial network

  • Published:
Multidimensional Systems and Signal Processing Aims and scope Submit manuscript

Abstract

Most of the existing deraining methods cannot preserve the details of the image while removing the rain streaks. To solve this problem, we propose a single image de-raining method with dual U-Net generative adversarial network (DU-GAN). By using two U-Net with stronger learning ability as our generator DU-GAN can not only accurately remove more rain streaks but also preserve image details. The network can make full use of image information and extract complete image features. The adversarial loss function using the proposed dual U-Net generator is utilized to generate de-rained images which are close to the ground truth. Furthermore, to obtain the better visual effects of the generated image, The L1 and structure similarity loss functions which are consistent with the human visual effect are applied to generate the final output. The synthetic rainy image datasets and real rainy image datasets are used to evaluate the effectiveness of the proposed network in the experiments. The quantitative and visual experimental results show that the proposed single image deraining method achieves state-of-the-art compared with the other single image deraining methods. The source code can be found at https://github.com/LuBei-design/DU-GAN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig.3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Barnum, P. C., Narasimhan, S., & Kanade, T. (2010). Analysis of rain and snow in frequency space. International Journal of Computer Vision, 86(2–3), 256.

    Article  Google Scholar 

  • Fu, X., Huang, J., & Ding, X. (2017a). Clearing the skies: A deep network architecture for single-image deraining. IEEE Transactions on Image Processing., 26(6), 2944–2956. https://doi.org/10.1109/TIP.2017.2691802

    Article  MathSciNet  MATH  Google Scholar 

  • Fu, X., Huang, J., & Zeng, D. (2017b). Removing rain from single images via a deep detail network. In: Proceedings of the 2017 IEEE Conference on Computer Vision Pattern Recognition. pp.1715–1723. https://doi.org/10.1109/CVPR.2017.186

  • Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. C., & Bengio, Y. (2014). Generative adversarial networks. Advances in Neural Information Processing Systems, 3, 2672–2680.

    Google Scholar 

  • Hu, X., Wang, W., & Pang, C. (2019). Rain-density squeeze-and-excitation residual network for single image rain-removal. In: Proceedings of the 11th Eleventh International Conference on Advanced Computational Intelligence. pp.284–289. https://doi.org/10.1109/ICACI.2019.8778583

  • Kim, J.H., Sim, J.Y., & Kim, C.S. (2014). Stereo video deraining and desnowing based on spatiotemporal frame warping. In: Proceedings of the 2014 IEEE International Conference on Image Processing. pp.5432–5436

  • Kim, J. H., Sim, J. Y., & Kim, C. S. (2015). Video deraining and desnowing using temporal correlation and low-rank matrix completion. IEEE Transactions Image Processing, 24(9), 2658–2670. https://doi.org/10.1109/TIP.2015.2428933

    Article  MathSciNet  MATH  Google Scholar 

  • Li, M., Xie, Q., & Zhao, Q. (2018). Video rain streak removal by multiscale convolutional sparse coding. In: Proceedings of the 2018 IEEE Conference on Computer Vision Pattern Recognition. pp. 6644–6653. https://doi.org/10.1109/CVPR.2018.00695

  • Li, Y., Tan, R.T., & Guo, X. (2016). Rain streak removal using layer priors. In: Proceedings of the 2016 IEEE Conference on Computer Vision Pattern Recognition. pp.2736–2744

  • Liu, C., Pang, Y., & Wang, J. (2014). Frequency domain directional filtering based rain streaks removal from a single color image. In International Conference on Intelligent Computing. pp: 415–424.

  • Liu, J., Teng, S., Li, Z. (2019a). Removing rain from single image based on details preservation and background enhancement. In: Proceedings of the 2019 IEEE 2nd International Conference on Information Communication and Signal Processing. pp. 322–326. https://doi.org/10.1109/ICICSP48821.2019.8958586

  • Liu, J., Yang, W., Yang, S., & Guo, Z. (2018). Erase or Fill? Deep joint recurrent rain removal and reconstruction in videos. In: Proceedings of the 2018 IEEE Conference on Computer Vision Pattern Recognition. pp. 3233–3242. https://doi.org/10.1109/CVPR.2018.00341

  • Liu, J., Yang, W., Yang, S., & Guo, Z. (2019b). D3R-Net: Dynamic routing residue recurrent network for video deraining. IEEE Transactions Image Processing., 28, 699–712. https://doi.org/10.1109/TIP.2018.2869722

    Article  MathSciNet  MATH  Google Scholar 

  • Luo, Y., Xu, Y., & Ji, H. (2015) Removing rain from a single image via discriminative sparse coding. In: 2015 IEEE International Conference on Computer Vision (ICCV). pp. 3397–3405

  • Matsui, T., & Ikehara, M. (2020). GAN-based rain noise removal from single-image considering rain composite models. IEEE Access, 8, 40892–40900. https://doi.org/10.1109/ACCESS.2020.2976761

    Article  Google Scholar 

  • Mi, Z., Shang, J., Zhou, H., & Wang, M. (2016). Image fusion-based video deraining using sparse representation. Electronics Letters., 52, 1528–1529. https://doi.org/10.1049/el.2016.1451

    Article  Google Scholar 

  • Park, K., Yu, S., & Jeong, J. (2018). A contrast restoration method for effective single image deraining algorithm. International Workshop on Advanced Image Technology., 2018, 1–4. https://doi.org/10.1109/IWAIT.2018.8369644

    Article  Google Scholar 

  • Peng, L., Jiang, A., Yi, Q., & Wang, M. (2020). Cumulative rain density sensing network for single image derain. IEEE Signal Processing Letters., 27, 406–410. https://doi.org/10.1109/LSP.2020.2974691

    Article  Google Scholar 

  • Pu, J., Chen, X., & Zhang, L. (2018). Removing rain based on a cycle generative adversarial network. In Proceedings of the 13th IEEE Conference Industrial Electronics Apply. pp. 621–626. https://doi.org/10.1109/ICIEA.2018.8397790

  • Qian, R., Tan, R. T., & Yang, W. (2018). Attentive generative adversarial network for raindrop removal from a single image. In: Proceedings of the 2018 IEEE Conference on Computer Vision Pattern Recognition. pp. 2482–2491. https://doi.org/10.1109/CVPR.2018.00263

  • Ren, W., Tian, J., & Han, Z. (2017). Video desnowing and deraining based on matrix decomposition. In: Proceedings of the 2017 IEEE Conference on Computer Vision Pattern Recognition. pp. 2838–2847. https://doi.org/10.1109/CVPR.2017.303

  • Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention. https://doi.org/10.1007/978-3-319-24574-4_28

    Article  Google Scholar 

  • Tripathi, A. K., & Mukhopadhyay, S. (2012). Video post processing: Low-latency spatiotemporal approach for detection and removal of rain. IET Image Process, 6(2), 181–196.

    Article  MathSciNet  Google Scholar 

  • Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transaction Image Processing, 13(4), 600–612.

    Article  Google Scholar 

  • Xiao, J., Zou, W., & Chen, Y. (2018). Single image deraining based on depth of field and sparse coding. Pattern Recognition Letters. https://doi.org/10.1016/j.patrec.2018.10.006

    Article  Google Scholar 

  • Yang, W., Liu, J., & Feng, J. (2019). Frame-consistent recurrent video deraining with dual-level flow. In: Proceedings of the 2019 IEEE Conference on Computer Vision Pattern Recognition. pp. 1661–1670. https://doi.org/10.1109/CVPR.2019.00176

  • Zhang, H., & Patel, V. M. (2018). Density-aware single image de-raining using a multi-stream dense network. In: Proceedings of the 2018 IEEE Conference on Computer Vision Pattern Recognition. pp. 695–704. https://doi.org/10.1109/CVPR.2018.00079

  • Zhang, H., Sindagi, V., & Patel, V. M. (2019). Image de-raining using a conditional generative adversarial network. IEEE Transactions Circuits Systems Video Technology., 30(11), 3943–3956. https://doi.org/10.1109/tcsvt.2019.2920407

    Article  Google Scholar 

  • Zhao, H., Gallo, O., Frosio, I., & Kautz, J. (2017). Loss functions for image restoration with neural networks. IEEE Transactions Computing Imaging., 3(1), 47–57.

    Article  Google Scholar 

  • Zhu, L., Fu, C. W., Lischinski, D., & Heng, P. A. (2017). Joint bi-layer optimization for single-image rain streak removal. In: Proceedings of the 2017 IEEE Conference on Computer Vision. pp. 2545–2553. https://doi.org/10.1109/ICCV.2017.276

  • Zhu, M., Fang, C., & Du, H. (2019). Application research on improved CGAN in image raindrop removal. The Journal of Engineering., 2019, 8404–8408. https://doi.org/10.1049/joe.2019.1092

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 62061032, 61866027; Natural Science Foundation of Youth Key Project of Jiangxi Province under Grant 20192ACB21032; Jiangxi Science Foundation under Grant 20202BABL202038.

Author information

Authors and Affiliations

  1. School of Information Engineering, Nanchang Hangkong University, Nanchang, 330063, Jiangxi, People’s Republic of China

    Bei Lu, Shan Gai, Bangshu Xiong & Jiazhou Wu

Authors
  1. Bei Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shan Gai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bangshu Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiazhou Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shan Gai.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, B., Gai, S., Xiong, B. et al. Single image deraining with dual U-Net generative adversarial network. Multidim Syst Sign Process 33, 485–499 (2022). https://doi.org/10.1007/s11045-021-00806-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11045-021-00806-8

Keywords

Search

Navigation