Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Learning to remove sandstorm for image enhancement

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In the dusty weather, the suspended sand particles in the air lead to various degrees of degradation for the captured images. Commonly, severe color deviations, low contrast, and blurring details arise in the images. In this paper, a method equipped with color correction and an unsupervised learning network is proposed to enhance sandstorm images. First, the color correction is carried out by color compensation followed by the radical histogram equalization which enables the recovery of the natural color of an image and relieves the pressure of subsequent network that focuses on the enhancement of details. Second, the three sub-networks including atmospheric light with accurate initialization, transmission map, and clear image estimation layers are introduced to remove haze-like effects and reconstruct the clear image by unsupervised learning, which solves the lacking of paired sandstorm images and clear counterparts. To perform a comprehensive study of the state-of-the-art sandstorm image enhancement methods, we propose a Sandstorm Image Enhancement (SIE) dataset which can benchmark the performance of different methods and make it possible to train sandstorm image enhancement networks. The experimental results demonstrate that our method outperforms several state-of-the-arts both qualitatively and quantitatively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Zhen, W., Wang, X., Duan, W., Li, F., Chen, F.: Well Production Real-Time Intelligent Monitoring Based on Convolutional Neural Network. Springer Series in Geomechanics and Geoengineering, pp. 39–49 (2019)

  2. Manzanilla, A., Reyes Sanchez, S., Garcia Rangel, M.A., Mercado Ravell, D.A., Lozano, R.: Autonomous navigation for unmanned underwater vehicles: real-time experiments using computer vision. IEEE Robot. Autom. Lett. 4(2), 1351–1356 (2019)

    Article  Google Scholar 

  3. Al-Shakarji, N. M., Bunyak, F., Seetharaman, G., Palaniappan, K.: Vehicle Tracking in Wide Area Motion Imagery using KC- LoFT Multi-Feature Discriminative Modeling. In: IEEE Applied Imagery Pattern Recognition Workshop (AIPR), pp. 1–6 (2017)

  4. Chen, Y., Yang, W., Tan, H., Yang, Y., Hao, N., Yang, K.: Image enhancement for LD based imaging in turbid water. Optik 127(2), 517–521 (2016)

    Article  Google Scholar 

  5. Long, M., Li, Z., Xie, X., Li, G., Wang, Z.: Adaptive image enhancement based on guide image and fraction-power transformation for wireless capsule endoscopy. IEEE Trans. Biomed. Circuits Syst. 12, 993–1003 (2018)

    Article  Google Scholar 

  6. Xiao, L., Heide, F., Heidrich, W., Scholkopf, B., Hirsch, M.: Discriminative transfer learning for general image restoration. IEEE Trans. Image Process. 27(99), 4091–4104 (2017)

    MathSciNet  MATH  Google Scholar 

  7. Zhang, Y., Sun, L., Yan, C., Ji, X., Dai, Q.: Adaptive residual networks for high-quality image restoration. IEEE Trans. Image Process. 27(7), 3150–3163 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  8. Fattal, R.: Single image dehazing. ACM Trans. Graphics 27(3), 1–9 (2008)

    Article  Google Scholar 

  9. Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: Dehazenet: an end-to-end system for single image haze removal. IEEE Trans. Image Process. 25(11), 5187–5198 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  10. Zhu, H., Xi, P., Chandrasekhar, V., Li, L., Lim, J. H.: DehazeGAN: when image dehazing meets differential programming. In: Twenty-Seventh International Joint Conferences on Artificial Intelligence (IJCAI), pp. 1234–1240 (2018)

  11. Li, B., Gou, Y., Liu, J.Z., Zhu, H., Peng, X.: Zero-Shot Image dehazing. IEEE Trans. Image Process. 29, 8457–8466 (2020)

    Article  MATH  Google Scholar 

  12. Li, C., Guo, J., Guo, C.: Emerging from water: underwater image color correction based on weakly supervised color transfer. IEEE Signal Process. Lett. 25(3), 323–327 (2018)

    Article  Google Scholar 

  13. Li, C., Guo, C., Chen, C. L.: Learning to enhance low-light image via zero-reference deep curve estimation. IEEE Trans. Pattern Anal. Mach. Intell. 1–1 (2021)

  14. Wang, Y., Guo, J., Gao, H., Yue, H.: UIEC2-Net: CNN-based underwater image enhancement using two color space. Signal Process.: Image Commun. 96, 1–1 (2021)

    Google Scholar 

  15. Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M. H.: Multi-stage progressive image restoration. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), (2021)

  16. Peng, Y.-T., Cao, K., Cosman, P.C.: Generalization of the dark channel prior for single image restoration. IEEE Trans. Image Process. 27(6), 2856–2868 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  17. Buchsbaum, G.: A spatial processor model for object colour perception. J. Franklin Inst. 310(1), 337–350 (1980)

    Article  Google Scholar 

  18. Brainard, D.H.: Analysis of the retinex theory of color vision. J. Opt. Soc. Am. A-Opt. Image Sci. Vis. 3(10), 1651–1661 (1986)

    Article  Google Scholar 

  19. Weijer, J., Gevers, T., Gijsenij, A.: Edge-based color constancy. IEEE Trans. Image Process. 16(9), 2207–2214 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  20. Kobus, Barnard.: Improvements to gamut mapping colour constancy algorithms. In: European Conference on Computer Vision (ECCV), pp. 390–403 (2000)

  21. Brainard, D.H., Freeman, W.T.: Bayesian color constancy. J. Opt. Soc. Am. A 14(7), 1393–1411 (1997)

    Article  Google Scholar 

  22. Jobson, D.J., Rahman, Z.U., Woodell, G.A.: Properties and performance of a center/surround retinex. IEEE Trans. Image Process. 6(3), 451–462 (1997)

    Article  Google Scholar 

  23. Rahman, Z. U., Jobson, D. J., Woodell, G. A.: Multi-scale retinex for color image enhancement. In: Proceedings of 3rd IEEE International Conference on Image Processing, pp. 1003–1006 (1996)

  24. Jobson, D.J., Rahman, Z., Woodell, G.A.: A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans. Image Process. 6(7), 965–976 (2002)

    Article  Google Scholar 

  25. Fu, X., Yue, H., Zeng, D., Zhang, X. P., Ding, X.: A fusion-based enhancing approach for single sandstorm image. In: 2014 IEEE 16th International Workshop on Multimedia Signal Processing (MMSP), pp. 1–5 (2014)

  26. Yan, T., Wang, L., Wang, J.: Method to enhance degraded image in dust environment. J. Software. 9(10), 2672–2677 (2014)

    Article  Google Scholar 

  27. Huang, S.C., Ye, J.H., Chen, B.H.: An advanced single-image visibility restoration algorithm for real-world hazy scenes. IEEE Trans. Industr. Electron. 62(5), 2962–2972 (2015)

    Article  Google Scholar 

  28. Al-Ameen, Z.: Visibility enhancement for images captured in dusty weather via tuned tri-threshold fuzzy intensification operations. Int. J. Intell. Syst. Technol. Appl. 8(8), 10–17 (2016)

    Google Scholar 

  29. Wang, J., Pang, Y., He, Y., Liu, C.: Enhancement for dust-sand storm images. In: International Conference on Multimedia Modeling (MMM), pp. 842–849 (2016)

  30. Shi, Z., Feng, Y., Zhao, M., Zhang, E., He, L.: Normalised gamma transformation-based contrast-limited adaptive histogram equalisation with colour correction for sand-dust image enhancement. IET Image Process. 14(4), 747–756 (2019)

    Article  Google Scholar 

  31. Park, T.H., Eom, I.K.: Sand-dust image enhancement using successive color balance with coincident chromatic histogram. IEEE Access. 9, 19749–19760 (2021)

    Article  Google Scholar 

  32. Ning, Z., Mao, S., Mei, L.: Visibility restoration algorithm of dust-degraded images. J. Image Graph. 21(12), 1585–1592 (2016)

    Google Scholar 

  33. Yu, S., Zhu, H., Wang, J., Fu, Z., Xue, S., Shi, H.: Single sand-dust image restoration using information loss constraint. J. Mod. Opt. 63(21), 2121–2130 (2016)

    Article  Google Scholar 

  34. Pan, H., Tian, R., Liu, C., Gong, C.: A sand-dust degraded image enhancement algorithm based on color correction and information loss constraints. Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao/J. Comput.-Aid. Design Comput. Graph. 30(6), 992–999 (2018)

    Google Scholar 

  35. Shi, Z., Feng, Y., Zhao, M., Zhang, E., He, L.: Let you see in sand dust weather: a method based on halo-reduced dark channel prior dehazing for sand-dust image enhancement. IEEE Access. 7(99), 116722–116733 (2019)

    Article  Google Scholar 

  36. Gao, G.X., Lai, H.C., Jia, Z., Liu, Y.Q., Wang, Y.L.: Sand-dust image restoration based on reversing the blue channel prior. IEEE Photon. J. 12(2), 1–16 (2020)

    Google Scholar 

  37. Cheng, Y., Jia, Z., Lai, H., Yang, J., Kasabov, N.K.: Blue channel and fusion for sandstorm image enhancement. IEEE Access. 8, 66931–66940 (2020)

    Article  Google Scholar 

  38. Kim, S.E., Park, T.H., Eom, I.K.: Fast single image dehazing using saturation based transmission map estimation. IEEE Trans. Image Process. 29, 1985–1998 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  39. Wang, B., Wei, B., Kang, Z., Hu, L., Li, C.: Fast color balance and multi-path fusion for sandstorm image enhancement. Signal Image Video Process. 15(9), 637–644 (2021)

    Article  Google Scholar 

  40. Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmentation. In: International Conference on Medical image computing and computer-assisted intervention, pp. 234–241 (2015)

  41. Fu, X., Cao, X.: Underwater image enhancement with global–local networks and compressed-histogram equalization. Signal Process.: Image Commun. 86, 115892 (2020)

    Google Scholar 

  42. Gandelsman, Y., Shocher, A., Irani, M.:“Double-DIP”: unsupervised image decomposition via coupled deep-image-priors. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11018–11027 (2019)

  43. He, K., Jian, S., Fellow, I.E.E.E., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2011)

    Article  Google Scholar 

  44. D Berman, Treibitz, T., Avidan, S.: Air-light estimation using haze-lines. In: IEEE International Conference on Computational Photography (ICCP), pp. 1–9 (2017)

  45. Wang, Y., Liu, H., Chau, L.P.: Single underwater image restoration using adaptive attenuation-curve prior. IEEE Trans. Circ. Syst. I Regul. Pap. 65(3), 992–1002 (2018)

    Article  Google Scholar 

  46. He, Z., Patel, V. M.: Densely connected pyramid dehazing network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.3194–3203 (2018)

  47. Wang, Y., Chau, L. P., Ma, X.: Airtight estimation based on distant region segmentation. In: IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5 (2019)

  48. Zhou, W., Simoncelli, E. P., Bovik, A. C.: Multiscale structural similarity for image quality assessment. In: The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, pp. 1398–1402 (2003)

  49. Li, C., Anwar, S.: Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recogn. 98(1), 107038 (2019)

    Google Scholar 

  50. Ran, J., Ling, G., Geng, W., Ren, T., Wu, G.: Depth saliency based on anisotropic center-surround difference. In: IEEE International Conference on Image Processing (ICIP), pp. 1115–1119 (2014)

  51. Liu, N., Zhang, N., Han, J.: Learning selective self-mutual attention for RGB-D saliency detection. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13753–13762 (2020)

  52. Kenk, M. A., Hassaballah, M., Hameed, M. A., Bekhet, S.: Visibility enhancer: adaptable for distorted traffic scenes by dusty weather. In: 2020 2nd Novel Intelligent and Leading Emerging Sciences Conference (NILES), pp. 213–218 (2020)

  53. Zhang, L., Zhang, L., Mou, X., Zhang, D.: FSIM: a feature similarity index for image quality assessment. IEEE Trans. Image Process. 20(8), 2378–2386 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  54. Mittal, A., Fellow, I.E.E.E., Soundararajan, R., Bovik, A.C.: Making a ‘completely blind’ image quality analyzer. IEEE Signal Process. Lett. 20(3), 209–212 (2012)

    Article  Google Scholar 

  55. Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  56. The source code of the project, Available: https://github.com/xuyu44f

  57. Hautiere, N., Tarel, J.P., Aubert, D., Dumont, E.: Blind contrast enhancement assessment by gradient ratioing at visible edges. Image Anal. Stereol. 27(2), 87–95 (2011)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Tae et al. [31], J Ran et al. [50], Liu et al. [51], and Mourad A. Kenk et al. [52] for their supplied datasets. Meanwhile, we also would like to thank Li et al. [11] for their guidance and answers that greatly improve this work.

Funding

This research was supported by higher education scientific research project of Ningxia (Grant NGY2017009).

Author information

Authors and Affiliations

  1. School of Physics and Electronic-Electrical Engineering, Ningxia University, Yinchuan, China

    Pengwei Liang, Pengwei Dong, Fan Wang, Peng Ma, Jiajing Bai & Bo Wang

  2. School of Computer Science and Engineering, Nanyang Technological University, Singapore, Singapore

    Chongyi Li

Authors
  1. Pengwei Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pengwei Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peng Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiajing Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Chongyi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, P., Dong, P., Wang, F. et al. Learning to remove sandstorm for image enhancement. Vis Comput 39, 1829–1852 (2023). https://doi.org/10.1007/s00371-022-02448-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02448-8

Keywords

Search

Navigation