Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Underwater image enhancement method based on a cross attention mechanism

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Underwater image enhancement is a technique that improves the quality of underwater images, which makes them clearer and more realistic. However, because of the complexity of underwater environments, underwater image enhancement faces many challenges, such as the variation in underwater optical properties as well as low contrast, low brightness, and color distortion in underwater images. To extract underwater image features more effectively, this paper proposes an underwater image enhancement algorithm called cross attention-based underwater image enhancement (CAUIE). The algorithm combines cross large kernel attention and dynamic enhancement modules to build a U-Net model. Cross larger attention uses large kernel attention mechanism to capture the local and global information of underwater images alternately, thus enhancing the semantic representation of the images. The dynamic enhancement module, by contrast, dynamically adjusts the enhancement parameters according to different regions of the image to acquire detail information. In addition, this paper introduces a contrast regularization loss to construct a hybrid loss function for guiding the training and optimization of the model. The experimental results show that the proposed algorithm outperforms the comparison algorithm in both subjective visual and objective evaluation criteria. Moreover, the proposed model obtains PSNR and SSIM results of 34.86 dB and 0.996, respectively, increasing the results of the previous model by 7.97 dB and 0.099, which illustrates that the proposed algorithm can solve the color distortion problem and recover the contrast and clarity of underwater images.And CAUIE achieved good results in two no-reference underwater evaluation metrics UIQM and UCIQE.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availibility

The data that support the findings of this study are available from the corresponding author, Jinhua Wang, upon reasonable request.

References

  1. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27 (2014)

  2. Lan, Z., Zhou, B., Zhao, W., Wang, S.: An optimized gan method based on the que-attn and contrastive learning for underwater image enhancement. Plos One 18(1), 0279945 (2023)

    Article  Google Scholar 

  3. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv:2010.11929 (2020)

  4. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5–9, 2015, Proceedings, Part III 18, pp. 234–241. Springer (2015)

  5. Lin, H., Cheng, X., Wu, X., Shen, D.: Cat: cross attention in vision transformer. In: 2022 IEEE International Conference on Multimedia and Expo (ICME). IEEE, pp. 1–6 (2022)

  6. Guo, M.-H., Lu, C.-Z., Liu, Z.-N., Cheng, M.-M., Hu, S.-M.: Visual attention network. Comput. Vis. Media 1–20 (2023)

  7. Wu, H., Qu, Y., Lin, S., Zhou, J., Qiao, R., Zhang, Z., Xie, Y., Ma, L.: Contrastive learning for compact single image dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10551–10560 (2021)

  8. Han, J., Shoeiby, M., Malthus, T., Botha, E., Anstee, J., Anwar, S., Wei, R., Armin, M.A., Li, H., Petersson, L.: Underwater image restoration via contrastive learning and a real-world dataset. Remote Sens. 14(17), 4297 (2022)

    Article  ADS  Google Scholar 

  9. Li, C., Guo, C., Ren, W., Cong, R., Hou, J., Kwong, S., Tao, D.: An underwater image enhancement benchmark dataset and beyond. IEEE Trans. Image Process. 29, 4376–4389 (2019)

    Article  ADS  Google Scholar 

  10. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010)

    PubMed  Google Scholar 

  11. Drews, P., Nascimento, E., Moraes, F., Botelho, S., Campos, M.: Transmission estimation in underwater single images. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 825–830 (2013)

  12. Peng, Y.-T., Cosman, P.C.: Underwater image restoration based on image blurriness and light absorption. IEEE Trans. Image Process. 26(4), 1579–1594 (2017)

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  13. Akkaynak, D., Treibitz, T.: A revised underwater image formation model. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6723–6732 (2018)

  14. Akkaynak, D., Treibitz, T.: Sea-thru: a method for removing water from underwater images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1682–1691 (2019)

  15. Cao, K., Peng, Y.-T., Cosman, P.C.: Underwater image restoration using deep networks to estimate background light and scene depth. In: 2018 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI). IEEE, pp. 1–4 (2018)

  16. Zhang, W., Zhuang, P., Sun, H.-H., Li, G., Kwong, S., Li, C.: Underwater image enhancement via minimal color loss and locally adaptive contrast enhancement. IEEE Trans. Image Process. 31, 3997–4010 (2022)

    Article  ADS  Google Scholar 

  17. Zhuang, P., Wu, J., Porikli, F., Li, C.: Underwater image enhancement with hyper-Laplacian reflectance priors. IEEE Trans. Image Process. 31, 5442–5455 (2022)

    Article  ADS  PubMed  Google Scholar 

  18. Zhou, J., Liu, Q., Jiang, Q., Ren, W., Lam, K.-M., Zhang, W.: Underwater camera: Improving visual perception via adaptive dark pixel prior and color correction. Int. J. Comput. Vis. 1–19 (2023)

  19. Zhou, J., Wang, Y., Li, C., Zhang, W.: Multicolor light attenuation modeling for underwater image restoration. IEEE J. Ocean. Eng. (2023)

  20. Zhou, J., Pang, L., Zhang, D., Zhang, W.: Underwater image enhancement method via multi-interval subhistogram perspective equalization. IEEE J. Ocean. Eng. (2023)

  21. Han, J., Shoeiby, M., Malthus, T., Botha, E., Anstee, J., Anwar, S., Wei, R., Petersson, L., Armin, M.A.: Single underwater image restoration by contrastive learning. In: 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS. IEEE, pp. 2385–2388 (2021)

  22. Li, J., Skinner, K.A., Eustice, R.M., Johnson-Roberson, M.: Watergan: unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robot. Autom. Lett. 3(1), 387–394 (2017)

    Google Scholar 

  23. Li, C., Anwar, S., Porikli, F.: Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recognit. 98, 107038 (2020)

    Article  Google Scholar 

  24. Li, C., Anwar, S., Hou, J., Cong, R., Guo, C., Ren, W.: Underwater image enhancement via medium transmission-guided multi-color space embedding. IEEE Trans. Image Process. 30, 4985–5000 (2021)

    Article  ADS  PubMed  Google Scholar 

  25. Fu, Z., Wang, W., Huang, Y., Ding, X., Ma, K.-K.: Uncertainty inspired underwater image enhancement. In: European Conference on Computer Vision, pp. 465–482. Springer (2022)

  26. Kar, A., Dhara, S.K., Sen, D., Biswas, P.K.: Zero-shot single image restoration through controlled perturbation of Koschmieder’s model. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16205–16215 (2021)

  27. Zhang, D., Zhou, J., Zhang, W., Lin, Z., Yao, J., Polat, K., Alenezi, F., Alhudhaif, A.: Rex-net: a reflectance-guided underwater image enhancement network for extreme scenarios. Expert Syst. Appl. 120842 (2023)

  28. Kang, Y., Jiang, Q., Li, C., Ren, W., Liu, H., Wang, P.: A perception-aware decomposition and fusion framework for underwater image enhancement. IEEE Trans. Circuits Syst. Video Technol. 33(3), 988–1002 (2022)

    Article  Google Scholar 

  29. Zhou, J., Li, B., Zhang, D., Yuan, J., Zhang, W., Cai, Z., Shi, J.: Ugif-net: an efficient fully guided information flow network for underwater image enhancement. IEEE Trans. Geosci. Remote Sens. (2023)

  30. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)

  31. Song, Y., He, Z., Qian, H., Du, X.: Vision transformers for single image dehazing. IEEE Trans. Image Process. 32, 1927–1941 (2023)

    Article  ADS  Google Scholar 

  32. Wang, W., Xie, E., Li, X., Fan, D.-P., Song, K., Liang, D., Lu, T., Luo, P., Shao, L.: Pyramid vision transformer: a versatile backbone for dense prediction without convolutions. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 568–578 (2021)

  33. Mehta, S., Rastegari, M.: Mobilevit: light-weight, general-purpose, and mobile-friendly vision transformer. arXiv:2110.02178 (2021)

  34. Yu, W., Luo, M., Zhou, P., Si, C., Zhou, Y., Wang, X., Feng, J., Yan, S.: Metaformer is actually what you need for vision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10819–10829 (2022)

  35. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017)

  36. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861 (2017)

  37. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXiv:1511.07122 (2015)

  38. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014)

  39. Zhao, H., Gallo, O., Frosio, I., Kautz, J.: Loss functions for image restoration with neural networks. IEEE Trans. Comput. Imaging 3(1), 47–57 (2016)

    Article  Google Scholar 

  40. Fu, Z., Lin, H., Yang, Y., Chai, S., Sun, L., Huang, Y., Ding, X.: Unsupervised underwater image restoration: from a homology perspective. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 643–651 (2022)

  41. Panetta, K., Gao, C., Agaian, S.: Human-visual-system-inspired underwater image quality measures. IEEE J. Ocean. Eng. 41(3), 541–551 (2015)

    Article  Google Scholar 

  42. Yang, M., Sowmya, A.: An underwater color image quality evaluation metric. IEEE Trans. Image Process. 24(12), 6062–6071 (2015)

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  43. Berman, D., Levy, D., Avidan, S., Treibitz, T.: Underwater single image color restoration using haze-lines and a new quantitative dataset. IEEE Trans. Pattern Anal. Mach. Intell. 43(8), 2822–2837 (2020)

    Google Scholar 

  44. Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

  45. Park, T., Efros, A.A., Zhang, R., Zhu, J.-Y.: Contrastive learning for unpaired image-to-image translation. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IX 16, pp. 319–345. Springer (2020)

  46. Zhang, S., Wang, T., Dong, J., Yu, H.: Underwater image enhancement via extended multi-scale retinex. Neurocomputing 245, 1–9 (2017)

    Article  Google Scholar 

  47. Ancuti, C., Ancuti, C.O., Haber, T., Bekaert, P.: Enhancing underwater images and videos by fusion. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp. 81–88 (2012)

  48. Li, K., Wu, L., Qi, Q., Liu, W., Gao, X., Zhou, L., Song, D.: Beyond single reference for training: underwater image enhancement via comparative learning. IEEE Trans. Circuits Syst. Video Technol. 33(6), 2561–2576 (2023). https://doi.org/10.1109/TCSVT.2022.3225376

    Article  Google Scholar 

  49. Qi, Q., Li, K., Zheng, H., Gao, X., Hou, G., Sun, K.: Sguie-net: semantic attention guided underwater image enhancement with multi-scale perception. IEEE Trans. Image Process. 31, 6816–6830 (2022)

    Article  ADS  Google Scholar 

  50. Guo, C., Wu, R., Jin, X., Han, L., Zhang, W., Chai, Z., Li, C.: Underwater ranker: learn which is better and how to be better. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, pp. 702–709 (2023)

  51. Zhang, W., Zhou, L., Zhuang, P., Li, G., Pan, X., Zhao, W., Li, C.: Underwater image enhancement via weighted wavelet visual perception fusion. IEEE Trans. Circuits Syst. Video Technol. (2023)

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (No. 62172045 and No. 62272049), the Academic Research Projects of Beijing Union University (No. ZKZD202301).

Author information

Authors and Affiliations

  1. Smart City College, Beijing Union University, Chaoyang, Beijing, 100101, China

    Sunhan Xu, Jinhua Wang, Ning He, Xin Hu & Fengxi Sun

Authors
  1. Sunhan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinhua Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ning He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xin Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fengxi Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SX contributed to conceptualization, methodology, investigation, formal analysis, and writing of the original draft. JW was responsible for project administration, supervision, and funding acquisition and also contributed to writing the review and editing. NH assisted with software and wrote the review and editing. XH provided software support, conducted formal analysis, and contributed to visualization. FS conducted ablation experiments and contributed to validation.

Corresponding author

Correspondence to Jinhua Wang.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Communicated by B. Bao.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, S., Wang, J., He, N. et al. Underwater image enhancement method based on a cross attention mechanism. Multimedia Systems 30, 26 (2024). https://doi.org/10.1007/s00530-023-01224-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00530-023-01224-5

Keywords

Search

Navigation