Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Real-time underwater image resolution enhancement using super-resolution with deep convolutional neural networks

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

In this paper, a two-step image enhancement is presented. In the first step, color correction and underwater image quality enhancement are conducted if there are artifacts such as darkening, hazing and fogging. In the second step, the image resolution optimized in the previous step is enhanced using the convolutional neural network (CNN) with deep learning capability. The main reason behind the adoption of this two-step technique, which includes image quality enhancement and super-resolution, is the need for a robust strategy to visually improve underwater images at different depths and under diverse artifact conditions. The effectiveness and robustness of the real-time algorithm are satisfactory for various underwater images under different conditions, and several experiments have been undertaken for the two datasets of images. In both stages and for each of image datasets, the mean square error (MSE), peak signal to noise ratio (PSNR), and structural similarity (SSIM) evaluation measures were fulfilled. In addition, the low computational complexity and suitable outputs were obtained for different artifacts that represented divergent depths of water to achieve a real-time system. The super-resolution in the proposed structure for medium layers can offer a proper response. For this reason, time is also one of the major factors reported in the research. Applying this model to underwater imagery systems will yield more accurate and detailed information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Tang, C., von Lukas, U.F., Vahl, M., Wang, S., Wang, Y., Tan, M.: Efficient underwater image and video enhancement based on Retinex. SIViP 13(5), 1011–1018 (2019)

    Article  Google Scholar 

  2. Kanaev, A.V., Smith, L.N., Hou, W.W., Woods, S.: Restoration of turbulence degraded underwater images. Opt. Eng. 51(5), 057007 (2012)

    Article  Google Scholar 

  3. Yu, J., Wang, Y., Zhou, S., Zhai, R., Huang, S.: Unmanned aerial vehicle (UAV) image haze removal using dark channel prior. J. Phys. Conf. Ser. 1324(1), 012036 (2019)

    Article  Google Scholar 

  4. Golts, A., Freedman, D., Elad, M.: Unsupervised single image dehazing using dark channel prior loss. IEEE Transactions on Image Processing (2019).

  5. Khosravi, M.R., Samadi, S.: Data compression in ViSAR sensor networks using non-linear adaptive weighting. EURASIP J. Wirel. Commun. Netw. 2019(1), 1–8 (2019)

    Article  Google Scholar 

  6. Cho, Y., Malav, R., Pandey, G., Kim, A.: DehazeGAN: underwater haze image restoration using unpaired image-to-image translation. IFAC-PapersOnLine. 52(21), 82–85 (2019)

    Article  MathSciNet  Google Scholar 

  7. Schettini, R., Corchs, S.: Underwater image processing: state of the art of restoration and image enhancement methods. EURASIP J. Adv. Signal Process. 2010, 1–4 (2010)

    Article  Google Scholar 

  8. Lu, H., Li, Y., Zhang, Y., Chen, M., Serikawa, S., Kim, H.: Underwater optical image processing: a comprehensive review. Mobile Netw. Appl. 22(6), 1204–1211 (2017)

    Article  Google Scholar 

  9. Sethi, R., Sreedevi, I.: Adaptive enhancement of underwater images using multi-objective PSO. Multimed. Tools Appl. 78(22), 31823–31845 (2019)

    Article  Google Scholar 

  10. Lu, H., Li, Y., Xu, X., Li, J., Liu, Z., Li, X., Yang, J., Serikawa, S.: Underwater image enhancement method using weighted guided trigonometric filtering and artificial light correction. J. Vis. Commun. Image Represent. 1(38), 504–516 (2016)

    Article  Google Scholar 

  11. Chiang, J.Y., Chen, Y.C.: Underwater image enhancement by wavelength compensation and dehazing. IEEE Trans. Image Process. 21(4), 1756–1769 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  12. Khosravi, M.R., Moghimi, M.K.: Underwater optical image processing. Modern Approaches Oceanogr Petrochem. Sci 1(1), 1–2 (2018). https://doi.org/10.32474/MAOPS.2018.01.000101

    Article  Google Scholar 

  13. Anwer, A., Ali, S.S., Khan, A., Mériaudeau, F.: Real-time underwater 3D scene reconstruction using commercial depth sensor. In 2016 IEEE International Conference on Underwater System Technology: Theory and Applications (USYS) 2016 Dec 13 (pp. 67–70). IEEE.

  14. Çelebi, A.T., Ertürk, S.: Visual enhancement of underwater images using empirical mode decomposition. Expert Syst. Appl. 39(1), 800–805 (2012)

    Article  Google Scholar 

  15. Abas, P.E., De Silva, L.C.: Review of underwater image restoration algorithms. IET Image Proc. 13(10), 1587–1596 (2019)

    Article  Google Scholar 

  16. Han, M., Lyu, Z., Qiu, T., Xu, M.: A review on intelligence dehazing and color restoration for underwater images. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2018 Jan 23.

  17. Shortis, M., Abdo, E.H.: A review of underwater stereo-image measurement for marine biology and ecology applications. In Oceanography and Marine Biology 2016 Apr 19 (pp. 269–304). CRC Press.

  18. Li, C., Guo, J., Guo, C.: Emerging from water: underwater image color correction based on weakly supervised color transfer. IEEE Signal Process. Lett. 25(3), 323–327 (2018)

    Article  Google Scholar 

  19. Moghimi, M.K., Mohanna, F.: A joint adaptive evolutionary model towards optical image contrast enhancement and geometrical reconstruction approach in underwater remote sensing. SN Appl. Sci. 1(10), 1242 (2019)

    Article  Google Scholar 

  20. Ji, T., Wang, G.: An approach to underwater image enhancement based on image structural decomposition. J. Ocean Univ. Chin. 14(2), 255–260 (2015)

    Article  Google Scholar 

  21. Khosravi, M.R.: The shortfalls of underwater sensor network simulators. Sea Technol. 60(5), 41–41 (2019)

    Google Scholar 

  22. Ghani, A.S., Isa, N.A.: Underwater image quality enhancement through Rayleigh-stretching and averaging image planes. Int. J. Naval Archit. Ocean Eng. 6(4), 840–866 (2014)

    Article  Google Scholar 

  23. Seemakurthy, K., Rajagopalan, A.N.: Deskewing of underwater images. IEEE Trans. Image Process. 24(3), 1046–1059 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  24. Panetta, K., Gao, C., Agaian, S.: Human-visual-system-inspired underwater image quality measures. IEEE J. Oceanic Eng. 41(3), 541–551 (2015)

    Article  Google Scholar 

  25. Galdran, A., Pardo, D., Picón, A., Alvarez-Gila, A.: Automatic red-channel underwater image restoration. J. Vis. Commun. Image Represent. 1(26), 132–145 (2015)

    Article  Google Scholar 

  26. Bianco, G., Muzzupappa, M., Bruno, F., Garcia, R., Neumann, L.: A new color correction method for underwater imaging. Int. Arch. Photogramm., Remote Sens Spatial Inf. Sci. 40(5), 25 (2015)

    Article  Google Scholar 

  27. Fu, X., Fan, Z., Ling, M., Huang, Y., Ding, X.: Two-step approach for single underwater image enhancement. In 2017 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS) 2017 Nov 6 (pp. 789–794). IEEE.

  28. Iqbal, K., Salam, R.A., Osman, A., Talib, A.Z.: Underwater Image Enhancement Using an Integrated Colour Model. IAENG International Journal of computer science. 2007 Dec 1;34(2).

  29. Liu, Y., Xu, H., Shang, D., Li, C., Quan, X.: An underwater image enhancement method for different illumination conditions based on color tone correction and fusion-based descattering. Sensors 19(24), 5567 (2019)

    Article  Google Scholar 

  30. Fiuzy, M.M., Rezaei, K.F., Haddadnia, J.M.: A novel approach for segmentation special region in an image. Majlesi J. Multimed. Process. 2011 Sep 23;1(2).

  31. Arel, I., Rose, D.C., Karnowski, T.P.: Research frontier: deep machine learning–a new frontier in artificial intelligence research. IEEE Comput. Intell. Mag. 5(4), 13–18 (2010)

    Article  Google Scholar 

  32. He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2015)

    Article  Google Scholar 

  33. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems 2012 (pp. 1097–1105).

  34. Burger, H.C., Schuler, C.J., Harmeling, S.: Image denoising: Can plain neural networks compete with BM3D?. In 2012 IEEE conference on computer vision and pattern recognition 2012 Jun 16 (pp. 2392–2399). IEEE.

  35. Schuler, C.J., Christopher, B.H., Harmeling, S., Scholkopf, B.: A machine learning approach for non-blind image deconvolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2013 (pp. 1067–1074).

  36. Cui, Z., Chang, H., Shan, S., Zhong, B., Chen, X.: Deep network cascade for image super-resolution. In European Conference on Computer Vision 2014 Sep 6 (pp. 49–64). Springer, Cham.

  37. Arun, P.V., Buddhiraju, K.M.: A deep learning based spatial dependency modelling approach towards super-resolution. In 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) 2016 Jul 10 (pp. 6533–6536). IEEE.

  38. Aymaz, S., Köse, C.: A novel image decomposition-based hybrid technique with super-resolution method for multi-focus image fusion. Information Fusion. 1(45), 113–127 (2019)

    Article  Google Scholar 

  39. Yao, T., Luo, Y., Chen, Y., Yang, D., Zhao, L.: Single-image super-resolution: a survey. In International Conference in Communications, Signal Processing, and Systems 2018 Jul 14 (pp. 119–125). Springer, Singapore.

  40. Khosravi, M.R., Basri, H., Rostami, H., Samadi, S.: distributed random cooperation for vbf-based routing in high-speed dense underwater acoustic sensor networks. J. Supercomput. 74(11), 6184–6200 (2018)

    Article  Google Scholar 

  41. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In Proceedings of the European Conference on Computer Vision (ECCV) 2018 (pp. 286–301).

  42. Kim, J., Kwon, L.J., Mu, L.K.: Deeply-recursive convolutional network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition 2016 (pp. 1637–1645).

  43. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2015)

    Article  Google Scholar 

  44. Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In European conference on computer vision 2016 Oct 8 (pp. 391–407). Springer, Cham.

  45. Timofte, R., De Smet, V., Van Gool, L.: A+: Adjusted anchored neighborhood regression for fast super-resolution. In Asian conference on computer vision 2014 Nov 1 (pp. 111–126). Springer, Cham.

  46. Schulter, S., Leistner, C., Bischof, H.:Fast and accurate image upscaling with super-resolution forests. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2015 (pp. 3791–3799).

  47. Prabhakar, C.J., Kumar, P.U.: An image based technique for enhancement of underwater images. arXiv preprint, arXiv:1212.0291. 2012 Dec 3.

  48. Ramadass, G.A., Ramesh, S., Selvakumar, J.M., Ramesh, R., Subramanian, A.N., Sathianarayanan, D., Harikrishnan, G., Muthukumaran, D., Jayakumar, V.K., Chandrasekaran, E., Murugesh, M.: Deep-ocean exploration using remotely operated vehicle at gas hydrate site in Krishna-Godavari basin, Bay of Bengal. Curr. Sci. 99(6), 809–815 (2010)

    Google Scholar 

  49. Srividhya, K., Ramya, M.M.: Accurate object recognition in the underwater images using learning algorithms and texture features. Multimed. Tools Appl. 76(24), 25679–25695 (2017)

    Article  Google Scholar 

  50. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010)

    Google Scholar 

  51. Mobley, C.D.: Light and water: radiative transfer in natural waters. Academic press, Cambridge (1994)

    Google Scholar 

  52. Drews, P., Nascimento, E., Moraes, F., Botelho, S., Campos, M.: Transmission estimation in underwater single images. In Proceedings of the IEEE international conference on computer vision workshops 2013 (pp. 825–830).

  53. Tarel, J.P., Hautiere, N.: Fast visibility restoration from a single color or gray level image. In 2009 IEEE 12th International Conference on Computer Vision 2009 Sep 29 (pp. 2201–2208). IEEE.

  54. Ancuti, C., Ancuti, C.O., Haber, T., Bekaert, P.: Enhancing underwater images and videos by fusion. In 2012 IEEE Conference on Computer Vision and Pattern Recognition 2012 Jun 16 (pp. 81–88). IEEE.

  55. Barbosa, W.V., Amaral, H.G., Rocha, T.L., Nascimento, E.R.: Visual-quality-driven learning for underwater vision enhancement. In 2018 25th IEEE International Conference on Image Processing (ICIP) 2018 Oct 7 (pp. 3933–3937). IEEE.

  56. Hitam, M.S., Awalludin, E.A., Yussof, W.N., Bachok, Z.: Mixture contrast limited adaptive histogram equalization for underwater image enhancement. In2013 International conference on computer applications technology (ICCAT) 2013 Jan 20 (pp. 1–5). IEEE.

  57. Khosravi, M.R., Basri, H., Rostami, H.: Efficient routing for dense UWSNs with high-speed mobile nodes using spherical divisions. J. Supercomput. 74(2), 696–716 (2018)

    Article  Google Scholar 

  58. Zhang, S., Wang, T., Dong, J., Yu, H.: Underwater image enhancement via extended multi-scale Retinex. Neurocomputing. 5(245), 1–9 (2017)

    Google Scholar 

  59. Mercado, M.A., Ishii, K., Ahn, J.: Deep-sea image enhancement using multi-scale retinex with reverse color loss for autonomous underwater vehicles. InOCEANS 2017-Anchorage 2017 Sep 18 (pp. 1–6). IEEE.

  60. Torres-Méndez, L.A., Dudek, G.: Color correction of underwater images for aquatic robot inspection. InInternational Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition 2005 Nov 9 (pp. 60–73). Springer, Berlin, Heidelberg.

  61. Petit, F., Capelle-Laizé, A.S., Carré, P.: Underwater image enhancement by attenuation inversionwith quaternions. In 2009 IEEE International Conference on Acoustics, Speech and Signal Processing 2009 Apr 19 (pp. 1177–1180). IEEE.

  62. Rizzi, A., Gatta, C., Marini, D.: A new algorithm for unsupervised global and local color correction. Pattern Recogn. Lett. 24(11), 1663–1677 (2003)

    Article  Google Scholar 

  63. Lu, H., Li, Y., Serikawa, S.: Single underwater image descattering and color correction. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2015 Apr 19 (pp. 1623–1627). IEEE.

  64. Li, Y., Lu, H., Li, J., Li, X., Li, Y., Serikawa, S.: Underwater image de-scattering and classification by deep neural network. Comput. Electr. Eng. 1(54), 68–77 (2016)

    Article  Google Scholar 

  65. Hollinger, G.A., Mitra, U., Sukhatme, G.S.: Active classification: Theory and application to underwater inspection. In Robotics Research 2017 (pp. 95–110). Springer, Cham.

  66. Mahmood, A., Bennamoun, M., An, S., Sohel, F., Boussaid, F., Hovey, R., Kendrick, G., Fisher, R.B.: Deep learning for coral classification. InHandbook of Neural Computation 2017 Jan 1 (pp. 383–401). Academic Press, Cambridge

  67. Kim, J.H., Dowling, D.R.: Blind deconvolution of extended duration underwater signals. J. Acoust. Soc. Am. 135(4), 2200 (2014)

    Article  Google Scholar 

  68. Lu, H., Li, Y., Hu, X., Yang, S., Serikawa, S.: Real-time underwater image contrast enhancement through guided filtering. In International Conference on Image and Graphics 2015 Aug 13 (pp. 137–147). Springer, Cham.

  69. Li, J., Skinner, K.A., Eustice, R.M., Johnson-Roberson, M.: WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robot. Autom. Lett. 3(1), 387–394 (2017)

    Google Scholar 

  70. Reza, A.M.: Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement. J. VLSI Signal Process. Syst. Signal Image Video Technol. 38(1), 35–44 (2004)

    Article  Google Scholar 

  71. Yang, J., Jiang, B., Lv, Z., Jiang, N.: A real-time image dehazing method considering dark channel and statistics features. J. Real-Time Image Proc. 13(3), 479–490 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Communication Engineering, University of Sistan and Baluchestan, Zahedan, Iran

    Mohammad Kazem Moghimi & Farahnaz Mohanna

Authors
  1. Mohammad Kazem Moghimi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Farahnaz Mohanna
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Farahnaz Mohanna.

Ethics declarations

Conflict of interest

We have no conflict of interest to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moghimi, M.K., Mohanna, F. Real-time underwater image resolution enhancement using super-resolution with deep convolutional neural networks. J Real-Time Image Proc 18, 1653–1667 (2021). https://doi.org/10.1007/s11554-020-01024-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-01024-4

Keywords

Search

Navigation