research-article

Public Access

Single Image Dehazing via Multi-scale Convolutional Neural Networks with Holistic Edges

Authors:

Ming-Hsuan YangAuthors Info & Claims

International Journal of Computer Vision, Volume 128, Issue 1

Pages 240 - 259

https://doi.org/10.1007/s11263-019-01235-8

Published: 01 January 2020 Publication History

Abstract

Single image dehazing has been a challenging problem which aims to recover clear images from hazy ones. The performance of existing image dehazing methods is limited by hand-designed features and priors. In this paper, we propose a multi-scale deep neural network for single image dehazing by learning the mapping between hazy images and their transmission maps. The proposed algorithm consists of a coarse-scale net which predicts a holistic transmission map based on the entire image, and a fine-scale net which refines dehazed results locally. To train the multi-scale deep network, we synthesize a dataset comprised of hazy images and corresponding transmission maps based on the NYU Depth dataset. In addition, we propose a holistic edge guided network to refine edges of the estimated transmission map. Extensive experiments demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods on both synthetic and real-world images in terms of quality and speed.

References

[1]

Ancuti CO and Ancuti C Single image dehazing by multi-scale fusion IEEE Transactions on Image Processing 2013 22 8 3271-3282

[2]

Ancuti, C. O., Ancuti, C., & Bekaert, P. (2010). Effective single image dehazing by fusion. In IEEE international conference on image processing, (pp. 3541–3544).

[3]

Berman, D., Treibitz, T., & Avidan, S. (2017). Air-light estimation using haze-lines. In IEEE international conference on computational photography, (pp. 1–9).

[4]

Berman, D., Treibitz, T., & Shai, A. (2016). Non-local image dehazing. In IEEE conference on computer vision and pattern recognition, (pp. 1674–1682).

[5]

Breiman L Random forests Machine Learning 2001 45 1 5-32

[6]

Cai B, Xu X, Jia K, Qing C, and Tao DDehazenet: An end-to-end system for single image haze removalIEEE Transactions on Image Processing201625115187-51983549336

[7]

Cai, B., Xu, X., & Tao, D. (2016). Real-time video dehazing based on spatio-temporal mrf. In Pacific Rim conference on multimedia, (pp. 315–325).

[8]

Canny J A computational approach to edge detection IEEE Transactions on Pattern Analysis and Machine Intelligence 1986 8 6 679-98

[9]

Caraffa, L., & Tarel, J. P. (2012). Stereo reconstruction and contrast restoration in daytime fog. In Asian conference on computer vision, (pp. 13–25).

[10]

Caraffa, L., & Tarel, J. P. (2013). Markov random field model for single image defogging. In IEEE intelligent vehicles symposium, (pp. 994–999).

[11]

Dong, C., Loy, C. C., He, K., & Tang, X. (2014). Learning a deep convolutional network for image super-resolution. In European conference on computer vision, (pp. 184–199).

[12]

Eigen, D., Puhrsch, C., & Fergus, R. (2014). Depth map prediction from a single image using a multi-scale deep network. In Advances in neural information processing systems, (pp. 2366–2374).

[13]

Fattal R Single image dehazing ACM Transactions on Graphics 2008 27 3 72

[14]

Fattal R Dehazing using color-lines ACM Transactions on Graphics 2014 34 1 13

[15]

Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The kitti vision benchmark suite. In IEEE conference on computer vision and pattern recognition, (pp. 3354–3361).

[16]

Gibson, K. B., & Nguyen, T. Q. (2013). Fast single image fog removal using the adaptive wiener filter. In IEEE international conference on image processing, (pp. 714–718).

[17]

Gibson KB, Vo DT, and Nguyen TQAn investigation of dehazing effects on image and video codingIEEE Transactions on Image Processing2012212662-6732920648

[18]

He, K., Sun, J., & Tang, X. (2009). Single image haze removal using dark channel prior. In IEEE conference on computer vision and pattern recognition, (pp. 1956–1963).

[19]

He K, Sun J, and Tang X Single image haze removal using dark channel prior IEEE Transactions on Pattern Analysis and Machine Intelligence 2011 33 12 2341-2353

[20]

He K, Sun J, and Tang X Guided image filtering IEEE Transactions on Pattern Analysis and Machine Intelligence 2013 35 6 1397-1409

[21]

Kopf J, Neubert B, Chen B, Cohen M, Cohen-Or D, Deussen O, Uyttendaele M, and Lischinski D Deep photo: Model-based photograph enhancement and viewing ACM Transactions on Graphics 2008 27 5 116

[22]

Koschmieder, H. (1924). Theorie der horizontalen sichtweite. Beitrage zur Physik der freien Atmosphare (pp. 33–53).

[23]

Kratz, L., & Nishino, K. (2009). Factorizing scene albedo and depth from a single foggy image. In IEEE international conference on computer vision, (pp. 1701–1708).

[24]

Li, B., Peng, X., Wang, Z., Xu, J., & Feng, D. (2017). Aod-net: All-in-one dehazing network. In IEEE international conference on computer vision.

[25]

Li B, Ren W, Fu D, Tao D, Feng D, Zeng W, and Wang ZBenchmarking single-image dehazing and beyondIEEE Transactions on Image Processing2018281492-5053863197

[26]

Li, S., Araujo, I. B., Ren, W., Wang, Z., Tokuda, E. K., Junior, R. H., Cesar-Junior, R., Zhang, J., Guo, X., & Cao, X. (2019). Single image deraining: A comprehensive benchmark analysis. In IEEE conference on computer vision and pattern recognition, (pp. 3838–3847).

[27]

Li, Y., Tan, R. T., & Brown, M. S. (2015). Nighttime haze removal with glow and multiple light colors. In IEEE international conference on computer vision, (pp. 226–234).

[28]

Li, Z., Tan, P., Tan, R. T., Zou, D., Zhou, S. Z., & Cheong, L. F. (2015). Simultaneous video defogging and stereo reconstruction. In IEEE conference on computer vision and pattern recognition, (pp. 4988–4997).

[29]

Liu, Y., Zhao, G., Gong, B., Li, Y., Raj, R., Goel, N., Kesav, S., Gottimukkala, S., Wang, Z., Ren, W., et al. (2018). Improved techniques for learning to dehaze and beyond: A collective study. arXiv preprintarXiv:1807.00202

[30]

Meng, G., Wang, Y., Duan, J., Xiang, S., & Pan, C. (2013). Efficient image dehazing with boundary constraint and contextual regularization. In IEEE international conference on computer vision, (pp. 617–624).

[31]

Nair, V., & Hinton, G. E. (2010). Rectified linear units improve restricted Boltzmann machines. In International conference on machine learning, (pp. 807–814).

[32]

Narasimhan, S. G., & Nayar, S. K. (2000). Chromatic framework for vision in bad weather. In IEEE conference on computer vision and pattern recognition, (pp. 598–605).

[33]

Narasimhan SG and Nayar SK Contrast restoration of weather degraded images IEEE Transactions on Pattern Analysis and Machine Intelligence 2003 25 6 713-724

[34]

Nayar, S. K., & Narasimhan, S. G. (1999). Vision in bad weather. In IEEE international conference on computer vision, (pp. 820–827).

[35]

Nishino K, Kratz L, and Lombardi SBayesian defoggingInternational Journal of Computer Vision2012983263-2782917054

[36]

Ren, W., & Cao, X. (2017). Deep video dehazing. In Pacific rim conference on multimedia, (pp. 14–24).

[37]

Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., & Yang, M. H. (2016). Single image dehazing via multi-scale convolutional neural networks. In European conference on computer vision, (pp. 154–169).

[38]

Ren, W., Ma, L., Zhang, J., Pan, J., Cao, X., Liu, W., & Yang, M. H. (2018). Gated fusion network for single image dehazing. In IEEE conference on computer vision and pattern recognition, (pp. 3253–3261).

[39]

Ren W, Zhang J, Xu X, Ma L, Cao X, Meng G, and Liu WDeep video dehazing with semantic segmentationIEEE Transactions on Image Processing20182841895-19083893160

[40]

Saxena A, Sun M, and Ng AY Make3d: Learning 3d scene structure from a single still image IEEE Transactions on Pattern Analysis and Machine Intelligence 2009 31 5 824-840

[41]

Scharstein D and Szeliski R A taxonomy and evaluation of dense two-frame stereo correspondence algorithms International Journal of Computer Vision 2002 47 1–3 7-42

[42]

Scharstein, D., & Szeliski, R. (2003). High-accuracy stereo depth maps using structured light. In IEEE conference on computer vision and pattern recognition, (Vol. 1, pp. I–195).

[43]

Schaul, L., Fredembach, C., & Süsstrunk, S. (2009). Color image dehazing using the near-infrared. In IEEE international conference on image processing, (pp. 1629–1632).

[44]

Schechner, Y. Y., Narasimhan, S. G., & Nayar, S. K. (2001). Instant dehazing of images using polarization. In IEEE conference on computer vision and pattern recognition, (Vol. 1, pp. I–325).

[45]

Shwartz, S., Namer, E., & Schechner, Y. Y. (2006). Blind haze separation. In IEEE conference on computer vision and pattern recognition, (pp. 1984–1991).

[46]

Silberman, N., Hoiem, D., Kohli, P., & Fergus, R. (2012). Indoor segmentation and support inference from RGBD images. In European conference on computer vision, (pp. 746–760).

[47]

Sulami, M., Glatzer, I., Fattal, R., & Werman, M. (2014). Automatic recovery of the atmospheric light in hazy images. In IEEE international conference on computational photography.

[48]

Tan, R. T. (2008). Visibility in bad weather from a single image. In IEEE conference on computer vision and pattern recognition, (pp. 1–8).

[49]

Tan, R. T., Pettersson, N., & Petersson, L. (2007). Visibility enhancement for roads with foggy or hazy scenes. In IEEE intelligent vehicles symposium, (pp. 19–24).

[50]

Tang, K., Yang, J., & Wang, J. (2014). Investigating haze-relevant features in a learning framework for image dehazing. In IEEE conference on computer vision and pattern recognition, (pp. 2995–3002).

[51]

Tarel, J. P., & Hautiere, N. (2009). Fast visibility restoration from a single color or gray level image. In IEEE international conference on computer vision, (pp. 2201–2208).

[52]

Tarel JP, Hautière N, Caraffa L, Cord A, Halmaoui H, and Gruyer D Vision enhancement in homogeneous and heterogeneous fog Intelligent Transportation Systems Magazine 2012 4 2 6-20

[53]

Treibitz, T., & Schechner, Y. Y. (2009). Polarization: Beneficial for visibility enhancement? In IEEE conference on computer vision and pattern recognition, (pp. 525–532).

[54]

Wang YK and Fan CTSingle image defogging by multiscale depth fusionIEEE Transactions on Image Processing201423114826-48373265422

[55]

Xie, S., & Tu, Z. (2015). Holistically-nested edge detection. In IEEE international conference on computer vision, (pp. 1395–1403).

[56]

Zhang, H., & Patel, V. M. (2018). Densely connected pyramid dehazing network. In IEEE conference on computer vision and pattern recognition.

[57]

Zhang, H., Sindagi, V., & Patel, V. M. (2018). Multi-scale single image dehazing using perceptual pyramid deep network. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, (pp. 902–911).

[58]

Zhang, J., Cao, Y., & Wang, Z. (2014). Nighttime haze removal based on a new imaging model. In IEEE international conference on image processing, (pp. 4557–4561).

[59]

Zhang S, He F, Ren W, and Yao J Joint learning of image detail and transmission map for single image dehazing The Visual Computer 2018 34 1-12

[60]

Zhang, S., Ren, W., & Yao, J. (2018). Feed-net: Fully end-to-end dehazing. In IEEE international conference on multimedia and expo, (pp. 1–6).

[61]

Zhang, X. S., Gao, S. B., Li, C. Y., & Li, Y. J. (2015). A retina inspired model for enhancing visibility of hazy images. Frontiers in Computational Neuroscience, 9, 1–13.

[62]

Zhu, Q., Mai, J., & Shao, L. (2014). Single image dehazing using color attenuation prior. In British machine vision conference

[63]

Zhu Q, Mai J, and Shao LA fast single image haze removal algorithm using color attenuation priorIEEE Transactions on Image Processing201524113522-35333371815

Cited By

Dhara GKumar R(2024)A novel multiscale cGAN approach for enhanced salient object detection in single haze imagesJournal on Image and Video Processing10.1186/s13640-024-00648-x2024:1Online publication date: 15-Sep-2024
https://dl.acm.org/doi/10.1186/s13640-024-00648-x
Zhang HXiao LCao XForoosh H(2024)Multiple Adverse Weather Conditions Adaptation for Object Detection via Causal InterventionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.316676546:3(1742-1756)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1109/TPAMI.2022.3166765
Bernabel SAgaian S(2024)NDELS: A Novel Approach for Nighttime Dehazing, Low-Light Enhancement, and Light SuppressionIEEE Transactions on Multimedia10.1109/TMM.2024.338842026(9292-9303)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3388420
Show More Cited By

Index Terms

Single Image Dehazing via Multi-scale Convolutional Neural Networks with Holistic Edges
1. Computing methodologies

Index terms have been assigned to the content through auto-classification.

Recommendations

Single Image Dehazing using CNN
Abstract
Haze is a natural phenomenon in which the dust, smoke and other particles alter the vision of the sky to reduce the visibility. Hazy images cause various visibility problems for traffic user, tourists everywhere, especially in hilly areas where ...
Single Image Dehazing via Regional Saturation-Value Translation
Abstract
This paper proposes an image dehazing prior, called Regional Saturation-Value Translation (RSVT), in order to address the color distortion issues produced by prevailing prior-based dehazing methods when processing hazy images with large sky ...
Image dehazing using two‐dimensional canonical correlation analysis

Image dehazing is an important issue that interests both image processing and computer vision. In this study, image dehazing is modelled as an example‐based learning problem, and a novel dehazing algorithm using two‐dimensional (2D) canonical correlation ...

Comments

Information & Contributors

Information

Published In

cover image International Journal of Computer Vision

International Journal of Computer Vision Volume 128, Issue 1

Jan 2020

259 pages

ISSN:0920-5691

Issue’s Table of Contents

© Springer Science+Business Media, LLC, part of Springer Nature 2019.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 January 2020

Accepted: 12 September 2019

Received: 24 December 2017

Author Tags

Qualifiers

Research-article

Funding Sources

Directorate for Computer and Information Science and Engineering

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

75
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dhara GKumar R(2024)A novel multiscale cGAN approach for enhanced salient object detection in single haze imagesJournal on Image and Video Processing10.1186/s13640-024-00648-x2024:1Online publication date: 15-Sep-2024
https://dl.acm.org/doi/10.1186/s13640-024-00648-x
Zhang HXiao LCao XForoosh H(2024)Multiple Adverse Weather Conditions Adaptation for Object Detection via Causal InterventionIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.316676546:3(1742-1756)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1109/TPAMI.2022.3166765
Bernabel SAgaian S(2024)NDELS: A Novel Approach for Nighttime Dehazing, Low-Light Enhancement, and Light SuppressionIEEE Transactions on Multimedia10.1109/TMM.2024.338842026(9292-9303)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3388420
Ashwini KNenavath HKumar Jatoth R(2024)EPQ-GAN: Evolutionary Perceptual Quality Assessment Generative Adversarial Network for Image DehazingIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.339485725:10(14710-14724)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3394857
Wang YYan XWang FXie HYang WZhang XQin JWei M(2024)UCL-Dehaze: Toward Real-World Image Dehazing via Unsupervised Contrastive LearningIEEE Transactions on Image Processing10.1109/TIP.2024.336215333(1361-1374)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2024.3362153
Chen ZHe ZLu Z(2024)DEA-Net: Single Image Dehazing Based on Detail-Enhanced Convolution and Content-Guided AttentionIEEE Transactions on Image Processing10.1109/TIP.2024.335410833(1002-1015)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2024.3354108
Feng YSu ZMa LLi XLiu RZhou F(2024)Bridging the Gap Between Haze Scenarios: A Unified Image Dehazing ModelIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.341467734:11_Part_1(11070-11085)Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3414677
Liang ZZhang WRuan RZhuang PXie XLi C(2024)Underwater Image Quality Improvement via Color, Detail, and Contrast RestorationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.329752434:3(1726-1742)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1109/TCSVT.2023.3297524
Kumari ASahoo S(2024)A new fast and efficient dehazing and defogging algorithm for single remote sensing imagesSignal Processing10.1016/j.sigpro.2023.109289215:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.sigpro.2023.109289
Dave CPatel HKumar A(2024)Unsupervised single image dehazing — A contour approachJournal of Visual Communication and Image Representation10.1016/j.jvcir.2024.104119100:COnline publication date: 17-Jul-2024
https://dl.acm.org/doi/10.1016/j.jvcir.2024.104119
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents