Abstract
Image dehazing is a pre-processing step in computer vision tasks, that has attracted considerable attention from the research community. Existing CNN-based methods ignore haze-related priors and rarely use a coarse-to-fine scheme in a feed-forward architecture to remove haze due to increasing network depth and parameters. This results in sub-optimal dehazing results. To address these problems, a multi-scale attentive recurrent network is proposed for image dehazing, which consists of a haze attention map predicted network and a recurrent encoder-decoder network. First, by assuming that haze in an image is formed by multiple layers with different depths, the haze attention map predicted network is designed for generating the map with multiple stages via a multi-scale recurrent framework. Second, the haze attention map is viewed as the haze-related prior and guides the subsequent recurrent encoder-decoder network to be aware of haze concentration information. Finally, for leveraging the intermediate information and optimizing the dehazing result with less parameters and more robust features, the recurrent residual operations which pass the features of selected layers at the current time step to the corresponding layers at the next time step are applied in the recurrent encoder-decoder network for removing haze following a coarse-to-fine strategy. Experiments on synthetic and real images demonstrate that our method outperforms state-of-the-art methods considering both visual and quantitative evaluations. In addition, our method is also suitable for real-time processing.











Similar content being viewed by others
References
Alajarmeh A, Salam R, Abdulrahim K, Marhusin M, Zaidan A, Zaidan B (2018) Real-time framework for image dehazing based on linear transmission and constant-time airlight estimation. Inf Sci 436:108–130
Berman D, Avidan S et al (2016) Non-local image dehazing. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1674–1682
Cai B, Xu X, Jia K, Qing C, Tao D (2016) Dehazenet: An end-to-end system for single image haze removal. IEEE Trans Image Process 25(11):5187–5198
Chen CF, Fan Q, Mallinar N, Sercu T, Feris R (2018) Big-little net: An efficient multi-scale feature representation for visual and speech recognition. arXiv:180703848
Chen D, He M, Fan Q, Liao J, Zhang L, Hou D, Yuan L, Hua G (2019a) Gated context aggregation network for image dehazing and deraining. In: 2019 IEEE Winter conference on applications of computer vision (WACV). IEEE, pp 1375–1383
Chen SZ, Ren ZG, Lian QS (2016) Single image dehazing algorithm based on improved dark channel prior and guided filter. Acta Automatica Sinica 42(3):455–465
Chen Y, Wang J, Chen X, Sangaiah AK, Yang K, Cao Z (2019b) Image super-resolution algorithm based on dual-channel convolutional neural networks. Appl Sci 9(11):2316
Chen Y, Wang J, Chen X, Zhu M, Yang K, Wang Z, Xia R (2019c) Single-image super-resolution algorithm based on structural self-similarity and deformation block features. IEEE Access 7:58791–58801
Chen Y, Wang J, Liu S, Chen X, Xiong J, Xie J, Yang K (2019d) Multiscale fast correlation filtering tracking algorithm based on a feature fusion model. Concurrency and Computation: Practice and Experience e5533
Chen Y, Wang J, Xia R, Zhang Q, Cao Z, Yang K (2019e) The visual object tracking algorithm research based on adaptive combination kernel. J Ambient Intell Human Comput 10(12):4855–4867
Chen Y, Xiong J, Xu W, Zuo J (2019f) A novel online incremental and decremental learning algorithm based on variable support vector machine. Clust Comput 22(3):7435–7445
Chen Y, Xu W, Zuo J, Yang K (2019g) The fire recognition algorithm using dynamic feature fusion and iv-svm classifier. Clust Comput 22(3):7665–7675
Chen Y, Tao J, Liu L, Xiong J, Xia R, Xie J, Zhang Q, Yang K (2020a) Research of improving semantic image segmentation based on a feature fusion model. Journal of Ambient Intelligence and Humanized Computing
Chen Y, Tao J, Zhang Q, Yang K, Chen X, Xiong J, Xia R, Xie J (2020b) Saliency detection via the improved hierarchical principal component analysis method. Wirel Commun Mob Comput 2020
Deng Z, Zhu L, Hu X, Fu CW, Xu X, Zhang Q, Qin J, Heng PA (2019) Deep multi-model fusion for single-image dehazing. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 2453–2462
Dong Y, Liu Y, Zhang H, Chen S, Qiao Y (2020) Fd-gan: Generative adversarial networks with fusion-discriminator for single image dehazing. Proceedings of the Innovative Applications of Artificial Intelligence Conference (AAAI)
Fan DP, Cheng MM, Liu JJ, Gao SH, Hou Q, Borji A (2018) Salient objects in clutter: Bringing salient object detection to the foreground. In: Proceedings of the european conference on computer vision (ECCV), pp 186–202
Fan DP, Wang W, Cheng MM, Shen J (2019) Shifting more attention to video salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8554–8564
Fattal R (2014) Dehazing using color-lines. ACM Trans Graphics (TOG) 34(1):1–14
Fu K, Zhao Q, Gu IYH, Yang J (2019) Deepside: A general deep framework for salient object detection. Neurocomputing 356:69–82
Galdran A (2018) Image dehazing by artificial multiple-exposure image fusion. Signal Process 149:135–147
Gao S, Cheng MM, Zhao K, Zhang XY, Yang MH, Torr PH (2019) Res2net: A new multi-scale backbone architecture. IEEE Trans Pattern Anal Mach Intell
Guan B, Yao J, Zhang G, Wang X (2019) Thigh fracture detection using deep learning method based on new dilated convolutional feature pyramid network. Pattern Recogn Lett 125:521–526
He K, Sun J, Tang X (2010) Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell 33(12):2341–2353
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778
Hernandez-Beltran JE, Diaz-Ramirez VH, Trujillo L, Legrand P (2019) Design of estimators for restoration of images degraded by haze using genetic programming. Swarm Evolution Comput 44:49–63
Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 7132–7141
Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1125–1134
Kim JH, Jang WD, Sim JY, Kim CS (2013) Optimized contrast enhancement for real-time image and video dehazing. J Vis Commun Image Represent 24(3):410–425
Li B, Peng X, Wang Z, Xu J, Feng D (2017) Aod-net: All-in-one dehazing network. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 4770–4778
Li B, Ren W, Fu D, Tao D, Feng D, Zeng W, Wang Z (2018a) Benchmarking single-image dehazing and beyond. IEEE Trans Image Process 28(1):492–505
Li J, Li G, Fan H (2018b) Image dehazing using residual-based deep cnn. IEEE Access 6:26831–26842
Ling Z, Fan G, Gong J, Guo S (2019) Learning deep transmission network for efficient image dehazing. Multimed Tools Appl 78(1):213–236
Liu R, Fan X, Hou M, Jiang Z, Luo Z, Zhang L (2018) Learning aggregated transmission propagation networks for haze removal and beyond. IEEE Trans Neural Netw Learn Syst 30(10):2973–2986
Liu X, Ma Y, Shi Z, Chen J (2019a) Griddehazenet: Attention-based multi-scale network for image dehazing. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 7314–7323
Liu Y, Pan J, Ren J, Su Z (2019b) Learning deep priors for image dehazing. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 2492–2500
Liu Y, Shang J, Pan L, Wang A, Wang M (2019c) A unified variational model for single image dehazing. IEEE Access 7:15722–15736
Luo Y, Qin J, Xiang X, Tan Y, Liu Q, Xiang L (2020) Coverless real-time image information hiding based on image block matching and dense convolutional network. J Real-Time Image Proc 17(1):125–135
Qian R, Tan RT, Yang W, Su J, Liu J (2018) Attentive generative adversarial network for raindrop removal from a single image. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2482–2491
Qin X, Wang Z, Bai Y, Xie X, Jia H (2020) Ffa-net: Feature fusion attention network for single image dehazing. In: AAAI, pp 11908–11915
Qu Y, Chen Y, Huang J, Xie Y (2019) Enhanced pix2pix dehazing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 8160–8168
Ren W, Ma L, Zhang J, Pan J, Cao X, Liu W, Yang MH (2018) Gated fusion network for single image dehazing. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 3253–3261
Tan RT (2008) Visibility in bad weather from a single image. In: 2008 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1–8
Tao X, Gao H, Shen X, Wang J, Jia J (2018) Scale-recurrent network for deep image deblurring. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 8174–8182
Wang JB, He N, Zhang LL, Lu K (2015) Single image dehazing with a physical model and dark channel prior. Neurocomputing 149:718–728
Wang W, Zhao S, Shen J, Hoi SC, Borji A (2019) Salient object detection with pyramid attention and salient edges. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1448–1457
Xu H, Guo J, Liu Q, Ye L (2012) Fast image dehazing using improved dark channel prior. In: 2012 IEEE international conference on information science and technology. IEEE, pp 663–667
Yang D, Sun J (2018) Proximal dehaze-net: A prior learning-based deep network for single image dehazing. In: Proceedings of the european conference on computer vision (ECCV), pp 702–717
Yin S, Wang Y, Yang YH (2019) A novel residual dense pyramid network for image dehazing. Entropy 21(11):1123
Yin S, Wang Y, Yang Y H (2020) A novel image-dehazing network with a parallel attention block. Pattern Recogn 102:107255
Yu F, Liu L, He B, Huang Y, Shi C, Cai S, Song Y, Du S, Wan Q (2019a) Analysis and fpga realization of a novel 5d hyperchaotic four-wing memristive system, active control synchronization, and secure communication application. Complexity 2019
Yu F, Liu L, Xiao L, Li K, Cai S (2019b) A robust and fixed-time zeroing neural dynamics for computing time-variant nonlinear equation using a novel nonlinear activation function. Neurocomputing 350:108–116
Yu T, Song K, Miao P, Yang G, Yang H, Chen C (2019c) Nighttime single image dehazing via pixel-wise alpha blending. IEEE Access 7:114619–114630
Zhang H, Patel VM (2018) Densely connected pyramid dehazing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 3194–3203
Zhang H, Sindagi V, Patel VM (2018) Multi-scale single image dehazing using perceptual pyramid deep network. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp 902–911
Zhang J, Xie Z, Sun J, Zou X, Wang J (2020) A cascaded r-cnn with multiscale attention and imbalanced samples for traffic sign detection. IEEE Access 8:29742–29754
Zhao JX, Liu JJ, Fan DP, Cao Y, Yang J, Cheng MM (2019) Egnet: Edge guidance network for salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 8779–8788
Zhu H, Peng X, Chandrasekhar V, Li L, Lim JH (2018) Dehazegan: When image dehazing meets differential programming. In: IJCAI, pp 1234–1240
Zhu Q, Mai J, Shao L (2015) A fast single image haze removal algorithm using color attenuation prior. IEEE Trans Image Process 24(11):3522–3533
Acknowledgements
This research was funded by National Natural Science Foundation of China: 61502396; the Education Department Foundation of Sichuan Province: 18ZB0484; and NSERC, Canada.
Author information
Authors and Affiliations
Contributions
All the authors conceived of the idea for the paper. Y.W. and S.Y. implemented the algorithm, performed the experiments and wrote the manuscript. A.B. provided technical guidance and revised the paper.
Corresponding author
Ethics declarations
Conflict of Interests
The authors declare no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, Y., Yin, S. & Basu, A. A multi-scale attentive recurrent network for image dehazing. Multimed Tools Appl 80, 32539–32565 (2021). https://doi.org/10.1007/s11042-021-11209-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-021-11209-z