Abstract
Smoke is a crucial indicator of early fires and gas leaks. By segmentation of smoke in the image, the detailed information such as smoke direction of spread, and source location can be obtained. Considering the popularity of video surveillance systems, smoke segmentation is of great significance. This paper uses an efficient boosted light-weighted network for smoke semantic segmentation with only 0.53M network parameters. Firstly, we propose a novel Smoke Feature Extractor (SFE) to improve the capability of smoke feature representation in encoders. The SFE achieves scale invariance by repeatedly stacking Multi-scale Foreground Enhancement Module (MSFEM) at different coding stages. The MSFEM increases the field of view of features by down-sampling the feature maps, and fusing information from different scale spaces into weights to enhance smoke foreground information. Secondly, we propose a novel Attention-guided Coupled Feature Fusion Module (ACFFM) that introduces Self-Refinement Coefficients (SRCs) generated from cross-layer fusion to weight the original layer images. This two-stage fusion approach effectively utilizes information from different scales to alleviate the impact of scale changes and performs hierarchical decoding. ACFFM serves as a step-by-step recovery model that guides cross-level feature mapping. It retains the original feature size to capture local information, while effectively preventing small objects from being ignored. Finally, we propose a Smoke Feature Decoder (SFD) and utilize a Global Coefficient Path to further aggregate feature expression capabilities. The smoke feature decoder fuses the outputs from different levels of ACFFM and weights them with deep global semantic attention coefficients. Experimental results on synthetic and forest smoke datasets show the effectiveness and superiority of our proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Muhammad, K., Khan, S., Baik, S.W.: Efficient convolutional neural networks for fire detection in surveillance applications. In: Deep Learning in Computer Vision: Principles and Applications (2020)
Finney, M.A.: The wildland fire system and challenges for engineering. Fire Saf. J. (2020)
Muhammad, K., Hussain, T., Tanveer, M., Sannino, G., de Albuquerque, V.: Cost-effective video summarization using deep CNN with hierarchical weighted fusion for IoT surveillance networks. IEEE Internet Things J. 7(5), 4455–4463 (2020)
Cui, F.: Deployment and integration of smart sensors with IoT devices detecting fire disasters in huge forest environment. Comput. Commun. 150, 818–827 (2020)
Yuan, F., Zhang, L., Xia, X., Huang, Q., Li, X.: A wave-shaped deep neural network for smoke density estimation. IEEE Trans. Image Process. 29, 2301–2313 (2020)
ByoungChul, K., JunOh, P., Jae-Yeal, N.: Spatiotemporal bag-of-features for early wildfire smoke detection. Image Vis. Comput. 31(10), 786–795 (2013)
Muhammad, K., Ahmad, J., Lv, Z., Bellavista, P., Yang, P., Baik, S.W.: Efficient deep CNN-based fire detection and localization in video surveillance applications. IEEE Trans. Syst. Man Cybern. Syst. 99, 1–16 (2018)
Jing, T., Meng, Q., Hou, H.: SmokeSeger: a transformer-CNN coupled model for urban scene smoke segmentation. IEEE Trans. Ind. Inform. (2023)
Nguyen, T.K.T., Kim, J.M.: Multistage optical smoke detection approach for smoke alarm systems. Opt. Eng. 52(5) (2013)
Dimitropoulos, K., Barmpoutis, P., Grammalidis, N.: Higher order linear dynamical systems for smoke detection in video surveillance applications. IEEE Trans. Circuits Syst. Video Technol. 27(5), 1143–1154 (2017)
Zhao, Y.: Candidate smoke region segmentation of fire video based on rough set theory. J. Electr. Comput. Eng. (2015)
Wang, H., Chen, Y.A.: Smoke image segmentation algorithm based on rough set and region growing. J. Forest Sci. 65(8) (2019)
Tung, T., Kim, J.: An effective four-stage smoke-detection algorithm using video images for early fire-alarm systems. Fire Saf. J. 46(5), 276–282 (2011)
Filonenko, A., Hernandez, D.C., Jo, K.-H.: Fast smoke detection for video surveillance using CUDA. IEEE Trans. Ind. Inf. 14(2), 725–733 (2018)
Yuan, F.: A fast accumulative motion orientation model based on integral image for video smoke detection. Pattern Recognit. Lett. 29(7), 925–932 (2008)
Tian, H., Li, W., Ogunbona, P.O., Wang, L.: Detection and separation of smoke from single image frames. IEEE Trans. Image Process. 27(3), 1164–1177 (2018)
Yuan, F., Fang, Z., Wu, S., Yang, Y., Fang, Y.: Real-time image smoke detection using staircase searching-based dual threshold AdaBoost and dynamic analysis. IET Image Process. 9(10), 849–856 (2015)
Appana, D.K., Islam, M.R., Khan, S.A., Kim, J.: A video-based smoke detection using smoke flow pattern and spatial-temporal energy analyses for alarm systems. Inf. Sci. 418, 91–101 (2017)
Alamgir, N., Nguyen, K., Chandran, V., Boles, W.: Combining multi-channel color space with local binary co-occurrence feature descriptors for accurate smoke detection from surveillance videos. Fire Saf. J. 102, 1–10 (2018)
Yuan, F., Zhang, L., Xia, X., Wan, B., Huang, Q., Li, X.: Deep smoke segmentation. Neurocomputing 357(10), 248–260 (2019)
Frizzi, S., Bouchouicha, M., Ginoux, J.-M., Moreau, E., Sayadi, M.: Convolutional neural network for smoke and fire semantic segmentation. IET Image Process. 15(6), 634–647 (2021)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of the International Conference on Learning Representation (2014)
Wang, Y., Luo, Z., Chen, D., Li, Y.: Semantic segmentation of fire and smoke images based on dual attention mechanism. In: 2022 4th International Conference on Frontiers Technology of Information and Computer (ICFTIC), pp. 185–190 (2022)
Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 833–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_49
Kundu, S., Maulik, U., Sheshanarayana, R., Ghosh, S.: Vehicle smoke synthesis and attention-based deep approach for vehicle smoke detection. IEEE Trans. Ind. Appl. 59(2), 2581–2589 (2023)
Cao, Y., Tang, Q., Wu, X., Lu, X.: EFFNet: Enhanced feature foreground network for video smoke source prediction and detection. IEEE Trans. Circuits Syst. Video Technol. 32(4), 1820–1833 (2022)
Tao, H., Duan, Q., Lu, M., Hu, Z.: Learning discriminative feature representation with pixel-level supervision for forest smoke recognition. Pattern Recognit. 143 (2023)
Yuan, F., Dong, Z., Zhang, L., Xia, X., Shi, J.: Cubic-cross convolutional attention and count prior embedding for smoke segmentation. Pattern Recognit. 131 (2022)
Xia, X., Zhan, K., Peng, Y., Fang, Y.: Texture-aware network for smoke density estimation. In: IEEE International Conference on Visual Communications and Image Processing, pp. 1–5 (2022)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)
Carrio, A., Sampedro, C., Rodriguez-Ramos, A., Campoy, P.: A review of deep learning methods and applications for unmanned aerial vehicles. J. Sens. (2017)
Anim Hossain, F.M., Zhang, Y.: MsFireD-Net: a lightweight and efficient convolutional neural network for flame and smoke segmentation. J. Autom. Intell. 2(3), 130–138 (2023)
Xia, W., Yu, F., Wang, H., Hong, R.: A high-precision lightweight smoke detection model based on SE attention mechanism. In: 2022 2nd International Conference on Consumer Electronics and Computer Engineering (ICCECE), pp. 941–944 (2022)
Yuan, F., Li, K. , Wang, C., Fang, Z.: A lightweight network for smoke semantic segmentation. Pattern Recognit. (2023)
Paszke, A., Chaurasia, A., Kim, S., Culurciello, E.: ENET: a deep neural network architecture for real-time semantic segmentation. arXiv preprint arXiv:1606.02147 (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Wei, J., Wang, S.H., Huang, Q.M.: F3Net: fusion, feedback and focus for salient object detection. In: AAAI (2020)
Romera, E., Álvarez, J.M., Bergasa, L.M., Arroyo, R.: ERFNet: efficient residual factorized ConvNet for real-time semantic segmentation. IEEE Trans. Intell. Transp. Syst. 19(1), 263–272 (2018)
Wang, Y., Zhou, Q., Liu, J., Xiong, J., Latecki. L.J.: LEDNet: a lightweight encoder-decoder network for real-time semantic segmentation. In: Proceedings of the IEEE International Conference on Image Processing, pp. 1860–1864 (2019)
Li, H., Xiong, P., Fan, H., Sun, J.: DFANet: deep feature aggregation for real-time semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9514–9523 (2019)
Wu, T., Tang, S., Zhang, R., Cao, J., Zhang, Y.: CGNet: a light-weight context guided network for semantic segmentation. IEEE Trans. Image Process. 30, 1169–1179 (2021)
Mehta, S., Rastegari, M., Caspi, A., Shapiro, L., Hajishirzi, H.: ESPNet: efficient spatial pyramid of dilated convolutions for semantic segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11214, pp. 561–580. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01249-6_34
Guo, W., Xiao, X., Hui, Y., Yang, W., Sadovnik, A.: Heterogeneous attention nested u-shaped network for blur detection. IEEE Signal Process. Lett. 29, 140–144 (2022)
Acknowledgments
This work was partially supported by the National Natural Science Foundation of China (61862029), and Capacity Construction Project of Shanghai Local Colleges (23010504100).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, K., Wang, C., Meng, C., Yuan, F. (2024). LightNet+: Boosted Light-Weighted Network for Smoke Semantic Segmentation. In: Zhai, G., Zhou, J., Ye, L., Yang, H., An, P., Yang, X. (eds) Digital Multimedia Communications. IFTC 2023. Communications in Computer and Information Science, vol 2066. Springer, Singapore. https://doi.org/10.1007/978-981-97-3623-2_6
Download citation
DOI: https://doi.org/10.1007/978-981-97-3623-2_6
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-3622-5
Online ISBN: 978-981-97-3623-2
eBook Packages: Computer ScienceComputer Science (R0)