A Deep Learning Framework for Segmentation of Road Defects Using ResUNet-a
Authors: 
Iason Katsamenis, George Kopsiaftis, Athanasios Voulodimos, Ioannis Rallis, Ioannis Georgoulas, Charalampos Zafeiropoulos, Anastasios DoulamisAuthors Info & Claims
Iason Katsamenis, George Kopsiaftis, Athanasios Voulodimos, Ioannis Rallis, Ioannis Georgoulas, Charalampos Zafeiropoulos, Anastasios DoulamisAuthors Info & ClaimsPages 449 - 455
Abstract
We present a deep learning framework leveraging the ResUNet-a framework for pixel-wise semantic segmentation of cracks and potholes. By integrating key components including a U-Net encoder/decoder backbone, residual connections, atrous convolutions, pyramid scene parsing pooling, and multi-tasking inference, the proposed method exhibits robustness in capturing intricate spatial details and inter-pixel contextual relationships essential for accurate road defect segmentation. Experimental results validate the efficacy of the proposed approach, with ResUNet-a consistently surpassing the conventional U-Net model, demonstrating its superior performance in crack and pothole segmentation tasks and thus providing a useful auxiliary tool for road maintenance and safety.
References
[1]
T. U. Ahmed 2019. An Integrated CNN-RNN Framework to Assess Road Crack. In 2019 22nd International Conference on Computer and Information Technology (ICCIT). 1–6.
[2]
M. Z. Alom 2018. Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv preprint:1802.06955 (2018).
[3]
Ji-Won Baek and Kyungyong Chung. 2020. Pothole classification model using edge detection in road image. Applied Sciences 10, 19 (2020), 6662.
[4]
Gedas Bertasius, Jianbo Shi, and Lorenzo Torresani. 2016. Semantic segmentation with boundary neural fields. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3602–3610.
[5]
Gunilla Borgefors. 1986. Distance transformations in digital images. Computer vision, graphics, and image processing 34, 3 (1986), 344–371.
[6]
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. 2017. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence 40, 4 (2017), 834–848.
[7]
Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. 2017. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587 (2017).
[8]
Amita Dhiman and Reinhard Klette. 2019. Pothole detection using computer vision and learning. IEEE Transactions on Intelligent Transportation Systems 21, 8 (2019), 3536–3550.
[9]
Foivos I Diakogiannis, François Waldner, Peter Caccetta, and Chen Wu. 2020. ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data. ISPRS Journal of Photogrammetry and Remote Sensing 162 (2020), 94–114.
[10]
Markus Eisenbach, Ronny Stricker, Daniel Seichter, Karl Amende, Klaus Debes, Maximilian Sesselmann, Dirk Ebersbach, Ulrike Stoeckert, and Horst-Michael Gross. 2017. How to get pavement distress detection ready for deep learning? A systematic approach. In 2017 international joint conference on neural networks (IJCNN). IEEE, 2039–2047.
[11]
Rui Fan, Xiao Ai, and Naim Dahnoun. 2018. Road surface 3d reconstruction based on dense subpixel disparity map estimation. IEEE Transactions on Image Processing 27, 6 (2018), 3025–3035.
[12]
Rui Fan and Ming Liu. 2019. Road damage detection based on unsupervised disparity map segmentation. IEEE Transactions on Intelligent Transportation Systems 21, 11 (2019), 4906–4911.
[13]
Rui Fan, Umar Ozgunalp, Brett Hosking, Ming Liu, and Ioannis Pitas. 2019. Pothole detection based on disparity transformation and road surface modeling. IEEE Transactions on Image Processing 29 (2019), 897–908.
[14]
Rui Fan, Hengli Wang, Mohammud J Bocus, and Ming Liu. 2020. We learn better road pothole detection: from attention aggregation to adversarial domain adaptation. In Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16. Springer, 285–300.
[15]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Identity mappings in deep residual networks. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14. Springer, 630–645.
[16]
H. Huang 2020. UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation. In 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 1055–1059. https://doi.org/10.1109/ICASSP40776.2020.9053405
[17]
M. D. Jenkins 2018. A deep convolutional neural network for semantic pixel-wise segmentation of road and pavement surface cracks. In 2018 26th European Signal Processing Conference (EUSIPCO). IEEE, 2120–2124.
[18]
Iason Katsamenis, Nikolaos Bakalos, Eftychios Protopapadakis, Eleni Eirini Karolou, Georgios Kopsiaftis, and Athanasios Voulodimos. 2023. Real time road defect monitoring from UAV visual data sources. In Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments. 603–609.
[19]
Iason Katsamenis, Agapi Davradou, Eleni Eirini Karolou, Eftychios Protopapadakis, Anastasios Doulamis, Nikolaos Doulamis, and Dimitris Kalogeras. 2022. Evaluating YOLO transferability limitation for road infrastructures monitoring. In Novel & Intelligent Digital Systems Conferences. Springer, 349–358.
[20]
Iason Katsamenis, Nikolaos Doulamis, Anastasios Doulamis, Eftychios Protopapadakis, and Athanasios Voulodimos. 2022. Simultaneous Precise Localization and Classification of metal rust defects for robotic-driven maintenance and prefabrication using residual attention U-Net. Automation in Construction 137 (2022), 104182.
[21]
Iason Katsamenis, Eleni Eirini Karolou, Agapi Davradou, Eftychios Protopapadakis, Anastasios Doulamis, Nikolaos Doulamis, and Dimitris Kalogeras. 2022. TraCon: A novel dataset for real-time traffic cones detection using deep learning. In Novel & Intelligent Digital Systems Conferences. Springer, 382–391.
[22]
Iason Katsamenis, Eftychios Protopapadakis, Nikolaos Bakalos, Andreas Varvarigos, Anastasios Doulamis, Nikolaos Doulamis, and Athanasios Voulodimos. 2023. A Few-Shot Attention Recurrent Residual U-Net for Crack Segmentation. In International Symposium on Visual Computing. Springer, 199–209.
[23]
Iason Katsamenis, Eftychios Protopapadakis, Anastasios Doulamis, Nikolaos Doulamis, and Athanasios Voulodimos. 2020. Pixel-level corrosion detection on metal constructions by fusion of deep learning semantic and contour segmentation. In International Symposium on Visual Computing. Springer, 160–169.
[24]
Iason Katsamenis, Athanasios Sakelliou, Nikolaos Bakalos, Eftychios Protopapadakis, Christos Klaridopoulos, Nikolaos Frangakis, Matthaios Bimpas, and Dimitris Kalogeras. 2023. Deep transformer networks for precise pothole segmentation tasks. In Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments. 596–602.
[25]
J. König 2019. A Convolutional Neural Network for Pavement Surface Crack Segmentation Using Residual Connections and Attention Gating. In 2019 IEEE International Conference on Image Processing (ICIP). 1460–1464. https://doi.org/10.1109/ICIP.2019.8803060
[26]
S. L. Lau 2020. Automated Pavement Crack Segmentation Using U-Net-Based Convolutional Neural Network. IEEE Access 8 (2020), 114892–114899. https://doi.org/10.1109/ACCESS.2020.3003638
[27]
Yahui Liu, Jian Yao, Xiaohu Lu, Renping Xie, and Li Li. 2019. DeepCrack: A deep hierarchical feature learning architecture for crack segmentation. Neurocomputing 338 (2019), 139–153.
[28]
J. Long, E. Shelhamer, and T. Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3431–3440.
[29]
Dimitrios Marmanis, Konrad Schindler, Jan Dirk Wegner, Silvano Galliani, Mihai Datcu, and Uwe Stilla. 2018. Classification with an edge: Improving semantic image segmentation with boundary detection. ISPRS Journal of Photogrammetry and Remote Sensing 135 (2018), 158–172.
[30]
N. Ogawa 2020. Distress Level Classification of Road Infrastructures via CNN Generating Attention Map. In 2020 IEEE 2nd Global Conference on Life Sciences and Technologies (LifeTech). 97–98. https://doi.org/10.1109/LifeTech48969.2020.1570619126
[31]
O. Oktay 2018. Attention u-net: Learning where to look for the pancreas. arXiv preprint:1804.03999 (2018).
[32]
H. Oliveira and P. L. Correia. 2009. Automatic road crack segmentation using entropy and image dynamic thresholding. In 2009 17th European Signal Processing Conference. 622–626.
[33]
Yashon O Ouma and M Hahn. 2017. Pothole detection on asphalt pavements from 2D-colour pothole images using fuzzy c-means clustering and morphological reconstruction. Automation in Construction 83 (2017), 196–211.
[34]
A. K. Pandey 2022. Convolution neural networks for pothole detection of critical road infrastructure. Comp. and Electrical Engineering 99 (2022), 107725. https://doi.org/10.1016/j.compeleceng.2022.107725
[35]
Vosco Pereira, Satoshi Tamura, Satoru Hayamizu, and Hidekazu Fukai. 2019. Semantic segmentation of paved road and pothole image using u-net architecture. In 2019 International Conference of Advanced Informatics: Concepts, Theory and Applications (ICAICTA). IEEE, 1–4.
[36]
Eftychios Protopapadakis, Iason Katsamenis, and Anastasios Doulamis. 2020. Multi-label deep learning models for continuous monitoring of road infrastructures. In Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive Environments. 1–7.
[37]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18. Springer, 234–241.
[38]
Sebastian Ruder. 2017. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv:1706.05098 (2017).
[39]
Bhavan Kumar SB, S Guhan, Manyam Kishore, R Santhosh, 2023. Deep Learning Approach for Pothole Detection-A Systematic Review. In 2023 Second International Conference on Electronics and Renewable Systems (ICEARS). IEEE, 1410–1414.
[40]
Yong Shi, Limeng Cui, Zhiquan Qi, Fan Meng, and Zhensong Chen. 2016. Automatic road crack detection using random structured forests. IEEE Transactions on Intelligent Transportation Systems 17, 12 (2016), 3434–3445.
[41]
Nikola Slavkovic and Milan Bjelica. 2019. Risk prediction algorithm based on image texture extraction using mobile vehicle road scanning system as support for autonomous driving. Journal of Electronic Imaging 28, 3 (2019), 033034–033034.
[42]
N. Tanaka and K. Uematsu. 1998. A Crack Detection Method in Road Surface Images Using Morphology.MVA 98 (1998), 17–19.
[43]
A. Voulodimos 2021. A Few-Shot U-Net Deep Learning Model for COVID-19 Infected Area Segmentation in CT Images. Sensors 21, 6 (2021). https://doi.org/10.3390/s21062215
[44]
Athanasios Voulodimos, Eftychios Protopapadakis, Iason Katsamenis, Anastasios Doulamis, and Nikolaos Doulamis. 2021. Deep learning models for COVID-19 infected area segmentation in CT images. In Proceedings of the 14th PErvasive Technologies Related to Assistive Environments Conference. 404–411.
[45]
Tao Wang, Yasin Amara Sekou S Dra, Xiaopei Cai, Zhiqiang Cheng, De Zhang, Yi Lin, and Huayang Yu. 2022. Advanced cold patching materials (CPMs) for asphalt pavement pothole rehabilitation: State of the art. Journal of Cleaner Production 366 (2022), 133001.
[46]
X Yu and E Salari. 2011. Pavement pothole detection and severity measurement using laser imaging. In 2011 IEEE International Conference on Electro/Information Technology. IEEE, 1–5.
[47]
Lei Zhang, Fan Yang, Yimin Daniel Zhang, and Ying Julie Zhu. 2016. Road crack detection using deep convolutional neural network. In 2016 IEEE international conference on image processing (ICIP). IEEE, 3708–3712.
[48]
Yajun Zhang, Shusheng Zhang, Rui Huang, Bo Huang, Lei Yang, and Jiachen Liang. 2021. A deep learning-based approach for machining process route generation. The International Journal of Advanced Manufacturing Technology 115, 11 (2021), 3493–3511.
[49]
Z. Zhang, Q. Liu, and Y. Wang. 2018. Road Extraction by Deep Residual U-Net. IEEE Geoscience and Remote Sensing Letters 15, 5 (2018), 749–753. https://doi.org/10.1109/LGRS.2018.2802944
[50]
H. Zhao, G. Qin, and X. Wang. 2010. Improvement of canny algorithm based on pavement edge detection. In 2010 3rd International Congress on Image and Signal Processing, Vol. 2. 964–967. https://doi.org/10.1109/CISP.2010.5646923
[51]
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. 2017. Pyramid scene parsing network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2881–2890.
[52]
Qin Zou, Yu Cao, Qingquan Li, Qingzhou Mao, and Song Wang. 2012. CrackTree: Automatic crack detection from pavement images. Pattern Recognition Letters 33, 3 (2012), 227–238.
Index Terms
- A Deep Learning Framework for Segmentation of Road Defects Using ResUNet-a
Recommendations
Deep transformer networks for precise pothole segmentation tasks
PETRA '23: Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive EnvironmentsPotholes on the road surface are a significant safety hazard and can cause severe damage to vehicles. Identifying and repairing potholes is a challenging task that requires efficient and accurate methods. In recent years, deep learning models, such as U-...
Automatic Segmentation of the Prostate on 3D CT Images by Using Multiple Deep Learning Networks
ICBBE '18: Proceedings of the 2018 5th International Conference on Biomedical and Bioinformatics EngineeringAutomatic segmentation of the prostate on CT images has many applications in prostate cancer diagnosis and therapy. However, prostate segmentation from CT images is a very challenging task due to the low contrast of soft tissue and the large variations ...
Accurate Kidney Segmentation in CT Scans Using Deep Transfer Learning
Smart MultimediaAbstractA competitive model for kidney segmentation in CT scans is trained using the publicly-available KiTS19 dataset. The model performed well against the KiTS19 test dataset, achieving a Sørensen–Dice coefficient of 0.9620 when generating kidney ...
Comments
Information & Contributors
Information
Published In
June 2024
708 pages
ISBN:9798400717604
DOI:10.1145/3652037
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 26 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- RESEARCH ? CREATE ? IN- NOVATE
Conference
PETRA '24
PETRA '24: The PErvasive Technologies Related to Assistive Environments Conference
June 26 - 28, 2024
Crete, Greece
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 267Total Downloads
- Downloads (Last 12 months)267
- Downloads (Last 6 weeks)76
Reflects downloads up to 13 Jan 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in