research-article

Pothole Detection Based on Disparity Transformation and Road Surface Modeling

Authors:

Ioannis PitasAuthors Info & Claims

IEEE Transactions on Image Processing, Volume 29

Pages 897 - 908

https://doi.org/10.1109/TIP.2019.2933750

Published: 01 January 2020 Publication History

Abstract

Pothole detection is one of the most important tasks for road maintenance. Computer vision approaches are generally based on either 2D road image analysis or 3D road surface modeling. However, these two categories are always used independently. Furthermore, the pothole detection accuracy is still far from satisfactory. Therefore, in this paper, we present a robust pothole detection algorithm that is both accurate and computationally efficient. A dense disparity map is first transformed to better distinguish between damaged and undamaged road areas. To achieve greater disparity transformation efficiency, golden section search and dynamic programming are utilized to estimate the transformation parameters. Otsu’s thresholding method is then used to extract potential undamaged road areas from the transformed disparity map. The disparities in the extracted areas are modeled by a quadratic surface using least squares fitting. To improve disparity map modeling robustness, the surface normal is also integrated into the surface modeling process. Furthermore, random sample consensus is utilized to reduce the effects caused by outliers. By comparing the difference between the actual and modeled disparity maps, the potholes can be detected accurately. Finally, the point clouds of the detected potholes are extracted from the reconstructed 3D road surface. The experimental results show that the successful detection accuracy of the proposed system is around 98.7% and the overall pixel-level accuracy is approximately 99.6%.

References

[1]

J. S. Miller and W. Y. Bellinger, “Distress identification manual for the long-term pavement performance program,” Office Infrastruct. Res. Develop., Federal Highway Admin., Washington, DC, USA, Tech. Rep. FHWA-HRT-13-092, 2014.

[2]

S. Mathavan, K. Kamal, and M. Rahman, “A review of three-dimensional imaging technologies for pavement distress detection and measurements,” IEEE Trans. Intell. Transp. Syst., vol. 16, no. 5, pp. 2353–2362, Oct. 2015.

Digital Library

[3]

T. Kim and S. K. Ry, “Review and analysis of pothole detection methods,” J. Emerg. Trends Comput. Inf. Sci., vol. 5, no. 8, pp. 603–608, Aug. 2014.

[4]

R. Fan, J. Jiao, J. Pan, H. Huang, S. Shen, and M. Liu, “Real-time dense stereo embedded in a UAV for road inspection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, Jun. 2019. [Online]. Available: http://openaccess.thecvf.com/content_CVPRW_2019/html/UAVision/Fan_Real-Time_Dense_Stereo_Embedded_in_a_UAV_for_Road_Inspection_CVPRW_2019_paper.html

[5]

C. Koch, K. Georgieva, V. Kasireddy, B. Akinci, and P. Fieguth, “A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure,” Adv. Eng. Inform., vol. 29, no. 2, pp. 196–210, 2015.

Digital Library

[6]

Y.-C. Tsai and A. Chatterjee, “Pothole detection and classification using 3D technology and watershed method,” J. Comput. Civil Eng., vol. 32, no. 2, Mar. 2018, Art. no.

[7]

I. Pitas, Digital Image Processing Algorithms and Applications. Hoboken, NJ, USA: Wiley, 2000.

Digital Library

[8]

S. Li, C. Yuan, D. Liu, and H. Cai, “Integrated processing of image and GPR data for automated pothole detection,” J. Comput. Civil Eng., vol. 30, no. 6, Nov. 2016, Art. no.

[9]

E. Buza, S. Omanovic, and A. Huseinovic, “Pothole detection with image processing and spectral clustering,” in Proc. 2nd Int. Conf. Inf. Technol. Comput. Netw., Oct. 2013, p. 4853.

[10]

C. Koch and I. Brilakis, “Pothole detection in asphalt pavement images,” Adv. Eng. Inform., vol. 25, no. 3, pp. 507–515, 2011.

Digital Library

[11]

M. R. Jahanshahi, F. Jazizadeh, S. F. Masri, and B. Becerik-Gerber, “Unsupervised approach for autonomous pavement-defect detection and quantification using an inexpensive depth sensor,” J. Comput. Civil Eng., vol. 27, no. 6, pp. 743–754, 2013.

[12]

R. Fan, X. Ai, and N. Dahnoun, “Road surface 3D reconstruction based on dense subpixel disparity map estimation,” IEEE Trans. Image Process., vol. 27, no. 6, pp. 3025–3035, Jun. 2018.

[13]

C. Zhang and A. Elaksher, “An unmanned aerial vehicle-based imaging system for 3D measurement of unpaved road surface distresses,” Comput.-Aided Civil Infrastruct. Eng., vol. 27, no. 2, pp. 118–129, 2012.

Digital Library

[14]

R. Fan, “Real-time computer stereo vision for automotive applications,” Ph.D. dissertation, Dept. Elect. Electron. Eng., Univ. Bristol, Bristol, U.K., 2018.

[15]

Z. Zhang, X. Ai, C. K. Chan, and N. Dahnoun, “An efficient algorithm for pothole detection using stereo vision,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), May 2014, pp. 564–568.

[16]

A. Mikhailiuk and N. Dahnoun, “Real-time pothole detection on TMS320c6678 DSP,” in Proc. IEEE Int. Conf. Imaging Syst. Techn. (IST), Oct. 2016, pp. 123–128.

[17]

L. Cruz, D. Lucio, and L. Velho, “Kinect and RGBD images: Challenges and applications,” in Proc. 25th SIBGRAPI Conf. Patterns Images Tuts. (SIBGRAPI-T), Aug. 2012, pp. 36–49.

[18]

R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge, U.K.: Cambridge Univ. Press, 2003.

Digital Library

[19]

M. A. Fischler and R. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Commun. ACM, vol. 24, no. 6, pp. 381–395, 1981.

Digital Library

[20]

P. Pedregal, Introduction to Optimization, vol. 46. New York, NY, USA: Springer-Verlag, 2006.

[21]

U. Ozgunalp, R. Fan, X. Ai, and N. Dahnoun, “Multiple lane detection algorithm based on novel dense vanishing point estimation,” IEEE Trans. Intell. Transp. Syst., vol. 18, no. 3, pp. 621–632, Mar. 2017.

Digital Library

[22]

N. Otsu, “A threshold selection method from gray-level histograms,” IEEE Trans. Syst., Man, Cybern., vol. SMC-9, no. 1, pp. 62–66, Jan. 1979.

[23]

M. B. Dillencourt, H. Samet, and M. Tamminen, “A general approach to connected-component labeling for arbitrary image representations,” J. ACM, vol. 39, no. 2, pp. 253–280, Apr. 1992.

Digital Library

[24]

H. E. Tseng, L. Xu, and D. Hrovat, “Estimation of land vehicle roll and pitch angles,” Vehicle Syst. Dyn., vol. 45, no. 5, pp. 433–443, May 2007.

[25]

J. Oh and S. B. Choi, “Vehicle roll and pitch angle estimation using a cost-effective six-dimensional inertial measurement unit,” Proc. Inst. Mech. Eng., D, J. Automobile Eng., vol. 227, no. 4, pp. 577–590, Apr. 2013.

[26]

J. Ryu, E. J. Rossetter, and J. C. Gerdes, “Vehicle sideslip and roll parameter estimation using GPS,” in Proc. AVEC Int. Symp. Adv. Vehicle Control, Sep. 2002, pp. 373–380.

[27]

J. Ryu and J. C. Gerdes, “Estimation of vehicle roll and road bank angle,” in Proc. Amer. Control Conf., vol. 3, Portland, OR, USA, Jun./Jul. 2005, pp. 2110–2115.

[28]

R. Labayrade and D. Aubert, “A single framework for vehicle roll, pitch, yaw estimation and obstacles detection by stereovision,” in Proc. IEEE Intell. Vehicles Symp., Jun. 2003, pp. 31–36.

[29]

P. Skulimowski, M. Owczarek, and P. Strumiłło, “Ground plane detection in 3D scenes for an arbitrary camera roll rotation through ‘V-disparity’ representation,” in Proc. Federated Conf. Comput. Sci. Inf. Syst. (FedCSIS), Sep. 2017, pp. 669–674.

[30]

M. Evans, R. Fan, and N. Dahnoun, “Iterative roll angle estimation from dense disparity map,” in Proc. 7th Medit. Conf. Embedded Comput. (MECO), Jun. 2018, pp. 1–4.

[31]

R. Fan, M. J. Bocus, and N. Dahnoun, “A novel disparity transformation algorithm for road segmentation,” Inf. Process. Lett., vol. 140, pp. 18–24, Dec. 2018.

[32]

I. Haller and S. Nedevschi, “Design of interpolation functions for subpixel-accuracy stereo-vision systems,” IEEE Trans. Image Process., vol. 21, no. 2, pp. 889–898, Feb. 2012.

Digital Library

[33]

K. Klasing, D. Althoff, D. Wollherr, and M. Buss, “Comparison of surface normal estimation methods for range sensing applications,” in Proc. IEEE Int. Conf. Robot. Autom., Kobe, Japan, May 2009, pp. 3206–3211.

[34]

C. Wang, H. Tanahashi, H. Hirayu, Y. Niwa, and K. Yamamoto, “Comparison of local plane fitting methods for range data,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 1, Dec. 2001, p. 1.

[35]

T. Vaudrey, C. Rabe, R. Klette, and J. Milburn, “Differences between stereo and motion behaviour on synthetic and real-world stereo sequences,” in Proc. 23rd Int. Conf. Image Vis. Comput. New Zealand, Nov. 2008, pp. 1–6.

[36]

A. Wedel, C. Rabe, T. Vaudrey, T. Brox, U. Franke, and D. Cremers, “Efficient dense scene flow from sparse or dense stereo data,” in Proc. 10th Eur. Conf. Comput. Vis., Marseille, France, Oct. 2008, pp. 739–751.

[37]

A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the KITTI vision benchmark suite,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2012, pp. 3354–3361.

[38]

M. Menze, C. Heipke, and A. Geiger, “Joint 3D estimation of vehicles and scene flow,” ISPRS Ann. Photogramm., Remote Sens. Spatial Inf. Sci., vol. 2, p. 427, Aug. 2015.

Cited By

Misra MSharma RTiwari SShankar A(2024)An optimized deep belief network based pothole detection model for asphalt roadIntelligent Decision Technologies10.3233/IDT-24012718:4(3041-3055)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.3233/IDT-240127
Wei CTan HZhang JFeng MLiu ZWang JZhang P(2024)Apple image recognition model based on optimized sYOLOv5 network with small object detectionProceedings of the 2024 3rd International Conference on Cryptography, Network Security and Communication Technology10.1145/3673277.3673371(548-552)Online publication date: 19-Jan-2024
https://dl.acm.org/doi/10.1145/3673277.3673371
Katsamenis IKopsiaftis GVoulodimos ARallis IGeorgoulas IZafeiropoulos CDoulamis A(2024)A Deep Learning Framework for Segmentation of Road Defects Using ResUNet-aProceedings of the 17th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3652037.3663935(449-455)Online publication date: 26-Jun-2024
https://dl.acm.org/doi/10.1145/3652037.3663935
Show More Cited By

Index Terms

Pothole Detection Based on Disparity Transformation and Road Surface Modeling
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding
  2. Computer graphics
    1. Shape modeling

Index terms have been assigned to the content through auto-classification.

Recommendations

An optimized deep belief network based pothole detection model for asphalt road
High-Performance Computing

The poor quality of asphalt roads has a significant impact on driver safety, damages the mechanical structure of vehicles, increases fuel consumption, annoys passengers and is sometimes also responsible for accidents. Further, the poor quality of the ...
Pothole Detection and Warning System: Infrastructure Support and System Design
ICECT '09: Proceedings of the 2009 International Conference on Electronic Computer Technology

a Many on-going projects in the field of vehicular networks are working in the direction of providing driver with relevant information about roads and traffic movements. In this paper, we propose a novelWi-Fi based architecture for Pothole Detection and ...
A deep learning approach to automatic road surface monitoring and pothole detection
Abstract
Anomalies in road surface not only impact road quality but also affect driver safety, mechanic structure of the vehicles, and fuel consumption. Several approaches have been proposed to automatic monitoring of the road surface condition in order to ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Image Processing

IEEE Transactions on Image Processing Volume 29, Issue

2020

3918 pages

ISSN:1057-7149

Issue’s Table of Contents

1057-7149 © 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 January 2020

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

32
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Misra MSharma RTiwari SShankar A(2024)An optimized deep belief network based pothole detection model for asphalt roadIntelligent Decision Technologies10.3233/IDT-24012718:4(3041-3055)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.3233/IDT-240127
Wei CTan HZhang JFeng MLiu ZWang JZhang P(2024)Apple image recognition model based on optimized sYOLOv5 network with small object detectionProceedings of the 2024 3rd International Conference on Cryptography, Network Security and Communication Technology10.1145/3673277.3673371(548-552)Online publication date: 19-Jan-2024
https://dl.acm.org/doi/10.1145/3673277.3673371
Katsamenis IKopsiaftis GVoulodimos ARallis IGeorgoulas IZafeiropoulos CDoulamis A(2024)A Deep Learning Framework for Segmentation of Road Defects Using ResUNet-aProceedings of the 17th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3652037.3663935(449-455)Online publication date: 26-Jun-2024
https://dl.acm.org/doi/10.1145/3652037.3663935
Jeffreys ZKumar KXie ZBae WAlkobaisi SNarayanappa SHong JPark J(2024)PotholeVision: An Automated Pothole Detection and Reporting System using Computer VisionProceedings of the 39th ACM/SIGAPP Symposium on Applied Computing10.1145/3605098.3636118(695-697)Online publication date: 8-Apr-2024
https://dl.acm.org/doi/10.1145/3605098.3636118
Ma NFan RXie L(2024)UP-CrackNet: Unsupervised Pixel-Wise Road Crack Detection via Adversarial Image RestorationIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.339803725:10(13926-13936)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3398037
Yu JJiang JFichera SPaoletti PLayzell LMehta DLuo S(2024)Road Surface Defect Detection—From Image-Based to Non-Image-Based: A SurveyIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.338283725:9(10581-10603)Online publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3382837
Shon HChoi SHuh K(2024)Real-Time Terrain Condition Detection for Off-Road Driving Based on TransformerIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.336847625:9(11726-11738)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3368476
Li LLiu JXing JLiu ZLin KDu B(2024)Road Pothole Detection Based on Crowdsourced Data and Extended Mask R-CNNIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.336072525:9(12504-12516)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3360725
Dong JWang NFang HShen YLi BDi DZhai K(2024)CBAM-Optimized Automatic Segmentation and Reconstruction System for Monocular Images With Asphalt Pavement PotholesIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.335325725:8(10313-10330)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3353257
Guo SLi JFeng YZhou DZhang DChen CSu SZhu XChen QFan R(2024)UDTIRI: An Online Open-Source Intelligent Road Inspection Benchmark SuiteIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.335120925:8(9920-9931)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TITS.2024.3351209
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents