A structure‐oriented loss function for automated semantic segmentation of bridge point clouds
Pages 801 - 816
Abstract
Focusing on learning‐based semantic segmentation (SS) methods for bridge point cloud data (PCD), this study proposes a structure‐oriented concept (SOC) with training focused on the spatial distribution patterns of bridge components, including both the horizontally absolute location of each component and its vertically relative position compared with other components. Then a structure‐oriented loss (SOL) function, which embodies the core of SOC, is defined accordingly, and it is compared to five cutting‐edge loss functions on a collected bridge PCD dataset. In contrast to the limitations of other loss functions, SOL significantly improves the overall evaluation metrics of overall accuracy (6.53%) and mean intersection over union (mean IoU: 8.67%). The IoU of the category “others” is improved by 8.44%, which is very important for automating the time‐consuming denoising process. Furthermore, the demonstrated robustness of SOC and SOL reveal great potential to improve the performance of other SS models.
References
[1]
Adeli, H., & Kim, H. (2009). Wavelet‐based vibration control of smart buildings and bridges. CRC Press. https://doi.org/10.1201/9780367813451
[2]
Adeli, H., & Saleh, A. (1997). Optimal control of adaptive/smart bridge structures. Journal of Structural Engineering, 123(2), 218–226. https://doi.org/10.1061/(ASCE)0733-9445(1997)123:2(218)
[3]
Adeli, H., & Saleh, A. (1999). Control, optimization, and smart structures: High‐performance bridges and buildings of the future. John Wiley. https://www.wiley.com/en‐us/Control%2C+Optimization%2C+and+Smart+Structures%3A+High‐Performance+Bridges+and+Buildings+of+the+Future‐p‐9780471350941
[4]
Alam, K. M., Siddique, N., & Adeli, H. (2020). A dynamic ensemble learning algorithm for neural networks. Neural Computing and Applications, 32(10), 8675–8690. https://doi.org/10.1007/s00521-019-04359-7
[5]
American Society of Civil Engineers (ASCE) . (2021). ASCE's 2021 infrastructure report card. https://infrastructurereportcard.org/cat‐item/bridges‐infrastructure/
[6]
Chen, D., Li, Y., Tao, J., Li, Y., Zhang, S., Shan, X., Wang, T., Qiao, Z., Zhao, R., Fan, X., & Zhou, Z. (2024). Deep learning‐based segmentation model for permeable concrete meso‐structures. Computer‐Aided Civil and Infrastructure Engineering, 39, 3626–3645. https://doi.org/10.1111/mice.13300
[7]
Chern, W. C., Nguyen, T. V., Asari, V. K., & Kim, H. (2023). Impact of loss functions on semantic segmentation in far‐field monitoring. Computer‐Aided Civil and Infrastructure Engineering, 38(3), 372–390. https://doi.org/10.1111/mice.12832
[8]
Chu, H., & Chun, P. J. (2024). Fine‐grained crack segmentation for high‐resolution images via a multiscale cascaded network. Computer‐Aided Civil and Infrastructure Engineering, 39(4), 575–594. https://doi.org/10.1111/mice.13111
[9]
Chu, H., Long, L., Guo, J., Yuan, H., & Deng, L. (2024). Implicit function‐based continuous representation for meticulous segmentation of cracks from high‐resolution images. Computer‐Aided Civil and Infrastructure Engineering, 39, 539–558. https://doi.org/10.1111/mice.13052
[10]
Chun, P. J., & Kikuta, T. (2024). Self‐training with Bayesian neural networks and spatial priors for unsupervised domain adaptation in crack segmentation. Computer‐Aided Civil and Infrastructure Engineering, 39(17), 2642–2661. https://doi.org/10.1111/mice.13315
[11]
Chun, P. J., Yamane, T., & Maemura, Y. (2021). A deep learning‐based image captioning method to automatically generate comprehensive explanations of bridge damage. Computer‐Aided Civil and Infrastructure Engineering, 37(11), 1387–1401. https://doi.org/10.1111/mice.12793
[12]
Deng, L., Yuan, H., Long, L., Chun, P. J., Chen, W., & Chu, H. (2024). Cascade refinement extraction network with active boundary loss for segmentation of concrete cracks from high‐resolution images. Automation in Construction, 162, 105410. https://doi.org/10.1016/j.autcon.2024.105410
[13]
Ghaedi, K., Ibrahim, Z., Adeli, H., & Javanmardi, A. (2017). Invited review: Recent developments in vibration control of building and bridge structures. Journal of Vibroengineering, 19(5), 3564–3580. https://doi.org/10.21595/jve.2017.18900
[14]
Graham, B., Engelcke, M., & Van Der Maaten, L. (2018). 3D semantic segmentation with submanifold sparse convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT (pp. 9224–9232). https://doi.org/10.1109/cvpr.2018.00961
[15]
Gutierrez Soto, M., & Adeli, H. (2019). Semi‐active vibration control of smart isolated highway bridge structures using replicator dynamics. Engineering Structures, 186, 536–552. https://doi.org/10.1016/j.engstruct.2019.02.031
[16]
Han, X., Dong, Z., & Yang, B. (2021). A point‐based deep learning network for semantic segmentation of MLS point clouds. ISPRS Journal of Photogrammetry and Remote Sensing, 175, 199–214. https://doi.org/10.1016/j.isprsjprs.2021.03.001
[17]
Hattori, K., Oki, K., Sugita, A., Sugiyama, T., & Chun, P. J. (2024). Deep learning‐based corrosion inspection of long‐span bridges with BIM integration. Heliyon, 10(15), e35308. https://doi.org/10.1016/j.heliyon.2024.e35308
[18]
Hu, Q., Yang, B., Xie, L., Rosa, S., Guo, Y., Wang, Z., Trigoni, N., & Markham, A. (2020). RandLA‐Net: Efficient semantic segmentation of large‐scale point clouds. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA (pp. 11108–11117). https://doi.org/10.1109/cvpr42600.2020.01112
[19]
Huang, Y., Liu, Y., Liu, F., & Liu, W. (2024). A lightweight feature attention fusion network for pavement crack segmentation. Computer‐Aided Civil and Infrastructure Engineering, 39, 2811–2825. https://doi.org/10.1111/mice.13225
[20]
Hui, L., & Belkin, M. (2020). Evaluation of neural architectures trained with square loss vs cross‐entropy in classification tasks. arXiv preprint arXiv:2006.07322. https://doi.org/10.48550/arXiv.2006.07322
[21]
Jing, Y., Sheil, B., & Acikgoz, S. (2022). Segmentation of large‐scale masonry arch bridge point clouds with a synthetic simulator and the BridgeNet neural network. Automation in Construction, 142, 104459. https://doi.org/10.1016/j.autcon.2022.104459
[22]
Jing, Y., Sheil, B., & Acikgoz, S. (2024). A lightweight Transformer‐based neural network for large‐scale masonry arch bridge point cloud segmentation. Computer‐Aided Civil and Infrastructure Engineering, 39(16), 2427–2438. https://doi.org/10.1111/mice.13201
[23]
Kim, H., & Adeli, H. (2005). Wavelet‐hybrid feedback linear mean squared algorithm for robust control of cable‐stayed bridges. Journal of Bridge Engineering, 10(2), 116–123. https://doi.org/10.1061/(ASCE)1084-0702(2005)10:2(116)
[24]
Kim, H., & Kim, C. (2020). Deep‐learning‐based classification of point clouds for bridge inspection. Remote Sensing, 12(22), 3757. https://doi.org/10.3390/rs12223757
[25]
Kim, H., Yoon, J., & Sim, S. H. (2020). Automated bridge component recognition from point clouds using deep learning. Structural Control and Health Monitoring, 27(9), e2591. https://doi.org/10.1002/stc.2591
[26]
Kunlamai, T., Yamane, T., Suganuma, M., Chun, P. J., & Okatani, T. (2024). Improving visual question answering for bridge inspection by pre‐training with external data of image‐text pairs. Computer‐Aided Civil and Infrastructure Engineering, 39(3), 345–361. https://doi.org/10.1111/mice.13086
[27]
Lamas, D., Justo, A., Soilán, M., Cabaleiro, M., & Riveiro, B. (2023). Instance and semantic segmentation of point clouds of large metallic truss bridges. Automation in Construction, 151, 104865. https://doi.org/10.1016/j.autcon.2023.104865
[28]
Landrieu, L., & Boussaha, M. (2019). Point cloud oversegmentation with graph‐structured deep metric learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA (pp. 7440–7449). https://doi.org/10.1109/cvpr.2019.00762
[29]
Landrieu, L., & Simonovsky, M. (2018). Large‐scale point cloud semantic segmentation with superpoint graphs. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT (pp. 4558–4567). https://doi.org/10.1109/cvpr.2018.00479
[30]
Lee, J. S., Park, J., & Ryu, Y. M. (2021). Semantic segmentation of bridge components based on hierarchical point cloud model. Automation in Construction, 130, 103847. https://doi.org/10.1016/j.autcon.2021.103847
[31]
Liang, H., Yeoh, J. K. W., & Chua, D. K. H. (2024). Material augmented semantic segmentation of point clouds for building elements. Computer‐Aided Civil and Infrastructure Engineering, 39(15), 2312–2329. https://doi.org/10.1111/mice.13198
[32]
Liang, Z., Yang, M., Li, H., & Wang, C. (2020). 3D instance embedding learning with a structure‐aware loss function for point cloud segmentation. IEEE Robotics and Automation Letters, 5(3), 4915–4922. https://doi.org/10.1109/lra.2020.3004802
[33]
Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2017). Focal loss for dense object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Venice, Italy (pp. 2980–2988). https://doi.org/10.1109/iccv.2017.324
[34]
Lin, Y. C., & Habib, A. (2022). Semantic segmentation of bridge components and road infrastructure from mobile LiDAR data. ISPRS Open Journal of Photogrammetry and Remote Sensing, 6, 100023. https://doi.org/10.1016/j.ophoto.2022.100023
[35]
Liu, K., Gao, Z., Lin, F., & Chen, B. M. (2020). FG‐Net: Fast large‐scale LiDAR point clouds understanding network leveraging correlated feature mining and geometric‐aware modelling. 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi'an, China (pp. 12896–12902). https://doi.org/10.1109/icra48506.2021.9561496
[36]
Lu, R., & Brilakis, I. (2019). Digital twinning of existing reinforced concrete bridges from labelled point clusters. Automation in Construction, 105, 102837. https://doi.org/10.1016/j.autcon.2019.102837
[37]
Lu, R., Brilakis, I., & Middleton, C. R. (2019). Detection of structural components in point clouds of existing RC bridges. Computer‐Aided Civil and Infrastructure Engineering, 34(3), 191–212. https://doi.org/10.1111/mice.12407
[38]
Milioto, A., Vizzo, I., Behley, J., & Stachniss, C. (2019). Rangenet++: Fast and accurate lidar semantic segmentation. 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China (pp. 4213–4220). https://doi.org/10.1109/iros40897.2019.8967762
[39]
Milletari, F., Navab, N., & Ahmadi, S. A. (2016). V‐Net: Fully convolutional neural networks for volumetric medical image segmentation. In B. Chen (Ed.), 2016 fourth international conference on 3D vision (3DV) (pp. 565–571). IEEE. https://doi.org/10.1109/3dv.2016.79
[40]
Ministry of Land, Infrastructure, Transport and Tourism (MLIT) . (2024). Road maintenance in Japan: Problems and solutions. https://www.mlit.go.jp/road/road_e/s3_maintenance.html
[41]
Noichl, F., Collins, F. C., Braun, A., & Borrmann, A. (2024). Enhancing point cloudsemantic segmentation in the data‐scarce domain of industrial plants throughsynthetic data. Computer‐Aided Civil and Infrastructure Engineering, 39, 1530–1549. https://doi.org/10.1111/mice.13153
[42]
Park, H. S., Lee, H. M., Adeli, H., & Lee, I. (2007). A new approach for health monitoring of structures: Terrestrial laser scanning. Computer‐Aided Civil and Infrastructure Engineering, 22(1), 19–30. https://doi.org/10.1111/j.1467-8667.2006.00466.x
[43]
Park, S. W., Park, H. S., Kim, J. H., & Adeli, H. (2015). 3D displacement measurement model for health monitoring of structures using a motion capture system. Measurement, 59, 352–362. https://doi.org/10.1016/j.measurement.2014.09.063
[44]
Pereira, D. R., Piteri, M. A., Souza, A. N., Papa, J., & Adeli, H. (2020). FEMa: A finite element machine for fast learning. Neural Computing and Applications, 32(10), 6393–6404. https://doi.org/10.1007/s00521-019-04146-4
[45]
Phan, T. H., & Yamamoto, K. (2020). Resolving class imbalance in object detection with weighted cross entropy losses. arXiv preprint arXiv:2006.01413. https://doi.org/10.48550/arXiv.2006.01413
[46]
Qi, C. R., Su, H., Mo, K., & Guibas, L. J. (2017). Pointnet: Deep learning on point sets for 3d classification and segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI (pp. 652–660). https://doi.org/10.1109/cvpr.2017.16
[47]
Qi, C. R., Yi, L., Su, H., & Guibas, L. J. (2017). PointNet++: Deep hierarchical feature learning on point sets in a metric space. Advances in Neural Information Processing Systems, 30, Long Beach, CA.
[48]
Qin, S., Qi, T., Deng, T., & Huang, X. (2024). Image segmentation using Vision Transformer for tunnel defect assessment. Computer‐Aided Civil and Infrastructure Engineering, 39, 3243–3268. https://doi.org/10.1111/mice.13181
[49]
Rafiei, M. H., & Adeli, H. (2017). A new neural dynamic classification algorithm. IEEE Transactions on Neural Networks and Learning Systems, 28(12), 3074–3083. https://doi.org/10.1109/tnnls.2017.2682102
[50]
Rafiei, M. H., Gauthier, L., Adeli, H., & Takabi, D. (2024a). Self‐supervised learning for electroencephalography. IEEE Transactions on Neural Networks and Learning Systems, 35(2), 1457–1471. https://doi.org/10.1109/tnnls.2022.3190448
[51]
Rafiei, M. H., Gauthier, L. V., Adeli, H., & Takabi, D. (2024b). Self‐supervised learning for near‐wild cognitive workload estimation. Journal of Medical Systems, 48(1), 107. https://doi.org/10.1007/s10916-024-02122-7
[52]
Riveiro, B., DeJong, M. J., & Conde, B. (2016). Automated processing of large point clouds for structural health monitoring of masonry arch bridges. Automation in Construction, 72, 258–268. https://doi.org/10.1016/j.autcon.2016.02.009
[53]
Romero‐Jarén, R., & Arranz, J. J. (2021). Automatic segmentation and classification of BIM elements from point clouds. Automation in Construction, 124, 103576. https://doi.org/10.1016/j.autcon.2021.103576
[54]
Sun, L., Yang, Y., Zhou, G., Chen, A., Zhang, Y., Cai, W., & Li, L. (2024). An integration‐competition network for bridge crack segmentation under complex scenes. Computer‐Aided Civil and Infrastructure Engineering, 39, 617–634. https://doi.org/10.1111/mice.13113
[55]
Wang, N., & Adeli, H. (2015). Self‐constructing wavelet neural network algorithm for nonlinear control of large structures. Engineering Applications of Artificial Intelligence, 41, 249–258. https://doi.org/10.1016/j.engappai.2015.01.018
[56]
Wang, Q., Ma, Y., Zhao, K., & Tian, Y. (2020). A comprehensive survey of loss functions in machine learning. Annals of Data Science, 9, 187–212. https://doi.org/10.1007/s40745-020-00253-5
[57]
Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., & Solomon, J. M. (2019). Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics (TOG), 38(5), 1–12. https://doi.org/10.1145/3326362
[58]
Wen, Y., Zhang, K., Li, Z., & Qiao, Y. (2016). A discriminative feature learning approach for deep face recognition. In B. Leibe, J. Matas, N. Sebe, & M. Welling (Eds.), Lecture notes in computer science: Vol. 9911. Computer vision—ECCV 2016 (pp. 499–515). Springer. https://doi.org/10.1007/978-3-319-46478-7_31
[59]
Xia, T., Yang, J., & Chen, L. (2022). Automated semantic segmentation of bridge point cloud based on local descriptor and machine learning. Automation in Construction, 133, 103992. https://doi.org/10.1016/j.autcon.2021.103992
[60]
Xiao, J. L., Fan, J. S., Liu, Y. F., Li, B. L., & Nie, J. G. (2024). Region of interest (ROI) extraction and crack detection for UAV‐based bridge inspection using point cloud segmentation and 3D‐to‐2D projection. Automation in Construction, 158, 105226. https://doi.org/10.1016/j.autcon.2023.105226
[61]
Yang, X., del Rey Castillo, E., Zou, Y., & Wotherspoon, L. (2023). Semantic segmentation of bridge point clouds with a synthetic data augmentation strategy and graph‐structured deep metric learning. Automation in Construction, 150, 104838. https://doi.org/10.1016/j.autcon.2023.104838
[62]
Yang, X., del Rey Castillo, E., Zou, Y., Wotherspoon, L., & Tan, Y. (2022). Automated semantic segmentation of bridge components from large‐scale point clouds using a weighted superpoint graph. Automation in Construction, 142, 104519. https://doi.org/10.1016/j.autcon.2022.104519
[63]
Yu, G., & Adeli, H. (1993). Object‐oriented finite element analysis using EER model. Journal of Structural Engineering, 119(9), 2763–2781. https://doi.org/10.1061/(asce)0733-9445(1993)119:9(2763)
[64]
Yu, J., Jiang, Y., Wang, Z., Cao, Z., & Huang, T. (2016). Unitbox: An advanced object detection network. In A. Hanjalic, C. Snoek, & M. Worring (Eds.), Proceedings of the 24th ACM international conference on multimedia (pp. 516–520). ACM. https://doi.org/10.1145/2964284.2967274
[65]
Zhu, B., Jiang, Z., Zhou, X., Li, Z., & Yu, G. (2019). Class‐balanced grouping and sampling for point cloud 3d object detection. arXiv preprint arXiv:1908.09492. https://doi.org/10.48550/arXiv.1908.09492
Index Terms
- A structure‐oriented loss function for automated semantic segmentation of bridge point clouds
Index terms have been assigned to the content through auto-classification.
Recommendations
Boundary Loss for Remote Sensing Imagery Semantic Segmentation
Advances in Neural Networks – ISNN 2019AbstractIn response to the growing importance of geospatial data, its analysis including semantic segmentation becomes an increasingly popular task in computer vision today. Convolutional neural networks are powerful visual models that yield hierarchies ...
The effect of loss function on conditional generative adversarial networks
AbstractConditional Generative Adversarial Network (cGAN) is a general purpose approach for many image-to-image translation tasks, which aims to translate images from one form to another resulting in high-quality translated images. In this ...
AutoLoss: Automated Loss Function Search in Recommendations
KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data MiningDesigning an effective loss function plays a crucial role in training deep recommender systems. Most existing works often leverage a predefined and fixed loss function that could lead to suboptimal recommendation quality and training efficiency. Some ...
Comments
Information & Contributors
Information
Published In
© 2025 The Author(s). Computer‐Aided Civil and Infrastructure Engineering published by Wiley Periodicals LLC on behalf of Editor.
This is an open access article under the terms of the Creative Commons Attribution‐NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.
Publisher
John Wiley & Sons, Inc.
United States
Publication History
Published: 13 February 2025
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 08 Mar 2025