research-article

Computer vision‐based recognition of 3D relationship between construction entities for monitoring struck‐by accidents

Authors:

Heng LiAuthors Info & Claims

Computer‐Aided Civil and Infrastructure Engineering, Volume 35, Issue 9

Pages 1023 - 1038

https://doi.org/10.1111/mice.12536

Published: 24 August 2020 Publication History

Abstract

Struck‐by accidents often cause serious injuries in construction. Monitoring of the struck‐by hazards in terms of spatial relationship between a worker and a heavy vehicle is crucial to prevent such accidents. The computer vision‐based technique has been put forward for monitoring the struck‐by hazards but there exists shortages such as spatial relationship distortion due to two‐dimensional (2D) image pixels‐based estimation and self‐occlusion of heavy vehicles. This study is aimed to address these problems, including the detection of workers and heavy vehicles, three‐dimensional (3D) bounding box reconstruction for the detected objects, depth and range estimation in the monocular 2D vision, and 3D spatial relationship recognition. A series of experiments were conducted to evaluate the proposed method. The proposed method is expected to estimate 3D spatial relationship between construction worker and heavy vehicle in a real‐time and view‐invariant manner, thus enhancing struck‐by hazards monitoring at the construction site.

References

[1]

Andreas, G., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The Kitti Vision Benchmark Suite. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI.

[2]

Arsalan, M., Anguelov, D., Flynn, J., & Košecká, J. (2017). 3D bounding box estimation using deep learning and geometry. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 5632–5640). Honolulu, HI.

[3]

Bang, S., Park, S., Kim, H., & Kim, H. (2019). Encoder–decoder network for pixel‐level road crack detection in black‐box images. Computer‐Aided Civil and Infrastructure Engineering, 34(8), 713–727.

Digital Library

[4]

The Center for Construction Research and Training (CPWR). (2017). Struck‐by injuries and prevention in the construction industry. Silver Spring, MD: Author.

[5]

Cha, Y.‐J., Choi, W., & Büyüköztürk, O. (2017). Deep learning‐based crack damage detection using convolutional neural networks. Computer‐Aided Civil and Infrastructure Engineering, 32(5), 361–378.

Digital Library

[6]

Cha, Y.‐J., Choi, W., Suh, G., Mahmoudkhani, S., & Büyüköztürk, O. (2018). Autonomous structural visual inspection using region‐based deep learning for detecting multiple damage types. Computer‐Aided Civil and Infrastructure Engineering, 33(9), 731–747.

Digital Library

[7]

Chen, X., Ma, H., Wan, J., Li, B., & Xia, T. (2017). Multi‐view 3D object detection network for autonomous driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI.

[8]

Choe, S., Leite, F., Seedah, D., & Caldas, C. (2013). Application of sensing technology in the prevention of backing accidents in construction work zones. Proceedings of the ASCE International Workshop on Computing in Civil Engineering, Los Angeles, CA.

[9]

Eigen, D., & Fergus, R. (2015). Predicting depth, surface normals and semantic labels with a common multi‐scale convolutional architecture. Proceedings of the IEEE International Conference on Computer Vision (ICCV) (pp. 2650–2658). Washington, DC.

[10]

Fang, D., Zhao, C., & Zhang, M. (2016). A cognitive model of construction workers' unsafe behaviors. Journal of Construction Engineering and Management, 142(9), 04016039.

[11]

Fang, Q., Li, H., Luo, X., Ding, L., Luo, H., Rose, T. M., & An, W. (2018). Detecting non‐hardhat‐use by a deep learning method from far‐field surveillance videos. Automation in Construction, 85, 1–9.

[12]

Forsyth, D., & Ponce, J. (2003). Computer vision: A modern approach. Upper Saddle River, NJ: Prentice‐Hall.

Digital Library

[13]

Golovina, O., Teizer, J., & Pradhananga, N. (2016). Heatmap generation for predictive safety planning: Preventing struck‐by and near miss interactions between workers‐on‐foot and construction equipment. Automation in Construction, 71, 99–115.

[14]

Hartley, R., & Zisserman, A. (2004). Multiple view geometry in computer vision. Cambridge: Cambridge University Press.

[15]

He, K., Zhang, X., Ren, S., & Sun, J. (2015). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA.

[16]

Hinze, J., & Godfrey, R. (2003). An evaluation of safety performance measures for construction projects. Journal of Construction Research, 4(1), 5–15.

[17]

Kim, D., Liu, M., Lee, S., & Kamat, V. R. (2019). Remote proximity monitoring between mobile construction resources using camera‐mounted UAVs. Automation in Construction, 99, 168–182.

[18]

Kim, H., Kim, K., & Kim, H. (2016). Vision‐based object‐centric safety assessment using fuzzy inference: Monitoring struck‐by accidents with moving objects. Journal of Computing in Civil Engineering, 30(4), 04015075.

[19]

Kim, K., Kim, H., & Kim, H. (2017). Image‐based construction hazard avoidance system using augmented reality in wearable device. Automation in Construction, 83, 390–403.

[20]

Ku, J., Mozifian, M., Lee, J., Harakeh, A., & Waslander, S. (2018). Joint 3D proposal generation and object detection from view aggregation. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 1–8). Madrid, Spain.

[21]

Li, R., Yuan, Y., Zhang, W., & Yuan, Y. (2018). Unified vision‐based methodology for simultaneous concrete defect detection and geolocalization. Computer‐Aided Civil and Infrastructure Engineering, 33(7), 527–544.

Digital Library

[22]

Li, S., Zhao, X., & Zhou, G. (2019). Automatic pixel‐level multiple damage detection of concrete structure using fully convolutional network. Computer‐Aided Civil and Infrastructure Engineering, 34(7), 616–634.

Digital Library

[23]

Liang, X. (2018). Image‐based post‐disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization. Computer‐Aided Civil and Infrastructure Engineering, 34(5), 415–430.

[24]

Liu, C.‐W., & Kang, S.‐C. (2014). A video‐enabled dynamic site planner. 2014 International Conference on Computing in Civil and Building Engineering (pp. 1562–1569). Orlando, FL.

[25]

Luo, X., Li, H., Cao, D., Dai, F., Seo, J., & Lee, S. (2018). Recognizing diverse construction activities in site images via relevance networks of construction‐related objects detected by convolutional neural networks. Journal of Computing in Civil Engineering, 32(3), 04018012.

[26]

Luo, X., Li, H., Yang, X., Yu, Y., & Cao, D. (2019). Capturing and understanding workers’ activities in far‐field surveillance videos with deep action recognition and Bayesian nonparametric learning. Computer‐Aided Civil and Infrastructure Engineering, 34(4), 333–351.

Digital Library

[27]

Maeda, H., Sekimoto, Y., Seto, T., Kashiyama, T., & Omata, H. (2018). Road damage detection and classification using deep neural networks with smartphone images. Computer‐Aided Civil and Infrastructure Engineering, 33(12), 1127–1141.

Digital Library

[28]

Park, J., Marks, E., Cho, Y., & Suryanto, W. (2016). Performance test of wireless technologies for personnel and equipment proximity sensing in work zones. Journal of Construction Engineering and Management, 142(1), 04015049.

[29]

Pramadihanto, D., Alfarouq, A., Waskitho, S. A., & Sukaridhoto, S. (2017). Merging of depth image between stereo camera and structure sensor on robot “Flow” vision. International Journal on Advanced Science, Engineering and Information Technology, 7(3), 1014–1025.

[30]

Rafiei, M. H., & Adeli, H. (2017). A novel machine learning‐based algorithm to detect damage in high‐rise building structures. The Structural Design of Tall and Special Buildings, 26(18), e1400.

[31]

Rafiei, M. H., & Adeli, H. (2018a). Novel machine‐learning model for estimating construction costs considering economic variables and indexes. Journal of Construction Engineering and Management, 144(12), 04018106.

[32]

Rafiei, M. H., & Adeli, H. (2018b). A novel unsupervised deep learning model for global and local health condition assessment of structures. Engineering Structures, 156, 598–607.

[33]

Rafiei, M. H., Khushefati, W. H., Demirboga, R., & Adeli, H. (2017). Supervised deep restricted Boltzmann machine for estimation of concrete. ACI Materials Journal, 114(2), 237–244.

[34]

Ray, S. J., & Teizer, J. (2012). Real‐time construction worker posture analysis for ergonomics training. Advanced Engineering and Informormatics, 26(2), 439–455.

Digital Library

[35]

Ren, S., He, K. M., Girshick, R., & Sun, J. (2015). Faster R‐CNN: Towards real‐time object detection with region proposal networks. Advances in Neural Information Processing Systems (pp. 91–99). Montreal, Canada.

[36]

Rijsbergen, C. J. V. (1979). Information retrieval (2nd ed.). London: Butterworth.

Digital Library

[37]

Roy, A., & Todorovic, S. (2016). Monocular depth estimation using neural regression forest. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 5506–5514). Las Vegas, NV.

[38]

Saxena, A., Chung, S. H., & Ng, A. Y. (2006). Learning depth from single monocular images. Proceedings of the Advances in Neural Information Processing Systems (pp. 1161–1168). Vancouver, Canada.

[39]

Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., … Blake, A. (2011). Real‐time human pose recognition in parts from single depth images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1297–1304). Providence, RI.

[40]

Taneja, S., Akinci, B., Garrett, J. H., Soibelman, L., Ergen, E., Pradhan, A., & Anil, E. B. (2011). Sensing and field data capture for construction and facility operations. Journal of Construction Engineering and Management, 137(10), 870–881.

[41]

Tateno, K., Tombari, F., Laina, I., & Navab, N. (2017). CNN‐slam: Real‐time dense monocular slam with learned depth prediction. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI.

[42]

Teizer, J., Allread, B. S., Fullerton, C. E., & Hinze, J. (2010). Autonomous proactive real‐time construction worker and equipment operator proximity safety alert system. Automation in Construction, 19(5), 630–640.

[43]

Teizer, J., & Cheng, T. (2015). Proximity hazard indicator for workers‐on‐foot near miss interactions with construction equipment and geo‐referenced hazard areas. Automation in Construction, 60, 58–73.

[44]

Torres, J. F., Galicia, A., Troncoso, A., & Martínez‐Álvarez, F. (2018). A scalable approach based on deep learning for big data time series forecasting. Integrated Computer‐Aided Engineering, 25(4), 335–348.

Digital Library

[45]

The U.S. Bureau of Labor Statistics (BLS). (2011–2015). Census of Fatal Occupational Injuries (CFOI). Washington, DC: Author.

[46]

The U.S. Occupational Safety and Health Administration (OSHA). (2013). OSHA construction Etools. Washington, DC: Author.

[47]

Wang, P., & Bai, X. (2018). Regional parallel structure based CNN for thermal infrared face identification. Integrated Computer‐Aided Engineering, 25(3), 247–260.

[48]

Weerasinghe, I. P. T., Ruwanpura, J. Y., Boyd, J. E., & Habib, A. F. (2012). Application of Microsoft Kinect sensor for tracking construction workers. Construction Research Congress 2012: Construction Challenges in a Flat World (pp. 858–867). West Lafayette, IN.

[49]

Wu, W., Yang, H., Li, Q., & Chew, D. (2013). An integrated information management model for proactive prevention of struck‐by‐falling‐object accidents on construction sites. Automation in Construction, 34, 67–74.

[50]

Xue, Y., & Li, Y. (2018). A fast detection method via region‐based fully convolutional neural networks for shield tunnel lining defects. Computer‐Aided Civil and Infrastructure Engineering, 33(8), 638–654.

Digital Library

[51]

Yan, X., Li, H., Wang, C., Seo, J., Zhang, H., & Wang, H. (2017). Development of ergonomic posture recognition technique based on 2D ordinary camera for construction hazard prevention through view‐invariant features in 2D skeleton motion. Advanced Engineering Informatics, 34, 152–163.

Digital Library

[52]

Yang, X., Li, H., Yu, Y., Luo, X., Huang, T., & Yang, X. (2018). Automatic pixel‐level crack detection and measurement using fully convolutional network. Computer‐Aided Civil and Infrastructure Engineering, 33(12), 1090–1109.

Digital Library

[53]

Yuan, C., Li, S., & Cai, H. (2017). Vision‐based excavator detection and tracking using hybrid kinematic shapes and key nodes. Journal of Computing in Civil Engineering, 31(1), 04016038.

[54]

Zhu, Z., Park, M.‐W., Koch, C., Soltani, M., Hammad, A., & Davari, K. (2016). Predicting movements of onsite workers and mobile equipment for enhancing construction site safety. Automation in Construction, 68, 95–101.

Cited By

Lashkov IYuan RZhang G(2024)Computing‐efficient video analytics for nighttime traffic sensingComputer-Aided Civil and Infrastructure Engineering10.1111/mice.1329539:22(3392-3411)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1111/mice.13295
Zhang MWang LHan SWang SLi H(2024)Deep learning framework with Local Sparse Transformer for construction worker detection in 3D with LiDARComputer-Aided Civil and Infrastructure Engineering10.1111/mice.1323839:19(2990-3007)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1111/mice.13238
Ding YLuo X(2024)A virtual construction vehicles and workers dataset with three-dimensional annotationsEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.107964133:PAOnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.engappai.2024.107964
Show More Cited By

Index Terms

Computer vision‐based recognition of 3D relationship between construction entities for monitoring struck‐by accidents
1. Applied computing
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics

Index terms have been assigned to the content through auto-classification.

Recommendations

Predicting workers’ inattentiveness to struck-by hazards by monitoring biosignals during a construction task: A virtual reality experiment
Abstract
At construction workplaces, workers should be consistently attentive to approaching and nearby safety hazards. However, workers tend to allocate most of their attentional resources to a work task and often exhibit inattentive behaviors ...
Types and Characteristics of Safety Accidents Induced by Metro Construction
ICIII '09: Proceedings of the 2009 International Conference on Information Management, Innovation Management and Industrial Engineering - Volume 03

Due to the special geological conditions and the surrounding environmental conditions, the urban metro constructions are faced with greater technical difficulty and higher safety risk, which cause many kinds of safety accidents. Taking Beijing metro as ...
A Survey of Vision-Based Traffic Monitoring of Road Intersections

Visual surveillance of dynamic objects, particularly vehicles on the road, has been, over the past decade, an active research topic in computer vision and intelligent transportation systems communities. In the context of traffic monitoring, important ...

Comments

Information & Contributors

Information

Published In

cover image Computer-Aided Civil and Infrastructure Engineering

Computer-Aided Civil and Infrastructure Engineering Volume 35, Issue 9

September 2020

136 pages

ISSN:1093-9687

EISSN:1467-8667

DOI:10.1111/mice.v35.9

Issue’s Table of Contents

© 2020 Computer‐Aided Civil and Infrastructure Engineering.

Publisher

John Wiley & Sons, Inc.

United States

Publication History

Published: 24 August 2020

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lashkov IYuan RZhang G(2024)Computing‐efficient video analytics for nighttime traffic sensingComputer-Aided Civil and Infrastructure Engineering10.1111/mice.1329539:22(3392-3411)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1111/mice.13295
Zhang MWang LHan SWang SLi H(2024)Deep learning framework with Local Sparse Transformer for construction worker detection in 3D with LiDARComputer-Aided Civil and Infrastructure Engineering10.1111/mice.1323839:19(2990-3007)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1111/mice.13238
Ding YLuo X(2024)A virtual construction vehicles and workers dataset with three-dimensional annotationsEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.107964133:PAOnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.engappai.2024.107964
Lu YYou KZhou CChen JWu ZJiang YHuang C(2024)Video surveillance-based multi-task learning with swin transformer for earthwork activity classificationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107814131:COnline publication date: 1-May-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107814
Zhou XLiao P(2023)Brain‐regulated learning for classifying on‐site hazards with small datasetsComputer-Aided Civil and Infrastructure Engineering10.1111/mice.1307839:3(458-472)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1111/mice.13078
Shen JYan WQin SZheng X(2023)A self‐supervised monocular depth estimation model with scale recovery and transfer learning for construction scene analysisComputer-Aided Civil and Infrastructure Engineering10.1111/mice.1293838:9(1142-1161)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1111/mice.12938
Assadzadeh AArashpour MLi HHosseini RElghaish FBaduge S(2023)Excavator 3D pose estimation using deep learning and hybrid datasetsAdvanced Engineering Informatics10.1016/j.aei.2023.10187555:COnline publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.aei.2023.101875
Jung SJeoung JKang HHong T(2021)3D convolutional neural network‐based one‐stage model for real‐time action detection in video of construction equipmentComputer-Aided Civil and Infrastructure Engineering10.1111/mice.1269537:1(126-142)Online publication date: 20-Dec-2021
https://dl.acm.org/doi/10.1111/mice.12695
Bang SHong YKim H(2021)Proactive proximity monitoring with instance segmentation and unmanned aerial vehicle‐acquired video‐frame predictionComputer-Aided Civil and Infrastructure Engineering10.1111/mice.1267236:6(800-816)Online publication date: 13-Apr-2021
https://dl.acm.org/doi/10.1111/mice.12672
Assadzadeh AArashpour MBab‐Hadiashar ANgo TLi H(2021)Automatic far‐field camera calibration for construction scene analysisComputer-Aided Civil and Infrastructure Engineering10.1111/mice.1266036:8(1073-1090)Online publication date: 26-Feb-2021
https://dl.acm.org/doi/10.1111/mice.12660

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents