Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

SpringerLink
Log in

Intelligent vision-based online inspection system of screw-fastening operations in light-gauge steel frame manufacturing

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Offsite construction shifts construction work to the factory environment, enabling automation of traditionally manual operations. While this approach has proven to be advantageous for the construction industry in Canada, changes in the process of manufacturing panelized walls have brought new challenges and opportunities. With the aim of providing alternatives to current manual quality control activities, an automated vision-based online inspection system for screw-fastening operations in light-gauge steel frame manufacturing is proposed in this paper. Targeting two common defects in the framing process, squareness and the quality of the screw-fastening operation itself, real-time quality assessment of the frame is achieved. The proposed system uses novel machine vision algorithms to generate the inspection data: a squareness estimation algorithm is developed, based on edge detection and the Hough transform, that can determine the squareness of a steel connection with a margin of error less than two degrees, whereas a R-CNN is designed based on a pre-trained ResNet-50 for detection and classification of screw-fastening operations, achieving an overall accuracy of 91.67%. All the inspection results are made accessible through a Python-based interface in order to direct operators towards either identifying potential quality issues or confirming the quality of the product leaving the station. The inspection system is validated over a real frame in a semi-automated machine environment. In future work, the inspection system will be deployed in continuous production setups in order to further test its accuracy and applicability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Liu H, Benjamin H, Christoph S, Gurjeet S, Al-Hussein M (2017) Optimizing multi-wall panel configuration for panelized construction using BIM. The 9th International Structural Engineering and Construction Conference (ISEC), Valencia, pp 1–6. https://doi.org/10.14455/isec.res.2017.15

  2. Bock T (2015) The future of construction automation: technological disruption and the upcoming ubiquity of robotics. Autom Constr 59:113–121. https://doi.org/10.1016/j.autcon.2015.07.022

    Article  Google Scholar 

  3. Kamali M, Hewage K (2016) Life cycle performance of modular buildings: a critical review. Renew Sust Energ Rev 62:1171–1183. https://doi.org/10.1016/j.rser.2016.05.031

    Article  Google Scholar 

  4. Leitão P, Colombo AW, Karnouskos S (2016) Industrial automation based on cyber-physical systems technologies: prototype implementations and challenges. Comput Ind 81:11–25. https://doi.org/10.1016/j.compind.2015.08.004

    Article  Google Scholar 

  5. Malik N, Ahmad R, Al-Hussein M (2019) Generation of safe tool-paths for automatic manufacturing of light gauge steel panels in residential construction. Autom Constr 98:46–60. https://doi.org/10.1016/j.autcon.2018.11.023

    Article  Google Scholar 

  6. Di Leo G, Liguori C, Pietrosanto A, Sommella P (2016) A vision system for the online quality monitoring of industrial manufacturing. Opt Lasers Eng 89:162–168. https://doi.org/10.1016/j.optlaseng.2016.05.007

    Article  Google Scholar 

  7. Semeniuta O, Dransfeld S, Martinsen K, Falkman P (2018) Towards increased intelligence and automatic improvement in industrial vision systems. Procedia CIRP 67:256–261. https://doi.org/10.1016/j.procir.2017.12.209

    Article  Google Scholar 

  8. Kang HS, Lee JY, Choi S, Kim H, Park JH, Son JY, Kim BH, Noh SD (2016) Smart manufacturing: past research, present findings, and future directions. Int J Precis Eng Manuf Green Technol 3:111–128. https://doi.org/10.1007/s40684-016-0015-5

    Article  Google Scholar 

  9. Martinez P, Ahmad R, Al-Hussein M (2019) A vision-based system for pre-inspection of steel frame manufacturing. Autom Constr 97:151–163. https://doi.org/10.1016/j.autcon.2018.10.021

    Article  Google Scholar 

  10. Kusiak A (2018) Smart manufacturing. Int J Prod Res 56:508–517. https://doi.org/10.1080/00207543.2017.1351644

    Article  Google Scholar 

  11. Lasi H, Fettke P, Kemper H-G, Feld T, Hoffmann M (2014) Industry 4.0. Bus Inf Syst Eng 6:239–242. https://doi.org/10.1007/s12599-014-0334-4

    Article  Google Scholar 

  12. Wang S, Wan J, Zhang D, Li D, Zhang C (2016) Towards smart factory for industry 4.0: a self-organized multi-agent system with big data based feedback and coordination. Comput Netw 101:158–168. https://doi.org/10.1016/j.comnet.2015.12.017

    Article  Google Scholar 

  13. Boton C, Forgues D (2017) Construction industrialization and it integration: how tall wood buildings can show the right path towards construction 4.0. Modular and Offsite Construction (MOC) Summit Proceedings. https://doi.org/10.29173/mocs64

  14. Craveiro F, Duarte JP, Bartolo H, Bartolo PJ (2019) Additive manufacturing as an enabling technology for digital construction: a perspective on Construction 4.0. Autom Constr 103:251–267. https://doi.org/10.1016/j.autcon.2019.03.011

    Article  Google Scholar 

  15. Oesterreich TD, Teuteberg F (2016) Understanding the implications of digitisation and automation in the context of Industry 4.0: a triangulation approach and elements of a research agenda for the construction industry. Comput Ind 83:121–139. https://doi.org/10.1016/j.compind.2016.09.006

    Article  Google Scholar 

  16. García de Soto B, Agustí-Juan I, Joss S, Hunhevicz J. Implications of Construction 4.0 to the workforce and organizational structures. Int J Constr Manag 2019:1–13. https://doi.org/10.1080/15623599.2019.1616414

  17. Dallasega P, Rauch E, Linder C (2018) Industry 4.0 as an enabler of proximity for construction supply chains: a systematic literature review. Comput Ind 99:205–225. https://doi.org/10.1016/j.compind.2018.03.039

    Article  Google Scholar 

  18. Aslan F, Langlois L, Mangin P, Balan T (2018) Identification of drilling parameters during the flow drill screw driving process. Key Eng Mater 767:465–471. https://doi.org/10.4028/www.scientific.net/KEM.767.465

    Article  Google Scholar 

  19. Psarommatis F, May G, Dreyfus P-A, Kiritsis D (2019) Zero defect manufacturing: state-of-the-art review, shortcomings and future directions in research. Int J Prod Res 58:1–17. https://doi.org/10.1080/00207543.2019.1605228

    Article  Google Scholar 

  20. Lebedynska Y, Berger U (2012) Intelligent knowledge-based system for the automated screwing process control. Proceedings of the 11th WSEAS international conference on Artificial Intelligence, Knowledge Engineering and Data Bases, p 175–80

  21. Chumakov R (2008) Optimal control of screwing speed in assembly with thread-forming screws. Int J Adv Manuf Technol 36:395–400. https://doi.org/10.1007/s00170-006-0839-1

    Article  Google Scholar 

  22. Jia Z, Bhatia A, Aronson RM, Bourne D, Mason MT (2019) A survey of automated threaded fastening. IEEE Trans Autom Sci Eng 16:298–310. https://doi.org/10.1109/TASE.2018.2835382

    Article  Google Scholar 

  23. Kosarev VA, Sugrobova ND (2014) Determination of the quality of an internal screw thread in treatment by means of plastic deformation. Meas Tech 56:1257–1260. https://doi.org/10.1007/s11018-014-0363-x

    Article  Google Scholar 

  24. Gadelmawla ES (2017) Computer vision algorithms for measurement and inspection of external screw threads. Measurement 100:36–49. https://doi.org/10.1016/j.measurement.2016.12.034

    Article  Google Scholar 

  25. Hunsicker RJ, Patten J, Ledford A, Ferman C, Allen M, Ellis C (1994) Automatic vision inspection and measurement system for external screw threads. J Manuf Syst 13:370–384. https://doi.org/10.1016/0278-6125(94)P2586-4

    Article  Google Scholar 

  26. Liu Y, Li S, Wang J, Zeng H, Lu J (2015) A computer vision-based assistant system for the assembly of narrow cabin products. Int J Adv Manuf Technol 76:281–293. https://doi.org/10.1007/s00170-014-6274-9

    Article  Google Scholar 

  27. Martinez P, Al-Hussein M, Ahmad R (2019) A scientometric analysis and critical review of computer vision applications for construction. Autom Constr 107:102947. https://doi.org/10.1016/j.autcon.2019.102947

    Article  Google Scholar 

  28. Hamledari H, McCabe B, Davari S (2017) Automated computer vision-based detection of components of under-construction indoor partitions. Autom Constr 74:78–94. https://doi.org/10.1016/j.autcon.2016.11.009

    Article  Google Scholar 

  29. Martinez P, Ahmad R, Al-Hussein M (2019) Real-time visual detection and correction of automatic screw operations in dimpled light-gauge steel framing with pre-drilled pilot holes. Procedia Manuf 34:798–803. https://doi.org/10.1016/j.promfg.2019.06.204

    Article  Google Scholar 

  30. Zhang Z (2002) A flexible new technique for camera calibration (technical report). IEEE Trans Pattern Anal Mach Intell 22:1330–1334. https://doi.org/10.1109/34.888718

    Article  Google Scholar 

  31. Huo Y-K, Wei G, Zhang Y-D, Wu L-N (2010) An adaptive threshold for the Canny Operator of edge detection. International conference on image analysis and signal processing. IEEE 2010(p):371–374. https://doi.org/10.1109/IASP.2010.5476095

    Article  Google Scholar 

  32. Weimer D, Scholz-Reiter B, Shpitalni M (2016) Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. CIRP Ann 65:417–420. https://doi.org/10.1016/j.cirp.2016.04.072

    Article  Google Scholar 

  33. Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. 2014 IEEE Conference on Computer Vision and Pattern Recognition, IEEE; 2014, p. 580–587. https://doi.org/10.1109/CVPR.2014.81

  34. Cha Y-J, Choi W, Suh G, Mahmoudkhani S, Büyüköztürk O (2018) Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput Aided Civil Infrastruct Eng 33:731–747. https://doi.org/10.1111/mice.12334

    Article  Google Scholar 

  35. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, p 770–8. https://doi.org/10.1109/CVPR.2016.90

  36. Han D, Liu Q, Fan W (2018) A new image classification method using CNN transfer learning and web data augmentation. Expert Syst Appl 95:43–56. https://doi.org/10.1016/j.eswa.2017.11.028

    Article  Google Scholar 

  37. Wuest T, Weimer D, Irgens C, Thoben K-D (2016) Machine learning in manufacturing: advantages, challenges, and applications. Prod Manuf Res 4:23–45. https://doi.org/10.1080/21693277.2016.1192517

    Article  Google Scholar 

  38. Choudhary AK, Harding JA, Tiwari MK (2009) Data mining in manufacturing: a review based on the kind of knowledge. J Intell Manuf 20:501–521. https://doi.org/10.1007/s10845-008-0145-x

    Article  Google Scholar 

  39. An S, Martinez P, Ahmad R, Al-Hussein M (2019) Ontology-based knowledge modeling for frame assemblies manufacturing. https://doi.org/10.22260/ISARC2019/0095

Download references

Acknowledgements

The authors gratefully acknowledge the support of all personnel involved in the development of the steel framing machine prototype, especially Shi An for his valuable help in setting up the visual system and initial data collection.

Funding

This study was financially supported by the Natural Sciences and Engineering Research Council of Canada (File No. IRCPJ 419145-15).

Author information

Authors and Affiliations

  1. Department of Civil and Environmental Engineering, University of Alberta, Edmonton, Canada

    Pablo Martinez & Mohamed Al-Hussein

  2. Laboratory of Intelligent Manufacturing, Design and Automation (LIMDA), Department of Mechanical Engineering, University of Alberta, Edmonton, Canada

    Rafiq Ahmad

Authors
  1. Pablo Martinez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed Al-Hussein
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rafiq Ahmad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rafiq Ahmad.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Martinez, P., Al-Hussein, M. & Ahmad, R. Intelligent vision-based online inspection system of screw-fastening operations in light-gauge steel frame manufacturing. Int J Adv Manuf Technol 109, 645–657 (2020). https://doi.org/10.1007/s00170-020-05695-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-020-05695-y

Keywords

Search

Navigation