Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Welding seam profiling techniques based on active vision sensing for intelligent robotic welding

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Intelligent robotic welding involves replicating the role of a manual professional welder to adaptively control the welding process. This is necessary to achieve accurate, fast and high-quality welding process in addition to the challenging factors for humans to operate in the welding environment. Therefore, robotic welding exists since the early days of robotics and it is still an active research area. This is why there have been numerous researches in this area for a very long time. Among various techniques proposed by researchers for the adaptive control of the robotic welding process, vision-based control is the most popular due to its non-invasiveness. Therefore, in this paper, we review, analyse and categorise the proposed vision-based techniques with the aim of covering the different image processing and feature extraction aspect of the techniques. The focus is mainly on the active vision system where various image processing techniques have been utilised in extracting the welding seam features. The challenges and difficulties to extract seam features in active vision system have been highlighted. The trends and new approaches have been indicated in order to provide a comprehensive source for researchers who are planning to carry out research related to the intelligent robot vision techniques for welding automation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Pires JN, Loureiro A, Bölmsjo G (2006) Welding robots: technology, system issues and application. Springer Science & Business Media

  2. Wilson WJ, Hulls CCW, Bell GS (1996) Relative end-effector control using Cartesian position based visual servoing. IEEE Trans Robot Autom 12(5):684–696

    Article  Google Scholar 

  3. Xu D, Tan M, Zhao X, Tu Z (2004) Seam tracking and visual control for robotic arc welding based on structured light stereovision. Int J Autom Comput 1(1):63–75

    Article  Google Scholar 

  4. Xu D, Wang L, Tan M (2004) Image processing and visual control method for arc welding robot. In: Robotics and biomimetics, 2004. ROBIO 2004. IEEE International Conference on (pp. 727–732). IEEE

  5. Xu D, Jiang Z, Wang L, Tan M (2004) Features extraction for structured light image of welding seam with arc and splash disturbance. In: Control, Automation, Robotics and Vision Conference, 2004. ICARCV 2004 8th (Vol. 3, pp. 1559–1563). IEEE

  6. Lanzetta M, Santochi M, Tantussi G (2001) On-line control of robotized Gas Metal Arc Welding. CIRP Ann Manuf Technol 50(1):13–16

    Article  Google Scholar 

  7. Clarke T, Wang X (2000) The control of a robot end-effector using photogrammetry. Int Arch Photogramm Remote Sens 33(B5/1; PART 5), 137–142

  8. Chen XZ, Huang YM, Chen SB (2012) Model analysis and experimental technique on computing accuracy of seam spatial position information based on stereo vision for welding robot. Ind Robot: Int J 39(4):349–356

    Article  Google Scholar 

  9. Cook GE, Andersen K, Fernandez KR, Shepard ME, Wells Jr AM (1987) Electric arc sensing for robot positioning control. IFS(Publications) Ltd., Robotic Welding, 181–216

  10. Park YW, Rhee S (2008) Process modeling and parameter optimization using neural network and genetic algorithms for aluminum laser welding automation. Int J Adv Manuf Technol 37(9–10):1014–1021

    Article  Google Scholar 

  11. Chaki S, Shanmugarajan B, Ghosal S, Padmanabham G (2015) Application of integrated soft computing techniques for optimisation of hybrid CO 2 laser–MIG welding process. Appl Soft Comput 30:365–374

    Article  Google Scholar 

  12. Nele L, Sarno E, Keshari A (2013) Modeling of multiple characteristics of an arc weld joint. Int J Adv Manuf Technol 69(5–8):1331–1341

    Article  Google Scholar 

  13. Fridenfalk M (2003) Development of intelligent robot systems based on sensor control. Lund University

  14. Fang Z, Xu D, Tan M (2013) Vision-based initial weld point positioning using the geometric relationship between two seams. Int J Adv Manuf Technol 66(9–12):1535–1543

    Article  Google Scholar 

  15. Kiddee P, Fang Z, Tan M (2014) Visual recognition of the initial and end points of lap joint for welding robots. In: Information and Automation (ICIA), 2014 I.E. International Conference on (pp. 513-518). IEEE

  16. Kong M, Shi FH, Chen SB, Lin T (2007) Recognition of the initial position of weld based on the corner detection for welding robot in global environment. In: Robotic welding. Springer, Berlin, pp 249–255

    Google Scholar 

  17. Zhu ZY, Lin T, Piao YJ, Chen SB (2005) Recognition of the initial position of weld based on the image pattern match technology for welding robot. Int J Adv Manuf Technol 26(7–8):784–788

    Article  Google Scholar 

  18. Zhou L, Lin T, Chen SB (2006) Autonomous acquisition of seam coordinates for arc welding robot based on visual servoing. J Intell Robot Syst 47(3):239–255

    Article  Google Scholar 

  19. Dinham M, Fang G (2013) Autonomous weld seam identification and localisation using eye-in-hand stereo vision for robotic arc welding. Robot Comput Integr Manuf 29(5):288–301

    Article  Google Scholar 

  20. Micallef K, Fang G, Dinham M (2011) Automatic seam detection and path planning in robotic welding. In: Robotic welding, intelligence and automation. Springer, Berlin, pp 23–32

    Chapter  Google Scholar 

  21. Li L, Fu L, Zhou X, Li X (2007) Image processing of seam tracking system using laser vision. In: Robotic welding, intelligence and automation. Springer, Berlin, pp 319–324

    Chapter  Google Scholar 

  22. Chen XZ, Chen SB (2010) The autonomous detection and guiding of start welding position for arc welding robot. Ind Robot: Int J 37(1):70–78

    Article  Google Scholar 

  23. Wei SC, Wang J, Lin T, Chen SB (2012) Application of image morphology in detecting and extracting the initial welding position. J Shanghai Jiaotong Univ (Sci) 17:323–326

    Article  Google Scholar 

  24. Chen XZ, Chen SB, Lin T (2007) Recognition of macroscopic seam for complex robotic welding environment. In: Robotic welding, intelligence and automation. Springer, Berlin, pp 171–178

    Chapter  Google Scholar 

  25. Dinham M, Fang G (2014) Detection of fillet weld joints using an adaptive line growing algorithm for robotic arc welding. Robot Comput Integr Manuf 30(3):229–243

    Article  Google Scholar 

  26. Chen X, Chen S, Lin T, Lei Y (2006) Practical method to locate the initial weld position using visual technology. Int J Adv Manuf Technol 30(7-8):663–668

    Article  Google Scholar 

  27. Nele L, Sarno E, Keshari A (2013) An image acquisition system for real-time seam tracking. Int J Adv Manuf Technol 69(9–12):2099–2110

    Article  Google Scholar 

  28. Chen SB, Chen XZ, Qiu T, Li JQ (2005) Acquisition of weld seam dimensional position information for arc welding robot based on vision computing. J Intell Robot Syst 43(1):77–97

    Article  Google Scholar 

  29. Shen HY, Wu J, Lin T, Chen SB (2008) Arc welding robot system with seam tracking and weld pool control based on passive vision. Int J Adv Manuf Technol 39(7–8):669–678

    Article  Google Scholar 

  30. Xu Y, Fang G, Chen S, Zou JJ, Ye Z (2014) Real-time image processing for vision-based weld seam tracking in robotic GMAW. Int J Adv Manuf Technol 73(9–12):1413–1425

    Article  Google Scholar 

  31. Gao X, Zhong X, You D, Katayama S (2013) Kalman filtering compensated by radial basis function neural network for seam tracking of laser welding. IEEE Trans Control Syst Technol 21(5):1916–1923

    Article  Google Scholar 

  32. Baskoro AS, Masuda R, Kabutomori M, Suga Y (2009) An application of genetic algorithm for edge detection of molten pool in fixed pipe welding. Int J Adv Manuf Technol 45(11–12):1104–1112

    Article  Google Scholar 

  33. Baskoro AS, Masuda R, Kabutomori M, Suga Y (2008) A comparison of particle swarm optimization and genetic algorithm for edge detection of molten pool in fixed pipe welding. In: SCIS & ISIS (Vol. 2008, No. 0, pp. 1743–1748). Japan Society for Fuzzy Theory and Intelligent Informatics

  34. Baskoro AS, Kabutomori M, Suga Y (2009) Welding penetration control of fixed pipe in TIG welding using fuzzy inference system. J Solid Mech Mater Eng 3(1):38–48

    Article  Google Scholar 

  35. Baskoro AS, Kabutomori M, Suga Y (2008) Automatic welding system of aluminium pipe by monitoring backside image of molten pool using vision sensor. J Solid Mech Mater Eng 2(5):582–592

    Article  Google Scholar 

  36. Ge J, Zhu Z, He D, Chen L (2005) A vision-based algorithm for seam detection in a PAW process for large-diameter stainless steel pipes. Int J Adv Manuf Technol 26(9–10):1006–1011

    Article  Google Scholar 

  37. Gao J, Wu C, Liu X, Xia D (2007) Vision-based weld seam tracking in gas metal arc welding. Front Mater Sci Chin 1(3):268–273

    Article  Google Scholar 

  38. Gao XD, Na SJ (2005) Detection of weld position and seam tracking based on Kalman filtering of weld pool images. J Manuf Syst 24(1):1–12

    Article  Google Scholar 

  39. Shen H, Lin T, Chen S, Li L (2010) Real-time seam tracking technology of welding robot with visual sensing. J Intell Robot Syst 59(3–4):283–298

    Article  MATH  Google Scholar 

  40. Ohshima K, Morita M, Fujii K, Yamamoto M, Kubota T (1992) Sensing and digital control of weld pool in pulsed MIG welding. Trans Jpn Weld Soc 23(1):17–23

    Google Scholar 

  41. Bae KY, Lee TH, Ahn KC (2002) An optical sensing system for seam tracking and weld pool control in gas metal arc welding of steel pipe. J Mater Process Technol 120(1):458–465

    Article  Google Scholar 

  42. Gao X, Ding D, Bai T, Katayama S (2011) Weld-pool image centroid algorithm for seam-tracking vision model in arc-welding process. IET Image Process 5(5):410–419

    Article  Google Scholar 

  43. Gu WP, Xiong ZY, Wan W (2013) Autonomous seam acquisition and tracking system for multi-pass welding based on vision sensor. Int J Adv Manuf Technol 69(1–4):451–460

    Article  Google Scholar 

  44. Iakovou D, Aarts R, Meijer J (2005) Sensor integration for robotic laser welding processes

  45. Forest J, Salvi J, Cabruja E, Pous C. (2004) Laser stripe peak detector for 3D scanners. A FIR filter approach. In: Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on (Vol. 3, pp. 646–649). IEEE

  46. Xu P, Xu G, Tang X, Yao S (2008) A visual seam tracking system for robotic arc welding. Int J Adv Manuf Technol 37(1–2):70–75

    Article  Google Scholar 

  47. Zhang L, Ye Q, Yang W, Jiao J (2014) Weld line detection and tracking via spatial-temporal cascaded hidden Markov models and cross structured light. IEEE Trans Instrum Meas 63(4):742–753

    Article  Google Scholar 

  48. Xiao Z (2011) Research on a trilines laser vision sensor for seam tracking in welding. In: Robotic welding, intelligence and automation. Springer, Berlin, pp 139–144

    Chapter  Google Scholar 

  49. Villán AF, Acevedo RG, Alvarez EA, López AC, García DF, Fernández RU, … & Sánchez JMG (2011) Low-cost system for weld tracking based on artificial vision. IEEE Trans Ind Appl 47(3):1159–1167

  50. Caggiano A, Nele L, Sarno E, Teti R (2014) 3D digital reconfiguration of an automated welding system for a railway manufacturing application. Procedia CIRP 25:39–45

    Article  Google Scholar 

  51. Hang K, Pritschow G (1999) Reducing distortions caused by the welding arc in a laser stripe sensor system for automated seam tracking. In: Industrial Electronics, 1999. ISIE’99. Proceedings of the IEEE International Symposium on (Vol. 2, pp. 919–924). IEEE

  52. Huang W, Kovacevic R (2012) Development of a real-time laser-based machine vision system to monitor and control welding processes. Int J Adv Manuf Technol 63(1–4):235–248

    Article  Google Scholar 

  53. Li Y, Li YF, Wang QL, Xu D, Tan M (2010) Measurement and defect detection of the weld bead based on online vision inspection. IEEE Trans Instrum Meas 59(7):1841–1849

    Article  Google Scholar 

  54. Kim JK, Hong JI, Kim JW, Choi DJ, Rhee SY (2012) Geometrical measurement about welding shape using dual laser vision system. In: Control, Automation and Systems (ICCAS), 2012 12th International Conference on (pp. 195–198). IEEE

  55. Sung K, Lee H, Choi YS, Rhee S (2009) Development of a multiline laser vision sensor for joint tracking in welding. Weld J

  56. Iakovou D (2009) Sensor development and integration for robotized laser welding. University of Twente (Ph.D Thesis)

  57. Shi YH, Wang GR, Li GJ (2007) Adaptive robotic welding system using laser vision sensing for underwater engineering. In: Control and Automation, 2007. ICCA 2007. IEEE International Conference on (pp. 1213–1218). IEEE

  58. Lee JP, Wu QQ, Park MH, Park CK, Kim IS (2014) A study on optimal algorithms to find joint tracking in GMA welding. Int J Eng Sci Innov Technol 3:370–380

    Google Scholar 

  59. Zhang L, Wu C, Zou Y (2009) An on-line visual seam tracking sensor system during laser beam welding. In: Information Technology and Computer Science, 2009. ITCS 2009. International Conference on (Vol. 2, pp. 361–364). IEEE

  60. Villán AF, Acevedo RG, Alvarez EA, López AC, García DF, Fernández RU, … & Sánchez JMG (2011) Low-cost system for weld tracking based on artificial vision. IEEE Trans Ind Appl 47(3), 1159–1167

  61. Huang W, Kovacevic R (2011) A laser-based vision system for weld quality inspection. Sensors 11(1):506–521

    Article  Google Scholar 

  62. Fang Z, Xu D (2009) Image-based visual seam tracking system for fillet joint. In: Robotics and Biomimetics (ROBIO), 2009 I.E. International Conference on (pp. 1230–1235). IEEE

  63. Fang Z, Xu D, Tan M (2010) Visual seam tracking system for butt weld of thin plate. Int J Adv Manuf Technol 49(5–8):519–526

    Article  Google Scholar 

  64. Piccardi M (2004) Background subtraction techniques: a review. In: Systems, man and cybernetics, 2004 I.E. international conference on (Vol. 4, pp. 3099–3104). IEEE

  65. Otsu N (1975) A threshold selection method from gray-level histograms. Automatica 11(285–296):23–27

    Google Scholar 

  66. Toft PA, Sørensen JA (1996) The Radon transform-theory and implementation (Doctoral dissertation, Technical University of Denmark Danmarks Tekniske Universitet, Department of Informatics and Mathematical Modeling, Institut for Informatik og Matematisk Modellering)

  67. Ballard DH (1981) Generalizing the Hough transform to detect arbitrary shapes. Pattern Recogn 13(2):111–122

    Article  MATH  Google Scholar 

  68. Li Y, Wang QL, Li YF, Xu D, Tan M (2008) On-line visual measurement and inspection of weld bead using structured light. In: Instrumentation and Measurement Technology Conference Proceedings, 2008. IMTC 2008. IEEE (pp. 2038–2043). IEEE

  69. Naidu DK, Fisher RB (1991) A comparative analysis of algorithms for determining the peak position of a stripe to sub-pixel accuracy. In: BMVC91, 1991st edn. Springer, London, pp 217–225

    Chapter  Google Scholar 

  70. Usamentiaga R, Molleda J, García DF (2012) Fast and robust laser stripe extraction for 3D reconstruction in industrial environments. Mach Vis Appl 23(1):179–196

    Article  Google Scholar 

  71. Haug K, Pritschow G (1998) Robust laser-stripe sensor for automated weld-seam-tracking in the shipbuilding industry. In: Industrial Electronics Society, 1998. IECON’98. Proceedings of the 24th Annual Conference of the IEEE (Vol. 2, pp. 1236–1241). IEEE

  72. Gong Y, Dai X, Li X (2010). Structured-light based joint recognition using bottom-up and top-down combined visual processing. In: Image Analysis and Signal Processing (IASP), 2010 International Conference on (pp. 507–512). IEEE

  73. Blais F, Rioux M (1986) Real-time numerical peak detector. Signal Process 11(2):145–155

    Article  Google Scholar 

  74. Trucco E, Fisher RB, Fitzgibbon AW, Naidu DK (1998) Calibration, data consistency and model acquisition with laser stripers. Int J Comput Integr Manuf 11(4):293–310

    Article  Google Scholar 

  75. Forest Collado J (2004) New methods for triangulation-based shape acquisition using laser scanners. Universitat de Girona

  76. Khoury JM (1999) U.S. Patent No. 5,931,898. U.S. Patent and Trademark Office, Washington, DC

    Google Scholar 

  77. Kim JW, Bae HS (2005) A study on a vision sensor system for tracking the I-butt weld joints. J Mech Sci Technol 19(10):1856–1863

    Article  Google Scholar 

  78. Fang Z, Xu D, Tan M (2011) A vision-based self-tuning fuzzy controller for fillet weld seam tracking. IEEE/ASME Trans Mechatron 16(3):540–550

    Article  Google Scholar 

  79. Eddy SR (1996) Hidden Markov models. Curr Opin Struct Biol 6(3):361–365

    Article  Google Scholar 

  80. Zhang L, Jiao J, Ye Q, Han Z, Yang W (2012) Robust weld line detection with cross structured light and Hidden Markov Model. In: Mechatronics and Automation (ICMA), 2012 International Conference on (pp. 1411–1416). IEEE

  81. Zhang L, Ke W, Han Z, Jiao J (2013) A cross structured light sensor for weld line detection on wall-climbing robot. In: Mechatronics and Automation (ICMA), 2013 I.E. International Conference on (pp. 1179–1184). IEEE

  82. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 6:679–698

    Article  Google Scholar 

  83. Marr D, Hildreth E (1980) Theory of edge detection. Proc R Soc Lond B Biol Sci 207(1167):187–217

    Article  Google Scholar 

  84. Kim JS, Son YT, Cho HS, Koh KI (1996) A robust visual seam tracking system for robotic arc welding. Mechatronics 6(2):141–163

    Article  Google Scholar 

  85. Kim JS, Son YT, Cho HS, Koh KI (1995) A robust method for vision-based seam tracking in robotic arc welding. In: Intelligent Control, 1995. Proceedings of the 1995 I.E. International Symposium on (pp. 363–368). IEEE

  86. Xiao Y, Zou JJ, Yan H (2001) An adaptive split-and-merge method for binary image contour data compression. Pattern Recogn Lett 22(3):299–307

    Article  MATH  Google Scholar 

  87. Aubert G, Kornprobst P (2006) Mathematical problems in image processing: partial differential equations and the calculus of variations (Vol. 147). Springer Science & Business Media

  88. Sicard P, Levine MD (1989) Joint recognition and tracking for robotic arc welding. IEEE Trans Syst Man Cybern 19(4):714–728

    Article  Google Scholar 

  89. Nan H, Beattie RJ, Davey PG (1988) A rule-based system for interpreting weld seam images. Int J Adv Manuf Technol 3(3):111–121

    Article  Google Scholar 

  90. Pavlidis T, Horowitz SL (1974) Segmentation of plane curves. IEEE Trans Comput 8:860–870

    Article  MathSciNet  MATH  Google Scholar 

  91. Berkan RC, Trubatch S (1997) Fuzzy system design principles. Wiley-IEEE Press

  92. Wu J, Smith JS, Lucas J (1996) Weld bead placement system for multipass welding. IEE Proc: Sci Meas Technol 143(2):85–90

    Google Scholar 

  93. Harris C, Stephens M (1988) A combined corner and edge detector. In: Alvey vision conference (Vol. 15, p. 50)

  94. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395

    Article  MathSciNet  Google Scholar 

  95. York D (1968) Least squares fitting of a straight line with correlated errors. Earth Planet Sci Lett 5:320–324

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mevlana Universitesi, Konya, Konya, Turkey

    Jawad Muhammad & Essam Abo-Serie

  2. Karatay University, Konya, Konya, Turkey

    Halis Altun

Authors
  1. Jawad Muhammad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Halis Altun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Essam Abo-Serie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jawad Muhammad.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Muhammad, J., Altun, H. & Abo-Serie, E. Welding seam profiling techniques based on active vision sensing for intelligent robotic welding. Int J Adv Manuf Technol 88, 127–145 (2017). https://doi.org/10.1007/s00170-016-8707-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-016-8707-0

Keywords

Search

Navigation