Real-Time and Automatic Detection of Welding Joints Using Deep Learning
Publication: Construction Research Congress 2022
ABSTRACT
Welding technique plays a pivotal role in many industries, such as construction, automobile manufacturing, and nuclear power plants (NPPs). However, the shortage of skilled welding workers is still controversial due to the severe working environment and conditions. Therefore, to conserve human labor and improve manufacturing efficiency, an automated welding process is necessary. Also, welding efficiency and quality are vital indicators requiring attention for automatic welding. Notably, in NPPs, minor welding defects can occur serious safety issues. Therefore, our research’s ultimate goal is to develop an automatic welding system to improve welding quality and manufacturing efficiency using visual sensors [e.g., a camera and light detection and ranging (LiDAR)], a robotic arm, and a welding machine. As the first step, this paper presents a method for automatically detecting different welding joints in real-time. Then, the different target joints are trained using a deep learning algorithm and detected by the camera. The results demonstrate the accuracy and effectiveness of the proposed method.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Cha, Y. J., Choi, W., Suh, G., Mahmoudkhani, S., and Büyüköztürk, O. (2018). “Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types.” Computer-Aided Civil and Infrastructure Engineering, 33(9), 731–747.
Chen, X., and Yuille, A. (2014). “Articulated pose estimation by a graphical model with image dependent pairwise relations.” Advances in Neural Information Processing Systems, 2(January), 1736–1744.
Cho, J., Lee, K., Shin, E., Choy, G., and Do, S. (2015). “How much data is needed to train a medical image deep learning system to achieve necessary high accuracy?”
Devereux, M. G., Murray, P., and West, G. (2021). “Automated Object Detection for Visual Inspection of Nuclear Reactor Cores.” Nuclear Technology, Taylor & Francis, 00(00), 1–14.
Dinham, M., and Fang, G. (2012). “Weld seam detection using computer vision for robotic Arc Welding.” IEEE International Conference on Automation Science and Engineering, IEEE, 771–776.
Dinham, M., and Fang, G. (2013). “Autonomous weld seam identification and localisation using eye-in-hand stereo vision for robotic arc welding.” Robotics and Computer-Integrated Manufacturing, Elsevier, 29(5), 288–301.
Golparvar-Fard, M., Heydarian, A., and Niebles, J. C. (2013). “Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers.” Advanced Engineering Informatics, Elsevier Ltd, 27(4), 652–663.
Han, K., and Golparvar-Fard, M. (2017). “Potential of big visual data and building information modeling for construction performance analytics: An exploratory study.” Automation in Construction, Elsevier B.V., 73, 184–198.
Jeelani, I., Asadi, K., Ramshankar, H., Han, K., and Albert, A. (2020). “Real-time Vision-based Worker Localization & Hazard Detection for Construction (under review).” Automation in Construction.
Kothari, J. D. (2018). “Detecting Welding Defects in Steel Plates using Computer Vision Algorithms.” International Journal of Advanced Research in Electrical, Electronics and Instrumentation Engineering, 7(9), 5.
Lin, L., Wang, K., Zuo, W., Wang, M., Luo, J., and Zhang, L. (2016). “A Deep Structured Model with Radius–Margin Bound for 3D Human Activity Recognition.” International Journal of Computer Vision, Springer US, 118(2), 256–273.
Mstafa, R. J., and Elleithy, K. M. (2016). “A video steganography algorithm based on Kanade-Lucas-Tomasi tracking algorithm and error correcting codes.” Multimedia Tools and Applications, 75(17), 10311–10333.
Nie, G., Guo, Y., Liu, Y., and Wang, Y. (2018). “Real-time salient object detection based on fully convolutional networks.” Communications in Computer and Information Science, 757, 189–198.
Noghabaei, M., and Han, K. (2021). “Object manipulation in immersive virtual environments: Hand Motion tracking technology and snap-to-fit function.” Automation in Construction, Elsevier B.V., 124(December 2020), 103594.
Noh, H., Hong, S., and Han, B. (2015). “Learning deconvolution network for semantic segmentation.” Proceedings of the IEEE International Conference on Computer Vision, 2015 Inter, 1520–1528.
Nuijten, R. J. G., Kooistra, L., and De Deyn, G. B. (2019). “Using Unmanned Aerial Systems (UAS) and Object-Based Image Analysis (OBIA) for Measuring Plant-Soil Feedback E ff ects on Crop Productivity.” Drones, 3(3)(54), 14.
Pathak, A. R., Pandey, M., and Rautaray, S. (2018). “Application of Deep Learning for Object Detection.” Procedia Computer Science, Elsevier B.V., 132(Iccids), 1706–1717.
Rahman, M. M., Tan, Y., Xue, J., and Lu, K. (2020). “Recent Advances in 3D Object Detection in the Era of Deep Neural Networks: A Survey.” IEEE Transactions on Image Processing, 29(10), 2947–2962.
Shafeek, H. I., Gadelmawla, E. S., Abdel-Shafy, A. A., and Elewa, I. M. (2004). “Assessment of welding defects for gas pipeline radiographs using computer vision.” NDT and E International, 37(4), 291–299.
Turan, E., Koçal, T., and Ünlügençoğlu, K. (2011). “Welding Technologies in Shipbuilding Industry.” Tojsat, 1(4), 24–30.
Voulodimos, A., Doulamis, N., Doulamis, A., and Protopapadakis, E. (2018). “Deep Learning for Computer Vision: A Brief Review.” Computational Intelligence and Neuroscience, 2018.
Xiao, B., and Kang, S.-C. (2021). “Development of an Image Data Set of Construction Machines for Deep Learning Object Detection.” Journal of Computing in Civil Engineering, 35(2), 05020005.
Yang, B., Yan, J., Lei, Z., and Li, S. Z. (2014). “Aggregate channel features for multi-view face detection.” IJCB 2014-2014 IEEE/IAPR International Joint Conference on Biometrics, IEEE.
Yang, L., Fan, J., Liu, Y., Li, E., Peng, J., and Liang, Z. (2020). “Automatic Detection and Location of Weld Beads With Deep Convolutional Neural Networks.” IEEE Transactions on Instrumentation and Measurement, 70, 1–12.
Yen, S. B., and Francisco, S. (1990). “Automatic Welding Seam Tracking and Identification.” Microbiology, 28(9), 1877–1880.
Information & Authors
Information
Published In
Construction Research Congress 2022
Pages: 601 - 609
History
Published online: Mar 7, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Doyun Lee, Guang-Yu Nie, Kevin Han, Automatic and Real-Time Joint Tracking and Three-Dimensional Scanning for a Construction Welding Robot, Journal of Construction Engineering and Management, 10.1061/JCEMD4.COENG-14135, 150, 3, (2024).