A Framework for Automated Quality Control of Wood-Framed Panels in Robotic-Based Manufacturing Using Computer Vision and Deep Learning
Publication: Construction Research Congress 2024
ABSTRACT
The advent of robotic systems has brought significant transformations across various industries, increasing the quality of products and services. However, due to construction projects’ intricacy and robotic manufacturing’s technological challenges, robotics in the construction sector is still in the nascent stages of development. The variability in construction materials presents a major challenge to the integration of robotics-based manufacturing. Lumber misalignments can cause costly reworks to the wood framing process due to lumber damage, structural deviations, and nail gun misfires. This paper seeks to address the critical quality control challenge for robotic-based manufacturing in industrialized construction. The proposed automated quality control system detects alignment issues using computer vision and deep learning technology. Detected misalignments are transmitted through a graphical user interface (GUI) to construction workers to allow them to determine whether corrective actions are required or not. The field experiments illustrated the significance of the proposed system in ensuring a proper framing process and enhancing the quality, safety, and productivity of robotic manufacturing in industrialized construction.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Agapaki, E., and I. Brilakis. 2020. “CLOI-NET: Class segmentation of industrial facilities’ point cloud datasets.” Advanced Engineering Informatics, 45: 101121. https://doi.org/10.1016/j.aei.2020.101121.
Azar, E. R., S. Dickinson, and B. Mccabe. 2012. “Server-Customer Interaction Tracker: Computer Vision–Based System to Estimate Dirt-Loading Cycles.” J Constr Eng Manag, 139 (7): 785–794. American Society of Civil Engineers. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000652.
Bock, T. 2007. “Construction robotics.” Auton Robots, 22 (3): 201–209. Springer. https://doi.org/10.1007/S10514-006-9008-5/FIGURES/20.
Bock, T. 2015. “The future of construction automation: Technological disruption and the upcoming ubiquity of robotics.” Autom Constr, 59: 113–121. Elsevier. https://doi.org/10.1016/J.AUTCON.2015.07.022.
Di Leo, G., C. Liguori, A. Pietrosanto, and P. Sommella. 2016. “A vision system for the online quality monitoring of industrial manufacturing.” Opt Lasers Eng, 89: 162–168. Elsevier Ltd. https://doi.org/10.1016/J.OPTLASENG.2016.05.007.
Ding, F., Z. Zhuang, Y. Liu, D. Jiang, X. Yan, and Z. Wang. 2020. “Detecting Defects on Solid Wood Panels Based on an Improved SSD Algorithm.” Sensors (Basel), 20 (18): 1–17. Sensors (Basel). https://doi.org/10.3390/S20185315.
Gong, J., and C. H. Caldas. 2009. “Computer Vision-Based Video Interpretation Model for Automated Productivity Analysis of Construction Operations.” Journal of Computing in Civil Engineering, 24 (3): 252–263. American Society of Civil Engineers. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000027.
IFR. 2022. “World Robotics Report: ‘All-Time High’ with Half a Million Robots Installed in one Year - International Federation of Robotics.” Accessed April 7, 2023. https://ifr.org/ifr-press-releases/news/wr-report-all-time-high-with-half-a-million-robots-installed.
Iyer, R. V., P. S. Ringe, and K. P. Bhensdadiya. 2021. “Comparison of YOLOv3, YOLOv5s and MobileNet-SSD V2 for Real-Time Mask Detection.” 08 (07).
Jocher, G. 2020. YOLOv5 by Ultralytics. Python.
Lin, K.-L., and J.-L. Fang. 2013. “Applications of computer vision on tile alignment inspection.” Automation in Construction, 35: 562–567. https://doi.org/10.1016/j.autcon.2013.01.009.
Lin, T. Y., P. Goyal, R. Girshick, K. He, and P. Dollar. 2017. “Focal Loss for Dense Object Detection.” Proceedings of the IEEE International Conference on Computer Vision, 2017-October: 2999–3007. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/ICCV.2017.324.
Liu, W., D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg. 2016. “SSD: Single shot multibox detector.” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9905 LNCS: 21–37. Springer Verlag. https://doi.org/10.1007/978-3-319-46448-0_2.
Tehrani, B.M., S. BuHamdan, and A. Alwisy. 2022. “Robotics in assembly-based industrialized construction: a narrative review and a look forward.” Int J Intell Robot Appl. Springer. https://doi.org/10.1007/S41315-022-00257-9.
Martinez, P., R. Ahmad, and M. Al-Hussein. 2019. “A vision-based system for pre-inspection of steel frame manufacturing.” Autom Constr, 97: 151–163. Elsevier B.V. https://doi.org/10.1016/J.AUTCON.2018.10.021.
Memarzadeh, M., M. Golparvar-Fard, and J. C. Niebles. 2013. “Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors.” Autom Constr, 32: 24–37. Elsevier. https://doi.org/10.1016/J.AUTCON.2012.12.002.
Orlowski, K. 2020. “Automated manufacturing for timber-based panelised wall systems.” Autom Constr, 109: 102988. Elsevier. https://doi.org/10.1016/J.AUTCON.2019.102988.
Pan, M., and W. Pan. 2020. “Stakeholder Perceptions of the Future Application of Construction Robots for Buildings in a Dialectical System Framework.” Journal of Management in Engineering, 36 (6): 04020080. American Society of Civil Engineers. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000846.
Park, M. W., and I. Brilakis. 2012. “Construction worker detection in video frames for initializing vision trackers.” Autom Constr, 28: 15–25. Elsevier. https://doi.org/10.1016/J.AUTCON.2012.06.001.
Redmon, J. A. F. n.d. “Yolov3: An incremental improvement.” arxiv.org.
Redmon, J., S. Divvala, R. Girshick, and A. Farhadi. 2016. “You only look once: Unified, real-time object detection.” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-December: 779–788. IEEE Computer Society. https://doi.org/10.1109/CVPR.2016.91.
Redmon, J., and A. Farhadi. 2017. “YOLO9000: Better, faster, stronger.” Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, 2017-January: 6517–6525. Institute of Electrical and Electronics Engineers Inc. https://doi.org/10.1109/CVPR.2017.690.
Ren, S., K. He, R. Girshick, and J. Sun. 2015. “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.” Adv Neural Inf Process Syst, 28.
Rezazadeh Azar, E., and B. McCabe. 2012. “Part based model and spatial-temporal reasoning to recognize hydraulic excavators in construction images and videos.” Autom Constr, 24: 194–202. Automation in Construction. https://doi.org/10.1016/J.AUTCON.2012.03.003.
Tehrani, B. M., S. BuHamdan, and A. Alwisy. 2022. “Robotics in industrialized construction: an activity-based ranking system for assembly manufacturing tasks.” Engineering, Construction and Architectural Management, ahead-of-print (ahead-of-print). Emerald Publishing Limited. https://doi.org/10.1108/ECAM-02-2022-0143.
Thoma, A., A. Adel, M. Helmreich, T. Wehrle, F. Gramazio, and M. Kohler. 2019. “Robotic Fabrication of Bespoke Timber Frame Modules.” Robotic Fabrication in Architecture, Art and Design 2018, 447–458. Springer, Cham. https://doi.org/10.1007/978-3-319-92294-2_34.
Yang, J., M. W. Park, P. A. Vela, and M. Golparvar-Fard. 2015. “Construction performance monitoring via still images, time-lapse photos, and video streams: Now, tomorrow, and the future.” Advanced Engineering Informatics, 29 (2): 211–224. Elsevier Limited. https://doi.org/10.1016/J.AEI.2015.01.011.
Information & Authors
Information
Published In
Construction Research Congress 2024
Pages: 456 - 465
History
Published online: Mar 18, 2024
ASCE Technical Topics:
- Automation and robotics
- Business management
- Computer vision and image processing
- Computing in civil engineering
- Construction engineering
- Construction management
- Engineering fundamentals
- Frames
- Industries
- Management methods
- Manufacturing
- Methodology (by type)
- Organizations
- Practice and Profession
- Quality control
- Structural engineering
- Structural members
- Structural systems
- Systems engineering
- Wood frames
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.