Intelligent Defect Diagnosis of Appearance Quality for Prefabricated Concrete Components Based on Target Detection and Multimodal Fusion Decision
Publication: Journal of Computing in Civil Engineering
Volume 37, Issue 6
Abstract
The quality of prefabricated concrete (PC) components during the construction phase is crucial for project safety. However, manual inspections are no longer sufficient to meet the demands of efficient and large-scale quality inspections of PC components. While computer vision (CV) can quickly inspect the surface quality of PC components, it fails to effectively prioritize critical quality defects among different components. Treating all quality defects equally would result in resource wastage. To address the efficient detection of external quality in PC components during the construction phase, this study proposes an appearance quality diagnosis method based on object detection and multimodal fusion decision. By integrating human and machine intelligence in quality inspections and implementing multimodal fusion decision-making, the intelligent quality diagnosis method becomes more targeted. By utilizing image object detection, the accuracy of identifying quality defects reached 87.70%. The fusion decision approach combining human and machine intelligence is applied to make informed decisions regarding structures with quality defects. Through the utilization of point cloud data, high-precision quality inspections of problematic components with an accuracy of 0.1 mm have been achieved. The developed case library enables defect tracking and provides recommendations for optimization solutions. The results demonstrate that the proposed engineering quality diagnostic method can effectively and quickly identify quality defects in PC components and provide improvement suggestions.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
The authors’ special thanks go to all survey participants and reviewers of the paper, and appreciation to National Key Research and Development Program (2022YFC3803602), National Natural Science Foundation of China (72071043), Natural Science Foundation of Jiangsu Province (BK20201280), and the Ministry of Education of Humanities and Social Science Project in China (20YJAZH114).
References
Arashpour, M., A. Heidarpour, A. Akbar Nezhad, Z. Hosseinifard, N. Chileshe, and R. Hosseini. 2020. “Performance-based control of variability and tolerance in off-site manufacture and assembly: Optimization of penalty on poor production quality.” Construct. Manage. Econ. 38 (6): 502–514. https://doi.org/10.1080/01446193.2019.1616789.
Arashpour, M., V. Kamat, A. Heidarpour, M. R. Hosseini, and P. Gill. 2022. “Computer vision for anatomical analysis of equipment in civil infrastructure projects: Theorizing the development of regression-based deep neural networks.” Autom. Constr. 137 (May): 104193. https://doi.org/10.1016/j.autcon.2022.104193.
Babic, M., M. A. Farahani, and T. Wuest. 2021. “Image based quality inspection in smart manufacturing systems: A literature review.” Procedia CIRP 103 (Jan): 262–267. https://doi.org/10.1016/j.procir.2021.10.042.
Bae, J., and S. Han. 2021. “Vision-based inspection approach using a projector-camera system for off-site quality control in modular construction: Experimental investigation on operational conditions.” J. Comput. Civ. Eng. 35 (5): 04021012. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000978.
Daniel, F., P. Kucherbaev, C. Cappiello, B. Benatallah, and M. Allahbakhsh. 2018. “Quality control in crowdsourcing: A survey of quality attributes, assessment techniques, and assurance actions.” ACM Comput. Surv. 51 (1): 1–40. https://doi.org/10.1145/3148148.
Darko, A., A. P. C. Chan, E. E. Ameyaw, E. K. Owusu, E. Pärn, and D. J. Edwards. 2019. “Review of application of analytic hierarchy process (AHP) in construction.” Int. J. Constr. Manage. 19 (5): 436–452. https://doi.org/10.1080/15623599.2018.1452098.
Delhi, V. S. K., R. Sankarlal, and A. Thomas. 2020. “Detection of personal protective equipment (PPE) compliance on construction site using computer vision based deep learning techniques.” Front. Built Environ. 6 (Sep): 136. https://doi.org/10.3389/fbuil.2020.00136.
Ding, L., K. Li, Y. Zhou, and P. E. Love. 2017. “An IFC-inspection process model for infrastructure projects: Enabling real-time quality monitoring and control.” Autom. Constr. 84 (Dec): 96–110. https://doi.org/10.1016/j.autcon.2017.08.029.
Dong, C. Z., and F. N. Catbas. 2021. “A review of computer vision–based structural health monitoring at local and global levels.” Struct. Health Monit. 20 (2): 692–743. https://doi.org/10.1177/1475921720935585.
Eissa, R., M. S. Eid, and E. Elbeltagi. 2021. “Current applications of game theory in construction engineering and management research: A social network analysis approach.” J. Constr. Eng. Manage. 147 (7): 04021066. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002085.
Gao, J., P. Li, Z. Chen, and J. Zhang. 2020. “A survey on deep learning for multimodal data fusion.” Neural Comput. 32 (5): 829–864. https://doi.org/10.1162/neco_a_01273.
Ge, Z., S. Liu, F. Wang, Z. Li, and J. Sun. 2021. “Yolox: Exceeding yolo series in 2021.” Preprint, submitted January16, 2022. http://arxiv.org/abs/2107.08430.
Guo, J., and Q. Wang. 2019. “Dimensional quality inspection of prefabricated MEP modules with 3D laser scanning.” In Proc., Int. Conf. on Computing in Civil Engineering 2019, 171–177. Reston, VA: ASCE. https://doi.org/10.1061/9780784482438.022.
Hussain, S., Z. Fangwei, A. F. Siddiqi, Z. Ali, and M. S. Shabbir. 2018. “Structural equation model for evaluating factors affecting quality of social infrastructure projects.” Sustainability 10 (5): 1415. https://doi.org/10.3390/su10051415.
Jin, L., F. Tan, and S. Jiang. 2020. “Generative adversarial network technologies and applications in computer vision.” Comput. Intell. Neurosci. 2020 (Aug): 1459107. https://doi.org/10.1155/2020/1459107.
Kim, M. K., J. C. Cheng, H. Sohn, and C. C. Chang. 2015. “A framework for dimensional and surface quality assessment of precast concrete elements using BIM and 3D laser scanning.” Autom. Constr. 49 (Jan): 225–238. https://doi.org/10.1016/j.autcon.2014.07.010.
Lahat, D., T. Adali, and C. Jutten. 2015. “Multimodal data fusion: An overview of methods, challenges, and prospects.” Proc. IEEE 103 (9): 1449–1477. https://doi.org/10.1109/JPROC.2015.2460697.
Lazli, L., M. Boukadoum, and O. A. Mohamed. 2020. “A survey on computer-aided diagnosis of brain disorders through MRI based on machine learning and data mining methodologies with an emphasis on Alzheimer disease diagnosis and the contribution of the multimodal fusion.” Appl. Sci. 10 (5): 1894. https://doi.org/10.3390/app10051894.
Levy, S. M. 2018. Project management in construction. New York: McGraw-Hill Education.
Liu, A., and S. Zhao. 2018. “High-performance target tracking scheme with low prediction precision requirement in WSNs.” Int. J. Ad Hoc Ubiquitous Comput. 29 (4): 270–289. https://doi.org/10.1504/IJAHUC.2018.096081.
Liu, J., T. Li, P. Xie, S. Du, F. Teng, and X. Yang. 2020. “Urban big data fusion based on deep learning: An overview.” Inf. Fusion 53 (Jan): 123–133. https://doi.org/10.1016/j.inffus.2019.06.016.
Liu, K., and J. Boehm. 2015. “Classification of big point cloud data using cloud computing.” ISPRS Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 40 (Aug): 553–557. https://doi.org/10.5194/isprsarchives-XL-3-W3-553-2015.
Liu, Y., R. D. Chang, J. Zuo, F. Xiong, and N. Dong. 2023. “What leads to the high capital cost of prefabricated construction in China: Perspectives of stakeholders.” Eng. Constr. Archit. Manage. 30 (2): 805–832. https://doi.org/10.1108/ECAM-06-2021-0538.
Martínez-Lorente, A. R., F. Dewhurst, and B. G. Dale. 1998. “Total quality management: Origins and evolution of the term.” TQM Mag. 10 (5): 378–386. https://doi.org/10.1108/09544789810231261.
MOHURD (Ministry of Housing and Urban-Rural Development of the People’s of China). 2015. Code for quality acceptance of concrete structure construction. GB50204. Beijing: MOHURD.
Nemati, S., R. Rohani, M. E. Basiri, M. Abdar, N. Y. Yen, and V. Makarenkov. 2019. “A hybrid latent space data fusion method for multimodal emotion recognition.” IEEE Access 7 (Nov): 172948–172964. https://doi.org/10.1109/ACCESS.2019.2955637.
Nguyen, P. T., V. D. B. Huynh, K. D. Vo, P. T. Phan, M. Elhoseny, and D. N. Le. 2021. “Deep learning based optimal multimodal fusion framework for intrusion detection systems for healthcare data.” CMC-Comput. Mater. Continua 66 (3): 2555–2571. https://doi.org/10.32604/cmc.2021.012941.
Ning, J., and J. Wang. 2022. “Automatic driving scene target detection algorithm based on improved YOLOv5 network.” In Proc., Int. Conf. on Computer Network, Electronic and Automation (ICCNEA), 218–222. New York: IEEE. https://doi.org/10.1109/ICCNEA57056.2022.00056.
Paneru, S., and I. Jeelani. 2021. “Computer vision applications in construction: Current state, opportunities and challenges.” Autom. Constr. 132 (Dec): 103940. https://doi.org/10.1016/j.autcon.2021.103940.
Pang, J., H. Zhang, C. Feng, and L. Li. 2020. “Research on crack segmentation method of hydro-junction project based on target detection network.” KSCE J. Civ. Eng. 24 (9): 2731–2741. https://doi.org/10.1007/s12205-020-1896-y.
Raghunandan, A., P. Raghav, and H. R. Aradhya. 2018. “Object detection algorithms for video surveillance applications.” In Proc., Int. Conf. on Communication and Signal Processing (ICCSP), 563–568. New York: IEEE. https://doi.org/10.1109/ICCSP.2018.8524461.
Ruodan, L., B. Ioannis, and C. R. Middleton. 2018. “Detection of structural components in point clouds of existing RC bridges.” Comput.-Aided Civ. Infrastruct. Eng. 34 (3): 191–212. https://doi.org/10.1111/mice.12407.
Son, R. H., and K. Han. 2022. “Automated model-based 3D scan planning for prefabricated building components.” J. Comput. Civ. Eng. 37 (2): 04022058. https://doi.org/10.1061/(ASCE)CP.1943-5487.0001055.
Sullivan, J., M. E. Asmar, J. Chalhoub, and H. Obeid. 2017. “Two decades of performance comparisons for design-build, construction manager at risk, and design-bid-build: Quantitative analysis of the state of knowledge on project cost, schedule, and quality.” J. Constr. Eng. Manage. 143 (6): 04017009. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001282.
Thomas, P., H. B. El Haouzi, M. C. Suhner, A. Thomas, E. Zimmermann, and M. Noyel. 2018. “Using a classifier ensemble for proactive quality monitoring and control: The impact of the choice of classifiers types, selection criterion, and fusion process.” Comput. Ind. 99 (Aug): 193–204. https://doi.org/10.1016/j.compind.2018.03.038.
Tseng, K. K., J. Lin, C. M. Chen, and M. M. Hassan. 2021. “A fast instance segmentation with one-stage multi-task deep neural network for autonomous driving.” Comput. Electr. Eng. 93 (Jul): 107194. https://doi.org/10.1016/j.compeleceng.2021.107194.
Wan, S., and S. Goudos. 2020. “Faster R-CNN for multi-class fruit detection using a robotic vision system.” Comput. Networks 168 (Feb): 107036. https://doi.org/10.1016/j.comnet.2019.107036.
Woo, S., J. Park, J. Y. Lee, and I. S. Kweon. 2018. “CBAM: Convolutional block attention module.” In Proc., European Conf. on Computer Vision (ECCV), 3–19. Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-030-01234-2_1.
Wu, Q., C. Shen, P. Wang, A. Dick, and A. Van Den Hengel. 2017. “Image captioning and visual question answering based on attributes and external knowledge.” IEEE Trans. Pattern Anal. Mach. Intell. 40 (6): 1367–1381. https://doi.org/10.1109/TPAMI.2017.2708709.
Xu, Z., Y. Liang, Y. Xu, Z. Fang, and U. Stilla. 2022. “Geometric modeling and surface-quality inspection of prefabricated concrete components using sliced point clouds.” J. Constr. Eng. Manage. 148 (9): 04022087. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002345.
Ye, F., J. Chen, and Y. Li. 2017. “Improvement of DS evidence theory for multi-sensor conflicting information.” Symmetry 9 (5): 69. https://doi.org/10.3390/sym9050069.
Yeh, Y. R., T. C. Lin, Y. Y. Chung, and Y. C. F. Wang. 2012. “A novel multiple kernel learning framework for heterogeneous feature fusion and variable selection.” IEEE Trans. Multimedia 14 (3): 563–574. https://doi.org/10.1109/TMM.2012.2188783.
Zhao, W., Y. Jiang, Y. Liu, and J. Shu. 2022. “Automated recognition and measurement based on three-dimensional point clouds to connect precast concrete components.” Autom. Constr. 133 (Jan): 104000. https://doi.org/10.1016/j.autcon.2021.104000.
Zhong, B., H. Wu, L. Ding, P. E. Love, H. Li, H. Luo, and L. Jiao. 2019. “Mapping computer vision research in construction: Developments, knowledge gaps and implications for research.” Autom. Constr. 107 (Nov): 102919. https://doi.org/10.1016/j.autcon.2019.102919.
Information & Authors
Information
Published In
Journal of Computing in Civil Engineering
Volume 37 • Issue 6 • November 2023
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Apr 14, 2023
Accepted: Jun 25, 2023
Published online: Aug 8, 2023
Published in print: Nov 1, 2023
Discussion open until: Jan 8, 2024
ASCE Technical Topics:
- Business management
- Concrete
- Construction engineering
- Construction equipment
- Construction industry
- Construction management
- Construction methods
- Defects and imperfections
- Diagnosis
- Engineering fundamentals
- Engineering materials (by type)
- Equipment and machinery
- Forensic engineering
- Human and behavioral factors
- Inspection
- Materials characterization
- Materials engineering
- Offsite construction
- Practice and Profession
- Structural engineering
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.