Deep Learning–Based Automation of Scan-to-BIM with Modeling Objects from Occluded Point Clouds
Publication: Journal of Management in Engineering
Volume 38, Issue 4
Abstract
As-built building information modeling (BIM) currently is regarded as a tool with the potential to manage buildings efficiently in the operation and maintenance phases. However, as-built BIM modeling is a labor-intensive process that requires considerable cost and time in modeling existing buildings. Although active research on scan-to-BIM automation has addressed this issue, previous studies modeled only major objects such as walls, floors, and ceilings, consequently requiring modeling other objects in indoor spaces. In addition, there was a limitation in modeling objects located in the occluded areas of scanned point clouds. Therefore, this study extracted various indoor objects from a point cloud based on deep-learning, and compensated for incomplete object information from occluded point clouds for automating the process of scan-to-BIM. The number of object classes extracted from the semantic segmentation of a deep learning network was increased to 13, and spatial relationships between objects were defined to improve the accuracy of bounding boxes extracted from point clouds. Furthermore, a parametric algorithm was developed to match the bounding boxes and objects in a BIM library to generate BIM models automatically. In a case study involving an office room, the accuracy of the bounding boxes of some object classes improved by as much as 53.33%. The study verified the feasibility of the proposed method of scan-to-BIM automation for the three-dimensional (3D) reality capture of existing buildings.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some of all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This work was supported by the National Research Foundation of Korea (NRF) grant funded by Ministry of Science and ICT (NRF-2020R1A4A2002855), and the Korea Agency for Infrastructure Technology Advancement (KAIA) grant funded by the Ministry of Land, Infrastructure and Transport (Grant 22AATD-C163269-02).
References
Akcamete, A., B. Akinci, and J. H. Garrett. 2010. “Potential utilization of building information models for planning maintenance activities.” In Proc., Int. Conf. on Computing in Civil and Building Engineering, edited by W. Tizani, 151–157. Nottingham, UK: Nottingham University Press.
Armeni, I., S. Sax, A. R. Zamir, and S. Savarese. 2017. “Joint 2D-3D-semantic data for indoor scene understanding.” Accessed May 6, 2017. https://arxiv.org/abs/1702.01105.
Austin, M., P. Delgoshaei, M. Coelho, and M. Heidarinejad. 2020. “Architecting smart city digital twins: Combined semantic model and machine learning approach.” J. Manage. Eng. 36 (4): 04020026.
Banfi, F., S. Fai, and R. Brumana. 2017. “BIM automation: Advanced modeling generative process for complex structures.” In Proc., 26th Int. CIPA Symp. on Digital Workflows for Heritage Conservation 2017, 9–16. Rawalpindi, Pakistan: Copernicus GmbH.
Bassier, M., L. Mattheuwsen, and M. Vergauwen. 2019. “BIM reconstruction: Automated procedural modeling from point cloud data.” Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLII-2/W17 (2/W17): 53–60. https://doi.org/10.5194/isprs-archives-XLII-2-W17-53-2019.
Braun, A., and A. Borrmann. 2019. “Combining inverse photogrammetry and BIM for automated labeling of construction site images for machine learning.” Autom. Constr. 106 (Oct): 102879. https://doi.org/10.1016/j.autcon.2019.102879.
Cheng, Y.-J., W.-G. Qiu, and D.-Y. Duan. 2019. “Automatic creation of as-is building information model from single-track railway tunnel point clouds.” Autom. Constr. 106 (Oct): 102911. https://doi.org/10.1016/j.autcon.2019.102911.
Cherabier, I., C. Hane, M. R. Oswald, and M. Pollefeys. 2016. “PointNet.” In Proc., 4th Int. Conf. on 3D Vision, 3DV 2016, 601–610. New York: Cornell Univ.
Daszykowski, M., and B. Walczak. 2009. “Density-based clustering methods.” Compr. Chemom. 2: 635–654.
Douillard, B., J. Underwood, N. Kuntz, V. Vlaskine, A. Quadros, P. Morton, and A. Frenkel. 2011. “On the segmentation of 3D LIDAR point clouds.” In Proc., IEEE Int. Conf. on Robotics and Automation, 2798–2805. New York: IEEE.
Engelmann, F., T. Kontogianni, A. Hermans, and B. Leibe. 2017. “Exploring spatial context for 3D semantic segmentation of point clouds.” In Proc., 2017 IEEE Int. Conf. on Computer Vision Workshops, ICCVW 2017, 716–724. New York: IEEE.
Everingham, M., S. M. A. Eslami, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. 2015. “The PASCAL visual object classes challenge: A retrospective.” Int. J. Comput. Vision 111 (1): 98–136. https://doi.org/10.1007/s11263-014-0733-5.
Graham, B., M. Engelcke, and L. Van Der Maaten. 2018. “3D semantic segmentation with submanifold sparse convolutional networks.” In Proc., IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 9224–9232. New York: IEEE.
Hammad, A. W. A. 2019. “Minimising the deviation between predicted and actual building performance via use of neural networks and BIM.” Buildings 9 (5): 131. https://doi.org/10.3390/buildings9050131.
Huber, D., B. Akinci, A. Adan, E. B. Anil, B. Okorn, and X. Xiong. 2011. “Methods for automatically modeling and representing as-built building information models.” In Proc., NSF Engineering Research and Innovation Conf. Atlanta: NSF Engineering Research and Innovation Conference.
Ilter, D., and E. Ergen. 2015. “BIM for building refurbishment and maintenance: Current status and research directions.” Struct. Surv. 33 (3): 228–256. https://doi.org/10.1108/SS-02-2015-0008.
Jung, J., S. Hong, S. Jeong, S. Kim, H. Cho, S. Hong, and J. Heo. 2014. “Productive modeling for development of as-built BIM of existing indoor structures.” Autom. Constr. 42 (Jun): 68–77. https://doi.org/10.1016/j.autcon.2014.02.021.
Kufuor, J., D. D. Mohanty, E. Valero, and F. Bosché. 2021. “Automatic MEP component detection with deep learning.” In Proc., ICPR Int. Workshops and Challenges, edited by A. Del Bimbo, R. Cucchiara, S. Sclaroff, G. M. Farinella, T. Mei, M. Bertini, H. J. Escalante, and R. Vezzani, 373–388. Cham, Switzerland: Springer International.
Lawin, F. J., M. Danelljan, P. Tosteberg, G. Bhat, F. S. Khan, and M. Felsberg. 2017. “Deep projective 3D semantic segmentation.” In Computer analysis of images and patterns, edited by M. Felsberg, A. Heyden, and N. Krüger, 95–107. Cham, Switzerland: Springer International.
Li, Q., H.-Y. Chong, C.-Y. Lee, and Y. Zhang. 2021. “BIM’s formal and informal collaborative networks in traditional procurement: Insights from the construction phase of a hospital case study.” J. Manage. Eng. 37 (6): 05021008. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000965.
Lin, Y. C., and W. F. Cheung. 2020. “Developing WSN/BIM-based environmental monitoring management system for parking garages in smart cities.” J. Manage. Eng. 36 (3): 04020012. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000760.
Lu, Q., A. K. Parlikad, P. Woodall, G. Don Ranasinghe, X. Xie, Z. Liang, E. Konstantinou, J. Heaton, and J. Schooling. 2020. “Developing a digital twin at building and city levels: Case study of West Cambridge campus.” J. Manage. Eng. 36 (3): 05020004. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000763.
Ma, L., R. Sacks, U. Kattel, and T. Bloch. 2018. “3D object classification using geometric features and pairwise relationships.” Comput.-Aided Civ. Infrastruct. Eng. 33 (2): 152–164. https://doi.org/10.1111/mice.12336.
Macher, H., L. Roy, and T. Landes. 2021. “Automation of windows detection from geometric and radiometric information of point clouds in a scan-to-BIM process.” Int. Arch. Photogram. Remote Sens. Spatial Inf. Sci. XLIII-B2-2021 (B2-2021): 193–200. https://doi.org/10.5194/isprs-archives-XLIII-B2-2021-193-2021.
Mill, T., A. Alt, and R. Liias. 2013. “Combined 3D building surveying techniques–terrestrial laser scanning (TLS) and total station surveying for BIM data management purposes.” Supplement, J. Civ. Eng. Manage. 19 (S1): S23–S32. https://doi.org/10.3846/13923730.2013.795187.
Pătrăucean, V., I. Armeni, M. Nahangi, J. Yeung, I. Brilakis, and C. Haas. 2015. “State of research in automatic as-built modelling.” Adv. Eng. Inf. 29 (2): 162–171. https://doi.org/10.1016/j.aei.2015.01.001.
Perez-Perez, Y., M. Golparvar-Fard, and K. El-Rayes. 2021. “Scan2BIM-NET: Deep learning method for segmentation of point clouds for scan-to-BIM.” J. Constr. Eng. Manage. 147 (9): 04021107. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002132.
Schnabel, R., R. Wahl, and R. Klein. 2007. “Efficient RANSAC for point-cloud shape detection.” Comput. Graphics Forum 26 (2): 214–226. https://doi.org/10.1111/j.1467-8659.2007.01016.x.
Shahi, K., B. Y. McCabe, and A. Shahi. 2019. “Framework for automated model-based e-permitting system for municipal jurisdictions.” J. Manage. Eng. 35 (6): 04019025. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000712.
Tang, P., D. Huber, B. Akinci, R. Lipman, and A. Lytle. 2010. “Automatic reconstruction of as-built building information models from laser-scanned point clouds: A review of related techniques.” Autom. Constr. 19 (7): 829–843. https://doi.org/10.1016/j.autcon.2010.06.007.
Tao, F., H. Zhang, A. Liu, and A. Y. C. Nee. 2019. “Digital twin in industry: State-of-the-art.” IEEE Trans. Ind. Inf. 15 (4): 2405–2415. https://doi.org/10.1109/TII.2018.2873186.
Tayeh, R., and R. R. A. Issa. 2020. “Interactive holograms for construction coordination and quantification.” J. Manage. Eng. 36 (6): 04020079. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000847.
Tchapmi, L., C. Choy, I. Armeni, J. Gwak, and S. Savarese. 2017. “SEGCloud: Semantic segmentation of 3D Point clouds.” In Proc., Int. Conf. on 3D Vision (3DV), 537–547. New York: Cornell Univ.
Thomson, C., and J. Boehm. 2015. “Automatic geometry generation from point clouds for BIM.” Remote Sens. 7 (9): 11753–11775. https://doi.org/10.3390/rs70911753.
Trzeciak, M., and A. Borrmann. 2019. “Towards registration of construction drawings to building information models using knowledge-based extended geometric hashing.” In Proc., CEUR Workshop, 2394. London: CEUR Workshop.
Volk, R., J. Stengel, and F. Schultmann. 2014. “Building information modeling (BIM) for existing buildings—Literature review and future needs.” Autom. Constr. 38 (Mar): 109–127. https://doi.org/10.1016/j.autcon.2013.10.023.
Xu, J., L. Ding, and P. E. D. Love. 2017. “Digital reproduction of historical building ornamental components: From 3D scanning to 3D printing.” Autom. Constr. 76 (Apr): 85–96. https://doi.org/10.1016/j.autcon.2017.01.010.
Yang, L., J. C. P. Cheng, and Q. Wang. 2020. “Semi-automated generation of parametric BIM for steel structures based on terrestrial laser scanning data.” Autom. Constr. 112 (Apr): 103037. https://doi.org/10.1016/j.autcon.2019.103037.
Zhang, J., J. C. Cheng, W. Chen, and K. Chen. 2022. “Digital twins for construction sites: Concepts, LoD definition, and applications.” J. Manage. Eng. 38 (2): 04021094. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000948.
Information & Authors
Information
Published In
Journal of Management in Engineering
Volume 38 • Issue 4 • July 2022
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Nov 20, 2021
Accepted: Feb 18, 2022
Published online: Mar 28, 2022
Published in print: Jul 1, 2022
Discussion open until: Aug 28, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Yejin Shin, Yujin Choi, Jaeseung Won, Taehoon Hong, Choongwan Koo, A New Benchmark Model for the Automated Detection and Classification of a Wide Range of Heavy Construction Equipment, Journal of Management in Engineering, 10.1061/JMENEA.MEENG-5630, 40, 2, (2024).
- Jong Won Ma, Jaehoon Jung, Fernanda Leite, Deep Learning–Based Scan-to-BIM Automation and Object Scope Expansion Using a Low-Cost 3D Scan Data, Journal of Computing in Civil Engineering, 10.1061/JCCEE5.CPENG-5751, 38, 6, (2024).
- Sohyun Kim, Hyeonggyun Kim, Jaewook Lee, Taehoon Hong, Kwangbok Jeong, An Integrated Assessment Framework of Economic, Environmental, and Human Health Impacts Using Scan-to-BIM and Life-Cycle Assessment in Existing Buildings, Journal of Management in Engineering, 10.1061/JMENEA.MEENG-5600, 39, 5, (2023).
- Sohyun Kim, Kwangbok Jeong, Taehoon Hong, Jaehong Lee, Jaewook Lee, Deep Learning–Based Automated Generation of Material Data with Object–Space Relationships for Scan to BIM, Journal of Management in Engineering, 10.1061/JMENEA.MEENG-5143, 39, 3, (2023).
- Zhongming Xiang, Abbas Rashidi, Ge Ou, Integrating Inverse Photogrammetry and a Deep Learning–Based Point Cloud Segmentation Approach for Automated Generation of BIM Models, Journal of Construction Engineering and Management, 10.1061/JCEMD4.COENG-13020, 149, 9, (2023).
- Ziwei Luo, Zhong Xie, Jie Wan, Ziyin Zeng, Lu Liu, Liufeng Tao, Indoor 3D Point Cloud Segmentation Based on Multi-Constraint Graph Clustering, Remote Sensing, 10.3390/rs15010131, 15, 1, (131), (2022).