Automated Scaling of Point Cloud Rebar Model via ArUco-Supported Controlled Markers
Publication: Journal of Construction Engineering and Management
Volume 150, Issue 3
Abstract
Photogrammetry has gained the interest of professionals and researchers for activities related to construction projects’ progress monitoring via attaining precise 3D point models. However, the precision of the generated models is directly linked with the precise scaling of the point cloud to ground truth dimensions (GTDs). Available scaling-up procedures for the close-range photogrammetry technique are complex, time consuming, and require human intervention, which adds the risk of error in the scaled-up model dimensions. Such a scenario creates hesitation among industry professionals toward implementing point cloud technologies. This paper devises an automated scaling-up methodology to overcome the said concerns by considering the construction progress monitoring theme. The intact process of automated scaling up of point cloud model to GTDs is controlled by two main parameters, that is, Python-based modules and designed ArUco-supported controlled markers. Remarkable outcomes are achieved with less than 1% scaled-up error compared with GTDs, which will improve the confidence of industry professionals toward point cloud technologies.
Practical Applications
Photogrammetry applications have been adopted in several domains and the optimum usage of attained models can be executed with 3D replicas having precise details of surface features and geometry. Therefore, to attain 3D point cloud models with ground truth dimensions (GTDs), or actual dimensions of the targeted object the practitioners mostly follow the markers/ground control points (GCPs) technique (minimum three GCPs/markers), manual scaling, or georeferencing data. However, the accuracy of traditional GCPs/markers’ technique and manual rescaling is dependent on the experience of the site staff/operator, and error chances may increase with the increasing number of GCPs/markers, whereas the georeferencing data-based technique is more technical and complex. Therefore, this paper developed an automated system for scaling up 3D point cloud models to GTDs with minimal human involvement. The system works with the help of specialized designed markers known as ArUco-supported controlled markers (ASCM). Only one ASCM marker is placed beside the targeted object for imaging; the devised system detects the marker in the images and rescales the developed point cloud model following the designed strategy. The system has high accuracy and can easily be implemented for scaling up close-range photogrammetry models in any domain.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data generated or analyzed during the study are included in the published paper.
Acknowledgments
The project team appreciates Technologist Raja Intan Shafinaz Bt Raja Mohd Noor (Department of Civil & Environmental Engineering, University Technology PETRONAS, Malaysia) for her efforts and assistance in the accomplishment of this paper. Two authors (Abdul Hannan Qureshi and Wesam Salah Alaloul) are inventors in the provisional patent application filed in Malaysia (MyIPO) on 11 November 2022 (Ref: IP2022-0203). The claims in this patent cover ASCM design and application.
References
Agisoft. 2015. “Guidelines for calibrated scale bar placement and processing.” Accessed November 25, 2023. https://www.agisoft.com/pdf/tips_and_tricks/CHI_Calibrated_Scale_Bar_Placement_and_Processing.pdf.
Agisoft. 2023. “How to install Metashape stand-alone Python module: Helpdesk Portal.” Accessed September 8, 2023. https://agisoft.freshdesk.com/support/solutions/articles/31000148930-how-to-install-metashape-stand-alone-python-module.
Agisoft, LLC. 2021. “Metashape python reference.” Release 2.0.0. Accessed December 8, 2023. https://www.agisoft.com/pdf/metashape_python_api_1_8_2.pdf.
Alaloul, W. S., A. H. Qureshi, M. A. Musarat, and S. Saad. 2021. “Evolution of close-range detection and data acquisition technologies towards automation in construction progress monitoring.” J. Build. Eng. 43 (Jun): 102877. https://doi.org/10.1016/j.jobe.2021.102877.
Alizadehsalehi, S., and I. Yitmen. 2019. “A concept for automated construction progress monitoring: Technologies adoption for benchmarking project performance control.” Arabian J. Sci. Eng. 44 (5): 4993–5008. https://doi.org/10.1007/s13369-018-3669-1.
Chen, C., B. Yang, S. Song, M. Tian, J. Li, W. Dai, and L. Fang. 2018. “Calibrate multiple consumer RGB-D cameras for low-cost and efficient 3D indoor mapping.” Remote Sens. 10 (2): 328. https://doi.org/10.3390/rs10020328.
Chikushi, S., et al. 2020. “Automated image presentation for backhoe embankment construction in unmanned construction site.” In Proc., IEEE/SICE Int. Symp. on System Integration (SII), 22–27. New York: IEEE. https://doi.org/10.1109/SII46433.2020.9026298.
Cultural Heritage Imaging. 2015. “Guidelines for calibrated scale bar placement and processing.” Accessed December 8, 2023. https://www.agisoft.com/pdf/tips_and_tricks/CHI_Calibrated_Scale_Bar_Placement_and_Processing.pdf.
Dai, F., A. Rashidi, I. Brilakis, and P. Vela. 2012. “Comparison of image-based and time-of-flight-based technologies for three-dimensional reconstruction of infrastructure.” J. Constr. Eng. Manage. 139 (1): 69–79. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000565.
Delgado-Vera, C., M. Aguirre-Munizaga, M. Jiménez-Icaza, N. Manobanda-Herrera, and A. Rodríguez-Méndez. 2017. “A photogrammetry software as a tool for precision agriculture: A case study.” In Vol. 749 of Proc., Int. Conf. on Technologies and Innovation, 282–295. Berlin: Springer. https://doi.org/10.1007/978-3-319-67283-0_21.
Deng, H., H. Hong, D. Luo, Y. Deng, and C. Su. 2019. “Automatic indoor construction process monitoring for tiles based on BIM and computer vision.” J. Constr. Eng. Manage. 146 (1): 1–12. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001744.
Faltýnová, M., E. Matoušková, J. Šedina, and K. Pavelka. 2016. “Building facade documentation using laser scanning and photogrammetry and data implementation into BIM.” In Vol. 41 of Proc., Int. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives, 215–220. Göttingen, Germany: Copernicus GmbH. https://doi.org/10.5194/isprsarchives-XLI-B3-215-2016.
Fauzan, K. N., D. Suwardhi, A. Murtiyoso, I. Gumilar, and T. P. Sidiq. 2021. “Close-range photogrammetry method for SF6 gas insulated line (GIL) deformation monitoring.” In Vol. 43 of Proc., Int. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives, 503–510. Göttingen, Germany: Copernicus GmbH. https://doi.org/10.5194/isprs-archives-XLIII-B2-2021-503-2021.
Forlani, G., F. Diotri, U. M. di Cella, and R. Roncella. 2019. “Indirect UAV strip georeferencing by on-board GNSS data under poor satellite coverage.” Remote Sens. 11 (15): 1765. https://doi.org/10.3390/rs11151765.
Gao, J., F. Li, C. Zhang, W. He, J. He, and X. Chen. 2021. “A method of D-type weld seam extraction based on point clouds.” IEEE Access 9 (Feb): 65401–65410. https://doi.org/10.1109/ACCESS.2021.3076006.
Garrido-Jurado, S., R. Muñoz-Salinas, F. J. Madrid-Cuevas, and M. J. Marín-Jiménez. 2014. “Automatic generation and detection of highly reliable fiducial markers under occlusion.” Pattern Recognit. 47 (6): 2280–2292. https://doi.org/10.1016/j.patcog.2014.01.005.
Garrido-Jurado, S., R. Muñoz-Salinas, F. J. Madrid-Cuevas, and R. Medina-Carnicer. 2016. “Generation of fiducial marker dictionaries using mixed integer linear programming.” Pattern Recognit. 51 (May): 481–491. https://doi.org/10.1016/j.patcog.2015.09.023.
Goda, I., G. L’Hostis, and P. Guerlain. 2019. “In-situ non-contact 3D optical deformation measurement of large capacity composite tank based on close-range photogrammetry.” Opt. Lasers Eng. 119 (Jan): 37–55. https://doi.org/10.1016/j.optlaseng.2019.02.006.
Guo, M., Y. Zhou, J. Zhao, T. Zhou, B. Yan, and X. Huang. 2021. “Urban Geospatial Information Acquisition Mobile Mapping System based on close-range photogrammetry and IGS site calibration.” Geo-spatial Inf. Sci. 24 (4): 558–579. https://doi.org/10.1080/10095020.2021.1924084.
Hamledari, H., S. Sajedi, B. McCabe, and M. Fischer. 2020. “Automation of inspection mission planning using 4D BIMs and in support of unmanned aerial vehicle–based data collection.” J. Constr. Eng. Manage. 147 (3): 04020179. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001995.
Han, K., J. Gwak, M. Golparvar-Fard, K. Saidi, G. Cheok, M. Franaszek, and R. Lipman. 2013. “Vision-based field inspection of concrete reinforcing bars.” In Proc., 13th Int. Conf. on Construction Applications of Virtual Reality, 30–31. Middlesbrough, UK: Teesside Univ.
Hurník, J., A. Zatočilová, and D. Paloušek. 2021. “Circular coded target system for industrial applications.” Mach. Vision Appl. 32 (1): 39. https://doi.org/10.1007/s00138-020-01159-1.
Jiang, Y., J. Shu, J. Ye, and W. Zhao. 2023. “Virtual trail assembly of prefabricated structures based on point cloud and BIM.” Autom. Constr. 155 (Jul): 105049. https://doi.org/10.1016/j.autcon.2023.105049.
Kim, M.-K., J. P. P. Thedja, H.-L. Chi, and D.-E. Lee. 2021. “Automated rebar diameter classification using point cloud data based machine learning.” Autom. Constr. 122 (Sep): 103476. https://doi.org/10.1016/j.autcon.2020.103476.
Kim, M.-K., J. P. P. Thedja, and Q. Wang. 2020. “Automated dimensional quality assessment for formwork and rebar of reinforced concrete components using 3D point cloud data.” Autom. Constr. 112 (Jan): 103077. https://doi.org/10.1016/j.autcon.2020.103077.
Li, Y., and C. Liu. 2019. “Applications of multirotor drone technologies in construction management.” Int. J. Constr. Manage. 19 (5): 401–412. https://doi.org/10.1080/15623599.2018.1452101.
Liu, S., H. Zhang, S. Kawai, and H. Nobuhara. 2020. “3D point cloud reconstruction based on stereo camera and RTK-GPS with high accuracy under sequential correction.” In Proc., IEEE 9th Global Conf. on Consumer Electronics (GCCE), 90–93. New York: IEEE.
Liu, X., X. He, Z. Liu, D. He, A. Li, W. He, and X. Peng. 2012. “Automatic registration of range images combined with the system calibration and global ICP.” In Vol. 8499 of Applications of digital image processing XXXV, edited by A. G. Tescher, 84991X. Bellingham, WA: Society of Photo-Optical Instrumentation Engineers. https://doi.org/10.1117/12.929733.
Luhmann, T. 2016. “Learning photogrammetry with interactive software Tool PhoX.” In Proc., ISPRS–Int. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLI-B6, 39–44. Göttingen, Germany: Copernicus GmbH. https://doi.org/10.5194/isprsarchives-XLI-B6-39-2016.
Madeira, T., M. Oliveira, and P. Dias. 2020. “Enhancement of RGB-D image alignment using fiducial markers.” Sensors 20 (5): 1497. https://doi.org/10.3390/s20051497.
Mahami, H., F. Nasirzadeh, A. H. Ahmadabadian, F. Esmaeili, and S. Nahavandi. 2019a. “Imaging network design to improve the automated construction progress monitoring process.” Constr. Innovation 19 (3): 386–404. https://doi.org/10.1108/CI-07-2018-0059.
Mahami, H., F. Nasirzadeh, A. H. Ahmadabadian, and S. Nahavandi. 2019b. “Automated progress controlling and monitoring using daily site images and building information modeling.” Buildings 9 (3): 70. https://doi.org/10.3390/buildings9030070.
Marzulli, M. I., P. Raumonen, R. Greco, M. Persia, and P. Tartarino. 2020. “Estimating tree stem diameters and volume from smartphone photogrammetric point clouds.” For.: Int. J. For. Res. 93 (3): 411–429. https://doi.org/10.1093/forestry/cpz067.
Matiukas, V., and D. Miniotas. 2011. “Point cloud merging for complete 3D surface reconstruction.” Electron. Electr. Eng. 113 (7): 73–76. https://doi.org/10.5755/j01.eee.113.7.616.
Monkman, G. G., K. Hyder, M. J. Kaiser, and F. P. Vidal. 2019. “Using machine vision to estimate fish length from images using regional convolutional neural networks.” Methods Ecol. Evol. 10 (12): 2045–2056. https://doi.org/10.1111/2041-210X.13282.
Murtiyoso, A., E. Pellis, P. Grussenmeyer, T. Landes, and A. Masiero. 2022. “Towards semantic photogrammetry: Generating semantically rich point clouds from architectural close-range photogrammetry.” Sensors 22 (3): 966. https://doi.org/10.3390/s22030966.
Omar, H., L. Mahdjoubi, and G. Kheder. 2018. “Towards an automated photogrammetry-based approach for monitoring and controlling construction site activities.” Comput. Ind. 98 (Jun): 172–182. https://doi.org/10.1016/j.compind.2018.03.012.
Pan, Y., Y. Dong, D. Wang, A. Chen, and Z. Ye. 2019. “Three-dimensional reconstruction of structural surface model of heritage bridges using UAV-based photogrammetric point clouds.” Remote Sens. 11 (10): 1204. https://doi.org/10.3390/rs11101204.
pix4d. 2017. “Offline getting started and manual (pdf).” Accessed September 8, 2023. https://support.pix4d.com/hc/en-us/articles/204272989-Offline-Getting-Started-and-Manual-pdf#gsctab=0.
Puliti, M., G. Montaggioli, and A. Sabato. 2021. “Automated subsurface defects’ detection using point cloud reconstruction from infrared images.” Autom. Constr. 129 (Feb): 103829. https://doi.org/10.1016/j.autcon.2021.103829.
Qureshi, A. H. 2022a. “Video1: ASCM operation for data set 1.” Accessed December 8, 2023. https://www.youtube.com/watch?v=EB3Z2bCfzOM.
Qureshi, A. H. 2022b. “Video2: ASCM operation for data set 4.” Accessed December 8, 2023. https://www.youtube.com/watch?v=YstBcMm0fYk.
Qureshi, A. H., W. S. Alaloul, S. J. Hussain, A. Murtiyoso, S. Saad, K. M. Alzubi, S. Ammad, and A. O. Baarimah. 2022a. “Evaluation of photogrammetry tools following progress detection of rebar towards sustainable construction processes.” Sustainability 15 (1): 21. https://doi.org/10.3390/su15010021.
Qureshi, A. H., W. S. Alaloul, A. Murtiyoso, S. Saad, and B. Manzoor. 2022b. “Comparison of Photogrammetry Tools Considering Rebar Progress Recognition.” In Vol. 2 of Proc., Int. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 141–146. Göttingen, Germany: Copernicus GmbH. https://doi.org/10.5194/isprs-archives-XLIII-B2-2022-141-2022.
Qureshi, A. H., W. S. Alaloul, W. K. Wing, S. Saad, S. Ammad, and M. Altaf. 2023a. “Characteristics-based framework of effective automated monitoring parameters in construction projects.” Arabian J. Sci. Eng. 48 (4): 4731–4749. https://doi.org/10.1007/s13369-022-07172-y.
Qureshi, A. H., W. S. Alaloul, W. K. Wing, S. Saad, S. Ammad, and M. A. Musarat. 2022c. “Factors impacting the implementation process of automated construction progress monitoring.” Ain Shams Eng. J. 13 (6): 101808. https://doi.org/10.1016/j.asej.2022.101808.
Qureshi, A. H., W. S. Alaloul, W. K. Wing, S. Saad, M. A. Musarat, S. Ammad, and A. F. Kineber. 2023b. “Automated progress monitoring technological model for construction projects.” Ain Shams Eng. J. 14 (10): 102165. https://doi.org/10.1016/j.asej.2023.102165.
Rahimian, F. P., S. Seyedzadeh, S. Oliver, S. Rodriguez, and N. Dawood. 2020. “On-demand monitoring of construction projects through a game-like hybrid application of BIM and machine learning.” Autom. Constr. 110 (Oct): 103012. https://doi.org/10.1016/j.autcon.2019.103012.
Reljić, I., I. Dunder, and S. Seljan. 2019. “Photogrammetric 3D scanning of physical objects: Tools and workflow.” TEM J. 8 (2): 383–388. https://doi.org/10.18421/TEM82-09.
Romero-Ramirez, F. J., R. Muñoz-Salinas, and R. Medina-Carnicer. 2018. “Speeded up detection of squared fiducial markers.” Image Vision Comput. 76 (May): 38–47. https://doi.org/10.1016/j.imavis.2018.05.004.
Ruescas Nicolau, A. V., H. De Rosario, F. Basso Della-Vedova, E. Parrilla Bernabé, M. C. Juan, and J. López-Pascual. 2022. “Accuracy of a 3D temporal scanning system for gait analysis: Comparative with a marker-based photogrammetry system.” Gait Posture 97 (Mar): 28–34. https://doi.org/10.1016/j.gaitpost.2022.07.001.
Siki, Z., and B. Takács. 2021. “Automatic recognition of ArUco codes in land surveying tasks.” Baltic J. Mod. Comput. 9 (1): 115–125. https://doi.org/10.22364/bjmc.2021.9.1.06.
Štroner, M., R. Urban, J. Seidl, T. Reindl, and J. Brouček. 2021. “Photogrammetry using UAV-mounted GNSS RTK: Georeferencing strategies without GCPs.” Remote Sens. 13 (7): 1336. https://doi.org/10.3390/rs13071336.
Tungol, Z. P. L., H. Toriya, N. Owada, I. Kitahara, F. Inagaki, M. Saadat, H. D. Jang, and Y. Kawamura. 2021. “Model scaling in smartphone gnss-aided photogrammetry for fragmentation size distribution estimation.” Minerals 11 (12): 1301. https://doi.org/10.3390/min11121301.
Turkan, Y., F. Bosché, C. T. Haas, and R. Haas. 2014. “Tracking of secondary and temporary objects in structural concrete work.” Constr. Innovation 14 (2): 145–167. https://doi.org/10.1108/CI-12-2012-0063.
Tuttas, S., A. Braun, A. Borrmann, and U. Stilla. 2017. “Acquisition and consecutive registration of photogrammetric point clouds for construction progress monitoring using a 4D BIM.” J. Photogramm. Remote Sens. Geoinf. Sci. 85 (1): 3–15. https://doi.org/10.1007/s41064-016-0002-z.
Wang, Q., J. C. P. Cheng, and H. Sohn. 2017. “Automated estimation of reinforced precast concrete rebar positions using colored laser scan data.” Comput.-Aided Civ. Infrastruct. Eng. 32 (9): 787–802. https://doi.org/10.1111/mice.12293.
Wang, Y., Y. Liu, Q. Xie, Q. Wu, X. Guo, Z. Yu, and J. Wang. 2021. “Density-invariant registration of multiple scans for aircraft measurement.” IEEE Trans. Instrum. Meas. 70 (Jun): 1–15. https://doi.org/10.1109/TIM.2020.3016410.
Xiao, Z., J. Liang, D. Yu, and A. Asundi. 2011. “Large field-of-view deformation measurement for transmission tower based on close-range photogrammetry.” Measurement 44 (9): 1705–1712. https://doi.org/10.1016/j.measurement.2011.07.009.
Xiao, Z., J. Liang, D. Yu, and J. Liu. 2010. “Rapid three-dimension optical deformation measurement for transmission tower with different loads.” Opt. Lasers Eng. 48 (9): 869–876. https://doi.org/10.1016/j.optlaseng.2010.04.003.
Yang, Z., and Y. Han. 2020. “A low-cost 3D phenotype measurement method of leafy vegetables using video recordings from smartphones.” Sensors 20 (21): 1–15. https://doi.org/10.3390/s20216068.
Yoo, J. S., D. H. Kim, and G. W. Kim. 2018. “Improved LiDAR-camera calibration using marker detection based on 3D plane extraction.” J. Electr. Eng. Technol. 13 (6): 2530–2544. https://doi.org/10.5370/JEET.2018.13.6.2530.
Yu, K., C. Zhang, M. Shooshtarian, W. Zhao, and J. Shu. 2021. “Automated finite element modeling and analysis of cracked reinforced concrete beams from three dimensional point cloud.” Struct. Concr. 22 (6): 3213–3227. https://doi.org/10.1002/suco.202100194.
Yu, R., P. Li, J. Shan, and H. Zhu. 2022. “Structural state estimation of earthquake-damaged building structures by using UAV photogrammetry and point cloud segmentation.” Meas.: J. Int. Meas. Confederation 202 (Aug): 111858. https://doi.org/10.1016/j.measurement.2022.111858.
Yuan, X., A. Smith, F. Moreu, R. Sarlo, C. D. Lippitt, M. Hojati, S. Alampalli, and S. Zhang. 2023. “Automatic evaluation of rebar spacing and quality using LiDAR data: Field application for bridge structural assessment.” Autom. Constr. 146 (Dec): 104708. https://doi.org/10.1016/j.autcon.2022.104708.
Yuan, X., A. Smith, R. Sarlo, C. D. Lippitt, and F. Moreu. 2021. “Automatic evaluation of rebar spacing using LiDAR data.” Autom. Constr. 131 (Aug): 103890. https://doi.org/10.1016/j.autcon.2021.103890.
Zhao, W., Y. Jiang, Y. Liu, and J. Shu. 2022. “Automated recognition and measurement based on three-dimensional point clouds to connect precast concrete components.” Autom. Constr. 133 (Nov): 104000. https://doi.org/10.1016/j.autcon.2021.104000.
Zhu, H., B.-G. Hwang, J. Ngo, and J. P. S. Tan. 2022. “Applications of smart technologies in construction project management.” J. Constr. Eng. Manage. 148 (4): 1–12. https://doi.org/10.1061/(ASCE)CO.1943-7862.0002260.
Zhu, H., W. Wu, J. Chen, G. Ma, X. Liu, and X. Zhuang. 2016. “Integration of three dimensional discontinuous deformation analysis (DDA) with binocular photogrammetry for stability analysis of tunnels in blocky rockmass.” Tunnelling Underground Space Technol. 51 (Jun): 30–40. https://doi.org/10.1016/j.tust.2015.10.012.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 150 • Issue 3 • March 2024
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Jul 18, 2023
Accepted: Oct 19, 2023
Published online: Dec 26, 2023
Published in print: Mar 1, 2024
Discussion open until: May 26, 2024
ASCE Technical Topics:
- Automation and robotics
- Construction engineering
- Construction methods
- Continuum mechanics
- Dynamics (solid mechanics)
- Engineering fundamentals
- Engineering materials (by type)
- Engineering mechanics
- Environmental engineering
- Errors (statistics)
- Fouling
- Geomatics
- Materials engineering
- Mathematics
- Metals (material)
- Models (by type)
- Motion (dynamics)
- Photogrammetry
- Reinforcing steel
- Solid mechanics
- Statistics
- Steel
- Surveying methods
- Systems engineering
- Three-dimensional models
- Uncertainty principles
- Waste management
- Waste treatment
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.