Computer Vision-Based Geometry Mapping and Matching of Building Elements for Construction Robotic Applications
Publication: Construction Research Congress 2022
ABSTRACT
Robotic automation of construction tasks is a growing area of research. For robots to successfully operate in a construction environment, sensing technology must be developed which allows for accurate detection of site geometry in a wide range of conditions. Much of the existing body of research on computer vision systems for construction automation focuses on pick-and-place operations such as stacking blocks or placing masonry elements. Very little research has focused on framing and related tasks. The research presented here aims to address this gap by designing and implementing computer vision algorithms for detection and measurement of building framing elements and testing those algorithms using realistic framing structures. These algorithms allow for a stationary RGB-D camera to accurately detect, identify, and measure the geometry of framing elements in a construction environment and match the detected geometry to provided building information modeling (BIM) data. The algorithms reduce identified framing elements to a simplified 3D geometric model, which allows for robust and accurate measurement and comparison with BIM data. This data can then be used to direct operations of construction robotic systems or other machines/equipment. The proposed algorithms were tested in a laboratory setting using an Intel RealSense D455 RGB-D camera, and initial results indicate that the system is capable of measuring the geometry of timber-frame structures with accuracy on the order of a few centimeters.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Akanbi, T., Zhang, J., and Lee, Y.-C. (2020). “Data-Driven Reverse Engineering Algorithm Development Method for Developing Interoperable Quantity Takeoff Algorithms Using IFC-Based BIM.” Journal of Computing in Civil Engineering, 34(5), 04020036.
Dawod, M., and Hanna, S. (2019). “BIM-assisted Object Recognition for the On-Site Autonomous Robotic Assembly of Discrete Structures.” Construction Robotics, 3(1-4), 69–81.
Deng, H., Hong, H., Luo, D., Deng, Y., and Su, C. (2020). “Automatic Indoor Construction Process Monitoring for Tiles Based on BIM and Computer Vision.” Journal of Construction Engineering and Management, 146(1), 04019095.
Dubuisson, M.-P., and Jain, A. K. (1994). “A Modified Hausdorff Distance for Object Matching.” Proceedings of 12th International Conference on Pattern Recognition, IEEE, Piscataway, NJ, 566–568.
Feng, C., Taguchi, Y., and Kamat, V. R. (2014). “Fast Plane Extraction in Organized Point Clouds Using Agglomerative Hierarchical Clustering.” 2014 IEEE International Conference on Robotics and Automation (ICRA), IEEE, Piscataway, NJ.
Feng, C., Xiao, Y., Willette, A., McGee, W., and Kamat, V. R. (2015). “Vision guided Autonomous Robotic Assembly and As-Built Scanning on Unstructured Construction Sites.” Automation in Construction, 59, 128–138.
Golparvar-Fard, M., Pena-Mora, F., and Savarese, S. (2011). “Integrated Sequential As-Built and As-Planned Representation with D4AR Tools in Support of Decision-Making Tasks in the AEC/FM Industry.” Journal of Construction Engineering and Management, 137(12), 1099–1116.
Grunnet-Jepsen, A., Sweetser, J., and Woodfill, J. (2020). “Tuning Depth Cameras for Best Performance.” <https://dev.intelrealsense.com/docs/tuning-depth-cameras-for-best-performance>(May 24, 2021).
Han, K., Degol, J., and Golparvar-Fard, M. (2018). “Geometry- and Appearance-Based Reasoning of Construction Progress Monitoring.” Journal of Construction Engineering and Management, 144(2): 04017110.
Intel Corporation. (2021). “Intel® RealSense™ SDK 2.0.” <https://github.com/IntelRealSense/librealsense>(June 9, 2021).
Li, L., Yang, F., Zhu, H., Li, D., Li, Y., and Tang, L. (2017). “An Improved RANSAC for 3D Point Cloud Plane Segmentation Based on Normal Distribution Transformation Cells.” Remote Sensing, 9(5), 433.
Pérez, L., Rodríguez, Í., Rodríguez, N., Usamentiaga, R., and García, D. (2016). “Robot Guidance Using Machine Vision Techniques in Industrial Environments: A Comparative Review.” Sensors, 16(3), 335.
Python Software Foundation. (2021a). “opencv-python 4.5.2.54.” <https://pypi.org/project/opencv-python/>(June 9, 2021).
Python Software Foundation. (2021b). “Python 3.7.10.” <https://www.python.org/downloads/release/python-3710/>(June 9, 2021).
Schwarz, M., Milan, A., Periyasamy, A. S., and Behnke, S. (2017). “RGB-D Object Detection and Semantic Segmentation for Autonomous Manipulation in Clutter.” The International Journal of Robotics Research, 37(4-5), 437–451.
Tish, D., King, N., and Cote, N. (2020). “Highly Accessible Platform Technologies for Vision-Guided, Closed-Loop Robotic Assembly of Unitized Enclosure Systems.” Construction Robotics, 4(1-2), 19–29.
Troncoso-Pastoriza, F., López-Gómez, J., and Febrero-Garrido, L. (2018). “Generalized Vision-Based Detection, Identification and Pose Estimation of Lamps for BIM Integration.” Sensors, 18(7), 2364.
Vähä, P., Heikkilä, T., Kilpeläinen, P., Järviluoma, M., and Gambao, E. (2013). “Extending Automation of Building Construction — Survey on Potential Sensor Technologies and Robotic Applications.” Automation in Construction, 36, 168–178.
Wang, J., Garratt, M., and Anavatti, S. (2015). “Dominant Plane Detection Using a RGB-D Camera for Autonomous Navigation.” 2015 6th International Conference on Automation, Robotics and Applications (ICARA).
Zhang, J. (2018). “Towards Systematic Understanding of Geometric Representations in BIM Standard: An Empirical Data-Driven Approach.” Proc., ASCE Construction Research Congress, ASCE, Reston, VA, 96–105.
Zhang, J., and El-Gohary, N. M. (2015). “Automated Extraction of Information from Building Information Models into a Semantic Logic-Based Representation.” Proc., 2015 International Workshop on Computing in Civil Engineering., ASCE, Reston, VA, 173–180.
Information & Authors
Information
Published In
History
Published online: Mar 7, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.