Automated Part Placement for Precast Concrete Component Manufacturing: An Intelligent Robotic System Using Target Detection and Path Planning
Publication: Journal of Computing in Civil Engineering
Volume 39, Issue 1
Abstract
Placing embedded parts (EPs), e.g., junction boxes or plastic cable ducts, in a precast concrete (PC) component is a fundamental and repetitive trade in its manufacturing. Yet, such trade is far from being automated to enhance PC component manufacturing productivity. This study presents an intelligent robotic system for automated part placement for PC component manufacturing by using target detection and path planning. The proposed system consists of an Aubo-i5 robotic arm, a Robotiq 2F-85 clamping claw, and an Intel Realsense D435i depth camera. An improved YOLOv5 target detection algorithm is proposed to automatically detect EPs with high precision, and a two-way two-threaded informed RRT* path planning algorithm is developed to optimize the robot movement. Using junction box placement as an experiment, performance of the system was evaluated by examining EP detection, clamping, path planning, and placement. The visual detection model achieved a mAP value of 99.5%. The efficiency of the path planning algorithm was improved by 37.7% compared with Bidirectional RRT* with close pathfinding quality. The final success rate of EP placement reached 99.8%. The research contributes to the field of PC component production by providing an automated system for EPs placement.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data that support the findings of this study is available from the corresponding author upon reasonable request.
Acknowledgments
The study was conducted with the support of the Shenzhen Newly Introduced High-end Talents Scientific Research Start-up Project (Grant No. 827000656).
Author contributions: Huanyu Wu: methodology, conceptualization, supervision, and writing–review and editing. Wei Zhang: investigation, visualization, methodology, and writing–original draft. Weisheng Lu: supervision and writing–review and editing. Junjie Chen: writing–original draft and review and editing. Jianqiu Bao: investigation, visualization, and methodology. Yongqi Liu: investigation and conceptualization.
References
Ahn, S., S. Han, and M. Al-Hussein. 2019. “2D drawing visualization framework for applying projection-based augmented reality in a panelized construction manufacturing facility: Proof of concept.” J. Comput. Civ. Eng. 33 (5): 04019032. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000843.
Avermann. 2023. “Precast concrete technology.” Accessed April 5, 2023. https://beton.avermann.de/zh/.
Balafas, V., E. Karantoumanis, M. Louta, and N. Ploskas. 2023. “Machine learning and deep learning for plant disease classification and detection.” IEEE Access 11 (Sep): 114352–114377. https://doi.org/10.1109/ACCESS.2023.3324722.
Cen, H. 2023. “Target location detection of mobile robots based on R-FCN deep convolutional neural network.” Int. J. Syst. Assur. Eng. Manage. 14 (2): 728–737. https://doi.org/10.1007/s13198-021-01514-z.
Chea, C. P., Y. Bai, X. B. Pan, M. Arashpour, and Y. P. Xie. 2020. “An integrated review of automation and robotic technologies for structural prefabrication and construction.” Transp. Saf. Environ. 2 (2): 81–96. https://doi.org/10.1093/tse/tdaa007.
Du, G. G., K. Wang, S. G. Lian, and K. Y. Zhao. 2021. “Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: A review.” Artif. Intell. Rev. 54 (3): 1677–1734. https://doi.org/10.1007/s10462-020-09888-5.
Du, J., J. Zhang, D. Castro-Lacouture, and Y. Hu. 2023. “Lean manufacturing applications in prefabricated construction projects.” Autom. Constr. 150 (Jun): 104790. https://doi.org/10.1016/j.autcon.2023.104790.
Ebawe. 2023. “Concrete precast component production equipment.” Accessed January 10, 2023. https://www.ebawe.de/zh/.
Ergen, E., B. Akinci, and R. Sacks. 2007. “Tracking and locating components in a precast storage yard utilizing radio frequency identification technology and GPS.” Autom. Constr. 16 (3): 354–367. https://doi.org/10.1016/j.autcon.2006.07.004.
Fang, B., X. Ma, J. Wang, and F. Sun. 2020. “Vision-based posture-consistent teleoperation of robotic arm using multi-stage deep neural network.” Rob. Auton. Syst. 131 (Sep): 103592. https://doi.org/10.1016/j.robot.2020.103592.
Gammell, J. D., S. S. Srinivasa, and T. D. Barfoot. 2014. “Informed RRT*: Optimal sampling-based path planning focused via direct sampling of an admissible ellipsoidal heuristic. In Proc., IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 2997–3004. New York: IEEE.
Guo, T. Z., J. Wang, W. Chen, Z. M. Wang, G. J. Chen, Y. L. Lin, and J. H. Liu. 2024. “Obstacle avoidance path planning for mobile robotic arm with multi-degree of freedom based on TBPS-RRT algorithm.” Hum.-Centric Comput. Inf. Sci. 14 (Feb): 008. https://doi.org/10.22967/hcis.2024.14.008.
Hu, C., H. Wang, and H. Shi. 2021. “Robotic arm reinforcement learning control method based on autonomous visual perception.” Xibei Gongye Daxue Xuebao/J. Northwest. Polytech. Univ. 39 (5): 1057–1063. https://doi.org/10.1051/jnwpu/20213951057.
Hwang, P. J., C. C. Hsu, P. Y. Chou, W. Y. Wang, and C. H. Lin. 2022. “Vision-based learning from demonstration system for robot arms.” Sensors 22 (7): 2678. https://doi.org/10.3390/s22072678.
Ilyas, M., H. Y. Khaw, N. M. Selvaraj, Y. Jin, X. Zhao, and C. C. Cheah. 2021. “Robot-assisted object detection for construction automation: Data and information-driven approach.” IEEE/ASME Trans. Mechatron. 26 (6): 2845–2856. https://doi.org/10.1109/TMECH.2021.3100306.
Jhang, J.-Y., C.-J. Lin, and K.-Y. Young. 2019. “Cooperative carrying control for multi-evolutionary mobile robots in unknown environments.” Electronics 8 (3): 298. https://doi.org/10.3390/electronics8030298.
Jiao, L. C., F. Zhang, F. Liu, S. Y. Yang, L. L. Li, Z. X. Feng, and R. Qu. 2019. “A survey of deep learning-based object detection.” IEEE Access 7 (May): 128837–128868. https://doi.org/10.1109/ACCESS.2019.2939201.
Jordan, M., and A. Perez. 2013. Optimal bidirectional rapidly-exploring random trees. Cambridge, MA: Massachusetts Institute of Technology.
Karaman, S., M. R. Walter, A. Perez, E. Frazzoli, and S. Teller. 2011. “Anytime motion planning using the RRT*.” In Proc., IEEE Int. Conf. on Robotics and Automation, 1478–1483. New York: IEEE.
Kim, M.-K., Q. Wang, J.-W. Park, J. C. P. Cheng, H. Sohn, and C.-C. Chang. 2016. “Automated dimensional quality assurance of full-scale precast concrete elements using laser scanning and BIM.” Autom. Constr. 72 (Dec): 102–114. https://doi.org/10.1016/j.autcon.2016.08.035.
LaValle, S. M. 1998. “Rapidly-exploring random trees: A new tool for path planning.” In The annual research report. Ames, IA: Iowa State Univ.
Li, C., M. M. Shu, L. Du, H. Y. Tan, L. Wei, and I. C. Soc. 2022a. “Design of automatic recycling robot based on YOLO target detection.” In Proc., Asia Conf. on Algorithms, Computing and Machine Learning (CACML), 310–315. Oxfordshire, UK: Taylor & Francise.
Li, D., X. Li, H. Feng, Y. Wang, and S. Fan. 2022b. “ISM-based relationship among critical factors that affect the choice of prefabricated concrete buildings in China.” Int. J. Construct. Manage. 22 (6): 977–992. https://doi.org/10.1080/15623599.2019.1675306.
Liang, Y., G. Chen, S. Li, and Z. Xu. 2023. “Intelligent defect diagnosis of appearance quality for prefabricated concrete components based on target detection and multimodal fusion decision.” J. Comput. Civ. Eng. 37 (6): 04023032. https://doi.org/10.1061/JCCEE5.CPENG-5460.
Mohammad Hossein Fallah, M., and F. Janabi-Sharifi. 2021. “Conjugated visual predictive control for constrained visual servoing.” J. Intell. Rob. Syst. 101 (2): 33. https://doi.org/10.1007/s10846-020-01299-6.
Ning, Y. M., T. J. Li, C. Yao, W. Q. Du, and Y. Zhang. 2024. “HMS-RRT: A novel hybrid multi-strategy rapidly-exploring random tree algorithm for multi-robot collaborative exploration in unknown environments.” Expert Syst. Appl. 247 (Aug): 123238. https://doi.org/10.1016/j.eswa.2024.123238.
Ourak, M., B. Tamadazte, O. Lehmann, and N. Andreff. 2019. “Direct visual servoing using wavelet coefficients.” IEEE/ASME Trans. Mechatron. 24 (3): 1129–1140. https://doi.org/10.1109/TMECH.2019.2898509.
Terven, J., D. M. Córdova-Esparza, and J. A. Romero-González. 2023. “A comprehensive review of YOLO architectures in computer vision: From YOLOv1 to YOLOv8 and YOLO-NAS.” Mach. Learn. Knowl. Extr. 5 (4): 1680–1716. https://doi.org/10.3390/make5040083.
Vollert. 2023. “Precast construction machinery.” Accessed April 5, 2023. https://www.vollert.de/en/home/.
Wang, Z. L., H. Li, and X. L. Zhang. 2019. “Construction waste recycling robot for nails and screws: Computer vision technology and neural network approach.” Autom. Constr. 97 (Jan): 220–228. https://doi.org/10.1016/j.autcon.2018.11.009.
Wareham, T. 2019. “Designing robot teams for distributed construction, repair, and maintenance.” ACM Trans. Auton. Adapt. Syst. 14 (1): 1–29. https://doi.org/10.1145/3337797.
Wei, A. H., and B. Y. Chen. 2020. “Robotic object recognition and grasping with a natural background.” Int. J. Adv. Rob. Syst. 17 (2): 1729881420921102. https://doi.org/10.1177/1729881420921102.
Wu, M. H., H. H. Yue, J. Wang, Y. X. Huang, M. Liu, Y. H. Jiang, C. Ke, and C. Zeng. 2020. “Object detection based on RGC mask R-CNN.” IET Image Proc. 14 (8): 1502–1508. https://doi.org/10.1049/iet-ipr.2019.0057.
Xu, Z., R. Kang, and R. Lu. 2020. “3D reconstruction and measurement of surface defects in prefabricated elements using point clouds.” J. Comput. Civ. Eng. 34 (5): 04020033. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000920.
Yu, X., S. Zhang, L. Sun, Y. Wang, C. Xue, and B. Li. 2020. “Cooperative control of dual-arm robots in different human-robot collaborative tasks.” Assem. Autom. 40 (1): 95–104. https://doi.org/10.1108/AA-12-2018-0264.
Yuan, Z., C. Sun, and Y. Wang. 2018. “Design for Manufacture and Assembly-oriented parametric design of prefabricated buildings.” Autom. Constr. 88 (Apr): 13–22. https://doi.org/10.1016/j.autcon.2017.12.021.
Yue, X. B., H. Y. Li, M. Shimizu, S. Kawamura, and L. Meng. 2022. “YOLO-GD: A deep learning-based object detection algorithm for empty-dish recycling robots.” Machines 10 (5): 294. https://doi.org/10.3390/machines10050294.
Information & Authors
Information
Published In
Journal of Computing in Civil Engineering
Volume 39 • Issue 1 • January 2025
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Jan 13, 2024
Accepted: Jul 2, 2024
Published online: Sep 24, 2024
Published in print: Jan 1, 2025
Discussion open until: Feb 24, 2025
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.