Robotic Assembly of Interlocking Blocks for Construction Based on Large Language Models
Publication: Construction Research Congress 2024
ABSTRACT
Using robots to automate construction processes can enhance productivity and safety and reduce cost. However, the construction robots haven’t been widely accepted primarily due to two challenges: lack of data and computational resources to train robots for performing specific construction tasks, and lack of the capability for robots to deal with new tasks. This paper introduces a framework leveraging large language models (LLMs) to enable robots to perform assembly tasks based on in situ worker instructions without extensive prior robot training. By using predefined prompts tailored to specific tasks, the LLM can interpret worker instructions and generate corresponding action sequences and robot functions without additional training. Hence, this LLM-based approach can effectively reduce training costs and facilitate robot adaptability for new tasks and environments through simple prompt modification. To validate this framework, several interlocking block assembly tasks performed by LLM-based robots were evaluated. It is found simple instructions were accurately executed by the robots, but future research is needed to improve the model to handle complex instructions from humans. Overall, this study demonstrates the potential of LLM-based robots in performing assembly tasks in construction with human collaboration.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Balaguer, C., Gambao, E., Barrientos, A., Puente, E. A., and Aracil, R. (1996). “Site Assembly in Construction Industry by means of a Large Range Advanced Robot,” Proceedings of the 13th International Symposium on Automation and Robotics in Construction, International Association for Automation and Robotics in Construction (IAARC), 65–72. https://doi.org/10.22260/ISARC1996/0010.
Belousov, B., Wibranek, B., Schneider, J., Schneider, T., Chalvatzaki, G., Peters, J., and Tessmann, O. (2022). “Robotic architectural assembly with tactile skills: Simulation and optimization,” Automation in Construction, Elsevier, 133, 104006. https://doi.org/10.1016/J.AUTCON.2021.104006.
Benyahia, K., Seriket, H., Prod’hon, R., Gomes, S., André, J. C., Qi, H. J., and Demoly, F. (2022). “A computational design approach for multi-material 4D printing based on interlocking blocks assembly,” Additive Manufacturing, Elsevier, 58, 102993. https://doi.org/10.1016/J.ADDMA.2022.102993.
Changali, S., Mohammad, A., and Nieuwland, M. (2005). “The construction productivity imperative,” Available athttps://www.mckinsey.com/capabilities/operations/our-insights/the-construction-productivity-imperative.
Coumans, E., and Bai, Y. (2016). Pybullet, a python module for physics simulation for games, robotics and machine learning.
Ding, Y., Zhang, X., Paxton, C., and Zhang, S. (2023). Task and Motion Planning with Large Language Models for Object Rearrangement.
Ferdous, W., Bai, Y., Ngo, T. D., Manalo, A., and Mendis, P. (2019). “New advancements, challenges and opportunities of multi-storey modular buildings – A state-of-the-art review,” Engineering Structures, Elsevier, 183, 883–893. https://doi.org/10.1016/J.ENGSTRUCT.2019.01.061.
Fu, C. W., Song, P., Yan, X., Yang, L. W., Jayaraman, P. K., and Cohen-Or, D. (2015). “Computational interlocking furniture assembly,” ACM Transactions on Graphics (TOG), ACM PUB27 New York, NY, USA, 34. https://doi.org/10.1145/2766892.
Funk, N., Chalvatzaki, G., Belousov, B., and Peters, J. (2022). “Learn2assemble with structured representations and search for robotic architectural construction,” in Conference on Robot Learning, PMLR, pp. 1401–1411.
González-Santamarta, M. A., Fernández-Becerra, L., Sobrín-Hidalgo, D., Guerrero-Higueras, Á. M., González, I., and Lera, F. J. R. (2023). Using Large Language Models for Interpreting Autonomous Robots Behaviors.
Hauser, S., Mutlu, M., Léziart, P. A., Khodr, H., Bernardino, A., and Ijspeert, A. J. (2020). “Roombots extended: Challenges in the next generation of self-reconfigurable modular robots and their application in adaptive and assistive furniture,” Robotics and Autonomous Systems, North-Holland, 127, 103467. https://doi.org/10.1016/J.ROBOT.2020.103467.
Inagaki, T., Kato, A., Takahashi, K., Ozaki, H., and Kanda, G. N. (2023). LLMs can generate robotic scripts from goal-oriented instructions in biological laboratory automation.
Javan, A. R., Seifi, H., Xu, S., and Xie, Y. M. (2016). “Design of a new type of interlocking brick and evaluation of its dynamic performance,” in Proceedings of the International Association for Shell and Spatial Structures Annual Symposium, International Association for Shell and Spatial Structures (IASS), pp. 1–8.
Koubaa, A. (2023). “ROSGPT: Next-Generation Human-Robot Interaction with ChatGPT and ROS,”. https://doi.org/10.20944/PREPRINTS202304.0827.V3.
Liang, J., Huang, W., Xia, F., Xu, P., Hausman, K., Ichter, B., Florence, P., and Zeng, A. (2022). Code as Policies: Language Model Programs for Embodied Control.
Liu, Y., Belousov, B., Funk, N., Chalvatzaki, G., Peters, J., and Tessmann, O. (2023). “Auto(mated)nomous Assembly,” Lecture Notes in Civil Engineering, Springer Science and Business Media Deutschland GmbH, 306 LNCE, 167–181. https://doi.org/10.1007/978-3-031-20241-4_12/COVER.
Mai, J., Chen, J., Li, B., Qian, G., Elhoseiny, M., and Ghanem, B. (2023). LLM as A Robotic Brain: Unifying Egocentric Memory and Control.
Mirkhalaf, M., Zhou, T., and Barthelat, F. (2018). “Simultaneous improvements of strength and toughness in topologically interlocked ceramics,” Proceedings of the National Academy of Sciences of the United States of America, National Academy of Sciences, 115, 9128–9133. https://doi.org/10.1073/PNAS.1807272115/SUPPL_FILE/PNAS.1807272115.SAPP.PDF.
Reynolds, A. (2022). “Challenges and opportunities in the post Covid-19 world,” Available athttps://www.rlb.com/oceania/insight/challenges-and-opportunities-in-the-post-covid-19-world/.
Saidi, K. S., Bock, T., and Georgoulas, C. (2016). “Robotics in Construction,” Springer Handbooks, Springer Science and Business Media Deutschland GmbH, 1493–1520. https://doi.org/10.1007/978-3-319-32552-1_57/FIGURES/34.
Wang, Z., Song, P., and Pauly, M. (2018). “DESIA: a general framework for designing interlocking assemblies,” ACM Transactions on Graphics (TOG), ACM PUB27 New York, NY, USA, 37. https://doi.org/10.1145/3272127.3275034.
Yokoi, K., Nakashima, K., Kobayashi, M., Mihune, H., Hasunuma, H., Yanagihara, Y., Ueno, T., Gokyuu, T., and Endou, K. (2016). “A Tele-operated Humanoid Operator,” http://dx.doi.org/10.1177/0278364906065900, SAGE Publications, 25, 593–602. https://doi.org/10.1177/0278364906065900.
You, H., Ye, Y., Zhou, T., Zhu, Q., and Du, J. (2023). Robot-Enabled Construction Assembly with Automated Sequence Planning based on ChatGPT: RoboGPT, Doi Number. https://doi.org/10.1109/ACCESS.2022.Doi.
Yu, S. N., Ryu, B. G., Lim, S. J., Kim, C. J., Kang, M. K., and Han, C. S. (2009). “Feasibility verification of brick-laying robot using manipulation trajectory and the laying pattern optimization,” Automation in Construction, Elsevier, 18, 644–655. https://doi.org/10.1016/J.AUTCON.2008.12.008.
Zhang, Y., Koga, Y., and Balkcom, D. (2021). “Interlocking Block Assembly with Robots,” IEEE Transactions on Automation Science and Engineering, Institute of Electrical and Electronics Engineers Inc., 18, 902–916. https://doi.org/10.1109/TASE.2021.3069742.
Information & Authors
Information
Published In
Construction Research Congress 2024
Pages: 777 - 786
History
Published online: Mar 18, 2024
ASCE Technical Topics:
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.