Intelligent Hoisting with Car-Like Mobile Robots
Publication: Journal of Construction Engineering and Management
Volume 146, Issue 12
Abstract
The construction industry’s traditional hoisting system always needs workers to complete the tasks involved, with concomitant extra labor costs and attention to the workers’ safety. This paper describes the development of an intelligent hoisting system to optimize the hoisting process, involving the application of robotic cars and vision-based recognition. In the design of the hoisting system, electric hooks were used and maneuvered by a robotic car while the vision-based recognition system—based on capturing images by the camera—arranges the robotic motion. The Yolo v2 recognition algorithm was used, which provides fast and efficient vision-based recognition. More than 30 trials in an experimental prefabrication factory indicated that the system had a significant success rate of approximately 92.5% (3.7/4)—the proportion of hooks successfully grappling the hoist points—verifying the feasibility of the system. The primary contribution of this paper is in the development and demonstration of an intelligent hoisting system to optimize the hoisting process, involving the application of robotic cars and vision-based recognition, thus furthering the application of computer vision techniques and robotics to construction work.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Data generated or analyzed during the study are available from the corresponding author by request. Information about the Journal’s data-sharing policy can be found here: http://ascelibrary.org/doi/10.1061/(ASCE)CO.1943-7862.0001263.
References
Amoretti, M., and M. Reggiani. 2010. “Architectural paradigms for robotics applications.” Adv. Eng. Inf. 24 (1): 4–13. https://doi.org/10.1016/j.aei.2009.08.004.
Bay, H., A. Ess, T. Tuytelaars, and L. Van Gool. 2008. “Speeded-up robust features (SURF).” Comput. Vision Image Understanding 110 (3): 346–359. https://doi.org/10.1016/j.cviu.2007.09.014.
Bay, H., T. Tuytelaars, and L. Van Gool. 2006. “Surf: Speeded up robust features.” In Proc., European Conf. on Computer Vision, 404–417. Berlin: Springer.
Bock, T. 2015. “The future of construction automation: Technological disruption and the upcoming ubiquity of robotics.” Autom. Constr. 59 (Nov): 113–121. https://doi.org/10.1016/j.autcon.2015.07.022.
Borrmann, D., A. Nüchter, M. Ðakulović, I. Maurović, I. Petrović, D. Osmanković, and J. Velagić. 2014. “A mobile robot based system for fully automated thermal 3D mapping.” Adv. Eng. Inf. 28 (4): 425–440. https://doi.org/10.1016/j.aei.2014.06.002.
Boysen, N., D. Briskorn, and S. Emde. 2017. “Parts-to-picker based order processing in a rack-moving mobile robots environment.” Eur. J. Oper. Res. 262 (2): 550–562. https://doi.org/10.1016/j.ejor.2017.03.053.
Bridgwater, T., G. Griffiths, A. Winfield, and T. Pipe. 2016. “A novel design for a robot grappling hook for use in a nuclear cave environment.” IFAC-PapersOnLine 49 (21): 288–293. https://doi.org/10.1016/j.ifacol.2016.10.569.
Caruso, L., R. Russo, and S. Savino. 2017. “Microsoft Kinect V2 vision system in a manufacturing application.” Rob. Comput. Integr. Manuf. 48 (Apr): 174–181. https://doi.org/10.1016/j.rcim.2017.04.001.
Chi, H. L., Y. C. Chen, S. C. Kang, and S. H. Hsieh. 2012. “Development of user interface for tele-operated cranes.” Adv. Eng. Inf. 26 (3): 641–652. https://doi.org/10.1016/j.aei.2012.05.001.
Chu, B., K. Jung, M. T. Lim, and D. Hong. 2013. “Robot-based construction automation: An application to steel beam assembly (Part I).” Autom. Constr. 32 (Jul): 46–61. https://doi.org/10.1016/j.autcon.2012.12.016.
Darwish, W., S. Tang, W. Li, and W. Chen. 2017. “A new calibration method for commercial RGB-D sensors.” Sensors 17 (6): 1204–1215. https://doi.org/10.3390/s17061204.
Feng, C., Y. Xiao, A. Willette, W. McGee, and V. R. Kamat. 2015. “Vision guided autonomous robotic assembly and as-built scanning on unstructured construction sites.” Autom. Constr. 59 (Nov): 128–138. https://doi.org/10.1016/j.autcon.2015.06.002.
Hamledari, H., B. McCabe, and S. Davari. 2017. “Automated computer vision-based detection of components of under-construction indoor partitions.” Autom. Constr. 74 (Feb): 78–94. https://doi.org/10.1016/j.autcon.2016.11.009.
Han, S., S. Lee, and F. Peña-Mora. 2013. “Vision-based detection of unsafe actions of a construction worker: Case study of ladder climbing.” J. Comput. Civ. Eng. 27 (6): 635–644. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000279.
Hartley, R. I. 1997. “In defense of the eight-point algorithm.” IEEE Trans. Pattern Anal. Mach. Intell. 19 (6): 580–593. https://doi.org/10.1109/34.601246.
Herrero, H., A. Abou Moughlbay, J. L. Outón, D. Sallé, and K. L. de Ipiña. 2017. “Skill based robot programming: Assembly, vision and workspace monitoring skill interaction.” Neurocomputing 255 (Sep): 61–70. https://doi.org/10.1016/j.neucom.2016.09.133.
Hong, J., G. Q. Shen, Z, Li, B. Zhang, and W. Zhang. 2018. “Barriers to promoting prefabricated construction in China: A cost–benefit analysis.” J. Cleaner Prod. 172 (Jan): 649–660. https://doi.org/10.1016/j.jclepro.2017.10.171.
Jaillon, L., and C. S. Poon. 2014. “Life cycle design and prefabrication in buildings: A review and case studies in Hong Kong.” Autom. Constr. 39 (Apr): 195–202. https://doi.org/10.1016/j.autcon.2013.09.006.
Jog, G. M., H. Fathi, and I. Brilakis. 2011. “Automated computation of the fundamental matrix for vision based construction site applications.” Adv. Eng. Inf. 25 (4): 725–735. https://doi.org/10.1016/j.aei.2011.03.005.
Kim, H., K. Kim, and H. Kim. 2016. “Data-driven scene parsing method for recognizing construction site objects in the whole image.” Autom. Constr. 71 (Part 2): 271–282. https://doi.org/10.1016/j.autcon.2016.08.018.
Kim, K., H. Kim, and H. Kim. 2017. “Image-based construction hazard avoidance system using augmented reality in wearable device.” Autom. Constr. 83 (Aug): 390–403. https://doi.org/10.1016/j.autcon.2017.06.014.
Kurien, M., M. K. Kim, M. Kopsida, and I. Brilakis. 2018. “Real-time simulation of construction workers using combined human body and hand tracking for robotic construction worker system” Autom. Constr. 86 (Jan): 125–137. https://doi.org/10.1016/j.autcon.2017.11.005.
Lee, C., G. Lee, S. Park, and J. Cho. 2012. “Analysis of field applicability of the rotation-controllable tower-crane hook block.” Autom. Constr. 21 (1): 81–88. https://doi.org/10.1016/j.autcon.2011.05.015.
Lee, C., and G. Lee. 2014. “Feasibility of beam erection with a motorized hook-block.” Autom. Constr. 41 (May): 25–32. https://doi.org/10.1016/j.autcon.2014.01.003.
Li, Z., G. Q. Shen, and M. Alshawi. 2014a. “Measuring the impact of prefabrication on construction waste reduction: An empirical study in China.” Resour. Conserv. Recycl. 91 (Sep): 27–39. https://doi.org/10.1016/j.resconrec.2014.07.013.
Li, Z., G. Q. Shen, and X. Xue. 2014b. “Critical review of the research on the management of prefabricated construction.” Habitat Int. 43 (Jul): 240–249. https://doi.org/10.1016/j.habitatint.2014.04.001.
Liao, L., K. Li, K. Li, C. Yang, and Q. Tian. 2018. “UHCL-Darknet: An OpenCL-based deep neural network framework for heterogeneous multi-/many-core clusters.” In Proc., 47th Int. Conf. on Parallel Processing, 1–10. New York: Association for Computing Machinery.
Lillywhite, K., D. J. Lee, B. Tippetts, and J. Archibald. 2013. “A feature construction method for general object recognition.” Pattern Recognit. 46 (12): 3300–3314. https://doi.org/10.1016/j.patcog.2013.06.002.
Lisowski, E., and G. Filo. 2015. “Automated heavy load lifting and moving system using pneumatic cushions.” Autom. Constr. 50 (Feb): 91–101. https://doi.org/10.1016/j.autcon.2014.12.004.
Mao, C., Q. Shen, W. Pan, and K. Ye. 2015. “Major barriers to off-site construction: The developers perspective in China.” J. Manage. Eng. 31 (3): 4014043. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000246.
Moses, M. S., H. Ma, K. C. Wolfe, and G. S. Chirikjian. 2014. “An architecture for universal construction via modular robotic components.” Rob. Auton. Syst. 62 (7): 945–965. https://doi.org/10.1016/j.robot.2013.08.005.
Nam, M., J. Kim, J. Lee, D. Kim, D. Lee, and J. Lee. 2017. “Cooperative control system of the floating cranes for the dual lifting.” Int. J. Nav. Archit. Ocean Eng. 10 (1): 95–102. https://doi.org/10.1016/j.ijnaoe.2017.03.003.
Napoli, A., S. Glass, C. Ward, C. Tucker, and I. Obeid. 2017. “Performance analysis of a generalized motion capture system using microsoft kinect 2.0.” Biomed. Signal Process. Control 38 (Sep): 265–280. https://doi.org/10.1016/j.bspc.2017.06.006.
Nerakae, P., P. Uangpairoj, and K. Chamniprasart. 2016. “Using machine vision for flexible automatic assembly system.” Procedia Comput. Sci. 96 (Jan): 428–435. https://doi.org/10.1016/j.procs.2016.08.090.
Plantard, P., H. P. Shum, A. S. Le Pierres, and F. Multon. 2017. “Validation of an ergonomic assessment method using Kinect data in real workplace conditions.” Appl. Ergon. 65 (Nov): 562–569. https://doi.org/10.1016/j.apergo.2016.10.015.
Redmon, J., S. Divvala, R. Girshick, and A. Farhadi. 2015. “You only look once: Unified, real-time object detection.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 779–788. New York: IEEE. https://doi.org/10.1109/CVPR.2016.91.
Redmon, J., and A. Farhadi. 2016. “YOLO9000: Better, faster, stronger.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 7263–7271. New York: IEEE. https://doi.org/10.1109/CVPR.2017.690.
Rezazadeh Azar, E., and B. McCabe. 2012. “Part based model and spatial-temporal reasoning to recognize hydraulic excavators in construction images and videos.” Autom. Constr. 24 (Jul): 194–202. https://doi.org/10.1016/j.autcon.2012.03.003.
Sangdani, M. H., A. R. Tavakolpour-Saleh, and A. Lotfavar. 2018. “Genetic algorithm-based optimal computed torque control of a vision-based tracker robot: Simulation and experiment.” Eng. Appl. Artif. Intell. 67 (Jan): 24–38. https://doi.org/10.1016/j.engappai.2017.09.014.
Scardua, L. A., J. J. Da Cruz, and A. H. R. Costa. 2002. “Optimal control of ship unloaders using reinforcement learning.” Adv. Eng. Inf. 16 (3): 217–227. https://doi.org/10.1016/S1474-0346(03)00002-8.
Soltani, A. R., H. Tawfik, J. Y. Goulermas, and T. Fernando. 2002. “Path planning in construction sites: Performance evaluation of the Dijkstra, A*, and GA search algorithms.” Adv. Eng. Inf. 16 (4): 291–303. https://doi.org/10.1016/S1474-0346(03)00018-1.
Soltani, M. M., Z. Zhu, and A. Hammad. 2016. “Automated annotation for visual recognition of construction resources using synthetic images.” Autom. Constr. 62 (4): 14–23. https://doi.org/10.1016/j.autcon.2015.10.002.
Spadafora, F., M. Muzzupappa, F. Bruno, D. Ribas, and P. Ridao. 2015. “Design and construction of a robot hand prototype for underwater applications.” IFAC-PapersOnLine 48 (2): 294–299. https://doi.org/10.1016/j.ifacol.2015.06.048.
Sun, G., M. Kleeberger, and J. Liu. 2005. “Complete dynamic calculation of lattice mobile crane during hoisting motion.” Mech. Mach. Theory 40 (4): 447–466. https://doi.org/10.1016/j.mechmachtheory.2004.07.014.
Sung, C., and P. Y. Kim. 2016. “3D terrain reconstruction of construction sites using a stereo camera.” Autom. Constr. 64 (Apr): 65–77. https://doi.org/10.1016/j.autcon.2015.12.022.
Tam, V. W., C. M. Tam, S. X. Zeng, and W. C. Ng. 2007. “Towards adoption of prefabrication in construction.” Build. Environ. 42 (10): 3642–3654. https://doi.org/10.1016/j.buildenv.2006.10.003.
Tam, V. W., I. W. Fung, M. C. Sing, and S. O. Ogunlana. 2015. “Best practice of prefabrication implementation in the Hong Kong public and private sectors.” J. Cleaner Prod. 109 (Dec): 216–231. https://doi.org/10.1016/j.jclepro.2014.09.045.
Tho, H. D., R. Tasaki, K. Terashima, M. Tsume, and K. Suzuki. 2017. “Robust sliding mode control of a rotary hook.” IFAC-PapersOnLine 50 (1): 6060–6066. https://doi.org/10.1016/j.ifacol.2017.08.1378.
Tran, T. H., T. L. Le, V. N. Hoang, and H. Vu. 2017. “Continuous detection of human fall using multimodal features from Kinect sensors in scalable environment.” Comput. Methods Programs Biomed. 146 (Jul): 151–165. https://doi.org/10.1016/j.cmpb.2017.05.007.
Vähä, P., T. Heikkilä, P. Kilpeläinen, M. Järviluoma, and E. Gambao. 2013. “Extending automation of building construction—Survey on potential sensor technologies and robotic applications.” Autom. Constr. 36 (Dec): 168–178. https://doi.org/10.1016/j.autcon.2013.08.002.
Walambe, R., N. Agarwal, S. Kale, and V. Joshi. 2016. “Optimal trajectory generation for car-type mobile robot using spline interpolation.” IFAC-PapersOnLine 49 (1): 601–606. https://doi.org/10.1016/j.ifacol.2016.03.121.
Wang, B., S. Li, J. Guo, and Q. Chen. 2017. “Car-like mobile robot path planning in rough terrain using multi-objective particle swarm optimization algorithm.” Neurocomputing 282 (Mar): 42–51. https://doi.org/10.1016/j.neucom.2017.12.015.
Xu, Z., R. Hess, and K. Schilling. 2012. “Constraints of potential field for obstacle avoidance on car-like mobile robots.” IFAC-PapersOnLine. 45 (4): 169–175. https://doi.org/10.3182/20120403-3-DE-3010.00077.
Yang, J., Z. Shi, and Z. Wu. 2016. “Vision-based action recognition of construction workers using dense trajectories.” Adv. Eng. Inf. 30 (3): 327–336. https://doi.org/10.1016/j.aei.2016.04.009.
Yin, X. F., L. P. Khoo, and C. H. Chen. 2011. “A distributed agent system for port planning and scheduling.” Adv. Eng. Inf. 25 (3): 403–412. https://doi.org/10.1016/j.aei.2010.10.004.
Zhang, C., and A. Hammad. 2012. “Improving lifting motion planning and re-planning of cranes with consideration for safety and efficiency.” Adv. Eng. Inf. 26 (2): 396–410. https://doi.org/10.1016/j.aei.2012.01.003.
Zhao, Y., L. Gong, Y. Huang, and C. Liu. 2016. “A review of key techniques of vision-based control for harvesting robot.” Comput. Electron. Agric. 127 (Sep): 311–323. https://doi.org/10.1016/j.compag.2016.06.022.
Zhu, H., et al. 2018. “The exploration of the life-cycle energy saving potential for using prefabrication in residential buildings in China.” Energy Build. 166 (May): 561–570. https://doi.org/10.1016/j.enbuild.2017.12.045.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 146 • Issue 12 • December 2020
Copyright
© 2020 American Society of Civil Engineers.
History
Received: Jan 14, 2020
Accepted: Jun 16, 2020
Published online: Sep 28, 2020
Published in print: Dec 1, 2020
Discussion open until: Feb 28, 2021
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.