Automated Assembly Progress Monitoring in Modular Construction Factories Using Computer Vision-Based Instance Segmentation
Publication: Computing in Civil Engineering 2023
ABSTRACT
Modular construction has recently gained interest as a transformative construction method. In this method, a large portion of the construction is performed inside factories, where processes are fast-paced and interdependent; therefore, any deviation from the schedule can delay the production. Such deviations are frequent in modular factories due to the labor-intensive nature of the tasks. This propagation of delays can be mitigated by continuously monitoring each process; however, current manual monitoring methods are laborious, and recently proposed contact sensor-based methods are intrusive to the work. In addition, recent computer vision-based monitoring methods inside factories are limited to detection algorithms that fail to provide the pixel-level accuracy required for assembly progress monitoring in highly occluded factory scenes, and they require a large number of manual annotations. Therefore, this paper proposes a method to monitor the installation of subassemblies in modular construction factories using mask R-CNN instance segmentation and improves the data efficiency of the model using a copy-paste augmentation method. This method was validated on the CCTV videos captured from a modular construction factory in the US, resulting in a 9% mAP improvement in segmentation.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Alsakka, F., I. El-Chami, H. Yu, and M. Al-Hussein. 2023. “Computer Vision-Based Process Time Data Acquisition for Offsite Construction.” Automation in Construction 149 (May): 104803. https://doi.org/10.1016/j.autcon.2023.104803.
Altaf, M. S., A. Bouferguene, H. Liu, M. Al-Hussein, and H. Yu. 2018. “Integrated Production Planning and Control System for a Panelized Home Prefabrication Facility Using Simulation and RFID.” Automation in Construction 85 (January): 369–83. https://doi.org/10.1016/j.autcon.2017.09.009.
Arashpour, M., and R. Wake. 2015. “Autonomous Production Tracking for Augmenting Output in Off-Site Construction.” Automation in Construction, 9.
Chu, W., S. Han, X. Luo, and Z. Zhu. 2020. “Monocular Vision–Based Framework for Biomechanical Analysis or Ergonomic Posture Assessment in Modular Construction.” Journal of Computing in Civil Engineering 34 (4): 04020018. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000897.
Dutta, A., and A. Zisserman. 2019. “The VIA Annotation Software for Images, Audio and Video.” In Proceedings of the 27th ACM International Conference on Multimedia, 2276–79. Nice France: ACM. https://doi.org/10.1145/3343031.3350535.
Ghiasi, G., Y. Cui, A. Srinivas, R. Qian, T.-Y. Lin, E. D. Cubuk, Q. V. Le, and B. Zoph. 2021. “Simple Copy-Paste Is a Strong Data Augmentation Method for Instance Segmentation.” arXiv. http://arxiv.org/abs/2012.07177.
Gong, J., and C. H. Caldas. 2009. “Computer Vision-Based Video Interpretation Model for Automated Productivity Analysis of Construction Operations.” Journal of Computing in Civil Engineering 24 (3): 252–63.
Khodabandelu, A., J. W. Park, J. O. Choi, and M. Sanei. 2022. Analysis of a Long Volumetric Module Lift Using Single and Multiple Cranes.
Martinez, P., B. Barkokebas, F. Hamzeh, M. Al-Hussein, and R. Ahmad. 2021. “A Vision-Based Approach for Automatic Progress Tracking of Floor Paneling in Offsite Construction Facilities.” Automation in Construction 125 (May): 103620. https://doi.org/10.1016/j.autcon.2021.103620.
Pal Singh Bhatia, A., S. Han, O. Moselhi, Z. Lei, and C. Raimondi. 2019. “Data Analytics of Production Cycle Time for Offsite Construction Projects.” Modular and Offsite Construction (MOC) Summit Proceedings, May, 25–32. https://doi.org/10.29173/mocs73.
Panahi, R., J. Louis, N. Aziere, A. Podder, and C. Swanson. 2021. Identifying Modular Construction Worker Tasks Using Computer Vision.
Panahi, R., J. Louis, A. Podder, and C. Swanson. 2022. “Tracking Volumetric Units in Modular Factories for Automated Progress Monitoring Using Computer Vision.” In Construction Research Congress 2022, 822–29. Arlington, Virginia: American Society of Civil Engineers. https://doi.org/10.1061/9780784483961.086.
Panahi, R., J. Louis, A. Podder, C. Swanson, and S. Pless. 2023. “Bottleneck Detection in Modular Construction Factories Using Computer Vision.” Sensors 23 (8): 3982. https://doi.org/10.3390/s23083982.
Park, K., and S. Ergan. 2022. “Toward Intelligent Agents to Detect Work Pieces and Processes in Modular Construction: An Approach to Generate Synthetic Training Data.” In Construction Research Congress 2022, 802–11. Arlington, Virginia: American Society of Civil Engineers. https://doi.org/10.1061/9780784483961.084.
Pless, S., et al. 2022. “The Energy in Modular (EMOD) Buildings Method: A Guide to Energy-Efficient Design for Industrialized Construction of Modular Buildings.”, MainId:83220. https://doi.org/10.2172/1875070.
Rashid, K. M., and J. Louis. 2020. “Activity Identification in Modular Construction Using Audio Signals and Machine Learning.” Automation in Construction 119 (November): 103361. https://doi.org/10.1016/j.autcon.2020.103361.
Rashid, K. M., and J. Louis. 2021. “Automated Active and Idle Time Measurement in Modular Construction Factory Using Inertial Measurement Unit and Deep Learning for Dynamic Simulation Input.” In 2021 Winter Simulation Conference (WSC), 1–8. Phoenix, AZ, USA: IEEE. https://doi.org/10.1109/WSC52266.2021.9715446.
Ren, S., K. He, R. Girshick, and J. Sun. 2015. “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.” In Advances in Neural Information Processing Systems 28, edited by C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, 91–99. Curran Associates, Inc. http://papers.nips.cc/paper/5638-faster-r-cnn-towards-real-time-object-detection-with-region-proposal-networks.pdf.
Tehrani, B. M., S. BuHamdan, and A. Alwisy. 2022. “Robotics in Industrialized Construction: An Activity-Based Ranking System for Assembly Manufacturing Tasks.” Engineering, Construction and Architectural Management, December. https://doi.org/10.1108/ECAM-02-2022-0143.
Wuni, I. Y., and G. Q. P. Shen. 2019. “Risks Identification and Allocation in the Supply Chain of Modular Integrated Construction (MiC).” Modular and Offsite Construction (MOC) Summit Proceedings, 189–97.
Xiao, B., H. Xiao, J. Wang, and Y. Chen. 2022. “Vision-Based Method for Tracking Workers by Integrating Deep Learning Instance Segmentation in off-Site Construction.” Automation in Construction 136 (April): 104148. https://doi.org/10.1016/j.autcon.2022.104148.
Zheng, Z., Z. Zhang, and W. Pan. 2020. “Virtual Prototyping- and Transfer Learning-Enabled Module Detection for Modular Integrated Construction.” Automation in Construction 120 (December): 103387. https://doi.org/10.1016/j.autcon.2020.103387.
Mohammadi, P., A. Rashidi, M. Malekzadeh, and S. Tiwari. 2023. “Evaluating Various Machine Learning Algorithms for Automated Inspection of Culverts.” Engineering Analysis with Boundary Elements 148 (March): 366–75. https://doi.org/10.1016/j.enganabound.2023.01.007.
Information & Authors
Information
Published In
Computing in Civil Engineering 2023
Pages: 290 - 297
History
Published online: Jan 25, 2024
ASCE Technical Topics:
- Automation and robotics
- Business management
- Computer vision and image processing
- Computing in civil engineering
- Construction engineering
- Construction methods
- Employment
- Engineering fundamentals
- Labor
- Management methods
- Methodology (by type)
- Mitigation and remediation
- Modular structures
- Personnel management
- Practice and Profession
- Scheduling
- Structural engineering
- Structures (by type)
- Systems engineering
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.