Target Domain Training Data Generation by Moving Object Detection and Label Propagation
Publication: Computing in Civil Engineering 2023
ABSTRACT
Although computer vision technology has shown great potential, its reliability can significantly degrade in the target domain where the model is applied. Collecting and labeling training data from the target domain can address this issue; however, it is a tedious and time-consuming task. To address this issue, this paper presents a novel method generating training data for construction site monitoring. The proposed method consists of extracting moving objects, classifying each region by comparing their features to the target classes, and then assigning class labels. The newly labeled data is copied and pasted to a clear background of the target domain where its foregrounds are removed by image inpainting. Experiments were conducted on construction site videos captured in far-field monitoring environments. The proposed method can significantly reduce the amount of effort required for data collection and labeling, thereby increasing the efficiency of developing robust computer vision models for construction site monitoring.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Bang, S., F. Baek, S. Park, W. Kim, and H. Kim. (2020). “Image augmentation to improve construction resource detection using generative adversarial networks, cut-and-paste, and image transformation techniques.” Autom. Constr., 115: 103198. https://doi.org/10.1016/j.autcon.2020.103198.
Bewley, A., Z. Ge, L. Ott, F. Ramos, and B. Upcroft. (2016). “Simple online and realtime tracking.” 2016 IEEE Int. Conf. Image Process. ICIP, 3464–3468.
Cheng, J. C. P., P. K.-Y. Wong, H. Luo, M. Wang, and P. H. Leung. (2022). “Vision-based monitoring of site safety compliance based on worker re-identification and personal protective equipment classification.” Autom. Constr., 139: 104312. https://doi.org/10.1016/j.autcon.2022.104312.
Chern, W.-C., J. Hyeon, T. V. Nguyen, V. K. Asari, and H. Kim. (2023). “Context-aware safety assessment system for far-field monitoring.” Autom. Constr., 149: 104779. https://doi.org/10.1016/j.autcon.2023.104779.
Duan, R., H. Deng, M. Tian, Y. Deng, and J. Lin. (2022). “SODA: A large-scale open site object detection dataset for deep learning in construction.” Autom. Constr., 142: 104499. https://doi.org/10.1016/j.autcon.2022.104499.
Ghiasi, G., Y. Cui, A. Srinivas, R. Qian, T.-Y. Lin, E. D. Cubuk, Q. V. Le, and B. Zoph. (2021). “Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation.” 2021 IEEECVF Conf. Comput. Vis. Pattern Recognit. CVPR, 2917–2927. https://doi.org/10.1109/CVPR46437.2021.00294.
Guo, B. H. W., Y. Zou, Y. Fang, Y. M. Goh, and P. X. W. Zou. (2021). “Computer vision technologies for safety science and management in construction: A critical review and future research directions.” Saf. Sci., 135: 105130. https://doi.org/10.1016/j.ssci.2020.105130.
Hong, Y., W.-C. Chern, T. V. Nguyen, H. Cai, and H. Kim. (2023). “Semi-supervised domain adaptation for segmentation models on different monitoring settings.” Autom. Constr., 149: 104773. https://doi.org/10.1016/j.autcon.2023.104773.
Iscen, A., G. Tolias, Y. Avrithis, and O. Chum. (2019). Label Propagation for Deep Semi-Supervised Learning. 5070–5079. https://doi.org/10.48550/arXiv.1904.04717.
Jiang, S., D. Campbell, Y. Lu, H. Li, and R. Hartley. (2021). Learning To Estimate Hidden Motions With Global Motion Aggregation. 9772–9781. https://doi.org/10.48550/arXiv.2104.02409.
Jocher, G., et al. (2022). ultralytics/yolov5: v7.0 - YOLOv5 SOTA Realtime Instance Segmentation. Zenodo.
Kim, H., Y. Ham, W. Kim, S. Park, and H. Kim. (2019). “Vision-based nonintrusive context documentation for earthmoving productivity simulation.” Autom. Constr., 102: 135–147. https://doi.org/10.1016/j.autcon.2019.02.006.
Kim, J., and S. Chi. (2020). “Multi-camera vision-based productivity monitoring of earthmoving operations.” Autom. Constr., 112: 103121. https://doi.org/10.1016/j.autcon.2020.103121.
Li, W., Z. Lin, K. Zhou, L. Qi, Y. Wang, and J. Jia. (2022). MAT: Mask-Aware Transformer for Large Hole Image Inpainting. 10758–10768. https://doi.org/10.48550/arXiv.2203.15270.
Lin, T.-Y., M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. (2014). “Microsoft COCO: Common Objects in Context.” Comput. Vis. – ECCV 2014, Lecture Notes in Computer Science, D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, eds., 740–755. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-10602-1_48.
Reja, V. K., K. Varghese, and Q. P. Ha. (2022). “Computer vision-based construction progress monitoring.” Autom. Constr., 138: 104245. https://doi.org/10.1016/j.autcon.2022.104245.
Tan, M., and Q. Le. (2021). “EfficientNetV2: Smaller Models and Faster Training.” Proc. 38th Int. Conf. Mach. Learn., 10096–10106. PMLR. https://doi.org/10.48550/arXiv.2104.00298.
Xuehui, A., Z. Li, L. Zuguang, W. Chengzhi, L. Pengfei, and L. Zhiwei. (2021). “Dataset and benchmark for detecting moving objects in construction sites.” Autom. Constr., 122: 103482. https://doi.org/10.1016/j.autcon.2020.103482.
Information & Authors
Information
Published In
Computing in Civil Engineering 2023
Pages: 380 - 387
History
Published online: Jan 25, 2024
ASCE Technical Topics:
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.