Detection, Tracking, and Segmentation of Transient Construction Objects in Video Frames
Publication: Computing in Civil Engineering 2023
ABSTRACT
In construction environments, transient construction objects such as moving vehicles, debris, materials, and construction tools can pose safety hazards and impede visibility. Monitoring their movement in video frames is crucial for effective project management, but it is challenging due to the complexity and dynamism of construction sites. This study proposes a two-step method to address this challenge by detecting, tracking, and segmenting transient construction objects in video frames. A synthetic dataset is used to train a deep learning-based detection model, and coarse and fine tracking models are then applied to track and segment objects based on detection results. The preliminary results reveal the significant performance of the proposed detection model, achieving an average precision of 75.75% at an IoU threshold of 0.5. It is demonstrated that the proposed detection method enables the detection of transient construction objects using synthetic datasets, reducing the need for manually annotating additional construction-related datasets. Besides, the produced segmentation results provide detailed information about the location and shape of objects, enabling enhanced safety control and analysis.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Azar, E. R., and McCabe, B. (2012). “Automated Visual Recognition of Dump Trucks in Construction Videos.” Journal of Computing in Civil Engineering, 26(6), 769–781.
Bewley, A., Ge, Z., Ott, L., Ramos, F., and Upcroft, B. “Simple online and realtime tracking.” Proc., 2016 IEEE International Conference on Image Processing (ICIP), 3464–3468.
Fang, Q., Li, H., Luo, X., Ding, L., Luo, H., and Li, C. (2018). “Computer vision aided inspection on falling prevention measures for steeplejacks in an aerial environment.” Automation in Construction, 93, 148–164.
Freeman, H., and Shapira, R. (1975). “Determining the minimum-area encasing rectangle for an arbitrary closed curve.” Commun. ACM, 18(7), 409–413.
Golparvar-Fard, M., Heydarian, A., and Niebles, J. C. (2013). “Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers.” Adv. Eng. Inform., 27(4), 652–663.
He, K., Gkioxari, G., Dollár, P., and Girshick, R. B. (2017). “Mask R-CNN.” 2017 IEEE International Conference on Computer Vision (ICCV), 2980–2988.
Hisham, M. B., Yaakob, S. N., Raof, R. A. A., Nazren, A. B. A., and Wafi, N. M. “Template Matching using Sum of Squared Difference and Normalized Cross Correlation.” Proc., 2015 IEEE Student Conference on Research and Development (SCOReD), 100–104.
Kim, C. (2017). “Vision-Based Safety Vest Detection in a Construction Scene.” Proceedings of the 34th International Symposium on Automation and Robotics in Construction (ISARC), Tribun EU, s.r.o., Brno, 288–293.
Lin, T.-Y., Maire, M., Belongie, S. J., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. “Microsoft COCO: Common Objects in Context.” Proc., ECCV.
Mneymneh, B. E., Abbas, M., and Khoury, H. (2019). “Vision-Based Framework for Intelligent Monitoring of Hardhat Wearing on Construction Sites.” Journal of Computing in Civil Engineering, 33(2), 04018066.
Park, M.-W., and Brilakis, I. (2016). “Continuous localization of construction workers via integration of detection and tracking.” Automation in Construction, 72, 129–142.
Rezazadeh Azar, E., and McCabe, B. (2012). “Part based model and spatial–temporal reasoning to recognize hydraulic excavators in construction images and videos.” Automation in Construction, 24, 194–202.
Wang, Q., Zhang, L., Bertinetto, L., Hu, W., and Torr, P. (2019). “Fast Online Object Tracking and Segmentation: A Unifying Approach.” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 1328–1338.
Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y., and Girshick, R. (2019). <https://github.com/facebookresearch/detectron2>.
Xiao, B., and Kang, S.-C. (2021). “Vision-Based Method Integrating Deep Learning Detection for Tracking Multiple Construction Machines.” Journal of Computing in Civil Engineering, 35(2), 04020071.
Zhu, Z., Ren, X., and Chen, Z. (2017). “Integrated detection and tracking of workforce and equipment from construction jobsite videos.” Automation in Construction, 81, 161–171.
Information & Authors
Information
Published In
Computing in Civil Engineering 2023
Pages: 298 - 307
History
Published online: Jan 25, 2024
ASCE Technical Topics:
- Computer vision and image processing
- Construction engineering
- Construction management
- Construction materials
- Construction methods
- Construction sites
- Continuum mechanics
- Detection methods
- Disaster risk management
- Disasters and hazards
- Dynamics (solid mechanics)
- Engineering fundamentals
- Engineering materials (by type)
- Engineering mechanics
- Frames
- Hazardous substances
- Man-made disasters
- Materials engineering
- Methodology (by type)
- Solid mechanics
- Structural engineering
- Structural members
- Structural systems
- Tracking
- Transient response
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.