Vision-Based Real-Time Posture Tracking for Multiple Construction Workers
Publication: Journal of Computing in Civil Engineering
Volume 38, Issue 4
Abstract
Tracking the postures of construction workers can provide precious information for safety management, occupational illness prevention, and productivity investigation. However, the posture data of construction workers is rarely utilized due to a lack of appropriate methods to track it. This research proposes a real-time multiworker posture tracking (MWPT) method to accurately track the postures of multiple workers onsite from video streams. It consists of three elements: image enhancement to adapt varying light conditions, posture detection for obtaining workers’ postures, and matching for tracking and retracking postures. In the field experiment, MWPT performed satisfactorily with an average of two ID switches (IDS), an average frame per second (FPS) of 11.0, and an average precision (AP@50) of 86.33. The results prove the capability of MWPT for tracking multiworker postures in real construction environments with high robustness and effectiveness. This research not only contributes an innovative tracking algorithm but also lays a stepping stone toward further worker posture-related research.
Practical Applications
This research introduces a posture tracking method for multiple construction workers. The motivation behind this research stems from the absence of an effective tracking method tailored to the demanding conditions of construction sites, such as variable lighting, extensive occlusions, and the challenge of distinguishing workers with similar appearances. The proposed solution leverages posture similarity for tracking workers and incorporates a retracking mechanism to address visual occlusions. The contrast limited adaptive histogram equalization method is employed to recover high-contrast visual features from dimly lit images, effectively addressing the issue of insufficient lighting. This method has been validated as both effective and efficient through customized test videos from actual construction sites. It shows promise for use in unsafe behavior detection, occupational disease prevention, and other posture-related research or applications as an effective tool.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this research are available from the corresponding author upon reasonable request (data set and python code for recognition model).
Acknowledgments
The authors would like to thank the National Natural Science Foundation of China (Grant Nos. 52278310 and 51578318) for supporting this research.
References
Aharon, N., R. Orfaig, and B.-Z. Bobrovsky. 2022. “BoT-SORT: Robust associations multi-pedestrian tracking.” Preprint, submitted June 29, 2022. http://arxiv.org/abs/2206.14651.
Alwasel, A., M. Nahangi, C. T. Haas, and E. M. Abdel-Rahman. 2017a. “Level-of-expertise classification for identifying safe and productive masons.” In Proc., ASCE Int. Workshop on Computing in Civil Engineering, 359–368. Reston, VA: ASCE.
Alwasel, A., A. Sabet, M. Nahangi, C. T. Haas, and E. Abdel-Rahman. 2017b. “Identifying poses of safe and productive masons using machine learning.” Autom. Constr. 84 (Dec): 345–355. https://doi.org/10.1016/j.autcon.2017.09.022.
Andriluka, M., U. Iqbal, E. Insafutdinov, L. Pishchulin, A. Milan, J. Gall, and B. Schiele. 2018. “PoseTrack: A benchmark for human pose estimation and tracking.” In Proc., 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 5167–5176. New York: IEEE.
Antwi-Afari, M. F., Y. Qarout, R. Herzallah, S. Anwer, W. Umer, Y. Zhang, and P. Manu. 2022. “Deep learning-based networks for automated recognition and classification of awkward working postures in construction using wearable insole sensor data.” Autom. Constr. 136 (Apr): 104181. https://doi.org/10.1016/j.autcon.2022.104181.
Bochkovskiy, A., C.-Y. Wang, and H.-Y. M. Liao. 2020. “YOLOv4: Optimal speed and accuracy of object detection.” Preprint, submitted April 23, 2020. http://arxiv.org/abs/2004.10934.
Buchholz, B., V. Paquet, L. Punnett, D. Lee, and S. Moir. 1996. “PATH: A work sampling-based approach to ergonomic job analysis for construction and other non-repetitive work.” Appl. Ergon. 27 (3): 177–187. https://doi.org/10.1016/0003-6870(95)00078-X.
Cao, Z., G. Hidalgo, T. Simon, S.-E. Wei, and Y. Sheikh. 2019. “OpenPose: Realtime multi-person 2D pose estimation using part affinity fields.” Preprint, submitted December 18, 2018. http://arxiv.org/abs/1812.08008.
Carini, F., M. Mazzola, C. Fici, S. Palmeri, M. Messina, P. Damiani, and G. Tomasello. 2017. “Posture and posturology, anatomical and physiological profiles: Overview and current state of art.” Acta Biomed. 88 (1): 11–16.
Chen, J., J. Qiu, and C. Ahn. 2017. “Construction worker’s awkward posture recognition through supervised motion tensor decomposition.” Autom. Constr. 77 (May): 67–81. https://doi.org/10.1016/j.autcon.2017.01.020.
Chen, Y., Z. Wang, Y. Peng, Z. Zhang, G. Yu, and J. Sun. 2018. “Cascaded pyramid network for multi-person pose estimation.” Preprint, submitted November 20, 2017. http://arxiv.org/abs/1711.07319.
COCO (Common Objects in Context). 2020a. “COCO 2020 keypoint detection task.” Accessed March 20, 2003. https://cocodataset.org/#keypoints-2020.
COCO (Common Objects in Context). 2020b. “Keypoint evaluation.” Accessed March 20, 2003. https://cocodataset.org/#keypoints-eval.
Doering, A., D. Chen, S. Zhang, B. Schiele, and J. Gall. 2022. “PoseTrack21: A dataset for person search, multi-object tracking and multi-person pose tracking.” In Proc., 2022 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 20931–20940. New York: IEEE.
Duan, P., Y. M. Goh, and J. Zhou. 2023. “Personalized stability monitoring based on body postures of construction workers working at heights.” Saf. Sci. 162 (Jun): 106104. https://doi.org/10.1016/j.ssci.2023.106104.
Fang, H.-S., J. Li, H. Tang, C. Xu, H. Zhu, Y. Xiu, Y.-L. Li, and C. Lu. 2022. “AlphaPose: Whole-body regional multi-person pose estimation and tracking in real-time.” Preprint, submitted November 7, 2022. http://arxiv.org/abs/2211.03375.
Ge, Z., S. Liu, F. Wang, Z. Li, and J. Sun. 2021. “YOLOX: Exceeding YOLO series in 2021.” Preprint, submitted July 18, 2021. http://arxiv.org/abs/2107.08430.
Girdhar, R., G. Gkioxari, L. Torresani, M. Paluri, and D. Tran. 2018. “Detect-and-Track: Efficient pose estimation in videos.” In Proc., 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 350–359. New York: IEEE.
Girdhar, R., G. Gkioxari, L. Torresani, D. Ramanan, M. Paluri, and D. Tran. 2017. “Simple, efficient and effective keypoint tracking.” In Proc., 2018 PoseTrack Int. Conf. on Computer Vision (ICCV). New York: IEEE.
Golabchi, A., S. Han, and A. R. Fayek. 2016. “A fuzzy logic approach to posture-based ergonomic analysis for field observation and assessment of construction manual operations.” Can. J. Civ. Eng. 43 (4): 294–303. https://doi.org/10.1139/cjce-2015-0143.
Guo, H., Y. Yu, Q. Ding, and M. Skitmore. 2018. “Image-and-skeleton-based parameterized approach to real-time identification of construction workers’ unsafe behaviors.” J. Constr. Eng. Manage. 144 (6): 04018042. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001497.
Guo, H., Z. Zhang, R. Yu, Y. Sun, and H. Li. 2023. “Action recognition based on 3d skeleton and LSTM for the monitoring of construction workers’ safety harness usage.” J. Constr. Eng. Manage. 149 (4): 04023015. https://doi.org/10.1061/JCEMD4.COENG-12542.
Han, S., S. Lee, and F. Peña-Mora. 2013. “Vision-based detection of unsafe actions of a construction worker: Case study of ladder climbing.” J. Comput. Civ. Eng. 27 (6): 635–644. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000279.
Jin, S., X. Ma, Z. Han, Y. Wu, W. Yang, W. Liu, C. Qian, and W. Ouyang. 2017. “Towards multi-person pose tracking: Bottom-up and top-down methods.” Accessed March 22, 2023. https://jin-s13.github.io/papers/BUTD.pdf.
Kreiss, S., L. Bertoni, and A. Alahi. 2021. “OpenPifPaf: Composite fields for semantic keypoint detection and spatio-temporal association.” Preprint, submitted March 3, 2021. http://arxiv.org/abs/2103.02440.
Lee, T.-H., and C.-S. Han. 2013. “Analysis of working postures at a construction site using the OWAS method.” Int. J. Occup. Saf. Ergon. 19 (2): 245–250. https://doi.org/10.1080/10803548.2013.11076983.
Li, H., G. Chan, T. Huang, M. Skitmore, T. Y. E. Tao, E. Luo, J. Chung, X. S. Chan, and Y. F. Li. 2015. “Chirp-spread-spectrum-based real time location system for construction safety management: A case study.” Autom. Constr. 55 (Jul): 58–65. https://doi.org/10.1016/j.autcon.2015.03.024.
Li, Z., and D. Li. 2022. “Action recognition of construction workers under occlusion.” J. Build. Eng. 45 (Jan): 103352. https://doi.org/10.1016/j.jobe.2021.103352.
Lin, X., and H. Guo. 2022. “A sensor-based method to detect near-miss struck-by on construction site.” In Proc., 26th Int. Symp. on Advancement of Construction Management and Real Estate, 1202–1217. Berlin: Springer.
Lindenmann, A., M. Uhl, T. Gwosch, and S. Matthiesen. 2021. “The influence of human interaction on the vibration of hand-held human-machine systems: The effect of body posture, feed force, and gripping forces on the vibration of hammer drills.” Appl. Ergon. 95 (Sep): 103430. https://doi.org/10.1016/j.apergo.2021.103430.
Maji, D., S. Nagori, M. Mathew, and D. Poddar. 2022. “YOLO-Pose: Enhancing YOLO for multi person pose estimation using object keypoint similarity loss.” Preprint, submitted April 14, 2022. http://arxiv.org/abs/2204.06806.
Nath, N. D., R. Akhavian, and A. H. Behzadan. 2017. “Ergonomic analysis of construction worker’s body postures using wearable mobile sensors.” Appl. Ergon. 62 (Jul): 107–117. https://doi.org/10.1016/j.apergo.2017.02.007.
Nibali, A., J. Millward, Z. He, and S. Morgan. 2021. “ASPset: An outdoor sports pose video dataset with 3D keypoint annotations.” Image Vis. Comput. 111 (Jul): 104196. https://doi.org/10.1016/j.imavis.2021.104196.
Ning, G., J. Pei, and H. Huang. 2020. “LightTrack: A generic framework for online top-down human pose tracking.” In Proc., 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), 4456–4465. Seattle: IEEE.
Osokin, D. 2018. “Real-time 2D multi-person pose estimation on CPU: Lightweight OpenPose.” Preprint, submitted November 29, 2022. http://arxiv.org/abs/1811.12004.
Park, M.-W., and I. Brilakis. 2016. “Continuous localization of construction workers via integration of detection and tracking.” Autom. Constr. 72 (Part 2): 129–142. https://doi.org/10.1016/j.autcon.2016.08.039.
Raaj, Y., H. Idrees, G. Hidalgo, and Y. Sheikh. 2019. “Efficient online multi-person 2D pose tracking with recurrent spatio-temporal affinity fields.” In Proc., 2019 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 4615–4623. New York: IEEE.
Roberts, D., W. T. Calderon, S. Tang, and M. Golparvar-Fard. 2020. “Vision-based construction worker activity analysis informed by body posture.” J. Comput. Civ. Eng. 34 (4): 04020017. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000898.
Song, L., G. Yu, J. Yuan, and Z. Liu. 2021. “Human pose estimation and its application to action recognition: A survey.” J. Vis. Commun. Image Represent. 76 (Apr): 103055. https://doi.org/10.1016/j.jvcir.2021.103055.
Subedi, S., and N. Pradhananga. 2021. “Sensor-based computational approach to preventing back injuries in construction workers.” Autom. Constr. 131 (Nov): 103920. https://doi.org/10.1016/j.autcon.2021.103920.
Sun, K., B. Xiao, D. Liu, and J. Wang. 2019. “Deep high-resolution representation learning for human pose estimation.” Preprint, submitted February 25, 2019. http://arxiv.org/abs/1902.09212.
Valero, E., A. Sivanathan, F. Bosché, and M. Abdel-Wahab. 2017. “Analysis of construction trade worker body motions using a wearable and wireless motion sensor network.” Autom. Constr. 83 (Nov): 48–55. https://doi.org/10.1016/j.autcon.2017.08.001.
Wang, C.-Y., A. Bochkovskiy, and H.-Y. M. Liao. 2022. “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors.” Preprint, submitted July 6, 2022. http://arxiv.org/abs/2207.02696.
Xiu, Y., J. Li, H. Wang, Y. Fang, and C. Lu. 2018. “Pose Flow: Efficient online pose tracking.” Preprint, submitted February 3, 2018. http://arxiv.org/abs/1802.00977.
Xu, L., R. Xu, and S. Jin. 2020. “HiEve ACM MM grand challenge 2020: Pose tracking in crowded scenes.” In Proc., 28th ACM Int. Conf. on Multimedia, 4689–4693. New York: Association of Computing Machinery.
Yang, J., O. Arif, P. A. Vela, J. Teizer, and Z. Shi. 2010. “Tracking multiple workers on construction sites using video cameras.” Adv. Eng. Inf. 24 (4): 428–434. https://doi.org/10.1016/j.aei.2010.06.008.
Yang, Y., and D. Ramanan. 2013. “Articulated human detection with flexible mixtures of parts.” IEEE Trans. Pattern Anal. Mach. Intell. 35 (12): 2878–2890. https://doi.org/10.1109/TPAMI.2012.261.
Yu, Y., H. Guo, Q. Ding, H. Li, and M. Skitmore. 2017. “An experimental study of real-time identification of construction workers’ unsafe behaviors.” Autom. Constr. 82 (Oct): 193–206. https://doi.org/10.1016/j.autcon.2017.05.002.
Yu, Y., W. Umer, X. Yang, and M. F. Antwi-Afari. 2021. “Posture-related data collection methods for construction workers: A review.” Autom. Constr. 124 (Apr): 103538. https://doi.org/10.1016/j.autcon.2020.103538.
Yu, Y. T., H. Li, X. C. Yang, L. L. Kong, X. C. Luo, and A. Y. L. Wong. 2019. “An automatic and non-invasive physical fatigue assessment method for construction workers.” Autom. Constr. 103 (Jul): 1–12. https://doi.org/10.1016/j.autcon.2019.02.020.
Zhang, H., X. Yan, and H. Li. 2018. “Ergonomic posture recognition using 3D view-invariant features from single ordinary camera.” Autom. Constr. 94 (Oct): 1–10. https://doi.org/10.1016/j.autcon.2018.05.033.
Zhao, J., and E. Obonyo. 2020. “Convolutional long short-term memory model for recognizing construction workers’ postures from wearable inertial measurement units.” Adv. Eng. Inf. 46 (Oct): 101177. https://doi.org/10.1016/j.aei.2020.101177.
Information & Authors
Information
Published In
Journal of Computing in Civil Engineering
Volume 38 • Issue 4 • July 2024
Copyright
© 2024 American Society of Civil Engineers.
History
Received: Nov 3, 2023
Accepted: Feb 13, 2024
Published online: May 15, 2024
Published in print: Jul 1, 2024
Discussion open until: Oct 15, 2024
ASCE Technical Topics:
- Business management
- Construction engineering
- Construction management
- Construction methods
- Construction sites
- Detection methods
- Employment
- Engineering fundamentals
- Information management
- Labor
- Methodology (by type)
- Occupational safety
- Personnel management
- Practice and Profession
- Public administration
- Public health and safety
- Safety
- Tracking
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.