A Computer Vision Approach to Assessing Work-Related Musculoskeletal Disorder (WMSD) Risk in Construction Workers
Publication: Construction Research Congress 2024
ABSTRACT
Work-related musculoskeletal disorders (WMSDs) are a group of painful disorders of muscles, tendons, and nerves caused due to improper work postures prevalent in construction workers. These disorders can cause temporary and/or permanent disabilities and seriously affect workers’ livelihoods. Previous research has applied machine learning (ML) for the recognition of WMSD risk. However, previous research used inertial sensors strapped to the body to measure the angle of the body parts. These sensors are expensive and uncomfortable to wear while working. This research aims to eliminate the need for additional hardware through the use of computer vision. In this project, an ML pipeline was built to identify WMSD risk from workers’ images. A pre-trained ML framework called Mediapipe Pose was used to generate features from the images. The relative positions of these landmarks were then used as the input for an artificial neural network (ANN) to classify ergonomic and non-ergonomic postures using the supervised learning approach. After hyper-parameter tuning, 100% training and 99.96% validation accuracy were achieved. Finally, the trained model was tested on real-life videos of construction workers and found to perform satisfactorily.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Akanmu, A. A., J. Olayiwola, O. Ogunseiju, and D. McFeeters. 2020. “Cyber-physical postural training system for construction workers.” Automation in Construction, 117: 103272. Elsevier B.V. https://doi.org/10.1016/j.autcon.2020.103272.
Alirezaei, S., H. Taghaddos, K. Ghorab, A. N. Tak, and S. Alirezaei. 2022. “BIM-augmented reality integrated approach to risk management.” Automation in Construction, 141: 104458. https://doi.org/10.1016/j.autcon.2022.104458.
Antwi-Afari, M. F., Y. Qarout, R. Herzallah, S. Anwer, W. Umer, Y. Zhang, and P. Manu. 2022. “Deep learning-based networks for automated recognition and classification of awkward working postures in construction using wearable insole sensor data.” Automation in Construction, 136: 104181. https://doi.org/10.1016/j.autcon.2022.104181.
Choi, B., S. Hwang, and S. Lee. 2017. “What drives construction workers’ acceptance of wearable technologies in the workplace?: Indoor localization and wearable health devices for occupational safety and health.” Automation in Construction, 84: 31–41. https://doi.org/10.1016/j.autcon.2017.08.005.
Ding, L., W. Fang, H. Luo, P. E. D. Love, B. Zhong, and X. Ouyang. 2018. “A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory.” Automation in Construction, 86: 118–124. https://doi.org/10.1016/j.autcon.2017.11.002.
Dong, X. S., E. Betit, A. M. Dale, G. Barlet, and Q. Wei. 2019. “Trends of musculoskeletal disorders and interventions in the construction industry.” CPWR quarterly data report ; 3rd quarter 2019, (CPWR--The Center for Construction Research and Training. National Institute for Occupational Safety and Health., ed.). Silver Spring, MD.
Ensafi, M., S. Alimoradi, X. Gao, and W. Thabet. 2022. “Machine Learning and Artificial Intelligence Applications in Building Construction: Present Status and Future Trends.” Construction Research Congress 2022, 116–124.
Géron, A. 2019. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow.
Golabchi, A., N. Jasimi Zindashti, L. Miller, H. Rouhani, and M. Tavakoli. 2023. “Performance and effectiveness of a passive back-support exoskeleton in manual material handling tasks in the construction industry.” Constr Robot. https://doi.org/10.1007/s41693-023-00097-4.
Google. 2023. “Pose landmark detection guide | MediaPipe.” Google Developers. Accessed January 15, 2023. https://developers.google.com/mediapipe/solutions/vision/pose_landmarker.
Halder, S., K. Afsari, J. Serdakowski, and S. DeVito. 2021. “A Methodology for BIM-enabled Automated Reality Capture in Construction Inspection with Quadruped Robots.” Proceedings of the International Symposium on Automation and Robotics in Construction, 17–24.
Hignett, S., and L. McAtamney. 2000. “Rapid Entire Body Assessment (REBA).” Applied Ergonomics, 31 (2): 201–205. https://doi.org/10.1016/S0003-6870(99)00039-3.
Luo, H., C. Xiong, W. Fang, P. E. D. Love, B. Zhang, and X. Ouyang. 2018a. “Convolutional neural networks: Computer vision-based workforce activity assessment in construction.” Automation in Construction, 94: 282–289. https://doi.org/10.1016/j.autcon.2018.06.007.
Luo, X., H. Li, D. Cao, Y. Yu, X. Yang, and T. Huang. 2018b. “Towards efficient and objective work sampling: Recognizing workers’ activities in site surveillance videos with two-stream convolutional networks.” Automation in Construction, 94: 360–370. https://doi.org/10.1016/j.autcon.2018.07.011.
Luo, X., H. Li, X. Yang, Y. Yu, and D. Cao. 2019. “Capturing and Understanding Workers’ Activities in Far-Field Surveillance Videos with Deep Action Recognition and Bayesian Nonparametric Learning.” Computer-Aided Civil and Infrastructure Engineering, 34 (4): 333–351. John Wiley & Sons, Ltd. https://doi.org/10.1111/mice.12419.
Pesenti, M., G. Invernizzi, J. Mazzella, M. Bocciolone, A. Pedrocchi, and M. Gandolla. 2023. “IMU-based human activity recognition and payload classification for low-back exoskeletons.” Scientific Reports, 13 (1): 1184. https://doi.org/10.1038/s41598-023-28195-x.
Pham, H. C., N. Dao, A. Pedro, Q. T. Le, R. Hussain, S. Cho, and C. Park. 2018. “Virtual field trip for mobile construction safety education using 360-degree panoramic virtual reality.” Int. J. Eng. Educ, 34 (4): 1174–1191.
Polygoncraft. 2023. “Construction Worker Motions Pack 1 | 3D Animations | Unity Asset Store.” Accessed April 25, 2023. https://assetstore.unity.com/packages/3d/animations/construction-worker-motions-pack-1-111378.
Ray, S. J., and J. Teizer. 2012. “Real-time construction worker posture analysis for ergonomics training.” Advanced Engineering Informatics, 26 (2): 439–455. Elsevier. https://doi.org/10.1016/j.aei.2012.02.011.
Cheng, T., G. C. Migliaccio, J. Teizer, and U. C. Gatti. 2013. “Data Fusion of Real-Time Location Sensing and Physiological Status Monitoring for Ergonomics Analysis of Construction Workers.” Journal of Computing in Civil Engineering, 27 (3): 320–335. American Society of Civil Engineers. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000222.
Tao, Y., H. Hu, F. Xu, and Z. Zhang. 2023. “Ergonomic Risk Assessment of Construction Workers and Projects Based on Fuzzy Bayesian Network and D-S Evidence Theory.” Journal of Construction Engineering and Management, 149 (6): 04023034. American Society of Civil Engineers. https://doi.org/10.1061/JCEMD4.COENG-12821.
TensorFlow Developers. 2023. TensorFlow (v2.12.0). Zenodo.
Wang, X., X. S. Dong, S. D. Choi, and J. Dement. 2017. “Work-related musculoskeletal disorders among construction workers in the United States from 1992 to 2014.” Occup Environ Med, 74 (5): 374–380. England. https://doi.org/10.1136/oemed-2016-103943.
Xu, S., J. Wang, W. Shou, T. Ngo, A.-M. Sadick, and X. Wang. 2021. “Computer Vision Techniques in Construction: A Critical Review.” Archives of Computational Methods in Engineering, 28 (5): 3383–3397. https://doi.org/10.1007/s11831-020-09504-3.
Yan, X., H. Li, A. R. Li, and H. Zhang. 2017a. “Wearable IMU-based real-time motion warning system for construction workers’ musculoskeletal disorders prevention.” Automation in Construction, 74: 2–11. https://doi.org/10.1016/j.autcon.2016.11.007.
Yan, X., H. Li, C. Wang, J. O. Seo, H. Zhang, and H. Wang. 2017b. “Development of ergonomic posture recognition technique based on 2D ordinary camera for construction hazard prevention through view-invariant features in 2D skeleton motion.” Advanced Engineering Informatics, 34: 152–163. Elsevier Ltd. https://doi.org/10.1016/j.aei.2017.11.001.
Yu, Y., X. Yang, H. Li, X. Luo, H. Guo, and Q. Fang. 2019. “Joint-Level Vision-Based Ergonomic Assessment Tool for Construction Workers.” Journal of Construction Engineering and Management, 145 (5): 04019025. American Society of Civil Engineers. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001647.
Zhao, J., and E. Obonyo. 2018. “Towards a Data-Driven Approach to Injury Prevention in Construction.” Advanced Computing Strategies for Engineering, Lecture Notes in Computer Science, I. F. C. Smith and B. Domer, eds., 385–411. Cham: Springer International Publishing.
Zhao, J., and E. Obonyo. 2020. “Convolutional long short-term memory model for recognizing construction workers’ postures from wearable inertial measurement units.” Advanced Engineering Informatics, 46: 101177. https://doi.org/10.1016/j.aei.2020.101177.
Zhao, J., and E. Obonyo. 2021. “Applying incremental Deep Neural Networks-based posture recognition model for ergonomics risk assessment in construction.” Advanced Engineering Informatics, 50: 101374. https://doi.org/10.1016/j.aei.2021.101374.
Information & Authors
Information
Published In
Construction Research Congress 2024
Pages: 678 - 687
History
Published online: Mar 18, 2024
ASCE Technical Topics:
- Artificial intelligence and machine learning
- Business management
- Computer programming
- Computer vision and image processing
- Computing in civil engineering
- Disaster risk management
- Employment
- Engineering fundamentals
- Equipment and machinery
- Labor
- Methodology (by type)
- Neural networks
- Personnel management
- Practice and Profession
- Probe instruments
- Risk management
- Structural engineering
- Structural members
- Structural systems
- Tendons
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.