Automatic Recognition of Workers’ Motions in Highway Construction by Using Motion Sensors and Long Short-Term Memory Networks
Publication: Journal of Construction Engineering and Management
Volume 147, Issue 3
Abstract
Monitoring and understanding construction workers’ behavior and working conditions are essential to achieve success in construction projects. The dynamic nature of construction sites has heightened the awareness of the need for improved monitoring of individual workers on sites. Although several studies indicated promising results in automated motion and activity recognition using wearable motion sensors, their technical and practical feasibility was not properly validated at actual job sites. Motion recognition models have to be evaluated in actual conditions because the motion sensor data collected in controlled conditions, and actual conditions can have different characteristics. This study proposes Long Short-Term Memory (LSTM) networks for recognizing construction workers’ motions. The LSTM networks were validated through case studies in one bridge construction site and two road pavement sites. The LSTM networks indicated classification accuracies of 97.6%, 95.93%, and 97.36% from three different field test sites, respectively. Through the case studies, the technical and practical feasibility of the LSTM networks was properly investigated. With LSTM networks, individual workers’ behavior and working conditions are expected to be automatically monitored and managed without excessive manual observation.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all of the data, models, or codes that support the findings of this study are available from the corresponding author on reasonable request.
Acknowledgments
This material is based on work supported by the National Science Foundation through Grant No. (1919068) and the Georgia Department of Transportation (GDOT) (RP18-17). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation and the Georgia Department of Transportation.
References
Akhavian, R., and A. H. Behzadan. 2016a. “Productivity analysis of construction worker activities using smartphone sensors.” In Proc., 16th Int. Conf. on Computing in Civil and Building Engineering, 1067–1074. Tokyo: Japan Society of Civil Engineers.
Akhavian, R., and A. H. Behzadan. 2016b. “Smartphone-based construction workers’ activity recognition and classification.” Autom. Constr. 71 (Nov): 198–209. https://doi.org/10.1016/j.autcon.2016.08.015.
Alwasel, A., A. Sabet, M. Nahangi, C. T. Haas, and E. Abdel-Rahman. 2017. “Identifying poses of safe and productive masons using machine learning.” Autom. Constr. 84 (Dec): 345–355. https://doi.org/10.1016/j.autcon.2017.09.022.
Antwi-Afari, M. F., H. Li, J. Seo, and A. Y. L. Wong. 2018. “Automated detection and classification of construction workers’ loss of balance events using wearable insole pressure sensors.” Autom. Constr. 96 (Dec): 189–199. https://doi.org/10.1016/j.autcon.2018.09.010.
Boyd, K., K. H. Eng, and C. D. Page. 2013. Area under the precision-recall curve: Point estimates and confidence intervals, 451–466. Berlin: Springer.
Chen, J., J. Qiu, and C. Ahn. 2017. “Construction worker’s awkward posture recognition through supervised motion tensor decomposition.” Autom. Constr. 77 (May): 67–81. https://doi.org/10.1016/j.autcon.2017.01.020.
Cho, Y. K., J. H. Youn, and D. Martinez. 2010. “Error modeling for an untethered ultra-wideband system for construction indoor asset tracking.” Autom. Constr. 19 (1): 43–54. https://doi.org/10.1016/j.autcon.2009.08.001.
CPWR (Center for Construction Research and Training). 2018. The construction chart book: The US construction industry and its workers. Silver Spring, MD: CPWR.
Fang, Y., Y. K. Cho, S. Zhang, and E. Perez. 2016. “Case study of BIM and cloud-enabled real-time RFID indoor localization for construction management applications.” J. Constr. Eng. Manage. 142 (7): 05016003. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001125.
Ghate, P. R., A. B. More, and P. R. Minde. 2016. “Importance of measurement of labour productivity in construction.” Int. J. Res. Eng. Technol.5 (7): 413–417.
Jebelli, H., C. R. Ahn, and T. L. Stentz. 2015. “Comprehensive fall-risk assessment of construction workers using inertial measurement units: Validation of the gait-stability metric to assess the fall risk of iron workers.” J. Comput. Civ. Eng. 30 (3): 04015034. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000511.
Kim, K., J. Chen, and Y. K. Cho. 2019. “Evaluation of machine learning algorithms for worker’s motion recognition using motion sensors.” In Proc., ASCE 2019 Int. Conf. on Computing in Civil Engineering (i3CE). Reston, VA: ASCE.
Kim, K., and Y. K. Cho. 2020. “Effective inertial sensor quantity and locations on a body for deep learning-based worker’s motion recognition.” Autom. Constr. 113 (May): 103126. https://doi.org/10.1016/j.autcon.2020.103126.
Kingma, D. P., and J. Lei Ba. 2015. “ADAM: A method for stochastic optimization.” In Proc., 3rd Int. Conf. for Learning Representations. San Diego: Computational and Biological Learning Society.
McKinsey & Company. 2015. “The construction productivity imperative.” Accessed January 22, 2018. https://www.mckinsey.com/industries/capital-projects-and-infrastructure/our-insights/the-construction-productivity-imperative.
McKinsey Global Institute. 2017. Reinventing construction: A route to higher productivity. New York: McKinsey & Company.
Nath, N. D., and A. H. Behzadan. 2017. “Construction productivity and ergonomic assessment using mobile sensors and machine learning.” In Computing in civil engineering 2017, 434–441. Reston, VA: ASCE.
NIOSH (The National Institute for Occupational Safety and Health). 2019. “Ergonomics and musculoskeletal disorders.” Accessed July 18, 2019. https://www.cdc.gov/niosh/topics/ergonomics/default.html.
Ordóñez, F., and D. Roggen. 2016. “Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition.” Sensors 16 (1): 115. https://doi.org/10.3390/s16010115.
Park, J., and Y. K. Cho. 2017. “Development and evaluation of a probabilistic local search algorithm for complex dynamic indoor construction sites.” J. Comput. Civ. Eng. 31 (4): 04017015. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000658.
Pei, L., R. Guinness, and J. Kaistinen. 2015. “Cognitive phone for sensing human behavior.” In Encyclopedia of mobile phone behavior, edited by Z. Yan, 1138–1150. Hershey, PA: IGI Global.
Ryu, J., J. Seo, H. Jebelli, and S. Lee. 2019. “Automated action recognition using an accelerometer-embedded wristband-type activity tracker.” J. Constr. Eng. Manage. 145 (1): 04018114. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001579.
Ryu, J., J. Seo, M. Liu, S. Lee, and C. T. Haas. 2016. “Action recognition using a wristband-type activity tracker: Case study of masonry work.” In Proc., Construction Research Congress 2016, 790–799. Reston, VA: ASCE.
Su, X., H. Tong, and P. Ji. 2014. “Activity recognition with smartphone sensors.” Tsinghua Sci. Technol. 19 (3): 235–249. https://doi.org/10.1109/TST.2014.6838194.
Valero, E., A. Sivanathan, F. Bosché, and M. Abdel-Wahab. 2017. “Analysis of construction trade worker body motions using a wearable and wireless motion sensor network.” Autom. Constr. 83 (Nov): 48–55. https://doi.org/10.1016/j.autcon.2017.08.001.
Yan, X., H. Li, A. R. Li, and H. Zhang. 2017. “Wearable IMU-based real-time motion warning system for construction workers’ musculoskeletal disorders prevention.” Autom. Constr. 74 (Feb): 2–11. https://doi.org/10.1016/j.autcon.2016.11.007.
Yang, K., and C. R. Ahn. 2019. “Inferring workplace safety hazards from the spatial patterns of workers’ wearable data.” Adv. Eng. Inf. 41 (Aug): 100924. https://doi.org/10.1016/j.aei.2019.100924.
Yang, K., C. R. Ahn, and H. Kim. 2019. “Validating ambulatory gait assessment technique for hazard sensing in construction environments.” Autom. Constr. 98 (Feb): 302–309. https://doi.org/10.1016/j.autcon.2018.09.017.
Yang, K., C. R. Ahn, M. C. Vuran, and S. S. Aria. 2016. “Semi-supervised near-miss fall detection for ironworkers with a wearable inertial measurement unit.” Autom. Constr. 68 (Aug): 194–202. https://doi.org/10.1016/j.autcon.2016.04.007.
Zhao, J., and E. Obonyo. 2019. “Convolutional long short-term memory model for recognizing postures from wearable sensor.” In CEUR Workshop Proc. Belgium: CEUR-WS.
Zhao, Y., R. Yang, G. Chevalier, X. Xu, and Z. Zhang. 2018. “Deep residual Bidir-LSTM for human activity recognition using wearable sensors.” Math. Probl. Eng. 2018: 13. https://doi.org/10.1155/2018/7316954.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 147 • Issue 3 • March 2021
Copyright
© 2020 American Society of Civil Engineers.
History
Received: Jan 7, 2020
Accepted: Sep 30, 2020
Published online: Dec 28, 2020
Published in print: Mar 1, 2021
Discussion open until: May 28, 2021
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.