Automated Action Recognition Using an Accelerometer-Embedded Wristband-Type Activity Tracker
Publication: Journal of Construction Engineering and Management
Volume 145, Issue 1
Abstract
Automated worker action recognition helps to understand the state of workers’ actions, enabling effective management of work performance in terms of productivity, safety, and health issues. A wristband equipped with an accelerometer (e.g., activity tracker) allows to collect the data related to workers’ hand activities without interfering with their ongoing work. Considering that many construction activities involve unique hand movements, the use of acceleration data from a wristband has great potential for action recognition of construction activities. In this context, the authors examine the feasibility of the wrist-worn accelerometer-embedded activity tracker for automated action recognition. Specifically, masonry work was conducted to collect acceleration data in a laboratory. The classification accuracy of four classifiers—the k-nearest neighbor, multilayer perceptron, decision tree, and multiclass support vector machine—was analyzed with different window sizes to investigate classification performance. It was found that the multiclass support vector machine with a 4-s window size showed the best accuracy (88.1%) to classify four different subtasks of masonry work. The present study makes noteworthy contributions to the current body of knowledge. First, the study allows for automatic construction action recognition using a single wrist-worn sensor without interfering with workers’ ongoing work, which can be widely deployed to construction sites. The use of a single sensor also greatly reduces the burden to carry multiple sensors while also reducing computational cost and memory. Second, influences associated with the variability of movement between subject and experience group were examined; thus, a consideration of data acquisition that reflects the characteristics of workers’ actions is suggested.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Data generated or analyzed during the study are available from the corresponding author by request. Information about the Journal’s data-sharing policy can be found here: http://ascelibrary.org/doi/10.1061/(ASCE)CO.1943-7862.0001263.
Acknowledgments
The authors would like to acknowledge colleagues from the Ontario Masonry Training Centre at Conestoga College in Waterloo, Canada, for their considerable help in collecting data, as well as the contributions of Professor Carl T. Haas, Professor Eihab Abdel-Rahman, and Dr. Abdullatif Alwasel at the University of Waterloo for their collaboration in the data acquisition phase.
References
Ahn, C. R., S. Lee, and F. Peña-Mora. 2013. “Accelerometer-based measurement of construction equipment operating efficiency for monitoring environmental performance.” In Proc., Int. Workshop on Computing in Civil Engineering, 565–572. Reston, VA: ASCE.
Akhavian, R., and A. H. Behzadan. 2014. “Construction activity recognition for simulation input modeling using machine learning classifiers.” In Proc., 2014 Winter Simulation Conf., 3296–3307. New York: IEEE.
Akhavian, R., and A. H. Behzadan. 2016. “Smartphone-based construction workers’ activity recognition and classification.” Autom. Constr. 71: 198–209. https://doi.org/10.1016/j.autcon.2016.08.015.
Alwasel, A., E. M. Abdel-Rahman, C. T. Haas, and S. Lee. 2017a. “Experience, productivity, and musculoskeletal injury among masonry workers.” J. Constr. Eng. Manage. 143 (6): 05017003. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001308.
Alwasel, A., A. Sabet, M. Nahangi, C. T. Haas, and E. Abdel-Rahman. 2017b. “Identifying poses of safe and productive masons using machine learning.” Autom. Constr. 84 (Dec): 345–355. https://doi.org/10.1016/j.autcon.2017.09.022.
Ann, O. C., and L. B. Theng. 2014. “Human activity recognition: A review.” In Proc., IEEE Int. Conf. on Control System, Computing and Engineering (ICCSCE), 389–393. New York: IEEE.
Arndt, V., D. Rothenbacher, U. Daniel, B. Zschenderlein, S. Schuberth, and H. Brenner. 2005. “Construction work and risk of occupational disability: A ten year follow up of 14 474 male workers.” Occup. Environ. Med. 62 (8): 559–566. https://doi.org/10.1136/oem.2004.018135.
Atallah, L., B. Lo, R. King, and G. Z. Yang. 2011. “Sensor positioning for activity recognition using wearable accelerometers.” IEEE Trans. Biomed. Circuits Syst. 5 (4): 320–329. https://doi.org/10.1109/TBCAS.2011.2160540.
Banos, O., J. M. Galvez, M. Damas, H. Pomares, and I. Rojas. 2014. “Window size impact in human activity recognition.” Sensors 14 (4): 6474–6499. https://doi.org/10.3390/s140406474.
Bao, L., and S. S. Intille. 2004. “Activity recognition from user-annotated acceleration data.” In Int. Conf. on Pervasive Computing. 1–17. Berlin: Springer.
Beak, J., G. Lee, W. Park, and B. J. Yun. 2004. “Accelerometer signal processing for user activity detection.” In Knowledge-based intelligent information and engineering systems. 610–617. Berlin: Springer.
Bouten, C. V. C., K. T. M. Koekkoek, M. Verduin, R. Kodde, and J. D. Janssen. 1997. “A triaxial accelerometer and portable data processing unit for the assessment of daily physical activity.” IEEE Trans. Biomed. Eng. 44 (3): 136–147. https://doi.org/10.1109/10.554760.
Brilakis, I., M. W. Park, and G. Jog. 2011. “Automated vision tracking of project related entities.” Adv. Eng. Inf. 25 (4): 713–724. https://doi.org/10.1016/j.aei.2011.01.003.
Bulling, A., U. Blanke, and B. Schiele. 2014. “A tutorial on human activity recognition using body-worn inertial sensors.” ACM Comput. Surv. (CSUR) 46 (3): 33. https://doi.org/10.1145/2499621.
Carbonari, A., A. Giretti, and B. Naticchia. 2011. “A proactive system for real-time safety management in construction sites.” Autom. Constr. 20 (6): 686–698. https://doi.org/10.1016/j.autcon.2011.04.019.
Cheng, T., G. C. Migliaccio, J. Teizer, and U. C. Gatti. 2012. “Data fusion of real-time location sensing and physiological status monitoring for ergonomics analysis of construction workers.” J. Comput. Civ. Eng. 27 (3): 320–335. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000222.
Cheng, T., J. Teizer, G. C. Migliaccio, and U. C. Gatti. 2013. “Automated task-level activity analysis through fusion of real time location sensors and worker’s thoracic posture data.” Autom. Constr. 29 (Jan): 24–39. https://doi.org/10.1016/j.autcon.2012.08.003.
Chernbumroong, S., A. S. Atkins, and H. Yu. 2011. “Activity classification using a single wrist-worn accelerometer.” In Proc., 5th Int. Conf. on Software, Knowledge Information, Industrial Management and Applications (SKIMA), 1–6. New York: IEEE.
Escorcia, V., M. A. Dávila, M. Golparvar-Fard, and J. C. Niebles. 2012. “Automated vision-based recognition of construction worker actions for building interior construction operations using RGBD cameras.” In Vol. 2012 of Proc., Construction Research Congress, 879–888. Reston, VA: ASCE.
Everett, J. G., and A. H. Slocum. 1994. “Automation and robotics opportunities: Construction versus manufacturing.” J. Constr. Eng. Manage. 120 (2): 443–452. https://doi.org/10.1061/(ASCE)0733-9364(1994)120:2(443).
Figo, D., P. C. Diniz, D. R. Ferreira, and J. M. Cardoso. 2010. “Preprocessing techniques for context recognition from accelerometer data.” Pers. Ubiquitous Comput. 14 (7): 645–662. https://doi.org/10.1007/s00779-010-0293-9.
Franc, V., and V. Hlavác. 2002. “Multi-class support vector machine.” Proc., 16th Int. Conf. on Pattern Recognition, 236–239. New York: IEEE.
Gatti, U. C., G. C. Migliaccio, S. M. Bogus, and S. Schneider. 2014. “An exploratory study of the relationship between construction workforce physical strain and task level productivity.” Constr. Manage. Econ. 32 (6): 548–564. https://doi.org/10.1080/01446193.2013.831463.
Golparvar-Fard, M., A. Heydarian, and J. C. Niebles. 2013. “Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers.” Adv. Eng. Inf. 27 (4): 652–663. https://doi.org/10.1016/j.aei.2013.09.001.
Gupta, P., and T. Dallas. 2014. “Feature selection and activity recognition system using a single triaxial accelerometer.” IEEE Trans. Biomed. Eng. 61 (6): 1780–1786. https://doi.org/10.1109/TBME.2014.2307069.
Hall, M. A. 1999. “Correlation-based feature selection for machine learning.” Ph.D. dissertation, Dept. of Computer Science, Univ. of Waikato.
Han, S., M. Achar, S. Lee, and F. Peña-Mora. 2013. “Empirical assessment of a RGB-D sensor on motion capture and action recognition for construction worker monitoring.” Visual. Eng. 1 (1): 6–13. https://doi.org/10.1186/2213-7459-1-6.
Han, S., and S. Lee. 2013. “A vision-based motion capture and recognition framework for behavior-based safety management.” Autom. Constr. 35 (Nov): 131–141. https://doi.org/10.1016/j.autcon.2013.05.001.
Hsu, C.-W., and C.-J. Lin. 2002. “A comparison of methods for multiclass support vector machines.” IEEE Trans. Neural Networks 13 (2): 415–425. https://doi.org/10.1109/72.991427.
Huynh, T., and B. Schiele. 2005. “Analyzing features for activity recognition.” In Proc., 2005 Joint Conf. on Smart Objects and Ambient Intelligence: Innovative Context-Aware Services: Usages and Technologies, 159–163. New York: Association for Computing Machinery.
Hwang, S., J. Seo, H. Jebelli, and S. Lee. 2016a. “Feasibility analysis of heart rate monitoring of construction workers using a photoplethysmography (PPG) sensor embedded in a wristband-type activity tracker.” Autom. Constr. 71 (Nov): 372–381. https://doi.org/10.1016/j.autcon.2016.08.029.
Hwang, S., J. Seo, J. Ryu, and S. Lee. 2016b. “Challenges and opportunities of understanding construction workers’ physical demands through field energy expenditure measurements using a wearable activity tracker.” In Proc., Construction Research Congress 2016, 2730–2739. Reston, VA: ASCE.
Jaselskis, E. J., and T. El-Misalami. 2003. “Implementing radio frequency identification in the construction process.” J. Constr. Eng. Manage. 129 (6): 680–688. https://doi.org/10.1061/(ASCE)0733-9364(2003)129:6(680).
Jebelli, H., C. R. Ahn, and T. L. Stentz. 2014. “The validation of gait-stability metrics to assess construction workers’ fall risk.” In Proc., Computing in Civil and Building Engineering 2014, 997–1004. Reston, VA: ASCE.
Jebelli, H., C. R. Ahn, and T. L. Stentz. 2015. “Comprehensive fall-risk assessment of construction workers using inertial measurement units: Validation of the gait-stability metric to assess the fall risk of iron workers.” J. Comput. Civ. Eng. 30 (3): 04015034. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000511.
Jebelli, H., C. R. Ahn, and T. L. Stentz. 2016. “Fall risk analysis of construction workers using inertial measurement units: Validating the usefulness of the postural stability metrics in construction.” Saf. Sci. 84 (Apr): 161–170. https://doi.org/10.1016/j.ssci.2015.12.012.
Jebelli, H., K. Yang, M. M. Khalili, C. Ahn, and T. Stentz. 2018. “Assessing the effects of tool-loading formation on construction workers’ postural stability.” In Proc., Construction Research Congress 2018, 292–302. Reston, VA: ASCE.
Joshua, L., and K. Varghese. 2010. “Accelerometer-based activity recognition in construction.” J. Comput. Civ. Eng. 25 (5): 370–379. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000097.
Joshua, L., and K. Varghese. 2014. “Automated recognition of construction labour activity using accelerometers in field situations.” Int. J. Productivity Perform. Manage. 63 (7): 841–862. https://doi.org/10.1108/IJPPM-05-2013-0099.
Ke, S. R., H. L. U. Thuc, Y. J. Lee, J. N. Hwang, J. H. Yoo, and K. H. Choi. 2013. “A review on video-based human activity recognition.” Computers 2 (2): 88–131. https://doi.org/10.3390/computers2020088.
Kim, H., C. R. Ahn, T. L. Stentz, and H. Jebelli. 2018. “Assessing the effects of slippery steel beam coatings to ironworkers’ gait stability.” Appl. Ergon. 68 (Apr): 72–79. https://doi.org/10.1016/j.apergo.2017.11.003.
Kohavi, R. 1995. “A study of cross-validation and bootstrap for accuracy estimation and model selection.” In Vol. 14 of Ijcai, 1137–1145. San Francisco: Morgan Kaufmann Publishers.
Koskimaki, H., V. Huikari, P. Siirtola, P. Laurinen, and J. Roning. 2009. “Activity recognition using a wrist-worn inertial measurement unit: A case study for industrial assembly lines.” In Proc., 17th Mediterranean Conf. on Control and Automation, 2009. MED’09, 401–405. New York: IEEE.
Kreil, M., B. Sick, and P. Lukowicz. 2014. “Dealing with human variability in motion based, wearable activity recognition.” In Proc., IEEE Int. Conf. on Pervasive Computing and Communications Workshops (PERCOM Workshops), 36–40. New York: IEEE.
Laguna, J. O., A. G. Olaya, and D. Borrajo. 2011. “A dynamic sliding window approach for activity recognition.” In Proc., Int. Conf. on User Modeling, Adaptation, and Personalization, 219–230. Berlin: Springer.
Lim, T. K., S. M. Park, H. C. Lee, and D. E. Lee. 2015. “Artificial neural network–based slip-trip classifier using smart sensor for construction workplace.” J. Constr. Eng. Manage. 142 (2): 04015065. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001049.
Lukowicz, P., J. A. Ward, H. Junker, M. Stäger, G. Tröster, A. Atrash, and T. Starner. 2004. “Recognizing workshop activity using body worn microphones and accelerometers.” In Proc., Int. Conf. on Pervasive Computing, 18–32. Berlin: Springer.
Madeleine, P., and T. M. T. Madsen. 2009. “Changes in the amount and structure of motor variability during a deboning process are associated with work experience and neck–shoulder discomfort.” Appl. Ergon. 40 (5): 887–894. https://doi.org/10.1016/j.apergo.2008.12.006.
Mannini, A., S. S. Intille, M. Rosenberger, A. M. Sabatini, and W. Haskell. 2013. “Activity recognition using a single accelerometer placed at the wrist or ankle.” Med. Sci. Sports Exercise 45 (11): 2193. https://doi.org/10.1249/MSS.0b013e31829736d6.
Menai, M. E. B., F. J. Mohder, and F. Al-mutairi. 2013. “Influence of feature selection on naïve Bayes classifier for recognizing patterns in cardiotocograms.” J. Med. Bioeng. 2 (1): 66–70. https://doi.org/10.12720/jomb.2.1.66-70.
Montaser, A., and O. Moselhi 2014. “RFID indoor location identification for construction projects.” Autom. Constr. 39 (Apr): 167–179. https://doi.org/10.1016/j.autcon.2013.06.012.
Oikonomopoulos, A., and M. Pantic. 2013. “Human activity recognition using hierarchically-mined feature constellations.” In Proc., Int. Symp. on Visual Computing, 150–159. Berlin: Springer.
Pal, S. K., and S. Mitra. 1992. “Multilayer perceptron, fuzzy sets, and classification.” IEEE Trans. Neural Networks 3 (5): 683–697. https://doi.org/10.1109/72.159058.
Peddi, A., L. Huan, Y. Bai, and S. Kim. 2009. “Development of human pose analyzing algorithms for the determination of construction productivity in real-time.” In Vol. 1 of Proc., Construction Research Congress, 11–20. Reston, VA: ASCE.
Preece, S. J., J. Y. Goulermas, L. P. Kenney, D. Howard, K. Meijer, and R. Crompton. 2009. “Activity identification using body-mounted sensors—A review of classification techniques.” Physiol. Meas. 30 (4): R1. https://doi.org/10.1088/0967-3334/30/4/R01.
Qian, H., Y. Mao, W. Xiang, and Z. Wang. 2010. “Recognition of human activities using SVM multi-class classifier.” Pattern Recognit. Lett. 31 (2): 100–111. https://doi.org/10.1016/j.patrec.2009.09.019.
Ravi, N., N. Dandekar, P. Mysore, and M. L. Littman. 2005. “Activity recognition from accelerometer data.” In Vol. 5 of AAAI, 1541–1546. Palo Alto, CA: Association for the Advancement of Artificial Intelligence.
Refaeilzadeh, P., L. Tang, and H. Liu. 2009. “Cross-validation.” In Encyclopedia of database systems. 532–538. New York: Springer.
Robnik-Šikonja, M., and I. Kononenko. 2003. “Theoretical and empirical analysis of ReliefF and RReliefF.” Mach. Learn. 53 (1–2): 23–69. https://doi.org/10.1023/A:1025667309714.
Ryu, J., J. Seo, M. Liu, S. Lee, and C. T. Haas. 2016. “Action recognition using a wristband-type activity tracker: Case study of masonry work.” In Proc., Construction Research Congress 2016, 790–799. Reston, VA: ASCE.
Seo, J., S. Han, S. Lee, and H. Kim. 2015. “Computer vision techniques for construction safety and health monitoring.” Adv. Eng. Inf. 29 (2): 239–251. https://doi.org/10.1016/j.aei.2015.02.001.
Seo, J., S. Lee, and J. Seo. 2016. “Simulation-based assessment of workers’ muscle fatigue and its impact on construction operations.” J. Constr. Eng. Manage. 142 (11): 04016063. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001182.
Shoaib, M., S. Bosch, O. D. Incel, H. Scholten, and P. J. Havinga. 2016. “Complex human activity recognition using smartphone and wrist-worn motion sensors.” Sensors 16 (4): 426. https://doi.org/10.3390/s16040426.
Sun, L., D. Zhang, B. Li, B. Guo, and S. Li. 2010. “Activity recognition on an accelerometer embedded mobile phone with varying positions and orientations.” In Proc., Int. Conf. on Ubiquitous Intelligence and Computing, 548–562. Berlin: Springer.
Sutton, O. 2012. Introduction to k-nearest neighbour classification and condensed nearest neighbour data reduction. Leicester, UK: Univ. of Leicester.
Suykens, J. A., and J. Vandewalle. 1999. “Least squares support vector machine classifiers.” Neural Process. Lett. 9 (3): 293–300. https://doi.org/10.1023/A:1018628609742.
Taneja, S., B. Akinci, J. H. Garrett, L. Soibelman, E. Ergen, A. Pradhan, and E. B. Anil. 2010. “Sensing and field data capture for construction and facility operations.” J. Constr. Eng. Manage. 137 (10): 870–881. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000332.
Texas Instruments. 2010. eZ430-Chronos development tool user’s guide. Dallas: Texas Instruments.
Tsai, M. K. 2014. “Automatically determining accidental falls in field surveying: A case study of integrating accelerometer determination and image recognition.” Saf. Sci. 66 (Jul): 19–26. https://doi.org/10.1016/j.ssci.2014.01.012.
Umer, W., H. Li, G. P. Y. Szeto, and A. Y. L. Wong. 2016. “Identification of biomechanical risk factors for the development of lower-back disorders during manual rebar tying.” J. Constr. Eng. Manage. 143 (1): 04016080. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001208.
Valero, E., A. Sivanathan, F. Bosché, and M. Abdel-Wahab. 2016. “Musculoskeletal disorders in construction: A review and a novel system for activity tracking with body area network.” Appl. Ergon. 54 (May): 120–130. https://doi.org/10.1016/j.apergo.2015.11.020.
Valero, E., A. Sivanathan, F. Bosché, and M. Abdel-Wahab. 2017. “Analysis of construction trade worker body motions using a wearable and wireless motion sensor network.” Autom. Constr. 83 (Nov): 48–55. https://doi.org/10.1016/j.autcon.2017.08.001.
Ward, J. A., P. Lukowicz, G. Troster, and T. E. Starner. 2006. “Activity recognition of assembly tasks using body-worn microphones and accelerometers.” IEEE Trans. Pattern Anal. Mach. Intell. 28 (10): 1553–1567. https://doi.org/10.1109/TPAMI.2006.197.
Weerasinghe, I. T., and J. Y. Ruwanpura. 2009. “Automated data acquisition system assess construction worker performance.” In Proc., Construction Research Congress 2009: Building a Sustainable Future, 61–70. Reston, VA: ASCE.
Weston, J., and C. Watkins. 1998. Multi-class support vector machines. London: Dept. of Computer Science, Royal Holloway, Univ. of London.
Witten, I. H., and E. Frank. 2005. Data mining: Practical machine learning tools and techniques. San Francisco: Morgan Kaufmann.
Yang, C. C., and Y. L. Hsu. 2010. “A review of accelerometry-based wearable motion detectors for physical activity monitoring.” Sensors 10 (8): 7772–7788. https://doi.org/10.3390/s100807772.
Yang, J. Y., J. S. Wang, and Y. P. Chen. 2008. “Using acceleration measurements for activity recognition: An effective learning algorithm for constructing neural classifiers.” Pattern Recognit. Lett. 29 (16): 2213–2220. https://doi.org/10.1016/j.patrec.2008.08.002.
Yang, K., H. Jebelli, C. Ahn, and M. Vuran. 2015. “Threshold-based approach to detect near-miss falls of iron workers using inertial measurement units.” In Proc., Computing in Civil Engineering 2015, 148–155. Reston, VA: ASCE.
Zappi, P., T. Stiefmeier, E. Farella, D. Roggen, L. Benini, and G. Tröster. 2007. “Activity recognition from on-body sensors by classifier fusion: Sensor scalability and robustness.” In Proc., 3rd Int. Conf. on Intelligent Sensors, Sensor Networks and Information, 2007. ISSNIP 2007, 281–286. New York: IEEE.
Zunino, A., J. Cavazza, and V. Murino. 2017 “Revisiting human action recognition: Personalization vs. generalization.” In Proc., Int. Conf. on Image Analysis and Processing, 469–480. Cham, Switzerland: Springer.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 145 • Issue 1 • January 2019
Copyright
©2018 American Society of Civil Engineers.
History
Received: Jan 22, 2018
Accepted: Jun 27, 2018
Published online: Oct 24, 2018
Published in print: Jan 1, 2019
Discussion open until: Mar 24, 2019
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.