Synthetic Training Image Dataset for Vision-Based 3D Pose Estimation of Construction Workers
Publication: Construction Research Congress 2022
ABSTRACT
Vision-based 3D pose estimation of construction workers has drawn attention for its usefulness in occupational ergonomics, safety, and productivity analysis. However, it is still challenging to develop an extensive training image dataset, which is essential for deep neural network-powered approaches, thus inhibiting the maximum potential of vision-based 3D pose estimation. To address this issue, we built a synthetic training image dataset and validated its effectiveness for 3D pose estimation. We trained and tested a state-of-the-art 3D pose estimation architecture using these synthetic images. The results show that the synthetic data-trained model can estimate 3D poses of construction workers with a Mean Per-Joint Position Error of 50.24 mm—comparable to real-data-trained model (46.5 mm). This finding indicates that synthesized construction images are effective in training a 3D pose estimation model, thus enabling the development of more accurate and scalable 3D pose estimation and alleviating the shortage of real-world construction training data.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Bogo, F., Kanazawa, A., Lassner, C., Gehler, P., Romero, J., and Black, M. J. (2016). “Keep it SMPL: Automatic estimation of 3D human pose and shape from a single image.” Lecture Notes in Computer Science, 9909 LNCS, 561–578.
Braun, A., and Borrmann, A. (2019). “Combining inverse photogrammetry and BIM for automated labeling of construction site images for machine learning.” Automation in Construction, 106, 102879.
Chen, T., Fang, C., Shen, X., Zhu, Y., Chen, Z., and Luo, J. (2021). “Anatomy-aware 3D human pose estimation with bone-based pose decomposition.” IEEE Transactions on Circuits and Systems for Video Technology, 1–11.
Chu, W., Han, S., Luo, X., and Zhu, Z. (2020). “Monocular vision–based framework for biomechanical analysis or ergonomic posture assessment in modular construction.” Journal of Computing in Civil Engineering, 34(4), 04020018.
CMU Graphics Lab. (2008). “CMU Graphics Lab Motion Capture Database.” <http://mocap.cs.cmu.edu/>(Apr. 7, 2021).
Han, S., and Lee, S. (2013). “A vision-based motion capture and recognition framework for behavior-based safety management.” Automation in Construction, 35, 131–141.
Han, S., Lee, S., and Peña-Mora, F. (2013). “Vision-based detection of unsafe actions of a construction worker: Case study of ladder climbing.” Journal of Computing in Civil Engineering, 27(6), 635–644.
Ionescu, C., Papava, D., Olaru, V., and Sminchisescu, C. (2014). “Human3.6M: Large scale datasets and predictive methods for 3D human sensing in natural environments.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(7), 1325–1339.
Kim, J. (2020). “Visual analytics for operation-level construction monitoring and documentation: state-of-the-art technologies, research challenges, and future directions.” Frontiers in Built Environment, 6, 575738.
Kim, J., and Chi, S. (2020). “Multi-camera vision-based productivity monitoring of earthmoving operations.” Automation in Construction, 112, 103121.
Kim, J., and Chi, S. (2021). “A few-shot learning approach for database-free vision-based monitoring on construction sites.” Automation in Construction, 124, 103566.
Kim, J., Chi, S., and Ahn, C. R. (2021). “Hybrid kinematic–visual sensing approach for activity recognition of construction equipment.” Journal of Building Engineering, 44, 102709.
Kim, J., Hwang, J., Chi, S., and Seo, J. (2020). “Towards database-free vision-based monitoring on construction sites: A deep active learning approach.” Automation in Construction, 120, 103376.
Liu, M., Han, S., and Lee, S. (2016). “Tracking-based 3D human skeleton extraction from stereo video camera toward an on-site safety and ergonomic analysis.” Construction Innovation, 16(3), 348–367.
Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., and Black, M. J. (2015). “SMPL: A skinned multi-person linear model.” ACM Transactions on Graphics, 34(6), 1–16.
Pavllo, D., Feichtenhofer, C., Grangier, D., and Auli, M. (2019). “3D human pose estimation in video with temporal convolutions and semi-supervised training.” Proceedings of 2019 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, Long Beach, CA, USA, 7753–7762.
Roberts, D., Calderon, W. T., Tang, S., and Golparvar-Fard, M. (2020). “Vision-based construction worker activity analysis informed by body posture.” Journal of Computing in Civil Engineering, 34(4), 04020017.
Seo, J., and Lee, S. (2021). “Automated postural ergonomic risk assessment using vision-based posture classification.” Automation in Construction, 128, 103725.
Seo, J., Starbuck, R., Han, S., Lee, S., and Armstrong, T. J. (2015). “Motion data-driven biomechanical analysis during construction tasks on sites.” Journal of Computing in Civil Engineering, 29(4), B4014005.
Soltani, M. M., Zhu, Z., and Hammad, A. (2016). “Automated annotation for visual recognition of construction resources using synthetic images.” Automation in Construction, 62, 14–23.
Torres Calderon, W., Roberts, D., and Golparvar-Fard, M. (2021). “Synthesizing pose sequences from 3D assets for vision-based activity analysis.” Journal of Computing in Civil Engineering, 35(1), 04020052.
Varol, G., Romero, J., Martin, X., Mahmood, N., Black, M. J., Laptev, I., and Schmid, C. (2017). “Learning from synthetic humans.” Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Honolulu, HI, USA, 109–117.
Yu, Y., Yang, X., Li, H., Luo, X., Guo, H., and Fang, Q. (2019). “Joint-level vision-based ergonomic assessment tool for construction workers.” Journal of Construction Engineering and Management, 145(5), 04019025.
Zhu, X., Vondrick, C., Fowlkes, C. C., and Ramanan, D. (2016). “Do we need more training data?” International Journal of Computer Vision, 119, 76–92.
Information & Authors
Information
Published In
Construction Research Congress 2022
Pages: 254 - 262
History
Published online: Mar 7, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.