International Conference on Construction and Real Estate Management 2019
Computer Vision Technologies and Machine Learning Algorithms for Construction Safety Management: A Critical Review
Publication: ICCREM 2019: Innovative Construction Project Management and Construction Industrialization
ABSTRACT
Computer vision technologies (CV) have been widely used for object detecting, tracking, and recognizing in the construction industry with machine learning algorithms (ML). This paper aim to make a comprehensive and systematic review of 438 CV related papers in construction safety management (CSM), and 169 camera related computer vision technologies (CRCV) papers were extracted. Finally, 29 camera related computer vision technologies and machine learning algorithms (CRCV-ML) papers were recognized for analyzing. Data collection, chronological, project types, CRCV, and learning algorithms distributions are proposed objectively by three-stage searching process. In discussion section, main research findings together with trends and gaps were gained. These results may help researchers identify the gaps and trends of CV and ML, and can serve as a guidance for future application research.
Get full access to this chapter
View all available purchase options and get full access to this chapter.
REFERENCES
Alwasel, A., Sabet, A., Nahangi, M., Haas, C.T. and Abdel-Rahman, E. (2017). “Identifying poses of safe and productive masons using machine learning.” Automation in Construction, 84(2017), 345-355.
Ang, J.C., Mirzal, A., Haron, H. and Hamed, H.N.A. (2016). “Supervised, unsupervised, and semi-supervised feature selection: a review on gene selection.” Ieee-Acm Transactions on Computational Biology and Bioinformatics, 13(5), 971-989.
Azar, E.R. and McCabe, B. (2012a). “Automated visual recognition of dump trucks in construction videos.” Journal of Computing in Civil Engineering, 26(6), 769-781.
Azar, E.R. and McCabe, B. (2012b). “Part based model and spatial-temporal reasoning to recognize hydraulic excavators in construction images and videos.” Automation in Construction, 24(7), 194-202.
Balali, V., Jahangiri, A. and Machiani, S.G. (2017). “Multi-class US traffic signs 3D recognition and localization via image-based point cloud model using color candidate extraction and texture-based recognition.” Advanced Engineering Informatics, 32(2017),263-274.
Barlow, H.B. (1983). “Vision: A computational investigation into the human representation and processing of visual information: David Marr. San Francisco: W. H. Freeman, 1982. pp. xvi+397.” Journal of Mathematical Psychology, 27(1), 107-110.
British Machine Vision Association (BMVA). (2016). “What is computer vision?” <http://www.bmva.org/visionoverview> (Dec. 21, 2016).
Brueckner, P., Fuetterer, R., Holder, S., Lerm, S., Schlegel, A. and Werner, P. (2015). “High resolution acquisition and processing of fluorescent microscopic images for quality assurance of food.” Measurement, 61(2015), 123-128.
Chaiyasarn, K., Kim, T.K., Viola, F., Cipolla, R. and Soga, K. (2016). “Distortion-free image mosaicing for tunnel inspection based on robust cylindrical surface estimation through structure from motion.” Journal of Computing in Civil Engineering, 30(3), 04015045.
Chen, J., Fang, Y. and Cho, Y.K. (2017). “Real-time 3d crane workspace update using a hybrid visualization approach.” Journal of Computing in Civil Engineering, 31(5), 04017049.
Chen, Y.C., Chi, H.L., Kang, S.C. and Hsieh, S.H. (2016). “Attention-based user interface design for a tele-operated crane.” Journal of Computing in Civil Engineering, 30(3), 04015030.
Costin, A.M., Teizer, J. and Schoner, B. (2015). “RFID and Bim-enabled worker location tracking to support real-time building protocol control and data visualization.” Journal of Information Technology in Construction, (20), 495-517.
Ding, L.Y., Fang, W.L., Luo, H.B., Love, P.E.D., Zhong, B.T. and Xi, O.Y. (2018). “A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory.” Automation in Construction, 86(2018), 118-124.
Falcone, R.W., Feinberg, B., Hussain, Z., Kirz, J., Krebs, G.F., Padmore, H.A., Robin, D.S. and Robinson, A.L. (2007). “ALS renewal moves forward.” Nuclear Instruments & Methods in Physics Research Section a-Accelerators Spectrometers Detectors and Associated Equipment, 582(1), 1-4.
Fang, Q., Li, H., Luo, X., Ding, L., Luo, H. and Li, C. (2018a). “Computer vision aided inspection on falling prevention measures for steeplejacks in an aerial environment.” Automation in Construction, 93(2018), 148-164.
Fang, W.L., Ding, L.Y., Luo, H.B. and Love, P.E.D. (2018b). “Falls from heights: A computer vision-based approach for safety harness detection.” Automation in Construction, 91(2018), 53-61.
Fang, W.L., Ding, L.Y., Zhong, B.T., Love, P.E.D. and Luo, H.B. (2018c). “Automated detection of workers and heavy equipment on construction sites: A convolutional neural network approach.” Advanced Engineering Informatics, 37(2018), 139-149.
Fang, Y., Cho, Y.K. and Chen, J. (2016). “A framework for real-time pro-active safety assistance for mobile crane lifting operations.” Automation in Construction, 72(2016), 367-379.
Fremont, V., Bui, M.T., Boukerroui, D. and Letort, P. (2016). “Vision-based people detection system for heavy machine applications.” Sensors, 16(1), 128.
Ganah, A.A. and John, G.A. (2017). “BIM and project planning integration for on-site safety induction.” Journal of Engineering Design and Technology, 15(3), 341-354.
German, S., Brilakis, I. and DesRoches, R. (2012). “Rapid entropy-based detection and properties measurement of concrete spalling with machine vision for post-earthquake safety assessments.” Advanced Engineering Informatics, 26(4), 846-858.
Gibril, M.B.A., Shafri, H.Z.M. and Hamedianfar, A. (2017). “New semi-automated mapping of asbestos cement roofs using rule-based object-based image analysis and Taguchi optimization technique from WorldView-2 images.” International Journal of Remote Sensing, 38(2), 467-491.
Goh, Y.M., Ubeynarayana, C.U., Wong, K.L.X. and Guo, B.H.W. (2018). “Factors influencing unsafe behaviors: A supervised learning approach.” Accident Analysis and Prevention, 118(2018), 77-85.
Gualdi, G., Prati, A. and Cucchiara, R. (2011). “Contextual information and covariance descriptors for people surveillance: an application for safety of construction workers.” Eurasip Journal on Image and Video Processing, 1(2011), 1-16.
Guo, H., Yu, Y., Ding, Q. and Skitmore, M. (2018). “Image-and-skeleton-based parameterized approach to real-time identification of construction workers’ unsafe behaviors.” Journal of Construction Engineering and Management, 144(6), 04018042.
Hale, A. (2009). “Why safety performance indicators?” Safety Science, 47(4), 479-480.
Hasanzadeh, S., Esmaeili, B. and Dodd, M.D. (2018). “Examining the relationship between construction workers’ visual attention and situation awareness under fall and tripping hazard conditions: using mobile eye tracking.” Journal of Construction Engineering and Management, 144(7), 04018060.
Jeelani, I., Albert, A. and Han, K. (2018). “Automating and scaling personalized safety training using eye-tracking data.” Automation in Construction, 93(2018), 63-77.
Kabir, S., Rivard, P., He, D.C. and Thivierge, P. (2009). “Damage assessment for concrete structure using image processing techniques on acoustic borehole imagery.” Construction and Building Materials, 23(10), 3166-3174.
Keller, C.G., Enzweiler, M., Rohrbach, M., Fernandez Llorca, D., Schnoerr, C. and Gavrila, D.M. (2011). “The benefits of dense stereo for pedestrian detection.” Ieee Transactions on Intelligent Transportation Systems, 12(4), 1096-1106.
Khaloo, A., Lattanzi, D., Cunningham, K., Dell’Andrea, R. and Riley, M. (2018). “Unmanned aerial vehicle inspection of the Placer River Trail Bridge through image-based 3D modelling.” Structure and Infrastructure Engineering, 14(2018), 124-136.
Kim, H., Kim, H., Hong, Y.W. and Byun, H. (2018). “Detecting construction equipment using a region-based fully convolutional network and transfer learning.” Journal of Computing in Civil Engineering, 32(2), 04017082.
Kim, K., Kim, H. and Kim, H. (2017). “Image-based construction hazard avoidance system using augmented reality in wearable device.” Automation in Construction, 83(2017), 390-403.
Kolar, Z., Chen, H. and Luo, X. (2018). “Transfer learning and deep convolutional neural networks for safety guardrail detection in 2D images.” Automation in Construction, 89, 58-70.
Larizza, M.F., Zukerman, I., Bohnert, F., Busija, L., Bentley, S.A., Russell, R.A. and Rees, G. (2014). “In-home monitoring of older adults with vision impairment: exploring patients’, caregivers’ and professionals views.” Journal of the American Medical Informatics Association, 21(1), 56-63.
LeCun, Y., Bengio, Y. and Hinton, G. (2015). “Deep learning.” Nature, 521(2015), 436-444.
Lee, G., Cho, J., Ham, S., Lee, T., Lee, G., Yun, S.H. and Yang, H.J. (2012). “A BIM-and sensor-based tower crane navigation system for blind lifts.” Automation in Construction, 26(2012), 1-10.
Lee, J.H., Yoo, H.S., Kim, Y.S., Lee, J.B. and Cho, M.Y. (2006). “The development of a machine vision-assisted, teleoperated pavement crack sealer.” Automation in Construction, 15(5), 616-626.
Lee, J., Lee, K.C. and Lee, Y.J. (2018). “Long-term deflection prediction from computer vision-measured data history for high-speed railway bridges.” Sensors, 18(5), 1488.
Lim, T.K., Park, S.M., Lee, H.C. and Lee, D.E. (2016). “Artificial neural network-based slip-trip classifier using smart sensor for construction workplace.” Journal of Construction Engineering and Management, 142(2), 04015065.
Liu, Z.W., Suandi, S.A., Ohashi, T. and Ejima, T. (2002). “A tunnel crack detection and classification systems based on image processing.” Machine Vision Applications in Industrial Inspection X, 4664(2002), 145-152.
McMahon, S., Sunderhauf, N., Upcroft, B. and Milford, M. (2018). “Multimodal trip hazard affordance detection on construction sites.” Ieee Robotics and Automation Letters, 3(1), 1-8.
Mendell, W.W. (2005). “Meditations on the new space vision: The moon as a stepping stone to mars.” Acta Astronautica, 57(2), 676-683.
Milan, S., Vaclav, H. and Roger, B. (2015) “Section 1: Introduction.” Image Processing, Analysis, and Machine Vision (Fourth Edition), Cengage Learning, Stamford, 1.1-1.3.
Ministry of Housing and Urban-Rural Development of the People’s Republic of China (MOHURD). (2011). “Standard for construction safety inspection.” <https://wenku.baidu.com/view/a00c33b8960590c69ec37622.html> (Mar. 29, 2012).
Nishiyama, S., Minakata, N., Kikuchi, T. and Yano, T. (2015). “Improved digital photogrammetry technique for crack monitoring.” Advanced Engineering Informatics, 29(4), 851-858.
Park, J., Kim, B., Kim, C. and Kim, H. (2011). “3D/4D CAD applicability for life-cycle facility management.” Journal of Computing in Civil Engineering, 25(2), 129-138.
Park, M.W. and Brilakis, I. (2016). “Continuous localization of construction workers via integration of detection and tracking.” Automation in Construction, 72(2016), 129-142.
Park, M.W., Elsafty, N. and Zhu, Z. (2015). “Hardhat-wearing detection for enhancing on-site safety of construction workers.” Journal of Construction Engineering and Management, 141(9), 04015024.
Park, T., Kim, M.K., Kim, C. and Kim, H. (2009). “Interactive 3D CAD for effective derrick crane operation in a cable-stayed bridge construction.” Journal of Construction Engineering and Management-Asce, 135(11), 1261-1270.
Poh, C.Q.X., Ubeynarayana, C.U. and Goh, Y.M. (2018). “Safety leading indicators for construction sites: A machine learning approach.” Automation in Construction, 93(2018), 375-386.
Quintana, M., Torres, J. and M, M.J. (2016). “A simplified computer vision system for road surface inspection and maintenance.” Ieee Transactions on Intelligent Transportation Systems, 17(2016), 608-619.
Radhika, S., Tamura, Y. and Matsui, M. (2015). “Cyclone damage detection on building structures from pre- and post-satellite images using wavelet based pattern recognition.” Journal of Wind Engineering and Industrial Aerodynamics, 136(2015), 23-33.
Ray, S.J. and Teizer, J. (2012). “Coarse head pose estimation of construction equipment operators to formulate dynamic blind spots.” Advanced Engineering Informatics, 26(1), 117-130.
Reizgevicius, M., Ustinovichius, L., Simanaviciene, R., Rasiulis, R. and Peliksa, M. (2014). “The evaluation and justification of the effectiveness of 4D CAD using multi-criteria analysis.” Journal of Civil Engineering and Management, 20(6), 884-892.
Rosenfeld, A. and Pfaltz, J.L. (1966). “Sequential operations in digital picture processing.” Journal of the ACM, 13(4), 471–494.
Rubaiyat, A.H.M., Toma, T.T., Kalantari-Khandani, M., Rahman, S.A., Chen, L., Ye, Y. and Pan, C.S. (2016). “Automatic detection of helmet uses for construction safety.”IEEE/WIC/ACM International Conference on Web Intelligence Workshops, Omaha, U.S., 135-142.
Samuel, A.L. (2000). “Some studies in machine learning using the game of checkers.” Ibm Journal of Research and Development, 3(3), 210-229.
Seo, J., Han, S., Lee, S. and Kim, H. (2015). “Computer vision techniques for construction safety and health monitoring.” Advanced Engineering Informatics, 29(2), 239-251.
Shapira, A. and Rosenfeld, Y. (2011). “Achieving construction innovation through academia-industry cooperation-keys to success.” Journal of Professional Issues in Engineering Education and Practice, 137(4), 223-231.
Siddula, M., Dai, F., Ye, Y. and Fan, J. (2016a). “Classifying construction site photos for roof detection A machine-learning method towards automated measurement of safety performance on roof sites.” Construction Innovation-England, 16(3), 368-389.
Siddula, M., Dai, F., Ye, Y.F. and Fan, J.P. (2016b). “Unsupervised feature learning for objects of interest detection in cluttered construction roof site images.” Procedia Engineering, 145(2016), 428-435.
Soltani, M.M., Zhu, Z. and Hammad, A. (2017). “Skeleton estimation of excavator by detecting its parts.” Automation in Construction, 82(2017), 1-15.
Sugimoto, N. and Ikeda, H. (1990). “The Construction of a radar-type safety device to complement human vision in safety prediction.” Ergonomics, 33(6), 735-762.
Sunindijo, R.Y. and Zou, P.X.W. (2012). “Political skill for developing construction safety climate.” Journal of Construction Engineering and Management-Asce, 138(5), 605-612.
Tomasi, R., Sottile, F., Pastrone, C., Mozumdar, M.M.R., Osello, A. and Lavagno, L. (2015). “Leveraging BIM interoperability for UWB-based WSN planning.” Ieee Sensors Journal, 15(10), 5988-5996.
Vandatikhaki, F., Langari, S.M., Taher, A., ElAmmari, K. and Hammad, A. (2017). “Enhancing coordination and safety of earthwork equipment operations using Multi-Agent System.” Automation in Construction, 81(2017), 267-285.
Yang, O. (2017). “Improved locating algorithm of railway tunnel personnel based on collaborative information fusion in Internet of Things.” Transactions of the Institute of Measurement and Control, 39(4), 446-454.
Yu, S.N., Jang, J.H. and Han, C.S. (2007). “Auto inspection system using a mobile robot for detecting concrete cracks in a tunnel.” Automation in Construction, 16(3), 255-261.
Zanen, P.P.A., Hartmann, T., Al-Jibouri, S.H.S. and Heijmans, H.W.N. (2013). “Using 4D CAD to visualize the impacts of highway construction on the public.” Automation in Construction, 32(8), 136-144.
Zeibak-Shini, R., Sacks, R., Ma, L. and Filin, S. (2016). “Towards generation of as-damaged BIM models using laser-scanning and as-built BIM: First estimate of as-damaged locations of reinforced concrete frame members in masonry infill structures.” Advanced Engineering Informatics, 30(3), 312-326.
Zhou, Z.P., Goh, Y.M. and Li, Q.M. (2015). “Overview and analysis of safety management studies in the construction industry.” Safety Science, 72(2015), 337-350.
Zhu, H., Li, M., Li, Y. and You, S. (2018). “Petrochemical factory inspection robot environment modeling and map building device, has industrial control computer for realizing accurate positioning of robot and constructing environment model and map based on positioning result.” Univ China Mining & Technology Beijing; Xuzhou Sengon Intelligent Technology Co Ltd.
Zou, Y., Kiviniemi, A. and Jones, S.W. (2016). “Developing a tailored RBS linking to BIM for risk management of bridge projects.” Engineering Construction and Architectural Management, 23(6), 727-750.
Information & Authors
Information
Published In
ICCREM 2019: Innovative Construction Project Management and Construction Industrialization
Pages: 67 - 81
Editors: Yaowu Wang, Ph.D., Harbin Institute of Technology, Mohamed Al-Hussein, Ph.D., University of Alberta, and Geoffrey Q. P. Shen, Ph.D., Hong Kong Polytechnic University
ISBN (Online): 978-0-7844-8230-8
Copyright
© 2019 American Society of Civil Engineers.
History
Published online: Aug 29, 2019
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.