Ontology-Based Semantic Modeling of Knowledge in Construction: Classification and Identification of Hazards Implied in Images
Publication: Journal of Construction Engineering and Management
Volume 146, Issue 4
Abstract
Identifying potential hazards of construction project is a data-intensive process that involves various types of information such as site data, specifications, and engineering documents. How to effectively convert the information into a machine processable format for safety management is a challenging task. To address this problem, in this paper, combining the HowNet and specific taxonomies from the relevant construction specifications, a semantic modeling approach is developed for the proactive construction hazard identification from images. A semantic scoring system is then introduced for quantifying the similarities between images, via comparing their annotations with the construction hazard specification. Furthermore, an image processing framework is developed to semantically annotate site images and further automatically classify the images into the categories. In this way, the potential hazards implied in the images can be identified automatically. Examples are developed to demonstrate the feasibility of the approach. The outcomes of this study have offered an alternative method to enhance site safety management on site.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Data generated or analyzed during the study are available from the corresponding author by request.
Acknowledgments
This research is partly supported by “National Natural Science Foundation of China” (Nos. 51878311, 71732001, 71821001, 51978302, and 51678265) and China Scholarship Council (CSC).
References
Albitar, S., S. Fournier, and B. Espinasse. 2014. “An effective TF/IDF-based text-to-text semantic similarity measure for text classification.” In Web information systems engineering—WISE 2014, edited by B. Benatallah, A. Bestavros, Y. Manolopoulos, A. Vakali, and Y. Zhang. New York: Springer.
Bottou, L. 2014. “From machine learning to machine reasoning: An essay.” Mach Learn 94 (2): 133–149.
Brilakis, I., M. W. Park, and G. Jog. 2011. “Automated vision tracking of project related entities.” Adv. Eng. Inf. 25 (4): 713–724. https://doi.org/10.1016/j.aei.2011.01.003.
Burdescu, D. D., C. G. Mihai, L. Stanescu, and M. Brezovan. 2013. “Automatic image annotation and semantic based image retrieval for medical domain.” Neurocomputing 109 (8): 33–48. https://doi.org/10.1016/j.neucom.2012.07.030.
Chen, H., A. Trouve, K. J. Murakami, and A. Fukuda. 2016. “An intelligent annotation-based image retrieval system based on RDF descriptions.” Comput. Electr. Eng. 58 (Feb): 537–550. https://doi.org/10.1016/j.compeleceng.2016.09.031.
Chen, J. 2006. “Concept similarity computation for domain ontology.” Comput. Eng. Appl. 33 (42): 163–166.
China Building Industry Press. 2016. Quality and safety inspection guide of Urban Rail Transit Engineering. Beijing: China Building Industry Press.
Ding, L., W. Fang, H. Luo, P. E. D. Love, B. Zhong, and X. Ouyang. 2018. “A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory.” Autom. Constr. 86 (Feb): 118–124. https://doi.org/10.1016/j.autcon.2017.11.002.
Ding, L., B. T. Zhong, S. Wu, and H. B. Luo. 2016. “Construction risk knowledge management in BIM using ontology and semantic web technology.” Saf. Sci. 87 (Aug): 202–213. https://doi.org/10.1016/j.ssci.2016.04.008.
Dong, Z., and Q. Dong. 2003. “HowNet: A hybrid language and knowledge resource.” In Proc., Int. Conf. on Natural Language Processing and Knowledge Engineering Proc., 820–824. New York: IEEE.
Dong, Z., and Q. Dong. 2006. HowNet and the computation of meaning. Singapore: World Scientific Publishing.
Dun, L. I., M. A. Yong-Tao, and J. L. Guo. 2009. “Words semantic orientation classification based on HowNet.” J. China Univ. Posts Telecommun. 16 (1): 106–110. https://doi.org/10.1016/S1005-8885(08)60188-0.
Fang, W., L. Ding, H. Luo, and P. E. D. Love. 2018a. “Falls from heights: A computer vision-based approach for safety harness detection.” Autom. Constr. 91 (Jul): 53–61. https://doi.org/10.1016/j.autcon.2018.02.018.
Fang, W., L. Ding, B. Zhong, P. E. D. Love, and H. Luo. 2018b. “Automated detection of workers and heavy equipment on construction sites: A convolutional neural network approach.” Adv. Eng. Inf. 37 (Aug): 139–149. https://doi.org/10.1016/j.aei.2018.05.003.
Fang, W., P. E. D. Love, H. Luo, and L. Ding. 2020. “Computer vision for behaviour-based safety in construction: A review and future directions.” Adv. Eng. Inf. 43 (Jan): 100980. https://doi.org/10.1016/j.aei.2019.100980.
Fang, W., B. Zhong, N. Zhao, P. E. D. Love, H. Luo, J. Xue, and S. Xu. 2019. “A deep learning-based approach for mitigating falls from height with computer vision: Convolutional neural network.” Adv. Eng. Inf. 39 (Jan): 170–177. https://doi.org/10.1016/j.aei.2018.12.005.
Fu, X., W. Liu, Y. Xu, and L. Cui. 2017. “Combine HowNet lexicon to train phrase recursive autoencoder for sentence-level sentiment analysis.” Neurocomputing 241 (Jun): 18–27. https://doi.org/10.1016/j.neucom.2017.01.079.
Goh, Y. M., and D. K. H. Chua. 2010. “Case-based reasoning approach to construction safety hazard identification: Adaptation and utilization.” J. Constr. Eng. Manage. 136 (2): 170–178. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000116.
Gonçalves, F. M. F., and I. R. Guilherme. 2017. “Semantic guided interactive image retrieval for plant identification.” Expert Syst. Appl. 91 (Jan): 138–153.
Guo, B. H. W., and Y. M. Goh. 2017. “Ontology for design of active fall protection systems.” Autom. Constr. 82 (SC): 138–153. https://doi.org/10.1016/j.autcon.2017.02.009.
Guo, S., L. Ding, H. Luo, and X. Jiang. 2016. “A big-data-based platform of workers’ behavior: Observations from the field.” Accid. Anal. Prev. 93 (Aug): 299–309. https://doi.org/10.1016/j.aap.2015.09.024.
Jiang, S., J. Zhang, and H. Zhang. 2013. “Ontology-based semantic retrieval for risk management of construction project.” J. Netw. 8 (5): 1212–1220.
Khoo, C. S. G., and J. C. Na. 2006. “Semantic relations in information science.” Ann. Rev. Inf. Sci. Technol. 1 (40): 157–228.
Kim, H., K. Kim, and H. Kim. 2016. “Vision-based object-centric safety assessment using fuzzy inference: Monitoring struck-by accidents with moving objects.” J. Comput. Civ. Eng. 30 (4): 04015075. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000562.
Kim, J., S. Chi, and J. Seo. 2018. “Interaction analysis for vision-based activity identification of earthmoving excavators and dump trucks.” Autom. Constr. 87 (Mar): 297–308. https://doi.org/10.1016/j.autcon.2017.12.016.
Konstantinou, E., J. Lasenby, and I. Brilakis. 2019. “Adaptive computer vision-based 2D tracking of workers in complex environments.” Autom. Constr. 103 (Jul): 168–184. https://doi.org/10.1016/j.autcon.2019.01.018.
Kumar, A., S. Dyer, J. Kim, C. Li, P. H. W. Leong, M. Fulham, and D. Feng. 2016. “Adapting content-based image retrieval techniques for the semantic annotation of medical images.” Computerized Med. Imaging Graphics 49 (Apr): 37. https://doi.org/10.1016/j.compmedimag.2016.01.001.
Kurtz, C., A. Depeursinge, S. Napel, C. F. Beaulieu, and D. L. Rubin. 2014. “On combining image-based and ontological semantic dissimilarities for medical image retrieval applications.” Med. Image Anal. 18 (7): 1082–1100. https://doi.org/10.1016/j.media.2014.06.009.
Li, W., and C. Yang. 2016. “Combining channel theory, HowNet and extension model to analyze big data.” Procedia Comput. Sci. 91: 452–459. https://doi.org/10.1016/j.procs.2016.07.118.
Lin, K. Y., and S. H. Hsieh. 2012. “Extracting construction safety domain ontology from job hazard analysis databases and construction safety standards.” In Proc., 14th Int. Conf. on Computing in Civil and Building Engineering, edited by V. Telichenko, A. Volkov, and I. Bilchuk. Reston, VA: ASCE.
Lu, Y., Q. Li, Z. Zhou, and Y. Deng. 2015. “Ontology-based knowledge modeling for automated construction safety checking.” Saf. Sci. 79 (Nov): 11–18. https://doi.org/10.1016/j.ssci.2015.05.008.
Luo, H., C. Xiong, W. Fang, P. E. D. Love, B. Zhang, and X. Ouyang. 2018. “Convolutional neural networks: Computer vision-based workforce activity assessment in construction.” Autom. Constr. 94 (Oct): 281–289. https://doi.org/10.1016/j.autcon.2018.06.007.
Magesh, N., and P. Thangaraj. 2011. “Semantic image retrieval based on ontology and SPARQL query.” In Int. Conf. on Advanced Computer Technology. New York: International Journal of Computer Applications.
Manzoor, U., A. Mohammed, B. Zafar, H. Umar, and M. Shoaib. 2015. “Semantic image retrieval: An ontology based approach.” Int. J. Adv. Res. Artif. Int. 4 (4). https://doi.org/10.14569/ijarai.2015.040401.
Mehmood, Z., T. Mahmood, and M. A. Javid. 2018. “Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine.” Appl. Intell. 48 (1): 166–181. https://doi.org/10.1007/s10489-017-0957-5.
Mei, J., M. Zhu, Y. Gao, and H. Yin. 1983. Tongyici cilin (Chinese Synonym Forest), edited by S. L. P. House. Shanghai, China: Shanghai Press of Lexicon and Books.
Russakovsky, O. et al. 2015. “ImageNet large scale visual recognition challenge.” Int. J. Comput. Vis. 115 (3): 211–252.
Sharma, M. K., and T. J. Siddiqui. 2016. “An ontology based framework for retrieval of museum artifacts.” Procedia Comput. Sci. 84: 169–176. https://doi.org/10.1016/j.procs.2016.04.083.
Tserng, H. P., S. Y. L. Yin, R. J. Dzeng, B. Wou, M. D. Tsai, and W. Y. Chen. 2009. “A study of ontology-based risk management framework of construction projects through project life cycle.” Autom. Constr. 18 (7): 994–1008. https://doi.org/10.1016/j.autcon.2009.05.005.
Wang, H., and F. Boukamp. 2011. “Ontology-based representation and reasoning framework for supporting job hazard analysis.” J. Comput. Civ. Eng. 25 (6): 442–456. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000125.
Yu, Y., Y. Guo, Q. Ding, H. Li, and M. Skimore. 2017. “An experimental study of real-time identification of construction workers’ unsafe behaviors.” Autom. Constr. 82 (Oct): 193–206. https://doi.org/10.1016/j.autcon.2017.05.002.
Zhong, B., and Y. Li. 2015. “An ontological and semantic approach for the construction risk inferring and application.” J Intell. Rob. Syst. 79 (3–4): 449–463. https://doi.org/10.1007/s10846-014-0107-9.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 146 • Issue 4 • April 2020
Copyright
©2020 American Society of Civil Engineers.
History
Received: Aug 9, 2018
Accepted: Jul 15, 2019
Published online: Jan 20, 2020
Published in print: Apr 1, 2020
Discussion open until: Jun 20, 2020
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.