Object Detection-Based Knowledge Graph Creation: Enabling Insight into Construction Processes
Publication: Computing in Civil Engineering 2023
ABSTRACT
Compared to other industries, the construction sector shows low productivity worldwide. However, holistic, data-oriented methods for investigating potential bottlenecks within the as-performed construction stage are scarce. Our research presents an approach to acquiring raw data from job sites and its subsequent processing to high-level information. First, images were captured over a period of one year in high frequency using multiple crane cameras. Second, an end-to-end deep learning based approach was developed to derive and link information about construction activities, covering the classification and localization of specific on-site objects. This information was subsequently integrated into a knowledge graph. Finally, additional data sources like the weather were exploited to interpret different on-site scenarios. We demonstrate that construction-related activities like working times can be detected. The presented approach provides a significant step toward exposing correlations on construction sites by using multiple data processing steps and showcases the possibility of identifying process patterns.
Get full access to this article
View all available purchase options and get full access to this chapter.
REFERENCES
Akyon, F. C., Altinuc, S. O., and Temizel, A. (2022). Slicing Aided Hyper Inference and Fine-tuning for Small Object Detection. https://doi.org/10.48550/arxiv.2202.06934.
Barbosa, F., Woetzel, J., and Mischke, J. (2017). Reinventing construction: A route of higher productivity. McKinsey Global Institute.
Braun, A., and Borrmann, A. (2019). Combining inverse photogrammetry and BIM for automated labeling of construction site images for machine learning. Automation in Construction, 106, 102879. https://doi.org/10.1016/J.AUTCON.2019.102879.
Collins, F., Pfitzner, F., and Schlenger, J. (2022). Scalable construction monitoring for an as-performed progress documentation across time. Proceedings of 33. Forum Bauinformatik. https://mediatum.ub.tum.de/doc/1688376/document.pdf.
Farmer, M. (2016). The Farmer Review of the UK Construction Labour Model Modernise or Die Time to decide the industry’s future. www.cast-consultancy.com.
Francis, N., Green, A., Guagliardo, P., Libkin, L., Lindaaker, T., Marsault, V., Plantikow, S., Rydberg, M., Selmer, P., and Taylor, A. (2018). Cypher: An evolving query language for property graphs. Proceedings of the ACM SIGMOD International Conference on Management of Data, 13, 1433–1445. https://doi.org/10.1145/3183713.3190657.
Guo, B. H. W., Zou, Y., Fang, Y., Goh, Y. M., and Zou, P. X. W. (2021). Computer vision technologies for safety science and management in construction: A critical review and future research directions. Safety Science, 135. https://doi.org/10.1016/j.ssci.2020.105130.
Hodler, A. E., Farnham, B., Tokyo, S., Boston, B., Sebastopol, F., and Beijing, T. (2019). Mark Needham Graph Algorithms Practical Examples in Apache Spark and Neo4j.
Li, H., Chan, G., Wong, J. K. W., and Skitmore, M. (2016). Real-time locating systems applications in construction. In Automation in Construction (Vol. 63, pp. 37–47). Elsevier. https://doi.org/10.1016/j.autcon.2015.12.001.
Mostafa, K., and Hegazy, T. (2021). Review of image-based analysis and applications in construction. In Automation in Construction (Vol. 122). Elsevier B.V. https://doi.org/10.1016/j.autcon.2020.103516.
Opoku, D. G. J., Perera, S., Osei-Kyei, R., and Rashidi, M. (2021). Digital twin application in the construction industry: A literature review. In Journal of Building Engineering (Vol. 40). Elsevier Ltd. https://doi.org/10.1016/j.jobe.2021.102726.
Pauwels, P., Zhang, S., and Lee, Y. C. (2017). Semantic web technologies in AEC industry: A literature overview. Automation in Construction, 73, 145–165. https://doi.org/10.1016/J.AUTCON.2016.10.003.
Pfitzner, F., Braun, A., and Borrmann, A. (2022). Towards data mining on construction sites: Heterogeneous data acquisition and fusion. ECPPM: European Conferences on Product and Process Modeling in the Building Industry.
Sacks, R., Brilakis, I., Pikas, E., Xie, H. S., and Girolami, M. (2020). Construction with digital twin information systems. Data-Centric Engineering, 1, e14. https://doi.org/10.1017/dce.2020.16.
Torabi, G., Hammad, A., and Bouguila, N. (2022). Two-Dimensional and Three-Dimensional CNN-Based Simultaneous Detection and Activity Classification of Construction Workers. Journal of Computing in Civil Engineering, 36(4). https://doi.org/10.1061/(ASCE)CP.1943-5487.0001024.
Xu, S., Wang, J., Shou, W., Ngo, T., Sadick, A.-M., and Wang, X. (2021). Computer Vision Techniques in Construction: A Critical Review. Archives of Computational Methods in Engineering, 28, 3383–3397. https://doi.org/10.1007/s11831-020-09504-3.
Xuehui, A., Li, Z., Zuguang, L., Chengzhi, W., Pengfei, L., and Zhiwei, L. (2021). Dataset and benchmark for detecting moving objects in construction sites. Automation in Construction, 122. https://doi.org/10.1016/j.autcon.2020.103482.
Zheng, Y., Törmä, S., and Seppänen, O. (2021). A shared ontology suite for digital construction workflow. Automation in Construction, 132. https://doi.org/10.1016/J.AUTCON.2021.103930.
Zou, Z., Shi, Z., Guo, Y., Ye, J., and Member, S. (2019). Object Detection in 20 Years: A Survey. Proceedings of the IEEE.
Information & Authors
Information
Published In
Computing in Civil Engineering 2023
Pages: 186 - 193
History
Published online: Jan 25, 2024
ASCE Technical Topics:
- Artificial intelligence and machine learning
- Business management
- Cameras
- Computer programming
- Computing in civil engineering
- Construction engineering
- Construction equipment
- Construction industry
- Construction management
- Construction methods
- Construction sites
- Cranes
- Engineering fundamentals
- Equipment and machinery
- Neural networks
- Personnel management
- Practice and Profession
- Productivity
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.