Semantic Rule-Based Construction Procedural Information Extraction to Guide Jobsite Sensing and Monitoring
Publication: Journal of Computing in Civil Engineering
Volume 35, Issue 6
Abstract
Existing construction monitoring systems rely on designated personnel to compare a planned procedure in the instructional documents with an execution procedure on the jobsite. It requires major human efforts and is time-consuming, costly, and human error–prone. To reduce manual efforts in collecting information from construction procedural documents, selecting appropriate sensing techniques to collect data on the jobsite, and giving in-time feedback for progress monitoring and compliance checking, the authors: (1) proposed a semantic rule-based information extraction (IE) method to extract construction execution steps from construction procedural documents automatically; (2) developed a construction procedure and data collection (CPDC) ontology to classify construction site information and provide guidance on selecting sensing techniques for collecting jobsite data based on the extracted information; and (3) proposed a construction procedural data integration (CPDI) framework, which could integrate textual data and sensing data to conduct construction execution steps compliance checking automatically. The proposed IE method offers a novel way to retrieve construction activities (e.g., install door and prepare surface) from the construction procedural documents that are an important source of information but were seldom analyzed in previous research. It focuses on extracting construction execution steps, which supports the incorporation of execution sequence information into construction jobsite management. A novel information classification application based on a newly developed CPDC ontology is provided to support IE. The IE results provide a new instrument for selecting sensing techniques based on different construction site data categories. The proposed method is developed to support a targeted unified CPDI framework that integrates the following: natural language processing (NLP) and sensing techniques to automatically extract, analyze, and process both planned procedures from textual data from construction procedural documents and executed procedures from sensing data of construction jobsites. It also can be extended to any domain applications that contain textual information and site activity information that needs to be matched or compared. In this paper, the authors focus on the NLP-based textual information extraction of the construction procedural documents, introduced the detailed steps involved, and proposed a CPDI framework on top of and as an application of the IE method. An experiment was conducted to evaluate the IE method with a set of open-source specifications. Comparing it to a manually developed gold standard, 97.08% precision and 93.23% recall were achieved using the proposed IE method for the extraction of construction execution steps. A qualitative analysis on sensing technique selection based on the IE results was also performed. The proposed IE method can be applied to the automation of construction site management tasks (e.g., construction monitoring) that integrates both textual information and sensing data to support construction decision-makings. It enables the transformation of construction monitoring/control from a human-intensive process to an automated one.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
References
Abul-Kasim, K., M. Thurnher, S. Puchner, and P. Sundgren. 2013. “Multimodal magnetic resonance imaging increases the overall diagnostic accuracy in brain tumours: Correlation with histopathology.” J. Radiol. 17 (1): 4–10. https://doi.org/10.7196/SAJR.812.
Ahmed, S. 2018. “A review on using opportunities of augmented reality and virtual reality in construction project management.” Organ. Technol. Manage. Constr. Int. J. 10 (1): 1839–1852.
Ajayi, O. G., M. Palmer, and A. A. Salubi. 2018. “Modelling farmland topography for suitable site selection of dam construction using unmanned aerial vehicle (UAV) photogrammetry.” Remote Sens. Appl.: Soc. Environ. 11 (Aug): 220–230. https://doi.org/10.1016/j.rsase.2018.07.007.
Akhavian, R., and A. H. Behzadan. 2016. “Smartphone-based construction workers’ activity recognition and classification.” Autom. Constr. 71: 198–209. https://doi.org/10.1016/j.autcon.2016.08.015.
AMBICO. 2020. “Decorative metal doors and frames.” Accessed November 11, 2020. https://www.arcat.com/sdspecs/htm04/08_11_00abl.htm.
Anwar, N., M. A. Izhar, and F. A. Najam. 2018. “Construction monitoring and reporting using drones and unmanned aerial vehicles (UAVs).” In Proc., 10th Int. Conf. on Construction in the 21st Century (CITC-10). Thailand: Asian Institute of Technology.
ARCAT. 2020. “CSI 3-part formatted specifications for openings.” Accessed November 11, 2020. https://www.arcat.com/content-type/spec/openings-08.
Artstein, R. 2017. “Inter-annotator agreement.” In Handbook of linguistic annotation, 297–313. Dordrecht, Netherlands: Springer. https://doi.org/10.1007/978-94-024-0881-211.
Asadi, K., and K. Han. 2018. “Real-time image-to-BIM registration using perspective alignment for automated construction monitoring.” In Vol. 2018 of Proc., Construction Research Congress, 388–397. Reston, VA: ASCE.
Bangaru, S. S., C. Wang, and F. Aghazadeh. 2020. “Data quality and reliability assessment of wearable EMG and IMU sensor for construction activity recognition.” Sensors 20 (18): 5264. https://doi.org/10.3390/s20185264.
Caldas, C. H., L. Soibelman, and J. Han. 2002. “Automated classification of construction project documents.” J. Comput. Civ. Eng. 16 (4): 234–243. https://doi.org/10.1061/(ASCE)0887-3801(2002)16:4(234).
Cline Doors. 2020. “Section 08 11 10 prefinished steel door framesAutomated classification of construction project documents.” Accessed November 11, 2020. https://www.arcat.com/sdspecs/htm04/08_11_16cli.htm.
Clough, R. H., G. A. Sears, S. K. Sears, R. O. Segner, and J. L. Rounds. 2015. Construction contracting: A practical guide to company management. New York: Wiley.
Coluccia, A., et al. 2019. Drone-vs-bird detection challenge at IEEE AVSS2019. In Proc., 16th IEEE Int. Conf. on Advanced Video and Signal Based Surveillance (AVSS) 2019, 1–7. New York: IEEE.
Deng, H., H. Hong, D. Luo, Y. Deng, and C. Su. 2020. “Automatic indoor construction process monitoring for tiles based on BIM and computer vision.” J. Constr. Eng. Manage. 146 (1): 04019095. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001744.
Dunbarton. 2020. “Section 08 11 10 prefinished steel door frames. Automated classification of construction project documents.” Accessed November 11, 2020. https://www.arcat.com/sdspecs/htm04/08_11_10dbc.htm.
Ergen, E., and B. Akinci. 2007. “An overview of approaches for utilizing RFID in construction industry.” In Proc., 2007 1st Annual RFID Eurasia, 1–5. New York: IEEE.
Esser, D., D. Schuster, K. Muthmann, and A. Schill. 2014. “Few-exemplar information extraction for business documents.” In Proc., 16th Int. Conf. on Enterprise Information Systems, (ICEIS-2014), 293–298. Setubal, Portugal: Science and Technology Publications.
Fagin, R., B. Kimelfeld, F. Reiss, and S. Vansummeren. 2016. “A relational framework for information extraction.” ACM SIGMOD Rec. 44 (4): 5–16. https://doi.org/10.1145/2935694.2935696.
Falotico, R., and P. Quatto. 2015. “Fleiss’ kappa statistic without paradoxes.” Quality Quan. 49 (2): 463–470. https://doi.org/10.1007/s11135-014-0003-1.
Fan, R., L. Wang, J. Yan, W. Song, Y. Zhu, and X. Chen. 2020. “Deep learning-based named entity recognition and knowledge graph construction for geological hazards.” ISPRS Int. J. Geo-Inf. 9 (1): 15. https://doi.org/10.3390/ijgi9010015.
Fang, W., P. E. Love, H. Luo, and L. Ding. 2020. “Computer vision for behaviour-based safety in construction: A review and future directions.” Adv. Eng. Inf. 43 (Jan): 100980. https://doi.org/10.1016/j.aei.2019.100980.
Fautsch, C., and J. Savoy. 2009. “Algorithmic stemmers or morphological analysis? An evaluation.” J. Am. Soc. Inf. Sci. Technol. 60 (8): 1616–1624. https://doi.org/10.1002/asi.21093.
Galaxy Metal Products. 2020. “Section 08 11 10 steel doors and frames.” Accessed November 11, 2020. https://www.arcat.com/sdspecs/htm04/08_11_10gal.htm.
Goutte, C., and E. Gaussier. 2005. “A probabilistic interpretation of precision, recall and F-score, with implication for evaluation.” In Proc., European Conf. on Information Retrieval, 345–359. Berlin: Springer.
Grishman, R., and B. M. Sundheim. 1996. “Message understanding conference-6: A brief history.” In Proc., COLING 1996 Volume 1: The 16th Int. Conf. on Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics.
Harris, C. M. 2006. Dictionary of architecture and construction. New York: McGraw-Hill.
Huang, Y., M. Yuan, Y. Sheng, X. Min, and Y. Cao. 2019. “Using geographic ontologies and geo-characterization to represent geographic scenarios.” ISPRS Int. J. Geo-Inf. 8 (12): 566. https://doi.org/10.3390/ijgi8120566.
Ibrahim, A., and M. Golparvar-Fard. 2019. “4D BIM based optimal flight planning for construction monitoring applications using camera-equipped UAVs.” In Computing in civil engineering 2019: Data, sensing, and analytics, 217–224. Reston, VA: ASCE.
Kim, H., Y. Ham, W. Kim, S. Park, and H. Kim. 2019b. “Vision-based nonintrusive context documentation for earthmoving productivity simulation.” Autom. Constr. 102 (Jun): 135–147. https://doi.org/10.1016/j.autcon.2019.02.006.
Kübler, S., R. McDonald, and J. Nivre. 2009. “Dependency parsing.” Synth. Lect. Hum. Lang. Technol. 2 (1): 1–127. https://doi.org/10.2200/S00169ED1V01Y200901HLT002.
Landis, J. R., and G. G. Koch. 1977. “The measurement of observer agreement for categorical data.” Biometrics 33 (1): 159–174. https://doi.org/10.2307/2529310.
Li, H., Z. Chen, L. Yong, and S. C. Kong. 2005. “Application of integrated GPS and GIS technology for reducing construction waste and improving construction efficiency.” Auto. Constr. 14 (3): 323–331.
Manning, C., and H. Schutze. 1999. Foundations of statistical natural language processing. Cambridge, MA: MIT Press.
Memon, Z. A., M. Z. A. Majid, and M. Mustaffar. 2006. “A systematic approach for monitoring and evaluating the construction project progress.” J. Inst. Eng. 67 (3): 26–32.
Moens, M. F. 2006. Vol. 21 of Information extraction: Algorithms and prospects in a retrieval context. New York: Springer.
Oliphant, T. E. 2007. “Python for scientific computing.” Comput. Sci. Eng. 9 (3): 10–20. https://doi.org/10.1109/MCSE.2007.58.
Omar, T., and M. L. Nehdi. 2016. “Data acquisition technologies for construction progress tracking.” Autom. Constr. 70: 143–155. https://doi.org/10.1016/j.autcon.2016.06.016.
Pathomvanich, S., F. T. Najafi, and P. A. Kopac. 2002. “Procedure for monitoring and improving effectiveness of quality assurance specifications.” Transp. Res. Rec. 1813 (1): 164–171. https://doi.org/10.3141/1813-20.
Pazhoohesh, M., and C. Zhang. 2015. “Automated construction progress monitoring using thermal images and wireless sensor networks.” Accessed November 18, 2020. https://www.researchgate.net/profile/Mehdi_Pazhoohesh/publication/319900597_Construction_progress_monitoring_based_on_thermal-image_analysis/links/5dc51f9ea6fdcc2d2ffc166e/Construction-progress-monitoring-based-on-thermal-image-analysis.pdf.
PGT Industries. 2020. “Section 08 15 00 vinyl doors.” Accessed November 11, 2020. https://www.arcat.com/sdspecs/htm04/08_15_00pgt.htm.
Piñeres-Espitia, G., A. Cama-Pinto, D. De La Rosa, F. Estevez, and D. Cama-Pinto. 2017. “Design of a low cost weather station for detecting environmental changes.” Rev. Espacios 38 (59): 13.
Poku, S. E., and D. Arditi. 2006. “Construction scheduling and progress control using geographical information systems.” J. Comput. Civ. Eng. 20 (5): 351–360. https://doi.org/10.1061/(ASCE)0887-3801(2006)20:5(351).
Ren, Z., C. J. Anumba, and J. H. M. T. Tah. 2011. “RFID-facilitated construction materials management (RFID-CMM)—A case study of water-supply project.” Adv. Eng. Inf. 25 (2): 198–207. https://doi.org/10.1016/j.aei.2010.02.002.
Roh, S., Z. Aziz, and F. Peña-Mora. 2011. “An object-based 3D walk-through model for interior construction progress monitoring.” Autom. Constr. 20 (1): 66–75. https://doi.org/10.1016/j.autcon.2010.07.003.
Serrano-Juan, A., E. Pujades, E. Vázquez-Suñè, M. Crosetto, and M. Cuevas-González. 2017. “Leveling vs. InSAR in urban underground construction monitoring: Pros and cons. Case of La Sagrera railway station (Barcelona, Spain).” Eng. Geol. 218: 1–11. https://doi.org/10.1016/j.enggeo.2016.12.016.
Shaw, J. 2006. Introduction to optical and infrared sensor systems. Orlando, FL: SPIE.
Sherafat, B., C. R. Ahn, R. Akhavian, A. H. Behzadan, M. Golparvar-Fard, H. Kim, Y. C. Lee, A. Rashidi, and E. R. Azar. 2020. “Automated methods for activity recognition of construction workers and equipment: State-of-the-art review.” J. Constr. Eng. Manage. 146 (6): 03120002. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001843.
Shi, L., Y. Zhang, J. Cheng, and H. Lu. 2019. “Two-stream adaptive graph convolutional networks for skeleton-based action recognition.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 2026–12035. New York: IEEE.
Singh, S. 2018. “Natural language processing for information extraction.” Preprint, submitted July 6, 2018. http://arxiv.org/abs/1807.02383.
spaCy. 2015. “Industrial-strength natural language processing in Python.” Accessed June 20, 2020. https://spacy.io/.
Sukkarieh, J. Z., and S. G. Pulman. 2005. “Information extraction and machine learning: Auto-marking short free text responses to science questions.” In Proc., 2005 Conf. on Artificial Intelligence in Education: Supporting Learning through Intelligent and Socially Informed Technology, 629–637. Amsterdam, Netherlands: IOS Press.
Sweets. 2020. “Master format products: All divisions.” Accessed November 18, 2020. https://sweets.construction.com/BrowseByDivision.
Teizer, J. 2008. “3D range imaging camera sensing for active safety in construction.” J. Inf. Technol. Constr. (ITcon) 13 (8): 103–117.
Teizer, J. and T. Kahlmann. 2007. “Range imaging as emerging optical three-dimension measurement technology.” ITcon 2040 (1). https://doi.org/10.3141/2040-03.
Tian, Y., X. Yang, and A. Arditi. 2010. “Computer vision-based door detection for accessibility of unfamiliar environments to blind persons.” In Proc., Int. Conf. on Computers for Handicapped Persons, 263–270. Berlin: Springer.
Wang, Y., et al. 2018. “Clinical information extraction applications: A literature review.” J. Biomed. Inf. 77: 34–49. https://doi.org/10.1016/j.jbi.2017.11.011.
Wiley, V., and T. Lucas. 2018. “Computer vision and image processing: A paper review.” Int. J. Artif. Intell. Res. 2 (1): 22–36. https://doi.org/10.29099/ijair.v2i1.42.
Yan, X., H. Li, A. R. Li, and H. Zhang. 2017. “Wearable IMU-based real-time motion warning system for construction workers’ musculoskeletal disorders prevention.” Autom. Constr. 74: 2–11. https://doi.org/10.1016/j.autcon.2016.11.007.
Yang, C. H., Y. Pang, and U. Soergel. 2017. “Monitoring of building construction by 4D change detection using multi-temporal SAR images.” ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci. 4: 35. https://doi.org/10.5194/isprs-annals-IV-1-W1-35-2017.
Yang, L. N., Z. F. Zhou, and L. J. Zhu. 2009. “Ceramic substrate’s detection system based on machine vision.” In Vol. 7283 of 4th Int. Symp. on Advanced Optical Manufacturing and Testing Technologies: Optical Test and Measurement Technology and Equipment, 72834I. Bellingham, WA: International Society for Optics and Photonics.
Yu, X., et al. 2020. “The 1st tiny object detection challenge: Methods and results.” Preprint, submitted September 16, 2020. http://arxiv.org/abs/2009.07506.
Zhang, J., and N. El-Gohary. 2017. “Integrating semantic NLP and logic reasoning into a unified system for fully-automated code checking.” Auto. Constr. 73 (1): 45–57.
Zhang, J., and N. M. El-Gohary. 2016. “Semantic NLP-based information extraction from construction regulatory documents for automated compliance checking.” J. Comput. Civ. Eng. 30 (2): 04015014. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000346.
Zhong, B., X. Xing, H. Luo, Q. Zhou, H. Li, T. Rose, and W. Fang. 2020. “Deep learning-based extraction of construction procedural constraints from construction regulations.” Adv. Eng. Inf. 43: 101003. https://doi.org/10.1016/j.aei.2019.101003.
Zhou, P., and N. El-Gohary. 2017. “Ontology-based automated information extraction from building energy conservation codes.” Autom. Constr. 74 (Feb): 103–117. https://doi.org/10.1016/j.autcon.2016.09.004.
Ziegler, S., R. C. Woodward, H. H. C. Iu, and L. J. Borle. 2009. “Current sensing techniques: A review.” IEEE Sens. J. 9 (4): 354–376. https://doi.org/10.1109/JSEN.2009.2013914.
Information & Authors
Information
Published In
Journal of Computing in Civil Engineering
Volume 35 • Issue 6 • November 2021
Copyright
© 2021 American Society of Civil Engineers.
History
Received: Sep 28, 2020
Accepted: Feb 2, 2021
Published online: Sep 6, 2021
Published in print: Nov 1, 2021
Discussion open until: Feb 6, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Farheen Hussain, Siddhant Mehta, Meta Soy, Jiansong Zhang, Natural Language Processing for Construction Management: A Literature Review, Construction Research Congress 2024, 10.1061/9780784485262.062, (607-618), (2024).