BIMASR: Framework for Voice-Based BIM Information Retrieval
Publication: Journal of Construction Engineering and Management
Volume 147, Issue 10
Abstract
Voice is the most convenient means for human beings to communicate with others, even if the objects of their communication are not other humans but machines or computers. Many industries, and even the architecture, engineering, construction, and operations (AECO) industry, have attempted to study and apply speech recognition systems in their operations to improve work efficiency and productivity. However, previous studies on speech recognition had two limitations: they used keywords requiring basic knowledge of building information modeling (BIM) commands for using them and in searching BIM data, they relied on the Industry Foundation Classes (IFC) format, which involves converting BIM data to IFC. Such methods did not conduce to direct retrieval in BIM software. In the latter case, data search was possible, but data manipulation was not. To improve on the limitations of previous studies, this study developed a building information modeling automatic speech recognition (BIMASR) framework that requires no knowledge of BIM commands, which allows for the input of natural language (NL)-based questions into BIM software using human voice to search and manipulate data. The framework consists of three modules: one for voice recognition, one for natural language processing (syntax and semantic analysis), and one for BIM data preprocessing and interworking with relational databases. The manipulation of BIM data with NL-based speech recognition converts the BIM operating environment from an expert-oriented into a user-oriented environment. This conversion allows for more BIM interaction and the popularization of BIM use and enhances the use of BIM in dynamic environments such as virtual reality, augmented reality, and holograms, where conventional input devices are typically absent.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
References
Allard, S., K. J. Levine, and C. Tenopir. 2009. “Design engineers and technical professionals at work: Observing information usage in the workplace.” J. Am. Soc. Inf. Sci. Technol. 60 (3): 443–454. https://doi.org/10.1002/asi.21004.
Al Qady, M., and A. Kandil. 2010. “Concept relation extraction from construction documents using natural language processing.” J. Constr. Eng. Manage. 136 (3): 294–302. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000131.
Asl, M. R., A. Stoupine, S. Zarrinmehr, and W. Yan. 2015. Optimo: A BIM-based multi-objective optimization tool utilizing visual programming for high performance building design. Vienna, Austria: Vienna Univ. of Technology.
Bansal, M., and J. Arora. 2016. “A review on ontology-based information retrieval system.” Int. J. Eng. Dev. Res. 4 (2): 263–265.
El-Gohary, N. M., and T. E. El-Diraby. 2010. “Domain ontology for processes in infrastructure and construction.” J. Constr. Eng. Manage. 136 (7): 730–744. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000178.
Gao, G., Y. S. Liu, M. Wang, M. Gu, and J. H. Yong. 2015. “A query expansion method for retrieving online BIM resources based on industry foundation classes.” Autom. Constr. 56 (11): 14–25. https://doi.org/10.1016/j.autcon.2015.04.006.
Gruber, T. R. 1993. “A translation approach to portable ontology specifications.” Knowl. Acquis. 5 (2): 199–220. https://doi.org/10.1006/knac.1993.1008.
Jennifer, A. 2012. “Expected BIM trends in 2013.” Accessed May 6, 2013. http://www10.aeccafe.com/blogs/aecsanjay/2012/12/20/expected-bim-trends-2013/.
Jiang, S., and J. Zhang. 2013. “Development of an ontology-based semantic retrieval method for construction project risk management.” In Proc., ICCREM 2013: Construction and Operation in the Context of Sustainability, 750–760. Reston, VA: ASCE.
Jiang, S. H., J. Zhang, and Q. Li. 2012. “Study on contract domain ontology supporting BIM interactivity.” In Proc., 2012 Int. Conf. on Construction and Real Estate Management (ICCREM 2012), 420–423. Reston, VA: ASCE.
Kassim, J. M., and M. Rahmany. 2009. “Introduction to semantic search engine.” In Proc., 2009 Int. Conf. on Electrical Engineering and Informatics, 380–386. New York: IEEE.
Khademi, H., and A. Behan. 2017. “A review of approaches to solving the problem of BIM search: Towards intelligence-assisted design.” In Proc., CitA BIM Gathering. Dublin, Republic of Ireland: TU Dublin.
Kim, H. 2018. “A research on utilization of speech recognition artificial intelligence for efficient BIM works.” Des. Convergence Study 17 (1): 1–16. https://doi.org/10.31678/SDC.68.1.
Knublauch, H., R. W. Fergerson, N. F. Noy, and M. A. Musen. 2004. “The Protégé OWL plugin: An open development environment for semantic web applications.” In Proc., Int. Semantic Web Conf., 229–243. Berlin: Springer.
Lange, H., A. Johansen, and M. B. Kjærgaard. 2018. “Evaluation of the opportunities and limitations of using IFC models as source of building metadata.” In Proc., 5th Conf. on Systems for Built Environments, 21–24. New York: Association for Computing Machinery.
Latiffi, A. A., J. Brahim, and M. S. Fathi. 2014. “The development of building information modeling (BIM) definition.” Appl. Mech. Mater. 567 (June): 625–630. https://doi.org/10.4028/www.scientific.net/AMM.567.625.
Le, T., and H. D. Jeong. 2016. “Technical term similarity model for natural language based data retrieval in civil infrastructure projects.” In Proc., Int. Symp. on Automation and Robotics in Construction (IAARC). New York: International Association for Automation and Robotics in Construction.
Li, S., H. Cai, and V. R. Kamat. 2016. “Integrating natural language processing and spatial reasoning for utility compliance checking.” J. Constr. Eng. Manage. 142 (12): 04016074. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001199.
Li, Z., V. Raskin, and K. Ramani. 2008. “Developing engineering ontology for information retrieval.” J. Comput. Inf. Sci. Eng. 8 (1): 13. https://doi.org/10.1115/1.2830851.
Lin, H. T., S. H. Hsieh, K. W. Chou, and K. Y. Lin. 2009. “Construction of engineering domain ontology through extraction of knowledge from domain handbooks.” In Proc., Int. Workshop on Computing in Civil Engineering, 207–216. Reston, VA: ASCE.
Lin, J. R., Z. Z. Hu, J. P. Zhang, and F. Q. Yu. 2016. “A natural-language-based approach to intelligent data retrieval and representation for cloud BIM.” Comput.-Aided Civ. Infrastruct. Eng. 31 (1): 18–33.
Liu, H., Y. S. Liu, P. Pauwels, H. Guo, and M. Gu. 2017. “Enhanced explicit semantic analysis for product model retrieval in construction industry.” IEEE Trans. Ind. Inf. 13 (6): 3361–3369. https://doi.org/10.1109/TII.2017.2708727.
Liu, H., M. Lu, and M. Al-Hussein. 2016. “Ontology-based semantic approach for construction-oriented quantity take-off from BIM models in the light-frame building industry.” Adv. Eng. Inf. 30 (2): 190–207. https://doi.org/10.1016/j.aei.2016.03.001.
Liu, K., W. R. Hogan, and R. S. Crowley. 2011. “Natural language processing methods and systems for biomedical ontology learning.” J. Biomed. Inf. 44 (1): 163–179. https://doi.org/10.1016/j.jbi.2010.07.006.
Lowe, A., C. McMahon, and S. Culley. 2004. “Characterising the requirements of engineering information systems.” Int. J. Inf. Manage. 24 (5): 401–422. https://doi.org/10.1016/j.ijinfomgt.2004.06.008.
Malve, A., and P. P. M. Chawan. 2015. “A comparative study of keyword and semantic based search engine.” Int. J. Innovative Res. Sci. Eng. Technol. 4 (11): 11156–11161.
Motawa, I. 2017. “Spoken dialogue BIM systems—An application of big data in construction.” Facilities 35 (13–14): 787–800. https://doi.org/10.1016/j.websem.2008.04.001.
Nepal, M. P., S. Staub-French, R. Pottinger, and J. Zhang. 2013. “Ontology-based feature modeling for construction information extraction from a building information model.” J. Comput. Civ. Eng. 27 (5): 555–569. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000230.
Niu, J., and R. R. A. Issa. 2012. “Framework for production of ontology-based construction claim documents.” In Proc., Int. Conf. on Computing in Civil Engineering, 9–16. Reston, VA: ASCE.
Paredes-Valverde, M. A., R. Valencia-García, M. Á. Rodríguez-García, R. Colomo-Palacios, and G. Alor-Hernández. 2016. “A semantic-based approach for querying linked data using natural language.” J. Inf. Sci. 42 (6): 851–862. https://doi.org/10.1177/0165551515616311.
Park, M., K. W. Lee, H. S. Lee, P. Jiayi, and J. Yu. 2013. “Ontology-based construction knowledge retrieval system.” KSCE J. Civ. Eng. 17 (7): 1654–1663. https://doi.org/10.1007/s12205-013-1155-6.
Rezgui, Y. 2006. “Ontology-centered knowledge management using information retrieval techniques.” J. Comput. Civ. Eng. 20 (4): 261–270. https://doi.org/10.1061/(ASCE)0887-3801(2006)20:4(261).
Roy Chowdhury, I., and P. R. Chowdhury. 2002. “International joint ventures: A welfare analysis.” J. Policy Reform 5 (1): 51–60. https://doi.org/10.1080/13841280212384.
Shalabi, F., and Y. Turkan. 2017. “IFC BIM-based facility management approach to optimize data collection for corrective maintenance.” J. Perform. Constr. Facilities 31 (1): 04016081.
Shengchun, D., L. Yuemeng, and G. Liren. 2007. “Research on integrated domain ontology construction method based on top-level ontology.” Inf. Stud. Theory Appl. 2 (Apr): 236–240.
Shi, X., Y. S. Liu, G. Gao, M. Gu, and H. Li. 2018. “IFCdiff: A content-based automatic comparison approach for IFC files.” Autom. Constr. 86 (Sept): 53–68. https://doi.org/10.1016/j.autcon.2017.10.013.
Shravankumar, C., and R. Tiwari. 2019. “Experimental identification of cracked rotor system parameters from the forward and backward whirl responses.” Archive Mech. Eng. 66 (3): 329.
Studer, R., V. R. Benjamins, and D. Fensel. 1998. “Knowledge engineering: Principles and methods.” Data Knowledge Eng. 25 (1–2): 161–197. https://doi.org/10.1016/S0169-023X(97)00056-6.
Sun, S., C. Luo, and J. Chen. 2017. “A review of natural language processing techniques for opinion mining systems.” Inf. Fusion 36 (2): 10–25. https://doi.org/10.1016/j.inffus.2016.10.004.
Uggla, G., and M. Horemuz. 2018. “Geographic capabilities and limitations of industry foundation classes.” Autom. Constr. 96 (Oct): 554–566. https://doi.org/10.1016/j.autcon.2018.10.014.
Uschold, M., and M. Gruninger. 1996. Ontologies: Principles, methods and applications. Cambridge, UK: Cambridge University Press.
Vimonsatit, V., and A. Chai Mui Foo. 2015. “Benefits of BIM in construction projects.” In Proc., 8th Int. Structural Engineering and Construction Conf., 1133–1138. Bangkok, Thailand: ISEC Press.
Wu, S., Q. Shen, Y. Deng, and J. Cheng. 2019. “Natural-language-based intelligent retrieval engine for BIM object database.” Comput. Ind. 108 (Sep): 73–88. https://doi.org/10.1016/j.compind.2019.02.016.
Zhang, J., and N. El-Gohary. 2012. “Extraction of construction regulatory requirements from textual documents using natural language processing techniques.” Comput. Civ. Eng. 2012 (1): 453–460. https://doi.org/10.1061/9780784412343.0057.
Zhang, L., and R. R. A. Issa. 2013. “Ontology-based partial building information model extraction.” J. Comput. Civ. Eng. 27 (6): 576–584. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000277.
Zhang, L., G. Wang, and H. Liu. 2014. “The development trend and government policies of open BIM in China.” In Proc., 17th Int. Symp. on Advancement of Construction Management and Real Estate, 981–993. Berlin: Springer.
Zhang, S., F. Boukamp, and J. Teizer. 2015. “Ontology-based semantic modeling of construction safety knowledge: Towards automated safety planning for job hazard analysis (JHA).” Autom. Constr. 52 (Apr): 29–41. https://doi.org/10.1016/j.autcon.2015.02.005.
Zhang, X., R. Di, and Y. Liang. 2010. “Ontology based knowledge modeling for structural engineering experiment information management.” In Proc., 2010 9th Int. Conf. on Grid and Cloud Computing, 40–45. New York: IEEE.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 147 • Issue 10 • October 2021
Copyright
© 2021 American Society of Civil Engineers.
History
Received: Aug 23, 2020
Accepted: Apr 27, 2021
Published online: Aug 3, 2021
Published in print: Oct 1, 2021
Discussion open until: Jan 3, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Jiming Liu, Liping Duan, Siwei Lin, Ji Miao, Jincheng Zhao, Semantic Web-Based CRUD Operation and Constraint Validation for IFC-Based Information Management, Journal of Construction Engineering and Management, 10.1061/JCEMD4.COENG-14276, 150, 10, (2024).
- Abdullahi B. Saka, Lukumon O. Oyedele, Lukman A. Akanbi, Sikiru A. Ganiyu, Daniel W.M. Chan, Sururah A. Bello, Conversational artificial intelligence in the AEC industry: A review of present status, challenges and opportunities, Advanced Engineering Informatics, 10.1016/j.aei.2022.101869, 55, (101869), (2023).
- You-Chieh Lee, Chao-Ming Wang, Using Voice User Interface to Assist Designers in the Early Stage of Product Design, 2022 IET International Conference on Engineering Technologies and Applications (IET-ICETA), 10.1109/IET-ICETA56553.2022.9971589, (1-2), (2022).
- Daniel Antonio Linares-Garcia, Nazila Roofigari-Esfahan, Kristina Pratt, Myounghoon Jeon, Voice-Based Intelligent Virtual Agents (VIVA) to Support Construction Worker Productivity, Automation in Construction, 10.1016/j.autcon.2022.104554, 143, (104554), (2022).
- Ning Wang, Raja R.A. Issa, Chimay J. Anumba, Transfer learning-based query classification for intelligent building information spoken dialogue, Automation in Construction, 10.1016/j.autcon.2022.104403, 141, (104403), (2022).
- Faris Elghaish, Jatin Kumar Chauhan, Sandra Matarneh, Farzad Pour Rahimian, M. Reza Hosseini, Artificial intelligence-based voice assistant for BIM data management, Automation in Construction, 10.1016/j.autcon.2022.104320, 140, (104320), (2022).
- Chengke Wu, Xiao Li, Yuanjun Guo, Jun Wang, Zengle Ren, Meng Wang, Zhile Yang, Natural language processing for smart construction: Current status and future directions, Automation in Construction, 10.1016/j.autcon.2021.104059, 134, (104059), (2022).