A Novel Approach to Capture Similarity in Capital Project Benchmarking: An Application to Healthcare Projects
Publication: Journal of Management in Engineering
Volume 38, Issue 3
Abstract
For credible project benchmarking, like-for-like project comparison is a prerequisite to set realistic targets for improvement, especially when a heterogeneous sample of healthcare projects is compared to another. However, the current method for determining the groups of similar projects relies on an ad-hoc technique that can lead to suboptimal target settings for improvement. To address this issue, this research proposes a novel approach to capture similarity for capital project benchmarking by leveraging Classification and Regression Trees (CART). This research focuses on healthcare projects. The data collected from a total of 89 healthcare projects were used to construct the trees by selecting a set of critical and flexible features that are closely associated with two metrics representing cost and schedule performance of the selected projects. The effectiveness of results derived from the proposed method was validated through statistical methods and comparative analysis. The proposed method allows for more targeted performance comparisons by capturing similarity using flexible sets of meaningful features, which reduces the search space of determining a group of similar projects. The new approach is, thus, expected to help organizations gain better insights into their relative performance position when benchmarking their capital projects.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
The authors thank the CII NHFBP participants for their contribution to this research by supporting benchmarking data. This research would not have been possible without their inputs throughout the development of this research. The conclusions herein are those of the authors and do not necessarily reflect the views of CII.
References
Beatham, S., C. Anumba, T. Thorpe, and I. Hedges. 2004. “KPIs: A critical appraisal of their use in construction.” Benchmarking 11 (1): 93–117. https://doi.org/10.1108/14635770410520320.
Breiman, L., J. H. Friedman, R. A. Olshen, and C. J. Stone. 1984. Classification and regression trees. Boca Raton, FL: Taylor & Francis Group.
Camp, R. C. 1995. Business process benchmarking: Finding and implementing best practices. Milwaukee, WI: ASQC Quality Press.
CEC (California Energy Commission). 2019. “Building energy efficiency standards—Title 24.” Accessed December 3, 2019. https://www.energy.ca.gov/programs-and-topics/programs/building-energy-efficiency-standards.
Chanmeka, A., S. R. Thomas, C. H. Caldas, and S. P. Mulva. 2012. “Assessing key factors impacting the performance and productivity of oil and gas projects in Alberta.” Can. J. Civ. Eng. 39 (3): 259–270. https://doi.org/10.1139/l11-128.
Choi, J., F. Leite, and D. P. de Oliveira. 2018. “BIM-based benchmarking system for healthcare projects: Feasibility study and functional requirements.” Autom. Constr. 96 (Dec): 262–279. https://doi.org/10.1016/j.autcon.2018.09.015.
Choi, J., F. Leite, and D. P. de Oliveira. 2020. “BIM-based benchmarking for healthcare construction projects.” Autom. Constr. 119 (Nov): 103347. https://doi.org/10.1016/j.autcon.2020.103347.
Choi, J., F. Leite, D. P. de Oliveira, and D. P. De Oliveira. 2017. “Model-based benchmarking for healthcare projects: System requirements and demonstration.” In Vol. 3 of Proc., ASCE Int. Workshop on Computing in Civil Engineering 2017, 125–132. Reston, VA: ASCE.
Choi, J., S. Yun, and D. P. de Oliveira. 2016. “Developing a cost normalization framework for phase-based performance assessment of construction projects.” Can. J. Civ. Eng. 43 (12): 1075–1086. https://doi.org/10.1139/cjce-2016-0223.
Choi, J., S. Yun, F. Leite, and S. P. Mulva. 2019. “Team integration and owner satisfaction: Comparing integrated project delivery with construction management at risk in health care projects.” J. Manage. Eng. 35 (1): 05018014. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000654.
Choi, J., S. Yun, S. P. Mulva, D. Oliveira, and Y. Kang. 2015. “A multi-perspective assessment method for measuring leading indicatiors in capital project benchmarking.” In Proc., CSCE/ICSC, 1–10. Vancouver, BC: Univ. of British Columbia.
Choi, M., and G. Lee. 2010. “Decision tree for selecting retaining wall systems based on logistic regression analysis.” Autom. Constr. 19 (7): 917–928. https://doi.org/10.1016/j.autcon.2010.06.005.
Chou, J. S., and C. Lin. 2013. “Predicting disputes in public-private partnership projects: Classification and ensemble models.” J. Comput. Civ. Eng. 27 (1): 51–60. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000197.
Chung, W. 2011. “Review of building energy-use performance benchmarking methodologies.” Appl. Energy 88 (5): 1470–1479. https://doi.org/10.1016/j.apenergy.2010.11.022.
CII (Construction Industry Institute). 2014. “National healthcare facility benchmarking summary report, 2011-2013.” Accessed January 7, 2018. https://www.construction-institute.org/resources/knowledgebase/best-practices/benchmarking-metrics.
Costa, D., C. Formoso, M. Kagioglou, L. Alarcón, and C. Caldas. 2006. “Benchmarking initiatives in the construction industry: Lessons learned and improvement opportunities.” J. Manage. Eng. 22 (4): 158–167. https://doi.org/10.1061/(ASCE)0742-597X(2006)22:4(158).
de Ville, B. 2006. Decision trees for business intelligence and data mining: Using SAS Enterprise Miner. Cary, NC: SAS Institute.
Du, J., R. Liu, and R. R. A. Issa. 2014. “BIM cloud score: Benchmarking BIM performance.” J. Constr. Eng. Manage. 140 (11): 04014054. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000891.
Ebrahimi, M., E. Ebrahimie, and M. Ebrahimi. 2009. “Searching for patterns of thermostability in proteins and defining the main features contributing to enzyme thermostability through screening, clustering, and decision tree algorithms.” EXCLI J 8: 218–233.
El-Mashaleh, M. S., R. E. Minchin Jr., W. J. O’Brien, R. E. Minchin, and W. J. O’Brien. 2007. “Management of construction firm performance using benchmarking.” J. Manage. Eng. 23 (1): 10–17. https://doi.org/10.1061/(ASCE)0742-597X(2007)23:1(10).
Eriksson, P. E., and M. Westerberg. 2011. “Effects of cooperative procurement procedures on construction project performance: A conceptual framework.” Int. J. Project Manage. 29 (2): 197–208. https://doi.org/10.1016/j.ijproman.2010.01.003.
Esmaeili, M., and F. Gabor. 2011. “Feature selection as an improving step for decision tree construction.” In Vol. 3 of Proc., Int. Conf. on Machine Learning (ICML), 35–39. Singapore: International Association of Computer Science and Information Technology Press.
Esposito, F., D. Malerba, and G. Semeraro. 1997. “A comparative analysis of methods for pruning decision trees.” IEEE Trans. Pattern Anal. Mach. Intell. 19 (5): 476–493. https://doi.org/10.1109/34.589207.
Fan, C., and F. Xiao. 2017. “Assessment of building operational performance using data mining techniques: A case study.” Energy Procedia 111 (Sep): 1070–1078. https://doi.org/10.1016/j.egypro.2017.03.270.
Farris, J. A., R. L. Groesbeck, E. M. Van Aken, and G. Letens. 2006. “Evaluating the relative performance of engineering design projects: A case study using data envelopment analysis.” IEEE Trans. Eng. Manage. 53 (3): 471–482. https://doi.org/10.1109/TEM.2006.878100.
Francisco, A., N. Mohammadi, and J. E. Taylor. 2020. “Smart city digital twin–enabled energy management: Toward real-time urban building energy benchmarking.” J. Manage. Eng. 36 (2): 04019045. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000741.
Goh, Y. M., and D. Chua. 2013. “Neural network analysis of construction safety management systems: A case study in Singapore.” Construct. Manage. Econ. 31 (5): 460–470. https://doi.org/10.1080/01446193.2013.797095.
Hwang, B., H. F. Tan, and S. Sathish. 2013. “Capital project performance measurement and benchmarking in Singapore.” Eng. Constr. Archit. Manage. 20 (2): 143–159. https://doi.org/10.1108/09699981311303017.
Hwang, B., S. R. Thomas, D. Degezelle, and C. H. Caldas. 2008. “Development of a benchmarking framework for pharmaceutical capital projects.” Construct. Manage. Econ. 26 (2): 177–195. https://doi.org/10.1080/01446190701802398.
Ibn-homaid, N. T., and I. A. Al-sulaihi. 2016. “Benchmarking financial performance: A case study of Saudi construction companies.” Benchmarking 2 (12): 1201–1210.
IPA (Independent Project Analysis). 2019. “IPA best practice in benchmarking.” Accessed November 5, 2019. https://www.gov.uk/government/publications.
Kamakura, W. A. 1988. “The use of categorical variables in data envelopment analysis.” Manage. Sci. 34 (10): 1273–1276. https://doi.org/10.1287/mnsc.34.10.1273.
Kass, G. V. 1980. “An exploratory technique for investigating large quantities of categorical data.” J. R. Stat. Soc. 29 (2): 119–127. https://doi.org/10.2307/2986296.
Kim, H., and W. Y. Loh. 2001. “Classification trees with unbiased multiway splits.” J. Am. Stat. Assoc. 96 (454): 589–604. https://doi.org/10.1198/016214501753168271.
Kim, H., L. Soibelman, and F. Grobler. 2008. “Factor selection for delay analysis using knowledge discovery in databases.” Autom. Constr. 17 (5): 550–560. https://doi.org/10.1016/j.autcon.2007.10.001.
Korkmaz, S., D. Riley, and M. Horman. 2010. “Piloting evaluation metrics for sustainable high-performance building project delivery.” J. Constr. Eng. Manage. 136 (8): 877–885. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000195.
Lam, E. W. M., A. P. C. Chan, and D. W. M. Chan. 2010. “Benchmarking success of building maintenance projects.” Facilities 28 (5–6): 290–305. https://doi.org/10.1108/02632771011031529.
Lee, H., J. Shin, and J.-K. Lee. 2016. “BIM-enabled definition of a path object and its properties to evaluate building circulation using numerical data.” J. Asian Archit. Build. Eng. 15 (3): 425–432. https://doi.org/10.3130/jaabe.15.425.
Lee, S.-H., S. P. Thomas, and R. L. Tucker. 2005. “Web-based benchmarking system for the construction industry.” J. Constr. Eng. Manage. 131 (7): 790–798. https://doi.org/10.1061/(ASCE)0733-9364(2005)131:7(790).
Li, Y. 2006. “Predicting materials properties and behavior using classification and regression trees.” Mater. Sci. Eng., A 433 (1–2): 261–268. https://doi.org/10.1016/j.msea.2006.06.100.
Li, Y., L. Cao, Y. Han, and J. Wei. 2020. “Development of a conceptual benchmarking framework for healthcare facilities management: Case study of Shanghai municipal hospitals.” J. Constr. Eng. Manage. 146 (1): 05019016. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001731.
Lin, N., D. Noe, and X. He. 2006. “Tree-based methods and their applications.” In Springer handbook of engineering statistics, edited by Pham H. New York: Springer. https://doi.org/10.1007/978-1-84628-288-1_30.
Liu, J., H. Chen, J. Liu, Z. Li, R. Huang, L. Xing, J. Wang, and G. Li. 2017a. “An energy performance evaluation methodology for individual office building with dynamic energy benchmarks using limited information.” Appl. Energy 206 (Nov): 193–205. https://doi.org/10.1016/j.apenergy.2017.08.153.
Liu, R., J. Du, R. R. A. Issa, and B. Giel. 2017. “bBIM cloud score: Building information model and modeling performance benchmarking.” J. Inf. Technol. Constr. 43: 04016109. https://doi.org/10.1061/(ASCE)CO.1943-7862.0001251.
Loh, W. Y. 2014. “Fifty years of classification and regression trees.” Int. Stat. Rev. 82 (3): 329–348. https://doi.org/10.1111/insr.12016.
Loh, W. Y., and Y. S. Shin. 1997. “Split selection methods for classification trees.” Stat. Sin. 7 (4): 815–840.
Loh, W.-Y. 2011. “Classification and regression trees.” Wiley Interdiscip. Rev.: Data Min. Knowl. Discovery 1 (1): 14–23.
Love, P. E. D. 2002. “Influence of project type and procurement method on rework costs in building construction projects.” J. Constr. Eng. Manage. 128 (1): 18–29. https://doi.org/10.1061/(ASCE)0733-9364(2002)128:1(18).
Mistikoglu, G., I. H. Gerek, E. Erdis, P. E. Mumtaz Usmen, H. Cakan, and E. E. Kazan. 2015. “Decision tree analysis of construction fall accidents involving roofers.” Expert Syst. Appl. 42 (4): 2256–2263. https://doi.org/10.1016/j.eswa.2014.10.009.
Mulva, S. P., and J. Dai. 2009. “Healthcare facility benchmarking.” HERD: Health Environ. Res. Des. J. 3 (1): 28–37. https://doi.org/10.1177/193758670900300104.
Newell, S. 2004. “Enhancing cross-project learning.” Eng. Manage. J. 16 (1): 12–20.
Olsen, I. S., N. H. Bertelsen, A. K. Frandsen, and K. Haugbølle. 2010. The benchmark centre for the Danish construction sector (BEC): Applying and improving key performance indicators (KPI) in the Danish construction sector CREDIT Case DK02. Aalborg, Denmark: Aalborg Univ.
Olubunmi, O. A., P. B. Xia, and M. Skitmore. 2016. “Green building incentives: A review.” Renewable Sustainable Energy Rev. 59 (Jun): 1611–1621. https://doi.org/10.1016/j.rser.2016.01.028.
Pakgohar, A., R. S. Tabrizi, M. Khalili, and A. Esmaeili. 2011. “The role of human factor in incidence and severity of road crashes based on the CART and LR regression: A data mining approach.” Procedia Comput. Sci. 3 (Jan): 764–769. https://doi.org/10.1016/j.procs.2010.12.126.
Park, H. S., M. Lee, H. Kang, T. Hong, and J. Jeong. 2016. “Development of a new energy benchmark for improving the operational rating system of office buildings using various data-mining techniques.” Appl. Energy 173 (Jul): 225–237. https://doi.org/10.1016/j.apenergy.2016.04.035.
Park, J., D. Ki, K. Kim, S. J. Lee, D. H. Kim, and K. J. Oh. 2011. “Using decision tree to develop a soil ecological quality assessment system for planning sustainable construction.” Expert Syst. Appl. 38 (5): 5463–5470. https://doi.org/10.1016/j.eswa.2010.10.007.
Poh, C. Q. X., C. U. Ubeynarayana, and Y. M. Goh. 2018. “Safety leading indicators for construction sites: A machine learning approach.” Autom. Constr. 93 (Sep): 375–386. https://doi.org/10.1016/j.autcon.2018.03.022.
Quinlan, J. R. 1999. “Simplifying decision trees.” Int. J. Hum. Comput. Stud. 51 (2): 497–510. https://doi.org/10.1006/ijhc.1987.0321.
Ramırez, R. R., L. F. C. Alarcon, and P. Knights. 2004. “Benchmarking system for evaluating management practices in the construction industry.” J. Manage. Eng. 20 (3): 110–117. https://doi.org/10.1061/(ASCE)0742-597X(2004)20:3(110).
Rankin, J., A. R. Fayek, G. Meade, C. Haas, and A. Manseau. 2008. “Initial metrics and pilot program results for measuring the performance of the Canadian construction industry.” Can. J. Civ. Eng. 35 (9): 894–907. https://doi.org/10.1139/L08-018.
Robichaud, L. B., and V. S. Anantatmula. 2011. “Greening project management practices for sustainable construction.” J. Manage. Eng. 27 (1): 48–57. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000030.
Rodríguez-Déniz, H., and A. Voltes-Dorta. 2014. “A frontier-based hierarchical clustering for airport efficiency benchmarking.” Benchmarking 21 (4): 486–508. https://doi.org/10.1108/BIJ-09-2012-0057.
Salimi, A., J. Rostami, and C. Moormann. 2017. “Evaluating the suitability of existing rock mass classification systems for TBM performance prediction by using a regression tree.” Procedia Eng. 191: 299–309. https://doi.org/10.1016/j.proeng.2017.05.185.
Sebastian, R., and L. van Berlo. 2010. “Tool for benchmarking BIM performance of design, engineering and construction firms in The Netherlands.” Archit. Eng. Des. Manage. 6 (4): 254–263. https://doi.org/10.3763/aedm.2010.IDDS3.
Shapiro, S., and M. B. Wilk. 1965. “An analysis of variance test for normality (complete samples).” Biometrika 52 (3–4): 591–611. https://doi.org/10.1093/biomet/52.3-4.591.
Soibelman, L., and H. Kim. 2002. “Data preparation process for construction knowledge generation through knowledge discovery in databases.” J. Comput. Civ. Eng. 16 (1): 39–48. https://doi.org/10.1061/(ASCE)0887-3801(2002)16:1(39).
Suk, S.-J., B.-G. Hwang, J. Dai, C. H. Caldas, and S. P. Mulva. 2012. “Performance dashboard for a pharmaceutical project benchmarking program.” J. Constr. Eng. Manage. 138 (7): 864–876. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000503.
Swarup, L., S. Korkmaz, and D. Riley. 2011. “Project delivery metrics for sustainable, high-performance buildings.” J. Constr. Eng. Manage. 137 (12): 1043–1051. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000379.
Tixier, A. J. P., M. R. Hallowell, B. Rajagopalan, and D. Bowman. 2016. “Application of machine learning to construction injury prediction.” Autom. Constr. 69 (Sep): 102–114. https://doi.org/10.1016/j.autcon.2016.05.016.
Tran, H., and D. G. Carmichael. 2013. “A contractor’s classification of owner payment practices.” Eng. Constr. Archit. Manage. 20 (1): 29–45. https://doi.org/10.1108/09699981311288664.
Tsanas, A., and A. Xifara. 2012. “Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools.” Energy Build. 49 (Jun): 560–567. https://doi.org/10.1016/j.enbuild.2012.03.003.
Vitner, G., S. Rozenes, and S. Spraggett. 2006. “Using data envelope analysis to compare project efficiency in a multi-project environment.” Int. J. Project Manage. 24 (4): 323–329. https://doi.org/10.1016/j.ijproman.2005.09.004.
Weeks, D. J., and F. Leite. 2022. “Minimizing facility corrective maintenance: Benchmarking preventative-to-corrective maintenance ratios using maintenance data and building age in dormitories.” J. Manage. Eng. 38 (1): 1–14. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000996.
Weng, J., and Q. Meng. 2011. “Decision tree-based model for estimation of work zone capacity.” Transp. Res. Rec. 2257 (1): 40–50. https://doi.org/10.3141/2257-05.
Wu, C., R. Estate, B. Xu, R. Estate, C. Mao, R. Estate, and X. Li. 2017. “Overview of BIM maturity measurement tools.” J. Inf. Technol. Construct. 22 (3): 34–62.
Yeung, J. F., A. P. C. Chan, and D. W. M. Chan. 2009. “Developing a performance index for relationship-based construction projects in Australia: Delphi study.” J. Manage. Eng. 25 (2): 59–68. https://doi.org/10.1061/(ASCE)0742-597X(2009)25:2(59).
Yeung, J. F. Y., A. P. C. Chan, D. W. M. Chan, Y. H. Chiang, and H. Yang. 2013. “Developing a benchmarking model for construction projects in Hong Kong.” J. Constr. Eng. Manage. 139 (6): 705–716. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000622.
Yun, S., J. Choi, D. P. de Oliveira, and S. P. Mulva. 2016a. “Development of performance metrics for phase-based capital project benchmarking.” Int. J. Project Manage. 34 (3): 389–402. https://doi.org/10.1016/j.ijproman.2015.12.004.
Yun, S., J. Choi, D. P. Oliveira, S. P. Mulva, and Y. Kang. 2016b. “Measuring project management inputs throughout capital project delivery.” Int. J. Project Manage. 34 (7): 1167–1182. https://doi.org/10.1016/j.ijproman.2016.06.004.
Zhang, Z., Z. Y. Koh, and F. Ling. 2020. “Benchmarking contractors’ financial performance: Case study of Singapore.” J. Financ. Manage. Property Constr. 25 (2): 183–199. https://doi.org/10.1108/JFMPC-03-2019-0024.
Zhou, J., X. Li, and H. S. Mitri. 2016. “Classification of rockburst in underground projects: Comparison of ten supervised learning methods.” J. Comput. Civ. Eng. 30 (5): 04016003. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000553.
Zuo, J., and Z. Y. Zhao. 2014. “Green building research-current status and future agenda: A review.” Renewable Sustainable Energy Rev. 30 (Feb): 271–281. https://doi.org/10.1016/j.rser.2013.10.021.
Information & Authors
Information
Published In
Journal of Management in Engineering
Volume 38 • Issue 3 • May 2022
Copyright
© 2022 American Society of Civil Engineers.
History
Received: Sep 29, 2021
Accepted: Jan 6, 2022
Published online: Feb 28, 2022
Published in print: May 1, 2022
Discussion open until: Jul 28, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Chan Young Park, Sungmin Yun, What Determines Infrastructure Investments: Critical Factors from Preliminary Feasibility Studies for Capital Projects in Korea, Journal of Management in Engineering, 10.1061/JMENEA.MEENG-6017, 40, 4, (2024).
- Yujin Choi, Seungwon Seo, Taehoon Hong, Choongwan Koo, A Classification Model Using Personal Biometric Characteristics to Identify Individuals Vulnerable to an Extremely Hot Environment, Journal of Management in Engineering, 10.1061/JMENEA.MEENG-5495, 40, 2, (2024).
- Jiyong Choi, Zhe Yin, Myungjin Chae, Sungmin Yun, Benchmarking the Material Use Efficiency of Building Projects, Journal of Management in Engineering, 10.1061/JMENEA.MEENG-5619, 39, 6, (2023).
- Taewoo Ko, H. David Jeong, JeeHee Lee, Natural Language Processing–Driven Similar Project Determination Using Project Scope Statements, Journal of Management in Engineering, 10.1061/JMENEA.MEENG-5229, 39, 3, (2023).