Analytic Overview of Citation Metrics in the Civil Engineering Domain with Focus on Construction Engineering and Management Specialty Area and Its Subdisciplines
Publication: Journal of Construction Engineering and Management
Volume 145, Issue 10
Abstract
Several data-based metrics have been developed to evaluate the quality of research output, including h-index, impact factor, and others. Considerable debates have focused on whether the aforementioned metrics represent effective and efficient indications of research impact or just provide an imperfect reflection. Because the academic community is witnessing a growing trend where most of the tangible rewards are related to publications and their quality, there is a dire necessity to obtain a more realistic and holistic understanding of such evaluation indicators. As such, the goal of this paper is to investigate the parameters associated with the citation metrics used to assess the quality and impact of research publications in the broad civil engineering (CE) domain with a focus on the construction engineering and management (CEM) specialty area and its subdisciplines. To this end, the authors (1) analyzed the different CE research areas—including CEM—with respect to the number of publications, community size, and potential ties with other interdisciplinary fields; (2) studied the citation metrics of the most-cited as well as award-winning peer-reviewed publications in both CE and CEM while examining the effect of the availability of research funds as well as the nature of the research—quantitative or qualitative—on the average number of citations for most-cited publications across the different CE specialty areas; and (3) investigated the impact of multiple factors on the citation metrics. Based on a thorough survey and analysis of 67,800 journal publications, it was found that both number of publications and citation metrics for the CEM specialty area is significantly lower compared with other CE specialty areas. Along the same vein, the citations across the various CEM subdisciplines are all different, where labor and personnel (306 average citations per paper) and contracting (266 average citations per paper) are the most cited, but cost and schedule (47 average citations per paper) and legal and contractual (23 average citations per paper) are the least cited. Also, with the exception of aerospace engineering as well as engineering education and practices, CEM was the least funded, had the highest percentage of qualitative research, and the least average number of citations for funded/unfunded projects as well as qualitative/quantitative research. Further, it was revealed that award-winning papers—which supposedly represent the highest quality—are not highly cited and in fact have way below average citation numbers. Ultimately, it was found that several factors such as research community size, trendiness of research field, amount of research output, connection with other interdisciplinary fields, characteristics and attributes of coauthors, timing of checking an h-index, publication access type, and the criteria adopted by the various archiving databases collectively affect the citation metrics for papers and authors. As such, citation metrics like h-index should not be used solely to assess the quality and impact of an author but rather be utilized as a single piece of information within a more holistic and broader context. This study furnishes various parameters and considerations that should be normalized before attempting to compare citations of papers and/or authors across the CE domain.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data generated or analyzed during the study are included in the published article.
Acknowledgments
The authors appreciate the comments, suggestions, and recommendations provided by anonymous reviewers as they collectively helped hone and strengthen the quality of this manuscript during the blind peer-review process.
References
Abotaleb, I. S., and I. H. El-adaway. 2018. “Managing construction projects through dynamic modeling: Reviewing the existing body of knowledge and deriving future research directions.” J. Manage. Eng. 34 (6): 04018033. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000633.
Abudayyeh, O., A. Dibert-DeYoung, and E. Jaselskis. 2004. “Analysis of trends in construction research: 1985-2002.” J. Constr. Eng. Manage. 130 (3): 433–439. https://doi.org/10.1061/(ASCE)0733-9364(2004)130:3(433).
Akhtar, M. 2018. “List of Scopus indexed journals July 2018.” Accessed October 25, 2018. https://www.researchgate.net/publication/324587178_List_of_Scopus_indexed_journals_July_2018.
Antelman, K. 2004. “Do open-access articles have a greater research impact?” College Res. Libraries 65 (5): 372–382. https://doi.org/10.5860/crl.65.5.372.
Aragón, A. M. 2013. “A measure for the impact of research.” Sci. Rep. 3 (1): 1649. https://doi.org/10.1038/srep01649.
Bornmann, L., R. Mutz, and H. D. Daniel. 2008. “Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine.” J. Am. Soc. Inf. Sci. Technol. 59 (5): 830–837. https://doi.org/10.1002/asi.20806.
Bröchner, J. A. N., and B. C. Björk. 2008. “Where to submit? Journal choice by construction management authors.” Constr. Manage. Econ. 26 (7): 739–749. https://doi.org/10.1080/01446190802017698.
Carpenter, C. R., D. C. Cone, and C. C. Sarli. 2014. “Using publication metrics to highlight academic productivity and research impact.” Acad. Emergency Med. 21 (10): 1160–1172. https://doi.org/10.1111/acem.12482.
Clarivate Analytics. 2018a. “Web of Science platform: Web of Science core collection.” Accessed October 24, 2018. https://clarivate.libguides.com/webofscienceplatform/woscc.
Clarivate Analytics. 2018b. “2018 journal citation reports.” https://clarivate.com/blog/science-research-connect/the-2018-jcr-release-is-here/.
Costas, R., and M. Bordons. 2008. “Is g-index better than h-index? An exploratory study at the individual level.” Scientometrics 77 (2): 267–288. https://doi.org/10.1007/s11192-007-1997-0.
de la Garza, J. M. 2007. “Sponsored research and its impact on universities, faculty, and journals.” J. Constr. Eng. Manage. 133 (9): 708–709. https://doi.org/10.1061/(ASCE)0733-9364(2007)133:9(708).
de Winter, J. C., A. A. Zadpoor, and D. Dodou. 2014. “The expansion of Google Scholar versus Web of Science: A longitudinal study.” Scientometrics 98 (2): 1547–1565. https://doi.org/10.1007/s11192-013-1089-2.
Elsevier. 2017. Scopus content coverage guide. Amsterdam, Netherlands: Elsevier.
Elsevier. 2018a. “Content policy and selection.” Accessed October 25, 2018. https://www.elsevier.com/solutions/scopus/how-scopus-works/content/content-policy-and-selection.
Elsevier. 2018b. “How stringent are our standards?” Accessed October 25, 2018. https://www.elsevier.com/solutions/scopus/how-scopus-works/content.
Enago Academy. 2018. “Understanding research metrics: Journal-level, article-level, and author-level.” Accessed October 25, 2018. https://www.enago.com/academy/what-are-different-research-metrics/.
Falagas, M. E., E. I. Pitsouni, G. A. Malietzis, and G. Pappas. 2008. “Comparison of PubMed, Scopus, Web of Science, and Google Scholar: Strengths and weaknesses.” FASEB J. 22 (2): 338–342. https://doi.org/10.1096/fj.07-9492LSF.
Harzing, A. W., and S. Alakangas. 2016. “Google Scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison.” Scientometrics 106 (2): 787–804. https://doi.org/10.1007/s11192-015-1798-9.
Hirsch, J. E. 2005. “An index to quantify an individual’s scientific research output.” Proc. Natl. Acad. Sci. 102 (46): 16569–16572. https://doi.org/10.1073/pnas.0507655102.
Hunter, J. D. 2007. “Matplotlib: A 2D graphics environment.” Comput. Sci. Eng. 9 (3): 90–95. https://doi.org/10.1109/MCSE.2007.55.
Jennings, M., and I. El-adaway. 2012. “Ethical issues in multiple-authored and mentor-supervised publications.” J. Prof. Issues Eng. Educ. Pract. 138 (1): 37–47. https://doi.org/10.1061/(ASCE)EI.1943-5541.0000087.
Koler-Povh, T., P. Južnič, and G. Turk. 2014. “Impact of open access on citation of scholarly publications in the field of civil engineering.” Scientometrics 98 (2): 1033–1045. https://doi.org/10.1007/s11192-013-1101-x.
Larsen, J. K., L. F. Ussing, and T. D. Brunø. 2013. “Trend-analysis and research direction in construction management literature.” In Proc., Construction and Operation in the Context of Sustainability, 73–82. Reston, VA: ASCE.
Leydesdorff, L., L. Bornmann, J. A. Comins, and S. Milojević. 2016. “Citations: Indicators of quality? The impact fallacy.” Front. Res. Metrics Anal. 1 (Aug): 1.
Marnett, A. 2018. “H-Index: What it is and how to find yours.” Accessed October 25, 2018. https://www.benchfly.com/blog/h-index-what-it-is-and-how-to-find-yours/.
McKinney, W. 2010. “Data structures for statistical computing in python.” In Vol. 445 of Proc., 9th Python in Science Conf., 51–56. Amsterdam, Netherlands: Elsevier Science B.V.
Neophytou, J. 2014. “How to navigate the world of citation metrics.” Accessed October 25, 2018. https://hub.wiley.com/community/exchanges/discover/blog/2014/05/14/how-to-navigate-the-world-of-citation-metrics.
Oliphant, T. E. 2006. Vol. 1 of A guide to NumPy, 85. Washington, DC: Trelgol Publishing.
Oswald, N. 2018. “Does your h-index measure up?” Accessed October 25, 2018. https://bitesizebio.com/13614/does-your-h-index-measure-up/.
Pietroforte, R., and M. A. Aboulezz. 2005. “ASCE Journal of Management in Engineering: Review of the years 1985-2002.” J. Manage. Eng. 21 (3): 125–130. https://doi.org/10.1061/(ASCE)0742-597X(2005)21:3(125).
Pietroforte, R., and T. P. Stefani. 2004. “ASCE Journal of Construction Engineering and Management: Review of the years 1983-2000.” J. Constr. Eng. Manage. 130 (3): 440–448. https://doi.org/10.1061/(ASCE)0733-9364(2004)130:3(440).
Raheel, M., S. Ayaz, and M. T. Afzal. 2018. “Evaluation of h-index, its variants and extensions based on publication age & citation intensity in civil engineering.” Scientometrics 114 (3): 1107–1127. https://doi.org/10.1007/s11192-017-2633-2.
Seglen, P. O. 1997. “Citations and journal impact factors: Questionable indicators of research quality.” Allergy 52 (11): 1050–1056. https://doi.org/10.1111/j.1398-9995.1997.tb00175.x.
Sen, R., and P. Patel. 2011. “Citation rates of award-winning ASCE papers.” J. Prof. Issues Eng. Educ. Pract. 138 (2): 107–113. https://doi.org/10.1061/(ASCE)EI.1943-5541.0000092.
Spicer, A. 2015. “Explainer: What is an h-index and how is it calculated?” Accessed October 25, 2018. http://theconversation.com/explainer-what-is-an-h-index-and-how-is-it-calculated-41162.
Swan, A. 2010. “The open access citation advantage: Studies and results to date.”https://eprints.soton.ac.uk/268516/.
Testa, J. 2018. “Journal selection process.” Accessed October 25, 2018. https://clarivate.com/essays/journal-selection-process/.
University of Minnesota Libraries. 2018. “Library system.” Accessed October 10, 2018. https://www.lib.umn.edu/faq/5341.
Weingart, P. 2005. “Impact of bibliometrics upon the science system: Inadvertent consequences?” Scientometrics 62 (1): 117–131. https://doi.org/10.1007/s11192-005-0007-7.
Yi, W., and A. P. Chan. 2013. “Critical review of labor productivity research in construction journals.” J. Manage. Eng. 30 (2): 214–225. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000194.
Information & Authors
Information
Published In
Copyright
©2019 American Society of Civil Engineers.
History
Received: Nov 2, 2018
Accepted: Mar 11, 2019
Published online: Aug 7, 2019
Published in print: Oct 1, 2019
Discussion open until: Jan 7, 2020
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.