Abstract
Due to globalization trends, engineers are increasingly expected to work effectively across national and cultural boundaries. However, there remains a lack of valid and reliable measures of global engineering competency. To address this gap, the research team has undertaken a large-scale research project to develop a suite of instruments to measure global engineering competency (GEC), including self-reported scales and situational judgment tests. This study reports on the development and initial validation of one such instrument, the Global Engineering Competency Scale (GECS). The GECS was administered to 400 practicing engineers, and exploratory and confirmatory factor analyses were conducted in sequence to further develop and evaluate the instrument. The results provide evidence of the construct validity of the GECS to measure two distinct dimensions of GEC: behavioral and cognitive competencies. In addition to describing some possible uses for this assessment tool, the need for further research to collect additional evidence regarding the validity and reliability of the instrument is also discussed.
Get full access to this article
View all available purchase options and get full access to this article.
References
Ang, S., and L. Van Dyne. 2008. Handbook of cultural intelligence: Theory, measurement, and applications. Armonk, NY: M.E. Sharpe.
Ang, S., L. Van Dyne, and C. Koh. 2006. “Personality correlates of the four-factor model of cultural intelligence.” Group Organ. Manage. 31 (1): 100–123. https://doi.org/10.1177/1059601105275267.
ASCE. 2008. Civil engineering body of knowledge for the 21st century: Preparing the civil engineer for the future. 2nd ed. Reston, VA: ASCE.
Awang, Z. H. 2012. “Validating the measurement model: CFA.” In A handbook on SEM. 4th ed., 54–73. Shah Alam, Malaysia: Universiti Teknologi MARA.
Ball, A. G., H. Zaugg, R. Davies, I. Tateishi, A. R. Parkinson, C. G. Gensen, and S. P. Magleby. 2012. “Identification and validation of a set of global competencies for engineering students.” Int. J. Eng. Educ. 28 (1): 156–168.
Bennett, M. J. 2017. “Developmental model of intercultural sensitivity.” In The international encyclopedia of intercultural communication, 1–10. Hoboken, NJ: Wiley.
Bentler, P. M., and T. Dijkstra. 1985. “Efficient estimation via linearization in structural models.” In Multivariate analysis—VI, edited by P. R. Krishnaiah, 9–42. Amsterdam, Netherlands: Elsevier.
Bielefeldt, A., K. Paterson, and C. Swan. 2010. “Measuring the value added from service learning in project-based engineering education.” Int. J. Eng. Educ. 26 (3): 535–546.
Byrne, B. M. 1994. Structural equation modelling with EQS and EQS/Windows: Basic concepts, applications, and programming. Thousand Oaks, CA: Sage.
Carrato, P., and R. Haryott. 2003. “Building leaders of a global society.” J. Civil Eng. Educ. 129 (3): 125–128. https://doi.org/10.1061/(ASCE)1052-3928(2003)129:3(125).
Comrey, A., and H. Lee. 1992. A first course in factor analysis. Hillsdale, NJ: Erlbaum.
Costello, A. B., and J. W. Osborne. 2005. “Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis.” Pract. Assess. Res. Eval. 10 (7): 1–9.
Curran, P. J., S. G. West, and J. F. Finch. 1996. “The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis.” Psychol. Methods 1 (1): 16–29. https://doi.org/10.1037/1082-989X.1.1.16.
Dalal, D. K., and N. T. Carter. 2015. “Negatively worded items negatively impact survey research.” In More statistical and methodological myths and urban legends, edited by C. E. Lance and R. J. Vandenberg, 112–132. New York: Routledge.
Davies, R., H. Zaugg, and I. Tateishi. 2015. “Design and development of a cross-cultural disposition inventory.” Eur. J. Eng. Educ. 40 (1): 81–94. https://doi.org/10.1080/03043797.2014.915289.
DeVellis, R. F. 2016. Vol. 26 of Scale development: Theory and applications. Los Angeles: Sage.
Douglas, K. A., and S. Purzer. 2015. “Validity: Meaning and relevancy in assessment for engineering education research.” J. Eng. Educ. 104 (2): 108–118. https://doi.org/10.1002/jee.20070.
Downey, G., et al. 2006. “The globally competent engineer: Working effectively with people who define problems differently.” J. Eng. Educ. 95 (2): 107–122. https://doi.org/10.1002/j.2168-9830.2006.tb00883.x.
Downey, G. L., and K. Beddoes. 2010. What is global engineering education for? The making of international educators. San Rafael, CA: Morgan and Claypool.
Fabrigar, L. R., D. T. Wegener, R. C. MacCallum, and E. J. Strahan. 1999. “Evaluating the use of exploratory factor analysis in psychological research.” Psychol. Methods 4 (3): 272. https://doi.org/10.1037/1082-989X.4.3.272.
Fantini, A. E. 2009. “Assessing intercultural competence: Issues and tools.” In The SAGE handbook of intercultural competence, edited by D. K. Deardorff, 456–476. Thousand Oaks, CA: Sage.
Fuertes, J. N., M. L. Miville, J. J. Mohr, W. E. Sedlacek, and D. Gretchen. 2019. “Factor structure and short form of the Miville-Guzman universality-diversity scale.” Meas. Eval. Couns. Dev. 33 (3): 157–170. https://doi.org/10.1080/07481756.2000.12069007.
Goldberg, L. R., J. A. Johnson, H. W. Eber, R. Hogan, M. C. Ashton, C. R. Cloninger, and H. C. Gough. 2006. “The international personality item pool and the future of public-domain personality measures.” J. Res. Personality 40 (1): 84–96. https://doi.org/10.1016/j.jrp.2005.08.007.
Gorsuch, R. L. 1983. Factor analysis. 2nd ed. Hillsdale, NJ: Erlbaum.
Grigg, N. S. 2014. “Broad, global, and multidisciplinary civil engineering education.” J. Civ. Eng. Educ. 140 (1): 02513002. https://doi.org/10.1061/(ASCE)EI.1943-5541.0000180.
Guadagnoli, E., and W. F. Velicer. 1988. “Relation of sample size to the stability of component patterns.” Psychol. Bull. 103 (2): 265–275. https://doi.org/10.1037/0033-2909.103.2.265.
Hammer, M. R., M. J. Bennett, and R. Wiseman. 2003. “Measuring intercultural sensitivity: The intercultural development inventory.” Int. J. Intercultural Relat. 27 (4): 421–443. https://doi.org/10.1016/S0147-1767(03)00032-4.
Hu, L., and P. M. Bentler. 1999. “Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternations.” Struct. Equation Model. 6 (1): 1–55. https://doi.org/10.1080/10705519909540118.
Hu, L. T., and P. M. Bentler. 1995. “Evaluating model fit.” In Structural equation modelling: Concepts, issues, and applications, edited by R. H. Hoyle, 76–99. London: Sage.
Hutcheson, G. D., and N. Sofroniou. 1999. The multivariate social scientist: Introductory statistics using generalized linear models. London: Sage.
Jesiek, B. K., and K. Beddoes. 2010. “From diplomacy and development to competitiveness and globalization: Historical perspectives on the internationalization of engineering education.” In What is global engineering education for? The making of international educators, edited by G. L. Downey and K. Beddoes, 45–76. San Rafael, CA: Morgan and Claypool.
Jesiek, B. K., Y. Haller, and J. Thompson. 2014a. “Developing globally competent engineering researchers: Outcomes-based instructional and assessment strategies from the IREE 2010 China research abroad program.” Adv. Eng. Educ. 4 (1): 1–31.
Jesiek, B. K., S. E. Woo, S. Parrigon, and C. Porter. Forthcoming. “Development and initial validation of a situational judgment test (SJT) for global engineering competency (GEC).” J. Eng. Educ.
Jesiek, B. K., Q. Zhu, S. E. Woo, J. Thompson, and A. Mazzurco. 2014b. “Global engineering competency in context: Situations and behaviors.” Online J. Global Eng. Educ. 8 (1): 1.
Jesiek, N. K., N. Trellinger, and Q. Zhu. 2018. “Global engineering competency: Assessment tools and training strategies.” In Proc., 2018 ASEE Annual Conf. and Exhibition. Washington, DC: American Society for Engineering Education.
Jha, M. K., and D. R. Lynch. 2007. “Role of globalization and sustainable engineering practice in the future civil engineering education.” WIT Trans. Ecol. Environ. 102: 641–650. https://doi.org/10.2495/SDP070622.
Johri, A., and B. K. Jesiek. 2014. “Global and international issues in engineering education.” In Cambridge handbook of engineering education research, edited by A. Johri and B. M. Olds, 655–672. New York: Cambridge University Press.
Kaiser, H. F. 1970. “A second generation little jiffy.” Psychometrika 35 (4): 401–415. https://doi.org/10.1007/BF02291817.
Kline, P. 1994. An easy guide to factor analysis. New York: Routledge.
LaFave, J. M., H.-S. Kang, and J. D. Kaiser. 2015. “Cultivating intercultural competencies for civil engineering students in the era of globalization: Case study.” J. Civ. Eng. Educ. 141 (3): 05014008. https://doi.org/10.1061/(ASCE)EI.1943-5541.0000234.
Lee, W. C., A. Godwin, and A. L. Hermunstad Nave. 2018. “Development of the engineering student integration instrument: Rethinking measures of integration.” J. Eng. Educ. 107 (1): 30–55. https://doi.org/10.1002/jee.20184.
Lohmann, J. R., H. A. Rollins Jr., and J. J. Hoey. 2006. “Defining, developing and assessing global competence in engineers.” Eur. J. Eng. Educ. 31 (1): 119–131. https://doi.org/10.1080/03043790500429906.
MacCallum, R. C., M. W. Browne, and H. M. Sugawara. 1996. “Power analysis and determination of sample size for covariance structure modeling.” Psychol. Methods 1 (2): 130–149. https://doi.org/10.1037/1082-989X.1.2.130.
Matsunaga, M. 2010. “How to factor-analyze your data right: Do’s, don’ts, and how-to’s.” Int. J. Psychol. Res. 3 (1): 97–110. https://doi.org/10.21500/20112084.854.
Mazzurco, A., B. K. Jesiek, and K. Ramane. 2012. “Are engineering students culturally intelligent?: Preliminary results from a multiple group study.” In Proc., 2012 ASEE Conf. and Exhibition. Washington, DC: American Society for Engineering Education.
Merkle, E. C., and D. You. 2018. “Getting started with nonnest2.” Accessed August 21, 2019. https://cran.r-project.org/web/packages/nonnest2/vignettes/nonnest2.pdf.
Miville, M. L., P. Holloway, C. Gelso, R. Pannu, W. Liu, P. Touradji, and J. Fuertes. 1999. “Appreciating similarities and valuing differences: The Miville-Guzman universality-diversity scale.” J. Couns. Psychol. 46 (3): 291–307. https://doi.org/10.1037/0022-0167.46.3.291.
Mohtar, R. H., and A. E. Dare. 2012. “Global design team: A global service-learning experience.” Int. J. Eng. Educ. 28 (1): 169–182.
Mullins-Sweatt, S. N., J. E. Jamerson, D. B. Samuel, D. R. Olson, and T. A. Widiger. 2006. “Psychometric properties of an abbreviated instrument of the five-factor model.” Assessment 13 (2): 119–137. https://doi.org/10.1177/1073191106286748.
Muthen, B., and D. Kaplan. 1992. “A comparison of some methodologies for the factor analysis of non-normal Likert variables: A note on the size of the model.” Br. J. Math. Stat. Psychol. 45 (1): 19–30. https://doi.org/10.1111/j.2044-8317.1992.tb00975.x.
NSF (National Science Foundation). 2018. “Science & engineering indicators 2018.” Accessed January 10, 2018. https://nsf.gov/statistics/2018/nsb20181/report/sections/science-and-engineering-labor-force/introduction.
Nunnally, J. C., and I. H. Bernstein. 1978. Psychometric theory. New York: McGraw-Hill.
Parkinson, A. 2007. “Engineering study abroad programs: Formats, challenges, best practices.” Online J. Global Eng. Educ. 2 (2): 2.
Parkinson, A. 2009. “The rationale for developing global competence.” Online J. Global Eng. Educ. 4 (2): 2.
Parkinson, A., J. Harb, and S. Magleby. 2009. “Developing global competence in engineers: What does it mean? What is most important?” In Proc., ASEE Annual Conf. and Exposition. Washington, DC: American Society for Engineering Education.
Pienaar, J., P. Wu, and N. Adams. 2016. “Development of virtual teamwork skills for distance students through simulated global virtual team projects.” J. Civ. Eng. Educ. 142 (1): 05015003. https://doi.org/10.1061/(ASCE)EI.1943-5541.0000252.
Podsakoff, P. M., S. B. MacKenzie, J. Lee, and N. P. Podsakoff. 2003. “Common method biases in behavioral research: A critical review of the literature and recommended remedies.” J. Appl. Psychol. 88 (5): 879–903. https://doi.org/10.1037/0021-9010.88.5.879.
Ragusa, G. 2011. “Engineering preparedness for global workforces: Curricular connections and experiential impacts.” In Proc., 2011 ASEE Annual Conf. and Exposition. Washington, DC: American Society of Engineering Education.
Raiche, G., and D. Magis. 2010. “Package ‘nFactors’.” Accessed August 21, 2019. https://cran.r-project.org/web/packages/nFactors/nFactors.pdf.
R Core Team and contributors worldwide. 2018. “The R stats package.” Accessed August 21, 2019. https://stat.ethz.ch/R-manual/R-devel/library/stats/html/00Index.html.
Revelle, W. 2018. “The ‘psych’ package.” Accessed August 21, 2019. https://cran.r-project.org/web/packages/psych/psych.pdf.
Robinson, M. A. 2018. “Using multi-item psychometric scales for research and practice in human resource management.” Hum. Res. Manage. 57 (3): 739–750. https://doi.org/10.1002/hrm.21852.
Rosseel, Y. 2012. “Lavaan: An R package for structural equation modelling.” J. Stat. Software 48 (2): 1–36. https://doi.org/10.18637/jss.v048.i02.
Schell, K. L., and F. L. Oswald. 2013. “Item grouping and item randomization in personality measurement.” Personality Individual Differences 55 (3): 317–321. https://doi.org/10.1016/j.paid.2013.03.008.
Sheehan, M. 2009. “Globalization: Conundrums and paradoxes for civil engineering.” Leadersh. Manage. Eng. 10 (1): 10–15. https://doi.org/10.1061/(ASCE)LM.1943-5630.0000051.
Spitzberg, B. H., and G. Changnon. 2009. “Conceptualizing intercultural competence.” In The SAGE handbook of intercultural competence, edited by D. K. Deardorff, 2–52. Thousand Oaks, CA: Sage.
Tabachnick, B. G., and L. S. Fidell. 2001. Using multivariate statistics. Needham Heights, MA: Allyn & Bacon.
Thorndike, R. L., and E. Hagen. 1997. Measurement and evaluation in psychology and education. 6th ed. Upper Saddle River, NJ: Prentice-Hall.
Trevelyan, J. 2007. “Technical coordination in engineering practice.” J. Eng. Educ. 96 (3): 191–204. https://doi.org/10.1002/j.2168-9830.2007.tb00929.x.
Woo, S. E., O. S. Chernyshenko, A. Longley, Z. X. Zhang, C. Y. Chiu, and S. E. Stark. 2014. “Openness to experience: Its lower level structure, measurement, and cross-cultural equivalence.” J. Personality Assess. 96 (1): 29–45. https://doi.org/10.1080/00223891.2013.806328.
Woolley, M. E., G. L. Bowen, and N. K. Bowen. 2006. “The development and evaluation of procedures to assess child self-report item validity.” Educ. Psychol. Meas. 66 (4): 687–700. https://doi.org/10.1177/0013164405282467.
Yates, J. K. 2007. “Globalization of engineering and construction legal issues.” J. Civ. Eng. Educ. 133 (3): 169–170. https://doi.org/10.1061/(ASCE)1052-3928(2007)133:3(169).
Information & Authors
Information
Published In
Journal of Civil Engineering Education
Volume 146 • Issue 2 • April 2020
Copyright
©2019 American Society of Civil Engineers.
History
Received: Oct 19, 2018
Accepted: Sep 10, 2019
Published online: Nov 21, 2019
Published in print: Apr 1, 2020
Discussion open until: Apr 21, 2020
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.