Design Assessment in Virtual and Mixed Reality Environments: Comparison of Novices and Experts
Publication: Journal of Construction Engineering and Management
Volume 145, Issue 9
Abstract
The construction industry is facing a severe shortage of skilled workforce. Higher education is challenged to develop innovative strategies to help college students develop career-specific competency and accelerate the transition from novice to expert. Technology innovations such as virtual reality (VR) and mixed reality (MR) have been successfully integrated into learning and training programs to create authentic learning experiences within simulated virtual learning environments to facilitate tacit knowledge acquisition and workplace expertise development, which traditionally takes years of empirical experience and apprenticeship training. This study aims to explore potential VR and MR interventions in construction education and workforce development. It is directed at enhancing understanding of key differences between novices and experts and how VR and MR may facilitate tacit knowledge acquisition and expertise development to address the current skills gap in the construction industry. A simulation of accessibility design review and assessment for a tiny house was conducted via VR and MR mock-ups with the participation of both student novices and professional experts to collect behavioral and perceptual data using instruments that included a think-aloud protocol, a pair of pre- and postsurvey questionnaires, and audio/video recordings. Comparative analyses were conducted, and the results indicated that student novices, despite their lack of expertise, demonstrated comparable patterns of behaviors and achieved design review outcomes similar to those of professional experts with the VR and MR mock-ups. The findings of this study contribute to the body of knowledge by providing preliminary evidence of learning affordances of VR and MR in bridging experience-related gaps and suggesting opportunities for accelerating workplace expertise development among college students via technology intervention. These findings also have the potential to inform instructional design and pedagogical approaches that integrate VR and MR technology in undergraduate construction and engineering curricula.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Date generated or analyzed during the study are available from the corresponding author by request. Information about the Journal’s data-sharing policy can be found here: http://ascelibrary.org/doi/10.1061/(ASCE)CO.1943-7862.0001263.
Acknowledgments
This material is based on work supported by the National Science Foundation under Awards IIS 1735804 and 1735878.
References
Akçayır, M., and G. Akçayır. 2017. “Advantages and challenges associated with augmented reality for education: A systematic review of the literature.” Educ. Res. Rev. 20 (Feb): 1–11. https://doi.org/10.1016/j.edurev.2016.11.002.
Allen, I. E., and C. A. Seaman. 2007. “Likert scales and data analyses.” Qual. Prog. 40 (7): 64–65.
Atman, C. J., R. S. Adams, M. E. Cardella, J. Turns, S. Mosborg, and J. Saleem. 2007. “Engineering design processes: A comparison of students and expert practitioners.” J. Eng. Educ. 96 (4): 359–379. https://doi.org/10.1002/j.2168-9830.2007.tb00945.x.
Ball, L. J., T. C. Ormerod, and N. J. Morley. 2004. “Spontaneous analogising in engineering design: A comparative analysis of experts and novices.” Des. Stud. 25 (5): 495–508. https://doi.org/10.1016/j.destud.2004.05.004.
Bandura, A. 1977. “Self-efficacy: Toward a unifying theory of behavioral change.” Psychol. Rev. 84 (2): 191–215. https://doi.org/10.1037/0033-295X.84.2.191.
Bassanino, M., T. Fernando, and K.-C. Wu. 2013. “Can virtual workspaces enhance team communication and collaboration in design review meetings?” Archit. Eng. Des. Manage. 10 (3–4): 200–217. https://doi.org/10.1080/17452007.2013.775102.
Becerik-Gerber, B., K. H. Ku, and F. Jazizadeh. 2012. “BIM-enabled virtual and collaborative construction engineering and management.” J. Prof. Issues Eng. Educ. Pract. 138 (3): 234–245. https://doi.org/10.1061/(ASCE)EI.1943-5541.0000098.
Bouchlaghem, D., H. Shang, J. Whyte, and A. Ganah. 2005. “Visualisation in architecture, engineering and construction (AEC).” Autom. Constr. 14 (3): 287–295. https://doi.org/10.1016/j.autcon.2004.08.012.
Bransford, J. D., A. L. Brown, and R. R. Cocking. 2000. How people learn: Brain, mind, experience, and school: Expanded edition. Washington, DC: National Academies Press.
Brogan, D. R., and M. H. Kutner. 2012. “Comparative analyses of pretest-posttest research designs.” Am. Statistician 34 (4): 229–232. https://doi.org/10.2307/2684066.
Brooke, J. 1996. “SUS-A quick and dirty usability scale.” In Usability evaluation in industry, edited by P. W. Jordan, B. Thomas, B. Weerdmeester, and I. L. McClelland, 189–194. Bristol, PA: Taylor & Francis.
Brown, A., M. S. Donovan, and J. W. Pellegrino. 2000. “How experts differ from novices.” In How people learn: Brain, mind, experience, and school: Expanded edition, edited by J. D. Bransford, A. L. Brown, and R. R. Cocking, 31–50. Washington, DC: National Academies Press.
Brunhaver, S. R., R. F. Korte, S. R. Barley, and S. D. Sheppard. 2017. “Bridging the gaps between engineering education and practice.” In US engineering in the global economy, edited by R. B. Freeman, and H. Salzman. Cambridge, MA: MIT Press.
Chalhoub, J., and S. K. Ayer. 2018. “Using mixed reality for electrical construction design communication.” Autom. Constr. 86 (Feb): 1–10. https://doi.org/10.1016/j.autcon.2017.10.028.
Cheng, K.-H., and C.-C. Tsai. 2013. “Affordances of augmented reality in science learning: Suggestions for future research.” J. Sci. Educ. Technol. 22 (4): 449–462. https://doi.org/10.1007/s10956-012-9405-9.
Chi, H.-L., S.-C. Kang, and X. Wang. 2013. “Research trends and opportunities of augmented reality applications in architecture, engineering, and construction.” Autom. Constr. 33 (Aug): 116–122. https://doi.org/10.1016/j.autcon.2012.12.017.
Chi, M. T. H., R. Glaser, and M. J. Farr. 1988. The nature of expertise. Hillsdale, NJ: Lawrence Erlbaum Associates.
Cross, N. 2004. “Expertise in design: An overview.” Des. Stud. 25 (5): 427–441. https://doi.org/10.1016/j.destud.2004.06.002.
Daily, M., M. Howard, J. Jerald, C. Lee, K. Martin, D. McInnes, and P. Tinker. 2000. “Distributed design review in virtual environments.” In Proc., CVE 2000, 57–63. New York: ACM.
Dainty, A. R. J., S. G. Ison, and D. S. Root. 2004. “Bridging the skills gap: A regionally driven strategy for resolving the construction labour market crisis.” Eng. Constr. Archit. Manage. 11 (4): 275–283. https://doi.org/10.1108/09699980410547621.
Du, J., Z. Zou, Y. Shi, and D. Zhao. 2018. “Zero latency: Real-time synchronization of BIM data in virtual reality for collaborative decision-making.” Autom. Constr. 85 (Jan): 51–64. https://doi.org/10.1016/j.autcon.2017.10.009.
Dugard, P., and J. Todman. 1995. “Analysis of pre-test-post-test control group designs in educational research.” Educ. Psychol. 15 (2): 181–198. https://doi.org/10.1080/0144341950150207.
Dunleavy, M., C. Dede, and R. Mitchell. 2009. “Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning.” J. Sci. Educ. Technol. 18 (1): 7–22. https://doi.org/10.1007/s10956-008-9119-1.
Dunston, P. S., L. L. Arns, and J. D. McGlothlin. 2010. “Virtual reality mock-ups for healthcare facility design and a model for technology hub collaboration.” J. Build. Perform. Simul. 3 (3): 185–195. https://doi.org/10.1080/19401490903580742.
Dunston, P. S., L. L. Arns, J. D. McGlothlin, G. C. Lasker, and A. G. Kushner. 2011. “An immersive virtual reality mock-up for design review of hospital patient rooms.” In Collaborative design in virtual environments, edited by X. Wang, and J. J.-H. Tsai, 167–176. Dordrecht, Netherlands: Springer.
Emerson, D. 2018. “Front line: Construction industry faces a labor shortage.” In Area development online. Minneapolis: Halcyon Business Publications.
Ericsson, K. A. 2009. “Enhancing the development of professional performance: Implications from the study of deliberate practice.” In Development of professional expertise: Toward measurement of expert performance and design of optimal learning environments, edited by K. A. Ericsson, 405–431. New York: Cambridge University Press.
Ericsson, K. A., R. T. Krampe, and C. Tesch-Römer. 1993. “The role of deliberate practice in the acquisition of expert performance.” Psychol. Rev. 100 (3): 363–406. https://doi.org/10.1037/0033-295X.100.3.363.
Friard, O., and M. Gamba. 2016. “BORIS: A free, versatile open-source event-logging software for video/audio coding and live observations.” Methods Ecol. Evol. 7 (11): 1325–1330. https://doi.org/10.1111/2041-210X.12584.
Ibáñez, M.-B., and C. Delgado-Kloos. 2018. “Augmented reality for STEM learning: A systematic review.” Comput. Educ. 123 (Aug): 109–123. https://doi.org/10.1016/j.compedu.2018.05.002.
Joe, J., S. Chaudhuri, T. Le, H. Thompson, and G. Demiris. 2015. “The use of think-aloud and instant data analysis in evaluation research: Exemplar and lessons learned.” J. Biomed. Inform. 56 (Aug): 284–291. https://doi.org/10.1016/j.jbi.2015.06.001.
Kassem, M., L. Benomran, and J. Teizer. 2017. “Virtual environments for safety learning in construction and engineering: Seeking evidence and identifying gaps for future research.” Visual. Eng. 5 (16): 1–15. https://doi.org/10.1186/s40327-017-0054-1.
Ke, F., S. Lee, and X. Xu. 2016. “Teaching training in a mixed-reality integrated learning environment.” Comput. Hum. Behav. 62 (Sep): 212–220. https://doi.org/10.1016/j.chb.2016.03.094.
Kim, T. K. 2015. “T test as a parametric statistic.” Korean J. Anesthesiol. 68 (6): 540–546. https://doi.org/10.4097/kjae.2015.68.6.540.
Kumar, S., M. Hedrick, C. Wiacek, and J. I. Messner. 2011. “Developing an experienced-based design review application for healthcare facilities using a 3D game engine.” J. Inf. Technol. Constr. 16: 85–104.
Kwon, O.-S., C.-S. Park, and C.-R. Lim. 2014. “A defect management system for reinforced concrete work utilizing BIM, image-matching and augmented reality.” Autom. Constr. 46 (Oct): 74–81. https://doi.org/10.1016/j.autcon.2014.05.005.
Lee, K. 2012. “Augmented reality in education and training.” TechTrends 56 (2): 13–21. https://doi.org/10.1007/s11528-012-0559-3.
Li, X., W. Yi, H.-L. Chi, X. Wang, and A. P. C. Chan. 2018. “A critical review of virtual and augmented reality (VR/AR) applications in construction safety.” Autom. Constr. 86 (Feb): 150–162. https://doi.org/10.1016/j.autcon.2017.11.003.
Litzinger, T., L. R. Lattuca, R. Hadgraft, and W. Newstetter. 2011. “Engineering education and the development of expertise.” J. Eng. Educ. 100 (1): 123–150. https://doi.org/10.1002/j.2168-9830.2011.tb00006.x.
Livorsi, E., M. Kennedy, and A. Patron. 2017. Talent development in the construction industry–2017 FMI industry survey. Raaleigh, NC: FMI Corporation.
MacFarland, T. W., and J. M. Yates. 2016. “Mann–Whitney U test.” In Introduction to nonparametric statistics for the biological sciences using R, 103–132. Cham, Switzerland: Springer.
Martín-Gutiérrez, J., P. Fabiani, W. Benesova, M. D. Meneses, and C. E. Mora. 2015. “Augmented reality to promote collaborative and autonomous learning in higher education.” Comput. Hum. Behav. 51 (Oct): 752–761. https://doi.org/10.1016/j.chb.2014.11.093.
Mayer, R. E. 2001. “Changing conceptions of learning: A century of progress in the scientific study of education.” In Education across the century: The centennial volume, edited by L. Corno, 35–75. Chicago: University of Chicago Press.
Mayer, R. E. 2005. “Introduction to multimedia learning.” In The Cambridge handbook of multimedia learning, edited by R. E. Mayer, 1–18. New York: Cambridge University Press.
Mayer, R. E. 2009. Multimedia learning. Cambridge, NY: Cambridge University Press.
McGraw-Hill Construction. 2012. Construction industry workforce shortages: Role of certification, training and green jobs in filling the gaps. Bedford, MA: McGraw-Hill.
Milgram, P., and H. Colquhoun. 1999. “A taxonomy of real and virtual world display integration.” In Mixed reality—Merging real and virtual worlds, edited by Y. Ohta, and H. Tamura, 1–26. Berlin: Springer.
Nightingale, P. 2009. “Tacit knowledge and engineering design.” In Philosophy of technology and engineering sciences, edited by A. Meijers, 351–374. Burlington, MA: Elsevier.
Oliver, R., and J. Herrington. 2010. “Exploring technology-mediated learning from a pedagogical perspective.” Interact. Learn. Environ. 11 (2): 111–126. https://doi.org/10.1076/ilee.11.2.111.14136.
Pan, Z., A. D. Cheok, H. Yang, J. Zhu, and J. Shi. 2006. “Virtual reality and mixed reality for virtual learning environments.” Comput. Graphics 30 (1): 20–28. https://doi.org/10.1016/j.cag.2005.10.004.
Park, C.-S., D.-Y. Lee, O.-S. Kwon, and X. Wang. 2013. “A framework for proactive construction defect management using BIM, augmented reality and ontology-based data collection template.” Autom. Constr. 33 (Aug): 61–71. https://doi.org/10.1016/j.autcon.2012.09.010.
Persky, A. M., and J. D. Robinson. 2017. “Moving from novice to expertise and its implications for instruction.” Am. J. Pharm. Educ. 81 (9): 6065. https://doi.org/10.5688/ajpe6065.
Quarles, J., S. Lampotang, I. Fischler, P. Fishwick, and B. Lok. 2009. “Scaffolded learning with mixed reality.” Comput. Graphics 33 (1): 34–46. https://doi.org/10.1016/j.cag.2008.11.005.
Rankohi, S., and L. Waugh. 2013. “Review and analysis of augmented reality literature for construction industry.” Visual. Eng. 1 (9): 1–18. https://doi.org/10.1186/2213-7459-1-9.
Reed, S. 2016. “The talent shortage: Will the training the AEC industry desperately needs arrive in time?” Accessed July 21, 2018. https://www.bdcnetwork.com/blog/talent-shortage-will-training-aec-industry-desperately-needs-arrive-time.
Rosner, B., R. J. Glynn, and M. L. Lee. 2006. “The Wilcoxon signed rank test for paired comparisons of clustered data.” Biometrics 62 (1): 185–192. https://doi.org/10.1111/j.1541-0420.2005.00389.x.
Ross, K. G., J. K. Phillips, G. Klein, and J. Cohn. 2005. “Creating expertise: A framework to guide simulation-based training.” In Proc., Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2005, 1–12. Orlando, FL: National Defense Industrial Association.
Ruge, G., and C. McCormack. 2017. “Building and construction students’ skills development for employability–Reframing assessment for learning in discipline-specific contexts.” Archit. Eng. Des. Manage. 13 (5): 365–383. https://doi.org/10.1080/17452007.2017.1328351.
Sacks, R., A. Perlman, and R. Barak. 2013. “Construction safety training using immersive virtual reality.” Constr. Manage. Econ. 31 (9): 1005–1017. https://doi.org/10.1080/01446193.2013.828844.
Sacks, R., and E. Pikas. 2013. “Building information modeling education for construction engineering and management. I: Industry requirements, state of the art, and gap analysis.” J. Constr. Eng. Manage. 139 (11): 04013016. https://doi.org/10.1061/(ASCE)CO.1943-7862.0000759.
Sheppard, S. D., K. Macatangay, A. Colby, and W. M. Sullivan. 2008. Educating engineers: Designing for the future of the field. San Francisco: Jossey-Bass.
Shin, D.-H. 2017. “The role of affordance in the experience of virtual reality learning: Technological and affective affordances in virtual reality.” Telematics Inf. 34 (8): 1826–1836. https://doi.org/10.1016/j.tele.2017.05.013.
Soibelman, L., L. Y. Liu, J. G. Kirby, E. W. East, C. H. Caldas, and K.-Y. Lin. 2003. “Design review checking system with corporate lessons learned.” J. Constr. Eng. Manage. 129 (5): 475–484. https://doi.org/10.1061/(ASCE)0733-9364(2003)129:5(475).
Sullivan, W. M., and M. S. Rosin. 2008. A new agenda for higher education: Shaping a life of the mind for practice. San Francisco: Jossey-Bass.
Wang, P., P. Wu, J. Wang, H. L. Chi, and X. Wang. 2018. “A critical review of the use of virtual reality in construction engineering education and training.” Int. J. Environ. Res. Public Health 15 (6): E1204. https://doi.org/10.3390/ijerph15061204.
Wang, X. 2009. “Augmented reality in architecture and design: Potentials and challenges for application.” Int. J. Archit. Comput. 7 (2): 309–326. https://doi.org/10.1260/147807709788921985.
Wang, X., and P. S. Dunston. 2013. “Tangible mixed reality for remote design review: A study understanding user perception and acceptance.” Visual. Eng. 1 (8): 1–15. https://doi.org/10.1186/2213-7459-1-8.
Wang, X., and M. A. Schnabel. 2009. Mixed reality in architecture, design and construction. Sydney: Springer.
Wenger, E. 1998. Communities of practice: Learning, meaning, and identity. Cambridge, UK: Cambridge University Press.
Whyte, J., and D. Nikolić. 2018. Virtual reality and the built environment. London: Routledge.
Wu, H.-K., S. W.-Y. Lee, H.-Y. Chang, and J.-C. Liang. 2013. “Current status, opportunities and challenges of augmented reality in education.” Comput. Educ. 62 (Mar): 41–49. https://doi.org/10.1016/j.compedu.2012.10.024.
Wu, P., Y. Feng, J. Pienaar, and Y. Zhong. 2015. “Educational attainment and job requirements: Exploring the gaps for construction graduates in Australia from an industry point of view.” J. Prof. Issues Eng. Educ. Pract. 141 (4): 06015001. https://doi.org/10.1061/(ASCE)EI.1943-5541.0000245.
Wu, W., and R. R. A. Issa. 2014. “BIM education and recruiting: Survey-based comparative analysis of issues, perceptions, and collaboration opportunities.” J. Prof. Issues Eng. Educ. Pract. 140 (2): 04013014. https://doi.org/10.1061/(ASCE)EI.1943-5541.0000186.
Yazici, B., and S. Yolacan. 2007. “A comparison of various tests of normality.” J. Stat. Comput. Simul. 77 (2): 175–183. https://doi.org/10.1080/10629360600678310.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 145 • Issue 9 • September 2019
Copyright
©2019 American Society of Civil Engineers.
History
Received: Oct 16, 2018
Accepted: Jan 22, 2019
Published online: Jun 28, 2019
Published in print: Sep 1, 2019
Discussion open until: Nov 28, 2019
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.