Efficacy of Annotated Video-Based Learning Environment for Drawing Students’ Attention to Construction Practice Concepts
Publication: Journal of Construction Engineering and Management
Volume 150, Issue 1
Abstract
To enhance students’ learning of construction practice, instructors use videos as multimedia pedagogical tools to bring practical experience into the classroom. However, cognitive load levels, individual differences, and multimedia design principles are important considerations in the effective use of videos for instruction. Therefore, this study investigates the effectiveness of an artificial intelligence (AI)–annotated video in guiding students’ attention to important construction practice concepts. Students were exposed to both annotated and unannotated videos illustrating construction safety practices. Eye tracking metrics and self-reported cognitive load were collected as students interacted with both learning environments. The effectiveness of the videos in drawing students’ attention were compared. The AI-annotated video was considered effective and the variations across individual differences were reported. No significant difference was observed in the cognitive loads of both learning environments. This study provides an understanding of the extent to which learners of different demographic characteristics allocate attention to signaled practice concepts. This study also illustrates the impact of the signaled concepts on learners’ cognitive loads. This study contributes to existing theories by elucidating how practice knowledge could be adapted to meet construction engineering students’ learning needs.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
All data, models, or code that support the findings of this study are available from the corresponding author upon reasonable request.
References
Abdulrahaman, M. D., N. Faruk, A. A. Oloyede, N. T. Surajudeen-Bakinde, L. A. Olawoyin, O. V. Mejabi, Y. O. Imam-Fulani, A. O. Fahm, and A. L. Azeez. 2020. “Multimedia tools in the teaching and learning processes: A systematic review.” Heliyon 6 (11): e05312. https://doi.org/10.1016/j.heliyon.2020.e05312.
Abudayyeh, O., J. Russell, D. Johnston, and J. Rowings. 2000. “Construction engineering and management undergraduate education.” J. Constr. Eng. Manage. 126 (3): 169–175. https://doi.org/10.1061/(ASCE)0733-9364(2000)126:3(169).
Ahn, Y. H., R. P. Annie, and H. Kwon. 2012. “Key competencies for U.S. construction graduates: Industry perspective.” J. Civ. Eng. Educ. 138 (2): 123–130. https://doi.org/10.1061/(ASCE)EI.1943-5541.0000089.
Alemdag, E., and K. Cagiltay. 2018. “A systematic review of eye tracking research on multimedia learning.” Comput. Educ. 125 (Oct): 413–428. https://doi.org/10.1016/j.compedu.2018.06.023.
AlMunifi, A., and A. Y. Aleryani. 2019. “Knowledge and skills level of graduate civil engineers employers and graduates’ perceptions.” Int. J. Eng. Pedagogy 9 (1): 84–101. https://doi.org/10.3991/ijep.v9i1.9744.
Alpizar, D., O. O. Adesope, and R. M. Wong. 2020. “A meta-analysis of signaling principle in multimedia learning environments.” Educ. Technol. Res. Dev. 68 (Oct): 2095–2119. https://doi.org/10.1007/s11423-020-09748-7.
Altan, T., and K. Cagiltay. 2022. “An eye tracking based investigation of multimedia learning design in science education textbooks.” Educ. Technol. Soc. 25 (2): 48–61.
Anmarkrud, Ø., A. Andresen, and I. Bråten. 2019. “Cognitive load and working memory in multimedia learning: Conceptual and measurement issues.” Educ. Psychologist 54 (2): 61–83. https://doi.org/10.1080/00461520.2018.1554484.
Austin, K. A. 2009. “Multimedia learning: Cognitive individual differences and display design techniques predict transfer learning with multimedia learning modules.” Comput. Educ. 53 (4): 1339–1354. https://doi.org/10.1016/j.compedu.2009.06.017.
Bahari, A. 2023. “Challenges and affordances of cognitive load management in technology-assisted language learning: A systematic review.” Int. J. Hum.-Comput. Interact. 39 (1): 85–100. https://doi.org/10.1080/10447318.2021.2019957.
Bajrami, L., and M. Ismaili. 2016. “The role of video materials in EFL classrooms.” Procedia-Social Behav. Sci. 232 (Mar): 502–506. https://doi.org/10.1016/j.sbspro.2016.10.068.
Becker, M. W., R. Alzahabi, and S. Jelinek. 2011. “Effective attentional filtering by the union of two distinct colors: Eye-tracking evidence.” J. Vision 11 (11): 1328. https://doi.org/10.1167/11.11.1328.
Brame, C. J. 2016. “Effective educational videos: Principles and guidelines for maximizing student learning from video content.” CBE—Life Sci. Educ. 15 (4): es6. https://doi.org/10.1187/cbe.16-03-0125.
Carter, B. T., and S. G. Luke. 2020. “Best practices in eye tracking research.” Int. J. Psychophysiol. 155 (Sep): 49–62. https://doi.org/10.1016/j.ijpsycho.2020.05.010.
Castro-Alonso, J. C., M. Wong, O. O. Adesope, P. Ayres, and F. Paas. 2019. “Gender imbalance in instructional dynamic versus static visualizations: A meta-analysis.” Educ. Psychol. Rev. 31 (Jun): 361–387. https://doi.org/10.1007/s10648-019-09469-1.
Çeken, B., and N. Taşkın. 2022. “Multimedia learning principles in different learning environments: A systematic review.” Smart Learn. Environ. 9 (1): 1–22. https://doi.org/10.1186/s40561-022-00200-2.
Cheng, B., Y. Wei, H. Shi, R. Feris, J. Xiong, and T. Huang. 2018. “Revisiting RCNN: On awakening the classification power of faster RCNN.” In Proc., Computer Vision—European Conf. on Computer Vision (ECCV)—ECCV 2018, Munich, Germany, edited by V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, 473–490. Berlin: Springer. https://doi.org/10.1007/978-3-030-01267-0_28.
Chun, D. M., and J. L. Plass. 1996. “Effects of multimedia annotations on vocabulary acquisition.” Mod. Lang. J. 80 (2): 183–198. https://doi.org/10.1111/j.1540-4781.1996.tb01159.x.
Clark, J. M., and A. Paivio. 1991. “Dual coding theory and education.” Educ. Psychol. Rev. 3 (Sep): 149–210. https://doi.org/10.1007/BF01320076.
Delacre, M., D. Lakens, and C. Leys. 2017. “Why psychologists should by default use Welch’s t-test instead of Student’s t-test.” Int. Rev. Social Psychol. 30 (1): 92–101. https://doi.org/10.5334/irsp.82.
De Winter, J. C. 2019. “Using the Student’s t-test with extremely small sample sizes.” Pract. Assess. Res. Eval. 18 (1): 1–12. https://doi.org/10.7275/e4r6-dj05.
Feldon, D. F., J. Franco, J. Chao, J. Peugh, and C. Maahs-Fladung. 2018. “Self-efficacy change associated with a cognitive load-based intervention in an undergraduate biology course.” Learn. Instr. 56 (Aug): 64–72. https://doi.org/10.1016/j.learninstruc.2018.04.007.
Ferreira, C., M. Baptista, and A. Arroio. 2013. “Teachers’ pedagogical strategies for integrating multimedia tools in science teaching.” J. Balt. Sci. Educ. 12 (4): 509. https://doi.org/10.33225/jbse/13.12.509.
Grimley, M. 2007. “Learning from multimedia materials: The relative impact of individual differences.” Educ. Psychol. 27 (4): 465–485. https://doi.org/10.1080/01443410601159795.
Guo, P. J., J. Kim, and R. Rubin. 2015. “How video production affects student engagement: An empirical study of MOOC videos.” In Proc., 1st ACM Conf. on Learning@scale Conf. 2014. New York: Association for Computing Machinery Press.
Heo, M., and N. Toomey. 2020. “Learning with multimedia: The effects of gender, type of multimedia learning resources, and spatial ability.” Comput. Educ. 146 (Mar): 103747. https://doi.org/10.1016/j.compedu.2019.103747.
Homer, B. D., J. L. Plass, and L. Blake. 2008. “The effects of video on cognitive load and social presence in multimedia-learning.” Comput. Hum. Behav. 24 (3): 786–797. https://doi.org/10.1016/j.chb.2007.02.009.
Horsley, M., M. Eliot, B. A. Knight, and R. Reilly. 2013. Current trends in eye tracking research. New York: Springer.
Ibrahim, M. 2012. “Implications of designing instructional video using cognitive theory of multimedia learning.” Crit. Questions Educ. 3 (2): 83–104.
Ibrahim, M., P. D. Antonenko, C. M. Greenwood, and D. Wheeler. 2012. “Effects of segmenting, signalling, and weeding on learning from educational video.” Learn. Media Technol. 37 (3): 220–235. https://doi.org/10.1080/17439884.2011.585993.
Ivory, M. Y., and M. A. Hearst. 2001. “The state of the art in automating usability evaluation of user interfaces.” ACM Comput. Surv. 33 (4): 470–516. https://doi.org/10.1145/503112.503114.
Jamet, E. 2014. “An eye-tracking study of cueing effects in multimedia learning.” Comput. Hum. Behav. 32 (Mar): 47–53. https://doi.org/10.1016/j.chb.2013.11.013.
Jevsikova, T., A. Berniukevičius, and E. Kurilovas. 2017. “Application of resource description framework to personalise learning: Systematic review and methodology.” Inf. Educ. 16 (1): 61–82. https://doi.org/10.15388/infedu.2017.04.
Kalyuga, S. 2013. “Effects of learner prior knowledge and working memory limitations on multimedia learning.” Procedia-Social Behav. Sci. 83 (Jul): 25–29. https://doi.org/10.1016/j.sbspro.2013.06.005.
Katona, J. 2022. “Measuring cognition load using eye-tracking parameters based on algorithm description tools.” Sensors 22 (3): 912. https://doi.org/10.3390/s22030912.
Kelley, K., B. Clark, V. Brown, and J. Sitzia. 2003. “Good practice in the conduct and reporting of survey research.” Int. J. Qual. Health Care 15 (3): 261–266. https://doi.org/10.1093/intqhc/mzg031.
Krebs, M.-C., A. Schüler, and K. Scheiter. 2021. “Do prior knowledge, model-observer similarity and social comparison influence the effectiveness of eye movement modeling examples for supporting multimedia learning?” Instr. Sci. 49 (5): 607–635. https://doi.org/10.1007/s11251-021-09552-7.
Latifzadeh, K., S. H. Amiri, A. Bosaghzadeh, M. Rahimi, and R. Ebrahimpour. 2020. “Evaluating cognitive load of multimedia learning by eye-tracking data analysis.” Technol. Educ. J. 15 (1): 33–50. https://doi.org/10.22061/tej.2020.6237.2368.
Law, E. L.-C., E. E. Mattheiss, M. D. Kickmeier-Rust, and D. Albert. 2010. Vicarious learning with a digital educational game: Eye-tracking and survey-based evaluation approaches. 471–488. New York: Springer.
Le, Q. T., A. Pedro, and C. S. Park. 2015. “A social virtual reality based construction safety education system for experiential learning.” J. Intell. Rob. Syst. 79 (Aug): 487–506. https://doi.org/10.1007/s10846-014-0112-z.
Lunsford, K. J. 2017. Challenges to implementing differentiated instruction in middle school classrooms with mixed skill levels. Minneapolis: Walden Univ.
MacHardy, Z., and Z. A. Pardos. 2015. “Evaluating the relevance of educational videos using BKT and big data.” In Proc., 8th Int. Conf. on Educational Data Mining, Madrid, Spain, 424–427. Boston, MA: International Educational Data Mining Society.
Martin-Gonzalez, A., A. Chi-Poot, and V. Uc-Cetina. 2016. “Usability evaluation of an augmented reality system for teaching Euclidean vectors.” Innovations Educ. Teach. Int. 53 (6): 627–636. https://doi.org/10.1080/14703297.2015.1108856.
Mason, L., M. C. Tornatora, and P. Pluchino. 2013. “Do fourth graders integrate text and picture in processing and learning from an illustrated science text? Evidence from eye-movement patterns.” Comput. Educ. 60 (1): 95–109. https://doi.org/10.1016/j.compedu.2012.07.011.
Mayer, R. 2009. “Multimedia learning.” Psychol. Learn. Motivation 41 (Feb): 85–139. https://doi.org/10.1017/CBO9780511811678.
Mayer, R. E. 2021. “Evidence-based principles for how to design effective instructional videos.” J. Appl. Res. Mem. Cognit. 10 (2): 229–240. https://doi.org/10.1016/j.jarmac.2021.03.007.
Mayer, R. E., and L. Fiorella. 2014. “Principles for reducing extraneous processing in multimedia learning: Coherence, signaling, redundancy, spatial contiguity, and temporal contiguity principles.” In The Cambridge handbook of multimedia learning, edited by R. E. Mayer, 279–315. Cambridge, UK: Cambridge University Press. https://doi.org/10.1017/CBO9781139547369.015.
Mayer, R. E., and R. Moreno. 2003. “Nine ways to reduce cognitive load in multimedia learning.” Educ. Psychologist 38 (1): 43–52. https://doi.org/10.1207/S15326985EP3801_6.
Milin, P., D. Divjak, and R. H. Baayen. 2017. “A learning perspective on individual differences in skilled reading: Exploring and exploiting orthographic and semantic discrimination cues.” J. Exp. Psychol. 43 (11): 1730. https://doi.org/10.1037/xlm0000410.
Molina, A. I., Ó. Navarro, M. Ortega, and M. Lacruz. 2018. “Evaluating multimedia learning materials in primary education using eye tracking.” Comput. Stand. Interfaces 59 (Aug): 45–60. https://doi.org/10.1016/j.csi.2018.02.004.
Moreno, R. 2002. “Who learns best with multiple representations? Cognitive theory implications for individual differences in multimedia learning.” In Proc., ED-MEDIA 2002–World Conf. on Educational Multimedia, Hypermedia & Telecommunications, edited by P. Barker and S. Rebelsky, 1380–1385. Denver: Association for the Advancement of Computing in Education.
Moreno, R., and T. Flowerday. 2006. “Students’ choice of animated pedagogical agents in science learning: A test of the similarity-attraction hypothesis on gender and ethnicity.” Contemp. Educ. Psychol. 31 (2): 186–207. https://doi.org/10.1016/j.cedpsych.2005.05.002.
Moreno, R., and R. E. Mayer. 1999. “Multimedia-supported metaphors for meaning making in mathematics.” Cognit. Instr. 17 (3): 215–248. https://doi.org/10.1207/S1532690XCI1703_1.
Mukherjee, S. 2018. “Role of multimedia in education.” Edelweiss Appl. Sci. Technol. 2 (1): 245–247. https://doi.org/10.33805/2576-8484.149.
Mutlu-Bayraktar, D., V. Cosgun, and T. Altan. 2019. “Cognitive load in multimedia learning environments: A systematic review.” Comput. Educ. 141 (Nov): 103618. https://doi.org/10.1016/j.compedu.2019.103618.
Nath, N. D., A. H. Behzadan, and S. G. Paal. 2020. “Deep learning for site safety: Real-time detection of personal protective equipment.” Autom. Constr. 112 (Apr): 103085. https://doi.org/10.1016/j.autcon.2020.103085.
Navarro, O., A. I. Molina, M. Lacruz, and M. Ortega. 2015. “Evaluation of multimedia educational materials using eye tracking.” Procedia-Social Behav. Sci. 197 (Jul): 2236–2243. https://doi.org/10.1016/j.sbspro.2015.07.366.
Olayiwola, J., A. Akanmu, X. Gao, H. Murzi, and K. Afsari. 2023a. “Design and usability evaluation of an annotated video-based learning environment for construction engineering education.” J. Comput. Civ. Eng. 37 (6): 04023033. https://doi.org/10.1061/JCCEE5.CPENG-5206.
Olayiwola, J., A. O. Yusuf, A. A. Akanmu, H. Murzi, X. Gao, and K. Afsari. 2023b. “Construction practice knowledge for complementing classroom teaching during site visits.” Smart Sustainable Built Environ. https://doi.org/10.1108/SASBE-07-2022-0144.
Ozcelik, E., I. Arslan-Ari, and K. Cagiltay. 2010. “Why does signaling enhance multimedia learning? Evidence from eye movements.” Comput. Hum. Behav. 26 (1): 110–117. https://doi.org/10.1016/j.chb.2009.09.001.
Paas, F., P. W. Van Gerven, and H. K. Tabbers. 2005. “The cognitive aging principle in multimedia learning.” In The Cambridge handbook of multimedia learning, edited by R. E. Mayer, 339–351. Cambridge, UK: Cambridge University Press.
Pandian, V. P. S., and S. Suleri. 2020. “NASA-TLX web app: An online tool to analyse subjective workload.” Preprint, submitted January 27, 2020. https://arxiv.org/abs/2001.09963.
Park, B., L. Knörzer, J. L. Plass, and R. Brünken. 2015. “Emotional design and positive emotions in multimedia learning: An eyetracking study on the use of anthropomorphisms.” Comput. Educ. 86 (Aug): 30–42. https://doi.org/10.1016/j.compedu.2015.02.016.
Pham, H. C., N. Dao, A. Pedro, Q. T. Le, R. Hussain, S. Cho, and C. S. I. K. Park. 2018. “Virtual field trip for mobile construction safety education using 360-degree panoramic virtual reality.” Int. J. Eng. Educ. 34 (4): 1174–1191.
Quinn, D., E. Cioffi, S. Hill, M. Kor, A.-C. Longford, R. Moller, and P. Rathore. 2019. “Implementing work-integrated learning in online construction management courses.” J. Univ. Teach. Learn. Pract. 16 (1): 9. https://doi.org/10.53761/1.16.1.9.
Refat, N., H. Kassim, and M. A. Rahman. 2019. “A cognitive approach-based instructional design for managing cognitive load and improving learning outcome.” In Proc., 2020 Emerging Technology in Computing, Communication and Electronics (ETCCE), 1–6. New York: IEEE.
Renkl, A., and K. Scheiter. 2017. “Studying visual displays: How to instructionally support learning.” Educ. Psychol. Rev. 29 (3): 599–621. https://doi.org/10.1007/s10648-015-9340-4.
Rosner, B., R. J. Glynn, and M. L. Ting Lee. 2003. “Incorporation of clustering effects for the Wilcoxon rank sum test: A large-sample approach.” Biometrics 59 (4): 1089–1098. https://doi.org/10.1111/j.0006-341X.2003.00125.x.
Saha, S., and S. Halder. 2016. “He or she: Does gender affect various modes of instructional visual design?” J. Res. Women and Gender 7 (1): 47–58.
Sanchez, C. A., and J. Wiley. 2010. “Sex differences in science learning: Closing the gap through animations.” Learn. Individual Differ. 20 (3): 271–275. https://doi.org/10.1016/j.lindif.2010.01.003.
Shojaei, A., S. Rokooei, A. Mahdavian, L. Carson, and G. Ford. 2021. “Using immersive video technology for construction management content delivery: A pilot study.” J. Inf. Technol. Constr. 26 (Nov): 886–901. https://doi.org/10.36680/j.itcon.2021.047.
Star, J. R., K. H. Lynch, and N. Perova. 2011. “Using video to improve mathematics’ teachers’ abilities to attend to classroom features: A replication study.” In Mathematics teachers’ noticing: Seeing through teachers’ eyes, edited by M. G. Sherin, V. R. Jacobs, and R. A. Philipp, 117–133. New York: Routledge.
Stark, L., R. Brünken, and B. Park. 2018. “Emotional text design in multimedia learning: A mixed-methods study using eye tracking.” Comput. Educ. 120 (May): 185–196. https://doi.org/10.1016/j.compedu.2018.02.003.
Sweller, J. 2010. “Cognitive load theory: Recent theoretical advances.” In Cognitive load theory, edited by J. L. Plass, R. Moreno, and R. Brünken, 29–47. Cambridge, UK: Cambridge Univ. Press.
Teng, F. 2022. “Vocabulary learning through videos: Captions, advance-organizer strategy, and their combination.” Comput. Assist. Lang. Learn. 35 (3): 518–550. https://doi.org/10.1080/09588221.2020.1720253.
Wang, C.-C., and W.-C. Lee. 2020. “Evaluation of the normality assumption in meta-analyses.” Am. J. Epidemiol. 189 (3): 235–242. https://doi.org/10.1093/aje/kwz261.
Wang, C. H., W. J. Lo, and M. J. J. Wang. 2022. “Usability evaluation of augmented reality-based maintenance instruction system.” Hum. Factors Ergon. Manuf. Serv. Ind. 32 (3): 239–255. https://doi.org/10.1002/hfm.20942.
Wood, D., J. S. Bruner, and G. Ross. 1976. “The role of tutoring in problem solving.” J. Child Psychol. Psychiatry 17 (2): 89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x.
Yezierski, E. J., and J. P. Birk. 2006. “Misconceptions about the particulate nature of matter. Using animations to close the gender gap.” J. Chem. Educ. 83 (6): 954. https://doi.org/10.1021/ed083p954.
Zhang, H., Z. Kyaw, S.-F. Chang, and T.-S. Chua. 2006. Visual translation embedding network for visual relation detection.” In Proc., IEEE Conf. on Computer Vision and Pattern Recognition, 5532–5540. New York: IEEE.
Information & Authors
Information
Published In
Journal of Construction Engineering and Management
Volume 150 • Issue 1 • January 2024
Copyright
© 2023 American Society of Civil Engineers.
History
Received: Mar 16, 2023
Accepted: Sep 18, 2023
Published online: Nov 9, 2023
Published in print: Jan 1, 2024
Discussion open until: Apr 9, 2024
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.