Differences between Professionals and Students in Their Visual Attention on Multiple Representation Types While Solving an Open-Ended Engineering Design Problem
This article has a reply.
VIEW THE REPLYThis article has a reply.
VIEW THE REPLYPublication: Journal of Civil Engineering Education
Volume 147, Issue 3
Abstract
Students and professionals from a variety of domains have demonstrated different approaches to problem solving. These two populations have displayed differences when using and perceiving multiple representations of problem-solving tools. In the domain of transportation engineering, this difference has yet to be evaluated in detail. This study addresses that knowledge gap. We used a mixed-methods approach with measurements of eye movements for visual attention and reflective interviews to gather participants’ reported representation use and problem-solving assumptions with three taxonomies of representations (equations, graphs, and flowchart) because they solved an open-ended design problem. Visual attention (VA) was recorded with a head-mounted eye-tracking device. Reflective interviews were used as a self-reported depiction of overt VA on the selected representation, and to record assumptions made by each participant to solve the problem. Equations, graphs, and flowcharts received different magnitudes of statistically significant VA for both groups. Professionals had a significant difference in VA between the flowchart/equation and graph representations, whereas students showed a difference in all three categories. Professionals generally chose representations with higher complexity (use of different combinations of representations) than students, as reflected by their frequency of conjoining representation choices and their associated assumptions. Professionals commonly approached problem solving by documenting specific assumptions, whereas students approached the problem more generically. Efficient information extraction occurred for professionals, but it took more time for them to solve the problem than novices. Novices frequently utilized the flowchart. Differences in performance may be explained by professionals using conceptual frameworks, or rules for interpreting and navigating the problems that were potentially developed through exposure to the domain of knowledge and deep understanding of the subject matter.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or codes that support the findings of this study are available from the corresponding author upon reasonable request.
Acknowledgments
This material is based upon work supported by the National Science Foundation under Grant No. 1463769.
References
Abadi, M. G., S. Gestson, S. Brown, and D. S. Hurwitz. 2019. “Traffic signal phasing problem-solving rationales of professional engineers developed from eye-tracking and clinical interviews traffic signal phasing problem-solving rationales of professional engineers developed from eye-tracking and clinical interviews.” Transport Res. Rec. 2673 (4): 685–696. https://doi.org/10.1177/0361198119837506.
Ainsworth, S. 1999. “The functions of multiple representations.” Comput. Educ. 33 (2–3): 131–152. https://doi.org/10.1016/S0360-1315(99)00029-9.
Ali, M. 2015. “An overview of research on experts and novice problem solvers in physics.” Buletin Persatuan Pendidikan Sains dan Matematik Johor 25 (1): 70–75.
Blascheck, T., K. Kurzhals, M. Raschke, M. Burch, D. Weiskopf, and T. Ertl. 2014. “State-of-the-art of visualization for eye tracking data.” In Proc., Eurographics Conf. on Visualization (EuroVis), 1–20. Godlar, Germany: Eurographics Association.
Boshuizen, H. P. A., and H. G. Schmidt. 2008. “The development of clinical reasoning expertise.” In Clinical reasoning in the health professions, 113–121. New York: Elsevier.
Breslow, L. A., J. G. Trafton, and R. M. Ratwani. 2009. “A perceptual process approach to selecting color scales for complex visualizations.” J. Exp. Psychol. 15 (1): 25–34.
Brown, S., B. Lutz, N. Perova-Mello, and O. Ha. 2019. “Exploring differences in statics concept inventory scores among students and practitioners.” J. Eng. Educ. 108 (1): 119–135. https://doi.org/10.1002/jee.20246.
Bruder, C., and C. Hasse. 2019. “Differences between experts and novices in the monitoring of automated systems.” Int. J. Ind. Ergon. 72 (Apr): 1–11. https://doi.org/10.1016/j.ergon.2019.03.007.
Chandler, P., and J. Sweller. 1992. “The split-attention effect as a factor in the design of instruction.” Br. J. Educ. Psychol. 62 (2): 233–246. https://doi.org/10.1111/j.2044-8279.1992.tb01017.x.
Cheville, A. 2014. “Defining engineering education.” In Proc., 121st ASEE Annual Conf. and Exposition. Washington, DC: American Society for Engineering Education.
Convertino, G., J. Chen, B. Yost, S. Ryu, and C. North. 2003. “Exploring context switching and cognition in dual-view coordinated visualizations.” In Proc., Int. Conf. on Coordinated and Multiple Views in Exploratory Visualization, 55–62. New York: IEEE.
Corbetta, M., and G. L. Shulman. 2002. “Control of goal-directed and stimulus-driven attention in the brain.” Nat. Rev. Neurosci. 3 (3): 201–215. https://doi.org/10.1038/nrn755.
Cristino, L. 2011. Bahia inicia uso de inseto transgênico contra dengue. Abingdon, UK: Taylor & Francis.
Dupont, L. 2014. “Eye-tracking analysis in landscape perception research: Influence of photograph properties and landscape characteristics.” Landsc. Res. 39 (4): 417–432. https://doi.org/10.1080/01426397.2013.773966.
Earnest, J. 2006. “ABET engineering technology criteria and competency based engineering education.” In Proc., Frontiers in Education 35th Annual Conf., 7–12. New York: IEEE.
Eye Tracking Incorporation. 2013. We’re going to need a bigger sample. River, NJ: Prentice Hall.
Fasano, G., and A. Franceschini. 1987. “A multidimensional version of the Kolmogorov-Smirnov test.” Mon. Not. Roy. Astron. Soc. 225 (1): 155–170. https://doi.org/10.1093/mnras/225.1.155.
Fitts, P. M., R. E. Jones, and J. L. Milton. 1950. “Eye movements of aircraft pilots during instrument-landing approaches.” Aeronaut. Eng. Rev. 9 (2): 1–6.
Frintrop, S., E. Rome, and H. I. Christensen. 2010. “Computational visual attention systems and their cognitive foundations.” ACM Trans. Appl. Percept. 7 (1): 1–39. https://doi.org/10.1145/1658349.1658355.
Galesic, M., R. Tourangeau, M. P. Couper, and F. G. Conrad. 2008. “Eye-tracking data new insights on response order effects and other cognitive shortcuts in survey responding.” Publ. Opin. Q. 72 (5): 892–913. https://doi.org/10.1093/poq/nfn059.
Gegenfurtner, A., E. Lehtinen, and R. Säljö. 2011. “Expertise differences in the comprehension of visualizations : A meta-analysis of eye-tracking research in professional domains.” Educ. Psychol. Rev. 23 (4): 523–552. https://doi.org/10.1007%2Fs10648-011-9174-7.
Guan, Z., S. Lee, E. Cuddihy, and J. Ramey. 2006. “The validity of the stimulated retrospective think-aloud method as measured by eye tracking.” In Proc., SIGCHI Conf. on Human Factors in Computing Sustems, 1253–1262. New York: Association for Computing Machinery.
Hagerty, M., N. H. Narayanan, and P. Freitas. 2002. “Understanding machines from multimedia and hypermedia presentations.” In The psychology of science text comprehenstion, 357–384. Hillsdale, NJ: Lawrence Erlbaum Associates.
Hornof, A., and T. Halverson. 2002. “Cleaning up systematic error in eye-tracking data by using fixation locations.” Behav. Res. Meth. Instrum. Comput. 34 (4): 592–604. https://doi.org/10.3758/BF03195487.
ITE (Institute of Traffic Engineers). 2000. Traffic engineering handbook. 5th ed. Washington, DC: ITE.
Jonassen, D., J. Strobel, and C. B. Lee. 2006. “Everyday problem solving in engineering: Lessons for engineering educators.” J. Eng. Educ. 95 (2): 139–151. https://doi.org/10.1002/j.2168-9830.2006.tb00885.x.
Kasarskis, P., J. Stehwien, J. Hickox, A. Aretz, and C. Wickens. 2001. “Comparison of expert and novice scan behaviors during VFR flight.” In Proc., 11th Int. Symp. on Aviation Psychology. Columbus, OH: Ohio State Univ.
Kokotovich, V. 2008. “Problem analysis and thinking tools: An empirical study of non-hierarchical mind mapping.” Des. Stud. 29 (1): 49–69. https://doi.org/10.1016/j.destud.2007.09.001.
Korte, R., S. Brunhaver, and S. Sheppard. 2015. “(Mis)Interpretations of organizational socialization: The expectations and experiences of newcomers and managers.” Hum. Res. Dev. Q. 26 (2): 185. https://doi.org/10.1002/hrdq.21206.
Lozner, T., U. Bailey, E. Lindstrom, K. Lee, S. Quayle, S. Beaird, S. Tsoi, P. Ryus, D. Gettman, S. Sunkari, K. Balke, D. Bullock, and A. Tanaka. 2015. Signal timing manual. 2nd ed. Washington, DC: NCHRP.
Morphew, J. W., J. P. Mestre, B. H. Ross, and N. E. Strand. 2015. “Do experts and novices direct attention differently in examining physics diagrams? A study of change detection using the flicker technique.” Phys. Rev. Spec. Top. Phys. Educ. Res. 11 (2): 1–6. https://doi.org/10.1103/PhysRevSTPER.11.020104.
Pavlović, N., and K. Jensen. 2009. “Eye tracking translation directionality.” Translat. Res. Project. 2: 93–109.
Rayner, K. 1977. “Visual attention in reading: Eye movements reflect cognitive processes.” Mem. Cognit. 5 (4): 443–448. https://doi.org/10.3758/BF03197383.
Rayner, K. 1998. “Eye movements in reading and information processing.” Psychol. Bull. 124 (3): 372–422. https://doi.org/10.1037/0033-2909.124.3.372.
Rayner, K., and M. Castelhano. 2007. “Eye movements.” Scholarperdia 2 (10): 3649. https://doi.org/10.4249/scholarpedia.3649.
Roess, R. P., E. S. Prassas, and W. R. McShane. 2011. Traffic engineering. 4th ed. Upper Saddle: Prentice Hall.
Salvucci, D. D., and J. H. Goldberg. 2000. “Identifying fixations and saccades in eye-tracking protocols.” In Proc., Symp. on Eye Tracking Research and Applications, 71–78. New York: Association for Computing Machinery.
Schwonke, R., K. Berthold, and A. Renkl. 2009. “How multiple external representations are used and how they can be made more useful.” Appl. Cognit. Psychol. 23 (9): 1227–1243. https://doi.org/10.1002/acp.1526.
Shulman, G. L., R. W. Remington, and J. P. McLean. 1979. “Moving attention through visual space.” J. Exp. Psychol.: Hum. Percept. Perform. 5 (3): 522–526. https://doi.org/10.1037/0096-1523.5.3.522.
Snowden, R., P. Thompson, and T. Troscianko. 2012. Basic vision an introduction to visual perception. Oxford, UK: Oxford University Press.
Sofaer, S. 2002. “Qualitative research methods.” Int. J. Qual. Health Care 14 (4): 329. https://doi.org/10.1093/intqhc/14.4.329.
Sudman, S., and N. Bradburn. 2003. “Thinking about answers: The application of cognitive processes to survey methodology.” Qual. Life Res. 12 (6): 719. https://doi.org/10.1023/A:1025127424627.
Tien, T., P. H. Pucher, M. H. Sodergren, K. Sriskandarajah, G. Yang, and A. Darzi. 2014. “Science direct eye tracking for skills assessment and training : A systematic review.” 191 (1): 169–178. https://doi.org/10.1016/j.jss.2014.04.032.
Tsui, A. 2003. Understanding expertise in teaching: Case studies of second language teachers. Cambridge, UK: Cambridge University Press.
Turochy, R. E., J. Fricker, H. G. Hawkins, D. S. Hurwitz, S. S. Ivey, M. A. Knodler, and R. K. Young. 2013. “Assessment of introductory transportation engineering course and general transportation engineering curriculum.” Transport. Res. Rec.: J. Transport. Res. Board 2328 (1): 9–15. https://doi.org/10.3141/2328-02.
Van Gog, T., F. Paas, J. J. G. Van Merrie, and P. Witte. 2005. “Uncovering the problem-solving process: Cued retrospective reporting versus concurrent and retrospective reporting.” J. Exp. Psychol.: Appl. 11 (4): 237–244. https://doi.org/10.1037/1076-898X.11.4.237.
Van Labeke, N., and S. E. Ainsworth. 2001. “Applying the DeFT framework to the design of multi-representational instructional simulations.” In Proc., 10th Int. Conf. on AI in Education, 314–321. San Antonio, TX: IOS Press.
Van Meeuwen, L. W., H. Jarodzka, S. Brand-Gruwel, P. A. Kirschner, J. J. P. R. De Bock, and J. J. G. Van Merriënboer. 2014. “Identification of effective visual problem solving strategies in a complex visual domain.” Learn. Instruct. 32 (Aug): 10–21. https://doi.org/10.1016/j.learninstruc.2014.01.004.
Van Someren, M. W., P. Reimann, and H. Boshuizen. 1998. Learning with multiple representations. Advances in learning and instruction series. New York: Elsevier.
Vine, S. J., R. S. W. Masters, and J. S. Mcgrath. 2012. “Cheating experience: Guiding novices to adopt the gaze strategies of experts expedites the learning of technical laparoscopic skills.” Surgery 152 (1): 32–40.
Voßkühler, A., V. Nordmeier, L. Kuchinke, and A. M. Jacobs. 2008. “OGAMA (Open Gaze and Mouse Analyzer): Open-Source software designed to analyze eye and mouse movements in slideshow study designs.” Behav. Res. Meth. 40 (4): 1150–1162. https://doi.org/10.3758/BRM.40.4.1150.
Wolff, C. E., H. Jarodzka, and N. Van Den. Bogert. 2016. “Teacher vision: Expert and novice teachers’ perception of problematic classroom management scenes.” Instr. Sci. 44 (3): 243–265. https://doi.org/10.1007/s11251-016-9367-z.
Information & Authors
Information
Published In
Journal of Civil Engineering Education
Volume 147 • Issue 3 • July 2021
Copyright
© 2021 American Society of Civil Engineers.
History
Received: Apr 7, 2020
Accepted: Dec 15, 2020
Published online: Mar 19, 2021
Published in print: Jul 1, 2021
Discussion open until: Aug 19, 2021
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- Hanliang Fu, Zhongjing Xia, Yubing Tan, Xiaotong Guo, Influence of Cues on the Safety Hazard Recognition of Construction Workers during Safety Training: Evidence from an Eye-Tracking Experiment, Journal of Civil Engineering Education, 10.1061/JCEECD.EIENG-1882, 150, 1, (2024).
- Sean L. Gestson, Shane Brown, Ananna Ahmed, David Hurwitz, Floraliza Bornasal, Renee M. Desing, Exploring the Differences in Hydraulic Engineering Problem-Solving Behavior between Undergraduate Students and Engineering Practitioners with Multiple Representations Using Eye-Tracking Techniques, Journal of Civil Engineering Education, 10.1061/JCEECD.EIENG-1872, 150, 1, (2024).
- Italo Rodeghiero Neto, Fernando Gonçalves Amaral, Teaching occupational health and safety in engineering using active learning: A systematic review, Safety Science, 10.1016/j.ssci.2023.106391, 171, (106391), (2024).
- Verena Ruf, Anna Horrer, Markus Berndt, Sarah Isabelle Hofer, Frank Fischer, Martin R. Fischer, Jan M. Zottmann, Jochen Kuhn, Stefan Küchemann, A Literature Review Comparing Experts’ and Non-Experts’ Visual Processing of Graphs during Problem-Solving and Learning, Education Sciences, 10.3390/educsci13020216, 13, 2, (216), (2023).
- Muhammed DAĞLI, Ahmet Feyzi SATICI, The effects of Multiple Representation Method and Prior Knowledge Level on Problem Solving Skills and Cognitive Load, Participatory Educational Research, 10.17275/per.23.78.10.5, 10, 5, (119-146), (2023).
- Kenji Wada, Günter Wallner, Steven Vos, Studying the Utilization of a Map-Based Visualization with Vitality Datasets by Domain Experts, Geographies, 10.3390/geographies2030024, 2, 3, (379-396), (2022).