Abstract
The ability to self-assess is a key component of learning and particularly of life-long learning. Knowledge surveys (KS) are a self-assessment tool where respondents rate their ability to answer a question or perform a skill rather than answer directly with knowledge content or by solving a problem as they would on an exam. KS were implemented in a junior level hydraulics class in three successive semesters, with a total of 64 students, to evaluate the accuracy of student self-assessments compared with instructor assessments (exams). Students’ self-reported behaviors and attitudes towards KS were assessed through qualitative feedback surveys. Results of this study show that: (1) students are reasonably accurate self-assessors, (2) accuracy improves with feedback and practice during the semester, and (3) higher exam scores are correlated with both earlier and more engaged use of KS, and with more positive self-reported attitudes. Given the successful use of KS in this study, civil engineering faculty should be encouraged to use KS in their courses to stimulate metacognitive reflection and learning. Further, KS and self-assessment have the potential to be a significant tool for achieving the Accreditation Board for Engineering and Technology (ABET) student outcome for acquiring and applying new knowledge, both in academic and professional settings as civil engineers adapt to an ever-changing world.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code generated or used during the study are proprietary or confidential in nature and may only be provided with restrictions. Anonymized student self-assessment and exam data are available from the corresponding author upon reasonable request.
Acknowledgments
The authors thank C. Dirk MacDonald who taught and collected data for the Fall 2019b section of Fundamental Hydraulics, and Regan Keener and Jake Polombo who assisted with coding of the open-ended questions.
Disclaimer
The views expressed in this paper are those of the authors and do not necessarily reflect the official policy or position of the United States Air Force Academy, the Air Force, the Department of Defense, or the US Government.
References
ABET (Accreditation Board for Engineering and Technology). 2021a. “Criteria for accrediting engineering programs, 2018–2019.” Accessed March 31, 2021. https://www.abet.org/accreditation/accreditation-criteria/criteria-for-accrediting-engineering-programs-2018–2019/#GC3.
ABET (Accreditation Board for Engineering and Technology). 2021b. “Criteria for accrediting engineering programs, 2019–2020.” Accessed March 31, 2021. https://www.abet.org/accreditation/accreditation-criteria/criteria-for-accrediting-engineering-programs-2019-2020/#GC3.
Anderson, L. W., and D. R. Krathwohl. 2001. A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman.
Andrade, H., and A. Valtcheva. 2009. “Promoting learning and achievement through self-assessment.” Theory Pract. 48 (1): 12–19. https://doi.org/10.1080/00405840802577544.
Ariel, R., and J. D. Karpicke. 2018. “Improving self-regulated learning with a retrieval practice intervention.” J. Exp. Psychol. 24 (1): 43–56.
Bahati, B., U. Fors, and M. Tedre. 2017. “Can student engagement in online courses predict performance on online knowledge surveys?” Int. J. Learn. Teach. Educ. Res. 16 (3): 73–87.
Beauregard, M. S., J. A. Sloan, and M. Brannon. 2019. “Summarizing the use of knowledge surveys to inform effective learning and teaching practices.” In Proc., Rocky Mountain Section of the American Society for Engineering Education’s Annual Conf. Washington, DC: American Society for Engineering Education.
Bell, P., and D. Volckmann. 2011. “Knowledge surveys in general chemistry: Confidence, overconfidence and performance.” J. Chem. Educ. 88 (Sep): 1469–1476. https://doi.org/10.1021/ed100328c.
Bloom, B. S., M. D. Englehart, E. J. Furst, W. H. Hill, and D. R. Krathwohl. 1956. “Taxonomy of educational objectives: The classification of educational goals.” In Handbook I: Cognitive domain. New York: David McKay Company.
Boud, D., and N. Falchikov. 2005. “Redesigning assessment for learning beyond higher education.” Res. Dev. Higher Educ. 28 (Jul): 34–41.
Boud, D., R. Lawson, and D. G. Thompson. 2013. “Does student engagement in self-assessment calibrate their judgement over time?” Assess. Eval. Higher Educ. 38 (8): 941–956. https://doi.org/10.1080/02602938.2013.769198.
Bowers, N., M. Brandon, and C. D. Hill. 2005. “The use of a knowledge survey as an indicator of student learning in an introductory biology course.” Cell Biol. Educ. 4 (4): 311–322. https://doi.org/10.1187/cbe.04-11-0056.
Brown, N. A., D. P. Dewey, and T. L. Cox. 2014. “Assessing the validity of can-do statements in retrospective (then-now) self-assessment.” Foreign Lang. Ann. 47 (2): 261–285. https://doi.org/10.1111/flan.12082.
Clauss, J., and K. Geedey. 2010. “Knowledge surveys: Students ability to self-assess.” J. Scholarship Teach. Learn. 10 (2): 14–24.
Correll, S. J. 2001. “Gender and the career choice process: The role of biased self-assessments.” Am. J. Sociol. 106 (6): 1691–1730. https://doi.org/10.1086/321299.
Cunningham, P., H. M. Matusovich, J. R. Morelock, and D. N. Hunter. 2016. Beginning to understand and promote engineering students’ metacognitive development.” In Proc., ASEE Annual Conf. and Exposition. Washington, DC: American Society for Engineering Education.
Cunningham, P., H. M. Matusovish, D. A. N. Hunter, and R. E. McCord. 2015. “Teaching metacognition: Helping engineering students take ownership of their own learning.” In Proc., Paper Presented at the 2015 IEEE Frontiers in Education Conf. New York: IEEE.
Ertmer, P. A., and T. J. Newby. 1996. “The expert learner: Strategic, self-regulated and reflective.” Instructional Sci. 24 (1): 1–24. https://doi.org/10.1007/BF00156001.
Favazzo, L., J. Willford, and R. Watson. 2014. “Correlating student knowledge and confidence using a graded knowledge survey to assess student learning in a general microbiology classroom.” J. Microbiol. Biol. Educ. 15 (2): 251–258. https://doi.org/10.1128/jmbe.v15i2.693.
Fink, L. D. 2003. Creating significant learning experiences: An integrated approach to designing college courses, 317. New York: Wiley.
Flavell, J. H. 1976. “Metacognitive aspects of problem solving.” In The nature of intelligence, edited by L. B. Resnick, 231–236. Hillsdale, NJ: Erlbaum.
Flavell, J. H. 1979. “Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry.” Am. Psychol. 34 (10): 906–911. https://doi.org/10.1037/0003-066X.34.10.906.
Ghanat, S. T., J. Kaklamanos, K. Ziotopoulou, S. Immanuel, and D. J. Fallon. 2016. “A multi-institutional study of pre- and post-course knowledge surveys in undergraduate geotechnical engineering courses.” In Proc., American Society for Engineering Education’s 123rd Annual Conf. and Exposition. Washington, DC: American Society for Engineering Education. https://doi.org/10.18260/p.26363.
Hall, K. D., D. G. Linzell, B. S. Minsker, J. F. Jajjar, and C. M. Saviz. 2020. Civil engineering education summit: Mapping the future of civil engineering education. Reston, VA: ASCE.
Ibabe, I., and J. Jauregizar. 2010. “Online self-assessment with feedback and metacognitive knowledge.” Higher Educ. 59 (2): 243–258. https://doi.org/10.1007/s10734-009-9245-6.
Karpicke, J. D., A. C. Butler, and H. L. Roediger III. 2009. “Metacognitive strategies in student learning: Do students practice retrieval when they study on their own?” Memory 17 (4): 471–479. https://doi.org/10.1080/09658210802647009.
Konrad, M., L. M. Joseph, and E. Eveleigh. 2009. “A meta-analytic review of guided notes.” Educ. Treat. Children 32 (3): 421–444. https://doi.org/10.1353/etc.0.0066.
Konrad, M., L. M. Joseph, and M. Itoi. 2011. “Using guided notes to enhance instruction for all students.” Intervention School Clinic 46 (3): 131–140. https://doi.org/10.1177/1053451210378163.
Kornell, N. 2009. “Optimising learning using flashcards: Spacing is more effective than cramming.” Appl. Cognit. Psychol. 23 (9): 1297–1317. https://doi.org/10.1002/acp.1537.
Kruger, J., and D. Dunning. 1999. “Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments.” J. Pers. Soc. Psychol. 77 (6): 1121–1134. https://doi.org/10.1037/0022-3514.77.6.1121.
Lew, M. D. N., W. A. M. Alwis, and H. G. Schmidt. 2010. “Accuracy of students’ self-assessment and their beliefs about its utility.” Assess. Eval. Higher Educ. 35 (2): 135–156. https://doi.org/10.1080/02602930802687737.
Luce, C., and J. P. Kirnan. 2016. “Using indirect vs. direct measures in the summative assessment of student learning in higher education.” J. Scholarship Teach. Learn. 16 (4): 75–91. https://doi.org/10.14434/josotl.v16i4.19371.
Marshman, E. M., Z. Y. Kalendar, T. Nokes-Malach, C. Schunn, and C. Sing. 2018. “Female students with A’s have similar physics self-efficacy as male students with C’s in introductory courses: A cause for alarm?” Phys. Rev. Phys. Educ. Res. 14 (2): 0200123. https://doi.org/10.1103/PhysRevPhysEducRes.14.020123.
McMillan, J. H., and J. Hearn. 2008. “Student self-assessment: The key to stronger student motivation and higher achievement.” Educ. Horiz. 87 (1): 40–49.
Millis, B. J. 2016. Using metacognition to promote learning. Manhattan, KS: IDEA Center.
Nilson, L. 2013. Creating self-regulated learners. Sterling, VA: Stylus Publishing.
Nuhfer, E. 1993. “Bottom-line disclosure and assessment.” Teach. Prof. 7 (7): 8.
Nuhfer, E., C. Cogan, S. Fleisher, E. Gaze, and K. Wirth. 2016a. “Random number simulations reveal how random noise affects the measurements and graphical portrayals of self-assessed competency.” Numeracy 9 (1): 4. https://doi.org/10.5038/1936-4660.9.1.4.
Nuhfer, E., S. Fleischer, C. Cogan, K. Wirth, and E. Gaze. 2017. “How random noise and a graphical convention subverted behavioral scientists’ explanations of self-assessment data: Numeracy underlies better alternatives.” Numeracy. 10 (1). https://doi.org/10.5038/1936-4660.10.1.4.
Nuhfer, E., and D. Knipp. 2003. “The knowledge survey: A tool for all reasons.” Improv. Acad. 21 (1): 59–78. https://doi.org/10.1002/j.2334-4822.2003.tb00381.x.
Nuhfer, E. B. 1996. “The place of formative evaluations in assessment and ways to reap their benefits.” J. Geosci. Educ. 44 (4): 385–394. https://doi.org/10.5408/1089-9995-44.4.385.
Nuhfer, E. B., C. B. Cogan, C. Klook, G. G. Wood, A. Goodman, N. Z. Delgado, and C. W. Wheeler. 2016b. “Using a concept inventory to assess the reasoning component of citizen-level science literacy: Results from a 17,000-student study.” J. Microbiol. Biol. Educ. 17 (1): 143–155. https://doi.org/10.1128/jmbe.v17i1.1036.
Pintrich, P. R. 2002. “The role of metacognitive knowledge in learning, teaching and assessing.” Theor. Pract. 41 (4): 219–225. https://doi.org/10.1207/s15430421tip4104_3.
Rivers, W. P. 2001. “Autonomy at all costs: An ethnography of metacognitive self-assessment and self-management among experienced language learners.” Mod. Lang. J. 85 (2): 279–290. https://doi.org/10.1111/0026-7902.00109.
Ross, J. A. 2006. “The reliability, validity, and utility of self-assessment.” Pract. Assess. Res. Eval. 11 (10): 1–13.
Strauss, A., and J. Corbin. 1997. Grounded theory in practice. 1st ed., 288. Thousand Oaks, CA: SAGE.
Sung, Y.-T., K.-E. Chang, S.-K. Chiou, and H.-T. Hou. 2005. “The design and application of a web-based self- and peer-assessment system.” Comput. Educ. 45 (2): 187–202. https://doi.org/10.1016/j.compedu.2004.07.002.
Thompson, G., A. Pilgrim, and K. Oliver. 2005. “Self-assessment and reflective learning for first-year university geography students: A simple guide or simply misguided?” J. Geogr. Higher Educ. 29 (3): 403–420. https://doi.org/10.1080/03098260500290959.
Watson, R. M., E. Nuhfer, K. N. Moon, S. Fleisher, and P. Walter. 2019. “Paired measures of competence and confidence illuminate impacts of privilege on college students.” Numeracy 12 (2).
Wirth, K. R. 2017a. “Keynote lecture.” In Proc., US Air Force Academy’s Scholarship of Teaching and Learning Forum. Air Force Academy, CO: United States Air Force Academy.
Wirth, K. R. 2017b. “Knowledge surveys.” In Proc., US Air Force Academy’s Scholarship of Teaching and Learning Forum. Air Force Academy, CO: United States Air Force Academy.
Wirth, K. R., and D. Perkins. 2005. “Knowledge surveys: An indispensable course design and assessment tool.” Innov. Scholarship Teach. Learn. 1–2.
Wu, S., S. Zha, and S. Mattson. 2020. “Integrating team-based learning modules to improve civil engineering students’ technical writing skills.” J. Civ. Eng. Educ. 146 (3): 04020005. https://doi.org/10.1061/(ASCE)EI.2643-9115.0000016.
Information & Authors
Information
Published In
Journal of Civil Engineering Education
Volume 148 • Issue 3 • July 2022
Copyright
© 2022 Published by American Society of Civil Engineers.
History
Received: May 25, 2021
Accepted: Jan 14, 2022
Published online: Mar 16, 2022
Published in print: Jul 1, 2022
Discussion open until: Aug 16, 2022
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
- José Díaz-Álvarez, Antonio Díaz-Álvarez, Ramiro Mantecón, María Henar Miguélez, “FLIPPED ASSESSMENT”: Proposal for a Self-Assessment Method to Improve Learning in the Field of Manufacturing Technologies, Education Sciences, 10.3390/educsci13080831, 13, 8, (831), (2023).
- Ayuni Ratnayake, Aditi Bansal, Natalie Wong, Theluckshan Saseetharan, Sapolnach Prompiengchai, Amy Jenne, Jeneni Thiagavel, Aarthi Ashok, All “wrapped” up in reflection: supporting metacognitive awareness to promote students’ self-regulated learning, Journal of Microbiology & Biology Education, 10.1128/jmbe.00103-23, (2023).
- Karin L. Becker, Joel A. Sloan, Implementing Technical Communication Instruction in a Civil Engineering Course: The Value of Knowledge Surveys and Peer Review, Journal of Civil Engineering Education, 10.1061/JCEECD.EIENG-1927, 149, 4, (2023).
- Michelle Naef, Lianne Lefsrud, Smooth operator: Aligning performance assessment methods with design and operating objectives, Journal of Loss Prevention in the Process Industries, 10.1016/j.jlp.2023.105158, 85, (105158), (2023).
- Li Yan, Charanjit Kaur Swaran Singh, Goh Hock Seng, Xu Meng, Yang Xuelan, Nadia Refat, Well-designed Electronic Portfolio as a Facilitator for EFL Engineering Learners Reading Skill, 2022 International Conference on Intelligent Technology, System and Service for Internet of Everything (ITSS-IoE), 10.1109/ITSS-IoE56359.2022.9990938, (1-5), (2022).