Artifact-Based Energy Literacy Assessment Utilizing Rubric Scoring
Publication: Journal of Professional Issues in Engineering Education and Practice
Volume 141, Issue 2
Abstract
This research focuses on the development of an energy literacy rubric for scoring project-type deliverables, an approach that has not been taken with respect to energy literacy assessment. The goal is to examine if a rubric-based rating approach is applicable for judging energy literacy based on competition or course deliverables or artifacts. The rubric was developed using a methodology involving related rubric review and adaptation, and an expert review. To examine if the rubric approach may be applicable, the rubric was applied to the Imagine Tomorrow competition, a high school energy competition, and trends in the results were examined to support rubric effectiveness. Competition deliverables include an abstract and a poster. Abstracts submitted to enter the competition from 5 years were available and evaluated for energy literacy and biofuel literacy by two raters. Posters were available only for 1 year and were rated by one rater. Besides identifying trends in the abstract scores, the abstract scores were used to test interrater reliability in which consistency was found to be high. Based on expected trends occurring in both the abstract and poster scores, and the high interrater consistency, it was determined that the rubric rating approach appears to be working effectively, at least for a preliminary evaluation. Future development and refinement of this type of energy literacy rubric may serve to enhance assessment of sustainability educational activity endeavors based purely on the artifacts involved.
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments
The authors would like to thank Ecoworks, the Bank of America, NARA, Boeing, and the many other sponsors of the Imagine Tomorrow Competition for their generous support. Appreciation is also for other members of the Imagine Tomorrow Competition steering committee, particularly Brian French for expertise in instrument assessment.
References
Barrow, L., and Morrisey, T. (1989). “Energy literacy of ninth-grade students: A comparison between Maine and New Brunswick.” J. Environ. Educ., 20(2), 22–25.
Blumenfeld, P. C., Soloway, E., Marx, R. W., Krajcik, J. S., Guzdial, M., and Palincsar, A. (1991). “Motivating project-based learning: Sustaining the doing, supporting the learning.” Educ. Psychol., 26(3/4), 369–398.
Bodzin, A. (2012). “Investigating urban eighth-grade students’ knowledge of energy resources” Int. J. Sci. Educ., 34(8), 1255–1275.
Brewer, R. S., Lee, G. E., and Johnson, P. M. (2011). “The Kukui Cup: A dorm energy competition focused on sustainable behavior change and energy literacy.” Proc., Hawaii Int. Conf. on System Sciences, Univ. of Hawaii at Manoa, Honolulu, HI, 1–10.
Cohen, J. (1960). “A coefficient of agreement for nominal scales.” Educ. Psychol. Measure., 20(1), 37–46.
Cohen, J. (1988). “The significance of a product moment rs.” Statistical power analysis for the behavioral sciences, L. Erlbaum Associates, Hillsdale, NJ, 75–107.
DeWaters, J. E., and Powers, S. E. (2011a). “Energy literacy of secondary students in New York State (USA): A measure of knowledge, affect, and behavior.” Energy Policy, 39(3), 1699–1710.
DeWaters, J. E., and Powers, S. E. (2011b). “Improving energy literacy among middle school youth with project-based learning pedagogies.” Frontiers in Education Conf. (FIE), IEEE, Rapid City, SD.
Dias, R. A., Mattos, C. R., and Balestieri, J. A. P. (2004). “Energy education: Breaking up the rational energy use barriers.” Energy Policy, 32(11), 1339–1347.
Hofmann, A. (2010). “Abstract.” Scientific writing and communication, Oxford University Press, New York, 312–326.
Howell, R. J. (2013). “Grading rubrics: Hoopla or help?” Innovations in education and teaching international, Routledge, New York, 1–11.
Kanter, D. E. (2010). “Doing the project and learning the content: Designing project-based science curricula for meaningful understanding.” Sci. Educ., 94(3), 525–551.
Kelly, G. J., and Takao, A. (2002). “Epistemic levels in argument: An analysis of university oceanography students’ use of evidence in writing.” Sci. Educ., 86(3), 314–342.
Landis, J. R., and Koch, G. G. (1977). “The measurement of observer agreement for categorical data.” Biometrics, 33(1), 159–174.
Lawrenz, F. (1983). “Student knowledge of energy issues.” School Sci. Math., 83(7), 587–595.
Marx, R. W., et al. (2004). “Inquiry-based science in the middle grades: Assessment of learning in urban systemic reform.” J. Res. Sci. Teach., 41(10), 1063–1080.
Mertler, C. A. (2001). “Designing scoring rubrics for your classroom.” Practical Assess. Res. Eval., 7(25), 1–9.
Moskal, B. M., and Leydens, J. A. (2000). “Scoring rubric development: Validity and reliability.” Practical Assess. Res. Eval., 7(10), 1–11, 〈http://PAREonline.net/getvn.asp?v=7&n=10〉 (Oct. 16, 2013).
The National Environmental Education & Training Foundation (NEETF). (2002). “Americans low ‘Energy IQ:’ A risk to our energy future.” 〈http://www.neefusa.org/pdf/roper/Roper2002.pdf〉 (Oct. 21, 2013).
Oakleaf, M. (2009). “Using rubrics to assess information literacy: An examination of methodology and interrater reliability.” J. Am. Soc. Inform. Sci. Technol., 60(5), 969–983.
Pasten, C., and Santamarina, J. C. (2012). “Energy and quality of life.” Energy Policy, 49, 468–476.
Popham, W. J. (1997). “What’s wrong—And what’s right—With rubrics.” Educ. Leader., 55(2), 72–75.
Reddy, Y. M., and Andrade, H. (2010). “A review of rubric use in higher education.” Assess. Eval. High. Educ., 35(4), 435–448.
Rhodes, T., and Finley, A. (2013). “Rubric calibration.” Using the VALUE rubrics for improvement of learning and authentic assessment, Association of American Colleges and Universities, Washington, DC, 23–25.
RubiStar. (2013). “Create rubrics for your project-based learning activities.” 〈http://rubistar.4teachers.org〉 (Oct. 15, 2013).
Schneider, R. M., Krajcik, J., Marx, R. W., and Soloway, E. (2002). “Performance of students in project-based science classrooms on a national measure of science achievement.” J. Res. Sci. Teach., 39(5), 410–422.
Stemler, S. E. (2004). “A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability.” Practical Assess. Res. Eval., 9(4), 1–19.
Switzky, H., and Gambro, J. (1999). “Variables associated with American high school students’ knowledge of environmental issues related to energy and pollution.” J. Environ. Educ. 30(2), 15–22.
Tariq, V. N., Stefani, L. A. J., Butcher, A. C., and Heylings, D. J. A. (1998). “Developing a new approach to the assessment of project work.” Assess. Eval. High. Educ., 23(3), 221–240.
Timmerman, B. E. C., Strickland, D. C., Johnson, R. L., and Payne, J. R. (2011). “Development of a ‘universal’ rubric for assessing undergraduates’ scientific reasoning skills using scientific writing.” Assess. Eval. High. Educ., 36(5), 509–547.
U.S. Department of Energy (DOE). (2013). “Energy literacy: Essential principles and fundamental concepts for energy education.” 〈http://www1.eere.energy.gov/education/energy_literacy.html〉 (Oct. 21, 2013).
Zografakis, N., Menegaki, A. N., and Tsagarakis, K. P. (2008). “Effective education for energy efficiency.” Energy Policy, 36, 3226–3232.
Information & Authors
Information
Published In
Journal of Professional Issues in Engineering Education and Practice
Volume 141 • Issue 2 • April 2015
Copyright
© 2014 American Society of Civil Engineers.
History
Received: Dec 13, 2013
Accepted: Mar 28, 2014
Published online: May 12, 2014
Discussion open until: Oct 12, 2014
Published in print: Apr 1, 2015
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.