Differences between Experts and Novices in the Review of Engineering Journal Papers
Publication: Journal of Professional Issues in Engineering Education and Practice
Volume 132, Issue 1
Abstract
This paper reports on a controlled experiment carried out to investigate the differences between novices and experts during the review of papers for engineering journals. A total of 17 young faculty members with PhD degrees were given instruction on the peer review process, and then had to review a short paper, for which the actual reviews were available. The quality of the reviews was assessed using criteria developed by editors of medical journals. The evidence obtained from this study helps to identify the critical areas in which reviewers fail to perform a good review, and then to develop strategies to overcome such limitations.
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments
The writer thanks the participants of this study, who should remain anonymous. This work was supported by a grant provided by the University of Puerto Rico at Mayagüez. The writer thanks the comments made by two anonymous reviewers, which helped to improve this manuscript.
References
Baxt, W. G., Waekerle, J. F., Tintinally, J. E., Knopp, R. K., and Callaham, M. L. (1996). “Evaluation of the peer-reviewer: Performance of reviewers of a fictitious submission.” Acad. Emerg. Med., 3, 504.
Black, N., van Rooyen, S., Godlee, F., Smith, R., and Evans, S. (1998). “What makes a good reviewer and a good review for a general medical journal.” J. Am. Med. Assoc., 280(3), 231–233.
Bransford, J. D., Brown, A. L., and Cocking, R. C. (1999). How people learn: Brain, mind, experience, and school, National Academy Press, Washington, D. C.
Callaham, M. L., Baxt, W., Waekerle, J., and Wears, R. (1998). “The reliability of editor’s subjective quality ratings of manuscript peer reviews.” J. Am. Med. Assoc., 280(3), 229–231.
Callaham, M. L., Knopp, R. K., and Gallagher, E. J. (2002). “Effect of written feedback by editors on quality of reviews.” J. Am. Med. Assoc., 287(21), 2781–2783.
Godoy, L. A., and Valeiras, N. (2001). “Initiative to strengthen publications by young faculty.” J. Prof. Issues Eng. Educ. Pract., 127(3), 116–121.
Harnad, S. (1982). Peer commentary on peer review, Cambridge University Press, Cambridge, U.K.
Ho, T. B. L. (2002). “How to survive peer review.” J. R. Soc. Med., 95(11), 571–572.
Judson, H. F. (1994). “Structural transformations of the Sciences and the end of peer-review.” J. Am. Med. Assoc., 272(2), 92–94.
Kassirer, J. P., and Campion, E. W. (1994). “Peer review: Crude and understated, but indispensable.” J. Am. Med. Assoc., 272(2), 96–97.
McCuen, R. H., ed. (1996). The elements of academic research, American Society of Civil Engineers, New York.
Miller, L. T., and Smith, A. E. (2004). “Engineering journal review process: A survey of engineering journal editors.” Proc., American Society of Engineering Education Southeast Section Conf. 2004, Auburn Ala.
Narayanan, R. R. (2003). “Academic leadership strategies for engineering faculty.” Int. J. Eng. Educ., 19(2), 241–251.
Nature Editor. (2001). “Bad peer reviewers”, Nature (London), 413(13), 93.
Peters, D., and Ceci, S. (1982). “Peer review practices of psychological journals: The fate of published articles, submitted again.” Behav. Brain Sci., 5(2), 187–255.
Peterson, G. A., and Logan, T. J. (1992). “An editing-writing workshop.” Research ethics, manuscript review and journal quality, H. F. Mayland and R. E. Sojka, eds., ACS Miscellaneous Publication, Madison, Wis.
Van Rooyen, S., Black, N., and Godlee, F. (1999). “Development of the review quality instrument (RQI) for assessing peer review of manuscripts.” J. Clin. Epidemiol., 52, 625–629.
Woodward, J., and Goodstein, D. (1996). “Conduct, misconduct and the structure of science.” Am. Sci., 84(5), 479–490.
Information & Authors
Information
Published In
Journal of Professional Issues in Engineering Education and Practice
Volume 132 • Issue 1 • January 2006
Pages: 24 - 28
Copyright
© 2006 ASCE.
History
Received: May 18, 2004
Accepted: May 10, 2005
Published online: Jan 1, 2006
Published in print: Jan 2006
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.