Skill Tests and Parametric Statistics for Model Evaluation
Publication: Journal of Hydraulic Engineering
Volume 112, Issue 2
Abstract
A series of traditional statistical methods and nonparametric skill tests are assembled into a model statistical analysis program (MSAP) with the intent of evaluating their ability to verify surface water models. Using two different hydrodynamic models, simulations of two Lake Erie storm surges are performed, with actual water levels for each storm surge obtained from monitoring stations surrounding the lake. The actual and computed water levels are analyzed by MSAP. Of the three major categories of statistical tests contained in MSAP, the skill tests are superior in demonstrating a model's ability or inability to predict the most important aspect of a storm‐surge simulation: its creation and die‐off. Time‐series statistical calculations such as correlation coefficients and root‐mean‐square deviation values are helpful only in assessing gross overall model performance. Statistical decision‐making tests, based on the acceptance or rejection of certain hypotheses within a specified level of significance, yield overly optimistic or incorrect results due to the violation of several assumptions necessary for proper use of the tests.
Get full access to this article
View all available purchase options and get full access to this article.
References
1.
Adams, E. E., et al., “Verification of Models of Hydrologic Transport and Dispersion—an Overview,” paper presented at ASCE Hydraulic Conference, Jackson, MS, 1982.
2.
Allender, T. H., “Comparison of Model and Observed Currents in Lake Michigan,” Journal of Physical Oceanography, Vol. 7, No. 5, 1977, pp. 711–718.
3.
Anthes, R. A., “Regional Models of the Atmosphere in Middle Latitudes,” Monthly Weather Review, Vol. 111, No. 6, 1983, pp. 1306–1335.
4.
Daniel, W. W., Applied Nonparametric Statistics, Houghton Mifflin Company, Boston, MA, 1978.
5.
Dingman, J. S., and Bedford, K. W., “The Lake Erie Response to the January 1978 Cyclone,” Final Report, US Environmental Protection Agency, Washington, DC, 1980, 161 pp.
6.
Fiering, M. B., and Jackson, B. B., Synthetic Streamflows, Water Resources Monograph No. 1, American Geophysical Union, Wasington, DC, 1971, 98 pp.
7.
Hamblin, P. F., “Great Lakes Storm Surge of April 6, 1979,” Journal of Great Lakes Research, Vol. 5, No. 3–4, 1979, pp. 312–315.
8.
Irish, S. M., and Platzman, G. W., “An Investigation of the Meteorological Conditions Associated with Extreme Wind Tides on Lake Erie,” Monthly Weather Review, Vol. 90, No. 1, 1962, pp. 39–47.
9.
Leendertse, J. J., and Gritton, E. C., A Water Quality Simulation Model for Well Mixed Estuaries and Coastal Seas, Computational Procedures, Vol. 2, Rand Institute, New York, NY, 1971, 55 pp.
10.
Lilliefors, H. W., “On the Kolmovogorov‐Smirnov Test For Normality With Mean and Variance Unknown,” American Statistical Association Journal, Vol. 62, No. 318, 1967, pp. 399–402.
11.
Mihram, G. A., “Some Practical Aspects of the Verification and Validation of Simulation Models,” Operational Research Quarterly, Vol. 23, No. 1, 1972, pp. 17–29.
12.
Ott, L., An Introduction to Statistical Methods and Data Analysis, Duxbury Press, North Scituate, MA, 1977.
13.
Reckhow, K. H., “Confirmation of Water Quality Simulation Models,” Applying Research to Hydraulic Practice, E. Smith, Ed., Conference Proceedings, Hydraulics Division, ASCE, New York, NY, 1982, pp. 604–613.
14.
Schwab, D. J., “Simulation and Forecasting of Lake Erie Storm Surges,” Monthly Weather Review, Vol. 106, No. 10, 1978, pp. 1476–1487.
15.
Shaeffer, D. L., “A Model Evaluation Methodology Applicable to Environmental Assessment Models,” Ecological Modelling, Vol. 8, 1970, pp. 275–295.
16.
Simons, T. J., “Verification of Numerical Models of Lake Ontario: Part I. Circulation in Spring and Early Summer,” Journal of Physical Oceanography, Vol. 4, 1974, pp. 507–523.
17.
Spaulding, M. L., and Beauchamp, C. H., “Modeling Tidal Circulation in Coastal Seas,” Journal of Hydraulic Engineering, Vol. 109, No. 1, 1983, pp. 116–132.
18.
Teweles, S., and Wobus, H., “Verification of Prognostic Charts,” Bulletin of the American Meteorological Society, Vol. 35, No. 10, 1954, pp. 455–463.
19.
Thomann, R. V., “Measures of Verification,” Workshop on Verification of Water Quality Models, US Environmental Protection Agency, Athens, GA, 1980 pp. 37–61.
20.
Thomann, R. V., “Verification of Water Quality Models,” Journal of the Environmental Engineering Division, ASCE, Vol. 108, No. EE5, 1982, pp. 923–940.
21.
Thomann, R. V., and Winfield, R. P., “On the Verification of a Three‐Dimensional Phytoplankton Model of Lake Ontario,” Proceedings of the Conference on Environmental Modeling and Simulation, US Environmental Protection Agency, Cincinnati, OH, 1976, pp. 568–572.
22.
Thorndike, R. M., Data Collection and Analysis—Basic Statistics, Gardner Press, New York, NY, 1982.
23.
“Workshop on Verification of Water Quality Models,” Hydroscience, Inc., US Environmental Protection Agency, Athens, GA, 1980.
Information & Authors
Information
Published In
Journal of Hydraulic Engineering
Volume 112 • Issue 2 • February 1986
Pages: 124 - 140
Copyright
Copyright © 1986 ASCE.
History
Published online: Feb 1, 1986
Published in print: Feb 1986
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.