Evaluating the Use of Nash-Sutcliffe Efficiency Coefficient in Goodness-of-Fit Measures for Daily Runoff Simulation with SWAT
Publication: Journal of Hydrologic Engineering
Volume 22, Issue 11
Abstract
Hydrological models must be carefully calibrated to ensure that predictions are scientifically sound and reliable. As a goodness-of-fit measure, an ideal threshold of the Nash-Sutcliffe efficiency coefficient () has been reported to be 0.65, based on yearly and monthly simulation results with the soil and water assessment tool (SWAT). To further explore the use of as a goodness-of-fit measure for daily runoff simulation with SWAT, the authors ran the model with five different parameter values, using Jinjiang River Basin on the southeast coast of China as the study area. In addition, to find how the model versions varied in predicting changes in flood and drought with changes in land use, each of the five model versions were run two times separately: once for the land use condition in 1985, and then again for the land use condition in 2006. The authors investigated the relationships between variation and model accuracy and discussed the differences in predicted runoff due to land use change as caused by the five model versions. The results show that the coefficient of determination () was highly associated with , with a high corresponding to a high ; the value was quite sensitive to simulation results of the maximum one-day (1d) runoff; 0.75; the value was not sensitive to simulation results of the minimum 1d and seven-day (7d) runoff, and unlike the flood runoff, the errors in drought flow increased adversely with increased , indicating the poor performance of the SWAT model regarding drought flows; and the variations in runoff simulations, due to the land use change, were sensitive to model version switching and were fundamentally similar to the changes in the error indicators said above. The five SWAT versions varied in performance () in daily runoff simulation and had different measures of goodness-of-fit for various characteristics of the flood and drought processes. As a result, an of 0.75 is suggested as a threshold for satisfactory simulation of maximum 1d discharge in the study watershed.
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments
The study was financially supported by the Science and Technology Major Project of Fujian Province (Grant No. 2015Y4002). We thank all data providers and previous students who worked on this topic. Chris McConnell provided a final proofread. We appreciate the three reviewers and the editors for their constructive comments.
References
Akpoti, K., Antwi, E. O., and Kabo-Bah, A. T. (2016). “Impacts of rainfall variability, land use and land cover change on stream flow of the black Volta Basin, West Africa.” Hydrology, 3(3), 26.
Arnold, J. G., et al. (2012). “SWAT: Model use, calibration, and validation.” Trans. ASABE, 55(4), 1491–1508.
Arnold, J. G., Srinivasan, R., Muttiah, R. S., and Williams, J. R. (1998). “Large area hydrologic modeling and assessment. I: Model development.” J. Am. Water Resour. Assoc., 34(1), 73–89.
Baker, T. J., and Miller, S. N. (2013). “Using the soil and water assessment tool (SWAT) to assess land use impact on water resources in an East African watershed.” J. Hydrol., 486(8), 100–111.
Bastidas, L. A., Gupta, H. V., Sorooshian, S., Shuttleworth, W. J., and Yang, Z. L. (1999). “Sensitivity analysis of a land surface scheme using multi criteria methods.” J. Geophys. Res., 104(D16), 19481–19490.
Boyle, D. P., Gupta, H. V., and Sorooshian, S. (2000). “Toward improved calibration of hydrologic models: Combining the strengths of manual and automatic methods.” Water Resour. Res., 36(12), 3663–3674.
Cibin, R., Sudheer, K. P., and Chaubey, I. (2010). “Sensitivity and identifiability of stream flow generation parameters of the SWAT model.” Hydrol. Process., 24(9), 1133–1148.
Clarke, R. T. (2008). “A critique of present procedures used to compare performance of rainfall-runoff models.” J. Hydrol., 352(3–4), 379–387.
Donigian, A. S., Imhoff, J. C., and Bicknell, B. R. (1983). “Predicting water quality resulting from agricultural nonpoint-source pollution via simulation-HSPF.” Agricultural management and water quality, Iowa State University Press, Ames, IA, 200–249.
Douglas-Mankin, K. R., Srinivasan, R., and Arnold, A. J. (2010). “Soil and water assessment tool (SWAT) model: Current developments and applications.” Trans. ASAE, 53(5), 1423–1431.
Gassman, P. W., Reyes, M. R., Green, C. H., and Arnold, J. G. (2007). “The soil and water assessment tool: Historical development, applications, and future research directions.” Trans. ASABE, 50(4), 1211–1250.
Gupta, H. V., and Kling, H. (2011). “On typical range, sensitivity, and normalization of mean squared error and Nash-Sutcliffe efficiency type metrics.” Water Resour. Res., 47(10), 125–132.
Gupta, H. V., Kling, H., Yilmaz, K. K., and Martinez, G. F. (2009). “Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling.” J. Hydrol., 377(1), 80–91.
Jain, S. K., and Sudheer, K. P. (2008). “Fitting of hydrologic model: A close look at the Nash–Sutcliffe index.” J. Hydrol. Eng., 981–986.
Legates, D. R., and McCabe, G. J. (1999). “Evaluating the use of ‘goodness-of-fit’ measures in hydrologic and hydroclimatic model validation.” Water Resour. Res., 35(1), 233–241.
Levesque, É., Anctil, F., Van Griensven, A. N. N., and Beauchamp, N. (2008). “Evaluation of streamflow simulation by SWAT model for two small watersheds under snowmelt and rainfall.” Hydrol. Sci. J., 53(5), 961–976.
Lin, B., et al. (2015). “Analyses of landuse change impacts on catchment runoff using different time indicators based on SWAT model.” Ecol. Indic., 58, 55–63.
Lin, B., Chen, X., Chen, Y., and Liu, M. (2014). “Simulations and analysis on the effects of landscape pattern change on flood and low flow based on SWAT model.” Acta Ecol. Sin., 34(7), 1772–1780 (in Chinese with English abstract).
Lin, B., Chen, Y., and Chen, X. (2013). “A study on regional difference of hydrological parameters of SWAT model.” J. Nat. Res., 28(11), 1988–1999 (in Chinese with English abstract).
Mccuen, R. H. (2006). “Evaluation of the Nash-Sutcliffe efficiency index.” J. Hydrol. Eng., 597–602.
Moriasi, D. N., Arnold, J. G., Van Liew, M. W., Binger, R. L., Harmel, R. D., and Veith, T. L. (2007). “Model evaluation guidelines for systematic quantification of accuracy in watershed simulations.” Trans. ASABE, 50(3), 885–900.
Motovilov, Y. G., Gottschalk, L., England, K., and Rodhe, A. (1999). “Validation of distributed hydrological model against spatial observations.” Agric. For. Meteorol., 98(74), 257–277.
Moussa, R. (2010). “When monstrosity can be beautiful while normality can be ugly: Assessing the performance of event-based flood models.” J. Hydrol. Sci., 55(6), 1074–1084.
Muleta, M. K. (2012). “Improving model performance using season-based evaluation.” J. Hydrol. Eng., 191–200.
Murphy, A. (1988). “Skill scores based on the mean square error and their relationships to the correlation coefficient.” Mon. Weather Rev., 116(12), 2417–2424.
Nash, J. E., and Sutcliffe, J. V. (1970). “River flow forecasting trough. 1: A conceptual model discussion of principles.” J. Hydrol., 10(3), 282–290.
Neitsch, S. L., Arnold, J. G., Kiniry, J. R., and Williams, J. R. (2009). “Soil and water assessment tool theoretical documentation version 2009.” Texas Water Resources Institute, College Station, TX.
Ramanarayanan, T. S., Williams, J. R., Dugas, W. A., Hauck, L. M., and McFarland, A. M. S. (1997). “Using APEX to identify alternative practices for animal waste management.” ASAE, St. Joseph, MI.
Santhi, C., Arnold, J. G., Williams, J. R., Dugas, W. A., Srinivasan, R., and Hauck, L. M. (2001). “Validation of the SWAT model on a large river basin with point and nonpoint sources.” J. Am. Water Resour. Assoc., 37(5), 1169–1188.
Saxton, K. E., and Rawls, W. J. (2006). “Soil water characteristic estimates by texture, and organic matter for hydrologic solutions.” Soil Sci. Soc. Am. J., 70(5), 1569–1578.
Singh, J., Knapp, H. V., Arnold, J. G., and Demissie, M. (2005). “Hydrologic modeling of the Iroquois River watershed using HSPF and SWAT.” J. Am. Water Resour. Assoc., 41(2), 343–360.
Singh, J., Knapp, H. V., and Demissie, M. (2004). “Hydrologic modeling of the Iroquois River watershed using HSPF and SWAT.” ⟨www.sws.uiuc.edu/pubdoc/CR/ISWSCR2004-08.pdf⟩ (Sep. 8, 2005).
SPAW [Computer software]. USDA, Washington, DC.
Stehr, A., Debels, P., Romero, F., and Alcayaga, H. (2008). “Hydrological modelling with SWAT under conditions of limited data availability: Evaluation of results from a Chilean case study.” Hydrol. Sci. J., 53(3), 588–601.
Strauch, M., Bernhofer, C., Koide, S., Volk, M., Lorz, C., and Makeschin, F. (2012). “Using precipitation data ensemble for uncertainty analysis in SWAT stream flow simulation.” J. Hydrol., 414–415, 413–424.
Ullrich, A., and Volk, M. (2009). “Application of the soil and water assessment tool (SWAT) to predict the impact of alternative management practices on water quality and quantity.” Agric. Water Manage., 96(8), 1207–1217.
Van Liew, M. W., Arnold, J. G., and Garbrecht, J. D. (2003). “Hydrologic simulation on agricultural watersheds: Choosing between two models.” Trans. ASAE, 46(6), 1539–1551.
Van Liew, M. W., Veith, T. L., Bosch, D. D., and Arnold, J. G. (2007). “Suitability of SWAT for the conservation effects assessment project: A comparison on USDA-ARS experimental watersheds.” J. Hydrol. Eng., 173–189.
Wang, L., and Chen, X. W. (2008). “Simulation of hydrological effects on vegetation restoration of degraded mountain in ecosystem with SWAT model.” J. Mt. Sci., 26(1), 71–75 (in Chinese with English abstract).
Yapo, P. O., Gupta, H. V., and Sorooshian, S. (1998). “Multi-objective global optimization for hydrologic models.” J. Hydrol., 204(1), 83–97.
Yaseen, Z. M., et al. (2016). “Boost stream-flow forecasting model with extreme learning machine data-driven: A case study in a semi-arid region in Iraq.” J. Hydrol., 542, 603–614.
Zhang, D., Chen, X., Yao, H., and Lin, B. (2015). “Improved calibration scheme of SWAT by separating wet and dry seasons.” Ecol. Modell., 301, 54–61.
Information & Authors
Information
Published In
Journal of Hydrologic Engineering
Volume 22 • Issue 11 • November 2017
Copyright
©2017 American Society of Civil Engineers.
History
Received: Dec 15, 2016
Accepted: May 15, 2017
Published online: Sep 1, 2017
Published in print: Nov 1, 2017
Discussion open until: Feb 1, 2018
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.