Comparing Sigmoid Transfer Functions for Neural Network Multistep Ahead Streamflow Forecasting
Publication: Journal of Hydrologic Engineering
Volume 15, Issue 4
Abstract
One of the main problems of neural networks is the lack of consensus on how to best implement them. This work targets the question of the transfer function selection—a vital part of neural network providing nonlinear mapping potential. Three nonlinear transfer functions bounded by −1 and 1 are selected for testing, based on a literature review: the Elliott sigmoid, the bipolar sigmoid, and the tangent sigmoid. They are used to design multilayer perceptron neural networks for multistep ahead streamflow forecasting over five diverse watersheds and lead times from 1 to 5 days. All multilayer perceptrons have shown a good performance on the account of the four selected criteria, which confirms that the selected multilayer perceptron implementation procedure was adequate, namely, the data set length, the Kohonen network clustering method to create the training and testing sets, and the Levenberg-Marquardt back-propagation training procedure with Bayesian regularization. Specifically, results endorsed the tangent sigmoid as the most pertinent transfer function for streamflow forecasting, over the bipolar (logistic) and Elliott sigmoids, but the latter requires less computing time and as such may be a valuable option for operational hydrology. Also, results averaged over five lead times confirmed the universal approximation theorem that a linear transfer function is suitable for the output layer—a nonlinear transfer function in the output layer failed to improve performance values.
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments
Financial support for the undertaking of this work has been provided by Hydro-Québec and by the Natural Science and Engineering Research Council of Canada.
References
Anctil, F., and Lauzon, N. (2004). “Generalisation for neural networks through data sampling and training procedures, with applications to streamflow predictions.” Hydrology Earth Syst. Sci., 8(5), 940–958.
Anctil, F., Michel, C., Perrin, C., and Andréassian, V. (2004a). “A soil moisture index as an auxiliary ANN input for stream flow forecasting.” J. Hydrol., 286, 155–167.
Anctil, F., Perrin, C., and Andreassian, V. (2004b). “Impact of the length of observed records on the performance of ANN and of conceptual parsimonious rainfall-runoff forecasting models.” Environ. Modell. Software, 19(4), 357–368.
ASCE. (2000). “Artificial neural networks in hydrology. I: Preliminary concepts.” J. Hydrol. Eng., 5(2), 115–123.
Bonafe, A., Galeati, G., and Sforna, M. (1994). “Neural networks for daily mean flow forecasting.” Hydraulic engineering software, V. W. R. Blain and K. L. Katsifarakis, eds., Vol. 1, Computational Mechanics Publications, Southampton, U.K., 131–138.
Bowden, G. J., Dandy, G. C., and Maier, H. R. (2005). “Input determination for neural network models in water resources applications. Part 1—Background and methodology.” J. Hydrol., 301(1–4), 75–92.
Carriere, P., Mohaghegh, S., and Gaskari, R. (1996). “Performance of a virtual runoff hydrograph system.” J. Water Resour. Plann. Manage., 122(6), 421–427.
Coulibaly, P., Anctil, F., and Bobée, B. (1999). “Hydrological forecasting with artificial neural networks: The state of the art.” Can. J. Civ. Eng., 26(3), 293–304.
Coulibaly, P., Anctil, F., and Bobée, B. (2000). “Daily reservoir inflow forecasting using artificial neural networks with stopped training approach.” J. Hydrol., 230, 244–257.
Dawson, C. W., and Wilby, R. (1998). “An artificial neural network approach to rainfall-runoff modelling.” Hydrol. Sci. J., 43(1), 47–66.
Dawson, C. W., and Wilby, R. L. (2001). “Hydrological modelling using artificial neural networks.” Prog. Phys. Geogr., 25(1), 80–108.
Elliott, D. L. (1993). A better activation function for artificial neural networks, Institute for Systems Research, Univ. of Maryland, College Park, Md.
Fahlman, S. E., and Lebiere, C. (1991). The cascade-correlation learning architecture, School of Computer Science, Carnegie Mellon Univ., Pittsburgh.
Foresse, F. D., and Hagan, M. T. (1997). “Gauss-Newton approximation to Bayesian regularization.” Proc., 1997 Int. Joint Conf. on Neural Networks, Houston, 1930–1935.
Franchini, M., and Pacciani, M. (1991). “Comparative-analysis of several conceptual rainfall runoff models.” J. Hydrol., 122(1–4), 161–219.
Hornik, K., Stinchcombe, M., and White, H. (1989). “Multilayer feedforward networks are universal approximators.” Neural Networks, 2(5), 359–366.
Hsu, K. L., Gupta, H. V., and Sorooshian, S. (1995). “Artificial neural-network modeling of the rainfall-runoff process.” Water Resour. Res., 31(10), 2517–2530.
Hughes, D. A. (1994). “Soil moisture and runoff simulations using four catchment rainfall-runoff models.” J. Hydrol., 158(3–4), 381–404.
Imrie, C. E., Durucan, S., and Korre, A. (2000). “River flow prediction using artificial neural networks: Generalisation beyond the calibration range.” J. Hydrol., 233(1–4), 138–153.
Jayawardena, A. W., Fernando, D. A. K., and Zhou, M. C.(1997). “Comparison of multilayer perceptron and radial basis function networks as tools for flood forecasting.” Destructive water: Water-caused natural disaster, their abatement and control, IAHS Publ. 239, Wallingford, U.K.
Jiang, N., Zhao, Z., and Ren, L. (2003). “Design of structural modular neural networks with genetic algorithm.” Adv. Eng. Software, 34(1), 17–24.
Kim, D., Kim, H., and Chung, D. (2005). “A modified genetic algorithm for fast training neural networks.” Lect. Notes Comput. Sci., 3496, 660–665.
Klemeš, V. (1986). “Dilettantism in hydrology: Transition or destiny?” Water Resour. Res., 22(9S), 177S–188S.
Kohonen, T. (1990). “The self-organizing map.” Proc. IEEE, 78(9), 1464–1480.
LeCun, Y. (1985). “Une procedure d’apprentissage pour reseau a seuil asymmetrique.” Proc., Cognitiva 85, Paris, 599–604 (in French).
MacKay, D. J. C. (1992). “A practical Bayesian framework for backpropagation networks.” Neural Comput., 4(3), 448–472.
Maier, H. R., and Dandy, G. C. (1996). “The use of artificial neural networks for the prediction of water quality parameters.” Water Resour. Res., 32(4), 1013–1022.
Maier, H. R., and Dandy, G. C. (2000). “Neural networks for the prediction and forecasting of water resources variables: A review of modelling issues and applications.” Environ. Modell. Software, 15, 101–124.
Minns, A. W., and Hall, M. J. (1996). “Artificial neural networks as rainfall-runoff models.” Hydrol. Sci. J., 41(3), 399–417.
Naef, F. (1981). “Can we model the rainfall-runoff process today.” Hydrol. Sci. Bull., 26(3), 281–289.
Nash, J. E., and Sutcliffe, J. V. (1970). “River flow forecasting through conceptual models: Part I—A discussion of principles.” J. Hydrol., 10(3), 282–290.
Parker, D. B. (1985). Learning logic: Casting the cortex of the human brain in silicon, Center for Computational Research in Economics and Management Science, Cambridge, Mass.
Perrin, C. (2000). “Vers une amélioration d’un modèle global pluie-débit au travers d’une approche comparative.” Ph.D. thesis, Institut National Polytechnique de Grenoble, Grenoble, France (in French).
Perrin, C., Michel, C., and Andreassian, V. (2001). “Does a large number of parameters enhance model performance? Comparative assessment of common catchment model structures on 429 catchments.” J. Hydrol., 242(3–4), 275–301.
Perrin, C., Michel, C., and Andreassian, V. (2003). “Improvement of a parsimonious model for streamflow simulation.” J. Hydrol., 279(1–4), 275–289.
Prechelt, L. (1998). “Automatic early stopping using cross validation: Quantifying the criteria.” Neural Networks, 11(4), 761–767.
Reed, R. D., and Marks, R. J. (1998). Neural smithing: Supervised learning in feedforward artificial neural networks, MIT Press, Cambridge, Mass.
Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986). Learning internal representations by error propagation: Parallel distributed processing: Explorations in the microstructures of cognition, MIT Press, Cambridge, Mass., 318–362.
Senbeta, D. A., Shamseldin, A. Y., and O’Connor, K. M. (1999). “Modification of the probability-distributed interacting storage capacity model.” J. Hydrol., 224(3–4), 149–168.
Shamseldin, A. Y., Nasr, A. E., and O’Connor, K. M. (2002). “Comparison of different forms of the multi-layer feed-forward neural network method used for river flow forecasting.” Hydrology Earth Syst. Sci., 6(4), 671–684.
Shamseldin, A. Y., O’Connor, K. M., and Liang, G. C. (1997). “Methods for combining the outputs of different rainfall-runoff models.” J. Hydrol., 197(1–4), 203–229.
Shrestha, R. R., Theobald, S., and Nestmann, F. (2005). “Simulation of flood flow in a river system using artificial neural networks.” Hydrology Earth Syst. Sci., 9(4), 313–321.
Smith, J., and Eli, R. N. (1995). “Neural-network models of rainfall-runoff process.” J. Water Resour. Plann. Manage., 121(6), 499–508.
Tan, Y. H., and Van Cauwenberghe, A. (1999). “Neural-network-based -step-ahead predictors for nonlinear systems with time delay.” Eng. Applic. Artif. Intell., 12(1), 21–35.
Tokar, A. S., and Markus, M. (2000). “Precipitation-runoff modeling using artificial neural networks and conceptual models.” J. Hydrol. Eng., 5(2), 156–161.
Toth, E., and Brath, A. (2007). “Multistep ahead streamflow forecasting: Role of calibration data in conceptual and neural network modeling.” Water Resour. Res., 43(11), W11405.
Werbos, P. J. (1974). “Beyond regression: New tools for prediction and analysis in the behavioral sciences.” Ph.D. thesis, Harvard Univ., Cambridge, Mass.
Wolpert, D. H. (1992). “Stacked generalization.” Neural Networks, 5(2), 241–259.
World Meteorological Organization (WMO). (1975). “Intercomparison of conceptual models used in operational hydrological forecasting.” Rep. No. 7, Geneva.
World Meteorological Organization (WMO). (1992). “Simulated real-time intercomparison of hydrological models.” Rep. No. 38, Geneva.
Ye, W., Bates, B. C., Viney, N. R., Sivapalan, M., and Jakeman, A. J. (1997). “Performance of conceptual rainfall-runoff models in low-yielding ephemeral catchments.” Water Resour. Res., 33(1), 153–166.
Information & Authors
Information
Published In
Journal of Hydrologic Engineering
Volume 15 • Issue 4 • April 2010
Pages: 275 - 283
Copyright
© 2010 ASCE.
History
Received: Apr 21, 2008
Accepted: Sep 24, 2009
Published online: Mar 15, 2010
Published in print: Apr 2010
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.