Comparing Model-Based and Model-Free Streamflow Simulation Approaches to Improve Hydropower Reservoir Operations
Publication: Journal of Water Resources Planning and Management
Volume 144, Issue 3
Abstract
This paper presents a comparison between an inflow model-based and an inflow model-free optimization method applied to a hydropower system. Widely used stochastic dynamic programming (SDP) and the evolutionary multiobjective direct policy search (EMODPS) methods are used, respectively, as model-based and model-free methods. Main results show that the model-free approach provides a better representation of the complex inflow correlations. Stochastic dynamic programming suffers from the temporal decomposition of the problem that allows only autoregressive exogenous (ARX) models to be used. However, because inflow uncertainty is implicitly represented through simulation with EMODPS, long time series are required to accurately characterize the probability of all possible events. To tackle this drawback and to avoid only learning the data set, a new regularization framework is introduced to improve the policy robustness on unseen data sets. Moreover, this study highlights how the preselected family of functions defining the policy can reduce the performance of the EMODPS method. This study is based on the real-world case of Kemano, located in British Columbia, Canada. This system is challenging because of the long-term inflow correlation due to long snow accumulation periods and its multiple objectives.
Get full access to this article
View all available purchase options and get full access to this article.
References
Barron, A. R. (1993). “Universal approximation bounds for superpositions of a sigmoidal function.” IEEE Trans. Inf. Theory, 39(3), 930–945.
Bertsekas, D. P. (1995). Dynamic programming and optimal control, Vol. 1, Athena Scientific, Belmont, MA.
Bertsekas, D. P., and Tsitsiklis, J. N. (1996). Neuro-dynamic programming, 1st Ed., Athena Scientific, Belmont, MA.
Castelletti, A., Corani, G., Rizzolli, A., Soncini-Sessa, R., and Weber, E. (2002). “Reinforcement learning in the operational management of a water system.” IFAC Workshop on Modeling and Control in Environmental Issues, Keio Univ., Yokohama, Japan, 325–330.
Castelletti, A., de Rigo, D., Rizzoli, A. E., Soncini-Sessa, R., and Weber, E. (2007). “Neuro-dynamic programming for designing water reservoir network management policies.” Control Eng. Pract., 15(8), 1031–1038.
Castelletti, A., Pianosi, F., and Restelli, M. (2013). “A multiobjective reinforcement learning approach to water resources systems operation: Pareto frontier approximation in a single run.” Water Resour. Res., 49(6), 3476–3486.
Charbonneau, R., Fortin, J., and Morin, G. (1977). “The CEQUEAU model: Description and examples of its use in problems related to water resource management/Le modèle CEQUEAU: Description et exemples d’utilisation dans le cadre de problèmes reliés à l’aménagement.” Hydrol. Sci. J., 22(1), 193–202.
Côté, P., Haguma, D., Leconte, R., and Krau, S. (2011). “Stochastic optimisation of Hydro-Quebec hydropower installations: A statistical comparison between SDP and SSDP methods.” Can. J. Civ. Eng., 38(12), 1427–1434.
Davidsen, C., et al. (2015). “Optimizing basin-scale coupled water quantity and water quality management with stochastic dynamic programming.” Geophysical research abstracts, Vol. 17, European Geosciences Union, Munich, Germany, 6457.
Deb, K., Pratap, A., Agarwal, S., and Meyarivan, T. (2002). “A fast and elitist multiobjective genetic algorithm: NSGA-II.” IEEE Trans. Evol. Comput., 6(2), 182–197.
Desreumaux, Q. (2016). “Amélioration de la représentation des processus stochastiques pour l’optimisation appliquée à la gestion des systèmes hydriques.” Ph.D. thesis, Dept. of Civil Engineering, Univeristé de Sherbrooke, Québec.
Desreumaux, Q., Côté, P., and Leconte, R. (2014). “Role of hydrologic information in stochastic dynamic programming: A case study of the Kemano hydropower system in British Columbia.” Can. J. Civ. Eng., 41(9), 839–844.
Faber, B. A., and Stedinger, J. (2001). “Reservoir optimization using sampling SDP with ensemble streamflow prediction (ESP) forecasts.” J. Hydrol., 249(1), 113–133.
Giuliani, M., Castelletti, A., Pianosi, F., Mason, E., and Reed, P. M. (2015). “Curses, tradeoffs, and scalable management: Advancing evolutionary multiobjective direct policy search to improve water reservoir operations.” J. Water Resour. Plann. Manage., 04015050.
Giuliani, M., Herman, J., Castelletti, A., and Reed, P. (2014). “Many-objective reservoir policy identification and refinement to reduce policy inertia and myopia in water management.” Water Resour. Res., 50(4), 3355–3377.
Hadka, D., and Reed, P. (2012). “Diagnostic assessment of search controls and failure modes in many-objective evolutionary optimization.” Evol. Comput., 20(3), 423–452.
Kim, Y.-O., and Palmer, R. N. (1997). “Value of seasonal flow forecasts in Bayesian stochastic programming.” J. Water Resour. Plann. Manage., 327–335.
Knowles, J., and Corne, D. (2002). “On metrics for comparing nondominated sets.” Proc., 2002 Congress on Evolutionary Computation, 2002. CEC’02, Vol. 1, IEEE, New York, 711–716.
Mordatch, I., Lowrey, K., Andrew, G., Popovic, Z., and Todorov, E. V. (2015). “Interactive control of diverse complex characters with neural networks.” Advances in neural information processing systems 28, C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, eds., Curran Associates, Inc., Red Hook, NY, 3114–3122.
Oliveira, R., and Loucks, D. P. (1997). “Operating rules for multireservoir systems.” Water Resour. Res., 33(4), 839–852.
Pianosi, F., Thi, X. Q., and Soncini-Sessa, R. (2011). “Artificial neural networks and multi objective genetic algorithms for water resources management: An application to the Hoabinh reservoir in Vietnam.” IFAC Proc. Volumes, 44(1), 10579–10584.
Salazar, J. Z., Reed, P. M., Herman, J. D., Giuliani, M., and Castelletti, A. (2016). “A diagnostic assessment of evolutionary algorithms for multi-objective surface water reservoir control.” Adv. Water Resour., 92, 172–185.
Soncini-Sessa, R., Weber, E., and Castelletti, A. (2007). Integrated and participatory water resources management-theory, Vol. 1, Elsevier, Amsterdam, Netherlands.
Tejada-Guibert, J. A., Johnson, S. A., and Stedinger, J. R. (1995). “The value of hydrologic information in stochastic dynamic programming models of a multireservoir system.” Water Resour. Res., 31(10), 2571–2579.
Turgeon, A. (2005). “Solving a stochastic reservoir management problem with multilag autocorrelated inflows.” Water Resour. Res., 41(12), in press.
Vincent, P., Larochelle, H., Bengio, Y., and Manzagol, P. A. (2008). “Extracting and composing robust features with denoising autoencoders.” Proc., 25th Int. Conf. on Machine Learning, Association for Computing Machinery, New York, 1096–1103.
Zoppoli, R., Sanguineti, M., and Parisini, T. (2002). “Approximating networks and extended Ritz method for the solution of functional optimization problems.” J. Optim. Theory Appl., 112(2), 403–440.
Information & Authors
Information
Published In
Journal of Water Resources Planning and Management
Volume 144 • Issue 3 • March 2018
Copyright
©2018 American Society of Civil Engineers.
History
Received: Aug 3, 2016
Accepted: Jul 11, 2017
Published online: Jan 9, 2018
Published in print: Mar 1, 2018
Discussion open until: Jun 9, 2018
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.