Performance of Implicit Stochastic Approaches to the Synthesis of Multireservoir Operating Rules
Publication: Journal of Water Resources Planning and Management
Volume 146, Issue 6
Abstract
With increasing pressure on water resources availability and dependability and constraints due to environmental concerns, the traditional approaches for defining reservoir management rules are often inadequate. In particular, in multireservoir systems, when multiple input variables (e.g., the storage of other reservoirs in the system, water demand in different districts) must be taken into account, it is almost impossible to figure out which shape the operating rule(s) could have. For these reasons, neural network (NN) based rules have been increasingly adopted in the last decade. NN-based rules are well known as universal approximators that can help determine the most interesting input variables, their mutual relations, and how they contribute to the definition of the optimal releases. Two approaches to the identification of neural management rules are discussed in the paper. The first solves a deterministic open-loop (i.e., with known inflows) problem and then identifies neural closed-loop policies using the classical regression method, so that the rules approximate as much as possible the solution found in the first step. The second approach, direct policy search, assumes that the operating rule is represented by an NN, the parameters of which are optimized directly by solving the optimal closed-loop problem. This work applies the two approaches to the case of the downstream portion of the Nile River basin system, which contains some large reservoirs, and for which several years of synthetic streamflows are available. The comparison of the two approaches highlights intrinsic differences, showing the benefits and disadvantages of each. In the specific case of the Nile, the first approach performs better in terms of global agricultural deficit and hydropower production.
Get full access to this article
View all available purchase options and get full access to this article.
Data Availability Statement
Some or all data, models, or code generated or used during the study are available from the corresponding author by request.
(Nile Basin model and FB approach implementation.)
Some or all data, models, or code used during the study were provided by a third party. Direct requests for these materials may be made to the provider, as indicated in the Acknowledgments.
(Synthetic series of the inflows and DPS approach implementation.)
Acknowledgments
The authors would like to thank Dr. M. Jeuland, Duke University, for providing the synthetic inflow time series of the Nile River basin, ing. L. Carelli, Euro-Mediterranean Center on Climate Change (CMCC), for implementing the DPS case, and the two anonymous reviewers for their constructive comments.
References
Ahmed, J. A., and A. K. Sarma. 2005. “Genetic algorithm for optimal operating policy of a multipurpose reservoir.” Water Resour. Manage. 19 (2): 145–161. https://doi.org/10.1007/s11269-005-2704-7.
Becker, L., and W. W.-G. Yeh. 1974. “Optimization of real time operation of a multiple-reservoir system.” Water Resour. Res. 10 (6): 1107–1112. https://doi.org/10.1029/WR010i006p01107.
Bellman, R. 1957. Dynamic programming. Princeton, NJ: Princeton University Press.
Bertsekas, D. P., and J. N. Tsitsiklis. 1995. “Neuro-dynamic programming: An overview.” In Vol. 1 of Proc., 34th IEEE Conf. on Decision and Control, 560–564. New York: IEEE.
Braga, B. P., Jr., W. W.-G. Yen, L. Becker, and M. T. Barros. 1991. “Stochastic optimization of multiple-reservoir-system operation.” J. Water Resour. Plann. Manage. 117 (4): 471–481. https://doi.org/10.1061/(ASCE)0733-9496(1991)117:4(471).
Castelletti, A., S. Galelli, M. Restelli, and R. Soncini-Sessa. 2010. “Treebased reinforcement learning for optimal water reservoir operation.” Water Resour. Res. 46 (9): W09507. https://doi.org/10.1029/2009WR008898.
Castelletti, A., F. Pianosi, and M. Restelli. 2013. “A multiobjective reinforcement learning approach to water resources systems operation: Pareto frontier approximation in a single run.” Water Resour. Res. 49 (6): 3476–3486. https://doi.org/10.1002/wrcr.20295.
Celeste, A., and M. Billib. 2009. “Evaluation of stochastic reservoir operation optimization models.” Adv. Water Resour. 32 (9): 1429–1443. https://doi.org/10.1016/j.advwatres.2009.06.008.
Celeste, A. B., W. F. Curi, and R. C. Curi. 2009. “Implicit stochastic optimization for deriving reservoir operating rules in semiarid Brazil.” Pesquisa Operacional 29 (1): 223–234. https://doi.org/10.1590/S0101-74382009000100011.
Cervellera, C., V. C. Chen, and A. Wen. 2006. “Optimization of a large-scale water reservoir network by stochastic dynamic programming with efficient state space discretization.” Eur. J. Oper. Res. 171 (3): 1139–1151. https://doi.org/10.1016/j.ejor.2005.01.022.
Chandramouli, V., and H. Raman. 2001. “Multireservoir modeling with dynamic programming and neural networks.” J. Water Resour. Plann. Manage. 127 (2): 89–98. https://doi.org/10.1061/(ASCE)0733-9496(2001)127:2(89).
Côté, P., and R. Arsenault. 2019. “Efficient implementation of sampling stochastic dynamic programming algorithm for multireservoir management in the hydropower sector.” J. Water Resour. Plann. Manage. 145 (4): 05019005. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001050.
Cybenko, G. 1989. “Approximation by superpositions of a sigmoidal function.” Math. Control Signals Syst. 2 (4): 303–314. https://doi.org/10.1007/BF02551274.
Dariane, A. B., and S. Momtahen. 2009. “Optimization of multireservoir systems operation using modified direct search genetic algorithm.” J. Water Resour. Plann. Manage. 135 (3): 141–148. https://doi.org/10.1061/(ASCE)0733-9496(2009)135:3(141).
Deb, K., A. Pratap, S. Agarwal, and T. A. M. T. Meyarivan. 2002. “A fast and elitist multiobjective genetic algorithm: NSGA-II.” IEEE Trans. Evol. Comput. 6 (2): 182–197. https://doi.org/10.1109/4235.996017.
De Rigo, D., A. E. Rizzoli, R. Soncini-Sessa, E. Weber, and P. Zenesi. 2001. “Neuro-dynamic programming for the efficient management of reservoir networks.” In Vol. 4 of Proc., MODSIM 2001, Int. Congress on Modelling and Simulation, 1949–1954. Acton, ACT: Modelling and Simulation Society of Australia.
Desreumaux, Q., P. Côté, and R. Leconte. 2018. “Comparing model-based and model-free streamflow simulation approaches to improve hydropower reservoir operations.” J. Water Resour. Plann. Manage. 144 (3): 05018002. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000860.
Dobson, B., T. Wagener, and F. Pianosi. 2019. “An argument-driven classification and comparison of reservoir operation optimization methods.” Adv. Water Resour. 128 (Jun): 74–86. https://doi.org/10.1016/j.advwatres.2019.04.012.
El-Fadel, M., Y. El-Sayegh, K. El-Fadl, and D. Khorbotly. 2003. “The Nile River basin: A case study in surface water conflict resolution.” J. Nat. Resour. Life Sci. Educ. 32: 107–117.
El-Shafie, A., M. Reda Taha, and A. Noureldin. 2007. “A neuro-fuzzy model for inflow forecasting of the Nile river at Aswan high dam.” Water Resour. Manage. 21 (3): 533–556. https://doi.org/10.1007/s11269-006-9027-1.
Galelli, S., and A. Castelletti. 2013. “Tree-based iterative input variable selection for hydrological modeling.” Water Resour. Res. 49 (7): 4295–4310. https://doi.org/10.1002/wrcr.20339.
Galelli, S., G. B. Humphrey, H. R. Maier, A. Castelletti, G. C. Dandy, and M. S. Gibbs. 2014. “An evaluation framework for input variable selection algorithms for environmental data-driven models.” Environ. Modell. Software 62 (Dec): 33–51. https://doi.org/10.1016/j.envsoft.2014.08.015.
Georgakakos, A. P., and D. H. Marks. 1987. “A new method for the real-time operation of reservoir systems.” Water Resour. Res. 23 (7): 1376–1390. https://doi.org/10.1029/WR023i007p01376.
Giuliani, M., A. Castelletti, F. Pianosi, E. Mason, and P. M. Reed. 2015a. “Curses, tradeoffs, and scalable management: Advancing evolutionary multiobjective direct policy search to improve water reservoir operations.” J. Water Resour. Plann. Manage. 142 (2): 04015050. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000570.
Giuliani, M., J. D. Herman, A. Castelletti, and P. M. Reed. 2014a. “Many-objective reservoir policy identification and refinement to reduce policy inertia and myopia in water management.” Water Resour. Res. 50 (4): 3355–3377. https://doi.org/10.1002/2013WR014700.
Giuliani, M., Y. Li, A. Cominola, S. Denaro, E. Mason, and A. Castelletti. 2016. “A Matlab toolbox for designing multi-objective optimal operations of water reservoir systems.” Environ. Modell. Software 85 (Nov): 293–298. https://doi.org/10.1016/j.envsoft.2016.08.015.
Giuliani, M., E. Mason, A. Castelletti, F. Pianosi, and R. Soncini-Sessa. 2014b. “Universal approximators for direct policy search in multi-purpose water reservoir management: A comparative analysis.” IFAC Proc. Volume 47 (3): 6234–6239. https://doi.org/10.3182/20140824-6-ZA-1003.01962.
Giuliani, M., F. Pianosi, and A. Castelletti. 2015b. “Making the most of data: An information selection and assessment framework to improve water systems operations.” Water Resour. Res. 51 (11): 9073–9093. https://doi.org/10.1002/2015WR017044.
Goor, Q., C. Halleux, Y. Mohamed, and A. Tilmant. 2010. “Optimal operation of a multipurpose multireservoir system in the Eastern Nile River Basin.” Hydrol. Earth Syst. Sci. 14 (10): 1895–1908. https://doi.org/10.5194/hess-14-1895-2010.
Guariso, G., S. Rinaldi, and R. Soncini-Sessa. 1986. “The management of Lake Como: A multiobjective analysis.” Water Resour. Res. 22 (2): 109–120. https://doi.org/10.1029/WR022i002p00109.
Guariso, G., and D. Whittington. 1987. “Implications of Ethiopian water development for Egypt and Sudan.” Int. J. Water Resour. Dev. 3 (2): 105–114. https://doi.org/10.1080/07900628708722338.
Hadka, D., and P. Reed. 2013. “Borg: An auto-adaptive many-objective evolutionary computing framework.” Evol. Comput. 21 (2): 231–259. https://doi.org/10.1162/EVCO_a_00075.
Hall, W. A., W. S. Butcher, and A. Esogbue. 1968. “Optimization of the operation of a multiple-purpose reservoir by dynamic programming.” Water Resour. Res. 4 (3): 471–477. https://doi.org/10.1029/WR004i003p00471.
Hornik, K. 1991. “Approximation capabilities of multilayer feedforward networks.” Neural Netw. 4 (2): 251–257. https://doi.org/10.1016/0893-6080(91)90009-T.
Hornik, K., M. Stinchcombe, and H. White. 1989. “Multilayer feedforward networks are universal approximators.” Neural Netw. 2 (5): 359–366. https://doi.org/10.1016/0893-6080(89)90020-8.
HydroWorld. 2018. “Sudan, Egypt and Ethiopia hold summit on 6,000-MW Grand Ethiopian renaissance dam.” Accessed April 23, 2018. www.hydroworld.com.
Jeuland, M. 2009. “Planning water resources development in an uncertain climate future: A hydroeconomic simulation framework applied to the case of the Blue Nile.” Ph.D. dissertation, Dept. of Environmental Sciences and Engineering, Univ. of North Carolina.
Jeuland, M., and D. Whittington. 2014. “Water resources planning under climate change: Assessing the robustness of real options for the Blue Nile.” Water Resour. Res. 50 (3): 2086–2107. https://doi.org/10.1002/2013WR013705.
Jeuland, M., X. Wu, and D. Whittington. 2017. “Infrastructure development and the economics of cooperation in the eastern Nile.” Water Int. 42 (2): 121–141. https://doi.org/10.1080/02508060.2017.1278577.
Kim, T., J. H. Heo, and J. Yi. 2000. “Monthly reservoir operating rules generated by implicit stochastic optimization.” In Proc., Watershed Management and Operations Management Conf. Reston, VA: ASCE.
Koutsoyiannis, D., and A. Economou. 2003. “Evaluation of the parametrization-simulation-optimization approach for the control of reservoir systems.” Water Resour. Res. 39 (6): 1170. https://doi.org/10.1029/2003WR002148.
Labadie, J. W. 2004. “Optimal operation of multireservoir systems: State-of-the-art review.” J. Water Resour. Plann. Manage. 130 (2): 93–111. https://doi.org/10.1061/(ASCE)0733-9496(2004)130:2(93).
Lee, J. H., and J. W. Labadie. 2007. “Stochastic optimization of multireservoir systems via reinforcement learning.” Water Resour. Res. 43 (11): W11408. https://doi.org/10.1029/2006WR005627.
Levy, B. S., and G. B. Baecher. 1999. “Nilesim: A windows-based hydrologic simulator of the Nile river basin.” J. Water Resour. Plann. Manage. 125 (2): 100–106. https://doi.org/10.1061/(ASCE)0733-9496(1999)125:2(100).
Li, X., S. Guo, P. Liu, and G. Chen. 2010. “Dynamic control of flood limited water level for reservoir operation by considering inflow uncertainty.” J. Hydrol. 391 (1–2): 124–132. https://doi.org/10.1016/j.jhydrol.2010.07.011.
Liao, Y., S.-C. Fang, and H. L. W. Nuttle. 2003. “Relaxed conditions for radial-basis function networks to be universal approximators.” Neural Netw. 16 (7): 1019–1028. https://doi.org/10.1016/S0893-6080(02)00227-7.
Libisch-Lehner, C., T. T. N. Hung, R. Taormina, H. P. Nachtnebel, and S. Galelli. 2019. “On the value of ENSO state for urban water supply system operators: Opportunities, trade-offs, and challenges.” Water Resour. Res. 55 (4): 2856–2875. https://doi.org/10.1029/2018WR023622.
Liu, P., X. Cai, and S. Guo. 2011. “Deriving multiple near-optimal solutions to deterministic reservoir operation problems.” Water Resour. Res. 47 (8): W08506. https://doi.org/10.1029/2011WR010998.
Liu, P., L. Li, G. Chen, and D. E. Rheinheimer. 2014. “Parameter uncertainty analysis of reservoir operating rules based on implicit stochastic optimization.” J. Hydrol. 514 (Jun): 102–113. https://doi.org/10.1016/j.jhydrol.2014.04.012.
Luchner, J., N. D. Riegels, and P. Bauer-Gottwein. 2019. “Benefits of cooperation in transnational water-energy systems.” J. Water Resour. Plann. Manage. 145 (5): 05019007. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001047.
Melesse, A. M., W. Abtew, G. Shimelis, S. G. Setegn, eds. 2014. Nile River Basin ecohydrological challenges, climate change and hydropolitics. Cham, Switzerland: Springer International Publishing.
Mizyed, N. R., J. C. Loftis, and D. G. Fontane. 1992. “Operation of large multireservoir systems using optimal-control theory.” J. Water Resour. Plann. Manage. 118 (4): 371–387. https://doi.org/10.1061/(ASCE)0733-9496(1992)118:4(371).
Momtahen, S., and A. B. Dariane. 2007. “Direct search approaches using genetic algorithms for optimization of water reservoir operating policies.” J. Water Resour. Plann. Manage. 133 (3): 202–209. https://doi.org/10.1061/(ASCE)0733-9496(2007)133:3(202).
Moriarty, D. E., and A. C. Schultz. 1999. “Evolutionary algorithms for reinforcement learning.” J. Artif. Intell. Res. 11: 241–276. https://doi.org/10.1613/jair.613.
Murray, D. M., and S. J. Yakowitz. 1979. “Constrained differential dynamic programming and its application to multireservoir control.” Water Resour. Res. 15 (5): 1017–1027. https://doi.org/10.1029/WR015i005p01017.
Nandalal, K., and J. J. Bogardi. 2007. Dynamic programming based operation of reservoirs: Applicability and limits. Cambridge, MA: Cambridge University Press.
Raso, L., M. Chiavico, and D. Dorchies. 2019. “Optimal and centralized reservoir management for drought and flood protection on the Upper Seine–Aube river system using stochastic dual dynamic programming.” J. Water Resour. Plann. Manage. 145 (3): 05019002. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001040.
Reed, P. M., D. Hadka, J. D. Herman, J. R. Kasprzyk, and J. B. Kollat. 2013. “Evolutionary multiobjective optimization in water resources: The past, present and future.” Adv. Water Resour. 51 (Jan): 438–456. https://doi.org/10.1016/j.advwatres.2012.01.005.
Sangiorgio, M. 2016. “A neural approach for multi-reservoir system operation.” M.Sc. thesis, Dept. of Electronics, Information and Bioengineering, Politecnico di Milano.
Sangiorgio, M., and G. Guariso. 2018. “NN-based implicit stochastic optimization of multi-reservoir systems management.” Water 10 (3): 303. https://doi.org/10.3390/w10030303.
Scholz, M., M. Fraunholz, and J. Selbig. 2008. “Nonlinear principal component analysis: Neural network models and applications.” In Principal manifolds for data visualization and dimension reduction, 44–67. Berlin: Springer.
Silver, D., G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller. 2014. “Deterministic policy gradient algorithms.” In Vol. 32 of Proc., 31st Int. Conf. Machine Learning. Brookline, MA: Microtome Publishing.
Soncini-Sessa, R., E. Weber, and A. Castelletti. 2007. Integrated and participatory water resources management-theory. Amsterdam, Netherlands: Elsevier.
Sutton, R. S., D. A. McAllester, S. P. Singh, and Y. Mansour. 2000. “Policy gradient methods for reinforcement learning with function approximation.” In Vol. 12 of Advances in neural information processing systems, 1057–1063. Cambridge, MA: MIT Press.
Tingsanchali, T., and T. Boonyasirikul. 2006. “Stochastic dynamic programming with risk consideration for transbasin diversion system.” J. Water Resour. Plann. Manage. 132 (2): 111–121. https://doi.org/10.1061/(ASCE)0733-9496(2006)132:2(111).
Turgeon, A. 1980. “Optimal operation of multireservoir power systems with stochastic inflows.” Water Resour. Res. 16 (2): 275–283. https://doi.org/10.1029/WR016i002p00275.
Whittington, D., and G. Guariso. 1983. Water management models in practice: A case study of the Aswan High Dam. Amsterdam, Netherlands: Elsevier.
Whittington, D., and E. McClelland. 1992. “Opportunities for regional and international cooperation in the Nile basin.” Water Int. 17 (3): 144–154. https://doi.org/10.1080/02508069208686134.
Whittington, D., J. Waterbury, and M. Jeuland. 2014. “The grand renaissance dam and prospects for cooperation on the Eastern Nile.” Water Policy 16 (4): 595–608. https://doi.org/10.2166/wp.2014.011b.
Whittington, D., X. Wu, and C. Sadoff. 2005. “Water resources management in the Nile basin: The economic value of cooperation.” Water Policy 7 (3): 227–252. https://doi.org/10.2166/wp.2005.0015.
Wu, X., S. Li, C. Cheng, S. Miao, and Q. Ying. 2019. “Simulation-optimization model to derive operation rules of multiple cascaded reservoirs for nash equilibrium.” J. Water Resour. Plann. Manage. 145 (5): 04019013. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001057.
Yao, H., and A. Georgakakos. 2003. Nile decision support tool river simulation and management. Atlanta: Georgia Tech.
Yeh, W. W. G. 1985. “Reservoir management and operations models: A state-of-the-art review.” Water Resour. Res. 21 (12): 1797–1818. https://doi.org/10.1029/WR021i012p01797.
Yitayew, M., and A. M. Melesse. 2011. “Critical water resources issues in the Nile River Basin.” In Nile river basin, 404–416. Dordrecht, Netherlands: Springer.
Zatarain Salazar, J., P. M. Reed, J. D. Herman, M. Giuliani, and A. Castelletti. 2016. “A diagnostic assessment of evolutionary algorithms for multi-objective surface water reservoir control.” Adv. Water Resour. 92 (Jun): 172–185. https://doi.org/10.1016/j.advwatres.2016.04.006.
Zhou, Y., S. Guo, P. Liu, C. Y. Xu, and X. Zhao. 2016. “Derivation of water and power operating rules for multi-reservoirs.” Hydrol. Sci. J. 61 (2): 359–370. https://doi.org/10.1080/02626667.2015.1035656.
Information & Authors
Information
Published In
Journal of Water Resources Planning and Management
Volume 146 • Issue 6 • June 2020
Copyright
©2020 American Society of Civil Engineers.
History
Received: Nov 8, 2018
Accepted: Nov 15, 2019
Published online: Mar 27, 2020
Published in print: Jun 1, 2020
Discussion open until: Aug 27, 2020
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.