Regression Dynamic Programming for Multiple-Reservoir Control

In stochastic dynamic programming models of reservoir control problems, the continuous state space typically is discretized and the optimal value function is computed at the state space grid points. Values of the optimal value function between these discrete grid points can be obtained by interpolation. Discretization of the state space for high dimensional problems, which may arise due to the large number of reservoirs in the system or multiple state variables for each reservoir, results in a prohibitively large computational requirement. A "Regression Dynamic Programming" approach is developed for approximating the optimal value function using penalized regression splines fitted over a subset of the full state space grid. Selection of the subset of state space grid points is made based on statistical methods of experimental design, one of which is Latin hypercube sampling. The Regression Dynamic Programming approach is demonstrated using a hypothetical reservoir system.