Network-Level Infrastructure Management Using Approximate Dynamic Programming
Publication: Journal of Infrastructure Systems
Volume 16, Issue 2
Abstract
This research introduces the use of approximate dynamic programming to overcome a variety of limitations of distinct infrastructure management problem formulations. The form, as well as the parameters, of a model specifying the long-term costs associated with alternate infrastructure maintenance policies are learned via simulation. The introduced methodology makes it possible to manage large heterogeneous networks of facilities related by budgetary restrictions and resource constraints as well as by dependencies in maintenance costs or deterioration. In addition, the methodology is particularly well suited to consideration of multiple types of infrastructure condition data at the same time, including continuous-valued data and relevant historical data. Introduced techniques will prove valuable when high-quality deterioration and cost estimation models are available but are ill suited for use in a Markov decision problem framework. Computational studies show that the introduced approach is able to find an optimal solution to a relatively simple infrastructure management problem, and is able to find increasingly good solutions to a more complex problem.
Get full access to this article
View all available purchase options and get full access to this article.
References
Carnahan, J., Davis, W., Shahin, M., Kean, P., and Wu, M. (1987). “Optimal maintenance decisions for pavement management.” J. Transp. Eng., 113(5), 554–572.
Chan, W., Fwa, T., and Tan, J. (2003). “Optimal fund-allocation analysis for multidistrict highway agencies.” J. Infrastruct. Syst., 9(4), 167–175.
Chu, C. -Y., and Durango-Cohen, P. (2007). “Estimation of infrastructure performance models using state-space specifications of time series models.” Transp. Res., Part C: Emerg. Technol., 15(1), 17–32.
Dekker, R., Wildeman, R., and Duyn Schouten, F. (1997). “A review of multi-component maintenance models with economic dependence.” Mathematical Models of Operations Research, 45, 411–534.
Durango, P., and Madanat, S. (2002). “Optimal maintenance and repair policies in infrastructure management under uncertain facility deterioration rates: An adaptive control approach.” Transp. Res., Part A: Policy Pract., 36(9), 763–778.
Durango-Cohen, P. (2004). “Maintenance and repair decision making for infrastructure facilities without a deterioration model.” J. Infrastruct. Syst., 10(1), 1–8.
Durango-Cohen, P., and Sarutipand, P. (2007). “Capturing interdependencies and heterogeneity in the management of multifacility transportation infrastructure systems.” J. Infrastruct. Syst., 13(2), 115–123.
Golabi, K., Kulkarni, R., and Way, G. (1982). “A statewide pavement management system.” Interfaces, 12(6), 5–21.
Golabi, K., and Shepard, R. (1997). “Pontis: A system for maintenance optimization and improvement of US bridge network.” Interfaces, 27(1), 71–88.
Gopal, S., and Majidzadeh, K. (1991). “Application of Markov decision processes to level-of-service-based systems.” Transp. Res. Rec., 1304, 12–17.
Madanat, S., Park, S., and Kuhn, K. (2006). “Adaptive optimization and systematic probing of infra-structure system maintenance policies under model uncertainty.” J. Infrastruct. Syst., 12(3), 192–198.
Mishalani, R., and Madanat, S. (2002). “Computation of infrastructure transition probabilities using stochastic duration models.” J. Infrastruct. Syst., 8(4), 139–148.
Parra-Hernández, R., and Dimoloulos, N. (2005). “A new heuristic for solving the multichoice multidimensional knapsack problem.” IEEE Trans. Syst. Man Cybern., Part A. Syst. Humans, 35(5), 708–717.
Powell, W. (2007). Approximate dynamic programming: Solving the curses of dimensionality, Wiley, New York.
Robelin, C. -A., and Madanat, S. (2007). “History-dependent bridge deck maintenance and replacement optimization with Markov decision processes.” J. Infrastruct. Syst., 13(3), 195–201.
Simão, H., Day, J., George, A., Gifford, T., Nienow, J., and Powell, W. (2009). “An approximate dynamic programming algorithm for large-scale fleet management: A case application.” Transportation Science, 43(2), 178–197.
Sutton, R., and Barto, A. (1998). Reinforcement learning, MIT Press, Cambridge, Mass.
Information & Authors
Information
Published In
Journal of Infrastructure Systems
Volume 16 • Issue 2 • June 2010
Pages: 103 - 111
Copyright
© 2010 ASCE.
History
Received: Apr 28, 2009
Accepted: Jul 31, 2009
Published online: Aug 3, 2009
Published in print: Jun 2010
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.