Markovian Models for Bridge Maintenance Management

The typical infrastructure maintenance decision‐making environment involves multiple objectives and uncertainty, and is dynamic. One of the most commonly used infrastructure models is a Markov decision process (MDP). MDP models have been applied to numerous sequential decision‐making situations involving uncertainty and multiple objectives, including applications related to infrastructure problems. In this paper we explore the use of Markov models for bridge management systems. In particular, we explore two critical issues associated with the use of MDP models. The first involves state‐space explosion, one of the most common problems with MDP models. We address the issues of state‐space cardinality and present approaches for dealing with the complexity. The second issue with MDP models is the compliance with the Markovian property. With both issues we use the Virginia bridge system and data to illustrate the concepts. Our research indicates that MDPs are a powerful and useful technique for bridge management systems; however, the data collection for repair and maintenance history can be improved in order to build more accurate and complete MDP‐based models.