Optimal Repair Policies for Infrastructure Systems with Life Cycle Cost Minimization and Annual Cost Leveling

In the field of infrastructure asset management, various methodologies based on mathematical models, optimization methods, and simulations have been developed for minimizing the expected life cycle cost of infrastructure systems that consist of multiple infrastructure facilities. When considering the repair policy for infrastructure systems comprising multiple infrastructure facilities, even if the expected annual repair cost does not change over the years, the uncertainty of the deterioration process causes variations in the timing of repair for each facility, and the actual annual repair cost of the system varies because of the finiteness of the number of the facilities in the system.

Here we study asset management systems that adopt the single-year budget principle, where the annual budget for repair work is usually fixed. When the optimal repair policy focusing only on minimizing the expected life cycle cost is applied, the actual repair cost may significantly differ from the given budget, and the possibly excessive difference between these amounts might hinder securing the budget and even lead to inefficient use of the latter. In a chain reaction, this might lead to underestimation of the next-year budget and consequently to an increased risk of deterioration because an underestimated budget might be insufficient for the necessary repair. Additionally, variation in the quantity of repair work not only affects the business conditions of the contractors by generating difficulties in securing order opportunities but also leads to difficulty in securing resources for repair work, including construction machines and material. Hence, a practicable, desirable repair policy should consider not only minimization of the life cycle cost but also leveling of the annual repair cost.

An intuitive concept of this study is represented in Fig. 1. However, although the figure illustrates deterministic deterioration processes, the proposed methodology assumes stochastic deterioration processes. In addition, we assume homogeneous stochastic deterioration processes (i.e., the same Markov transition probabilities) for each facility, but the deterioration processes realized according to the stochastic process can be heterogeneous as illustrated in the figure. In the figure, we consider an infrastructure system comprising two facilities and show the variation in the total cost of repairing them. It is assumed here that two repair methods are available as shown in the figure: corrective repair and preventive repair. The deterioration and repair process for each facility is represented by the time evolution of the condition state (CS). The graphs in the left column assume a situation where a repair policy that only considers minimizing the total repair cost (life cycle cost) is adopted. Here we suppose that a repair policy that executes only corrective repairs—the corrective repair policy—minimizes the life cycle cost. In this situation, the annual repair cost may vary over years, as shown in the bottom graph. The graphs in the right column assume a situation where a repair policy that considers cost leveling is adopted. In this situation, as shown in the bottom graph, by advancing the repairs to Facility 2, the variance in the annual repair cost is reduced; that is, cost leveling is achieved, although the life cycle cost is slightly increased. For cost leveling, the decision on whether or not to advance the repair of Facility 2 should be made by also considering the CS of Facility 1. This means that optimal repair policies for each facility are not guaranteed to be optimal for the system. Thus, the problem of optimizing the repair policy considering cost leveling becomes a network-level problem that determines the facility to be repaired according to the combination of the CSs of the individual facilities in the system. Additionally, advancing the repair of Facility 2 for cost leveling can be interpreted as executing preventive repair. Thus, it is likely that cost leveling can be attained through the effective execution of preventive repairs. Note that Fig. 1 assumes that preventive repairs for cost leveling are additionally executed in addition to the corrective repair policy, but in situations where preventive repairs are basically executed (e.g., a situation where a repair policy that includes the execution of preventive repairs minimizes the life cycle cost), cost leveling might be achieved by partially suppressing the execution of preventive repairs. It is presumed that the approximate solution method proposed in this study can be applied to such situations because it can also provide a solution that suppresses preventive repairs.

Although more details will be described in the literature review section, the following issues remain unresolved in state-of-the-art research on cost leveling: (1) there is no model to derive the optimal repair policy at the network level by considering by using a stochastic model to consider the uncertainty of the deterioration process of each facility in the system; and (2) there is no method to derive an optimal or approximate solution to the network-level cost-leveling problem with stochastic deterioration processes while eliminating combinatorial explosion. To bridge the gap, we set the objectives of this study as follows:

The remainder of this paper is organized into the following sections. In “Literature Review,” we review previous studies from two perspectives: optimization models and solution methods. In “Problem Formulation,” we formulate an optimal repair policy problem considering life cycle cost minimization and annual cost leveling. In “Determination of Repair Policy,” we propose the two solution methods. In “Numerical Study,” we present numerical studies of a small-scale system to compare the two solution methods and a large-scale system to investigate the performance of the proposed approximation method. In “Summary and Future Tasks,” we provide conclusions and future research directions. A list of notations is given at the end of this paper and can be referred to in reading the “Problem Formulation” and “Determination of Repair Policy” sections.

Here we review some mathematical optimization models for providing repair policies (also called management policies, maintenance policies, intervention policies, etc.) for infrastructure systems, focusing on the properties of their objective functions. They can be classified into project-level (also called object-level or facility-level) and network-level (also called system-level) models. In project-level models, the total repair cost or total social surplus is set as the objective function for a single facility (Friesz and Fernandez 1979; Madanat 1993; Madanat and Ben-Akiva 1994; Li and Madanat 2002; Brühwiler and Adey 2005; Ouyang and Madanat 2006; Jido et al. 2008). In network-level models, interdependencies among objects (facilities) are considered. For example, models have been proposed that consider the budget constraints of the entire system (Kuhn and Madanat 2005; Sathaye and Madanat 2011; Lee and Madanat 2015); resource constraints (human resources, equipment, etc.) for the entire system (Fwa et al. 1994; Chu and Huang 2018); user costs due to deterioration and repair works (Ouyang and Madanat 2004; Durango-Cohen and Sarutipand 2007; Ouyang 2007; Medury and Madanat 2013; Hu et al. 2015); and economies of scale by repair synchronization (Mizutani et al. 2020). Among them, many models represent the uncertainty of a deterioration process as stochastic.

As described in the “Introduction,” repair policy optimization models that consider cost leveling are network-level models that account for the interdependence among facilities. The models presented in previous studies on cost leveling can also be generally classified as network-level. Previous studies considering cost leveling have focused on the short-term (several years) construction-sequencing problem (i.e., determining the timing of each work process in a series of short-term construction works) rather than on long-term repair policy optimization problems. Such short-term problems originated with studies of the resource-leveling problem (Woodworth and Willie 1975; Easa 1989; Harris 1990) and have subsequently evolved into weighted total minimization problems of a cost and its variance (Takamoto et al. 1995; Savin et al. 1997), and with studies on achieving cost leveling by the minimum moment method (Kartam and Tongthong 1998; Leu et al. 2000; Hiyassat 2001; Christodoulou et al. 2010). In Yoon et al. (2014), cost leveling in maintenance and asset management were targeted; however, the considered planning period was short (three years), making it difficult to derive long-term repair policies. The deterioration of infrastructure is a long-term process, and the effects of advancements in repair, as shown in Fig. 1, remain relevant for a long time. Thus, to properly consider cost leveling, it is desirable to use a model that determines repair policies in a long-term framework. Additionally, in the aforementioned studies, the deterioration process was assumed to be deterministic. Although the models recalled were network-level, there were a few project-level models that were exceptionally similar to the concept of cost leveling. Yoon et al. (2017) proposed a model to find a long-term repair policy with cost leveling of a single facility assuming a deterministic deterioration process. The contribution of Boyles et al. (2010) can be interpreted as proposing a model similar to the one in this study in the sense that it considered nonlinear preferences for repair costs in a project-level model. However, since it was a project-level model, it would be unable to achieve the objectives of the present study.

The aforementioned studies, as well as this study, attempted to achieve cost leveling directly (an indicator of the variance in repair costs is explicitly included in the objective function). On the other hand, studies that consider the budget constraint for the entire network can consequently demonstrate a cost-leveling effect, even though they do not directly aim at cost leveling. Many studies have been performed to derive network-level optimal repair policies that satisfy the budget constraint assuming a deterministic deterioration process (Lee and Madanat 2015; France-Mensah et al. 2019; Kothari et al. 2022). However, when budget constraints are considered in a stochastic deterioration process, a feasible solution to satisfy the budget constraint does not exist, unless conditions are set so that the occurrence probability of the situations that do not satisfy the budget constraint is zero. Studies on long-term (a sufficiently long finite time horizon, such as decades, or an infinite time horizon) repair policy optimization considering budget constraints for stochastic deterioration processes set restrictions on the analytical framework, such as (1) budget constraints only on the expected values with respect to possible condition states (there is a possibility of budget overruns in reality) (Smilowitz and Madanat 2000); and (2) constraints on the stochastic deterioration process (e.g., assuming that no sudden and unexpected deterioration occurs and that only transitions to adjacent condition states occur) (Kuhn and Madanat 2005; Sathaye and Madanat 2011; Guo et al. 2020). To avoid such restrictions, it is necessary to consider cost leveling directly in a stochastic deterioration process, rather than attempting to indirectly achieve the cost-leveling effect by establishing a budget constraint. However, to the best of the authors’ knowledge, no network-level optimization model has ever been proposed to obtain a long-term repair policy considering the cost-leveling effect directly under the assumption of a stochastic deterioration process.

We review some solution methods for network-level models used for deriving long-term repair policies considering stochastic deterioration processes. As the previous models for cost-leveling consider a deterministic deterioration process, it would be significantly inaccurate to adopt their solution methods for dealing with the Markov decision process used in this study. For small-scale systems, an exact solution of the optimal repair policy at the network level can be found even if the deterioration process is stochastic (Mizutani et al. 2020). Conversely, in realistic-scale systems, because of the combinatorial explosions of both the possible states and the solution space, the problem of deriving optimal repair policies at the network level often leads to difficulties in deriving the exact solution.

Motivated by this, solution methods that approximate the objective function using machine learning (Medury and Madanat 2013; Li et al. 2014; Carvalho et al. 2019) or meta-heuristic methods such as genetic algorithms, simulated annealing, or ant colony optimization (Chan et al. 1994; Fwa et al. 1994; Lee and Madanat 2015; Kielhauser et al. 2017; Zhang et al. 2017) are sometimes used. Nevertheless, generally the repair policies derived by these methods are in the form of specifying the facilities to be repaired for each of a huge number of states, and their applicability in practice is questionable.

Since cost leveling is considered in this study to provide a practical model, it would be desirable to provide the solution in a practical form as well. As previous studies may be able to meet such practical requirements, there are solution methods that reduce the dimensionality of the state and decision variables when approximating the objective function, as follows. Smilowitz and Madanat (2000) suggested a solution method that simplifies the problem by defining state variables for an infrastructure system consisting of homogeneous facilities such that the deterioration state of each facility is aggregated into the number of facilities per deterioration state. Mizutani et al. (2020) proposed an approximation method for finding repair and work zone policies by solution space reduction techniques that (1) use a simplified rule to determine a huge number of decision variables that specify whether each facility in an infrastructure system needs to be repaired or work-zoned; and (2) optimize only a few of the operational variables in the rule. In the present study, we employ some ideas from the latter two sources to develop a solution method for the proposed model that considers cost leveling. Note that, according to the classification by Sathaye and Madanat (2011), the proposed solution method is classified as a top-down approach, which is for homogeneous facilities. Although it is conceivable to use a bottom-up approach that can consider heterogeneous facilities, we design the solution method in this study with emphasis on proposing a calculable solution method. Nevertheless, to the best of the authors’ knowledge, a solution method identical to the preventive repair rule–based policy proposed in this study has not been proposed in the literature. Additionally, it seems that the effect of preventive repairs on cost reduction has been the focus of conventional analysis, and there have been no studies discussing the effect and implementation of preventive repairs from the perspective of cost leveling as performed in this study.

We define a discrete time axis as consisting of time points $t_0,t_1,\dots$ that extends to infinite with a fixed interval $\xi (>0)$; that is, $t_i=t_0+i\xi$, $i\in {\mathbb{Z}}^{+}$. We denote the infinite set of inspection time points by $T\equiv \{t_i|i\in {\mathbb{Z}}^{+}\}$. At each time point $t_i\in T$, an inspection of the entire system is conducted and the CS of each facility in the system is observed. The system manager has to decide which facilities to repair and what types of repair work to perform for the entire system according to the CS inspected.

In this study, we target an infrastructure system comprising $N$ identical (homogeneous) facilities that have the same utility, deterioration process, and repair cost function. We assume the CS of a single facility to be represented by $M$ discrete ratings; the state space of the CS is defined as $M\equiv \{1,2,\dots ,M\}$. When CS takes the value $M$, it means that the facility is in the worst condition, whereas a CS value of 1 represents a facility in its best condition. Hence, the higher the CS, the more deteriorated the facility is. To simplify the expression for the condition of the entire system, information on the CS of each facility at time point $t_i$ is aggregated into a state vector representing the number of facilities per each CS. Specifically, the state vector of the system at time point is defined aswhere $n_{i,m}$ = number of facilities with CS $m$ at time point $t_i$. To distinguish them before and after repair at time point $t_i$, we use ${\mathit{n}}_i^{-}$ and ${\mathit{n}}_i^{+}$ to denote the state vectors before and after repair, respectively. The number of possible patterns for the state vector is $S=\{N+(M-1)\}!/(M-1)!N!$. Without loss of generality, we assign an element identification number $s\in \mathcal{S}$ to each state vector, where $S\equiv \{s=1,\dots ,S\}$ is the state space of the identification number. The state space of the state vectors is denoted $N\equiv \{{\mathit{n}}^{(1)},{\mathit{n}}^{(2)},\dots ,{\mathit{n}}^{(S)}\}$. Denote also by $n_m^{(s)}$ the number of facilities with CS $m$ corresponding to the state vector ${\mathit{n}}^{(s)}$.

A deterioration process of a facility is modeled as a stochastic progression to be able to consider the uncertainty inherent in it. Assuming that each facility in the system is identical, the same Markov process can be adopted for each of them. The Markov transition probability of the change of CS from $a$ to $b$ between $t_i$ and $t_{i+1}$ is defined aswhere $h_i^{-}$ and $h_i^{+}$ = CSs at time point $t_i$ before and after repair, respectively. The Markov transition probability ${\pi }_{a,b}$ has following properties:

The Markov transition probability matrix of a single facility is then defined as follows:where $\mathbf{\Pi }$ = upper triangular matrix because of Eq. (5).

To calculate the transition probability of the state vector, we consider a state variation matrix that represents the amount of change in the number of facilities in each CS. When CS changes from ${\mathit{n}}^{(s)}\in N$ to ${\mathit{n}}^{(r)}\in N$, a state variation matrix ${\mathit{x}}^{s,r}$ is defined aswhere $x_{a,b}^{s,r}$ = number of facilities whose CSs change from $a$ to $b$ when the state vector changes from ${\mathit{n}}^{(s)}$ to $\mathit{n}{s}^{(r)}$; and $n_m^{(s)}(m\in \mathcal{M})=m$th element of ${\mathit{n}}^{(s)}$. The occurrence probability of a state variation matrix ${\mathit{x}}^{s,r}$ (i.e., the conditional occurrence probability of ${\mathit{n}}^{(r)}$ with given ${\mathit{n}}^{(s)}$) can be given, based on the multinomial distribution, as

Since there exist multiple patterns of the state variation matrix ${\mathit{x}}^{s,r}$ that satisfy Eqs. (9) and (10) for the same pair $({\mathit{n}}^{(s)},{\mathit{n}}^{(r)})$, we define the set of all possible state variation matrices as follows:

Now we can derive the transition probability of the state vector from ${\mathit{n}}^{(s)}$ to ${\mathit{n}}^{(r)}$ as the sum of the probabilities of all state variation matrices in ${\mathbf{\Omega }}^{s,r}$; that is

The transition probability $p({\mathit{n}}^{(s)},{\mathit{n}}^{(r)})$ has the following properties:

The deterioration process of the entire system is expressed as the transition probability matrix for state vectors

We assume that one type of repair method is available for each CS before repair, and $w_m$ is the CS after repair when a facility with CS $m$ is repaired. With slight modifications to the framework of this study, the selection of multiple repair methods for a CS before repair can also be considered; however, the preventive repair rule–based policy may become cumbersome. When the state vector at time point $t_i$ before repair is ${\mathit{n}}_i^{-}\in \mathcal{N}$ and repair policy $d\in {\mathcal{D}}_N$ is applied, the repair process can be formulated as the transition of the state vector from ${\mathit{n}}_i^{-}$ to ${\mathit{n}}_i^{+}\in N$ at time point $t_i$ as follows (where we assume that the time for the determination and execution of the repair policy is relatively small, and hence can be ignored throughout this paper):

A transition probability of the Markov chain, $q_{a,b}^d({\mathit{n}}_i^{-})$ can also be interpreted as the ratio (repair ratio) of $n_{i,a}^{-}$ to be repaired. We denote by $d$ the identification number of the possible repair policies, and $D_N$ is the set of possible repair policies. For example, if there are three possible repair policies $d=1,2,3$, ${\mathcal{D}}_N=\{1,2,3\}$. Once is determined, the value of $q_{a,b}^d({\mathit{n}}_i^{-})$ is uniquely selected from $0,1/n_{i,a}^{-},2/n_{i,a}^{-},\dots ,1$, where $n_{i,a}^{-}$ is the $a$th element of ${\mathit{n}}_i^{-}$. Then, once $q_{a,b}^d({\mathit{n}}_i^{-})$ is determined, the correspondence of the transition from ${\mathit{n}}_i^{-}$ to ${\mathit{n}}_i^{+}\in \mathcal{N}$ is deterministically and uniquely specified. However, even if we define this transition correspondence, $q_{a,b}^d({\mathit{n}}_i^{-})$ may not be uniquely determined. Additionally, since the size of the set of $d$ varies with the number of facilities $N$, we denote ${\mathcal{D}}_N$ with the subscript $N$.

In Eq. (18), ${\mathit{Q}}^d({\mathit{n}}_i^{-})$ is the operator matrix that converts the state vector before the repair to that after the repair, and $q_{a,b}^d({\mathit{n}}_i^{-})\in \{0,1/n_{i,a}^{-},2/n_{i,a}^{-},\dots ,1\}$ means the ratio of the number of facilities with CS $a$ before repair whose CSs become $b$ after repair to $n_{i,a}^{-}$. Following the concept of “randomized policy” in Smilowitz and Madanat (2000), the ratio $q_{a,b}^d({\mathit{n}}_i^{-})$ is treated as a decision variable, and the facilities to be repaired are not specified.

Although it is possible to set the operator matrix ${\mathit{Q}}^d({\mathit{n}}_i^{-})$ to be time-variant, to obtain a time-invariant steady-state solution, the repair CS transition matrix ${\mathit{Q}}^d({\mathit{n}}_i^{-})$ is assumed to be constant with respect to time; namely

Additionally, it is supposed that the facilities with the maximum CS $M$ at each time point must be repaired; that isand the facilities with CS 1 need not be repaired; that is

With the operator matrix ${\mathit{Q}}^d({\mathit{n}}_i^{-})$, the repair process of the entire system with given $d$ is expressed as the deterministic transition matrix of the state vectors, so

We consider finding an optimal repair policy that considers both life cycle cost minimization and annual cost leveling. Specifically, we define the cost corresponding to each year as the expected value of the weighted sum of the annual repair cost and its variance and then find the optimal repair policy as the solution to the problem to minimize the total cost over years. The time evolution of the state of the system is modeled stochastically as a dynamic variation of the state vector, which aggregately represents the deterioration and repair processes of each facility. Note that the values of the elements of the deterioration transition probability matrix $\tilde{\mathit{P}}$ fluctuate between 0 and 1 because the deterioration process is a stochastic process, whereas the those of the repair transition matrix ${\tilde{\mathit{Q}}}^d$ are 0 or 1 because the repair process is not a stochastic one.

Supposing the repair policy $d$ is given, the occurrence probability vector of the state vector before repair at each time point, ${\mathit{\mu }}_i^d(i\in {\mathbb{Z}}^{+})$, can be calculated using $\tilde{\mathit{P}}$ and ${\tilde{\mathit{Q}}}^d$ aswhere ${\mu }_{i,s}^d$ = occurrence probability that the state vector before repair is ${\mathit{n}}^{(s)}$ at time point $t_i$. When the time point $t_i$ is close to infinity ($i\to \infty$), the deterioration and repair process converges into a steady state and its occurrence probability vector ${\tilde{\mathit{\mu }}}^d$ satisfies the conditionwhere ${\tilde{\mu }}_s^d$ = occurrence probability of the state vector before repair in the steady state. Supposing the repair policy is $d$ and the state vector before repair is ${\mathit{n}}^{(s)}$, the repair cost at that time point can be given aswhere $c_a$ = unit cost per facility to perform the repair that alters the CS from $a$ to $w_a$.

Since the state vector is a random vector, the repair cost at each time point also varies stochastically. For a given repair policy $d$, the expected value and variance of the annual repair cost at a time point in the steady state are given byrespectively. Based on Eqs. (35) and (36), the cost function $\mathrm{\Psi }({\mathit{n}}_0^{-},d)$ to be minimized can be defined as the discounted present value of the sum of the expected values of the annual repair costs and their variances over an infinite horizon; namelywhere ${\mathbb{E}}_{{\mathit{n}}_i^{-}}[·]$ = expected value of the input with respect to the random vector ${\mathit{n}}_i^{-}$; $\gamma$ = discount factor; and $\epsilon (0\le \epsilon \le 1)$ = relative weight between the annual repair cost and its variance. In this study, we interpret the variance ${\{C^d({\mathit{n}}_i^{-})-E^d\}}^2$ as an additional switching cost (e.g., financing cost) caused by cost variation. In this case, $\epsilon$ can be interpreted as a weight to convert the variance into the monetary scale. In order to treat the variance as the switching cost, which is preferably discounted with respect to time, we formulate the objective function in Eq. (37) by explicitly considering the variance at each time point. Note that we use variance as the index of dispersion of annual repair costs; namely, we assume linearity with respect to the variance in the total cost to be minimized. That is, we define the cost function as a nonlinear function of the repair cost variation on the monetary scale. For this reason, the cost function in Eq. (37) may be interpreted as considering the nonlinear preference (Boyles et al. 2010) for repair costs. In the proposed methodology, it is not necessary to use the variance as the index for the dispersion of annual repair costs; any index defined by $C^d({\mathit{n}}_i^{-})$ and $E^d$ (e.g., standard deviation) can be used with a slight modification. Furthermore, it is conceivable to formulate the objective function that explicitly minimizes the cost variation through the entire planning horizon. In such a case, the proposed approximate solution method may be applicable, but it cannot be guaranteed from our current analysis that the minimization problem of the objective function for the entire planning horizon can be divided into subproblems based on the principle of optimality, or that an exact solution can be derived using the Bellman equation. In this study, we only show one example of an objective function for cost leveling. Further investigation will be needed to modify the objective function in various ways depending on the asset management system.

By determining a repair policy $d$, the quantity of facilities in each CS to be repaired is uniquely determined. The optimal repair policy $d^{*}$ can be found as the policy to minimize the cost function $\mathrm{\Psi }({\mathit{n}}_0^{-},d)$ via dynamic programming from the Bellman equation, so

This optimization problem is a typical infinite-horizon Markov decision process, so the policy or value iteration (White and White 1989) can be used to find optimal repair policy $d^{*}$. In this study, we employ the policy iteration. Note that $E_{d^{*}}$ in Eq. (38) includes the occurrence probability of the state vector in the steady state; however, the general policy or value iteration can be directly used to find the steady-state solution.

Finding the optimal repair policy is a problem of selecting the optimal repair pattern from among ${\prod }_{a=2}^{M-1}(1+n_{i,a}^{-})$ possible candidates for each state vector ${\mathit{n}}_i^{-}$, whose state space has size $S=\{N+(M-1)\}!/(M-1)!N!$. Thus, when the system comprises a large number of facilities, the optimal repair policy $d^{*}$ is difficult to calculate because of combinatorial explosion. To overcome this difficulty, we develop an approximate solution method considering the ease of intuitive understanding and the feasibility of the solution method in practice. Specifically, an optimization problem is defined, in which facilities to be repaired are determined by a preventive repair rule that allows execution of preventive repair according to the observed state. Optimizing a small number of decision variables in a preventive repair rule is equivalent to solving an approximate problem that reduces the size of the solution space of the original optimization problem whose solution is given in Eq. (38).

Let us explain now the relationship between the preventive repair rule–based policy and cost leveling. In Fig. 1, the first repair (red arrow) in “Deterioration and repair process of facility 2” in “Repair policy considering cost leveling” can be regarded as a preventive repair in which the repair is executed before the CS reaches 4. By implementing the preventive repair rule–based policy, we can expect two types of cost-leveling effect. One, at a time point, if the total annual repair cost of the facilities that must be repaired does not reach the desirable annual repair cost (budget), additional preventive repairs can be made to bring the actual annual repair cost closer to the desirable annual repair cost (single time point cost-leveling effect); two, if it is likely that many facilities will need to be repaired simultaneously at a future time point, repairs for some of these facilities can be advanced as preventive repairs at the current time point to make the annual repair cost at the future time point closer to the desirable annual repair cost (long-term cost-leveling effect). In the framework of this study, we do not minimize the cost and variance in a single year, but minimize the sum of the total cost and variance over the entire planning horizon as shown in Eq. (37). Therefore, an optimal preventive repair rule–based policy can be obtained considering both the single time point cost-leveling effect and the long-term cost-leveling effect. The method for deriving the optimal preventive repair rule described below relies on optimizing the decision variables to determine the desirable annual repair cost and how many facilities of each CS should be repaired if additional preventive repairs are desired.

The preventive repair rule–based policy is the elements $q_{2,w_2}^d({\mathit{n}}_i^{-}),q_{3,w_3}^d({\mathit{n}}_i^{-}),\dots ,q_{M-1,w_{M-1}}^d({\mathit{n}}_i^{-})$ of the operator matrix ${\mathit{Q}}^d({\mathit{n}}_i^{-})$ determined by the algorithm with $2M-3$ decision variables, $\mathit{z}=(\phi ,{\theta }_{\mathrm{a},2},{\theta }_{\mathrm{a},3},\dots ,{\theta }_{\mathrm{a},M-1},{\theta }_{\mathrm{b},2},{\theta }_{\mathrm{b},3},\dots ,{\theta }_{\mathrm{b},M-1})$, whose state space is $\mathcal{Z}=\mathbf{\Phi }\times {\mathbf{\Theta }}_{\mathrm{a},2}\times \cdots \times {\mathbf{\Theta }}_{\mathrm{a},M-1}\times {\mathbf{\Theta }}_{\mathrm{b},2}\times \cdots \times {\mathbf{\Theta }}_{\mathrm{b},M-1}$, instead of the specification of $d$, when ${\mathit{n}}_i^{-}$ is given. Subscripts a and b in roman type are used to identify the two types of $\theta$. Once $q_{2,w_2}^d({\mathit{n}}_i^{-}),q_{3,w_3}^d({\mathit{n}}_i^{-}),\dots ,q_{M-1,w_{M-1}}^d({\mathit{n}}_i^{-})$ are found, the other elements of ${\mathit{Q}}^d({\mathit{n}}_i^{-})$ are uniquely determined. The intention of setting each decision variable is described in the following paragraphs.

The variable $\phi$ is the decision variable to set the desirable annual repair cost.

The decision variables ${\theta }_{\mathrm{a},2}\dots ,{\theta }_{\mathrm{a},M-1},{\theta }_{\mathrm{b},2}\dots ,{\theta }_{\mathrm{b},M-1}$ determine the ratio of preventive repair for each CS to (1) adjust the annual repair cost closer to the desirable annual repair cost, (2) advance some of the future repairs when it is expected that many repairs will be needed at a future time point, and (3) address situations where preventive repairs attain cost minimization if the required annual repair cost at the current time point is smaller than the desirable annual repair cost.

The formula ${\mathit{\theta }}_{\mathrm{a}}=({\theta }_{\mathrm{a},2},\dots ,{\theta }_{\mathrm{a},M-1})$ is the ratio of preventive repair for each CS when the total annual repair cost of the facilities that must be repaired does not reach the desirable annual repair costs (budget), and ${\mathit{\theta }}_{\mathrm{b}}=({\theta }_{\mathrm{b},2},\dots ,{\theta }_{\mathrm{b},M-1})$ is the ratio of preventive repair for each CS otherwise. The decision variable $\mathit{z}$ is supposed to take the same value regardless of ${\mathit{n}}_i^{-}$, and instead of optimizing $d\in {\mathcal{D}}_N$, only $2M-3$ decision variables are optimized. Note that the operator matrix determined as the preventive repair rule–based policy also varies depending on ${\mathit{n}}_i^{-}$. We denote the operation matrix determined by the preventive repair rule–based policy with $\mathit{z}$ asinstead of ${\mathit{Q}}^d({\mathit{n}}_i^{-})$. The symbol “⌣” indicates the preventive repair rule–based policy. In the following paragraphs, we first describe the specific algorithm for calculating the expected total cost for a given $\mathit{z}$ and then formulate the optimization problem corresponding to $\mathit{z}$.

Since the desirable annual repair cost $\tilde{C}$ does not depend on ${\mathit{n}}_i^{-}$, it must be determined only once in finding the optimal preventive repair rule–based policy $z^{*}={\mathit{z}}^{*}=({\phi }^{*},{\theta }_{\mathrm{a},2}^{*},{\theta }_{\mathrm{a},3}^{*},\dots ,{\theta }_{\mathrm{a},M-1}^{*},{\theta }_{\mathrm{b},2}^{*},{\theta }_{\mathrm{b},3}^{*},\dots ,{\theta }_{\mathrm{b},M-1}^{*})$. The $\tilde{C}$ is originally positioned as a decision variable and takes any positive value. In this study, to limit the search range of $\tilde{C}$, we introduce $\phi$, which represents the degree of deviation from the expected annual repair cost $C_0^{*}$ when the optimal repair policy is adopted for the case where $\epsilon =0$ is set in Eq. (38)—that is, the case where only the total repair cost is minimized—and we determine $\tilde{C}$ aswhere $d_0^{*}$ = optimal repair policy obtained via Eq. (38) with $\epsilon =0$; and ${\tilde{\mu }}_s^{d_0^{*}}$ = expected annual repair cost at steady state under $d_0^{*}$. Since the problem of finding $d_0^{*}$ is attributed to a project-level repair policy optimization problem, $d_0^{*}$ and ${\tilde{\mu }}_s^{d_0^{*}}$ can be easily obtained without a large computational load. Specifically, as the cost function in Eq. (34) has linearity, when $\epsilon =0$ it can be divided into subproblems (where the decision variable is whether or not to repair each facility) for each facility $n\in \mathcal{N}$. Since the Markov transition probability ${\pi }_{a,b}$ and the unit cost of repair $c_a$ are the same for all facilities, the optimal solutions to these subproblems are the same for all facilities. Thus, when $\epsilon =0$, the project-level problem of deciding whether or not to repair each system of a facility need only be solved once.

At any time point $t_i$ with given state vector ${\mathit{n}}_i^{-}$, the number of facilities to repair for each CS is found based on the preventive repair rule with variables $\phi$, ${\mathit{\theta }}_{\mathrm{a}}=({\theta }_{\mathrm{a},2},\dots ,{\theta }_{\mathrm{a},M-1})$, and ${\mathit{\theta }}_{\mathrm{b}}=({\theta }_{\mathrm{b},2},\dots ,{\theta }_{\mathrm{b},M-1})$ by the following algorithm.

Step 0: Initialize the set of the investigated CSs as $\mathcal{R}=\{M\}$ and the repair facility number vector $\mathrm{\Delta }\mathit{n}=(\mathrm{\Delta }{n}_2,\mathrm{\Delta }{n}_3,\dots ,\mathrm{\Delta }{n}_M)=(0,0,\dots ,n_{i,M}^{-})$. Set the decision variable $\mathit{\theta }=({\theta }_2,\dots ,{\theta }_{M-1})$ as

Step 1: Set the investigating CS as $\widehat{a}=\min \{\mathcal{R}\}-1$. If, finish the algorithm.

Step 2: Calculate the surplus budget as $\mathrm{\Delta }\tilde{C}=\tilde{C}-\sum _{m\in \mathcal{R}}{c}_m\mathrm{\Delta }{n}_m$. If the surplus budget $\mathrm{\Delta }\tilde{C}\le 0$, finish the algorithm.

Step 3: Calculate the possible repair facility number with CS $\widehat{a}$ as a real number as

Step 4: Calculate the actual repair facility number in an integer aswhere $\lfloor ·\rfloor$= floor function.

Step 5: Update the set of the investigated CSs as $\mathcal{R}=\mathcal{R}\cup \{\widehat{a}\}$, and return to Step 1.

An example of a specific procedure for determining $\mathrm{\Delta }\mathit{n}$ for a given ${\mathit{n}}_i^{-}$ is shown in Fig. 2. With the repair facility number vector $\mathrm{\Delta }\mathit{n}$ given by the algorithm, we can determine the elements of the operator matrix as follows:

With the above procedure, we can calculate ${\stackrel{⌣}{\mathit{Q}}}^{\mathit{z}}({\mathit{n}}_i^{-})$ given $\mathit{z}$ and ${\mathit{n}}_i^{-}$. This allows us to derive the cost function $\stackrel{⌣}{\mathrm{\Psi }}({\mathit{n}}_0^{-},\mathit{z})$ on the basis of the preventive repair rule–based policy as follows:where ${\tilde{\mu }}_s^z$ = occurrence probability of the state vector before repair in the steady state with given $\mathit{z}$. The preventive repair rule–based policy only comprises $2M-3$ decision variables, and the size of the search space of the decision variables becomes $|\mathcal{Z}|$. The operator $|·|$ indicates the number of elements (the cardinality) of the set. Thus, we can find an optimal preventive repair rule–based policy with the combinations of the decision variables to minimize the cost function $\stackrel{⌣}{\mathrm{\Psi }}({\mathit{n}}_0^{-},\mathit{z})$ by calculating the cost function with the Monte Carlo simulation with sufficiently many $G$ iterations for all possible combinations of the decision variables. In an iteration, given $\mathit{z}$, the annual repair costs at $H$ time points can be calculated, and these values can be used to numerically determine the value of $\stackrel{⌣}{\mathrm{\Psi }}({\mathit{n}}_0^{-},\mathit{z})$. Here we assume ${\mathit{n}}_0^{-}=(N,0,\dots ,0)$. By performing a total of $|\mathcal{Z}|$ simulations for each pattern of $\mathit{z}$ and comparing the results, the optimal value turns out to be

In the numerical study in this paper, we do not specify $\epsilon$ to derive the preventive repair rule–based policy, but calculate the expected value and variance in annual repair costs by simulation for each and then compare the results to extract the Pareto frontier of $\mathcal{Z}$.