Coping with the Wickedness of Public Policy Problems: Approaches for Decision Making under Deep Uncertainty

Robust decision making (RDM) (Lempert et al. 2006) emphasizes an iterative approach to planning in which candidate strategies are tested across a very large number of scenarios and, in light of insights gained from this model-based scenario analysis, candidate strategies can be improved. The overarching concern is with the development of a strategy that produces satisficing results in as large a set of scenarios as possible. In RDM, the first step is a generic policy analytic activity that aims at conceptualizing the system under study, the key uncertainties pertaining to the system, the main policy levers, and the outcomes of interest. The second step is case generation, or exploratory modeling (Bankes et al. 2013). In this step, the behavior of one or more models of the system under study is systematically explored across the identified uncertainties, and the performance of candidate strategies is assessed. The third step is scenario discovery (Bryant and Lempert 2010). Using statistical machine learning algorithms, the results of the exploratory modeling are analyzed to reveal the conditions under which strategies perform poorly. These conditions reveal vulnerabilities of the strategies, in light of which they can be modified. The fourth step is trade-off analysis, in which the performance of the different strategies are compared across the different outcome indicators, thus providing an additional source of information that can be used in redesigning strategies. The steps can be iterated until a satisficing robust strategy emerges.

Multiobjective robust decision making (MORDM) (Kasprzyk et al. 2013) is an extension of RDM that adds a multiobjective optimization search for solutions prior to performing the exploratory modeling and scenario discovery. The multiobjective optimization is used to generate a set of promising planning alternatives that illustrate the key trade-offs on the relevant objectives. RDM is subsequently used to assess the robustness of each of these planning alternatives to a wide range of deeply uncertain futures. Kasprzyk et al. (2013) also discuss various visual analytics techniques that can be used to assess the trade-offs across multiple objectives and the robustness of the various alternatives. A key point of the visual analytics is that both RDM and MORDM aim at facilitating a discussion among stakeholders and decision makers, rather than dictating a single optimal solution (Singh et al. 2015).

RDM has been applied to strategic planning problems in a diverse set of fields, including economic policy (Seong et al. 2005), climate change (Lempert et al. 1996, 2003), flood risk management (Fischbach 2010), sea level rise (Lempert et al. 2012), energy resource development (Popper et al. 2009), and water resources management (Groves 2005; Groves and Lempert 2007; Lempert and Groves 2010; Matrosov et al. 2013a, b). MORDM has been applied to water resources planning management (Herman et al. 2014; Kasprzyk et al. 2013) and ecosystem management (Singh et al. 2015).

The dynamic adaptive policy pathways (DAPP) (Haasnoot et al. 2013a, b) approach is based on the concept that, in light of deep uncertainties about the future, one needs to design dynamic adaptive plans. Such plans contain a strategic vision of the future, commit to short-term actions, and establish a framework to guide future actions. It is a fusion of adaptive policymaking (Hamarat et al. 2013; Kwakkel et al. 2010; Walker et al. 2001) and adaptation tipping points (Haasnoot et al. 2012; Kwakkel et al. 2010; Offermans 2012).

The first step in DAPP is to describe the setting, including objectives, constraints, major uncertainties, and a definition of success, and to assess current and future vulnerabilities and opportunities. The specified uncertainties are used to generate an ensemble of plausible futures in the form of (transient) scenarios. Next, the conditions under which the status quo starts to perform unacceptably (adaptation tipping points) are assessed for the relevant uncertainties using expert judgment and/or model simulations. The timing of an adaptation tipping point (use-by date) is derived from linking the use-by conditions with scenarios, or from the changing performance over time resulting from transient or semistatic model simulations. This reveals if, and when, policy actions are needed to reach the desired outcomes. Based on this problem analysis, policy actions are identified to address vulnerabilities and seize opportunities, and their conditions and timing of adaptation tipping points is assessed based on their efficacy in reaching the desired outcomes over changing conditions or time. Once the set of policy actions is deemed adequate, alternative pathways can be designed and evaluated. A pathway consists of a concatenation of policy actions, in which a new policy action is activated once its predecessor is no longer able to meet the definition of success. Based on the evaluation of the pathways, a manageable number of preferred pathways can be identified. These preferred pathways can be improved through contingency planning, which requires the specification of corrective, defensive, and capitalizing actions, and an associated monitoring system with trigger values that would result in the implementation of the actions. In light of the final adaptation pathways map, a plan for action can be made, which specifies the actions to be taken immediately, the developments to monitor, and when next actions of a pathway should be taken to stay on track of the preferred pathway.

Fig. 1 shows a stylized example of an adaptation pathways map. In the map, starting from the current situation, targets begin to be missed after four years. Following the line of the current policy, one can see that, after four years, there are four options. Actions A and D should be able to achieve the targets for the next 100 years in all climate scenarios. If Action B is chosen after the first four years, a tipping point is reached within about five years; a shift to one of the other three actions will then be needed to achieve the targets (follow the lines of Action B). If Action C is chosen after the first four years, a shift to Action A, B, or D will be needed in the case of Scenario X (follow the solid line of Action C). In all other scenarios, the targets will be achieved for the next 100 years (the dashed line of Action C).

Adaptation pathways can be developed in a variety of ways. Haasnoot et al. (2012) systematically assess adaptation tipping points and explore options after an adaptation tipping point across a range of transient climate scenarios through simulations; Haasnoot et al. (2013a, b) derive the pathways from expert judgment on adaptation tipping points; Haasnoot (2013) derives pathways from expert written storylines and game simulations, and Kwakkel et al. (2014) use a multiobjective robust optimization approach.

The adaptation pathway approach has been applied to a variety of cases. Most notably, it forms the underpinning of the Dutch Delta Programme (2014), and it has been used in the Thames Estuary 2100 project (Reeder and Ranger online). Haasnoot et al. (2013a, b) demonstrate the adaptation pathway approach with an example drawn from the Dutch Delta Programme focused on the Lake IJselmeer area in the Netherlands. Rosenzweig and Solecki (2014) adopt the notion of adaptation pathways to discuss climate adaptation in New York after Hurricane Sandy. Other applications are ongoing. For example, the approach is currently being used in Bangladesh and Indonesia.

The authors have presented RDM and DAPP as two distinct approaches to supporting decision making under deep uncertainty. There are, however, commonalities between the approaches. For example, both DAPP and RDM rely on a participatory scoping of the problem and the use of sets of scenarios to identify vulnerabilities. A vulnerability in the context of RDM is the set of uncertain developments under which a policy fails. This is closely related to the idea of an adaptation tipping point in DAPP. There are also complementarities between the approaches. RDM has a strong emphasis on the iterative process of scenario discovery and policy refinement. RDM is less well developed with respect to the architecture of policies that can be adapted over time. In contrast, DAPP focuses on the adaptive policy architecture, but is more open ended on how to design policies that fit this adaptive architecture. Hence, researchers are increasingly working on combining elements from both approaches (Groves et al. 2014).

Both RDM and DAPP emerged as planning approaches in the presence of deep uncertainty. Looking at these approaches in light of the characteristics of wicked problems, how well do they hold up?

Looking at the literature on RDM and MORDM, the authors observe that there is a strong focus on supporting deliberation through analysis. In an evaluative study of scenario discovery, Parker et al. (2015) found that scenario discovery is able to summarize the information contained in a large ensemble of simulation runs in an easily understandable way. Users appreciated the ability to analyze trade-offs, and found the results to be quite unambiguous. This ability to analyze trade-offs is particularly apparent in the multiobjective extension to RDM, where the set of solutions found through optimization is not handled as the final set of possible solutions. Instead it offers a starting point for learning about the problem, about possible solutions, and about trade-offs (Kasprzyk et al. 2013; Singh et al. 2015). If no clearly preferred solution is found, at least it is learned that the problem framing needs to be adapted. Moreover, the iterative process of policy refinement through modeling supports learning and computer-assisted reasoning (Bankes et al. 2001).

There are, however, several facets of wicked problems to which RDM does not offer a clear answer. RDM starts from the idea of scoping a problem by defining a system boundary and agreeing on outcomes of interest. Once these are set and models are developed or tuned to fit with this scoping, it will be hard and often expensive, although not impossible, to revise this in light of what is being learned. That is, RDM assumes substantial consensus among decision makers and stakeholders on the system under study. It is, therefore, not surprising that RDM practitioners often stress the importance of using existing models that are accepted by the various decision makers and stakeholders (R. J. Lempert et al., “Ensuring robust flood risk management in Ho Chi Minh City,” working paper, World Bank, Washington, District of Columbia). Another issue that is not extensively addressed in the RDM literature at present is the fact that, in many complex wicked problem situations, decisions are largely irreversible, there is no right to be wrong, and there is path dependency. RDM helps in reducing the scenarios under which an action fails with its iterative improvement of the robustness of candidate actions, but does not provide detailed guidance on how to design plans that can be adapted over time, nor does it offer support for analyzing path dependency and lock-ins. It is exactly here that there exist complementarities with the DAPP approach, which focuses more strongly on making the path dependency between actions, and the presence or absence of lock-ins, more transparent.

Examining DAPP as an approach for supporting decision making on wicked problems, there are several aspects that stand out. First, DAPP strongly emphasizes the importance of keeping multiple pathway options open to the future, which helps alleviate the irreversibility of decisions and reduces the risk of being wrong. Pathways make lock-ins transparent and help foster understanding of which options are left open given a certain choice now. Moreover, pathways specify future actions that can be taken if the initial actions prove to be insufficient. Second, some of the work on model-based support for the design of adaptation pathways has explicitly approached it as a multiobjective problem (Kwakkel et al. 2014), in which support is focused on creating clarity with respect to trade-offs among competing decision alternatives. Third, DAPP does not dictate a single solution; instead, it helps produce a map of possible routes into the future and can, for example in combination with the perspectives method (Offermans 2012; Offermans et al. 2011), present the consequences of different values and perspectives of stakeholders. In light of this, decision makers and stakeholders can have an informed debate on which actions they would like to take in the future, with an awareness of how these actions might affect their solution space in the future.

There are several facets of wicked problems to which adaptation pathways are less well suited. Similar to RDM, DAPP assumes that the outcomes and system boundaries are largely uncontested. The process envisioned by DAPP also limits possibilities to change the system conceptualization over the course of the analysis. This is not impossible, but might be costly. Another less well-developed aspect is the computer-assisted learning about a problem that is one of the strengths of RDM. DAPP is substantially more open ended in the methods, tools, and techniques one can employ for supporting adaptation pathway design.

In conclusion, both RDM and DAPP address somewhat different aspects of what makes wicked problems wicked. RDM facilitates the analysis of trade-offs and the iterative learning about a policy problem. DAPP helps in studying the reversibility of decisions and offers insight into future actions that can be taken if the initial actions prove to be insufficient. This suggests that research on combining RDM with DAPP is a fruitful direction for future work.

Both RDM and DAPP still struggle with the fact that, in many wicked problems, the problem definition itself is open to change and co-evolves with solutions that are suggested, and that rival system boundaries and conceptualizations may be present. In the context of model-based support for decision making, a relatively precise and unambiguous system conceptualization is required, which can be at odds with the wicked nature of the problem under study. The exploratory modeling approach advocated for supporting decision making under deep uncertainty (McInerney et al. 2012) can be used to at least partly alleviate this concern. Kwakkel et al. (2013), for example, apply scenario discovery using two models that represent substantially different conceptualizations of the system under study. Similarly, Auping et al. (2015) explore the consequences of alternative strategies for coping with societal aging using three distinct conceptualizations of how public support for societal aging policies develop. Pruyt and Kwakkel (2014) apply a similar multi-model approach to identify effective policies for reducing homegrown terrorism, in which the three models are inspired by rival explanations for the emergence of homegrown terrorists. These examples demonstrate that it is at least technically feasible to handle multiple partially incommensurable system conceptualizations in a single exploratory modeling approach.