Provoking More Productive Discussion of Wicked Problems

Water management is full of wicked problems. These problems often have many conflicting stakeholders with entrenched opposing positions, great complexity, shifting dynamics, and substantial uncertainties (Rittel and Webber 1973; Liebman 1976). Wicked problems elude clean technical and policy solutions.

The challenges of solving such wicked problems are borne out mathematically in game theory. For many problems, the parties involved have short-term incentives to behave unproductively. Games of “chicken” or other unproductive or counterproductive deadlocks result when parties follow near-sighted objectives as the overall system continues to deteriorate (Madani and Lund 2010).

Elected and agency officials have special challenges dealing with wicked problems because the controversies from addressing them can threaten their careers. One response to these controversies has been frequent use of consensus processes, in which a wide range of stakeholders are assembled to outline solutions themselves, with little governmental direction. Sometimes these processes have been very useful and produce win-win solutions for all parties. However, seeking consensus often becomes a prolonged process of endless discussions and posturing, with very little progress—an elegant way to prolong a deteriorating status quo. Many problems cannot be resolved by a consensus process alone. The demonstration of win-win opportunities is not a sufficient condition for consensus.

Stakeholders are usually not ready for the rhetorical and substantive compromises needed for consensus. Decades of work and failure are often needed for stakeholders to obtain the knowledge, perspective, experience, and political position on which consensus on long-term direction is possible (Blomquist 1992; Kelley 1989). Unfortunately, stakeholders are sometimes devoted to outsmarting each other more than solving the long-term problem.

External events or insights often are needed to provoke longer-term thinking and actions by stakeholders and policymakers. Floods, droughts, and lawsuits can serve this function, often violently (Hanak et al. 2011). Sometimes policy thinking can be aided by insightful, fair, and well-communicated studies by independent nonpartisan entities, such as think tanks or universities. Informed public discourse on technically complex problems requires thoughtful and detailed efforts to design even the outlines of major strategic actions.

When stakeholders perform this type of work, it is likely to be seen as partisan because it often is, or it is likely to be hidden from public discourse because it would often muddle short-term partisan messages. Universities have almost unique opportunities to be useful in providing such thought- and discussion-provoking studies (Cash et al. 2003). Public universities may even be thought of as having a duty in this regard.

Provoking more productive discussion on the way to solution or consensus has tremendous opportunities and a few risks for universities. For individual faculty and students, these types of studies provide remarkable access to interesting and important problems and people. Stakeholders are usually very smart and knowledgeable people. These problems almost always have important theoretical and methodological aspects and opportunities, which are of value for traditional scholarly activities of teaching and research. However, supporting more constructive discussion is time consuming and requires that researchers cultivate the middle ground and not be affiliated or funded by one side.

For universities, as institutions, provoking more productive discussion of wicked problems is of great importance and can contribute to their long-term reputation and independence. However, artless and overly prominent involvement can put a university at risk of becoming mired in controversy.

Trying to provoke more productive discussion of public policy problems among thousands of self-interested and fearful stakeholders is often ineffective, at least in the short term. Luck and timing are important. Doing a major study on flood policy just before a major flood is much more effective than doing such a study months after a flood. I have been fortunate to be involved in several major and relatively successful policy-provoking efforts. I’d like to reflect on some of these briefly.

Restoring Hetch Hetchy Valley by removing O'Shaughnessy Dam has been a goal of many Californians since the early 1900s when John Muir and others lost their battle to preserve this valley from San Francisco’s water system. Over time, water supply technologies have improved, and California’s water system has grown and diversified. Perhaps Hetch Hetchy is no longer vital as a water supply. In 2003, this topic became the focus of the master’s thesis of Sarah Null (now a professor at Utah State University). In this and subsequent work, using the CALVIN hydroeconomic optimization model, we found that a short intertie of the Hetch Hetchy Aqueduct to the larger downstream new Don Pedro Reservoir would allow the San Francisco service area to be supplied in almost all years, with water for remaining years being purchased from farmers (Null and Lund 2006).

The problem is not the provision of water. Rather, the problem is economic. Because of its purity and age, the Hetch Hetchy source does not currently require a filtration plant (estimated to cost approximately $1 billion, or more than $50 million annually at a 5% interest rate; lost hydropower and other costs are much less). Hetch Hetchy Valley is unlikely to be restored in the near future, but this work (and other work) has allowed a longer-term and more focused view of the problem. When drinking-water filtering is required, restoring Hetch Hetchy Valley will become much more feasible.

California’s Sacramento–San Joaquin Delta has been controversial since the 1920s. Its problems have evolved over time. During the past decade, it became apparent that the current situation is largely unsustainable, with rising sea levels, land subsidence, likely earthquakes, floods, and declining native ecosystems. Stakeholder processes became exhausted, and state agencies were unable to present an organized and effective understanding of the problem and potential solutions. By 2005, the demise of the CALFED Bay-Delta Program consensus process and the cacophony of stakeholder notions without a state policy framework were clearly unproductive.

In 2007, a group of independent university and think-tank researchers released the report Envisioning Futures for the Sacramento–San Joaquin Delta (Lund et al. 2007). The report organized the problem and potential solutions concisely on the basis of existing and new scientific and technical work. It proved to be useful in shaping emerging governmental decision processes and studies. In 2008, a second report (Lund et al. 2008), Comparing Futures for the Sacramento–San Joaquin Delta, with 10 supporting technical appendices, presented a quantitative and analytical comparison of some major policy choices for water and land management. This body of work, summarized in Lund et al. (2010), was important in shaping current discussion of the Sacramento–San Joaquin Delta and the 2009 state water legislation.

Overall, California’s water policy is at a crossroads on many fronts. Although innovations and experiments are widespread at the local level, broad state and federal policy discussions and agency actions are commonly deadlocked by conflicting interests and concerns. Policy discussions often deteriorate into competing stakeholder narratives, which often degenerate into common myths about water problems. In 2010, an expanded group of university and think-tank researchers produced a report and paper entitled California Water Myths (Hanak et al. 2009, 2010), identifying the scientific error of many myths and their unproductive policy implications. This was followed by a major book on California’s water policy (Hanak et al. 2011), Managing California’s Water: From Conflict to Reconciliation, which reviews California’s water history, current situation, and future prospects and then suggests a range of long-term approaches and policies for adapting California’s water policy for future conditions. Time will tell if this effort is successful in its broad goals. We have developed a blog to help with this discussion (californiawaterblog.com).

In these efforts to provoke more productive policy discussions, it has been important to bring something new to these conversations. In the preceding cases, the primary novelty was a coherent technical perspective on the problem, from which technical and policy insights could arise. For this work, systems analysis techniques from engineering, operations research, and economics have been most valuable for helping organize problems, solutions, and other knowledge for identifying misconceptions and promising policy insights. Much of this technical perspective arose through the development of the CALVIN model of California’s water system (Draper et al. 2003; Jenkins et al. 2004; Tanaka et al. 2011; Harou et al. 2009) and other technical activities over many years. Although all models are wrong in a pure sense, the development and use of a systems model for a large complex problem entails opportunities to develop insights and perspectives that are unavailable to others and may have value for policy discussions.

From these experiences, five principles for how to do broadly useful provocative studies have emerged.The policy environment that we are trying to educate usually resides above the direct stakeholders (characteristically legislators, legislative staff, senior officials, and media) but also involves the stakeholders themselves. These people are usually very smart but often lack a good broad understanding of the problems and promising solutions, except for their understanding of the political difficulties. Much can be learned from all of these people, and talking with them before, during, and after your research will improve your understanding and products.

In the field of systems analysis, we have special incentive to become involved in insightful and provocative studies. Our field depends economically on our reputation for producing information that helps resolve problems in better ways. This requires us to be engaged with real problems and with the wide range of experts and stakeholders involved in real problems. Aside from mathematical and quantitative skills, system analysts must be willing to learn whatever social and scientific fields are needed to assemble and communicate a system analysis study. These studies should occasionally be provocative, but they should almost always be insightful.

Wicked water controversies often involve thousands of people and evolve over historical time scales, so patience, persistence, and pointed diplomacy are needed to move the conversation along. We, as academics and researchers, generally can do more than just write papers. It is hard but more rewarding and sustainable for the profession.

This essay is adapted from part of the acceptance speech for ASCE’s 2011 Julian Hinds Award. I thank my wife and family, students, and many colleagues for their wonderful help over the years; the people of California for paying my salary; state and federal agencies for the many problems they have let me work on; and the many journal reviewers for making constructive and useful criticisms that have improved our work. Alas, those responsible are too numerous to mention individually.