Earthquake Early Warning Riskwork: ShakeAlert’s Operation with Institutionalized Uncertainties

The West Coast of the United States of America has had an earthquake early warning system, ShakeAlert, in development since 2007. Recently, though, it has experienced rapid growth. While the project began in 2007, it started offering public alerts in California in 2019. Things happened quickly for the system after that. By May 2021, ShakeAlert was out of the testing phase: according to the USGS, the federal entity responsible for ShakeAlert, more than 50 million people across California, Oregon, and Washington could receive warnings before earthquakes via the FEMA Wireless Emergency Alert system, third-party phone apps, and other technologies.

Earthquake early warning draws automated environmental monitoring into coordinated operation with a variety of technical and social systems. This allows earthquake early warning systems to detect quakes as they start and produce warnings for areas of concern, sometimes seconds before shaking reaches them (see Fig. 1). Such systems can take many configurations [see the excellent overview in Allen and Melgar (2019)]. The variety of earthquake early warning systems in operation today is broad. Some involve wide coverage. Their designers have optimized them for a variety of functions, such as ShakeAlert and other systems outlined by Kamigaichi et al. (2009), Sheen et al. (2017), and Espinosa-Aranda et al. (2011). Others produce alerts tailored to fewer purposes, including providing alerts to student dormitories and emergency management offices (Chamoli et al. 2021), to a gas utility and a subway tunnel (Clinton et al. 2016), or to a single nuclear research facility and a bridge (Satriano et al. 2010; Mărmureanu et al. 2010).

ShakeAlert is an especially ambitious system, bringing together many elements and collaborators into a single massive project. The scale of the challenges it presents can be hard to convey easily. When interviewed for this research, one scientist working on the system offered a high-level list: “There’s networks, there’s tectonics, there’s societal differences and complexities, there’s state, there’s political, jurisdictional, there’s operational levels of complexities.” In this paper, we attend to some of these “complexities” as they inform uncertainties that system operators must navigate.

As the previously quoted scientist summarized, ShakeAlert uses multiple regional seismic sensory networks that monitor places with different tectonic characteristics and communicate alerts to diverse groups (see Given et al. 2018). Local and regional government are involved in funding, regulating, and decision-making related to the system. So are politicians on the national stage. The USGS oversees the project. Operations and research involve multiple universities and technical partners with special permission to develop and deliver additional services using warnings generated by the system. ShakeAlert depends on existing infrastructure, including long-standing collaborations, physical networks, and ongoing projects (e.g., Eguchi et al. 1994). This new public-facing mandate and the influx of resources that it brings have meant substantial expansion (e.g., Cochran et al. 2018; Kohler et al. 2020), as have ongoing seismic motion and concerns about it. In this paper, we argue that what the scientist we quote called “complexities” have been core to the experience of operating ShakeAlert, especially as the system went live. The uncertainties that arise in relation to these complexities, which we consider in depth, have had consequences for the operation of the system, for the system itself, and for its users.

ShakeAlert is designed to involve diverse collaborations of the sort that disaster communication scholar Dennis Mileti advocated as part of an interdisciplinary hazards community (Mileti 1999; Mileti and Gailus 2005). As its 2018 technical implementation plan stipulates, ShakeAlert’s approach to alerting requires the development of “all practical pathways” beyond the scope of the “mission and abilities of the USGS” (Given et al. 2018). Some have commented that the complex network of public-private partnerships that animate ShakeAlert should be considered part of a “paradigm shift” that will make earthquake early warnings increasingly common around the world and available for different uses (Allen and Stogaitis 2022). However, as much as these advocates might visualize such coordinated efforts as key to a bright future for earthquake risk mitigation, in practice ShakeAlert’s many elements informed the specific kinds of uncertainties that its operators had to navigate. Indeed, as organizational studies scholars have shown, new partnerships and innovative undertakings may entail both rewards and complex challenges (e.g., Powell et al. 1996). This was very much the case for ShakeAlert. However, these aspects of work on ShakeAlert have not been substantially explored in scientific literature on the system. Because research-based guidelines in hazard risk mitigation suggest taking on collaborative projects, operators’ experiences developing and navigating these relationships merit attention.

In 1987, Dennis Mileti and his collaborator John Sorensen developed a typology of the major decision-making uncertainties that mark early warning systems because, as they wrote, “uncertainty can act as a major constraint to organizations providing timely public warnings in emergencies” (Sorensen and Mileti 1987, p. 35). They drew their readers’ attention to uncertainties related to interpreting impending events, communicating about such events, perceived impacts of warnings, and exogenous influences (including prior community experience and planning) (Sorensen and Mileti 1987). Substantial research on earthquake early warning, specifically, has sought to support decision-making related to issues within each of these categories.

With this paper, we take on a different task. Rather than supporting decision-making, we seek instead to characterize work with and through uncertainties. We borrow sociologist of science Janet Vertesi’s term “institutional uncertainties” to describe an ordinary lack of information, confusions, and disagreements that people sometimes express in relation to rapidly changing, collaborative projects like ShakeAlert undertaken by organizations like the USGS (Vertesi 2020). These institutionalized uncertainties may be about key concepts, approaches, roles, or planning. Crucially, they highlight issues that those involve cannot easily resolve, despite their expertise. They are perpetuated by institutional factors. In fact, we will describe these uncertainties themselves as “institutionalized” to highlight how they come to be ongoing conditions within organizations rather than simply passing issues. The uncertainties we describe here are not unresolvable, while they were certainly important the time of this research. If and when these specific uncertainties are put to rest, the potential for uncertainties like them to become institutionalized in this organizational context will not be. We write to draw attention to specific uncertainties, their effects, and the conditions in which they can emerge and become institutionalized.

This article is a study of such institutionalized uncertainties and their effects in what sociologist Michael Power has called “riskwork,” the practical work that, whether or not it is explicitly a matter of risk analysis or management, is nonetheless crucial to managing and mitigating risk (Power 2016). For Power, this is not necessarily a coherent category or specific kind of job. He uses the concept of riskwork to highlight what he calls “back office” work on risk, which he describes as “the work that creates order and representations of orderliness, the work by which rational organizational narratives of risk, control, and governance come to be assembled and exported to different audiences” (Power 2016, p. 8).

Centering the back office work that makes ShakeAlert possible means foregrounding parts of the system that may be often discussed but rarely written about. We first frame our research on this professional practice in the context of calls for complex collaboration that inform the specific uncertainties that the ShakeAlert community of practice (Lave and Wenger 1991; Li et al. 2009)—that is, the people who are involved in ShakeAlert’s operation in many different ways—negotiates. We then trace ShakeAlert’s history of growth, demonstrating the project’s form in relation to these calls. We explain this study’s research methods, showing how we conducted and analyzed participant observation and interviews from between 2019 and 2021 using grounded theory methodology (Charmaz 2006). Drawing on data about earthquake early warning’s community of practice, we reveal three kinds of institutionalized uncertainty that interviewees described as part of their experience of ShakeAlert’s complexity. First, we consider conceptual uncertainty around the purpose of the ShakeAlert system; second, uncertainty related to organization and collaboration; and third, uncertainty related to use and users. We discuss what these three modes of institutionalized uncertainty may mean for a project like ShakeAlert, raising concerns related to how these uncertainties have diminished opportunities for members of key groups, including social scientists and user communities, to engage with the ShakeAlert project.

This study is part of a growing effort to attend qualitatively and ethnographically to how experts take on the tasks of assessing and addressing risk (i.e., Knowles 2012; Tironi et al. 2014; Daipha 2015; Revet 2020; Reddy 2023; Remes and Horowitz 2021). Although the institutionalized uncertainties we document here are by no means unique to state-coordinated projects like ShakeAlert a point argued very well in DiTomaso (2001), considering uncertainties in this context can help us better understand what kinds of work earthquake early warning systems, and other projects like them, truly entail. We contribute to an understanding of institutionalized uncertainties around one earthquake early warning system, with implications for similar projects using this form of environmental monitoring technology and for collaborative hazard risk mitigation technology projects more broadly (including the many that are already in operation around the world and those still in development). Better understanding of disaster riskwork will not only be conceptually rewarding. It is our hope that it will facilitate a better awareness and management of institutionalized uncertainties the promises of which for increased organizational and project effectiveness are summarized in Hudson-Doyle et al. (2018) or, at the very least, mobilize support for those riskworkers who must navigate uncertainties like those we describe.

It is by now commonplace to argue that there are no natural disasters that is, to assert that disasters result from factors related to physical environment and social life (Maskrey 1993; Wisner et al. 1994, 2015). Disaster researchers have made the case that social conditions and physical ones must always be included among the causes of human suffering. Like the earthquake disasters it is designed to prevent, ShakeAlert is also the product of its conditions. It has come into being in a specific context, shaped by existing technical capabilities, organizational structures, physical conditions, and the labor of many people. As Power’s research on riskwork highlights, this back office labor may be significantly less orderly than the products it produces.

Academic research on hazards often has recommendations for risk mitigation work that emphasize increased collaboration across agencies and communities, which, in turn, leads to increases in technical and organizational complexity. In their discussion of calls for sustainable hazard mitigation work in the US during the latter part of the 20th century, Mileti and Gailus (2005) show how interdisciplinary and interagency partnerships have often been recommended strategies for achieving well-considered and well-supported mitigation and resilience-oriented work that exceeds the expertise of a single actor or agency [referring to the work of White and Haas (1975) as well as observations by Burby (1998), Kunreuther and Roth (1998), Mileti (1999), Cutter (2001), and Tierney et al. (2001)]. More recent accounts of the field show an ongoing drive toward various “cross-disciplinary and cross-organizational collaborations” to confront hazards (Peek et al. 2020) [see also Pulwarty et al. (2009) and Miller et al. (2016)]. Deliberate choices related to collaborations can not only mobilize new resources, but also facilitate better attention to historically marginalized communities. This can be especially useful for supporting meaningful and appropriate engagement with those most affected by disasters (e.g., Maldonado 2014; Seager et al. 2017; Reddy et al. 2022). On the complexity of meaningful inclusion in decision-making for members of such communities, see Schlosberg (2004) and Marion Suiseeya (2016).

The “hazard community” that Mileti worked so hard to support in his life is evident in the ways that academic, nongovernmental organizations, and government organizations alike have demonstrated interest in the power of collaborative risk mitigation activities. For example, the UN Office for Disaster Risk Reduction’s Sendai Framework emphasizes the need for local, regional, and national coordination of activities to mitigate hazard and promote resilience. The framework it laid out for 2015–2020 encouraged members to set specific goals related to reducing various disaster effects and support each other in capacity building work. In doing so, it highlighted the guiding principle of “coordination mechanisms within and across sectors and with relevant stakeholders at all levels” (UNISDR 2015, p. 13). While researchers have identified this issue as a particular challenge for work done in the US (Wilkins et al. 2021), it remains a goal of the nation’s disaster risk reduction efforts. USGS itself has taken up the cause of greater outward-looking work. The recent report “Science for a Risky World” addresses these themes as it encourages seeking out partnerships, making the case that “scientists and stakeholders must collaborate to match community needs with actionable insights, research, products, and tools, using advances in technology to improve information discovery and delivery” (Ludwig et al. 2018, p. 1). The authors noted that “effective engagement with external partners to understand their needs and to deliver the right research and products in a timely and appropriate format is essential to successful risk reduction” (Ludwig et al. 2018, p. 17). As is the case in many big-picture reports, these descriptions of the rewards of such collaborations are somewhat instrumentalist, focusing narrowly on specific outcomes. However, organizational studies scholars also remind us that interorganizational collaborations are never simply “a series of discrete transactions.” Instead, the effects of these activities accumulate. Cultivating collaborative practice over time facilitates the development of new skills, relations, and knowledge for the people and organizations involved (e.g., Powell et al. 1996, p. 119).

Collaborations that cross disciplinary and organizational boundaries are practically useful efforts to prevent and respond to disasters. Instrumentalist descriptions often neglect how partnerships, whatever forms they take, entail practical and conceptual challenges. Scholars have demonstrated that interdisciplinary and interagency disjunctures related to hazards are frequent and sometimes serious (Moezzi and Peek 2021; Browne et al. 2018; Sapat 2021), even among committed and thoughtful parties. Developing relevant practices, expectations, concepts, and goals to undertake a shared project can be difficult. Some forms of knowledge are simply hard to align. This challenge, especially when people trained in one discipline have more power over a project than those trained in another do, can be a recipe for tokenism or even complete disregard. Peek and Guikema put it pointedly: there are many barriers to interdisciplinary collaboration, and they “have been well-described” (Peek and Guikema 2021, p. 1050). Nonetheless, the specific conditions of riskwork remain important to development of hazard risk mitigation tools.

Collaboration challenges are one reason that hazards scholars Scolobig and Pelling make a point to describe projects like ShakeAlert as a matter of more than good science. They write, “Decisions…are not determined by technology or statistical rubric but by the relational interaction between interested and varyingly informed parties” (Scolobig and Pelling 2016, p. 9). They argue that there is a significant need to understand the effects of social and cultural processes on disaster risk mitigation. However, putting such insights into practice presents challenges. While substantial theoretical tools for grappling with expert material and social relationships pertinent to collaborative expert work exist, engaging with them requires fundamentally rejecting the traditional assumption that risk mitigation is a straightforwardly rational exercise of expert economic and technical decision-making. It may present conceptual difficulties, but interrogating riskwork in this way can have real benefits. For example, as Scolobig and Pelling note, doing so can facilitate a kind of critical reflection and learning that has become increasingly important within contemporary disaster risk mitigation practice (Scolobig and Pelling 2016, p. 8).

The attention to social and cultural practice that Scolobig and Pelling advocate can be found in scholarship in fields like history of science and science and technology studies. This work has a record of highlighting diverse social, political, and disciplinary agendas that emerge around earthquakes (e.g., Valencius 2013; Coen 2013; Finn 2018; Reddy 2023). Bringing this kind of attention to bear on ShakeAlert, we center institutionalized uncertainties that participants describe in relation to ShakeAlert’s development during and immediately before the research period. Vertesi writes about such uncertainties as a condition of labor in mission-driven organizations, in which organizations “contract their spending and compete in a resource-limited environment, while benchmarks for success and expectations for outcomes are high” (Vertesi 2020, p. 475). Scholarship on uncertainty often refers to the work of Frank Knight, who used it to refer to situations in which possibilities exist but evade calculability (Knight 1921). However, contemporary transformations in both social science theory and scientific practices have complicated this definition significantly in the years since he wrote. Considering the contemporary state of uncertainty, sociologist Phaedra Daipha writes that “the need to acknowledge—and address—the limits of human understanding appears more acute and inescapable than ever before” (Daipha 2012, p. 15). In many cases, she argues, mitigating the negative effects of uncertainties—when reducing the uncertainties themselves is not possible—has become crucial to knowledge work in general, and hazard risk mitigation work in particular, during the late 20th and early 21st century (see also Neale and Weir 2015). Indeed, the literatures relevant to hazard risk mitigation often engage uncertainties, and may do so in different ways. Sword-Daniels et al. (2018) suggested that they tend to take the topic on as either as epistemic or aleatory; that is, a product of limited knowledge or natural variability. Either way, uncertainty marks multiple aspects of riskwork in ways that they describe as “multifaceted and prevailing in different forms, and at different levels and time scales” (Sword-Daniels et al. 2018, p. 291).

ShakeAlert is built from complex interdisciplinary and interagency relations, and these contribute to the institutionalized uncertainties that those operating the system must work within. Vertesi showed that such uncertainties can have implications for the approaches that experts take to their projects—implications that may be seen in the ways they frame their research and goals. As we discovered through our empirical research, institutional uncertainty has a significant presence in the ShakeAlert system. Our sustained research allows us to consider this uncertainty and its implications in detail.

As the scientist who we quoted in the opening of this paper sought to convey, ShakeAlert is complex in ways that escape easy description. ShakeAlert is a single project, but also a network of collaborators in what some have called the “triple helix” of academia, enterprise, and government (Leydesdorff and Etzkowitz 1998); see Berman (2012) for a history of this shift in US academia and Pamatang et al. (2012) and Yoon et al. (2017) on how nonprofit organizations like the Gordon and Betty Moore Foundation, which has funded ShakeAlert, should be understood as actors within such context. The participants involved are drawn together from multiple disciplines and organizational spaces. Their institutionalized uncertainties are framed by inconsistent stop-and-start growth of the system, shaped by rapid changes in funding and the transformation of a small academic project to something much bigger, as well as the fundamental differences between places and organizations brought together in the project.

Many of those interviewed for this project suggested that the complex organizational space was challenging to describe, much less work within. One emergency manager explained that, “sometimes it looks like a big bowl of spaghetti…trying to figure out where all the pieces fit and where we are and where we should be.” To illustrate this point, we offer a brief overview of the system. ShakeAlert’s sensor network is made of many regional components of the Advanced National Seismic System: they are the Pacific Northwest Seismic Network (PNSN) and the California Integrated Seismic Network (CISN) [itself comprising the Southern California Seismic Network (SCSN) and the Northern California Seismic System (NCSS)]. ShakeAlert materials circulating at the time of this writing note a plan to one day incorporate 1,675 stations (Given et al. 2018). Stations are operated, and the data they produce processed, at universities, primarily by teams based at California Institute of Technology (CalTech); University of California, Berkeley; University of Oregon; University of Washington; University of Nevada, Reno; ETH Zurich; and the nonprofit UNAVCO or University NAVSTAR Consortium, now part of the EarthScope university consortium to facilitate geoscience research, and distribution partners like Google or California’s public transit systems designated “technical users” and given licenses to use ShakeAlert data in special ways. The USGS acts as a coordinator in context of ongoing tension between the needs for locally relevant alerting regimes and consistency across the US West Coast region.

ShakeAlert itself dates to 2007, but the effort grew to include partners throughout the West Coast–wide ShakeAlert efforts began in 2011, when University of Washington became a primary ShakeAlert center with seed funding from the Gordon and Betty Moore Foundation. New algorithms for creating accurate warnings for the earthquakes experienced throughout the region and partnerships with early adopters helped shape the system’s growth. Some national political support and funding came through in the final years of the Obama presidency (2009–2017), but despite perennial emphasis on infrastructure development, federal funding was challenging to obtain during the subsequent Trump years (2017–2021). People across public administration and science report feeling that support for their work was sparse during the Trump presidency (Goodsell 2019; Goldman et al. 2020). One ShakeAlert administrator in USGS commented on this obliquely in an interview, saying, “what I do is driven by a desire to counteract the effects of people from very high up who don’t really trust the federal workforce.” In this political context, the ShakeAlert project had to depend heavily on inconsistent financial support from state governments just as it launched.

As the project began, resources were primarily allocated to the physical scientists and engineers developing ShakeAlert’s technical function. However, when ShakeAlert prepared to go live, decision-makers began to fund other priorities to make the system usable. In 2016, the same year that USGS began operating a prototype ShakeAlert system in California and allowed test users to start taking actions in response to alerts, the ShakeAlert Joint Committee for Communication, Education, Outreach, and Technical Engagement was formed (de Groot et al. 2022). The USGS soon began to actively seek to fund social scientific work on earthquake early warning, supporting research with implications for ShakeAlert’s organization, messaging, and outreach plans (including this study). However, this support was late to develop. Social science about ShakeAlert only became a priority for the USGS in 2018, and the social scientists it funded were unable to meet as the Social Science Working Group before 2019. While the working group was a fertile site for the development of innovative and practical research on early warning, the findings its members eventually produced had little opportunity to influence ShakeAlert planning during the crucial time before the system began to go live.

By 2019, as these social scientists began to focus robust attention on topics relevant to ShakeAlert, California was already issuing public alerts. It was joined by Washington and Oregon in 2021. That year, further funding became available for system expansion: not just from the states involved, but from US Congress, which devoted $25.7 million to the USGS for ShakeAlert. The funding was welcome, but still not entirely adequate for the system’s goals. In an interview, one ShakeAlert project manager described struggling to prove the worth of ShakeAlert to people who expected “100% performance at 50% funding levels.” Others described how they see the same funding issues that framed different technology rollout timelines in the three states also impact decision-making power. One noted that, perhaps because Oregon and Washington are later adopters of ShakeAlert than California and perhaps because these states have been able to mobilize much less funding to the project than their neighbor to the south, “the Pacific Northwest always thinking that California is the oldest sibling is getting more of the pie.”

While important for the development of ShakeAlert, these conditions are rarely the focus of published research on the system. Instead, researchers have documented and analyzed other aspects of ShakeAlert’s various developments extensively, addressing such issues as motion thresholds and timeliness (Minson et al. 2018; Wald 2020; Saunders et al. 2022), platforms (Rochford et al. 2019; Allen et al. 2020), state-of-the-art in different regions (Allen and Kanamori 2003; Allen 2007; Wu et al. 2007; Allen et al. 2009a; Böse et al. 2012; Hartog et al. 2016), performance evaluations (Böse et al. 2009a; Cua et al. 2009; Chung et al. 2020), and new forms of analysis possible (Cua and Heaton 2007; Böse et al. 2009b; Thakoor et al. 2019). Members of the Social Science Working Group have generated research findings on topics including messaging (McBride et al. 2020; Sutton et al. 2020), education campaigns (Jenkins et al. 2022; Sumy et al. 2022), public perception of ShakeAlert and earthquake risk (Bostrom et al. 2022), and protective actions (Wood 2018; McBride et al. 2022).

One participant in this research simplified this story by diagramming ShakeAlert as a balanced system, a kind of blossoming Venn diagram, in which experts with different kinds of knowledge and embedded in different institutions united in service to a project they shared ownership of, for the benefit of a general, undefined public (see Fig. 2). This simplified figure helped him tell a story about dense interdisciplinary and interagency partnerships that are all crucial to ShakeAlert and, as demonstrated previously, deeply challenging to parse in their complexity. It revealed one way of seeing the system—in which the public is firmly on the outside of the process, some forms of expertise and components of riskwork are clearly delineated while other distinctions are lost, and ownership of “ShakeAlert Central” seems straightforward and shared. While this model offers a lovely symmetry, it is the purpose of this paper to tell a messier story.

In service of this effort, the authors far prefer the explanation given by a scientist based at a participating university. He noted the sensible reasons that the ShakeAlert project has taken the form we see today while critiquing their cumulative effects. “If you were designing it from scratch, it probably wouldn’t look like that at all,” he said. A changing project means things that once worked no longer do, and members of the ShakeAlert community need to find ways to deal with that.

Themes related to the experience of working in the context of institutional uncertainty were not predetermined topics for this investigation, but emerged during qualitative research using methods related to grounded theory. This research in follows Clarke (2005) and Charmaz (2006) in approaching grounded theory as a tool for constructivist research, which offers “systematic, yet flexible guidelines for collecting and analyzing qualitative data to construct theories ‘grounded’ in the data themselves” (Charmaz 2006, p. 2). We chose this method because it entailed comprehensive, inductive, and iterative analysis of data appropriate for a team of researchers with different levels of experience (including undergraduate and graduate students as well as an experienced ethnographer) to collaboratively seek theoretical insights. Through this process, we developed a theory of ShakeAlert from the perspective of the operators on whom it depends. This account highlights and explores uncertainties in order to offer fundamentally different insights into ShakeAlert than those that readers are likely to encounter elsewhere.

To develop these insights, the research team engaged in participant observation and semistructured interviews to generate empirical data, and then studied it by using qualitative coding, synthesis, and review of relevant literature. Preliminary findings also guided follow-up questions asked in subsequent interviews to facilitate deeper consideration of emergent themes. This process allowed us to engage in what anthropologists Ballestero and Winthereik have described as “a form of analysis that creates an opening for making sense of something that [researchers] cannot fully anticipate” (Ballestero and Winthereik 2021, p. 2). We put our insights developed through this into explicit relation with existing scholarship in the fields of science and technology studies and hazard risk mitigation.

Our research was originally undertaken to better understand how the ShakeAlert project integrates at least 200 people (as one USGS interviewee conservatively estimated) to bring their expertise to bear on ShakeAlert riskwork. Incorporating scientists, technicians, administrators, project managers, and emergency managers is no small achievement. However, this is only a small portion of the people whose work somehow involves or supports ShakeAlert. A more robust estimate of the size of the ShakeAlert team might include policy workers who deal with its regulations among their many tasks, users who spend time integrating ShakeAlert into the technical and social systems they manage, and many others who are not paid to focus on ShakeAlert but whose efforts are nonetheless required to operate the many infrastructures that ShakeAlert riskworkers depend on. Such an estimate might incorporate hundreds more people into the ShakeAlert project’s tally.

We planned this research to capture perishable data regarding how members of this community experience ShakeAlert System developments; document how members of this community understand and prioritize aspects of the system’s function and community membership; and note social and technical factors that members of this community understand to support and/or challenge their multisite and multidisciplinary collaborative work. We did this in four main phases, as demonstrated in Fig. 3: preliminary participant observation, scoping, and recruitment, in which a researcher attended and sometimes actively participated in ShakeAlert community meetings; intensive in-person participant observation and interviews in Washington accompanied by preliminary coding; further digital interviews undertaken with affiliates in California and elsewhere along with a second round of analysis; and, finally, member checking findings with participants.

First, participant observation at events for several months was performed to scope research. The first author began to engage in some ShakeAlert events digitally to learn about the community, documenting her experiences extensively. She approached ShakeAlert leadership and then site managers in order to gain permission to reach out to the people on their teams. She then developed a semistructured interview protocol in consultation with the ShakeAlert Social Science Working Group. These interview questions focused on several topics: first, how ShakeAlert might be defined; second, the experience of operating ShakeAlert; and third, the social practices and technical tools that provide help or challenges for these activities. Exploring these themes created opportunities for interviewees to delve into contemporary issues of relevance and broad reflections on ShakeAlert experiences.

While original research plans involved recruiting interviewees during participant observation at meetings and key workspaces to recruit interviewees with diverse professional relationships to ShakeAlert, this was only possible in Washington state in early 2020. In light of the COVID-19 pandemic, plans changed. The in-person visits planned as part of this project were put aside in favor of digital forms of co-presence (Beaulieu 2010). Casual recruitment plans changed to targeted recruitment plans. We focused on sampling with preference for those participants with experience to help contribute to cogent, thorough observation and help us better understand emerging themes and categories what Charmaz (2006) calls “theoretical sampling”. Interviews that we collected are summarized in Fig. 4. We present them in an amalgamated fashion to preserve interviewees’ anonymity.

In Washington, 19 interviews were conducted (mostly individually, but some in groups of two or three). These interviewees shared reflections about their experiences with ShakeAlert from positions at Washington Emergency Management (WAEMD); at PNSN at the University of Washington, Technical User Groups; and at USGS. Interview recordings were transcribed, and returned to interviewees for modifications. Two people did not return edited interviews with their approval, so we were only able to use 17.

Next, a series of digital interviews were undertaken. In California, 13 interviews were conducted using Zoom videoconferencing software. We recruited these participants to represent major groups and in relation to their ability to reflect on current and former experience at California Office of Emergency Systems (Cal OES); at SCSN; at NCSS; at University of California, Berkeley; at California Technological Institute (CalTech); and at USGS. Again, one person did not return edited interviews with their approval, so we only analyzed 12.

Finally, we pursued additional interviews with ShakeAlert affiliates within USGS as well as members of companies designated as technical users and licensed to develop products using ShakeAlert data. These research participants were not closely tied to any of the participant user states but had insights related to the whole system. We conducted seven interviews that fit into this category.

Participants at other sites are not represented in this study; and analysis holds their absence in tension with a commitment to considering the importance of including diverse voices from this community. For example, a number of key ShakeAlert riskworkers based in Oregon were unable to participate in this research. After repeated recruitment attempts throughout the course of data collection period were unsuccessful, we decided to focus research on Washington and California. Participants in Washington addressed the PNSN network and noted issues pertinent to working on earthquake safety in a region less likely to experience earthquakes and so less prepared for them than California [a connection shown in Aotearoa New Zealand by Becker et al. (2017) and borne out in survey research on the US West Coast by Bostrom et al. (2022)], if not to comment on the Oregon rollout itself. As such, findings in this paper do not directly address Oregon’s situation. It is also important to note that only one member of the ShakeAlert Social Science Working Group was interviewed as part of this research. Instead, extensive participant observation and consultation with this team throughout the research process has contributed insights related to their experience of ShakeAlert.

Some parts of interview treatment were consistent throughout the research and analysis process: each participant gave consent, each of the 36 interviews were transcribed, and each was returned to the relevant research participant to modify and correct. Overall, all but three participants returned their interviews and gave consent for the research to move forward (see Fig. 4). Each usable interview was then coded using Dedoose version 8.3.41 qualitative analysis software and password-protected Google document tools. However, processes differed there: we analyzed early interviews using open coding to identify themes and consider themes in relation to each other. We then developed a set of focused codes significant for this research. This set of codes informed the analysis of subsequent interviews. We established inter-rater agreement through extensive conversation, first between coders and then among the entire research team. We did not code notes from participant observation in the same way. Instead, we let these notes support insights into the context of the interviews. In this sense, notes from participant observation were also useful for developing theoretically meaningful and empirically grounded interpretations of interview data.

We wrote up summaries of qualitative insights for Washington, Northern California, Southern California, and those working out of USGS’s headquarters in Reston, Virginia, representing businesses exploring becoming licensed ShakeAlert partners, and other interested parties who we simply classified as “other” (a division that helped us to consider issues related to different regions and their characteristic forms of seismicity, relevant policies, and what it might mean to be embedded in them or external to them). We made use of these to member-check our findings. We recruited interested parties from among our interviewees and presented them with the summary most relevant to them. We asked people if these findings seemed in keeping with their feelings and experiences, or those that others had expressed to them. We also invited them to respond, and to support or provide counterpoints as they might see fit. All findings we share here are those we agreed on with the self-selected participants—among them technicians, scientists, and administrators as well as representatives of universities, USGS, and emergency management.

Our methods for systematic qualitative study required our ongoing participation in the ShakeAlert community. This engagement facilitated access that more rapid or lower-contact methods of research could not achieve. The research team developed familiarity with ShakeAlert and with members of the ShakeAlert community. This allowed us to pursue germane lines of questioning in interviews and to consider participants’ responses in relation to the events and conditions affecting the wider ShakeAlert community. Ethnographic methods like these succeed because they involve building relationships between researcher and communities researched. These relationships are sometimes understood as limitations because a researcher who undertakes them cannot pretend to have an objective or detached perspective. However, such relationships also form the basis of respect and accountability that can help researchers produce useful and appropriate critique (Reddy 2023). We understand this to be part of ethnographic research.

Earthquake early warning entails a great deal of operational complexity. The many institutions that facilitate ShakeAlert’s ambitious scope also perpetuate uncertainties. Interviews with participants highlight how they experience these uncertainties and navigate them. The people interviewed operate this system at a high level of competency, and there is much they knew about operating this system. Highlighting and exploring these uncertainties, we can better understand the riskwork of ShakeAlert. Through this case, we can also reflect on the complications of enacting contemporary hazard risk mitigation—a field importantly shaped by Dennis Mileti’s research and advocacy, but which may entail challenges that undermine his vision of effective interdisciplinary and interagency collaboration.

As we have shown, institutionalized uncertainties can be a matter of a project understood differently by different participants. Not only does ShakeAlert incorporate collaborators with their own agendas and priorities and subsystems that are more or less integrated, but the key purpose of the project as a whole may be contested. Interviewees suggest that ShakeAlert could conceivably be approached in different ways and held to a variety of standards. Here, institutionalized uncertainties means that the project can contain substantially different values and priorities. While these emerge because of the complexity involved, they are not necessarily represented by people in different collaborating organizations or even different disciplines. They certainly can be (emergency managers and scientists, for example), but the ways that USGS managers explain their work demonstrate that conceptual multiplicity can live and thrive within a single organization. As scholarship in science and technology studies shows, collaborators need not share an understanding of how their joint undertaking works (e.g., Star and Griesemer 1989; Mol 2002; Bowker et al. 2015). However, interviewees themselves highlighted these areas for us as spaces worthy of consideration that they cannot simply ignore. They need to make design decisions, and some of the values and priorities presented to them would entail mutually exclusive choices. These inform their conceptual and practical experience of their work on ShakeAlert and may be related to the project’s collaborative nature. Because of the ways that strongly interdisciplinary and interagency work has come to define large hazard risk mitigation efforts, such uncertainties merit attention.

Uncertainty related to management and collaboration procedures can arise in growing teams and expanding collaborations. Such confusion can be understood in the context of ongoing institutional changes. Reflection on the history of ShakeAlert reveals that the system has been influenced by factors including scientific and technical developments; institutional partnerships; state-specific laws; and influxes of funding from nonprofit organizations, states, and federal sources, as well as funding constrictions as one presidential administration gave way to the next. An interdisciplinary and interagency effort to manage what we have followed a ShakeAlert scientist in calling “the complexities” of earthquake early warning under these conditions had consequences for emerging organizational structure and decision-making responsibilities. The different needs and roles of collaborators were not necessarily clear, and this presented challenges for the project as it developed. The experience of working through ShakeAlert has been defined by interviewees as a matter of emergent and sometimes contested relationships that surfaced institutionally in the context of diverging and overlapping areas over which people claimed expertise and power.

It is well-established by empirical science and technology studies research that scientific and technical activities rely on social organization (e.g., Latour and Woolgar 1979; Traweek 1988; Knorr-Cetina 1999). Indeed, stories that interviewees shared about their experiences with ShakeAlert suggest that they found existing social connections were useful for smoothing rough interactions and even making their voices heard to shape ShakeAlert. However, such resources were not broadly shared by all involved in the project, which may well have been, and continue to be, a limiting factor for participation of collaborators. Where observers of earthquake hazard risk mitigation work in the US have noted that demonstrating how institutions can make meaningful collaboration across “theoretical and cultural-disciplinary gaps” (Kendra and Nigg 2014, p. 134), uncertain institutions cannot be relied on to do the same. Throughout its history, people trained in physical science have led the ShakeAlert project. While social scientists have become increasingly involved in the project, their inclusion was late in the project’s development. Most do not have the same long-term professional and social relationships with those in project decision-making positions that other ShakeAlert participants might, and the “theoretical and cultural-disciplinary gaps” they need to overcome in order to make collaborations work can be great. Perhaps these factors explain why social scientific insights and emergency management concerns related to public engagement did not substantially inform ShakeAlert’s design until the project was well underway. The comparatively late development of a working group focused on social science in ShakeAlert is particularly important to consider with respect to use and users.

Uncertainty around use and users is particularly significant for a risk mitigation project. Users are key to ShakeAlert’s successful function but proved difficult to address. Making warnings suitable for their intended recipients has been a significant challenge within the ShakeAlert project, despite the necessity of such work (Mileti and Sorensen 1990). Physical scientists that dominated this project in its early stages did not have relevant training to take on warning design and public education. They did not attend sufficiently to users’ practical and diverse education and warning communication needs. Appropriate partners in emergency management lost opportunities to involve the public in ShakeAlert’s design and build coalitions around the system early on in its development. Challenges conceptualizing the public are not unusual for projects related to disaster (e.g., Finn 2018), but this is the kind of situation in which the neglect of users or an unproductive, deficit-model approach to education and science communication is very likely (Wynne 1991, 1992). Here, uncertainty about how to take on novel tasks that were known to be necessary but fell within no central leaders’ specific expertise had consequences. If social scientists and emergency managers were relative latecomers to ShakeAlert, at least they are part of the diagram shared in Fig. 2. The user communities that both groups might advocate for including are even more removed. These groups are presented by the study participant who drew the diagram in Fig. 3 as just a backdrop for ShakeAlert activities, with no means of informing design decisions. A survey study by Bostrom et al. about public knowledge of ShakeAlert in California, Oregon, and Washington suggest that even without early stakeholder engagement and meaningful avenues for public participation, “majorities of respondents (over 60%) in all three states understood that ShakeAlert would send a warning that an earthquake was approaching” (Bostrom et al. 2022, p. 8). However, Bostrom and her team observe that participants still generally expressed a lack of knowledge about the system, and describe “gaps in understanding and expectations between both groups: the scientists and operators who manage the ShakeAlert system and the publics they seek to serve” (Bostrom et al. 2022, p. 19). Informed by their survey work, they suggest strategies including reducing alerting thresholds for interested users. One wonders what shape ShakeAlert might have taken if such survey results informed system design from the beginning, and how earlier community engagement might have influenced the popular knowledge about the system.

If we understand uncertainty as institutional rather than personal, it becomes even more important to consider what kinds of systemic effects it may have. Diverse interagency and interdisciplinary collaborations have been promoted to support effective hazard mitigation projects. This approach has been conceptualized as key to making earthquake early warning accessible and suitable for many uses. The complex partnerships supporting this project may well nurture certain kinds of innovation, but they also create conditions of possibility for uncertainties. The uncertainties described in this paper not only characterize the experiences of working on ShakeAlert, but have consequences for how the system has been, and will continue to be, designed, and maintained.

Study participants discussed these uncertainties at length in response to very general interview questions. Those who were involved in member checking preliminary research memos affirmed that these themes were important in their experience and merit attention. However, because work experiences, seismicity, legal regimes, and the events of ShakeAlert’s launch happened differently in each state, these insights may be more or less applicable to experiences in Oregon. Further, research primarily focused on different parts of the ShakeAlert community, particularly the ShakeAlert Social Science Working Group or ShakeAlert Technical Users, may reveal essential nuances that this study could not. Regardless of what additional research reveals, though, this much is worth contending with: while some interviewees cited convivial work environments and celebrated strong leaders who organize meetings, steer work, and advocate for team members within ShakeAlert, these operational strengths do not diminish the impact of institutionalized uncertainties. Such uncertainties impede ShakeAlert’s efforts to become what Mileti and Sorensen describe as “a warning system that takes advantage of existing social science knowledge and current technology to maximize the probability that the system will be effective when implemented” (Mileti and Sorensen 1990, pp. 1–2).

Hazard risk mitigation technology is shaped by the work it takes to produce it. This means that understanding ShakeAlert, and technologies like it, requires consideration of the riskwork done by the ShakeAlert community of practice. Our analysis of participant observation and interviews data leads us to highlight several forms of institutionalized uncertainties related to work on ShakeAlert. We situate these uncertainties in the context of the dizzying complexity of ShakeAlert’s historical and ongoing development. After showcasing how interviewees described organizational uncertainties, uncertainties about use and users, and uncertainties related to the nature of the system itself, we discuss these themes and consider the implications of research participants’ insights.

This research allows us to consider uncertainty as part of a large and rapidly developing collaborative project. ShakeAlert is unique in many ways, such as its slow start as a scientific research project, its rapid growth following funding from government sources, and its novelty for the many groups that contribute to it, including the USGS. ShakeAlert is one of many risk mitigation projects that relies on interdisciplinary and interorganizational collaborations. While we are sensitive to the great potential benefits of such collaborations, in this paper we seek to document the experience of working within them. Considering this experience can help us understand contemporary riskwork. Institutional factors create uncertainty, and that uncertainty, in turn, shapes risk mitigation tools and practices. It may not always be resolvable, but it—and its effects—must be accounted for.

Some or all data, models, or code generated or used during the study are proprietary or confidential in nature and may only be provided with restrictions. Raw data are confidential. Research memos (anonymized and approved by interviewees) may be shared upon request.

Research for this paper was supported by funding from the Earthquake Science Center of the USGS.