Boise State University

ScholarWorks

Public Policy and Administration Faculty Department of Public Policy and Administration
Publications and Presentations

7-2019

Problems, Politics, and Policy Streams in Policy Implementation

Luke Fowler
Boise State University

This is the peer reviewed version of the following article:

Fowler, L. (2019). Problems, Politics, and Policy Streams in Policy Implementation. Governance, 32(3), 403-420.
which has been published in final form at doi: 10.1111/gove.12382. This article may be used for non-commercial
purposes in accordance with Wiley Terms and Conditions for Self-Archiving.
 This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

Problems, Politics, and Policy Streams in Policy Implementation

Luke Fowler
Public Policy & Administration
Boise State University

Introduction

Kingdon’s (1995) Multiple Streams Framework (MSF) is one of the most widely applied and researched theoretical
frameworks of the policy process, and explains a wide variety of policy issues at national, supranational, and
subnational levels (Herweg, Zahariadis, and Zohlnhofer, 2018). Although originally developed to examine agenda-
setting, scholars have expanded MSF to explain policy adoption, and some argue for its applicability to policy
implementation. Although highly interrelated with policymaking, policy implementation involves different goals,
processes, mechanisms, actors, and relationships (Nakamura, 1987; Matland, 1995; Hill and Hupe, 2014; Zahariadis,
2014; Herweg, et al., 2018). Consequently, applying MSF to policy implementation meets several challenges in
reformulating conceptualizations of policy streams, windows, and entrepreneurs, which leads to limitations in current
scholarship (Herweg, et al., 2018). In fact, some scholars argue that effectively addressing policy implementation is
a key area of future research for MSF, along with developing more testable hypotheses and connecting findings to
democratic governance (Sabatier, 2007; Riddle, 2009; Zahariadis, 2014); although more recent developments alleviate
some of these concerns (Herweg, et al., 2018).

As such, we seek to further this line of research by applying MSF to explain policy implementation. In order to do
so, we first review extant literature on these topics, and then apply MSF’s core theoretical components to policy
implementation, as a distinct stage of the policy process that is separate but interrelated with policymaking. Second,
we use Travis and Zahariadis (2002)’s work on creating testable hypotheses from MSF to guide development and
empirically testing of conditional effects of policy, problems, and politics streams on policy outputs (i.e., results of
the policy process). Findings indicate that when any stream is held at moderate levels, streams only have marginal
effects on policy outputs; however, when all three streams are increased to extreme values, significant differences
emerge. Finally, our conclusions connect these findings to larger issues of applying MSF to policy implementation
and discuss how scholarship in this area may inform research on democratic governance.

Multiple Streams Framework

Kingdon (1995) argues a key variable in the policy process is ambiguity, where many ways of thinking about the same
conditions or phenomena results in vagueness, confusion, and stress (Kingdon, 1995; Zahariadis, 2014; Herweg, et
al., 2018). Ambiguity in policymaking results from: fluid participation, including legislative and bureaucratic turnover
or shifting advocacy coalitions; problematic preferences in which policymakers are unsure about their goals or how
policies affect them; and, unclear technology (e.g., organizational processes) that contributes to uncertainty about
individual responsibilities and how they relate to collective actions (Kingdon, 1995; Zahariadis, 2014; Herweg, et al.,
2018). Consequently, “we often don’t know what the problem is; its definition is vague and shifting. Distinguishing
between relevant and irrelevant information is problematic… Choice becomes less an exercise in solving problems
and more an attempt to make sense of a partially comprehensible world…[and] Who pays attention to what and when
is critical” (Zahariadis, 2014, p. 28). MSF seeks to add order to this complex set of circumstances. More specifically,
Kingdon (1995) identifies five structural elements where problems, politics, and policies stream independently until
policy entrepreneurs couple streams during open policy windows, leading to agenda-setting and decision-making.
Figure 1 diagrams these structural elements (Zahariadis, 2014).

First, problem streams consist of socio-economic or environmental conditions that are not ideal, policymakers and/or
citizens want to address, and require government (or collective) action. Problems gain attention through indicators of
conditions, focusing (or punctuating) events, or from feedback (Kingdon, 1995; Zahariadis, 2014; Baumgartner, Jones,
and Mortensen, 2018; Herweg, et al., 2018). Second, politics streams consist of the political atmosphere concerning
which issues are important and how interests should be balanced. However, advocacy coalitions and governmental
officials largely influence this by shaping public discourse on those issues (Kingdon, 1995; Zahariadis, 2014; Herweg,
et al., 2018; Jenkins-Smith, Nohrstedt, Weible, and Ingold, 2018). Third, policy streams consist of “a ‘primeval soup’

1
 This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

of ideas that compete to win acceptance in policy networks... generated by specialists in policy communities”
(Zahariadis, 2014, p. 33). While a number of ideas may exist, policy networks sort through these with only a few
obtaining serious consideration based on technical feasibility, value acceptability, and resource adequacy (Zahariadis,
2003, 2014; Herweg, et al., 2018).

[Figure 1 about here]

Fourth, policy entrepreneurs include individuals or organizations who serve as power brokers that advocate for their
preferences and attempt to couple streams (Kingdon, 1995; Mintrom and Norman, 2009; Zahariadis, 2014; Herweg,
et al., 2018). Policy entrepreneurs employ strategic political manipulation to ensure pluralistic interests interpret
ambiguity as serving diverse aims. Zahariadis (2014) describes it as “in a world replete with ambiguity, the most
important aspect of entrepreneurial activity is not to pursue self-interest but to clarify or create meaning for those
policymakers, and others, who have problematic preferences” (p. 30). As such, policy entrepreneurs employ strategies
to convince policymakers that policies shrouded in ambiguity are politically good enough to satisfy their problematic
preferences. These include framing perceptions of gains and losses, using symbols to elicit emotional responses, and
manipulating sequential decision-making (Zahariadis, 2005, 2014; Herweg, et al., 2018).

Finally, policy windows are fleeting “opportunit[ies] for advocates of proposals to push their pet solutions, or to push
attention to their special problems” (Kingdon, 1995, p. 165). When streams are coupled during open policy windows
created by events in the problems or politics streams (i.e., focusing events), policymaking is more likely to be
successful (Zahariadis, 2014; Baumgartner, et al., 2018; Herweg, et al., 2018). However, it depends on skills,
resources, and strategies from policy entrepreneurs, including avoiding a mismatch between policies, problems and
politics streams (Avery, 2004; Birkland, 2004; Zahariadis, 2014). Although recent developments address some
criticisms of MSF, there remains both shortcomings and directions for future research. These include incorporating
policy implementation, creating stronger links to democratic governance, and generating testable hypotheses and
empirical evidence. While a few scholars examine implementation through the MSF lens, there are few broad
explanations of implementation processes or of how streams link policymaking to implementation (Riddle, 2009;
Zahariadis, 2014; Boswell and Rodrigues, 2016; Zahariadis and Exadaktylos, 2016; Herweg, et al., 2018).
Consequently, implications for governance are still limited, as governance considers the whole policy process and not
just policymaking (Kettl, 2015; Herweg, et al., 2018).

Policy Implementation

Theoretical developments in policy implementation have stalled in recent decades, especially as networks and
governance emerged as a focal point for scholars. While first generation studies were largely atheoretical and case
study based, subsequent studies focused on developing theory. Consequently, two competing frameworks emerged:
top-down, which argued implementation behavior is centrally controlled with little or no administrative discretion;
and bottom-up, which argued largely ambiguous policies require interpretation causing street-level bureaucrats to use
administrative discretion. Subsequently, third generation studies developed meta-theories, employed large-N studies,
and attempted to reconcile these two perspective (Goggin, Bowman, Lester, and O’Toole, 1990; Matland, 1995;
O’Toole, 2000; deLeon and deLeon, 2002; Hill and Hupe, 2014). Contemporarily, implementation theories have
largely been supplanted by governance frameworks, which incorporate inter-organizational links that differ from
hierarchical or market arrangements and include both public agencies and non- governmental organizations (NGOs)
(O’Toole, 2000, 2015; Hill and Hupe, 2014). While theories differ along several important lines, a common trend is
an assumption that ambiguity invites administrative discretion, which influences how collective action is undertaken.

More specifically, when policies are ambiguous, implementers can interpret them in multiple ways. While it may
negatively impact organizational performance, ambiguity can be an effective tool for reducing conflicts during
implementation, in much the same way as it allows policy entrepreneurs to breed consensus in policymaking (Chun
and Rainey, 2005a, 2005b; Pandey and Rainey, 2006; Pandey and Wright, 2006; Davis and Stazyk, 2015). For
instance, Davis and Stazyk (2015) argue that “when high levels of political conflict are present, public organizations
must find some way to mollify disparate interests to attain (or even retain) political support. One method of doing so
rests in how organizations construct, shape, and convey their goals” (p. 754). Furthermore, in uniting top-down and
bottom-up perspectives, Matland (1995) argues that levels of ambiguity and conflict dictate different strategies,
including both centralized decisions and bottom-up discretion. Similarly, other scholars argue that when faced with
high degrees of ambiguity, organizations adapt their mechanisms to protect status quos and satisfy political coalitions
(Thompson, 1967; Donaldson, 2001; Stazyk and Goerdel, 2011).

2
 This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

Central to issues of ambiguity is administrative discretion, how it shapes implementer behavior, and how it affects
public service delivery (deLeon and deLeon, 2002; Hill and Hupe, 2014). Policymaking tends to conclude with a
decision on a policy to be used (i.e., policy adoption) that is preceded by a series of smaller decisions (i.e., agenda-
setting). However, policy implementation begins with a decision about what policy is to be implemented and is then
followed by a course of smaller decisions made by implementers as they interpret said policy. Thus, decision-making
is fundamentally different in implementation than in policymaking. As such, there is a rather rigorous debate over
administrative discretion, concerning how it is used, who uses it, and how to create accountability when it is used
(Holzer and Yang, 2005; Meier and O’Toole, 2006; Hupe and Hill, 2007; Kettl, 2015). While they may disagree
about specifics, scholars recognize that implementers have decision-making power, which pluralistic interests seek to
influence in order to align policy implementation with their goals.

Consequently, patterns of ambiguity and administrative discretion largely influence what types of collective actions
are undertaken. In many ways, policy implementation is a specific instance of collective action, as it requires
collective choices and responsibilities from numerous actors (Hill and Hupe, 2014). Second generation theoretical
frameworks largely viewed implementation as occurring within a single organization made up of individuals who
responded to organizational incentives, rules, and cultures (O’Toole, 2000; deLeon and deLeon, 2002). However, third
generation theoretical frameworks are mixed with some focusing on collective action within a single organization
(e.g., Matland, 1995) and others on collective action involving multiple organizations (e.g., Goggin, et al., 1990).
Nevertheless, in recent decades, these lines of inquiry evolved (or were absorbed) into governance frameworks that
seek to understand collective action in multi-level, interdependent layers of complex systems (Lynn, Heinrich, and
Hill, 2001; Hill and Hupe, 2006, 2011). Additionally, governance is also concerned with entire policy processes and
assumes that policymaking and implementation are intertwined (Nakamura, 1987; Hill and Hupe, 2014; Kettl, 2015).
Notably, this coincides with an emphasis on understanding how NGOs participate in public service delivery, both
directly and indirectly through contracting, partnerships, and/or advocacy coalitions (Feiock and Scholtz, 2009;
Feiock, 2013; Hill and Hupe, 2014; Jenkins-Smith, et al., 2018; Kettl, 2015).

Connecting MSF to policy implementation expands on these inherent links. As such, there are a few notable examples
of previous scholarship that uses MSF as a framework for analyzing implementation processes. For instance,
Aberbach and Christensen (2014) use Kingdon’s work to explain why implementation of administrative reforms are
prone to failure, as implementation unfolds through a complex decision-making process. More directly, in analyzing
policy targets in the United Kingdom, Boswell and Rodrigues (2016) argue different implementation approaches are
the result of conditional effects that occur when policy and politics streams interact. Alternatively, Riddle (2009), in
analyzing an international cooperation project in Burkina Faso, argues that implementation is a function of an
interaction between problems and policy streams, with politics streams only loosely coupled to the others.
Additionally, Zahariadis and Exadaktylos (2015) identify the use of entrepreneur strategies and political manipulation
during implementation of Greek higher education reforms that are similar to behavior of policy entrepreneurs in the
policymaking process. However, these provide only limited applications of MSF, with those scholars calling for
additional research.

Applying MSF to Policy Implementation

Previous scholarship provides three important theoretical underpinnings for applying MSF to policy implementation:
1) policy systems and processes are nested, with policymaking and policy implementation existing in organized
interdependent layers (Howlett, McConnell, and Perl, 2015; Zahariadis and Exadaktylos, 2016); 2) policy
entrepreneurs use political manipulation to influence implementation processes, similarly to policymaking processes
(Aberbach and Christensen, 2014; Boswell and Rodrigues, 2016; Zahariadis and Exadaktylos, 2016); and, 3)
implementer behavior is driven by threats to status quos with problems and politics streams affecting commitment to
policies (Boswell and Rodrigues, 2016; Zahariadis and Exadaktylos, 2016). Consequently, our initial assumption is
that there are separate policy, politics, and problems streams as well as outputs from policymaking and policy
implementation processes individually and the policy process as a whole (i.e., includes both policymaking and
implementation), which are interconnected but distinct. By extension, the classic MSF approach to policymaking only
describes the five structural elements for the policymaking portion of this model. As such, we then assume the basic
structural elements of MSF still exist during policy implementation, are intertwined with their policymaking
counterparts, and must be adapted to fit this new context. Figure 2 diagrams our MSF model of policy implementation
processes, based on the classic MSF approach and previous scholarship in this area where each structural element is
reformulated to the context of policy implementation.

3
 This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

[Figure 2 about here]

First, there is a complex relationship between policymaking outputs, implementation outputs, and policy outputs.
Importantly, policy outputs reflect results produced from the entire policy process, while policymaking outputs and
implementation outputs only reflect results from their respective portions of the entire policy process. To start,
policymaking outputs are new policies aimed at changing conditions that are not ideal (Zahariadis, 2014).
Policymaking outputs trigger policy implementation, where individual behaviors of implementers (e.g., enforcement
action) aggregate to implementation outputs. Implementation outputs constitutes the norms of policy in practice. In
turn, implementation outputs aggregate to policy outputs, or the resulting impacts of policy on a set of behaviors
related to socio-economic or environmental conditions. Policy outputs aggregate to policy outcomes, which are socio-
economic, or environmental conditions that have drawn the attention of policymakers. Policy outcomes, then,
constitute the problems stream in the policymaking process. Policymakers use policy outcomes to determine if
policies originally adopted during policymaking processes have achieved their goal. As policymaking tends to result
from non-incremental changes in socio-economic or environmental conditions (i.e., focusing events), policymaking
goals are to create status quos that are deemed acceptable, so issues disappear from the political agenda (Baumgartner,
et al., 2018). In other words, the ultimate goal of policy processes is to maintain status quos, so non- incremental
changes do not force creation of new policies. As such, successful implementation results in stable policy outputs
(and, by extension, policy outcomes) over time that align with socially defined goals established during policymaking
processes.

Second, implementers are the primary decision-makers in implementation processes. As a result, after policy adoption,
policy entrepreneurs shift focus from policymakers to implementers, whom use their administrative discretion to
determine how policies are applied (Lipsky, 2010; Aberbach and Christensen, 2014; Boswell and Rodrigues, 2016;
Zahariadis and Exadaktylos, 2016). While implementers are typically public administrators or their contractors, they
can potentially be anyone whose role in implementation or public service delivery has an impact on policy outputs.
Importantly, implementers have multi-faceted interests and may interpret policies differently based on their own
problematic preferences. These interests may include both their obligations as public servants and their own self-
interests (e.g., payoff maximization) (Moe, 1984; Wood and Waterman, 1991; Sowa and Selden, 2003; Meier and
O’Toole, 2006). Consequently, policy entrepreneurs strategically manipulate implementers navigating an inherently
ambiguous process (Aberbach and Christensen, 2014; Boswell and Rodrigues, 2016; Zahariadis and Exadaktylos,
2016). Additionally, quite often, organizations have capacity to shape uses of administrative discretion, through
incentives, rules, and organizational cultures (Wood and Waterman, 1991; Hill and Hupe, 2014). As such, policy
entrepreneurs may focus their efforts on individual bureaucrats or on organizations as a whole.

Third, problems streams consist of the same non-ideal socio-economic or environmental conditions that policymakers
want to address. Policymaking is the process by which government actions are decided upon, but implementation is
when those actions are carried out. At this stage, policymakers have already sorted through indicators of conditions,
focusing events, and citizen feedback in order to narrow down specific problems to be addressed (Kingdon, 1995;
Zahariadis, 2014; Baumgartner, Jones, and Mortensen, 2018; Herweg, et al., 2018). Implementers naturally focus on
what is measured or monitored, so they direct their efforts towards issues identified during policymaking (Brehm and
Gates, 1999). However, as policies tend to be ambiguous and do not always include specific performance indicators,
implementers may determine for themselves which measures are the most important (Chun and Rainey 2005a, 2005b).
Consequently, policy and politics streams as well as policy entrepreneurs influence implementer choices in how to
balance competing interests when directing their efforts.

Fourth, while policies already exist, implementers are faced with alternative uses of administrative discretion (Sowa
and Selden, 2003; Meier and O’Toole, 2006; Hill and Hupe, 2014). As such, policy streams consist of alternative
ways to interpret policies and incentives that encourage those interpretations. Due to inherent ambiguity in
policymaking, implementers regularly choose between multiple solutions when facing practical problems in applying
policy to real-world conditions (Matland, 1995; Lipsky, 2010). However, from successes and failures, implementers
learn what uses of administrative behavior satisfy competing needs in achieving a status quo (Jenkins-Smith, et al.,
2018). Additionally, organizations may use institutional mechanisms to constrain alternatives considered by
implementers, which can further fortify status quos (Wood and Waterman, 1991; Donaldson, 2001; Stazyk and
Goerdel, 2011). Importantly, policymakers use resources to incentivize or limit bureaucratic capacity to engage in

4
 This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

certain activities. As such, control over resources is a key tool used by policymakers to guide implementer behavior
on an ongoing basis (Wood and Waterman, 1991; Meier and O’Toole, 2006). In sum, policies consist of mechanisms
for directing implementer behavior, which aggregates to implementation outputs.

Fifth, politics streams here are the same as in policymaking processes, where politics promotes attention to certain
issues. Generally, publics tend to hold core beliefs and values in common, which link to how competing political
interests should be balanced in decision-making (Zahariadis, 2014; Herweg, et al., 2018). Similar to policymakers,
when implementers sense that political climate is for or against certain decisions, they can adjust their behavior
accordingly. Reed (2014) refers to this as operational localism, where implementers are responsible for “the
construction and maintenance of political coalitions that endorse policies… and legitimate administrative decisions”
(p. xiv). Importantly, this suggests that implementation decisions are responsive to their environment and not
necessarily uniform even if policy is. Additionally, implementers may need to adjust their behavior in responses to
shifts in public opinion following focusing events or when ideological or partisan compositions change over time.
Essentially, implementers must be aware and responsive to politics, or face challenges to their discretion (Meier and
O’Toole, 2006; Reed, 2014; Davis and Stazyk, 2015). As implementation requires cooperative behavior from political
coalitions, citizens, and/or NGOs, these challenges can create significant obstacles to maintaining status quos, and
draw renewed interest from policymakers (Hill and Hupe, 2014; Kettl, 2015; Jenkins-Smith, et al., 2018). As such,
implementers are motivated to maintain stasis in policy subsystems by being politically responsive.

Finally, when streams are coupled during critical moments, uses of administrative discretion and implementation
behavior change. Outside of policy windows, implementation outputs should be stable over time, as implementation
efforts remain consistent. That is, if there is a commonly accepted use of administrative discretion that all
implementers consistently apply, then implementation outputs should be stable. By extension, policy outputs resulting
from implementation outputs should then be stable too. If implementers focus on maintaining status quos, then even
when program funding changes, for example, their behavior should remain consistent and outputs, stable. On the
other hand, when streams are coupled during policy windows, implementers alter their behavior in order to challenge
status quos, which results in a change in implementation outputs (Zahariadis and Exadaktylos, 2016). While
implementers may be able to withstand challenges to the status quo from any single stream (i.e., shift in public opinion,
focusing events), when coupled, implementers have no choice but to respond. Otherwise, they risk a breakdown of
political coalitions, challenges to their legitimacy, implementation failures, or issues returning to policymaking
agendas as policy entrepreneurs shift pressure back to policymakers. Thus, while problems, politics, and policy
streams have independent effects on implementation processes, they also have conditional effects when coupled that
create divergences from status quos in implementation outputs.

Empirical Testing

While a key criticism of MSF as a theoretical framework of policymaking processes is lack of testable hypotheses and
empirical evidence, recent developments in scholarship ameliorate some of these concerns (Travis and Zahariadis,
2002; Riddle, 2009; Herweg, et al., 2018). Nevertheless, concepts such as coupling strategies and policy windows do
not lend themselves to testable hypotheses; although previous scholarship does examine these issues and is
incorporated into our theoretical model of MSF for policy implementation (Riddle, 2009; Zahariadis and Exadaktylos,
2016; Herweg, et al., 2018). On the other hand, previous scholarship tests the conditional nature and interaction of
policy, problems, and politics streams as part of the policymaking process but is underdeveloped within the MSF
implementation literature (Travis and Zahariadis, 2002; Riddle, 2009; Boswell and Rodrigues, 2016). Therefore, we
focus empirical testing on the conditional nature of problems, politics, and policy streams in affecting implementation
outputs, with policy outputs resulting from implementation outputs. As such, we follow Travis and Zahariadis (2002)’s
development of testable hypotheses and empirical models on this conditional nature within policymaking processes.

Importantly, Travis and Zahariadis (2002) make three alterations to MSF that inform our hypotheses. First, they
extend their model to include decision making so coupling relates to policy adoption, and not simply agenda setting.
Similarly, we focus on end-products of implementation processes and not superficial changes in behavior. As such,
we assume that policy outputs are a direct consequence of implementation, and if implementation behavior is
consistent, then policy outputs are consistent. Second, they assume that “identifying individual entrepreneurs is not a
prerequisite to modeling the process” (Travis and Zahariadis, 2002, p. 498). Correspondingly, we assume that since
such a large array of implementers are involved in processes that it is impossible to ascribe value to individual actions;
rather, it is aggregated actions across entire processes that creates changes in policy outputs. Third, they “incorporate

5
 This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

an element of inertia” (Travis and Zahariadis, 2002, p. 498) by assuming that public programs are largely based on
inheritance rather than choice. Likewise, we assume that implementer behavior is consistent over time and space
unless there is an impetus for change.

Based on these assumptions and other MSF implementation scholarship (Riddle, 2009; Boswell and Rodrigues, 2016),
we develop a general MSF hypothesis for implementation processes. Our null hypothesis is: there are no differences
in policy outputs across implementation processes. We assume that if the same policies with the same goals are
implemented, then any differences in policy outputs result from policy implementation processes. As such, if there is
no relationship between problems, politics, policies and implementation, then problems, politics, or policies will not
affect outputs. Our general hypothesis is: policy implementers continue status quos in implementation behavior by not
altering their administrative decisions, but during open windows, implementation behavior is adjusted conditional to
coupling of problems, policy, and politics streams (Travis and Zahariadis, 2002; Riddle, 2009). To quote Travis and
Zahariadis (2002), “[t]his conditional nature is the crux of our argument. Although we hold it possible that the problem
and political variables can have significant independent impacts, we hypothesize that their impact will be
condition[al]” (p. 503). As such, policy, problems, and politics only have a significant impact in affecting policy
outputs at particular levels of the other independent variables. Otherwise, only marginal differences in policy outputs
occur.

Data and Dependent Variable

We test this by analyzing state implementation of national environmental policy in the United States (U.S.), where
states implement the same national policies but differences in policy outputs are resultant from differences in state
implementation processes. Specifically, we focus on three national environmental policies: Clean Air Act (CAA),
Clean Water Act (CWA), and Resource Conservation and Recovery Act (RCRA). These three policies rely on a
federal-state partnership, where the U.S. Environmental Protection Agency (EPA) sets broad guidelines and provides
oversight, but states are responsible for implementation and day-to-day operations.Although incremental changes have
occurred in EPA’s approach to these policies over time, there have been no significant legislative changes since the
early 1990s. This creates both national and temporal consistency in policies but variation in implementation at the
state-level, which scholars argue contributes to differences in policy outputs (Ringquist, 1993; Bacot and Dawes,
1997; Travis, Morris, and Morris, 2004; Woods, 2006). As such, implementers should share similar behaviors, unless
state-level factors incentivize alternative behaviors.

We use a pooled cross-sectional panel dataset that includes 45 states over 10 years from 2001 to 2010, creating 450
observations. 1 Our dependent variable is change in policy outputs over a two-year period following initial observation
of predictor variables. The two-year lag allows us to determine more specifically that changes in the dependent
variable result from intrastate factors and for adequate time for predictor variables to affect implementer behavior and,
by extension, policy outputs. We measure policy outputs from the CAA, CWA, and RCRA as change in total state
on-site toxic releases per capita. We use data from the toxic releases inventory, which collects data on all toxic releases
regulated under these three policies (EPA, 2015b). Additionally, we use a per capita rate to create a comparable
measure between states. While there are some limitations to this measure, toxic releases serve as the best
approximation of U.S. environmental regulatory policy outputs.

We assume that: policymaking outputs are the CAA, CWA, and RCRA legislation; implementation outputs are
compliance, monitoring, and enforcement actions occurring at the state-level as part of these programs; and, policy
outputs are toxic releases by industries regulated under these programs. In other words, adoption of the CAA, CWA,
and RCRA lead to a set of implementation behaviors, and when aggregated together, these behaviors shape the
behavior of polluting industries when releasing toxic particles into the air, land, or water. Since the policies in place
are consistent across states and over time, any variation in the policy outputs should be a function of implementation
alone. As such, policy outputs can be measured by pollutants released by regulated industries. In effect, we assume
there are 450 different implementation processes in which implementers use the same implementation behavior, which
creates no changes in policy outputs over a two-year period unless streams are coupled and administrative behavior
changes. In other words, toxic releases per capita should be stable and consistent for all 450 implementation processes,
unless a set of circumstances causes a change in implementation outputs, which leads to changes in policy outputs. As
such, our null hypothesis is that there is no difference in toxic releases per capita between states.

1
Data for Nebraska, Nevada, New Hampshire, Rhode Island, and Utah was unavailable.

6
 This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

Streams

First, to control for policy streams, we identified state environmental expenditures as the key policy tool that fluctuates
between implementation processes and impacts implementer behavior. Since general requirements and guidelines for
the CAA, CWA, and RCRA are set at the national-level, we assume policy streams are consistent across states and
that implementers subscribed to industry standards in their implementer behavior. However, program funding still
results from state budgetary processes, which allows state-level policymakers to use budgeting to constrict or expand
resources available to implementers in order to manipulate their behavior (Newmark and Witco 2007). In fact,
scholars regularly link budgeting and resource allocations to political control of bureaucracy (Wood and Waterman,
1991; Meier and O’Toole, 2006). If policymaking alone controlled implementation processes, expenditures would
likely be a key predictor of policy outputs. However, previous scholarship on environmental spending and program
management indicates the relationship is more complex than this, and other socio- economic and political factors play
a key role (Bacot and Dawes, 1997; Sapat, 2004; Woods, 2006). We measure environmental expenditures with data
from the U.S. Census Bureau, as per capita state spending on natural resources.

Second, to control for politics streams, we use state-level public opinion estimates from Author (2016) that capture
both general public moods concerning the environment and how interests should be balanced. Although other
measures of state politics exist (e.g., legislative ideology), in most cases, these are not specific enough to
environmental policy to capture the political context of implementing environmental programs. More specifically,
environmental concerns may not be a political priority in some states, which may mitigate the impact of ideological
preferences. On the other hand, public opinion is a reflection of environmental beliefs, which captures perceptions of
both the government’s responsibility in managing environmental concerns and how they should be balanced against
other priorities (Carman, 1998; Author, 2016). Consequently, public opinion provides a measure of politics that is
specific enough to capture the context of implementing environmental policies. Using General Social Survey (GSS)
data pooled for 2000 through 2010, Author (2016) estimates state-level opinion with multi-level regression and
postestimation (MRP), which calculates positions of each demographic group within each state, and then weights
estimates based on actual portion of those demographic groups in state populations (See Author, 2016 for more detail).
Author (2016) reports logistic coefficient estimates that represent probability of state public opinion falling into the
pro-environmental category. 2

Finally, to control for the problems stream, we use existing policy outputs as a measure of environmental problems
occurring within each state. As such, we measure problems as total on-site toxic releases per capita. Related to the
dependent variable, implementation problems are defined by the conditions that policymakers and implementers seek
to alter, with efforts aimed to that end. As such, existing toxic releases captures the existing problems that
implementers are working to manage. Additionally, as environmental remediation tends to become more difficult as
pollutants approach zero, it also provides a mechanism to control for any initial variations in toxic releases attributed
to either implementation or factors outside of state regulatory control that may bias results (Breyer, 1993; Bacot and
Dawes, 1997; Sapat, 2004). Figure 3 diagrams our theoretical model of CAA, CWA, and RCRA policy
implementation. In applying our general hypothesis to this data, we hypothesize that effects of state expenditures
(policies) are conditional on values of both toxic releases (problems) and public opinion (politics), where there are
only marginal differences in policy outputs (and, therefore implementation processes) at moderate levels (i.e., 50th
percentile) of toxic releases and public opinion. However, at extreme levels (i.e., 95th percentile), both have significant
impacts on policy outputs. In other words, we expect to find that expenditures have negligible effects on two-year
changes in toxic releases when public opinion and total toxic releases are at median-levels, but substantively important
differences when public opinion and total toxic releases are at high-levels.

[Figure 3 about here]

2
Public opinion estimates are time invariant.

7
 This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

Co-Variates

Since environmental policy outputs are complex, we include four additional control variables based on previous
literature to ensure misspecification of models does not bias results. 3 First, to control for socio-economic differences
between states, we use data from the U.S. Bureau of Economic Analysis to measure per capita gross state product
(GSP) and per capita GSP generated from polluting industries monitored under the toxic release inventory program
(i.e., manufacturing, utilities, and mining industries) (Bacot and Dawes, 1997; Newmark and Witco, 2007). Second,
to control for organizational differences between state environmental agencies, we use program authority and mini-
EPA. Program authority is a dummy variable comparing states with authority over CAA, CWA, and RCRA programs
to those without authority (EPA, 2015a). Additionally, mini-EPA is a dummy variable comparing states with a solitary
agency focused on pollution control, to states with other types of agencies managing their environmental programs
(Author, 2012). Table 1 provides descriptions of variables used in analyses.

[Table 1 about here]

Analysis

To test this, we follow Travis and Zahariadis (2002) and use ordinary least squares (OLS) regression with interaction
terms. Since models using interaction terms can suffer from inflated standard errors and difficulty in interpreting
coefficients, we use a lagged dependent variable and robust standard errors. Since initial diagnostic tests indicated
autocorrelation, we used the Prais-Winsten transformation, which corrects for known biases in standard errors and
heteroscedasticity that arise in pooled cross-sectional panel data (Friedrich, 1982; Beck and Katz, 1995a, 1995).
Further diagnostics indicated these steps were sufficient in correcting issues, and no other assumptions of OLS were
violated (Chatterjee and Hadi 2006). To aid in interpretation, we graphed interaction terms to illustrate predictor
variable effects at different levels. Additionally, outlier analyses indicated five state-years fell outside normally
reported ranges for leverage and normalized residual values, so they were dropped from the analytical dataset. Model
1 includes only policy, politics, and problem variables, while model 2 also includes additional control variables. The
equation for the full model is:

𝛾𝛾 = 𝛽𝛽0 + 𝛽𝛽1 (𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒) + 𝛽𝛽2 (𝑒𝑒𝑡𝑡𝑒𝑒𝑒𝑒𝑡𝑡 𝑒𝑒𝑒𝑒𝑟𝑟𝑒𝑒𝑟𝑟𝑒𝑒𝑒𝑒𝑒𝑒) + 𝛽𝛽3 (𝑒𝑒𝑒𝑒𝑝𝑝𝑟𝑟𝑒𝑒𝑡𝑡 𝑡𝑡𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑡𝑡𝑒𝑒)

+ 𝛽𝛽𝑖𝑖1 (𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 𝑋𝑋 𝑒𝑒𝑡𝑡𝑒𝑒𝑒𝑒𝑡𝑡 𝑒𝑒𝑒𝑒𝑟𝑟𝑒𝑒𝑟𝑟𝑒𝑒𝑒𝑒𝑒𝑒) + 𝛽𝛽𝑖𝑖2 (𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 𝑋𝑋 𝑒𝑒𝑒𝑒𝑝𝑝𝑟𝑟𝑒𝑒𝑡𝑡 𝑡𝑡𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑡𝑡𝑒𝑒)
+ 𝛽𝛽𝑖𝑖3 (𝑒𝑒𝑡𝑡𝑒𝑒𝑒𝑒𝑡𝑡 𝑒𝑒𝑒𝑒𝑟𝑟𝑒𝑒𝑟𝑟𝑒𝑒𝑒𝑒𝑒𝑒 𝑋𝑋 𝑒𝑒𝑒𝑒𝑝𝑝𝑟𝑟𝑒𝑒𝑡𝑡 𝑡𝑡𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑡𝑡𝑒𝑒)
+ 𝛽𝛽𝑖𝑖4 (𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 𝑋𝑋 𝑒𝑒𝑡𝑡𝑒𝑒𝑒𝑒𝑡𝑡 𝑒𝑒𝑒𝑒𝑟𝑟𝑒𝑒𝑟𝑟𝑒𝑒𝑒𝑒𝑒𝑒 𝑋𝑋 𝑒𝑒𝑒𝑒𝑝𝑝𝑟𝑟𝑒𝑒𝑡𝑡 𝑡𝑡𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑡𝑡𝑒𝑒) + 𝛽𝛽4 (𝐺𝐺𝐺𝐺𝐺𝐺 𝑒𝑒𝑒𝑒𝑒𝑒 𝑡𝑡𝑟𝑟𝑒𝑒𝑒𝑒𝑒𝑒𝑟𝑟)
+ 𝛽𝛽5 (𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑖𝑖 𝑒𝑒𝑒𝑒𝑒𝑒 𝑡𝑡𝑟𝑟𝑒𝑒𝑒𝑒𝑒𝑒𝑟𝑟) + 𝛽𝛽6 (𝑒𝑒𝑒𝑒𝑡𝑡𝑝𝑝𝑒𝑒𝑟𝑟𝑝𝑝 𝑟𝑟𝑒𝑒𝑒𝑒ℎ𝑡𝑡𝑒𝑒𝑒𝑒𝑒𝑒𝑖𝑖)
+ 𝛽𝛽7 (𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑡𝑡𝑒𝑒𝑟𝑟𝑟𝑟 𝑡𝑡𝑟𝑟𝑒𝑒𝑟𝑟𝑡𝑡𝑒𝑒𝑒𝑒𝑖𝑖) + 𝛽𝛽8 (𝑝𝑝𝑒𝑒𝑒𝑒𝑒𝑒𝑚𝑚𝐺𝐺𝑚𝑚) + 𝜀𝜀

where, 𝛾𝛾 represents the dependent value, 𝛽𝛽0 represents the constant, 𝛽𝛽1…8 represents the coefficients for individual
variables, 𝛽𝛽𝑖𝑖1…𝑖𝑖4 represents interaction term coefficients, and 𝜀𝜀 represents the error term.

Results

Table 2 displays findings for OLS models. Coefficients for environmental expenditures, toxic releases, and public
opinion indicate negative relationships with change in toxic releases. As expected, increases in these variables
correlates to decreased toxic releases over time. More importantly, interactive term coefficients indicate a conditional
relationship exists between policies, problems, and politics. However, raw coefficients are difficult to interpret, so
figure 4 provides a graphical representation of interactions when toxic releases and public opinion are at 50th percentile
(i.e., 10.65 pounds per capita for toxic releases and .42 for public opinion estimate) and 95th percentile (i.e., 180.91
for toxic releases and .58 for public opinion) values and environmental expenditures range from 5th to 95th percentile
(i.e., $28.6 to $238.9 per capita) values. As the graph indicates, when either problems or politics are at 50th percentile
value there are only marginal effects of policies on policy outputs; however, when problems and politics are both at
extremely high levels, significant differences emerge.

3
Variance inflation factor (VIF) statistics did not indicate multicollinearity among co-variates in additive models (without interaction terms).
However, multicollinearity was detected when interaction terms were introduced, which is normal for models using interactive effects (Beck and
Katz, 1995a, 1995b).

8
 This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

[Table 2 about here]

For the solid black horizontal line through the center of the graph, toxic releases and public opinion are at the 50th
percentile value, and indicate there is no substantive change in toxic releases across state-years at these levels.
Furthermore, the two adjacent lines indicate that when values of one increase to the 95th percentile value, only marginal
differences occur compared to when both are at 50th percentile values. As such, there are still no substantive changes
in policy outputs across state-years. However, results for the fourth angled line are more complex. Here, both toxic
releases and public opinion increase to 95th percentile values, with the lines fluctuating widely depending on
environmental expenditures. When environmental expenditures approach extreme values in either direction,
significant differences in policy outputs emerge. Nevertheless, there are still only marginal differences in toxic releases
if expenditures are at moderate levels (i.e., center of the graph). As such, figure 4 indicates that if policies, problems,
or politics or any combination thereof remains at moderate levels, then there are no substantive changes in policy
outputs. On the other hand, if policies, problems, and politics all increase to extreme levels, then substantive changes
in policy outputs emerge. When comparing models 1 and 2, coefficients and statistical significant between models
were consistent, suggesting reliability of results. These findings support our hypotheses concerning the conditional
nature of policies, problems, and politics in affecting policy outputs.

[Figure 4 about here]

Finally, findings for program authority and mini-EPA indicate that states with authority over CAA, CWA, and RCRA
programs and with mini-EPA agencies have increased toxic releases, compared to states without authorities or mini-
EPA agencies, respectively. Additionally, GSP per capita is positively correlated and industry per capita negatively
correlated with changes in toxic releases, indicating economic development shares a complex relationship with
pollution emissions; however, these finding are not statistically significant, so it difficult to ascribe much meaning to
them. R2 statistics indicate models are moderately strong predictors of changes in toxic releases. Although these
additional control variables did not substantially increase explanatory power of model 2 compared to model 1, they
do indicate that administrative factors also influence implementation processes.

Discussion

Empirical testing indicates sufficient support for our hypothesis concerning the conditional effects of policy, problems,
and politics streams in affecting policy outputs. Findings suggest that when environmental expenditures (policies),
toxic releases (problems), or public opinion (politics) are at moderate-levels, there are only marginal differences in
policy outputs. However, when all three increase to extreme levels, significant differences in outputs emerge. More
specifically, if pollution levels or public opinion on the environment are relatively moderate, then spending has little
effect on changes in pollution over time. On the other hand, if pollution levels are extremely high and the public is
extremely concerned about the environment, then spending has a substantive impact on changes in pollution levels.
This indicates that neither expenditures, public opinion, nor toxic releases have particularly important independent
effects on implementation of environmental policy. However, when combined, their conditional effects explain a
notable portion of the inter-state variance that occurs. Consequently, this provides meaningful insights into
intergovernmental management of federal environmental policy in the U.S., where inter-state differences are as much
of a result of administrative processes as they are of political and environmental conditions.

We believe these findings provide evidence to our theoretical conceptualization of policy, problems, and politics
streams in policy implementation processes, and supports application of MSF to implementation. Existing research
already indicates core concepts of MSF exist in implementation, such as coupling strategies and policy windows
(Riddle, 2009; Zahariadis and Exadaktylos, 2016), but providing empirical evidence of interactions between streams
significantly expands on this scholarship. More specifically, while some scholars suggests conditional relationships
exist (Riddle, 2009; Boswell and Rodrigues, 2016), research in this area has not empirically tested interactive effects
as part of a MSF model of policy implementation. This provides an important step forward in establishing a foundation
for expanding MSF. Additionally, these findings may have implications for other research areas. For instance, findings
also indicate that expenditures only impact implementation if politics and problems are at high-levels; otherwise,
expenditures have little or no effect. As such, these findings provide additional nuance to observations from previous
scholarship on complex relationships between expenditures and environmental outcomes, which may suggest
implications for budgeting, performance management, or political control of the bureaucracy (Bacot and Dawes,
1997).

9
 This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

Nevertheless, there are some limitations to this analysis. First, as in any empirical model, we operationalize theoretical
concepts, which inevitably creates limitations. While we believe our choices in operationalizing policy, problems, and
politics streams as well as other co-variates are justified and representative of underlying concepts, other possible
measures do exist. This limitation is likely most apparent for environmental expenditures, which may not wholly
capture differences in the policy stream that may influence implementer behavior, such as statutory language,
mandatory spending requirements or limitations, or oversight from federal regulatory agencies. Furthermore, other
measures of the politics stream may better capture the general political climate in each state, such as gubernatorial
partisanship, legislative voting patterns, or citizen ideology. Second, our methodology assumes issues in the policy,
politics, and problems stream occur contemporarily, while policy outputs manifest over years. However, there may be
some temporal issues in streams not accounted for. For instance, a spending bill passed today is driven by
environmental conditions of yesterday, but affects implementation and policy outputs of tomorrow. Additionally,
while public opinion on the environment is mostly stable (Daniels, Krosnick, Tichy, and Tompson, 2013), Author
(2016)’s estimates are time invariant, which does not account for potential variations emerging within states between
2000 and 2010.

Third, we make broad assumptions about how implementation behavior relates to implementation outputs, and
implementation outputs to policy outputs. In sum, we assume that: 1) policymaking outputs trigger implementation
behavior; 2) implementation behavior aggregates to implementation outputs; 3) implementation outputs lead to policy
outputs; 4) policy outputs aggregate to policy outcomes; and 5) policy outcomes relate to the goals of policymaking.
Additionally, in making broad assumptions, we risk ecological fallacies, where aggregation obscures individual-level
behaviors or where there is a disconnection between individual-level and organizational-level behaviors (Schwartz,
1994). Consequently, our assumptions may be flawed and alternative explanations for our empirical findings may
exist. Finally, we do not empirically test all important concepts related to applying MSF to policy implementation.
However, as previous scholarship provides evidence to many of these concepts, our focus here was only on empirical
testing of the conditional nature of the policy, problems, and politics streams.

Conclusions

MSF provides many important insights for policy implementation, starting with providing structure to an organized
anarchy of collective action. With growing reliance on NGOs and emergence of networks, complexities in public
service delivery have grown exponentially in recent decades with policy implementation occurring through
cooperation of numerous interdependent organizations. As such, there are many gaps in understanding how a myriad
of actors connect while providing public services, and scholars struggle to rectify the expansive list of factors affecting
policy implementation (O’Toole, 2000; deLeon and deLeon, 2002). However, MSF can “shed new light on
implementation behavior by explaining why that behavior varies across time, policies, and units of government”
(Goggin, et al., 1990, p. 17). To this end, MSF offers a potential framework to unify these disparate variables into a
cohesive framework. For instance, MSF can be used to examine how individual or organizational actors make choices
that affect collective actions, and how political power and manipulation occur in non-hierarchical organizational
settings that include pluralistic interests.

Several scholars already supply some of these insights (Riddle, 2009; Howlett, McConnell, and Perl, 2015; Boswell
and Rodrigues, 2016; Zahariadis and Exadaktylos, 2016). However, our contribution is a more developed
understanding of conditional effects of policy, politics, and problems in influencing implementer behavior, and in
turn, policy outputs. As previous scholarship largely assumes additive effects of various factors in implementation
processes, our findings suggest future research should seriously consider theoretical models and research designs that
incorporate interactive effects. Additionally, our MSF model of policy implementation also creates a fuller articulation
of how streams work through nested, interdependent stages of policy processes. This further suggests the
interconnection between policymaking and policy implementation that previous scholars have criticized
implementation research for lacking (Nakamura, 1987; Sabatier, 1991; Hill and Hupe, 2014). Nevertheless, our
findings only provide initial evidence of these relationships and additional research is necessary. These efforts should
include different types of policies, with quantitative and qualitative inquiries to identify both broad overall trends and
specific mechanisms.

Finally, by expanding MSF to better include a nested policy implementation process that is separate from but
interdependent with policymaking, there are significant implications for examining implementation during the
governance era. More specifically, by connecting policymaking and policy implementation processes, MSF can be
used to analyze governance through multi-layered systems that include both policy decision-making and execution

10
 This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

(O’Toole, 2000; Hill and Hupe, 2014; Kettl, 2015; Schlager and Cox, 2018). Additionally, since MSF does not
necessarily limit itself to the public sector, this framework can easily incorporate NGOs as well as advocacy coalitions
and policy networks that affect both policymaking and implementation (Zahariadis, 2014; Herweg, et al., 2018;
Jenkins-Smith, et al., 2018). As such, MSF has potential to provide a stronger theoretical structure to examining these
relationships, as policy implementation becomes more complex and includes a variety of public agencies and NGOs
(Hill and Hupe, 2014). For more than two decades, MSF has been a core theory of policymaking but better
incorporating policy implementation has significant potential to expand its utility and applications.

References

Aberbach, J.D. and T. Christensen. 2014. Why Reforms so Often Disappoint. American Review of Public
Administration 44(1): 3-16.
Avery, G. 2004. Bioterrorism, Fear, and Public Health Reform: Matching Policy Solution to the Wrong Window.
Public Administration Review 64(3): 274-288.
Bacot, A.H. and R.A. Dawes. 1997. State Expenditures and Policy Outcomes in Environmental Program
Management. Policy Studies Journal 25(3): 355-370.
Baumgartner, F.R., B.D. Jones, and P.B. Mortensen. 2018. Punctuated Equilibrium Theory: Explaining Stability and
Change in Public Policymaking. In Theories of the Policy Process, 4th ed., edited by C.M. Weible and P.A.
Sabatier. Boulder, CO: Westview.
Beck, N. and J.N. Katz. 1995a. Nuisance vs. Substance: Specifying and Estimating Time-Series- Cross-Section
Models. Political Analysis 6(1): 1-36.
Beck, N. and J.N. Katz. 1995b. What to Do (and Not to Do) with Time-Series-Cross-Section Data in Comparative
Politics. American Political Science Review 89(3): 634-647.
Birkland, T.A. 2004. The World Changed Today: Agenda-Setting and Policy Change in the Wake of the September
11 Terrorist Attacks. Review of Policy Research 21(2): 179-200.
Boswell, C. and E. Rodrigues. 2016. Policies, Politics, and Organisational Problems: Multiple Streams and the
Implementation of Targets in UK Government. Policy & Politics 44(4).
Brehm, J. and S. Gates. 1999. Working, Shirking, and Sabotage. Ann Arbor, MI: University of Michigan.
Breyer, S. 1995. Breaking the Vicious Circle. Cambridge, MA: Harvard University.
Carman, C.J. 1998. Dimensions of Environmental Policy Support in the United States. Social Science Quarterly
79(4): 717-733.
Chatterjee, S. and A.S. Hadi. 2006. Regression Analysis by Example. Hoboken, NJ: John Wiley.
Chun, Y.H. and H.G. Rainey. 2005a. Goal Ambiguity in Organizational Performance in U.S. Federal Agencies.
Journal of Public Administration Research & Theory 15(4): 529-57.
Chun, Y.H. and H.G. Rainey. 2005b. Goal Ambiguity in U.S. Federal Agencies. Journal of Public Administration
Research & Theory 15(1): 1-30.
Cohen, M.D., J.G. March, and J.P. Olsen. 1972. A Garbage Can Model of Organizational Choice. Administrative
Science Quarterly 17(1): 1-25.
Daniels, D.P., J.A. Krosnick, M.P. Tichy, and T. Tompson. 2013. Public Opinion on Environmental Policy in the
United States. In Handbook of U.S. Environmental Policy edited by M. Kraft and S. Kamieniecki (pgs.
461-486). New York: Oxford University.
Davis, R.S. and E.C. Stazyk. 2015. Developing and Testing a New Goal Taxonomy: Accounting for the Complexity
of Ambiguity and Political Support. Journal of Public Administration Research & Theory 25(3): 751-775.
deLeon, P. and L. deLeon. 2002. What Ever Happened to Policy Implementation? Alternative Approach. Journal of
Public Administration Research & Theory 12(4): 467-492.
Donaldson, L. 2001. The Contingency Theory of Organizations. Thousand Oaks, CA: Sage. Feiock, R.C. 2013. The
Institutional Collective Action Framework. Policy Studies Journal 41(3): 397-425.
Feiock, R.C. and J.T. Scholtz. 2009. Self-Organizing Governance of Institutional Collective Action Dilemmas: An
Overview. In Self-Organizing Federalism edited by R.C. Feiock and J.T. Scholtz (pgs. 3-32). New York:
Cambridge University.
Friedrich, R.J. 1982. In Defense of Multiplicative Terms in Multiple Regression Equations. American Journal of
Political Science 26(3): 797-833.
Goggin, M.L., A.O’M. Bowman, J.P. Lester, and L.J. O’Toole. 1990. Implementation Theory and Practice.
Glenwood, IL: Scott Foresman/Little Brown.

11
 This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

Herweg, N., N. Zahariadis, and R. Zohlnhofer. 2018. The Multiple Stream Framework: Foundations, Refinements,
and Empirical Applications. In Theories of the Policy Process, 4th ed., edited by C.M. Weible and P.A.
Sabatier. Boulder, CO: Westview.
Hill, M. and P. Hupe. 2006. Analysing Policy Processes as Multiple Governance: Accountability in Social Policy.
Policy and Politics 34(3): 557-573.
Hill, M. and P. Hupe. 2014. Implementing Public Policy, 3rd ed.. Thousand Oaks, CA: Sage. Holzer, M. and K.
Yang. 2005. Administrative Discretion in a Turbulent Time: An Introduction. Public Administration
Quarterly 29(1/2): 128-139.
Howlett, M., A. McConnell, and A. Perl. 2015. Streams and Stages: Reconciling Kingdon and Policy Process
Theory. European Journal of Political Research 54(3): 419-434.
Hupe, P. and M. Hill. 2007. Street-level Bureaucracy and Public Accountability. Public Administration 85(2): 279-
299.
Jenkins-Smith, H.C., D. Nohrstedt, C.M. Weible, and K. Ingold. 2018. The Advocacy Coalition Framework: An
Overview of the Research Program. In Theories of the Policy Process, 4th ed., edited by C.M. Weible and
P.A. Sabatier. Boulder, CO: Westview.
Kettl, D. 2015. The Transformation of Governance. Baltimore, MD: Johns Hopkins University.
Kingdon, J. 1995. Agendas, Alternatives, and Public Policies, 2nd ed. New York: Harper Collins.
Lipsky, M. 2010. Street-Level Bureaucracy. New York: Russell Sage Foundation.
Lynn, L.E., C.J. Heinrich, and C.J. Hill. 2001. Improving Governance. Washington, DC: Georgetown University.
Matland, R.E. 1995. Synthesizing the Implementation Literature: The Ambiguity-Conflict Model of Policy
Implementation. Journal of Public Administration Research & Theory 5(2): 145-174.
Meier, K.J. and L.J. O’Toole. 2006. Political Control versus Bureaucratic Values: Reframing the Debates. Public
Administration Review 66(2): 177-192.
Mintrom, M. and P. Norman. 2009. Policy Entrepreneurship and Policy Change. Policy Studies Journal 37(4): 649-
667.
Moe, T.M. 1984. The New Economics of Organization. American Journal of Political Science 28(4): 739-777.
Nakamura, R.T. 1987. The Textbook Policy Process and Implementation Research. Review of Policy Research 7(1):
142-154.
Newmark, A.J. and C. Witco. 2007. Pollution, Politics, and Preferences for Environmental Spending in the States.
Review of Policy Research 24(4): 291-308.
O’Toole, L.J. 2000. Research on Policy Implementation: Assessment and Prospects. Journal of Public
Administration Research & Theory 10(2): 263-288.
O’Toole, L.J. 2015. Networks and Networking: The Public Administrative Agendas. Public Administration Review
75(3): 361-371.
Pandey, S.K. and H.G. Rainey. 2006. Public Managers’ Perceptions of Organizational Goal Ambiguity: Analyzing
Alternative Models. International Public Management Journal 9(2):85-112.
Pandey, S.K. and B.E. Wright. 2006. Connecting the Dots in Public Management: Political Environment,
Organizational Goal Ambiguity, and the Public Manager’s Role Ambiguity. Journal of Public
Administration Research & Theory 16(4): 511-532.
Reed, D.S. 2014. Building the Federal Schoolhouse. New York: Oxford University. Riddle, V. 2009. Policy
Implementation in an African State: An Extension of Kingdon’s Multiple-Streams Approach. Public
Administration 87(4): 938-954.
Ringquist, E.J. 1993. Environmental Protection at the State Level. New York: M.E. Sharpe. Sabatier, P.A. 1991.
Toward Better Theories of the Policy Process. PS: Political Science & Politics 24(2): 147-156.
Sabatier, P.A. 2007. Fostering the Development of Policy Theory. In Theories of the Policy Process, 2nd ed., edited
by P.A. Sabatier (pgs. 321-326). Boulder, CO: Westview.
Sapat, A. 2004. Devolution and Innovation: The Adoption of State Environmental Policy Innovations by
Administrative Agencies. Public Administration Review 64(2): 141-151.
Schlager, E. and M. Cox. 2014. The IAD Framework and the SES Framework: An Introduction and Assessment of
the Ostrom Workshop Frameworks. In Theories of the Policy Process, 4th ed., edited by C.M. Weible and
P.A. Sabatier. Boulder, CO: Westview.
Schwartz, S. 1994. The Fallacy of the Ecological Fallacy: The Potential Misuse of a Concept and the Consequences.
American Journal of Public Health 84(5): 819-824.
Sowa, J.E. and S.C. Selden. 2003. Administrative Discretion and Active Representation: An Expansion of the
Theory of Representative Bureaucracy. Public Administration Review 63(6): 700-710.

12
 This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

Stafford, S.L. 2006. State Adoption of Environmental Audit Initiatives. Contemporary Economic Policy 24(1): 172-
187.
Stazyk, E.C. and H.T. Goerdel. 2011. The Benefits of Bureaucracy: Public Managers’ Perceptions of Political
Support, Goal Ambiguity, and Organizational Effectiveness. Journal of Public Administration Research &
Theory 21(4): 645-72.
Thompson, J.D. 1967. Organizations in Action. New York, NY: McGraw-Hill.
Travis, R., J.C. Morris, and E.D. Morris. 2004. State Implementation of Federal Environmental Policy: Leveraging
in the State Clean Water Revolving Fund. Policy Studies Journal 32(3): 461-480.
Travis, R. and N. Zahariadis. 2002. A Multiple Streams Model of U.S. Foreign Aid Policy. Policy Studies Journal
30(4): 495-514.
U.S. Environmental Protection Agency. 2015a. State Review Framework for Compliance and Enforcement
Performance. Available at https://www.epa.gov/compliance/state-review- framework-compliance-and-
enforcement-performance [Retrieved August, 2015].
U.S. Environmental Protection Agency. 2015b. Toxic Releases Inventory. Available at https://www.epa.gov/toxics-
release-inventory-tri-program [Retrieved August, 2015].
Wood, B.D. and R.W. Waterman. 1991. The Dynamics of Political Control of the Bureaucracy. American Political
Science Review 85(3): 801-828.
Woods, N.D. 2006. Primacy Implementation of Environmental Policy in the US States. Publius 36(2): 259-276.
Zahariadis, N. 2003. Ambiguity and Choice in Public Policy. Washington, DC: Georgetown University.
Zahariadis, N. 2005. Essence of Political Manipulation. New York: Peter Lang.
Zahariadis, N. 2014. Ambiguity and Multiple Streams. In Theories of the Policy Process, 3rd ed., edited by P.A.
Sabatier and C.M. Weible (pgs. 25-58). Boulder, CO: Westview.
Zahariadis, N. and T. Exadaktylos. 2016. Policies that Succeed and Programs that Fail: Ambiguity, Conflict, and
Crisis in Greek Higher Education. Policy Studies Journal 44(1): 59-82.

Figures

Figure 1. Multiple Streams Framework for Policymaking Process

Adapted from Zahariadis (2014)

13
This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

Figure 2. Multiple Streams Framework for Policy Implementation Process

Figure 3. Multiple Streams Model for CAA, CWA, & RCRA Implementation

14
This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

Figure 4. Interactive Effects of Environmental Expenditures, Toxic Releases, and Public Opinion, with 95%
Confidence Intervals

15
 This is an author-produced, peer-reviewed version of this article. The final, definitive version of this document can be found online at
Governance, published by Wiley Periodicals, Inc. Copyright restrictions may apply. doi: 10.1111/gove.12382

Table 1. Variable Descriptive Statistics

Variable Description Mean Std. Min Max

Dev.
Toxic releases Two-year change in total on-site toxic releases per -3.21 23.55 -192.80 157.45
change (DV) capita, in pounds per person
Toxic releases total on-site toxic releases per capita, in pounds per 39.87 122.07 .41 1169.53
person
Expenditures $1s (real 2009) per capita in state spending on natural 92.46 85.03 18.39 750.54
resources
Public opinion Author (2016)’s state-level estimates of probability of pro- .42 .10 .23 .63
environmental response to: “are we spending too much, too
little, or about the right amount on improving and
protecting the environment?”
GSP $1000s (real 2009) of dollars per capita 45.42 8.25 16.64 69.90
Industry $1000s (real 2009) per capita generated from the 7.91 3.64 2.10 28.24
utilities, manufacturing, waste management, and
mining industries
Program Dummy variable comparing states with authority for .12 .32 0 1
authority CAA, CWA, and RCRA to states without authority (base
category)
Mini-EPA Dummy variable comparing states with mini-EPA .38 .49 0 1
agencies to states with other types of environmental
agencies (base category)

Table 2. OLS Model Results

Model 1 Model 2
Expenditures -.271 (.070)*** -.281 (.070)***
Toxic Releases -1.72 (.650)** -1.809 (.651)**
Public Opinion -54.680 (15.941)*** -60.915 (16.069)***
Expenditures X Toxic Releases .013 (.004)*** .013 (.004)***
Expenditures X Public Opinion Toxic .550 (.145)*** .570 (.142)***
Releases X Public Opinion 4.049 (1.437)** 4.241 (1.433)**
Expenditures X Toxic Releases X Public Opinion -.030 (.008)*** -.031 (.008)***

GSP .053 (.053)

Industry -.075 (.108)
Program authority 2.544 (1.103)*
Mini-EPA .990 (.578)+
Constant 27.156 27.584
2
R 0.214 0.221
BIC 3543.642 3563.798
N 445 445
Durbin-Watson (transformed) Rho 1.800 1.807
-.482 -.490

Note: +<.1, *<.05, **<.01, ***<.001. Standard errors in parenthesis.

16