Beietle: Using Sodal Goals to Evaluate Public Partidpation

USING SOCIAL
GOALS TO
EVALUATE PUBLIC
PARTICIPATION IN
ENVIRONMENTAL
DECISIONS
75
Thomas C. Beierle
Center for Risk Management,
Resources for the Future
Washington DC
Policy Studies Review, FaWWinter 1999 16314
ABSTRACT
The need to increase public participation in environmental decision-making
is receiving renewed attention at all levels of government. However, there
are few approaches to evaluating these processes that address the question:
What are we getting @om public participation? This article proposes one
way to answer this question using apamework that evaluates the outcomes
of participatory processes using a set of social goals. These social
goals are: I ) educating the public; 2) incorporating public values,
assumptions, and preferences into decision making; 3) increasing the
substantive quality of decisions; 4) fostering trust in institutions; 5)
reducing conflict; and 6) making decisions cost-effectively. Although these
goals apply to public participation writ large, there are a limited number
of formalized mechanisms available to public agencies for involving the
public. The article matches these si x social goals to the participatory
mechanisms by which they might be achieved. It concludes with areas for
fdrther research suggested by the framework.
INTRODUCTION
ublic managers are continually faced with the challenge of
76 P making high quality decisions while remaining responsive
to the citizens those decisions affect. Meeting the challenge in the
environmental policy arena poses particular problems because
issues are often technically complex and value-laden, and multiple
interests operate in what is frequently an atmosphere of conflict
and mistrust. A legacy of gridlock has widely discredited the
decide, announce, defend approach to environmental decision-
making, in which agencies confiont the public only after determining
a course of action. At the same time, experience with public
participation fails to support the position that it is an unmitigated
good and that more of it is always better. Rather, public
participation must be balanced and integrated with other important
aspects of the environmental decision-making process, such as
scientific evaluations, the environmental conditions of the system
of interest, and the regulatory context. Federal, state, and local
governments are increasingly seeking better ways to bring these
aspects of environmental decision-making together-to fulfill their
regulatory mandates and maintain scientific rigor while
constructively engaging the public. This article presents a
Beierk Using Sodal Goals to Evaluate Public Partiapation
framework for evaluating the success of the public participation
components of environmental decision-making processes and for
comparing the performance of a variety of different participatory
mechanisms.
While public participation is a desirable component of
public policymaking in many arenas, it is particularly important
in decision-making about environmental issues. Three reasons
stand out. First is the changing nature of pressing environmental
priorities as the national focus shifts from large point sources of
pollution to more diffuse and widely distributed sources, such as
urban and agricultural run-off. Because of their complexity, many
of the new priority problems are not conducive to centralized
hierarchical decision-making (Dryzek, 1997). Rather, they often
require the knowledge, commitment, and action of multiple levels
of government, special interests, and the general public over long
periods of time.
77
Second, there is an emerging realization that lay people
and experts bring valid but very different perspectives to decision-
making about risks. Gridlock over issues of chemical and nuclear
risks have shown that experts and the lay public view risks
differently (Krimsky and Golding, 1992). Recent high-profile
studies have noted how even the most technical tools of
environmental decision-making-risk assessment and cost-benefit
analysis-require significant subjective judgements that are most
appropriately made with explicit attention to public values (NRC,
1996; PCRARM, 1997).
Third, the public has demonstrated that it can be very
effective in holding up projects with environmental impacts if
such projects exclude a legitimate public voice in the decision-
making process. Public participation may be one of the most
effective ways to find alternatives to the not in may back yard
(NIMBY) syndrome (Rabe, 1994). Often, public opposition is a
symptom of the publics legitimate mistrust of the willingness or
ability of government and industry to manage risks appropriately.
Policy Studies Review, FaWWiiter 1999 163/4
In a study of trust and nuclear power plant management, Slovic
(1993) found that active public involvement may be one of the
few ways to start resolving these issues of mistrust.
For all three of these reasons, the National Research
Council, in its 1996 report understanding Risk, argued that public
participation is critical to ensure that all relevant information is
included, that it is synthesized in a way that addresses parties
concerns, and that those who may be affected by a risk decision
are sufficiently well informed and involved to participate
meaninghlly in the decision (NRC, 1996: 30).
Recent efforts at many levels of government show some
commitment to enhancing public participation in environmental
decision-making. The Environmental Protection Agency (EPA),
Department of Energy (DOE), and Department of Defense @OD)
have initiated over 200 citizen advisory groups at contaminated
sites around the country (FFERDC, 1996); a number of states
have incorporated public involvement into comparative risk efforts
(Perhac, 1998; WCED, 1997); and public advisory groups have
become important components of EPAs environmental justice
activities, place-based decision-making efforts, and reinvention
programs (Mlay, 1996; NEJAC, 1996; NAPA, 1997). Despite the
resurgence of interest in public participation, however, no
consistent method has emerged for evaluating the success of
individual processes or the desirability of the many participatory
methods. The evaluation of public participation still resembles
one researchers 1983 description: The participation concept is
complex and value laden; there are no widely held criteria for
judging success and failure; there are no agreed-upon evaluation
methods; and there are few reliable measurement tools (Rosener,
1983: 45).
78
The framework described in this qi cl e is one response
to the need for evaluations of public participation programs. It
focuses on the evaluation of participatory mechanisms intentionally
instituted by government to involve the lay public, or their
Bderle: Using Sodal Goah to Evaluate Public Pactidpation
representatives, in administrative decision-making on
environmental issues. The framework is designed with three
objectives in mind: 1) to identify the strengths and weaknesses of
a number of different participatory mechanisms, 2) to be
objective in the sense of not taking the perspective of any one
party to a decision, and 3) to measure-to the extent feasible-
tangible outcomes. Such a framework is useful for determining
whether participatory programs are working, how they can be
improved, which mechanisms work best for particular needs, and,
ultimately, whether and when participatory programs justify the
commitment of public and private resources.
These three objectives, however, preclude the use of two
approaches common to the evaluation of public participation
efforts. The first are process-oriented evaluations, which compare
how actual participatory processes compare to a participatory
ideal: Were participants representative? Was the membership
balanced? Did participation occur early in the process? Were
there face-to-face discussions between the public and agency
representatives? Was the agency committed to the participatory
process and responsive to public input? In an evaluation of public
participation in U.S. Forest Service land management activities,
for example, Blahna and Yonts-Shepard (1 989) evaluated programs
using these five questions. Crosby, et. al. (1986) used a similar
approach to evaluate a citizens panel on agriculture and water
quality in Minnesota. While these process-oriented evaluations
can be objective-by measuring the quality of participation for
everyone involved-they do less well in meeting the two other
objectives. By design, they measure processes rather than
outcomes, and there is little empirical support in the literature for
tying one to the other. Because they tend to identify a preferred
process apriori, they are also less useful for comparing the
strengths and weaknesses of a wide range of participatory
mechanisms, each of which may utilize a different process.
79
The second common type of evaluations are those which
are interest-oriented. These measure the extent to which different
Policy Studies Review, FalUWiitw 1999 16314
parties (such as a public agency or a community group) achieved
their own specific goals. In a survey of 22 evaluations of public
participation programs, for example, Sewell (1 979) reported that
those performed by agency personnel tended to measure success
in terms of the degree of public acceptance of agency programs
and improved agency image. While public acceptance and an
improved image may be an agencys primary goals, other
participants in the process are likely to have very different-and
often conflicting-goals (English, 199 1 ; Rosener, 1983). Indeed,
the Sewell article shows that evaluations performed by citizen
groups, independent observers, and consultants contained a much
broader set of criteria, including measures such as degree of
equity and diversity of views heard. While interest-oriented
evaluations do measure outcomes and can be applied to a number
of different types of mechanisms, because they take the perspective
of one (or a subset) of the many interests to a decision, they can
not be regarded as objective.
80
In order to arrive at evaluative criteria that do meet the
three objectives, it is necessary to return to the core tasks of
program evaluation-identifying the problems public participation
efforts are meant to help remedy and developing a set of goals that
these programs are intended to achieve. I start with the premise
that the environmental regulatory system has systemic ailments to
which public participation may provide at least a partial cure. A
number of recent analyses have highlighted some of the problems:
the public lacks basic knowledge about many environmental issues;
policymakers inadequately consider public values and preferences
in making regulatory decisions; opportunities to correct mistakes
or find innovative solutions go unexplored; the public mistrusts
agencies resolve to protect health and the environment, and a
culture of conflict prevails (Davies and Mazurek, 1998; Esty and
Chertow, 1997; John, 1994; NAPA, 1997; NEETF, 1997; NRC,
1996, Ruckleshaus, 1996).
Bderle: Using Social Goals to Bvaluate Public Partidpation
Five social goals emerge from this problem assessment
and form the basis of this articles evaluative framework. The
goals are:
+
+
+
+
+ Reducing conflict.
Educating and informing the public,
Incorporating public values into decision-making,
Improving the substantive quality of decisions,
Increasing trust in institutions, and
To these five goals can be added a sixth goal of cost-effectiveness,
that is, choosing the least resource-intensive decision-making
process needed to achieve the goals of interest.
These goals can be used to measure the outcomes of
participatory processes, but they take a broader view of outcomes
than is typical. Normally, the outcome of a decision-making
process refers to its substantive decisions, conclusions, or
recommendations-such as whether an incinerator should be built,
what environmental problems should receive priority attention,
or what emergency response system should be established at an
industrial facility. But narrowly interpreting outcome to refer
only to these substantive decisions misses some of the most
important results of participatory processes-and indeed those
which justify opening up decision processes to the public in the
first place. Social goals are those goals which transcend the
immediate interests of parties involved in a decision. The benefits
of achieving them spill over from the participants themselves to
the regulatory system as a whole. How well they are achieved
often depends as much on how participants feel about the decision-
making process as by the substantive decisions made during it.
81
These social goals may apply to a number of areas of
policy. They are, however, critically important to the arena of
environmental policy. The next section highlights their role in
environmental decision-making and describes the social goals
framework in greater detail. The article goes on to describe how
Policy Studies Review, FaWWmter 1999 16:3/4
the framework applies to the limited number of formalized
participatory mechanisms available. It concludes with a summary
of the evaluative framework and identifies areas for krther
research.
82
SOCIAL GOALS
Goal 1: Inform and Educate the Public.
Public education is increasingly important to a well-
functioning environmental regulatory system. Knowledge about
environmental issues allows the public to carry out the role
envisioned in major environmental legislation of identifying
violations, applying community pressure, enforcing laws, and
contributing to permitting and rulemaking. It is also critical for
ensuring that the technical complexity of issues does not hamper
the publics ability to participate in decision-making.
Although a large cadre of citizens well-informed about
pertinent environmental issues might fulfill a Jeffersonian ideal of
public participation, such a vision is clearly not realistic. Instead,
we can differentiate between what the actively involved public
and the wider affected public might reasonably be expected to
know. Ideally, the active public would gain sufficient knowledge
to enable them to deliberate issues and formulate alternatives with
government representatives and experts. Education would extend
beyond the scientific foundations of the particular issue to the
decision-making process itself, an understanding of the tradeoffs
involved in various outcomes, and knowledge about the interests
of the various stakeholders. It would facilitate what Dryzek (1 997:
198) argues is one of the key elements of a successful approach to
environmental decision-making : the capacity to facilitate and
engage in social learning in an ecological context.
The knowledge requirements for the active public are
clearly too ambitious for more than a handful of citizens in most
cases. But the wider public ought to know enough about relevant
issues so that, if called on to decide an issue or offer an opinion,
Bderk Using Socia Goals to Evaluate Public Participation
they would have a realistic understanding of the consequences of
their choice. Evidence suggests that even this moderate
requirement is ambitious. In Ropers most recent National Report
Card on environmental attitudes and knowledge, nearly two-thirds
of Americans received a failing grade on basic questions about
the environment (NEETF, 1997). This lack of information clearly
affects how well the public controls its own contribution to
environmental damage and hampers the publics ability to apply
pressure to other polluters.
Goal 2: Incorporate Public Values, Assumptions, and
Pxefexences into Decision-making.
While the first goal focused on educating the public, this
goal and the next focus on educating public agencies. The risk
perception and communication literature contains numerous
examples of the differences between public and expert perception
of risk (Krimsky and Golding, 1992). Differences over values,
assumptions and preferences need to be discussed in a process
that fosters mutual education and, ideally, results in analyses and
decisions reflecting the public perspective. Such a perspective is
obviously not monolithic-various interests within the public
can have widely differing views about how a particular
environmental issue should be resolved (Bauer and Randolph,
1998). In order to give the widest range to discussions about
values, assumptions, and preferences, all of the affected
stakeholders should be involved in the process on a level playing
field.
One model for explicitly incorporating the public
perspective into decision-making involves the use of comparative
risk assessments, which have been conducted at many levels of
government throughout the United States over the last decade.
These efforts often bring together a multi-stakeholder committee-
representing public values, assumptions, and preferences-with
technical experts in a discussion-based format that seeks to meld
science and values in risk management decision-making (Perhac,
1998).
83
Policy Studies Review, FaWWinter 1999 16W4
Arguing that participatory processes should facilitate the
incorporation of public values into decision-making does not
presuppose what those values should be. Indeed, it is a basic
tenet of pluralist democracy that there is no objective common
good, but that a relative common good arises out of the free
deliberation and negotiation among all relevant interests, fairly
represented (Williams and Matheney, 1995). There is no
guarantee, then, that public values will be the same as, or even
support, ecological values. While this may be a hdamental
weakness of participatory decision-making in the environmental
arena, some authors have suggested reasons to be optimistic-at
least for the more deliberative forms of public participation.
Dryzek (1 997), for example, notes that democratic deliberation
tends to generate interests oriented toward community values rather
than more selfish individual values. It is these community interests,
he argues, which are most likely to include ecological concerns.
Goal 3: Increase the Substantive Quality of Decisions.
84
Not only is the public a source of values, assumptions,
and preferences, but it is a source of facts and innovative
alternatives. This goal relies less on the normative argument of
Goal 2 and more on the substantive argument that public input can
make decisions more technically rigorous and satisfling to a wider
range of interests. In some cases, the contribution of participatory
efforts to the substantive quality of decisions may be explicit.
The Department of Energy, for example, credits the community
advisory board at its Fernald, Ohio facility with a cleanup solution
that will save taxpayers $2 billion over the life of the project
(Applegate, 1998). In other cases, evidence of substantive
improvement is contextual. Bingham (1 986) suggests that reaching
an agreement among parties engaged in mediation is sufficient
evidence that the mediation process substantively improved
outcomes for all parties. If not, the parties who would be worse
off would refuse to agree to the mediated solution.
Cases in which substantive quality can be measured,
however, may be rare. For many participatory efforts, it is very
Beierle Using Social Goals to Evaluate Public Participation
difficult to calculate costs and benefits relative to some baseline,
figure out whether participants have expanded the pie, or come
up with a decision about who ought to get what. Instead, an
evaluation has to settle for evidence that the public participation
process added useful substantive knowledge or ideas that would
not have been available otherwise. These might include:
identifying relevant factual information, discovering mistakes, or
generating alternatives that satisfy a wider range of interests.
Goal 4: Foster must in Institutions.
The percentage of Americans reporting that they trust the
government has dropped by roughly half fiom the time of the
Kennedy Administration to today (PRC, 1998). Parallel declines
in party identification, voter turnout, and confidence in institutional
leadership signal what has been described as a decline of
deference to societys authoritative institutions (Laird, 1989).
The precipitous drop in trust and deference may represent a healthy
public skepticism in the wake of scandals and mismanagement by
these same authoritative institutions. However, it is also
symptomatic of what some claim to be a general decline in the
norms of civil society (putnam, 1995). As trust in the institutions
responsible for solving complex environmental problems
decreases, the ability to resolve those same problems is seriously
circumscribed.
Three characteristics of many environmental issues-the
long time horizon to realize benefits and costs, the absence of
clear feedback on the success of management efforts, and the
diffuse nature of benefits-make agency trustworthiness
particularly important (DOE, 1993). A number of analyses of
public trust suggest that it is far easier to lose than to regain.
However, one of the most effective ways to regain public trust
may be to involve and empower the public in decision making
(Slovic, 1993; Schneider, et. al., 1997). This is not to say that
rebuilding trust is an easy task for participatory decision-making.
In a recent survey of DOES Site Specific Advisory Boards
85
Policy Studies Review, FaWWintm 1999 16314
(SSABs)--made up of representatives of communities surrounding
each of DOES major cleanup sites-the agency asked participants
explicitly about the impact of the SSAB process on their trust in
the agency. The answers were not particularly reassuring-less
than half of the respondents felt that the advisory board process
had contributed to trust and confidence in DOE (DOE, 1997).
86
Goal 5: Reduce Conflict Among Stakeholders.
The environmental regulatory system in the U.S. was born
of conflict between environmental and industrial interests. It is
not surprising that conflict has persisted as the system has matured.
Yet many have recognized that substantial amounts of money and
energy have been consumed by court battles and other kinds of
conflict while environmental problems remain unresolved. Goal
5 arises from the view that public participation can be a process
for identifying shared norms and values and can build the
foundation for cooperative rather than confrontational decision-
making. Even if parties cannot resolve a particular issue, the process
ought to help participants understand the goals and perspective of
others by fostering communication and building relationships.
Ideally, relationships (and decisions, if made) would remain stable
over time, reflecting an ongoing absence of conflict or agreed-
upon mechanisms for resolving emergent differences (Susskind
and Cruikshank 1987).
Resolving conflict is often regarded as one of the major
achievements of those participatory processes which bring
stakeholders together for face-to-face deliberation-such as
mediations and negotiations. The literature is full of successful
cases, such as the Applegate Partnership in Oregon which brought
ti mber and environmental interests together over forest management
issues or the Negrito Project in New Mexico which balanced the
environmental, economic, and grazing interests wi thi n the Negrito
River watershed (Bauer and Randolph, 1998). Conflict reduction,
however, is by no means guaranteed. Ina study of regulatory
negotiations, for example, Coglianese (1 997) found that these
Beietk Using Sodal Gods to Evaluate Public Participation
collaborative efforts did not in fact result in less litigation than
non-collaborative rulemaking.
Goal 6: Cost effective decision-making.
The goal of cost-effectiveness addresses the appropriate
use and scope of public participation mechanisms. Certainly, not
every environmental decision justifies an active public
participation program. Few can support as extensive a process
as many observers would like. The goal of cost-effectiveness can
be considered the goal which constrains the achievement of the
first five goals: was the public participation mechanism the most
cost-effective way (in terms of money, time, risk, and opportunity
cost) of achieving the benefits (in terms of Goals 1 though 5 )
relative to other mechanisms which reasonably could have been
expected to achieve the same results? The goal argues that public
participation programs must earn their keep by producing results
which justirjr the added effort.
LINKING PARTICIPATORY MECHANISMS AND GOALS
87
The six social goals outlined earlier apply to public involvement
writ large. In practice, participation occurs through only a limited
number of mechanisms. Each can be anticipated to be relatively
better at achieving some of public participations goals and worse
at others. This section suggests which goals each mechanism may
be expected to achieve. The discussion covers one-way flows of
information such as surveys, focus groups, and public education;
traditional participatory mechanisms, such as public hearings,
public comments, and advisory committees; mechanisms
associated with collaborative decision-making and conflict
resolution, such as mediation and regulatory negotiations; and
innovative forms of public deliberation, such as citizen juries and
consensus conferences.
These various mechanisms differ along four important
dimensions: 1) the direction of information flows, 2) the degree
Policy Studies Review, PWi nt e r 1999 163/4
Public Nolice
Public Education
88
Figure 1
Unct i on of Infomuti on Rom
One-way
Public to Go v e mn l T- Y
OIM-Way:
Government to Public
Low
Focus Group
PuMic Comment
m g m of
Int.nction
Among
Potentially Medium
0Ppo.h
I n t mSt S
Huh I
B
Consensus Conferenu,
Citizen JurylPanel
Advisory Cornrninees
Reg~l at ~r y Negotiation
Mediation
Figure 2 Type of R.pnHntl ti on
Sen-mpresenMiwn Public Reprsaentsfive lnlemsl Gmups/Expar(s
-
No w s i o n Right-to-Know Pu Mi Education PuMic Notiar t+
Making Role
survey Focus Group Puui c-
bei si on
Making Rola
of Public
Advisory
Ms i o n al
of interaction among potentially opposing interests, 3) the type of
representation, and 4) the decision-making role of the public (see
Figures 1 and 2). A consideration of the relationships between
each of these dimensions and the social goals outlined above can
suggest which social goals different mechanisms may be expected
to achieve. The proposed relationships are based on the public
participation literature and common sense, not on any empirical
evidence. Testing the hypothesized relationships is an important
area for Mher study.
Beierle Using Social Goals to Evaluate Public Pdapat i on
Information flows can be one-way, with information
flowing either from the public to the government (e.g., surveys or
focus groups) or from government to the public (e.g. right-to-know
information or public notices). The former will be mainly useful
for providing decision-makers with public values, assumptions,
and preferences (Goal 2) and substantive information to improve
decisions (Goal 3). The latter will be mainly useful for increasing
public knowledge (Goal 1). To the extent that these mechanisms
increase transparency, they may also lead to greater trust in
institutions (Goal 4). Information flows can also be two-way,
offering varying degrees of opportunity for deliberation among
participants (e.g., advisory committees or mediations).
Mechanisms which allow for two-way flows ought to be expected
to achieve all of these first four goals.
The degree of interaction among potentially opposing
interests can range fiom none, as in the case of a survey, to high,
as in the case of a multi-party mediation. By allowing opposing
interests to identifj shared interests and develop relationships
among themselves, face-to-face interaction is a key element of
dispute resolution (Susskind and Cruikshank, 1987). Those
mechanisms which bring together more (and ideally all) potentially
opposing interests therefore ought to present more opportunities
for reducing conflict among stakeholders (Goal 5).
The type of representation ranges fiomcitizens representing
themselves at a public hearing, to representative members of an
advisory committee, to professional public interest or
environmental group representatives engaged in a regulatory
negotiation. All else equal, mechanisms in which the public
represents itself (through direct participation) ought to be better
at achieving the goals of education (Goal 1) and trust formation
(Goal 4) than those where the general public is represented by
representative members or professionals (such as lobbyists).
89
The decision-making role of the public can range fiom
none, in the case of a focus group, to a direct decisional role in
Policy Studies Review, FalllWiite 1999 16:3/4
ratifying an agreement arrived at through mediation. All else equal,
mechanisms which give the public a direct decision-making role
are likely to be better at achieving the goal of trust formation
(Goal 4) than those which do not (Slovic, 1993; Schneider, et. al.,
1997).
90
Bringing together, for the various mechanisms, these
relationships between characteristics and goals leads to some
insights about how different mechanisms might be expected to
perform on the six social goals. While the discussion which
follows suggests the ideal performance of each mechanism, how
successful each mechanism actually is in practice will depend on
the decision-making context as well as the process by which the
various mechanisms are carried out (Landre and Knuth, 1993;
Peele, et. al., 1996). The following subsections discuss in more
detail the different participatory mechanisms and their relationships
to the goals of interest.
Surveys, Focus Groups, and Public Comments.
As a group, these mechanisms provide one-way flows of
information from the public to the government. They include
statutory procedures for soliciting public input through comments
on proposed rules or environmental impact statements. They also
include non-statutory mechanisms, such as surveys and focus
groups, that help public managers incorporate information from
the public into decision making. For example, Ropers
Environmental Report Card, mentioned earlier, could be used
by EPA to guide an education campaign. Likewise, polls may be
used to help decide between policy options.
In general, little to no deliberation among different
stakeholders takes place with these mechanisms, and input is rarely
binding on decision-makers. The source of public input differs,
however, among mechanisms: while surveys collect the views of
individual citizens, focus groups use representative citizens as
a proxy for public opinion, and public comments are usually
undertaken by professionals hired to represent public opinion.
Beictle: Using S M G d to Evaluate Public Partidpation
The primary goals against which surveys, focus groups,
and public comments may be judged include the degree to which
they facilitate the incorporation of public values into decision
making (Goal 2) and foster the generation of policy alternatives
(Goal 3). An interesting research question is how the achievement
of these goals changes as the consulted public changes from many
citizens (surveys) to representative citizens (focus groups) to
citizen representatives (public comments).
Information Provision, Public Notice, and Public Education.
At the other end of the information spectrum are one-way
flows of information from the government to the public in forms
such as public education campaigns, the provision of right-to-
know information, and public notices. The accessibility of
chemical emissions and transfer data through the Toxics Release
Inventory (TRI) may be the best known example. Although these
mechanisms are relatively passive, the intent is often to inspire
more active participation. For example, Advanced Notices of
Proposed Rulemaking-which alert the public through a Federal
Register notice of upcoming opportunities to comment on rules-
are intended to encourage public input into the rulemaking process.
For some of these mechanisms, such as the dissemination of TRI
data, the media or community groups play important roles in
identifling and disseminating information to a wider public.
91
These mechanisms may be expected primarily to inform
and educate the public (Goal 1). Whether information provision
informs a large number of people or educates a small number will
depend on the mechanism and how it is used. For example, on-
line access to Superfund databases has the potential to reach a
large number of people with summary data on listed sites, while
public education campaigns may reach a targeted audience with
in-depth information. To the extent that these mechanisms make
government and the regulated community more accountable and
transparent to citizens, they may also increase trust (Goal 4). As
with education, the type and quality of government information
provided to citizens will determine its actual impact on trust. If
Policy Studies Review, PaWWinter 1999 16314
these mechanisms bring disparate groups together, as may be the
case with public education workshops, there is at least some
opportunity for establishing the kind or relationships or joint
learning that can reduce conflict (Goal 5).
92
Public Hearings.
Public hearings remain the most common form of face-to-
face public involvement in spite of nearly universal criticism of
their ability to provide meaningful participation. EPA convenes
hundreds of hearings per year (Fiorino, 1990). Most are used to
defend agency decisions rather than to involve the public in the
decision-making process itself. Agencies often hold hearings late
i n the process, present technical information beyond the
understanding of the lay public, and seek to do little more than
fulfill administrative requirements (Fiorino, 1990).
The two-way flow of information would suggest that public
hearings should be able to achieve the first four goals: increasing
public knowledge (Goal l), providing decision-makers wi th public
values, assumptions, and preferences (Goal 2), providing
substantive information to improve decisions (Goal 3), and, to the
extent that hearings increase transparency, increase trust in
institutions (Goal 4). Because they often bring together potentially
opposing interests, public hearings also may be expected to reduce
conflict (Goal 5). However, the lack of real deliberation among
stakeholders and between stakeholders and government might lead
one to predict a priori that most public hearings will do a poor
job of achieving most of these goals.
Citizen Advisory Committees.
Citizen advisory committees (CACs) encompass a wide
variety of organizations which represent a relatively small group
of citizens who are called together to represent ideas and attitudes
of various groups and/or communities (Rosener, 1978: 188). They
should be distinguished from expert advisory committees, which
Bderk Using Social Goah to Evaluate Public Participation
agencies use extensively to bring outside scientific, economic,
and other technical information into government decision-making
processes (Jasanoff, 1990).
The forms and functions of CACs vary widely. Federally
endorsed committees established under the Federal Advisory
Committee Act (FACA) follow strict requirements regarding
balanced membership, transparency, and government involvement.
The Federal Facilities Environmental Restoration Dialogue
Committee (FFERDC), for example, was established under FACA
in 1992 to seek consensus among a variety of stakeholders on
issues related to the cleanup of contaminated federal facilities.
The committee produced an influential report in 1996, which,
among other things, recommended the formation of community
advisory committees at all federal facilities undergoing cleanup
(FFERDC, 1996).
93
CACs may also be quite informal, including groups which
were established without government involvement but that have
come to represent public views in policy making. For example,
the Anaconda-Deer Lodge Advisory Committee was formed by
local leaders and residents to represent the communitys interests
in the Anaconda, Montana Superfund site clean-up process
(WMREI, 1991).
Advisory committee members are intended to serve as the
voice of the larger public, although in practice this has been
interpreted to include elected officials and other elites as well as
typical members of the ~ommunity.~ Even in the latter case, a
number of studies have shown that participants are often not
representative of the wider community in terms of income and
education (Lynn and Kartez, 1995). CACs often present members
with the opportunity to engage in discussions with a number of
other interests, either internally in committees with balanced
representation or externally with other organized interest groups.
They typically play only an advisory role, but ideally their input
is explicitly incorporated into the decision-making process. Where
Policy Studies Review, FaWWimter 1999 16314
committees are balanced, the CAC can act like a voluntary
negotiating body where each participant represents broad
constituent interests (Lynn and Kartez, 1995).
94
The deliberative and representative nature of advisory
committees suggests that they ought to achieve the first four goals:
increasing public knowledge (Goal l), providing decision-makers
with public values, assumptions, and preferences (Goal 2),
providing substantive information to improve decisions (Goal 3),
and, increasing trust in institutions (Goal 4). To the extent that the
committees are balanced they ought to provide opportunities
for conflict reduction (Goal 5) between the stakeholders
represented.
A1 terna tive Dispute Resolution Mechanisms.
Collaborative methods of dispute resolution and problem
solving have arisen in the environmental policy arena largely as
an alternative to long and expensive legal battles (Bauer and
Randolph, 1998; Dryzek, 1997). The primary alternative dispute
resolution mechanisms in environmental decision making are
regulatory negotiations and stakeholder mediations. Regulatory
negotiations provide a formal process for stakeholders to negotiate
the content of federal regulations. They were initiated by EPA
eighteen times between 1983 and 1986, and, in twelve cases during
this period, resulted in at least one final rule on the topic under
negotiation (Coglianese, 1997). Stakeholder mediation describes
a far more diverse, and often non-governmental, set of approaches
for bringing together opposing interests to settle divisive issues.
Some of the most successful mediations have been over resource
issues in the western United States. For example, a grass roots
effort to seek consensus on water management issues in Montanas
Clark Fork River Basin brought miners, ranchers, municipal
officials, and environmentalists together after decades of
acrimonious conflict to successfidly resolve disputes over water
use (NAPA, 1997).
B d a k Using Sodal Goals to Evaluate Public Pactidpation
Regulatory negotiations and stakeholder mediations offer
substantial opportunity for two-way communication among a
variety of opposing interests. Their explicit purpose is to reduce
conflict and reach consensus, often in cases where other forms of
agreement or dispute settlement have failed. If parties reach a
decision, they are typically bound by it, at least informally. This
accountability to outcomes may be a critical aspect of successful
negotiations and mediations (Bingham, 1986). Participants are
often professional representatives rather than members of the lay
public. In fact, one of the principal criticisms of regulatory
negotiations is that they only involve the usual suspects of
lobbyists, NGOs, and government officials (Applegate, 1 998).
The deliberative nature of alternative dispute resolution
mechanisms would suggest that they would be likely to achieve
the first four goals. However, to the extent that participants are
the usual suspects, this limits opportunities for public education
outside of a small group of professionals. In spite of this trait, the
mechanisms are still likely to be excellent fora for interest groups
to provide decision-makers with public values, assumptions, and
preferences (Goal 2) and substantive information to improve
decisions (Goal 3). The binding nature of many agreements would
suggest opportunities for trust formation (Goal 4). However, if
these agreements are mostly among lobbyists and other
professionals, trust formation is unlikely to filter out to the public
at large. The explicit attention to consensus building and conflict
resolution among a wide range of stakeholders suggests that
negotiations and mediations provide ample opportunities to reduce
conflict among stakeholders (Goal 5) . As previously noted,
however, research on regulatory negotiations by Coglianese (1 997)
suggest that these mechanisms promise of conflict reduction may
be more encouraging than their actual performance.
95
Citizen Deliberations.
Mechanisms for citizen deliberation include citizen juries
(or the related citizen panels) and consensus conferences. Many
Policy Studies Review, PatyWinter 1999 16314
of the examples of these mechanisms in the U.S. have been non-
governmental experiments in participatory policy analysis on
complex issues such as education policy, energy planning, and
public spending priorities. Some states have used these
mechanisms to inform decisions about risk prioritization, water
quality planning, and sludge disposal (Jefferson Center, 1997;
Crosby et. al., 1986; Renn et. al., 1991). Although the format
varies across different mechanisms, their purpose is to help non-
expert citizens, acting as value consultants, analyze technically
complex subjects. Organizers provide a group of selected citizens
with access to expert information and sufficient time to engage in
deliberative analysis with experts and among themselves. They
are expected to combine the technical facts with public values
into a set of conclusions and recommendations.
96
These mechanisms are explicitly designed to allow two-
way communication between experts and the public, and sometimes
government. However, experts and the government are mainly
information resources, and most of the actual deliberation takes
place among the citizen members of the group. Participants are
not interest group representatives. Rather, they are regarded as
representative of the public. In some citizen juries, they may
even be selected through random sampling (Fiorino, 1990). All
of these factors would suggest that deliberative fora ought to be
particularly good at educating participants (Goal 1 ), providing
decision-makers with public values, assumptions, and preferences
(Goal 2), and generating substantive information to improve
decisions (Goal 3). In the past, many of these mechanisms have
had public or media outreach programs which extends educational
opportunities beyond those who actually participate.
Citizen deliberations offer only limited opportunities for
interaction between potentially opposing interests (other than the
extent to which participants identify themselves with various
interests in their daily lives). Opportunities to reduce conflict
(Goal 5 ) are therefore limited. Trust formation (Goal 4) is unlikely
as the results of the efforts are purely advisory, and many have
Bderle: Using Social Goals to Evaluate Public Partiapation
had no formal tie to government decision making processes. In
fact, in his evaluation of a citizens panel on Telecommunications
and the Future of Democracy held in 1997 near Boston, Guston
(1998) found no actual impact on the substance, agenda,
vocabularies or problem framing of telecommunications policy
as a result of the effort.
CONCLUSIONS
This article has presented a framework for evaluating the success
of public participation efforts as measured by social goals. The
framework meets the objectives set out in the Introduction. It is
objective in the sense that it does not explicitly take the
perspective of any one party to a particular decision, and it seeks
to measure tangible outcomes fiom participation (although it takes
a wider view of outcomes than normal). By linking social goals
to the characteristics of a number of participatory mechanisms, it
also suggests some hypotheses about what goals different
participatory mechanisms may be expected to achieve (see Table
1 for a summary). These suggested relationships, especially if
Table 1: Goals and Mechanisms
0 =not likely to schicvc to god
B - may bc sxpected to achieve goal
0 - ought to achieve goal
97
Policy Studies Review, FaWWintm 1999 16:3/4
confirmed by further research, should be useful for the practitioner
and evaluator in knowing which mechanisms ought to be useful
for different policy objectives and how mechanisms might be
combined to achieve a range of goals.
The evaluative framework also highlights some areas for
further research. An impact model which describes how an
intervention (the participatory process) affects an outcome (the
social goals) does not exist in the literature. Further research on
how various procedural factors affect the outcomes of interest
will be important for designing and evaluating participatory
programs in the future. The second research need was implied
earlier. Many of the relationships between the characteristics of
various mechanisms and the goals which they might accomplish
are merely hypothesized. Further research could address whether,
for example, bringing more stakeholders to a decision actually
does lead to more opportunities for conflict resolution or whether
giving the public a decision-making (rather than advisory) role
affects trust formation. Finally, further research should address
how the analysis of mechanisms and goals changes when dealing
with different environmental issues and other contextual issues.
How does participation in a controversial facility siting decision
differ from that of a relatively non-controversial comparative risk
assessment? What goals are important? Which mechanisms are
most effective?
98
A starting point for addressing these research needs would
be the application of the evaluative framework described here to
multiple case studies where different participatory mechanisms
were used to address a variety of environmental issues. Such
research may allow us to get beyond seeking ways to simply
increase public involvement, and help us tackle unanswered
questions of when, how, and why the public should be involved.
Beiede Using Sodal Goals to Evaluate Public Partidpation
ENDNOTES
1. The defmition explicitly excludes important conventional and regulated
methods of participation such as voting and lobbying as well as unconventional
and extralegal methods such as striking, picketing, and violence. It also
excludes public input through referenda, initiatives, and citizen suits although
the analysis could be extended to include these mechanisms.
2. These and many other criteria for what constitute good processes have
been derived from theory (Webler, 1995; Fiorino, 1990: 229-230) or rules
of thumb which practitioners and researchers have found to be consistently
successful over time (Ashford, 1984:79; Crosby, 1986: 171; Blahna and
Yonts-Shepard, 1989:211-2 13; Peelle, 1996). Additional criteria on which
a rough consensus has emerged include: clarity of goals and roles of
participants; sufficient resources, including financial support, time, and
information; recognition of the legitimacy of public input equal to that of
officials and technical experts; procedural independence of public to make
decisions, set the agenda, and acquire technical information; and, the presence
of a strong chairperson or facilitator.
3. Organizers of CACs, such as agency officials, often have considerable
power in picking committee members (Lynn and Kartez 1995). For the variety
this discretion fosters, see Perhac (1998) on public involvement in
comparative risk assessments.
99
REFERENCES
Applegate, J ohn S. 1998. Beyond the Usual Suspects: The Use of Citizens
Advisory Boards in Administrative Decisionmaking, Indiana Law Journal
73:903
Ashford, Nicholas A. 1984. Advisory Committees in OSHA and EPA: Their
Use in Regulatory Decisionmaking, Science, Technology, and Human
Values 9(1) Winter: 72-82.
Bauer, Michael R. and J ohn Randolph. 1998. Improving Environmental
Decision-making through Collaborative Methods, Paper prepared for the
National Conference on Environmental Decision Making, Knoxville,
Tennessee (May 4-6). .
Bingham, Gail. 1986. Resolving Environmental Disputes: A Decade of
Experience, Washington, DC: the Conservation Foundation.
Policy Studies Review, F M i t e r 1999 16314
Blahna, Dale J . and Susan Yonts-Shepard. 1989. Public Involvement in
Resource Planning: Toward Bridging the Gap Between Policy and
Implementation, Society and Natural Resources 2(3): 209-227.
100
Coglianese, Cary. 1997. Assessing Consensus: The Promise and
Performance of Negotiated Rulemaking, Duke Law Journal 46(6).
Crosby, Ned; J anet M. Kelly; Paul Schaefer. 1986. Citizen Panels: A New
Approach to Citizen Participation, Public Management Forum March/
April 1986:170-178.
Davies, J . Clarence and Jan Mazurek. 1998. Pollution Control in the United
States: Evaluating the System, Washington, DC: Resources for the Future.
Department of Energy (DOE). 1997. Site-Specijlc Advisory Board Initiative,
1997 Evaluation Survey Results: Volume I, Summary Report, Washington,
DC: U.S. Department of Energy (September).
Department of Energy (DOE). 1993. Earning Public Trust and Confidence:
Requisites for Managing Radioactive Wastes, Final Report of the Secretary
of Energy Advisory Board Task Force on Radioactive Waste Management,
Washington, DC: Department of Energy.
Dryzek, J ohn S. 1997. The Politics of the Earth: Environmental
Discourses, Oxford, England: Oxford University Press.
English, Mary R. 1991. Stakeholder Views of Superfund Sites, The
Superfund Process: Site-Level Experience Waste Management Research
and Education Institute: Knoxville, TN.
Esty, Daniel C. and Marian R. Chertow. 1997. Thinking Ecologically: An
Introduction, in Marian R. Chertow and Daniel C. Esty eds., Thinking
Ecologically: The Next Generation of Environmental Policy, New Haven,
CT: Yale University Press.
Federal Facilities Environmental Restoration Dialogue Committee
(FFERDC). 1996. Consensus Principles and Recommendations f or
Improving Federal Facilities Cleanup, Washington, DC: Environmental
Protection Agency (April).
Fiorino, Daniel J . 1990. Citizen Participation and Environmental Risk: A
Survey of Institutional Mechanisms, Science, Technologv and Human
Values 15(2): 226-243.
Bei el e Using Social Goals to Evaluate Public Partiapation
Guston, David H. 1998. Evaluating the Impact of the First U.S. Citizens Panel
on Telecommunications and the Future of Democracy, Prepared for delivery
at the 1998 Meeting of the American Political Science Association, Boston,
Massachusetts (September 3-6).
J asanoff, Sheila. 1990. The Fifth Branch: Science Advisors as
Policymakers, Cambridge, MA: Harvard University Press.
J efferson Center. 1997. Citizen J ury on Comparing Environmental Risks,
http://www.usinternet.com/users/jcenter/jcsumart.html
J ohn, Dewitt. 1994. Civic Environmentalism: Alternatives to Regulation
in States and Communities, Washington, DC: Congressional Quarterly
Press.
Krimsky, Sheldon and Dominic Golding, eds. 1992. Social Theories of
Risk, Westport, CT: Praeger.
Laird, Frank N. 1989. The Decline of Deference: The Political Context of
Risk Communication, Risk Analysis, 9(4):543-550.
Landre, Betsy Kiernan and Barbara Knuth. 1993. The Role of Agency Goals
and Local Context in Great Lakes Water Resources Public Involvement
Programs, Environmental Management 17(2): 153-165.
101
Lynn, Frances and J ack D. Kartez. 1995. The Redemption of Citizen
Advisory Committees, in Fairness and Competence in Citizen
Participation: Evaluating Models for Environmental Discourse, Ortwin
Renn; Thomas Webler, and Peter Wiedemann, eds., Kluwer Academic
Publishers: Dordrecht, Germany.
Mlay, Marian. 1996. Principles for Sponsors of Community-Based
Environmental Protecti ~n,~~ presented at the Annual Conference of the
American Society of Public Administration, Atlanta, Georgia (J uly 1).
National Academy of Public Administration (NAPA). 1997. Resolving the
Paradox of Environmental Protection, Washington, DC: National Academy
of Public Administration.
National Environmental Education and Training Foundation (NEETF). 1997.
The National Report Card on Environmental Knowledge, Attitudes, and
Behaviors, Washington, DC: NEETF.
Policy Studies Review, Fall/Wiiter 1999 16:3/4
102
National Environmental Justice Advisory Council (NEJAC). 1996. The Model
Plan for Public Participation, Washington, DC: Environmental Protection
Agency.
National Research Council (NRC). 1996. Understanding Risk: Informing
Decisions in a Democratic Society, Washington, DC: National Academy
Press.
Peelle, Elizabeth; Martin Schweitzer, J ohn Munro, Sam Cames, Amy Wolfe.
1996. Factors Favorable to Public Participation Success, Proceedings:
National Association of Environmental Professionals, Houston, TX, J une 2-
5, 1996, Oak Ridge, TN: Oak Ridge National Laboratory.
Perhac, Ralph M. 1997. Comparative Risk Assessment: Where Does the
Public Fit In?, Science, Technology, and Human Values 23(2):22 1-24 1 .
Pew Research Center (PRC). 1998. Deconstructing Trust: How Americans
View Government, http://www.people-press.org/trustrpt.html.
Presidential/Congressional Commission on Risk Assessment and Risk
Management (PCRARM). 1997. Framework for Environmental Health
Risk Management Final Report, Volume 1, Washington, DC: PCRARM.
Putnam, Robert D. 1995. Bowling Alone: Americas Declining Social
Capital, Journal of Democracy 6:l Jan 1995: 65-78.
Rabe, Barry. 1994. Beyond NIMBY, Washington, DC: The Brookings
Institution.
Renn, Ortwin; Thomas Webler; and Branden B. J ohnson. 199 1. Public
Participation in Hazard Management: The Use of Citizen Panels in the U.S.,
Risk-Issues in Health and Safety 197 (summer).
Rosener, J udith. 1978. Matching Method to Purpose: The Challenge of
Planning Citizen Participation Activities, in Stuart Langton (ed.), Citizen
Participation in America, Lexington Books: Lexington, MA
Rosener, J udy B. 1983. User-Oriented Evaluation: A New Way to View
Citizen Participation, in Gregory A. Daneke, Margot W. Garcia, and J erome
Delli Priscoli, eds., Public Involvement and Social Impact Assessment,
Westview Press: Boulder, CO. pp. 45-60.
Ruckleshaus, William D. 1996. Trust in Government: A Prescription for
Restoration, Webb Lecture, National Academy of Public Administration,
Washington, DC, November 15.
Bderle: Using Social Goals to Evaluate Public Participation
Schneider, Mark, Paul Teske and Melissa Marschall. 1997. Institutional
Arrangements and the Creation of Social Capital: The Effects of Public School
Choice, American Political Science Review 9 1 (1): 82-93.
Sewell, W.R Derrick and Susan D. Phillips. 1979. Models for the Evaluation
of Public Participation Programs, Natural Resources Journal 19:337-
58.
Slovic, Paul. 1993. Perceived Risk, Trust, and Democrilc! .* Ki sh . J)i t i / \ xi s
13(6): 675-682,
Susskind, Lawrence and J effrey Cruikshiiih 1987. Brwking the !tnpusse:
Consensual Approaches to Resolving Pirl~lrt I )r vpu~c~s. New York: Basic
Books
Waste Management Research and Education Institute WMIUZI 1. I )9 I . Tlw
Superfund Process: Site-Level Experience, Hazardous W;rs~c Ihilediation
Project. University of Tennessee: Knoxville, TN.
Webler, Thomas. 1995. Right Discourse in Citi/.cn Ianicipetion: An
Evaluative Yardstick, in Ortwin Renn; Thomas Webler. and Ielcr Wicdcinann. 103
eds., Fairness and Competence in Citizen PartiL.ipurio,i: 1;vciliiiiting
Models for Environmental Discourse, Kluwer Acadr m ic Iublishrrs:
Dordrecht, Germany.
Western Center for Environmental Decision-Making. ( WCl : b 1. 1 Y07.
Public Involvement in Comparative Risk Projects: I ri )i ci pl ~-.v . i d Bcw
Practices. Boulder, CO: Western Center for Environiiiental Lkcision-
Making.
Williams, Bruce A. and Albert R. Matheney. 1995. / k n i t i n i c ; i * , l ) i hgi t ~, .
and Environmental Disputes: The Contested 1.uJigiiugc.x 01 Socicil
Regulation, Yale University Press: New Haven, CT.
ABOUT THE AUTHOR
Thomas C. Beierle is a Research Associate with r h c - ( t vi i t -r f or Risk
Management at Resources for the Future in Wachirigrrni. l X*. O~ I C r! f hi s
primary research areas is the role of public involwnicvil in l i ~i ~~i r~i t i i ?i ~~~i i ~l
decision-making. His recent work includes an exminutioti of rhc Federul
Advisory Committee Act and a meta-analJ*sis 01 cusc sttidies 011
participatory decision-making using the l r t i ni ~wor k discussed in t hi s
article.