See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/273214852

Collaborative, Participatory, and Empowerment Evaluation:

Building a Strong Conceptual Foundation for Stakeholder
Involvement Approaches to Evaluation (A Resp....

Article  in  American Journal of Evaluation · February 2013

DOI: 10.1177/1098214013509875

CITATIONS READS
16 92

4 authors, including:

David M Fetterman Liliana Rodríguez-Campos

University of Charleston and San Jose State University University of South Florida
106 PUBLICATIONS   2,032 CITATIONS    199 PUBLICATIONS   126 CITATIONS   

SEE PROFILE SEE PROFILE

Abraham Wandersman
University of South Carolina
12 PUBLICATIONS   165 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

21st anniversary of empowerment evaluation View project

Hewlett-Packard's Digital Villages View project

All content following this page was uploaded by David M Fetterman on 31 May 2018.

The user has requested enhancement of the downloaded file.

 American Journal of Evaluation
http://aje.sagepub.com/

Collaborative, Participatory, and Empowerment Evaluation: Building a Strong

Conceptual Foundation for Stakeholder Involvement Approaches to Evaluation (A
Response to Cousins, Whitmore, and Shulha, 2013)
David Fetterman, Liliana Rodríguez-Campos, Abraham Wandersman and Rita Goldfarb O'Sullivan
American Journal of Evaluation published online 18 December 2013
DOI: 10.1177/1098214013509875

The online version of this article can be found at:

http://aje.sagepub.com/content/early/2013/12/13/1098214013509875

Published by:

http://www.sagepublications.com

On behalf of:

American Evaluation Association

Additional services and information for American Journal of Evaluation can be found at:

Email Alerts: http://aje.sagepub.com/cgi/alerts

Subscriptions: http://aje.sagepub.com/subscriptions

Reprints: http://www.sagepub.com/journalsReprints.nav

Permissions: http://www.sagepub.com/journalsPermissions.nav

>> OnlineFirst Version of Record - Dec 18, 2013

What is This?

Downloaded from aje.sagepub.com at American Evaluation Association on January 6, 2014

Letter to the Editor
American Journal of Evaluation
XX(X) 1-5
ª The Author(s) 201X
Collaborative, Participatory, Reprints and permission:
sagepub.com/journalsPermissions.nav
and Empowerment DOI: 10.1177/1098214013509875
aje.sagepub.com
Evaluation: Building a Strong
Conceptual Foundation for
Stakeholder Involvement
Approaches to Evaluation
(A Response to Cousins,
Whitmore, and Shulha, 2013)

David Fetterman1, Liliana Rodrı́guez-Campos2,

Abraham Wandersman3, and Rita Goldfarb O’Sullivan4

Defining, compartmentalizing, and differentiating among stakeholder involvement approaches to

evaluation, such as collaborative, participatory, and empowerment evaluation, enhance conceptual
clarity. It also informs practice, helping evaluators select the most appropriate approach for the task
at hand. This view of science and practice is presented in response to the argument of Cousins, Whit-
more, and Shulha (2013) that efforts to differentiate among approaches have been ‘‘unwarranted and
ultimately unproductive’’ (p. 15).
Over the past couple of decades, members of the American Evaluation Association’s (AEA) Col-
laborative, Participatory, and Empowerment Evaluation Topical Interest Group (CPE-TIG) have
labored to build a strong theoretical and empirical foundation of stakeholder involvement
approaches in evaluation. This includes identifying the essential features of collaborative, participa-
tory, and empowerment evaluation. It also includes highlighting similarities and differences among
these three major approaches to stakeholder involvement.
Our primary disagreement with the article by Cousins et al. concerns the value and appropriateness
of (1) differentiating among the stakeholder involvement approaches; (2) misleading characterization;
(3) confounding and comingling terms, and (4) using collaborative inquiry as the umbrella term for
stakeholder involvement approaches.

1
Fetterman & Associates, San Jose, CA, USA
2
University of South Florida, Tampa, FL, USA
3
University of South Carolina, Columbia, SC, USA
4
University of North Carolina, Chapel Hill, NC, USA

Corresponding Author:
David Fetterman, Fetterman & Associates, 5032 Durban Court, San Jose, CA 95138, USA.
Email: fettermanassociates@gmail.com

Downloaded from aje.sagepub.com at American Evaluation Association on January 6, 2014

2 American Journal of Evaluation XX(X)

Differentiating Among the Stakeholder Involvement Approaches

A long list of colleagues has recommended that evaluation approaches to stakeholder involvement
be differentiated (Miller & Campbell, 2006; Patton, 1997, 2005; Scriven, 1997, 2005; Sechrest,
1997; Stufflebeam, 1994), and many have helped to define and identify similarities and differences
among these approaches (Fetterman, 2001; Fetterman, Deitz, & Gesundheit, 2010; Fetterman,
Kaftarian, & Wandersman, 1996; Fetterman & Wandersman, 2005, 2007; Rodrı́guez-Campos &
Rincones-Gómez, 2013; O’Sullivan, 2004; Shulha, 2010).
However, Cousins et al. ask, ‘‘Why is it important to have sharp distinctions among these
approaches and to whose benefit?’’ (p.14) and conclude, ‘‘we find the investment in compartmen-
talizing genres of collaborative, participatory, and empowerment evaluation unwarranted and ulti-
mately unproductive’’ (p. 15). Our view is that it is the nature of science and good practice to be
precise, define terms, and explain differences among similar approaches in order to build on knowl-
edge and improve practice. Differentiation of approaches helps evaluators select the most appropri-
ate stakeholder involvement approach in the field. Hence, we advocate for distinguishing among
approaches as follows:

 Collaborative evaluators are in charge of the evaluation, but they create an ongoing engagement
between evaluators and stakeholders, contributing to stronger evaluation designs, enhanced data
collection and analysis, and results stakeholders understand and use. Collaborative evalua-
tion covers the broadest scope of practice, ranging from an evaluator’s consultation with the
client to full-scale collaboration with specific stakeholders in every stage of the evaluation
(Rodrı́guez-Campos & O’Sullivan, 2010).
 Participatory evaluators jointly share control of the evaluation. Participatory evaluations
range from program staff members and participants participating in the evaluator’s agenda
to participation in an evaluation that is jointly designed and implemented by the evaluator and
program staff members. They encourage participants to become involved in defining the eva-
luation, developing instruments, collecting and analyzing data, and reporting and disseminat-
ing results (Shulha, 2010). Typically ‘‘control begins with the evaluator but is divested to
program community members over time and with experience’’ (Cousins, Whitmore, &
Shulha., 2013, p.14).
 Empowerment evaluators view program staff members, program participants, and com-
munity members as in control of the evaluation. However, empowerment evaluators do not
abdicate their responsibility and leave the community to conduct the evaluation solely by
itself. They serve as critical friends or coaches to help keep the process on track, rigorous,
responsive, and relevant. Empowerment evaluations are not conducted in a vacuum. They are
conducted within the conventional constraints and requirements of any organization. How-
ever, participants determine how best to meet those external requirements and goals (Fetter-
man & Wandersman, 2010).

Misleading Characterization
Cousins et al. present a hypothetical about an evaluation approach that hands off the evaluation to
the community without the assistance of an evaluator (p. 14). They use a discussion with Donna
Mertens about ‘‘turning control over to stakeholders’’ as their example. In this conversation, she con-
cluded it would be ‘‘chaos’’ and that there needed to be ‘‘a partnership rather than a relinquishing of
responsibility on the part of the evaluator’’ (p. 14). This is a misleading characterization of any major
stakeholder involvement approach. Empowerment evaluations, for example, place program staff and
community members in control of the evaluation. However, empowerment evaluators do not

Downloaded from aje.sagepub.com at American Evaluation Association on January 6, 2014

Fetterman et al. 3

abdicate their responsibility. They work with communities as evaluation coaches and critical friends.
This should be apparent from Fetterman, Kaftarian, and Wandersman’s (1996) earliest writings to
case examples in Stanford University’s School of Medicine (Fetterman et al., 2010) and Hewlett-
Packard’s US$15 Million Digital Village empowerment evaluation (Fetterman, 2013).

Confounding and Comingling Terms

The Cousins et al. article confounds and comingles terms when they use ‘‘collaborative evaluation’’
(books have been written about the approach) interchangeably with ‘‘collaborative inquiry’’ (p. 14).
Their language is further confounded by shifting terminology. For example, their claim that
participatory evaluation ‘‘remains somewhat of an enigma to us’’ (p. 14) is perplexing since those
same authors were proponents of the approach (before changing their terms of reference). In a
previous article, Cousins and Whitmore (1998, p. 7) noted, ‘‘Cousins and Earl (1992, 1995) outlined
an approach they labeled participatory evaluation, which built on the conventional stakeholder
model by advocating joint ownership and control of technical evaluation decision making, a more
penetrating role for stakeholders, and restriction of participation to stakeholders most closely con-
nected with the program.’’ In another article, Cousins and Earl (1995) explain, ‘‘We believe our form
of participatory evaluation strikes an appropriate balance between the needs of technical rigor and
responsiveness in evaluation. Our approach differs from other forms of participatory evaluation in
that it does not explicitly hold the goals of emancipating oppressed groups, ameliorating social
inequities, or redefining power relationships.’’ In addition, they have replaced the term participatory
evaluation with the label collaborative evaluation and called collaborative evaluation practical par-
ticipatory evaluation. This practice of confounding, comingling, and shifting terminology adds noise
and confusion to the conversation.
This is not a purely academic exercise. Imprecise language has implications for practice and also
for research on evaluation. The use of a common language, or at least translating terms and not con-
founding, comingling, or changing them, facilitates communication, dialogue, and understanding.
While recognizing that a foolish consistency is the hobgoblin of small minds (Emerson, 1993), there
is some merit to using terms consistently.

Using Collaborative Inquiry as the Umbrella Term

Similarly, privileging one label (or approach) over all three approaches, such as stating that colla-
borative inquiry or collaborative evaluation is the umbrella term for all approaches (e.g., p. 14)
or earlier stating that participatory evaluation was the appropriate term for all approaches (Cousins
& Earl, 1995), simply muddies the waters and neglects work to date.
A more appropriate global or umbrella term is ‘‘stakeholder involvement approaches’’ that
encompass collaborative, participatory, and empowerment approaches (among others). (This differs
from a similar approach, stakeholder-based evaluation; Bryk, 1983 and Mark & Shotland, 1985; in
which the evaluator is in charge, widespread agreement about program goals is lacking, all legiti-
mate groups are involved, and stakeholder participation is limited.).
Collaborative and empowerment evaluators have accepted the umbrella terminology of ‘‘stake-
holder involvement approaches.’’ In addition, Cousins and Earl (1995) placed participatory evalua-
tion within the context of stakeholder-based models in the past: ‘‘participatory evaluation is an
extension of the traditional stakeholder-based model.’’
Although the term collaborative inquiry may be a suitable overarching term for some stakeholder
involvement approaches, it does not capture the underlying values driving empowerment evaluation,
including capacity building, self-determination, and empowerment. Moreover, collaborative inquiry
typically implies evaluator control of the evaluation and consultation with the stakeholder.

Downloaded from aje.sagepub.com at American Evaluation Association on January 6, 2014

4 American Journal of Evaluation XX(X)

Empowerment evaluation is the opposite. The stakeholder being in control is primary and collabora-
tion is secondary. Stakeholder involvement represents a more generic and descriptive term for all
three approaches. Building on the work of others, engaging in the dialogue, or at least acknowled-
ging previous work in the discourse, has proved to provide more light than heat and greater concep-
tual clarity in the long run.
Cousins et al.’s (2013) resistance (p. 14) and ‘‘discomfort’’ (p. 13) with defining terms, compart-
mentalizing approaches, and clarifying approaches is not in the best interests of participatory eva-
luation, stakeholder involvement approaches, or evaluation in general. Our disagreements with
Cousins et al. (2013), however, should not be used to divide and weaken strong bonds and relation-
ships. They should be used to refine and improve our efforts. There is an overlap between collabora-
tive, participatory, and empowerment approaches in practice. Synergistic strength is a function of
interrelated and reinforcing characteristics and features. We invite colleagues to continue to engage
in critical reflection and dialogue in an effort to strengthen the quality of our work. It is our hope that
our dialogue will continue to produce more light than heat in the field.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publi-
cation of this article.

Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.

References
Bryk, A. S. (1983). (Ed.) Stakeholder-based evaluation. New Directions for Program Evaluation, Vol. 17. San
Francisco, CA: Jossey-Bass.
Cousins, J. B., & Earl, L. M. (1992). The case for participatory evaluation. Educational Evaluation and Policy
Analysis, 14, 397–418.
Cousins, J. B., & Earl, L. M. (1995). Participatory Evaluation: Enhancing Evaluation Use and Organizational
Learning Capacity. The Evaluation Exchange. Issue Topic: Participatory Evaluation. Harvard Family
Research Project, 1. Retreived from http://www.hfrp.org/evaluation/the-evaluation-exchange/issue-
archive/participatory-evaluation/participatory-evaluation-enhancing-evaluation-use-and-organizational-
learning-capacity
Cousins, J. B., & Whitmore, E. (1998). Framing participatory evaluation. New Directions for Evaluation, 80,
5–23.
Cousins, J. B., Whitmore, E., & Shulha, L. (2013). Arguments for a common set of principles for collaborative
inquiry in evaluation. American Journal of Evaluation, 34, 7–22.
Emerson, R. W. (1993). Self-reliance and other essays. Nashville, TN: American Renaissance.
Fetterman, D. M. (2001). Foundations of empowerment evaluation. Thousand Oaks, CA: Sage.
Fetterman, D. M. (2013). Empowerment evaluation in the digital villages: Hewlett-Packard’s $15 million race
toward social justice. Stanford, CA: Stanford University Press.
Fetterman, D. M., Deitz, J., & Gesundheit, N. (2010). Empowerment evaluation: A collaborative approach to
evaluating and transforming a medical school curriculum. Academic Medicine, 85, 813–820.
Fetterman, D. M., Kaftarian, S., & Wandersman, A. (Eds.), (1996). Empowerment evaluation: Knowledge and
tools for self- assessment and accountability. Thousand Oaks, CA: Sage.
Fetterman, D. M., & Wandersman, A. (Eds.). (2005). Empowerment evaluation principles in practice. New
York, NY: Guilford.
Fetterman, D. M., & Wandersman, A. (2007). Empowerment evaluation: Yesterday, today, and tomorrow.
American Journal of Evaluation, 28, 179–198.

Downloaded from aje.sagepub.com at American Evaluation Association on January 6, 2014

Fetterman et al. 5

Fetterman, D. M., & Wandersman, A. (2010, November). Empowerment evaluation essentials: Highlighting
the essential features of empowerment evaluation. Paper presented at the American Evaluation Association
Conference, San Antonio, Texas.
Mark, M. M., & Shotland, R. L. (1995). Stakeholder-based evaluation and value judgments: The role of per-
ceived power and legitimacy in the selection of stakeholder groups. Evaluation Review, 9, 605–626.
Miller, R., & Campbell, R. (2006). Taking stock of empowerment evaluation: An empirical review. American
Journal of Evaluation, 27, 296–319.
O’Sullivan, R. (2004). Practicing evaluation: A collaborative approach. Thousand Oaks, CA: Sage.
Patton, M. Q. (1997). Toward distinguishing empowerment evaluation and placing it in a larger context. Amer-
ican Journal of Evaluation, 18, 147–163.
Patton, M. Q. (2005). Toward distinguishing empowerment evaluation and placing it in a larger context: Take
two. American Journal of Evaluation, 26, 408–414.
Rodrı́guez-Campos, L., & O’Sullivan, R. (2010, November). Collaborative evaluation essentials: Highlighting
the essential features of collaborative evaluation. Paper presented at the American Evaluation Association
Conference, San Antonio, Texas.
Rodrı́guez-Campos, L., & Rincones-Gómez, R. (2013). Collaborative evaluations: Step-by-step (2nd ed.).
Stanford, CA: Stanford University Press.
Scriven, M. (1997). Empowerment evaluation examined. American Journal of Evaluation, 18, 165–175.
Scriven, M. (2005). Empowerment evaluation principles in practice. American Journal of Evaluation, 26,
415–417.
Sechrest, L. (1997). Review of the book Empowerment evaluation: Knowledge and tools for self-assessment
and accountability. Environment and Behavior, 29, 422–426. Retrieved from http://www.davidfetterman.
com/SechrestBookReview.htm
Shulha, L. (2010, November). Participatory evaluation essentials: Highlighting the essential features of parti-
cipatory evaluation. Paper presented at the American Evaluation Association Conference, San Antonio,
Texas.
Stufflebeam, D. L. (1994). Empowerment evaluation, objectivist evaluation, and evaluation standards: Where
the future of evaluation should not go and where it needs to go. American Journal of Evaluation, 15,
321–338.

Downloaded from aje.sagepub.com at American Evaluation Association on January 6, 2014

View publication stats