Book reviews 123

Cook, T. D., & Campbell, D. T. (1979). Q uasi-experimentation: Design
and analysis issues forfield settings. Chicago: Rand McNally.
Dennis, M. J., & Boruch, R. F. (1994). Improving the quality of ran-
domized field experiments: Tricks of the trade. In K. J. Conrad
(Ed.), Critically evaluating the role of experiments. New Direcrions
for Program Evaluation no. 63 (pp. 87-101). San Francisco: Jossey-
Bass.
Fetterman, D. M. (1982). Ibsen’s baths: Reactivity and insensitivity.
Educational Evaluation and Policy Analysis, 4, 261- 279.
Fetterman, D. M. (1994). Keeping research on track. In K. J. Conrad
(Ed.), Critically evaluating the role of experiments. New Directions
for Program Evaluation no. 63 (pp. 1033105). San Francisco: Jossey-
Bass.

Utilization-Focused Evaluation: The New Century Text (3rd ter 3, Patton revisits the role of the personal factor that he
Edition) by M. Q. Patton. Thousand Oaks, CA: Sage 1996. identified in the book’s first edition. He continues to believe
Reviewer: Hallie Preskill, University of New Mexico. that, “the presence of an identifiable individual or group of
people who personally care about the evaluation and the fin-
Keywords: utilization-focused evaluation; evaluation use; evalu- dings it generates . .” (p. 44) is the key ingredient to evaluation
ation texts. use. Furthermore, he writes that the more diverse the stake-
holder group, the greater likelihood use of findings will occur.
In 1978, with the publication of the first edition of Utilization- In Chapter 4, Patton elaborates on the notion of intended
Focused Evaluation (UFE), Michael Q. Patton challenged evalu- use by intended users and identifies three uses of findings. He
ation practice by asking evaluators to consider evaluation use proposes that “evaluation findings can serve three primary pur-
as part of evaluation theory. Patton’s innovative writing style poses: rendering judgements, facilitating improvements and/or
incorporating the use of Sufi parables, children’s stories, every- generating knowledge” (p. 65). While all of these purposes may
day language and examples from his research generated come into play in any one evaluation, he suggests that “one is
increased interest in a new, user-focused evaluation approach. likely to become the dominant motif and prevail as the primary
By the second edition’s publication in 1986, the first edition had purpose of informing design decisions and priority uses .” (p.
sold nearly 15,000 copies. The second edition’s content while 65). Each of these purposes is described in detail with concrete
very similar to the first, provided readers with additional exam- examples.
ples of UFE evaluations, new stories, cartoons and references The final chapter in Part 1 represents an important addition
(Sonnichsen, 1987). In the ten years between 1986 and 1996, to the third edition and to the evaluation use literature in
nearly 13,000 copies were sold. In November 1996, the third general. Here Patton introduces the concept of process use:
edition was published and in its first nine months, Utilization-
Focused Evaluation: The New Century Text sold 4000 copies. Process use refers to and is indicated by individual chan-
In the preface to this new edition, Patton describes the evol- ges in thinking and behavior and program or organ-
ution and growth of UFE and his three books: izational changes in procedures and culture, that occur
among those involved in evaluation as a result of the
The first edition, published in 1978 and based on research learning that occurs during the evaluation process. Evi-
done in 1975, had the tone of a toddler throwing a temper dence of process use is represented by the following kind
tantrum because no one seemed to be paying attention. of statement after an evaluation: “The impact of our
The second edition, published in 1986, was alternatively program came not just from the findings but from going
brash and shy, assertive and uncertain, like an adolescent through the thinking process that the evaluation requi-
coming of age . I wanted the second edition to set the red” (p. 90).
record straight and clarify points of confusion . . . Now
that utilization-focused evaluation has survived to voting By emphasizing the concept of process use as a type of use,
age . I feel liberated to be more celebratory and less Patton is asking evaluators and others to broaden their defi-
argumentative in tone. (p. xiv) nition of evaluation use, expand their understanding and
appreciation of conceptual use and acknowledge the learning
This third edition is a celebration of 25 years of research and capabilities of evaluation practice. While maintaining that
practice on evaluation utilization that links the past to the “using evaluation to enhance shared understandings is a rela-
present with hopes of a user-focused evaluation future. Patton tively traditional use of evaluation logic”, he also suggests that
has remained consistent in his belief that utihzation-focused evaluation may be conducted using an intervention-oriented
evaluation is an orientation to evaluation that emphasizes mat- approach which is more controversial. This is where evaluation
ching intended use to intended users. becomes part of the design, delivery and implementation of
Part 1 of the book’s third edition is titled, “Toward More an intervention (i.e., program). Collaborative, participatory,
Useful Evaluation” and includes five chapters that set the stage developmental and empowerment types of evaluation are dis-
for understanding utilization-focused evaluation. The first two cussed as approaches that facilitate intervention-oriented evalu-
chapters define evaluation, provide a brief overview of evalu- ations
ation’s history and place evaluation within the context of Part 2 is titled, “Focusing Evaluations: Choices, Options
intended use by intended users. Patton maintains his long- and Decisions” and also contains five chapters. This section
standing view that evaluation theory can not be understood addresses evaluator roles, situational factors that effect evalu-
without being grounded in a theory of evaluation use. In Chap- ation and use, challenges faced by internal and external eval-
uators. the development of program goals and outcomes, understanding of evaluation in multiple organizational
alternatives to goals-based evaluation, focusing the evaluation. contexts.
evaluation implementation issues and the role of program
theory. Each of these topics is discussed in the context of
D~fftirem~es between the third edition und precious editions
enhancing evaluation use.
“Appropriate Methods” is the title of Part 3 which consists
In each successive edition, I/FE has grown; the second edition
of three chapters. This section focuses on the choices evaluators
was 20% longer than the first edition (367 pages compared to
have in making data collection method decisions. Again, Patton
303) and the third edition is not only 430 pages (a 15% growth),
urges evaluators to choose methods relative to enhancing evalu-
but it is larger in size (from 5.5” x 8.5” to 7” x IO”)! In addition to
ation use. The first chapter is called “Evaluations Worth Using”
providing new content, the third edition includes many stylistic
and approaches the concepts of reliability and validity from
differences as well. Because of the larger size, the text is pre-
a use perspective which emphasizes methodological quality,
sented in two columns, which I found much easier to read. The
credibility, believability. understandability and stakeholder
inclusion of more graphics, tables, figures and bold typeface all
involvement in determining appropriate methods. While many
contribute to make the work extremely accessible to different
people believe Patton’s preference is for qualitative forms of
kinds of learners.
data, he suggests this is not necessarily the case and attributes
Interwoven throughout this edition are the revised Evalu-
this perception to his previous books on using qualitative
ation Standards (Joint Committee on Standards for Program
methods in evaluation. He specifically states that “the primary
Evaluation, 1994) and the American Evaluation Association’s
focus in making evaluation methods decisions should be on
Guiding Principles (Shadish et al., 1995). However, the most
getting the best possible data to adequately answer primary
important and substantive addition to this edition is Patton’s
user’s evaluation questions given available resources and time”
discussion of process use and the use of participatory evaluation
(p, 247). A few pages later he adds, “Relevance and utility are
approaches as a means for organizational growth and change.
the driving forces in utilization-focused evaluation; methods
Related to this expanded view of evaluation practice, Patton
are employed in the service of relevance and use, not as their
addresses alternative roles for evaluators, ethical issues associ-
master” (p. 250). Here Patton continues to emphasize the idea
ated with participative and process oriented evaluations and
that use drives decisions about methods and not the other way
methods to involve a diversity of stakeholders who represent
around. He makes this argument in part, based on the fallibility
different sets of values and interests.
of all methods and designs. In the chapter on “The Paradigms
Criticisms of the previous editions have focused on UFEs
Debate and a Utilitarian Synthesis”, he reviews the history of
lack of generalizability to “multi-layered programs having
the quantitative vs quantitative paradigms of inquiry debate
many decision-makers and multiple information needs” (Smith
that has beleaguered the field for nearly three decades. An
and Young, 1987). Smith and Young lamented that in the
interesting note here is that in the second edition, the title
second edition Patton failed to use his experience to discuss
of the chapter on paradigms was called, “The Methodology
common problems in employing the I/FE approach. I believe
Dragon: The Paradigms Debate in Perspective”. In this third
that the third edition addresses this criticism by being more
edition the title reflects a truce in this debate and a need to
explicit about the goals of UFE and where and under what
move on. To this point he explains that since it would be
circumstances a UFE approach would be most desirable and
impossible to please the entire scientific community, his intent
appropriate. An additional criticism lodged by Smith and
is to please primary intended users. Finally he suggests that
Young was that several chapters in the first two editions were
“Issues of methodology are issues of strategy, not of morals”
based only on Patton’s experiences. They suggested that the
(p. 268). At the same time, he submits that the debate, while
book would benefit from grounding it more in the literature
having “withered”, is not yet dead. Of importance to Patton, is
of other evaluation researchers. Again, Patton seems to have
that Utili;uiion-FocusL’$ Ewluution does not require an eva-
responded to this criticism by providing several more references
luator to choose one paradigm over the other. Instead, he sees
from the evaluation research literature of the last ten years.
the C/FE evaluator as having many choices, all of which should
In Connor’s (I 980) review of UFE’s first edition he wondered
be guided by the stakeholders’ information needs.
if the book wouldn’t be of greater interest to evaluators inter-
Part 4 of the book is titled “Realities and Practicalities of
ested in instrumental evaluation. While still believing in the
Utilization, Focused Evaluation”. The first two chapters in this
value of instrumental formative use, Patton now emphasizes
section cover the power, politics and ethics of evaluation. Patton
learning as an important outcome of the evaluation process. As
provides the reader with examples of how evaluation is a pol-
organizational change becomes the norm, it behoves evaluators
itical activity, identifies ways in which an evaluator can work
not to consider how their practice must also change to most
to decrease the negative influence of political pressures and
effectively serve the organizations with which they work (Pre-
discusses the relationship between use and misuse. The book’s
skill and Torres, in press).
last chapter summarizes the benefits of employing a Ufilization-
Focused Evuhtion approach and offers a useful updated flow-
chart for designing Utilization-Focused Emluations. Using Utilization-Focused Evaluation: The New Century Text
As with his previous two editions of I/FE, Patton continues us a textbook
to include numerous examples that help explain the book’s
many concepts. An important addition to the third edition are Of all three editions I believe the third edition is the most
examples taken from many different kinds of organizations in useful for teaching graduate students the basics of program
the education, healthcare, government, private industry and evaluation. One of the challenges I face when teaching intro-
non-profit arenas. These examples contribute to the reader’s ductory evaluation courses is that for many students, “evalu-
 Book reviews 125

ation” equals a survey or test. To explore their initial
understanding of evaluation, I have students draw pictures of
what comes to mind when they think of the word “evaluation”.
The pictures they draw typically depict report cards, tests,
surveys, checklists and only sometimes a scale, light bulb, or
question mark. Students too often believe evaluation is about
methods and have little to no understanding of the complex
context in which evaluation occurs, or that it is a value-laden
political activity. Therefore, we spend about 30% of the sem-
ester on concepts such as evaluation use, evaluation logic, eva-
luator roles, politics of evaluation, evaluation standards and
ethics, stakeholder involvement, evaluation and organizational
learning and various evaluation theories. Only after this foun-
dation has been laid do we discuss evaluation designs and data
collection and analysis methods. While the students are at first
frustrated by not being able to jump in and develop instruments,
they soon learn that evaluation is much more than a bag of
technical tricks. Patton’s third edition of UFE is unquestionably
the most helpful book I have found to lay this foundation for
evaluation practice.
Throughout my eleven years of teaching graduate level
courses in program evaluation, I have used several of Michael
Q. Patton’s books. Students have found his work easy to read
and grounded in the practice and realities of organizational life.
In teaching the reader about evaluation, Patton skilfully uses
parables, stories and examples. Students who learn through
narrative often say they appreciate this characteristic of Pat-
ton’s writing. On the other hand, I have heard students say they
wish he’d just give them a checklist and forget the stories and
numerous examples! In the third edition, I believe Patton has
tried to attend to students’ different learning styles by keeping
the stories and examples, but by adding a number of “menus”
and “exhibits” (50 of them!) that summarize key points in
each chapter. These additions accomplish two things. First, the
menus and exhibits help those students who prefer to see an
overview of the chapter-those who want to know where they’re
going before they start the journey. Second, instructors can use
the menus and exhibits as a tool for engaging the class in large
and small group discussions and activities.
While I have few criticisms of the book, the one area in which
Patton might have done more is in the area of evaluation’s
relationship to organizational learning. He discusses organ-
ization development and developmental evaluation as a means
of learning and even refers to one of his experiences as having
resulted in organizational learning. However, the concept is
never defined, nor is other work linking evaluation to organ-
izational learning discussed or referenced.’ Given Patton’s inter-
est in process use, organizational change and development and
learning through evaluation, this omission was somewhat dis-
appointing.
There is no such thing as a perfect evaluation text. Every
semester I scour the publisher’s brochures just to make sure
there is no one book that addresses all the topics I teach in my
basic program evaluation course. This semester I am using five
books-including Patton’s UFE 3rd edition and it is one I will
continue to use.’ While the book lacks the level of detail I
need to cover topics such as data collection methods, analysis
techniques and issues of communicating and reporting, it is the
only text 1 have found that addresses the basic concepts of
program evaluation and focuses on the realities of program
evaluation within a local organizational context. It is grounded
in everyday practice students understand and relate to without
being overly simplistic. Patton never underestimates the intel-
ligence of his readers.
In summary, the third edition of Utilizution-Focused E~:ulu-
ation: The New Century Text is a welcome edition to the field
of evaluation and in particular to instructors of graduate level
program evaluation courses. It does not pretend to be a theor-
etical or empirically based research text as some may prefer.
What it is, however, is a solid mixture of a utilization-focused
evaluation philosophy and technology. Patton’s writing style
is engaging and entertaining. He includes excellent examples,
covers a broad range of topics and provides an update on an
alternative to the social science approach to evaluation. As
is true for the first two editions, Patton’s pragmatic view of
evaluation allows novice evaluators to more fully comprehend
what evaluation looks and feels like. For more experienced
evaluators, the book provides another perspective on our evolv-
ing practice and challenges us to find ways to respond to the
complexities of evaluation in changing organizations. Even if
readers have read the first two editions, this third edition has
enough new material to make its purchase worthwhile. For
those who find a kindred spirit in Patton’s philosophy of evalu-
ation, there is no better introduction.
References

Connor, R. (1980). Review of the book Utilixztion-Focused E&uution,

’ Several evaluation researchers have been actively writing in the area (1978). Knowledge, Creution. Diffusion, Utilization, l(4), 63 I-634.
of organizational learning and evaluation. They include: Cousins et al., Cousins, J. B., & Earl. L. M. (Eds) (1995). Purticipatorq ecaluntion
1996: Forss and Cracknell, 1994; Mathison, 1994; Owen and Lambert, in education: Studies in evaluation use and organizationul learning.
1995; Preskill, 1994; Preskill and Torres, 1996; Torres, 1994, Torres et London: Falmer Press.
al.. 1996). Forss, K., Cracknell, B., & Samset, K. (1994). Can evaluation help an
‘In addition to Patton’s (IF,!?. 3rd Edition, I have used the following organization to learn? Evaluation Review, 18(5), 574-59 1.
books this last year: Bourque and Fielder (1995). How to conduct self- Joint Committee on Standards for Educational Evaluation. (1994). The
administered and mail surreys. Thousand Oaks, CA: Sage; Patton, M. program evaluation standards. 2nd Edition. Thousand Oaks, CA:
Q. (1987). How IO use qualiratire methods in ecalucttion. Thousand Oaks, Sage.
CA: Sage; Joint Committee on Standards for Educational Evaluation, Mathison, S. (1994). Rethinking the evaluator role: Partnerships
(1994). The program evaluation stunduds. 2nd Edition. Thousand Oaks, between organizations and evaluators. Euduation and Program
CA: Sage and Torres et al. (1996). Etwluaiion strategies ,for com- P/arming, 17(3), 299-304.
municating and reporting.. Enhancing [earning in organizations. Thou- Owen, J. M., & Lambert, F. C. (1995). Roles for evaluation in learning
sand Oaks. CA: Sage. Though students bristle at having to buy so many organizations. Evaluation, I(2), 237-250.
books, I make the argument that these books will be ones they will refer Preskill, H. (1994). Evaluation’s role in enhancing organizational learn-
to over and over again as they practice evaluation, and therefore are ing: A model for practice. Evaluation atld Program Planning. I7(3),
more than mere “textbooks” to be traded in at the end of the semester. 29 1-297.
126 Book reviews

Preskill, H., & Torres, R. T. (1996). From evaluation to evaluative inquir?, Sonnichsen, R. C. (1987). Review of the book Utilization-Focused
for organizational learning. Paper presented at the 1996 American Evaluation, 2nd Edition (1986). Evaluation and Program Planning,
Evaluation Association conference, Atlanta, CA. M(3), 301-302.
Preskill, H., & Torres, R. T. (in press). Evaluative inquiryfor learning Torres, R. T. (1994). Concluding remarks: Evaluation and learning
in organizations. Thousand Oaks, CA: Sage. organizations: Where do we go from here? Evaluation and Program
Shadish, W. R., Newman, D. L., Scheirer, M. A., & Wye, C. (1995). Planning, 17(3), 339-340.
Guiding principles for evaluators. NeM; Directions ,fbr Program Torres, R. T., Preskill, H., & Piontek, M. (1996). Evaluation strategies
Evaluation, 66. ,for communicating and reporting.. Enhancing learning in organ-
Smith, N. L., & Young, M. J. (1987). Review of the book Utilization- izations. Thousand Oaks, CA: Sage.
Focused Evaluation. 2nd Edition (1986). Evaluation Practice, a(4).