This article was downloaded by: [69.253.48.

90]
On: 20 June 2014, At: 20:16
Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

The Clinical Supervisor

Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/wcsu20

Does Psychotherapy Supervision

Contribute to Patient Outcomes?
Considering Thirty Years of Research
a
C. Edward Watkins Jr.
a
University of North Texas , Denton, Texas, United States
Published online: 29 Nov 2011.

To cite this article: C. Edward Watkins Jr. (2011) Does Psychotherapy Supervision Contribute to
Patient Outcomes? Considering Thirty Years of Research, The Clinical Supervisor, 30:2, 235-256, DOI:
10.1080/07325223.2011.619417

To link to this article: http://dx.doi.org/10.1080/07325223.2011.619417

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the
“Content”) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources
of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or
howsoever caused arising directly or indirectly in connection with, in relation to or arising
out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
 The Clinical Supervisor, 30:235–256, 2011
Copyright # Taylor & Francis Group, LLC
ISSN: 0732-5223 print=1545-231X online
DOI: 10.1080/07325223.2011.619417

Does Psychotherapy Supervision Contribute

to Patient Outcomes? Considering Thirty
Years of Research

C. EDWARD WATKINS, JR.

University of North Texas, Denton, Texas, United States

After a century of psychotherapy supervision and over half a

Downloaded by [69.253.48.90] at 20:17 20 June 2014

century of supervision research, what do we know empirically

about the contribution of psychotherapy supervision to patient out-
comes? In this article, I address that question by (1) assembling all
identified supervision-patient outcome studies, from 1981 to 2006,
referenced in four reviews (Ellis & Ladany, 1997; Freitas, 2002;
Inman & Ladany, 2008; Wheeler & Richards, 2007) and (2) iden-
tifying additional post-review studies by means of computer
searches, spanning January 2006 through May 2011. A total of
18 supervision outcome studies emerged, spanning the past gener-
ation of supervision scholarship. Unfortunately, after closely scruti-
nizing each investigation, eliminating misidentified studies
(constituting over one-third of the 18 studies), and weighing the
gravity of various methodological deficiencies across investiga-
tions, the collective data appeared to shed little new light on the
matter: We do not seem to be any more able to say now (as opposed
to 30 years ago) that psychotherapy supervision contributes to
patient outcome. What did emerge of considerable promise, how-
ever, were three recent studies that were developed, organized,
and prosecuted with the primary objective of evaluating the effects
of supervision on patient outcome. Those investigations were high-
lighted because, in my view, they point the way for future studies to
follow (in ways not done before) and are prototypal in their design
and execution. Although the difficulty in researching the
supervision-patient outcome matter has long been lamented in
the supervision literature, those few studies (especially Bambling,
King, Raue, Schweitzer, & Lambert, 2006) indeed show us for

Address correspondence to C. Edward Watkins, Jr., 1155 Union Circle # 311280,

Psychology, UNT, Denton, TX 76203-5017. E-mail: watkinsc@unt.edu

235
 236 C. E. Watkins, Jr.

the first time that research on supervision-patient outcome can be

done and be done well.

KEYWORDS clinical supervision efficacy, clinical supervision

outcome, patient outcome, psychotherapy supervision efficacy,
psychotherapy supervision outcome

INTRODUCTION

In psychiatric and psychological practice, accountability has become an

increasingly pressing, preeminent issue (Thomason, 2010). The concepts of
competent practice, evidence-based practice, and empirically supported
therapies have now become an ever more vital part of our treatment lexicon;
they have substantially impacted how we think about treatment and treat-
Downloaded by [69.253.48.90] at 20:17 20 June 2014

ment education, and are now seemingly forever inextricably intertwined with
our conceptualizations of responsible, informed, and ethical therapeutic
implementation (American Psychological Association, 2006; American
Psychiatric Association, 2001; Fisher & O’Donohue, 2006; Spring & Walker,
2007; Weisz & Kazdin, 2010). We now also see the concepts of competent
practice and evidence-based practice increasingly becoming more central
considerations in how we think about, conduct, teach, and even supervise
psychotherapy supervision (Falender & Shafranske, 2007; Milne, 2009). Much
as psychotherapy has been called upon to account, psychotherapy super-
vision now finds itself called to do so as well.
While concerns about evidence for and impact of psychotherapy have
actually been with us for more than a century (Coriat, 1917; Eysenck, 1952;
Freud, 1909=1959; Smith, Glass, & Miller, 1980; Wampold, 2008), concerns
about evidence for and impact of psychotherapy supervision have compara-
tively been a much more recent phenomenon. The first empirical efforts to
investigate supervision did not occur until the 1950s (Harkness & Poertner,
1989), and in many respects, it has only been within the past 30 years that
supervision scholarship and study have truly exploded (cf. Bernard, 2005).
As psychotherapy supervision has matured and become ever more substan-
tial, questions about its efficacy have increasingly emerged (Inman & Ladany,
2008; Lambert & Ogles, 1997). Those questions have tended to take one of
two forms: Does supervision have a beneficial effect on supervisees (the
positive impact of supervisor on supervisee)?, or Does supervision actually
have a beneficial effect on supervisees’ patients (the positive impact of
supervisor on supervisee, which in turn positively impacts patients)?
Research thus far suggests that psychotherapy supervision indeed has a
most beneficial effect on supervisees. Some of those positive effects include
supervisee enhanced self-awareness, enhanced treatment knowledge, skill
acquisition and utilization, enhanced self-efficacy, and strengthening of the
 Psychotherapy Supervision Outcomes 237

supervisee–patient relationship (Beutler & Kendall, 1995; Goodyear &

Guzzardo, 2000; Holloway & Neufeldt, 1995; Inman & Ladany, 2008; Lambert
& Ogles, 1997; Wheeler & Richards, 2007). But research that examines the
impact of psychotherapy supervision on patient outcomes has proven much
more of a challenge. Admittedly, it is a most difficult endeavor to trace triadic
impact—the effect of the supervisor=supervision experience as processed
through the supervisee upon the patient (Wampold & Holloway, 1997).
Yet the importance of this type of outcome research cannot be overempha-
sized; it was identified as a significant press or need for twenty-first-century
psychotherapy supervision well over a decade ago (Watkins, 1998), con-
tinues to be referred to as the real ‘‘acid test’’ or ‘‘gold standard’’ of super-
vision efficacy (Bernard & Goodyear, 2009; Ellis & Ladany, 1997), and calls
for that test or standard to be substantively addressed continue to be issued
(Lichtenberg, 2007; Watkins, 2011; Westefeld, 2009).
Attention to the supervision-patient outcome issue began to take more
Downloaded by [69.253.48.90] at 20:17 20 June 2014

solid form and gather momentum in the mid to late 1990s. Holloway and
Neufeldt (1995), in their review of supervision effects, indicated that research
on the supervision-patient outcome matter was virtually nonexistent (cf.
Neufeldt, Beutler, & Banchero, 1997). But shortly thereafter, Ellis and Ladany
(1997) identified what to my knowledge was the first list of supervision-
patient outcome studies, which included nine such investigations; their brief
critique highlighted a host of methodological problems with that research
and they concluded that there were ‘‘few justifiable conclusions [that could
be drawn] from this set of studies’’ (p. 488). Building on that list, however,
Freitas (2002) provided a more detailed summary and analysis of most of
the nine studies identified by Ellis and Ladany (e.g., number of participants
involved, specific measures used) and included and detailed four other
studies that had not earlier been referenced; he reported no supervision
outcome studies appearing between 1997 and 2001. While not skirting the
methodological problems evident across studies, Freitas presented a more
optimistic view about the available research and focused his attention on
what the data had to offer for future research considerations. Five years later,
Wheeler and Richards (2007) devoted but a paragraph to supervision-patient
outcome, referenced only two other supervision outcome studies appearing
since the Freitas review, and noted the limited attention given to this
all-important matter. Inman and Ladany (2008) also devoted just 1 paragraph
to supervision-patient outcome research, indicated that about 18 such studies
had been done so far, identified 1 new study, and much like previous
reviewers, accentuated the difficulty in researching this subject.
In my view, the essence of this outcome concern—despite its problem-
atic researchability—was perfectly captured by Lichtenberg (2007) in his brief
commentary: ‘‘ . . . the reason for providing supervision and the ethical justi-
fication for requiring it are that it makes a difference with respect to client
outcomes’’ . . . . ‘‘supervisors’ impact on psychotherapy outcomes is critical
 238 C. E. Watkins, Jr.

in terms of evaluating the effectiveness of supervision and the ethics of

supervision practices’’ (p. 275). If we cannot show that supervision affects
patient outcome, then how can we continue to justify supervision? The ben-
efits of supervision on supervisees alone are not necessarily sufficient; while
valuable, they at best only provide us with an indirect link to patient out-
come. At this point in time, evidence of a direct link seems to be increasingly
imperative. As Watkins (2011) has stated, ‘‘the effectiveness question must be
compellingly addressed if supervision is to ever move beyond ‘the reason-
able but unproven practice stage’ and convincingly justify itself . . . . After a
century of existence, proof of supervision effectiveness for patient outcome
seems long, long overdue’’ (p. 63).
As calls for supervision-patient outcome research continue to be made
(Bernard & Goodyear, 2009; Ellis & Ladany, 1997; Lichtenberg, 2007;
Watkins, 1998, 2011; Westefeld, 2009), as supervision has been and is being
increasingly called upon to meet the competence and evidence-based chal-
Downloaded by [69.253.48.90] at 20:17 20 June 2014

lenge (see Milne, 2009, 2010; Stoltenberg, 2009; Stoltenberg & McNeill, 2009),
as some study of the supervision-patient outcome issue continues to be
conducted (Bambling, King, Raue, Schweitzer, & Lambert, 2006; Bradshaw,
Butterworth, & Mairs, 2007; White & Winstanley, 2010), and as we now
already have about 18 such studies (Inman & Ladany, 2008) spanning a gen-
eration of supervision research (from 1981 through mid-2011), I thought it
might be interesting to take a fresh look at this matter. Since Freitas’ (2002)
review, which largely built upon the work of Ellis and Ladany (1997), no
effort has been made to integrate the first two decades of supervision-patient
outcome research with research produced in the past decade. Where do we
now stand with regard to those ‘‘acid test’’ data? I would like to consider that
question by assembling all previously identified supervision-patient outcome
studies, complement those by including any recent studies that have been
conducted, and then examine the group of investigations as a whole for
any new insights or directions that they might have to offer. What does
our first generation of supervision-patient outcome research tell us?

DEFINITIONS AND METHOD

Psychotherapy will be defined as psychological treatment for issues that are

primarily psychological in nature, offered by mental health professionals
from various disciplines (e.g., psychology, psychiatry, social work, psychi-
atric nursing, and counseling). Psychotherapy supervision (or clinical super-
vision) will be defined as follows:

. . . an intervention provided by a more senior member of a profession to a

more junior member or members of that same profession. This relation-
ship is evaluative and hierarchical, extends over time, and has the simul-
taneous purposes of enhancing the professional functioning of the more
 Psychotherapy Supervision Outcomes 239

junior person(s); monitoring the quality of professional services offered to

clients that she, he, or they see; and serving as a gatekeeper for those who
are to enter the particular profession. (Bernard & Goodyear, 2009, p. 7)

That definition will be used here because (1) it is a widely accepted super-
vision definition in the United States and abroad (Milne, 2007), (2) its ele-
ments seem to nicely capture the essence of supervisory practice across
mental health specialties (cf. Gold, 2006; Hess, Hess, & Hess, 2008; Munson,
2001; Watkins, 1997), and (3) it distinguishes supervision as an actual
intervention separate from graduate course work and psychotherapy skills
training (cf. Hill & Lent, 2006; Robertson, 1995; Stein & Lambert, 1995). A
supervision-patient outcome study will be defined as follows: A study in
which (1) one of its stated objectives is the investigation of a supervision-
patient outcome link and=or (2) a measure (or measures) of patient outcome
is taken over time and that is then related back to supervision in some way. I
Downloaded by [69.253.48.90] at 20:17 20 June 2014

chose not to adopt stringent exclusionary review criteria that would only
further restrict an already restricted field of supervision-patient outcome stu-
dies. Instead, I chose to include any quantitative or qualitative investigations
that emerged, but then subject them to critical examination.
Articles to be included in this review were drawn from two sources: (1)
studies that had been previously identified by Ellis and Ladany (1997), Freitas
(2002), Wheeler and Richards (2007), and Inman and Ladany (2008); and (2)
a computer search for any new studies that would have appeared in the past
few years. Drawing on the reviews of Ellis and Ladany, Freitas, Wheeler and
Richards, and Inman and Ladany, 16 studies—which spanned from 1981
through 2006—were identified for inclusion (see Table 1). Using PsycINFO
and Google Scholar databases, computer searches were conducted for the
January 2006 through May 2011 period, with such keywords as ‘‘psycho-
therapy supervision outcomes,’’ ‘‘clinical supervision outcomes,’’ and ‘‘super-
vision outcomes’’ being inputted for article identification purposes. For that
approximate five-year period, two additional articles were identified for
inclusion (Bradshaw et al., 2007; White & Winstanley, 2010), bringing the
total number of supervision-patient outcome articles from 1981 through
mid-2011 to 18. A synopsis of each of the 18 articles is provided in Table 1.

RESULTS AND DISCUSSION

Those 18 studies have involved the full spectrum of mental health disciplines,
with the most recent investigations emerging from psychiatric nursing
(Bradshaw et al., 2007; White & Winstanley, 2010). The studies have ranged
from experimental research (Bambling et al., 2006), correlational research
(Harkness, 1995), case studies (Alpher, 1991), to surveys of perceptions
and opinions (Vallance, 2004). Participating supervisors and patients have
 Downloaded by [69.253.48.90] at 20:17 20 June 2014

TABLE 1 Eighteen Actual or Purported Psychotherapy Supervision-Patient Outcome Studies, 1981–May 2011

Author Study description Findings Limitations

Alpher (1991) NSOS: Intensive case study analysis involving 1 female patient, 1
male therapist, and 1 male supervisor. Purpose of study to
examine parallel process between therapy and supervision.
Patient improvement ratings across treatment were made by
patient, therapist, and supervisor. Structural Analysis of Social
Behavior ratings made by patient (for therapist) and therapist
(for patient and supervisor). Supervisor provided ratings of his
relationship with therapist. The effects of supervision on
patient outcome not considered at any point in article.
Bambling, 127 patients (87 f, 40 m), 127 therapists (96 f, 31 m), and 40 Patients in supervised as Supervision pretreatment
King, Raue, supervisors (31 f, 9 m) participated. Patients with major opposed to unsupervised training session and therapist
Schweitzer, depression were randomly assigned to receive 8 sessions of treatment rated the working allegiance effects were
& Lambert problem-solving therapy (PST) from either a supervised or alliance higher, their identified as potential
(2006) unsupervised therapist; the 3 supervision conditions, to which symptoms lower, their confounds; total power was

240
therapists were randomly assigned, were alliance skill focus, satisfaction with treatment insufficient to eliminate
alliance process focus, and no supervision: all therapists higher, and were more apt to possibility of Type II errors.
received manual-driven training on PST; all supervisors stay in treatment.
received manual-driven training in either alliance skill or
alliance process supervision. Effects of supervision on
client-rated working alliance and symptom reduction were
evaluated.
Bradshaw, 89 schizophrenic patients (sex not specified), 23 mental health Both experimental and control Nurses in experimental group
Butterworth, nurses (14 f, 9 m), and several nurse supervisors (number not groups showed significant significantly older and more
& Mairs specified) participated. Supervisors received 2-day course increases in case experienced than control
(2007) about clinical supervision from study’s first author. All nurses management knowledge and group nurses; only
received 36 days of formal training in Psychosocial their patients demonstrated supervision training for
Intervention (PSI) and small-group clinical supervision. In significant reductions in supervisors was a 2-day
addition, those nurses assigned to experimental group also affective and positive course, which was not
received workplace clinical supervision (whereas control symptoms and significant described; no non-PSI
group nurses did not). Nurses’ knowledge about serious improvements in social education control group;
mental illness and patients’ symptom changes were assessed. functioning; nurses in retrospective comparison
 Downloaded by [69.253.48.90] at 20:17 20 June 2014
Data were gathered twice—at very beginning of PSI training experimental group,
and at its end. however, also showed
greater knowledge about
psychological intervention
and schizophrenia and their
patients demonstrated
significantly greater
reductions in both positive
and total symptoms.
Couchon & 32 clients (19 f, 13 m), 21 counselors (17 f, 4 m), and 7 supervisors Supervision approximately
Bernard (3 f, 4 m) participated. Clients were being seen at a university 4 hours before next
(1984) clinic for a variety of personal issues. Effects of timing of counseling session emerged
supervision were examined across three conditions: as more of a ‘‘planning
supervision occurring (1) within 4 hours of next counseling session’’ comparatively.
session, (2) 1 day before next counseling session, or (3) 2 days Supervisor tended to
before next counseling session. All supervision and counseling function as more of a
241
sessions were audio-taped and rated for strategies generated consultant and be more
and time orientation employed. Client satisfaction and focused than in other 2
counselor satisfaction ratings were taken. treatment conditions.
Supervision session timing,
however, had no significant
effect on either client or
counselor satisfaction.
Dodenhoff 59 master’s-level counseling student therapists (34 f, 25 m) and 12 Student therapists who scored
(1981) supervisors (5 f, 7 m; 8 PhDs, 2 master’s degrees, 2 doctoral higher on interpersonal
students) participated; number of patients involved not attraction toward supervisor
specified; focus of study was on supervision as a social were rated to be more
influence process; student therapists completed an effective by their supervisors;
interpersonal attraction (to supervisor) measure at week 3 of higher ratings of patient
semester, supervisors completed (1) an effectiveness measure outcome were associated
for their supervisees (week 3 and at semester’s end) and (2) an with a direct supervisory
outcome measure for their supervisees’ patients (around week style.
group used; study
quasi-experimental in
design; ‘‘[s]upervisors’ fidelity
to the model of clinical
supervision was assessed via
monthly meetings with the
first author’’ (p. 6) but
nothing beyond that single
statement was offered to
help us understand what that
entailed.
Inadequate sample size (Ellis &
Ladany, 1997); while each
counselor was to see a
different client in each of the
3 timing conditions, that
objective was not achieved;
instead, ‘‘some counselors
saw the same client twice
(under different treatment
conditions)’’ (Couchon &
Bernard, 1984, p. 6).
Homemade measures
created to assess counselor
satisfaction with supervision
and counseling.
Patient outcome only rated at
one point in time (around
fifth session); no pre-post
patient outcome data
collected; no random
assignment; modification of
non-equivalent groups
design used.
(Continued )
 Downloaded by [69.253.48.90] at 20:17 20 June 2014

TABLE 1 Continued

Author Study description Findings Limitations

5), and patients also completed the same outcome measure

around week 5.
Friedlander, NSOS: Intensive case study analysis involving 1 patient, 1 female
Siegel, & therapist, and 1 female supervisor. Purpose of study to
Brenock examine parallel process between therapy and supervision.
(1989) Across triad, 7 pre-treatment measures taken, 12 measures
taken during treatment, and 4 measures taken posttreatment.
Therapist rated patient satisfaction at end of treatment; patient
reported being ‘‘too busy’’ to complete any posttreatment
measures. If a prime objective was to investigate effects of
supervision on patient outcome, that objective was never
stated, and absent any patient posttreatment data, that seems
virtually impossible to achieve.
Harkness Same data set=measures=participants from Harkness and Significant relationships found ‘‘Weak experimental control

242
(1995) Hensley (1991; see below) used. Test of Shulman’s between therapist ratings of over volunteer subjects
interactional helping theory. 36 correlations performed to supervisory empathy and increased error variance’’
examine possible relations between supervisory skills, client ratings of contentment, (Harkness, 1995, p. 70);
relationship, and helpfulness and practice and client supervisory helpfulness and ‘‘Causal inferences cannot be
outcomes. client contentment, and made from the findings of
supervisory relationship and this investigation, and the
client contentment and goal narrow scope of its sample
attainment. mitigates against
generalization about skills,
relationships and outcomes
of practice in other settings’’
(Harkness, 1995, p. 70);
‘‘ . . . one is left wondering
what can be inferred . . .’’
(Freitas, 2002, p. 361).
Harkness Same data set=measures=participants from Harkness and Findings interpreted as Large number of causal tests
(1997) Hensley (1991; Harkness, 1995; see subsequent entry) again supporting and altering conducted; limitations under
used; test of Interactional Social Work theory; multiple interactional view of social Harkness (1995) and
 Downloaded by [69.253.48.90] at 20:17 20 June 2014

regression in cross-lagged panel design employed; examined work practice; supervision Harkness and Hensley
causal connections among the skills, relationships, and problem-solving and (1991) entries also apply
outcomes of supervised practice; 81 causal tests of association empathic skills found to have here (see above).
conducted. a causal influence on ratings
of supervisory relationship
and supervisory helpfulness
but not in ways expected;
empathy actually found to
have a direct negative effect
on the supervision
experience (a truly unique
finding in empathy=
supervision research).
Harkness & 161 patients (87 f, 74 m), 2 male therapists (master’s-level Under client-focus=mixed Multiple-baseline research
Hensley psychologists), 2 female therapists (master’s-level social focus-supervision as design; ‘‘any combination of
(1991) workers), and 1 female supervisor (certified social worker) opposed to mixed-focus order effects, sampling error,
participated; 4 therapists first exposed to 8 weeks of (only) supervision, patient and the interaction of testing
mixed-focus supervision (case management=consultation) depression decreased; and treatment may have

243
followed by mix of 8 weeks of client-focus (individual=group ratings of therapist confounded the findings’’
supervision)=mixed-focus supervision. Effects of mixed-focus helpfulness, goal attainment, (Harkness & Hensley, 1991,
versus client-focus=mixed-focus supervision were evaluated and patient-therapist p. 511); throughout article,
for: depressive symptoms, patient satisfaction with therapist partnership all increased. client-focused supervision is
helpfulness, goal attainment, and patient-therapist partnership. contrasted with mixed-focus
supervision but in reality
client-focused group was a
supervision amalgam, not a
pure type.
Iberg (1991) NSOS: Examined what were designated as 3 supervision themes:
Give more empathic responses, don’t give advice or
suggestions, and don’t ask questions. 6 doctoral student
therapists in clinical psychology (5 f, 1 m) each recruited 8
volunteer pseudo-clients (27 f, 21 m) to participate. Therapists
were assigned to each level of the 3 experimental conditions
(empathy: few versus many statements; suggestions:

(Continued )
 Downloaded by [69.253.48.90] at 20:17 20 June 2014

TABLE 1 Continued

Author Study description Findings Limitations

volunteered versus withheld; questions: asked versus not

asked) across their 8 volunteer clients. Results analyzed via
Statistical Control Theory. No supervisors or supervision
involved. This was a study of therapy skill development
specifically where pseudo-clients (e.g., friends) were used for
role-play purposes. Results could be useful in informing
supervision but had nothing to do with the actual effects of
supervision on patient outcome.
Kivlighan, 48 undergraduate volunteers (39 f, 9 m; awarded extra course Patients in live as opposed to Due to clerical error, half
Angelone, & credit), 48 master’s-level counseling student therapists (29 f, videotaped supervision rated of the working alliance
Swafford 19 m) and 17 supervisors (sex unspecified); 1 PhD and 16 the working alliance higher questionnaire data could
(1991) doctoral students) participated; recruited volunteer patients and their sessions as not be used; no random
seen for 4 50-minute therapy sessions; effects of videotaped rougher; authors concluded assignment involved;
supervision (tape review) versus live supervision were that live supervision may pre-experimental,

244
compared on patients’ ratings of working alliance and session have an accelerative learning non-equivalent groups
smoothness-ease and depth-value. effect on supervisees. design used.
Mallinckrodt & NSOS: 50 counselor-client dyads participated; after third session,
Nelson counselor and client completed Working Alliance Inventory;
(1991) effects of counselor training level (novice, advanced,
experienced) on working alliance formation investigated;
though having potential supervision implications, this study
involved no supervision at all.
Milne, NSOS: Intensive qualitative and quantitative case study analysis
Pilkington, involving 1 male patient, 1 female therapist, and 1 male
Gracie, & supervisor. Purpose of study to examine thematic content
James (2003) similarities between cognitive-behavioral therapy and
supervision. 10 supervision and 10 therapy sessions coded
using qualitative and quantitative content analysis
methodology. Supervision themes nicely reflected in therapy
process; a ‘‘parallelism’’ occurred. No measures of patient
outcome taken, effects of supervision on patient outcome not
addressed. Focus of study on the process of supervision to
specifically impact therapist behavior.
 Downloaded by [69.253.48.90] at 20:17 20 June 2014

Sandell (1985) NSOS: 20 psychiatric outpatients (sex unspecified) and 5

psychotherapists (sex unspecified) participated. Purpose of
study to test influence of patient’s ego level, therapist
competence, and supervision on effects of Mann’s time-limited
psychodynamic psychotherapy. Interview summary ratings
provided the bulk of the data. Supervision was a binary
variable, either received or not and did not involve a
professional supervisor but was peer in nature. Supervision
was conducted ‘‘in groups of 3-4 persons who met once a
week for mutual supervision of 1 case each’’ (Sandell, 1985,
p. 105). Data were analyzed by means of path analysis, and
supervision was judged to have ‘‘had, if anything, a negative
influence’’ (p. 103).
Steinhelber, Data gathered from 237 psychiatric patients (154 f, 83 m) of When therapist trainees and Survey study; ex post facto
Patterson, mixed diagnoses and 51 therapist trainees (sex not specified; supervisors shared similar design; patient outcome
Cliffe, & primarily psychiatric residents and clinical psychology interns). theoretical orientation, measured by but a
LeGoullon Therapist trainees (1) completed questionnaire about patients, therapist trainees’ GAS single-item global rating
(1984) their treatment, and supervision matters and (2) completed a ratings reflected greater made by trainee at 2 different

245
Global Assessment Scale (GAS) for each patient at treatment’s patient improvement; times; cause-effect
beginning and at time of study. Effects of amount of amount of supervision was conclusions not possible.
supervision and therapist-supervisor theoretical congruence unrelated to trainee ratings
on patient outcome were evaluated. of patient GAS.
Triantafillou 14 supervisory=management staff and 10 direct care workers (all Both treatment and control Pilot study with very small
(1997) with at least a 3-year child=youth worker college diploma) groups showed reduction in sample size; no random
received 4, 3-hour weekly training sessions in solution-focused serious incidence behaviors assignment; extremely
supervision (SFS). To test effects of training, 5 problem during study but treatment limited study details (entire
residents from one residential facility (where SFS was taught to group showed substantially study given but a single page
staff=workers) were compared against 7 problem residents more (75% less than control in 23-page article).
from a different facility (where SFS not taught) on serious group).
incidence behaviors and medication usage. Pilot study of
training program package. No actual supervision observed or
measured.

(Continued )
 Downloaded by [69.253.48.90] at 20:17 20 June 2014

TABLE 1 Continued

Author Study description Findings Limitations

Vallance (2004) NSOS: Qualitative study of 19 practicing counselors receiving

individual supervision; objective to examine counselors’
perceptions of supervision’s impact on their clients;
perceptions assessed via open-ended questionnaires or
semi-structured interviews; no actual supervision observed,
measured, or studied.
White & 170 patients (sex not specified), 186 mental health nurses (sex Nurse supervisors scored Supervision in this study
Winstanley not specified), 54 unit staff (sex not specified), and 24 nurse significantly higher on seemingly defined as
(2010) supervisors (17 f, 7 m) participated. 2 nurses received 4-day Manchester Clinical occurring in ‘‘small groups of
residential course in clinical supervision, which involved Supervision Scale (MCSS) individuals (n-6) attending a
theory, practice, and direct feedback, and were then assigned after 4-day training course pre-arranged meeting with

246
to conduct yearlong group supervision of neophyte and maintained difference
supervisees at their worksites. The control group was a scores 12 months later;
no-supervision condition. Qualitative diary data were collected supervisee MCSS scores did
monthly from supervisors; quantitative data were collected not change significantly over
from mental health nurses (baseline=12 months), patients 12 months of supervision;
(baseline=6 and 12 months), and unit staff (6 months). statistically significant
differences not demonstrated
in either quality of care or
patient satisfaction.
an appropriately trained
clinical supervisor, for 45–60
minutes per session, on a
monthly frequency, for
facilitated reflective
discussion, in confidence,
around matters of
professional relevance and
importance’’ (p. 152);
influence of middle
managers facilitated or
frustrated supervision efforts
across work settings.

Notes. NSOS ¼ Not Supervision Outcome Study; if a study was judged NSOS, then no Findings or Limitations were provided. f ¼ females, m ¼ males.
 Psychotherapy Supervision Outcomes 247

run the full gamut as well—from recruited undergraduate or community vol-

unteer pseudo-clients (Iberg, 1991) to actual patients with schizophrenia
(Bradshaw et al., 2007), from experienced supervisors (Bambling et al.,
2006) to those given only two days or less of supervisory training (Bradshaw
et al., 2007; Triantafillou, 1997) before being designated ‘‘supervisor.’’ With
but two exceptions, where qualitative analysis was employed exclusively
or in part (Milne, Pilkington, Gracie, & James, 2003; Vallance, 2004), all of
the studies were quantitative in design and analysis.
In surveying the first 20 years (1981–2000) and last 10 years (2001–
mid-2011) of this outcome research, one of the most surprising results from
my perspective is this: Approximately 40% of those investigations actually
identified as being supervision-patient outcome in nature have been misi-
dentified and are not really supervision-patient outcome studies at all. Seven
of the 18 studies fall into that category by my review: Alpher (1991);
Friedlander, Siegel, and Brenock (1989); Iberg (1991); Kivlighan, Angelone,
Downloaded by [69.253.48.90] at 20:17 20 June 2014

and Swafford (1991); Mallinckrodt and Nelson (1991); Milne and colleagues
(2003); and Sandell (1985). Alpher (1991) and Friedlander and colleagues
(1989) were both case studies of the parallel process phenomenon. While
study of parallel process would seem to readily lend itself to supervision-
patient outcome inferences, that is not necessarily a given. In Alpher
(1991), patient improvement ratings were taken, but at no point in his article
were those in any way considered in relation to supervision and patient
outcome. Friedlander and colleagues (1989) were also appropriately cautious
in considering the implications of their data; while the therapist provided
ratings about her patient’s outcome, her client failed to provide any post-
treatment outcome data, thus limiting any conclusions that could be drawn.
Friedlander and colleagues (1989) mentioned no supervision-outcome link at
any point in their article. Milne and colleagues (2003) studied thematic simi-
larities between cognitive-behavioral therapy and supervision (parallelism)
in an intensive case design; their investigation’s focus was on how super-
vision actually impacted therapist behavior. The effects of supervision on
patient, however, were not considered.
Mallinckrodt and Nelson (1991), while conducting a study that may have
supervision implications, investigated the impact of therapist training level
(novice, advanced, experienced) on formation of the therapeutic working
alliance. This was strictly a study of psychotherapy; no supervision was
involved. Iberg (1991), while orienting his study around ‘‘supervision
themes,’’ actually conducted an investigation of psychotherapy skills training
(e.g., learning empathic skills), not supervision. Six doctoral-level clinical psy-
chology students recruited pseudo-clients for role-play purposes, with the
skills focus being on empathy, questions, and suggestions. No supervision
or supervisors were involved at any time. Sandell (1985) provided data about
‘‘voluntary small-group peer supervision’’ on patient outcome in short-term
psychodynamic psychotherapy; again, trained professional supervision was
 248 C. E. Watkins, Jr.

not part of this study. Kivlighan and colleagues (1991) found that recruited,
compensated (extra course credit) undergraduate ‘‘clients,’’ when exposed
to supervisees receiving live versus videotaped supervision, rated their four
counseling sessions as rougher and the working alliance as stronger during
the live supervision condition. No actual change ratings were taken or made;
this appeared to primarily be an investigation of how live versus videotaped
supervision modalities affected clients’ session and working alliance percep-
tions. How that impacted overall outcome, however, was not assessed. It is
my contention that those seven studies have been identified as supervision-
patient outcome studies (see Ellis & Ladany, 1997; Freitas, 2002; Wheeler &
Richards, 2007) when in fact they are not. While some supervision implica-
tions may be drawn from some of those studies, I believe we should be
quite cautious about specifically drawing any supervision-patient outcome
implications from them.
Unfortunately, I do not believe those are the only studies in this group
Downloaded by [69.253.48.90] at 20:17 20 June 2014

of 18 that give us reason for question or concern. We also have studies here
that are purely survey or opinion based (Steinhelber, Patterson, Cliffe, &
LeGoullon, 1984; Vallance, 2004), pilot in nature (Triantafillou, 1997), involve
data duplication (see Harkness, 1995, 1997), or failed to adequately assess
patient outcome (Dodenhoff, 1981). Steinhelber and colleagues (1984) eval-
uated therapist trainees’ questionnaire and single-item rating scale responses
about patient diagnosis and progress, treatment variables, and supervision;
this was really a study about therapists’ perceptions of self, their supervisors,
and their patients. Though the trainees were all in supervision, no actual
supervision was observed or studied. Vallance (2004) conducted a qualitative
study in which she investigated 19 counselors’ opinions about the impact
of supervision on their treatment efforts and its ultimate impact on patient
outcome. This, too, was a study of counselors’ perceptions about self, their
clients, and their supervisors, tapped by means of either an open-ended
questionnaire or semi-structured interview. Again, though all participants
were being supervised, no actual supervision was observed or studied in this
research either. (While I think that therapist perceptual data alone can be
important and have a place in supervision research, I also think that at this
particular point in time we need much more than that for the supervision-
patient outcome area of inquiry to move forward.)
With Triantafillou (1997), we have a supervision training package (four
3-hour meetings of supervision instruction) pilot tested on 12 behaviorally
disturbed residents (5 in treatment group versus 7 in control group); super-
visors trained in solution-focused supervision served the treatment group,
whereas the control group served as a no-supervision condition. While much
attention and care appears to have been given to the training package pres-
entation, no monitoring, observation, or study of actual supervision occurred
once that was complete. Harkness and Hensley (1991) conducted an interest-
ing study that involved 1 supervisor, 4 supervisees, and 161 mental health
 Psychotherapy Supervision Outcomes 249

center patients, but much like cutting the same pie up time and time again,
those very same data were reanalyzed on two other occasions (Harkness,
1995, 1997). While Harkness’ two later studies have been recognized as part
of our body of supervision outcome studies (see Freitas, 2002, who did men-
tion the matter of piecemeal publication), I believe that we can certainly
question that wisdom. If nothing else, those are not new data sets, and while
we may cautiously draw implications from them, any such implications
would probably best be placed within the confines of Harkness and
Hensley’s first study (1991). In Dodenhoff’s (1981) investigation, both client
and supervisor completed a client outcome rating scale around the fifth
counseling session; no pretreatment or posttreatment measure was taken,
so any type of pre=post comparison was not possible. This was foremost a
study about supervisor social influence, so the supervisor and client outcome
ratings were used exclusively as measures of counselor trainee effectiveness
(seen as being influenced by supervisor social power). Consideration of
Downloaded by [69.253.48.90] at 20:17 20 June 2014

supervision-patient outcome was not a prime objective, if an objective at all.

What does this misidentification, data duplication, pilot, survey=
opinion, and inadequate measurement attention and mention mean for our
examination of psychotherapy supervision-patient outcome? We may have
thought that we had a body of 18 studies on this subject but I do not believe
that that is really the case at all. From my perspective, if we are to truly con-
sider the more solidly constructed and conducted investigations, we actually
have only three such studies upon which we can now draw, all published
within the past five years: Bambling and colleagues (2006); Bradshaw and
colleagues (2007); and White and Winstanley (2010). While investigations
by Couchon and Bernard (1984) and Harkness and Hensley (1991) also
examined supervision-patient outcome, those each suffered from some ser-
ious methodological deficiencies that sorely limit the strength of their find-
ings (e.g., condition confounds, homemade measures, uncontrolled Type I
and Type II error; Ellis & Ladany, 1997; Freitas, 2002). And while the research
of Bambling and colleagues (2006), Bradshaw and colleagues (2007), and
White and Winstanley (2010) is certainly not deficiency free (see Limitations
in Table 1), their studies in my view provide us with new light—giving us
substantive and prototypal direction for thinking about and tackling the
ever-challenging conundrum of supervision-patient outcome. Let us look
more closely at each of those.
The truly stellar, model study is Bambling and colleagues (2006; see
Table 1 for more detail). This was foremost an investigation undertaken to
plainly and simply research the supervision-patient outcome issue, and four
of its hypotheses directly compared supervised versus unsupervised treat-
ment on patient variables, with depression symptom reduction and treatment
completion being two of them. Experienced therapists and supervisors
(primarily from psychology and social work) were used, all patients had a
diagnosis of major depression, and eight sessions of supervision were
 250 C. E. Watkins, Jr.

provided. There was one other major study difference: A supervision manual
was used to guide the work of supervision. (Therapists also were provided
with their own treatment manual.) To my knowledge, this is only the second
supervision study (other than Patton & Kivlighan, 1997) to ever use a super-
vision manual. While manuals have a definite history of being referenced and
recommended in the supervision literature (Bernard & Goodyear, 2009;
Goodyear & Bernard, 1998; Goodyear & Guzzardo, 2000; Holloway, 1992;
Lambert & Arnold, 1987; Lambert & Ogles, 1997; Watkins, 1998), their use
in supervision research has been virtually absent. It may indeed be that the
‘‘most promising possibility for outcome research on the effects of supervi-
sion . . . appears to be in the area of large-scale outcome studies of manua-
lized treatments . . .’’ (Neufeldt et al., 1997, pp. 519–520). Perhaps what
Bambling and colleagues have done is to also show us that ‘‘manualized’’
has a place in supervision too: Just as psychotherapy research has benefited
from a manualized approach, supervision research could do so as well.
Downloaded by [69.253.48.90] at 20:17 20 June 2014

While manuals are by no means a panacea (Scott & Binder, 2002), they do
provide us with one viable means of gathering useful research data (see
Lambert, 2004) and might be useful in exposing a supervision data mine that
has yet to be unearthed, explored, and excavated.
Two recent research investigations from psychiatric mental health nurs-
ing (Bradshaw et al., 2007; White & Winstanley, 2010; see Table 1 for more
detail), though not as refined as Bambling and colleagues’ study, also present
a highly focused approach to the specific study of supervision and patient
outcome that could prove quite informative for future study (NURSINGtimes.
net, 2010; Proctor, 2010). Bradshaw and colleagues (2007) examined the
effects of supervision (provided by nurse ‘‘supervisors’’ receiving a two-day
course in clinical supervision) on nurses receiving Psychosocial Intervention
Training (PIT; family and cognitive-behavioral intervention) plus workplace
supervision (treatment group) versus a group of nurses receiving PTI only
(no workplace supervision control group). Supervision was provided every
other week, with there being one supervisor and two supervisees per group.
Eighty-nine patients with schizophrenia served as the patient group. Patients
whose caregivers received PIT plus workplace supervision (compared with
the control group) experienced greater reductions in positive and total psy-
chotic symptoms measured. While the Bradshaw and colleagues investi-
gation was not a randomized control trial (RCT) and was certainly the
weakest methodologically of the three studies highlighted here, I include it
because (1) it had as a preeminent intent the examination of supervision-
patient outcome; (2) it provided a corresponding structure that directly
allowed that to be done (contrary to most of the other 18 studies); and (3)
it clearly has the potential to stimulate useful heuristic thought about pushing
the matter under study forward. White and Winstanley (2010) examined the
effects of supervision (provided by nurse ‘‘supervisors’’ receiving a four-day
residential course in clinical supervision) on mental health nurses receiving
 Psychotherapy Supervision Outcomes 251

yearlong supervision (treatment group) versus nurses who received no

supervision (control group). One hundred seventy patients from various
mental health facilities served as the patient group, but after a year of study
and multiple measures being taken, no significant differences emerged
between patients in the treatment and control groups on treatment satisfac-
tion and quality of care. While this was a much-involved RCT, it appears that
management issues across some of the participating mental health facilities
may have greatly affected the prosecution of the entire investigation, the
supervision provided, and the resulting outcomes (Proctor, 2010; White &
Winstanley, 2010).
The Bradshaw and colleagues (2007) and White and Winstanley (2010)
investigations raise two questions: (1) Does a two-day or four-day course in
clinical supervision ‘‘make’’ a supervisor? In psychology, psychiatry, and
social work, the project of becoming and being a supervisor has tended to
be viewed as a developmental process (Rodenhauser, 1994; Watkins, 2010)
Downloaded by [69.253.48.90] at 20:17 20 June 2014

that involves growth over time. Would such a brief training period be suffi-
ciently powerful to render one a good-enough supervisor? (2) How fre-
quently must supervision occur for it to have a beneficial effect? In
Bambling and colleagues (2006), individual supervision was provided once
a week; in Bradshaw and colleagues (2007), supervision was provided to
two supervisees at a time every other week; in White and Winstanley
(2010), supervision was provided in groups consisting of anywhere from
six to nine individuals once a month. If the supervisory alliance is a critical
component of supervision, and accumulating research suggests that it is
(Inman & Ladany, 2008; Ladany, 2004), can such an alliance actually be
established in groups of 6 to 9 supervisees meeting only once a month for
about 45 minutes? It may well be that another reason White and Winstanley’s
nicely conducted RCT failed to find supervision-patient effects was that the
amount of time devoted to supervision was not sufficiently powerful to have
an effect. As Proctor (2010) stated, ‘‘. . . makes me shake my head at the idea
of nine supervisees in a 1 hour (or less) monthly group’’ (p. 171). It may also
be possible that some of what led to positive results in Bradshaw and collea-
gues’ (2007) study was the more frequent supervision conducted with but
two supervisees. Those possibilities would at least be reasonable hypotheses
to consider by means of future research.
Of all the studies considered here, those three—Bambling and collea-
gues (2006), Bradshaw and colleagues (2007), and White and Winstanley
(2010)—provide the best and clearest directions for further thought about
conducting future successful research in the supervision-patient outcome
area. Those investigations show us that, despite jeremiads about the difficult-
ies of researching this subject, it can be done and be done well. As concern
mounts about more substantively demonstrating the effectiveness of super-
vision on patient outcome, we have our first vestiges, tentative though they
may be, of evidence to that effect. But beyond that, I really do not believe
 252 C. E. Watkins, Jr.

that we can say any more right now. When we take into account study mis-
identifications along with other issues mentioned earlier (e.g., data dupli-
cation), I am not sure that we can safely draw any further conclusions
about supervision outcome from this very limited group of research investi-
gations. What we do have, in my opinion, is a beginning, and I believe some
of the work done in the past few years opens a door that had not been
opened before. For the first time in supervision’s 100-year history, the
psychotherapy supervision-patient outcome problem does not have to be
the will-o’-the-wisp it has always been.

CONCLUSION

Does psychotherapy supervision positively affect patient outcomes? After a

century of psychotherapy supervision and over half a century of supervision
Downloaded by [69.253.48.90] at 20:17 20 June 2014

research, we still cannot empirically answer that question. But in this age of
accountability, we can be assured that that question will continue to be an
increasingly preeminent press for psychotherapy supervision in the
twenty-first century. In surveying the last 30 years of supervision outcome
research (actual and purported), the drawing of any conclusions about
supervision’s effects on patient outcome seems premature. In my view, we
have not arrived at the point where we can safely do that. But some recent,
nicely done studies produced in the past few years do provide us with sub-
stantive examples, exciting possibilities, and charted directions upon which
we can build in our quest to further unravel the psychotherapy
supervision-patient outcome riddle.

REFERENCES

Alpher, V. S. (1991). Interdependence and parallel processes: A case study of struc-

tural analysis of social behavior in supervision and short-term dynamic psycho-
therapy. Psychotherapy, 28, 218–231.
American Psychiatric Association. (2001). Practice guidelines for the treatment of
psychiatric disorders. Washington, DC: American Psychiatric Press.
APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based
practice in psychology. American Psychologist, 61, 271–285.
Bambling, M., King, R., Raue, P., Schweitzer, R., & Lambert, W. (2006). Clinical
supervision: Its influence on client-rated working alliance and client symptom
reduction in the brief treatment of major depression. Psychotherapy Research,
16, 317–331.
Bernard, J. M. (2005). Tracing the development of clinical supervision. The Clinical
Supervisor, 24, 3–21.
Bernard, J. M., & Goodyear, R. K. (2009). Fundamentals of clinical supervision (4th
ed.). Upper Saddle River, NJ: Merrill.
 Psychotherapy Supervision Outcomes 253

Beutler, L. E., & Kendall, P. C. (Eds.) (1995). Introduction to the special section: The
case for training in the provision of psychological therapy. Journal of Consult-
ing and Clinical Psychology, 63, 179–183.
Bradshaw, T., Butterworth, A., & Mairs, H. (2007). Does structured clinical super-
vision during psychosocial intervention education enhance outcome for mental
health nurses and the service users they work with? Journal of Psychiatric and
Mental Health Nursing, 14, 4–12.
Coriat, I. H. (1917). Some statistical results of the psychoanalytic treatment of the
psychoneuroses. The Psychoanalytic Review, 4, 209–216.
Couchon, W. D., & Bernard, J. M. (1984). Effects of timing of supervision on super-
visor and counselor performance. The Clinical Supervisor, 2, 3–20.
Dodenhoff, J. T. (1981). Interpersonal attraction and direct-indirect supervisor influ-
ence as predictors of counselor trainee effectiveness. Journal of Counseling
Psychology, 28, 47–52.
Ellis, M. V., & Ladany, N. (1997). Inferences concerning supervisees and clients in
clinical supervision: An integrative review. In C. E. Watkins, Jr. (Ed.), Handbook
Downloaded by [69.253.48.90] at 20:17 20 June 2014

of psychotherapy supervision (pp. 447–507). New York: Wiley.

Eysenck, H. J. (1952). The effects of psychotherapy: An evaluation. Journal of
Consulting Psychology, 16, 319–324.
Falender, C. A., & Shafranske, E. P. (2007). Competence in competency-based super-
vision practice: Construct and application. Professional Psychology: Research
and Practice, 38, 232–240.
Fisher, J. E., & O’Donohue, W. T. (2006). Practitioner’s guide to evidence-based
psychotherapy. New York: Springer.
Freitas, G. J. (2002). The impact of psychotherapy supervision on client outcome: A
critical examination of 2 decades of research. Psychotherapy, 39, 354–367.
Freud, S. (1909=1959). Analysis of a phobia in a five-year-old boy. Collected papers
(vol. 3, pp. 149–289). London: Hogarth Press.
Friedlander, M. L., Siegel, S., & Brenock, K. (1989). Parallel process in counsel-
ing and supervision: A case study. Journal of Counseling Psychology, 36,
149–157.
Gold, J. H. (Ed.). (2006). Psychotherapy supervision and consultation in clinical
practice. New York: Aronson.
Goodyear, R. K., & Bernard, J. M. (1998). Clinical supervision: Lessons from the
literature. Counselor Education and Supervision, 38, 6–22.
Goodyear, R. K., & Guzzardo, C. R. (2000). Psychotherapy supervision and training.
In S. D. Brown & R. W. Lent (Eds.), Handbook of counseling psychology (3rd ed.,
pp. 83–108). New York: Wiley.
Harkness, D. (1995). The art of helping in supervised practice: Skills, relationships,
and outcomes. The Clinical Supervisor, 13, 63–76.
Harkness, D. (1997). Testing interactional social work theory: A panel analysis of
supervised practice and outcomes. The Clinical Supervisor, 15, 33–50.
Harkness, D., & Hensley, H. (1991). Changing the focus of social work supervision:
Effects on client satisfaction and generalized contentment. Social Work, 36,
506–512.
Harkness, D., & Poertner, A. (1989). Research and social work supervision: A
conceptual review. Social Work, 34, 115–119.
 254 C. E. Watkins, Jr.

Hess, A. K., Hess, K. D., & Hess, T. H. (Eds.). (2008). Psychotherapy supervision:
Theory, research, and practice (2nd ed.). Hoboken, NJ: Wiley.
Hill, C. E., & Lent, R. W. (2006). A narrative and meta-analytic review of helping
skills training: Time to revive a dormant area of inquiry. Psychotherapy, 43,
154–172.
Holloway, E. L. (1992). Supervision: A way of teaching and learning. In S. D. Brown
& R. W. Lent (Eds.), Handbook of counseling psychology (pp. 177–214). New
York: Wiley.
Holloway, E. L., & Neufeldt, S. A. (1995). Supervision: Its contributions to treatment
efficacy. Journal of Consulting and Clinical Psychology, 63, 207–213.
Iberg, J. R. (1991). Applying Statistical Control Theory to bring together clinical
supervision and psychotherapy research. Journal of Consulting and Clinical
Psychology, 59, 575–586.
Inman, A. G., & Ladany, N. (2008). Research: The state of the field. In A. K. Hess,
K. D. Hess, & T. H. Hess (Eds.), Psychotherapy supervision: Theory, research,
and practice (2nd ed., pp. 500–517). Hoboken, NJ: Wiley.
Downloaded by [69.253.48.90] at 20:17 20 June 2014

Kivlighan, D. M., Angelone, E. O., & Swafford, K. G. (1991). Live supervision in indi-
vidual psychotherapy: Effects on therapist’s intention use and client’s evaluation
of session effect and working alliance. Professional Psychology: Research and
Practice, 22, 489–495.
Ladany, N. (2004). Psychotherapy supervision: What lies beneath. Psychotherapy
Research, 14, 1–19.
Lambert, M. J. (Ed.). (2004). Bergin and Garfield’s handbook of psychotherapy and
behavior change. New York: Wiley.
Lambert, M. J., & Arnold, R. C. (1987). Research and the supervision process.
Professional Psychology: Research and Practice, 18, 217–224.
Lambert, M. J., & Ogles, B. M. (1997). The effectiveness of psychotherapy
supervision. In C. E. Watkins, Jr. (Ed.), Handbook of psychotherapy supervision
(pp. 421–446). New York: Wiley.
Lichtenberg, J. W. (2007). What makes for effective supervision? In search of clinical
outcomes. Professional Psychology: Research and Practice, 38, 27–28.
Mallinckrodt, B., & Nelson, M. L. (1991). Counselor training level and the formation
of the psychotherapeutic working alliance. Journal of Counseling Psychology,
38, 133–138.
Milne, D. (2007). An empirical definition of clinical supervision. British Journal of
Clinical Psychology, 46, 437–447.
Milne, D. (2009). Evidence-based clinical supervision: Principles and practice.
Hoboken, NJ: Wiley.
Milne, D. (2010). Can we enhance the training of clinical supervisors? A national pilot
study of an evidence-based approach. Clinical Psychology and Psychotherapy,
17, 321–328.
Milne, D., Pilkington, J., Gracie, J., & James, I. (2003). Transferring skills from super-
vision to therapy: A qualitative and quantitative N ¼ 1 analysis. Behavioural and
Cognitive Psychotherapy, 31, 193–202.
Munson, C. (2001). Handbook of clinical social work supervision (3rd ed.).
Binghamton, NY: Haworth Press.
 Psychotherapy Supervision Outcomes 255

Neufeldt, S. A., Beutler, L. E., & Banchero, R. (1997). Research on supervisor vari-
ables in psychotherapy supervision. In C. E. Watkins, Jr. (Ed.), Handbook of
psychotherapy supervision (pp. 508–524). New York: Wiley.
NURSINGtimes.net. (2010). Does clinical supervision lead to better patient outcomes
in mental health nursing? (Research report). (Originally appeared April 26,
2010). Retrieved August 20, 2010, from http://www.nursingtimes.net./nursing-
practice-clinical-research/mental-health/does-clinical-supervision-lead-to-better-
patient-outcomes-in-mental-health-nursing/5013977.article
Patton, M. J., & Kivlighan, D. M., Jr. (1997). Relevance of the supervisory alliance
to the counseling alliance and to treatment adherence in counselor training.
Journal of Counseling Psychology, 44, 108–115.
Proctor, B. (2010). Review: A randomised controlled trial of clinical supervision:
Selected findings from a novel Australian attempt to establish the evidence base
for causal relationships with quality of care and patient outcomes, as an
informed contribution to mental health nursing practice development. Journal
of Research in Nursing, 15, 169–172.
Downloaded by [69.253.48.90] at 20:17 20 June 2014

Robertson, M. H. (1995). Psychotherapy education and training. Madison, CT:

International Universities Press.
Rodenhauser, P. (1994). Toward a multidimensional model for psychotherapy super-
vision based on developmental stages. Journal of Psychotherapy Practice and
Research, 3, 1–15.
Sandell, R. (1985). Influence of supervision, therapist’s competence, and patient’s
ego level on the effects of time-limited psychotherapy. Psychotherapy and
Psychosomatics, 44, 103–109.
Scott, S. J., & Binder, J. L. (2002). The effects of manual-based training on treatment
fidelity and outcome: A review of the literature on adult individual psycho-
therapy. Psychotherapy, 39, 184–198.
Smith, M. L., Glass, G. V., & Miller, T. I. (1980). The benefits of psychotherapy.
Baltimore, MD: Johns Hopkins University Press.
Spring, B., & Walker, B. B. (Eds.). (2007). Special issue: Evidence-based practice in
clinical psychology. Journal of Clinical Psychology, 63, 607–705.
Stein, D. M., & Lambert, M. J. (1995). Graduate training in psychotherapy: Are therapy
outcomes enhanced? Journal of Consulting and Clinical Psychology, 63, 182–196.
Steinhelber, J., Patterson, V., Cliffe, K., & LeGoullon, M. (1984). An investigation of
some relationships between psychotherapy supervision and patient change.
Journal of Clinical Psychology, 40, 1346–1353.
Stoltenberg, C. D. (2009). Evidence-based clinical supervision. Living Proof Podcast
Series [Audio Podcast]. Retrieved August 26, 2010, from http://www.socialwork.
buffaloedu=podcast=episode.asp?ep ¼ 12
Stoltenberg, C. D., & McNeill, B. W. (2009). IDM supervision: An integrated develop-
mental model for supervising counselors and therapists (3rd ed.). San Francisco:
Jossey-Bass.
Thomason, T. C. (2010). The trend toward evidence-based practice and the future of
psychotherapy. American Journal of Psychotherapy, 64, 29–38.
Triantafillou, N. (1997). A solution-focused approach to mental health supervision.
Journal of Systemic Therapies, 16, 305–328.
 256 C. E. Watkins, Jr.

Vallance, K. (2004). Exploring counsellor perceptions of the impact of counselling

supervision on clients. British Journal of Guidance and Counselling, 32, 559–574.
Wampold, B. E. (2008). The great psychotherapy debate (2nd ed.). Mahwah, NJ:
Lawrence Erlbaum.
Wampold, B. E., & Holloway, E. L. (1997). Methodology, design, and evaluation in
psychotherapy supervision research. In C. E. Watkins, Jr. (Ed.), Handbook of
psychotherapy supervision (pp. 11–27). New York: Wiley.
Watkins, C. E., Jr. (Ed.). (1997). Handbook of psychotherapy supervision. New York:
Wiley.
Watkins, C. E., Jr. (1998). Psychotherapy supervision in the 21st century: Some press-
ing needs and impressing possibilities. Journal of Psychotherapy Practice and
Research, 7, 93–101.
Watkins, C. E., Jr. (2010). Psychoanalytic developmental psychology and the
supervision of psychotherapy supervisor trainees. Psychodynamic Practice,
16, 393–407.
Watkins, C. E., Jr. (2011). Psychotherapy supervision since 1909: Some friendly
Downloaded by [69.253.48.90] at 20:17 20 June 2014

observations about its first century. Journal of Contemporary Psychotherapy,

41, 57–67.
Weisz, J. R., & Kazdin, A. E. (Eds.). (2010). Evidence-based psychotherapies for
children and adolescents (2nd ed.). New York: Guilford.
Westefeld, J. S. (2009). Supervision of psychotherapy: Models, issues, and recom-
mendations. The Counseling Psychologist, 37, 296–316.
Wheeler, S., & Richards, K. (2007). The impact of clinical supervision on counselors
and therapists, their practice and their clients: A systematic review of the litera-
ture. Counselling and Psychotherapy Research, 7, 54–65.
White, E., & Winstanley, J. (2010). A randomised controlled trial of clinical super-
vision: Selected findings from a novel Australian attempt to establish the evi-
dence base for causal relationships with quality of care and patient outcomes,
as an informed contribution to mental health nursing practice development.
Journal of Research in Nursing, 15, 151–167.