Professional Documents
Culture Documents
CL Psychiatry - Performance Measures - PSM 2014
CL Psychiatry - Performance Measures - PSM 2014
discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/38034487
CITATIONS READS
3 199
3 authors, including:
Colin Harrington
Brown University
17 PUBLICATIONS 103 CITATIONS
SEE PROFILE
All content following this page was uploaded by Colin Harrington on 19 June 2015.
The user has requested enhancement of the downloaded file. All in-text references underlined in blue are added to the original document
and are linked to publications on ResearchGate, letting you access and read them immediately.
Psychosomatics 2013:54:567574 & 2013 The Academy of Psychosomatic Medicine. Published by Elsevier Inc. All rights reserved.
Mallika Lavakumar, M.D., Emily D. Gastelum, M.D., Filza Hussain, M.B.B.S., Jon Levenson, M.D.,
Ralph N. Wharton, M.D., Philip R. Muskin, M.D., Peter A. Shapiro, M.D.
Background: There is no consensus in the literature on in the literature as performance measures. Consultee
measures for evaluating the performance of general satisfaction was endorsed as a valuable performance
hospital Consultation-Liaison psychiatry services. indicator by 67.7% of them, but no satisfaction rating
Objective: The purpose of this study was to investigate what instrument was identied. The internal survey of consultees
indicators might be used to this end. Methods: We surveyed identied 11 of 16 candidate parameters as important or
United States Psychosomatic Medicine fellowship directors very important to consultee satisfaction, of which con-
(n 53) about the use of performance measures for their sultant understands the core situation and the core question
psychiatric consultation services. Results of this survey led being asked received the highest rating. Conclusions:
to the construction of a second survey, which was Consultee satisfaction is perceived as a useful global
distributed to the representatives of services calling for measure of the effectiveness of a psychiatric consult service.
psychiatric consultations at our hospital (n 21); this We elicited parameters that can be used to create a
survey sought to determine the importance of various measurement tool for consultee satisfaction with
performance parameters to overall consultee satisfaction. Consultation-Liaison services. The use of such a tool merits
Results: Sixty-three percent of responding psychiatric testing in a larger multicenter study.
consult services do not use any of the parameters identied (Psychosomatics 2013; 54:567574)
in the delivery of clinical services, and can guide settings. Outpatient mental health centers in the UK used
interventions to improve outcomes. Quality assessment consultee satisfaction along with patient satisfaction to
and performance data may be used by hospitals to inform evaluate the quality of the services they were providing.7
which services may be retained or eliminated. We sought An Australian outpatient telepsychiatry consultation
to identify quality indicators for psychiatric consultation service sought to evaluate consultee satisfaction in
services by review of the literature. addition to patient satisfaction with the goal that the
A number of metrics for C-L service quality have information would help further develop the service.8 In
been considered in the literature, including the effect of this study, surveys were distributed to both primary care
psychiatric consultation services on length of stay, physicians and nurses to collect information. An out-
hospital costs, overall medical costs, readmission rates, patient psychiatry consultation service specializing in
patient health status, and detection of alcohol and mood and anxiety disorders, also located in Australia,
substance abuse disorders.14 In the absence of a com- used consultee satisfaction, patient satisfaction, and
parison condition (i.e., either no functioning consult patient impressions of clinical improvement to collec-
service or some alternative model of psychiatric con- tively evaluate the work of their service.9
sultation), it is not possible to assess if a C-L service is Our search for published literature from other
having a benecial effect on any of these outcomes. medical specialties addressing how they evaluate the
Moreover, although these studies suggest certain specic performance of their consultation services identied only
parameters for evaluation of a psychiatric C-L service, one other study. A radiology department sought to
they do not help psychiatrists to answer the more global develop a questionnaire for measuring consultee satis-
questions of: Is my consultation service generally doing faction.10 The investigators conducted a pilot study in
a good job? and Overall, do our consultants provide a which a questionnaire was distributed to all physicians
useful service within the hospital? referring patients to their department. Respondents were
Two studies offered a wider perspective on this issue. asked to rate 29 separate parameters of their satisfaction
Karasu et al. created a survey for physicians and nursing with radiology services. The investigators concluded that
staff to gain a better understanding of which aspects of the widespread deployment of such a questionnaire could
psychiatric consultation referring clinicians found most serve to establish best practice benchmarks. Aside from
useful, and discovered that the elements most valued this study, we found no other published studies of how
depended upon the level of training and specialty of the medical consult services based in a general hospital
referring clinician.5 This study highlighted that knowledge setting evaluate the performance of their service.
of the perceptions of referring physicians (hereafter referred In the absence of clear guidance from the literature,
to as consultees) was key to understanding the utility of we determined to ask our Psychosomatic Medicine
a C-L psychiatry service. Most recently, Solomons et al. colleagues if they had a clear sense of the parameters
turned to consultees to help generate performance indicators relevant to evaluating their services, and whether they
and to identify aspects of consultation that were valuable. He used standardized measures to assess the quality and
and his colleagues conducted face-to-face interviews of 25 effectiveness of the work of their consultation services.
senior medical physicians and nursing staff in the United
Kingdom (UK) and thereby identied several measures of
performance that might be used in conducting more SURVEY NO. 1: PSYCHOSOMATIC MEDICINE
structured quality assessments, including response time, TRAINING DIRECTORS
length of stay of patients, outpatient psychiatric follow-up,
medical readmission rates, clarity of documentation, and We surveyed Psychosomatic Medicine fellowship training
satisfaction of those requesting the consultation (also here- directors in the United States to determine the parameters
after referred to as consultees).6 Both these studies they considered important in evaluating the effectiveness
suggest that consultee satisfaction is relevant to answering and quality of their psychiatry consultation services.
the question, Is my consult service doing a good job?
Consultee satisfaction can be construed as an indi- Methods, Survey No. 1
cator of consultants success in meeting the needs of those
who utilize their services and has been used as a quality Based on individual domains identied in the literature
assurance measure in other mental health treatment review described previously, a survey was developed by
consensus of the authors. The purpose of the survey was States who were included in the fellowship directory of
to understand: the Academy of Psychosomatic Medicine (n 53).
Fellowship directors were selected as the study popula-
1. if other C-L services had means of assessing their tion both for convenience and because we believed they
effectiveness; would have knowledge of the service evaluation practices
2. whether there was a consensus in the C-L community of the largest and most evaluation-minded psychiatric
about which measures are valuable indicators of C-L consultation services in the United States, so that their
service performance; and, replies would be likely to represent state-of-the-art
3. how C-L psychiatrists perceived consultee satisfaction practices. No incentives for reply were offered. Two
as a measure of C-L service performance. follow-up e-mails were sent at intervals of 1 week to all
programs to improve the response rate. The survey was
The survey contained 6 questions in multiple-choice closed on April 6, 2012.
and open-ended (free-text) formats, including free-text
responses for suggestions about measuring service qual- Results, Survey No. 1
ity and effectiveness. The survey concluded with a
request to forward any existing tools used to measure Thirty (a 56.6% response rate) of 53 survey recipients
the effectiveness of the respondents C-L service. Table 1 submitted responses. Because the responses were anon-
includes the questions in this survey. An online survey ymous, we cannot characterize respondents vs. non-
tool (SurveyMonkey) was used to create and distribute respondents.
the survey and to collect results. Anonymity of respond- The most commonly cited domains for which C-L
ents was preserved. services collect data included consultee satisfaction
In March 2012, an explanatory e-mail with a link to (26.7%), length of stay (16.7%), and patient satisfaction
the web-based survey was delivered to all Psychosomatic (13.3%). The effect of psychiatric consultations on
Medicine fellowship training directors across the United clinical costs and detection of psychiatric or substance
Question 2) Does your service use any of the following measures to assess the effectiveness of the C-L service?
J Consultee satisfaction
J Patient satisfaction
J Decrease in clinical cost
J Increase in referral to outpatient psychiatric services
J Decrease in length of stay
J Improvement in detection of psychiatric disorders and/or substance use disorders
J None
Question 3) Do you think consultee satisfaction would be a valuable indicator of the C-L services performance?
J Yes
J No
Question 7) Please share any other ideas that you may have about measuring the quality and effectiveness of C-L services
With your permission, please forward us a copy of your assessment tool to Mallika Lavakumar at ml3328@columbia.edu.
C-L Consultation-Liaison.
n
Indicates questions to which the answers are free-text responses.
FIGURE 1. Responses to Question, Does Your Service Use Any of the Following Measures to Assess the Effectiveness of the CL Service:
Decrease in Length of Stay, Referral to Outpatient Psychiatric Services, Improvement in Detection of Psychiatric and/or Alcohol
Use Disorders, Patient Satisfaction, Consultee Satisfaction, Decrease in Clinical Cost, and/or None?
disorders were less frequently assessed (3.3%). None of In summary, a majority of respondents (66.7%) felt
the programs surveyed reported rates of referral of that consultee satisfaction is a valuable measure of C-L
patients to outpatient psychiatric care as an outcome service effectiveness, only 26.7% of respondents are
measure of consultation effectiveness. A majority (63%) currently collecting information, and no existing instru-
of respondents reported not using any of these parameters ment to measure consultee satisfaction was identied
as quality indicators (Figure 1). either by literature search or by the survey.
In response, to question 3 in the survey, Do you
think consultee satisfaction is a useful measure of C-L SURVEY NO. 2: A PILOT STUDY OF CONSULTEE
service performance?, 66.7% responded with yes and SATISFACTION PARAMETERS
33.3% responded with no. Free-text comments about
this question were revealing. Many respondents endorsed The results of the rst survey highlighted the important
the value of consultee satisfaction as an indicator because role of consultee satisfaction in evaluating the effective-
C-L psychiatrists are providing a service not only to ness of a C-L service, and the absence of a suitable
patients but also to colleagues, and requests for consults assessment tool. We therefore undertook a second survey
might increase by assessing and responding to consultee inside our own institution to understand how consultees
satisfaction. Some respondents, however, were concerned rate various parameters as contributing to their overall
because poor consultee satisfaction might reect factors satisfaction with our C-L service.
not fully within the psychiatric consultants control (e.g.,
rapidity with which patients respond to psychiatric Methods, Survey No. 2
medications and time to transfer off the medical or
surgical service) or circumstances in which what is best This survey contained 3 questions, which are listed in
for the patient is inconvenient for the primary service; in Table 2. Respondents were asked to assess 16 parameters
these instances, consultee satisfaction might not be a good in determining satisfaction with the effectiveness and
measure of the consultants effectiveness. quality of psychiatric consultation services. These param-
Despite our request, no respondent provided an eters were derived from the results of the literature search
existing instrument to measure consultee satisfaction, described previously and by consensus of the authors.
suggesting that no such instrument exists. Responses were recorded and quantied using a 5-point
TABLE 2. Survey Sent to Representatives of Medical and Surgical Services at New York-Presbyterian Hospital, Columbia Campus
Question 1) For each of the following parameters, please rate how important the parameter is in determining satisfaction with the effectiveness and
quality of psychiatric consultation services using one of the following options: Very important Important Neutral Not important not at all
Important N/A
Question 2) What additional parameters determine your satisfaction with the effectiveness and quality of psychiatric consultation services provided
to your service?n
Question 3) We welcome any other comments you would like to share with us.n
n
Indicates questions to which the answers are free-text responses.
Likert scale, with scores ranging from very important Results, Survey No. 2
(5) to not at all important (1). The degree to which each
parameter was valued as important to consultee satisfac- Seventeen of 21 recipients of the survey request provided
tion was expressed as a mean value of the responses for responses, resulting in a response rate of 81%.
that parameter. The 2 other questions were free-text Generally, respondents did not consider any of the 16
responses for suggestions about other parameters that parameters not important or not at all important.
consultees thought were important for measuring Table 4 captures the value that the respondent group
satisfaction. collectively assigned to each of the 16 parameters in the
Chiefs of Service of Medicine, Surgery, and other rst question of the survey. The parameters are tabulated
specialties across New York-Presbyterian Hospital, in order of decreasing value with the most important
Columbia Campus, known to be high-volume utilizers parameter being listed rst and the least important
of the psychiatric consultation service were contacted parameter being listed last. Mean scores of 4 or higher
via e-mail to aid in identifying optimal survey respond- indicate that the respondent group rated the parameter as
ents from their service (i.e., people actively involved in either important or very important, so these scores
clinical care). The survey respondents included attend- warrant special attention. Items rated as important to
ings on teaching services, hospitalists, and chief resi- very important were the following: Consultant under-
dents. The identied service representatives (n 21) stands the situation and the core question being asked,
received an e-mail message explaining our purpose, which received the highest rating, Consultant has
requesting their participation, and providing a link to the practical and helpful management suggestions for the
web-based survey. Table 3 describes the target group of medical staff, Consultant has practical and helpful
respondents. Reminders were sent via e-mail and phone management suggestions for the nursing staff, Con-
calls at intervals of 1 week to all prospective respondents sultation occurs within 24 hours of request, Consultant
so as to improve response rate. The results were quickly manages patients behavioral problems, Pro-
collected anonymously between May 31, 2012 and July viding diagnostic clarication that aided in manage-
11, 2012. ment, Communicating verbally with the primary
TABLE 3. Breakdown of Recipients of Survey at New York- 24 hours of request as being either important or very
Presbyterian Hospital, Columbia Campus important and 70% of respondents rated Consultation
Number of people occurs within 4 hours of request as being either
contacted important or very important, highlighting the role
Service (N 21)
of timeliness in consultee satisfaction.
Internal medicine teaching service 4 Free-text responses demonstrated the importance to
Internal medicine hospitalist service 1 the consulting teams of having psychiatric consultants
Neurology 2
present and accessible. One respondent requested twice
Physical medicine and rehabilitation medicine 1
HIV 1 daily C-L rounds on patients in restraints or on constant
Gynecological oncology 1 observation and for routine psychiatric evaluations to be
General surgery 1 provided on weekends and holidays. A few others
Obstetrics and gynecology 3
Transplant (lung, liver, and heart) 3 requested easy availability for questions by phone and
Minimally invasive surgery 1 expressed appreciation for the presence of a psychiatrist at
Surgical ICU and cardiothoracic ICU 2 family meetings. The presence of a psychiatric consultant
Orthopedic surgery 1
on the primary teams rounds and identication or
ICU intensive care unit.
availability of an attending psychiatrist were all suggested
as additional points to include in a satisfaction measure.
TABLE 4. Responses of Service Representatives to the Question, For Each of the Following Parameters, Please Rate How Important the
Parameter is in Determining Satisfaction with the Effectiveness and Quality of Psychiatric Consultation Services Using One of
the Following Options: Very Important (5), Important (4), Neutral (3), Not Important (2) or Not at All Important (1). The
Survey had 17 Respondents. Numerical Responses to Each Question Were Added and Divided by 17 to Yield the Mean
Consultant understands the situation and the core question being asked 4.94 0.24
Consultant has practical and helpful management suggestions for medical staff 4.76 0.42
Consultant has practical and helpful management suggestions for nursing staff 4.71 0.46
Consultation occurs within 24 hours of request 4.65 0.76
Consultant quickly manages patients behavioral problems 4.53 0.78
Consultant provides diagnostic clarication leading to management and disposition planning 4.53 0.78
Consultant communicates verbally with the medical team in addition to placing notes in the chart 4.53 0.61
Consultant does not increase length of stay 4.35 0.90
Consultant facilitates transfer to inpatient psychiatry service 4.35 0.68
Consultant reduces length of stay 4.24 0.73
After the initial consult, consultant follows up for the duration of the admission 4.00 0.69
Consultant arranges disposition that removes problem patient from my service 3.94 0.87
Consultation occurs within 4 hours of request 3.88 0.83
Consultant is involved in educational activities with the medical staff 3.82 0.71
Consultant helps manage substance intoxication and/or withdrawal 3.76 1.11
Consultant is involved in educations activities with the nursing staff 3.53 0.61
for inpatient psychiatric units since 2008.11 General In some respects, our survey study was similar to that by
hospital psychiatric C-L services are not guided by any Solomons et al. (in the UK) who attempted to understand
performance measures even though they operate in a what consultees valued.6 They interviewed 25 referrers,
climate of great accountability. Directors of C-L services including nursing staff and medical consultants. Response
of 2 hospitals in Rhode Island have acknowledged the times, follow-ups and readmissions, quality of documenta-
need for such measures and have grappled with determin- tion, satisfaction of patients and clinicians, and educational
ing what quality measures are the most relevant to activities were found to be important in these interviews.
measuring the effectiveness of a service.12 Most of these parameters were highly valued in our survey.
Several barriers hinder the development of a com- The UK study described consultee satisfaction in qualitative
prehensive quality assessment tool for hospital-based terms, whereas we sought to dene it in quantitative terms.
psychiatric consultations. The spectrum of clinical sce- Karasu et al., in the 1970s, addressed the question of what
narios addressed by psychiatric consultants is vast. There aspects of psychiatric consultation are valued by physi-
is signicant clinical heterogeneity in the problems that cians.5 That study focused more on understanding how
entail our involvement, and consequently, the specic physicians differ in their expectations of consult services
outcomes that would dene an effective consultation vary based on their level of experience and their medical or
from one consultation to another. Psychiatric consultants surgical specialty, rather than on developing a tool which
are often called upon to assist with challenging ethical and might have broad applicability in general hospitals. It did,
systems-level problems, solutions to which are not easily however, employ some of the same parameters in its
quantied. Many people are involved in psychiatric questionnaire that we used in our survey of clinical services
consultations, including patients, their families, primary at New York-Presbyterian Hospital.
treating physicians, nurses, and social workers, and the It comes as no surprise that understanding the
agendas and needs of the many parties to the consultation situation and the core question being asked was ranked
may differ. It is unlikely that a single measurement tool the highest in our survey of consultees. This parameter is
can capture input from each of these parties and reconcile not always clear when a psychiatric consultation is
conicting viewpoints. Thus it is challenging to create a requested. The interplay of abstract ideas, transference,
universally applicable tool that measures consult service and psychic conicts make formulation of accurate
effectiveness in all situations. questions challenging. Often the consultant has to help
There has been no systematic exploration of percep- the consultee formulate his or her question after a brief
tions within the C-L community regarding the use of conversation about the case. For example, consults for
consultee satisfaction as a performance measure. Our Help us manage a patient with a personality disorder
survey of 53 fellowship directors from the United States is who is sabotaging treatment might be cloaked as does
the rst organized attempt to understand how C-L the patient have the capacity to leave AMA?13 In such a
psychiatrists view consultee satisfaction as a performance situation if the consultant simply answers the question
measure. A majority (66.7%) were of the opinion that asked and declares that the patient has the capacity to
consultee satisfaction was a valuable measure of C-L leave against advice, it does the patient a disservice and
service performance. Of the total respondents, 26.7% may leave the primary team feeling that the goals of care
appear to already be employing consultee satisfaction in were not accomplished. The survey highlights that
formal or informal ways as a measure of C-L service successful psychiatric consultants are those who are able
effectiveness. Consultee satisfaction has face validity as a to understand both the explicit and implicit questions
relevant performance measure of C-L service effective- posed by the team. Similarly, the other ndings of the
ness. A review of the literature and our survey indicates survey on consultee opinion systematically document and
that there are no operational denitions of satisfaction. validate what might appear to be intuitively known to
Our internal survey of consultees is novel in that it is clinicians, and provide a means to measure these dimen-
the rst attempt to dene consultee satisfaction in sions of consultation quality.
quantitative terms. We elicited parameters that are There are limitations to using consultee satisfaction
valuable to general hospital medical and surgical services, as a measurement of consult service effectiveness. In
and that can be used in a measurement tool for consultee some situations doing what is in the best interests of the
satisfaction and thereby, as one measure of C-L service patient may result in disappointment or frustration for the
effectiveness. consultee. Occasionally, a troublesome patient is not
medically stable enough to be transferred to a psychiatric other settings cannot be known without additional studies
unit, and consultees insist on this intervention under- using more sites and larger samples. Another potential
standably but prematurely. Sometimes referring services limitation is the possibility of a social desirability
are unyielding in their belief that an inpatient psychiatric response bias, despite the anonymity of responses, arising
hospitalization is clinically necessary, when in the con- because we surveyed colleagues who are personally
sultants judgment it might be futile. Patients may known to us and who may have been inuenced by
demonstrate problems with behavior or judgment as a previous discussions with us. This limitation can also be
result of chronic conditions, such as mental retardation or addressed through a larger, multicenter study.
dementia, that an inpatient psychiatric hospitalization is We think consultee satisfaction is a useful global
unlikely to resolve. Even when the patient is medically measure of consult service effectiveness, and we have
stable for transfer and the referral to a psychiatric unit is identied a discrete and manageably sized list of param-
appropriate, a shortage of available beds might delay eters that can be included in a consultee satisfaction
transfer, which dissatises the consultee but is beyond the measure. We do not believe that consultee satisfaction
control of the consultant. alone is a perfect index of consult service effectiveness.
Our study has several limitations. We rst surveyed We envision consultee satisfaction being used in con-
fellowship training directors, and their programs may not junction with others measures of service performance,
reect service evaluation practices in institutions without such as nursing satisfaction, patient satisfaction, and
Psychosomatic Medicine fellowships. We suspect, how- additional process and outcomes measures. We also note
ever, that this limitation, if anything, introduces a that this work has potential for generalizability to hospital
conservative bias to our nding that the practice of consultation services of other medical specialties striving
C-L service performance evaluation is underdeveloped, to develop performance measures. If the results of this
as training programs tend to occur in larger and better- survey hold true in a multicenter study, they can be used
established C-L services. If even these services lack by psychiatric C-L services as a self-assessment tool and
performance measures, it is likely that less established by hospital regulatory bodies as benchmarks.
services also lack them. Our survey of consultees
to establish important parameters of consultee satisfaction Disclosure: The authors disclosed no proprietary or
was limited to a small sample at a single teaching hospital commercial interest in any product mentioned or concept
in New York City. The generalizability of its ndings to discussed in this article.
References
1. Levitan SJ, Kornfeld DS: A study of liaison psychiatry 7. Milne D, Gibson L: Quality assurance in the voluntary sector.
effectiveness: clinical and cost benets. Am J Psychiatry Int J Health Care Qual Assur 1994; 7:1619
1981; 136:790793 8. Clarke PH: A referrer and patient evaluation of a telepsychiatry
2. Levenson JL, Hamer RM, Rossiter LF: A randomized controlled consultation-liaison service in South Australia. J Telemed
study of psychiatric consultation guided by screening in general Telecare 1997; 3:1214
medical inpatients. Am J Psychiatry 1992; 149:631637 9. Lewis R, Musella E, Berk M, Dodd S, McKenzie H, Hyland M:
3. Desan PH, Zimbrean PC, Weinstein AJ, Bozzo JE, Sledge WH: An audit of clinical outcomes and client and referrer satisfaction
Proactive psychiatric consultation services reduce length of with a Mood and Anxiety Disorders Unit. J Eval Clin Pract
stay for admissions to an inpatient medical team. Psychoso- 2004; 10:549552
matics 2011; 52:513520 10. Kubik-Huch RA, Rexroth M, Porst R, Durselen L, Otto R,
4. Diehl A, Nakovics HB, Reinhard I, Kiefer F, Mann K: Szucs T: Referrer satisfaction as a quality criterion: devel-
Consultation-liaison psychiatry in general hospitals: improve- oping an questionnaire for measuring the quality of services
ment in physicians detection rates of alcohol use disorders. provided by a radiology department. Rofo 2005; 177:
Psychosomatics 2009; 50:599604 429434
5. Karasu TB, Plutchik R, Conte H, Siegel B, Steinmuller R, 11. Available from: http://www.jointcommission.org/core_measure_
Rosenbaum M: What do physicians want from a psychiatric sets.aspx. 2012.
consultation service? Compr Psychiatry 1977; 18:7381 12. Goldberg RJ, Burock J, Harrington CJ: Quality indicators in
6. Solomons LC, Thachil A, Burgess C, et al: Quality of consultation-liaison psychiatry. Psychosomatics 2009; 50:550
psychiatric care in the general hospital: referrer perceptions 13. Kornfeld DS, Muskin PR, Tahil FA: Psychiatric evaluation of
of an inpatient liaison psychiatry service. Gen Hosp Psychiatry mental capacity in the general hospital: a signicant teaching
2011; 33:260266 opportunity. Psychosomatics 2009; 50:468473