Received 12/29/14

Revised 11/27/15
Accepted 12/15/15
DOI: 10.1002/jmcd.12047

Validation of a
Standardized Multiple-Choice
Multicultural Competence Test:
Implications for Training,
Assessment, and Practice
Angela R. Gillem, Eleonora Bartoli, Kristin N. Bertsch,
Maureen A. McCarthy, Kerra Constant, Sheila Marrero-Meisky,
Steven J. Robbins, and Scarlett Bellamy
The Multicultural Counseling and Psychotherapy Test (MCPT), a measure of
multicultural counseling competence (MCC), was validated in 2 phases. In Phase
1, the authors administered 451 test items derived from multicultural guidelines
in counseling and psychology to 32 multicultural experts and 30 nonexperts. In
Phase 2, the authors administered the top 50 discriminative items to licensed
mental health professionals (N = 227) and compared MCPT scores to external
indicators of MCC. Evidence was found for the construct validity and internal
consistency reliability of the MCPT.
Keywords: multicultural competence, counselor training, assessment
El Test de Consejería Multicultural y Psicoterapia (MCPT, por sus siglas en inglés),
una medida de la competencia en consejería multicultural (MCC, por sus siglas
en inglés) fue validado en dos fases. En la Fase 1, los autores administraron
451 elementos del test derivados de las pautas multiculturales en consejería
y psicología a 32 expertos multiculturales y 30 no expertos. En la Fase 2, los
autores administraron los 50 elementos más discriminativos a profesionales
licenciados de la salud mental (N = 227) y compararon los puntajes del MCPT
con indicadores externos de MCC. Se hallaron evidencias de la validez del
constructo y la fiabilidad de la consistencia interna del MCPT.
Palabras clave: competencia multicultural, formación de consejeros, evaluación

M
ulticultural competence (MCC) involves being able to use counsel-
ing skills in a way that is relevant to client experiences, having basic
knowledge of cultural norms, being able to empathize with a client’s

Angela R. Gillem, Eleonora Bartoli, Kristin N. Bertsch, Kerra Constant, and Sheila Marrero-Meisky, Graduate
Program in Counseling, and Steven J. Robbins, Department of Psychology, Arcadia University; Maureen A.
McCarthy, Department of Psychology, Kennesaw State University; Scarlett Bellamy, Department of Biostatistics
and Epidemiology, University of Pennsylvania. Kristin N. Bertsch is now at Counseling Center, Drexel Univer-
sity. Kerra Constant is now at Stepping Stones Partial Hospitalization Program, Philadelphia, Pennsylvania.
Sheila Marrero-Meisky is now at Creative Health Services, Spring City, Pennsylvania. The authors acknowledge
Kristen English for her assistance in editing the Multicultural Counseling and Psychotherapy Test (MCPT),
Toby Ayash for being instrumental in facilitating the literature review for this article, Melony Parkhurst for
her assistance with data management, and Jeffray Feliciana for his work on checking citation–reference list
correspondence. Monica Williams was coauthor of the items on the MCPT with Angela R. Gillem and Eleonora
Bartoli. Correspondence concerning this article should be addressed to Angela R. Gillem, Graduate Program in
Counseling, Arcadia University, 450 South Easton Road, Glenside, PA 19038 (e-mail: gillema@arcadia.edu).
© 2016 American Counseling Association. All rights reserved.

Journal of Multicultural Counseling and Development • July 2016 • Vol. 44 209

cultural perspective, and being aware of oneself and one’s attitudes as cultur-
ally grounded (Vasquez, 2010). Mental health professionals recognized the
need for MCC as early as 1973, but the counseling profession did not consider
integrating cultural content into graduate curricula until approximately 2
decades later (Ridley & Kleiner, 2003).
Research has suggested that traditional treatment modalities and the coun-
selors who implement them may carry biases that may not be responsive to
diverse racial/ethnic communities (Kim, Cartwright, Asay, & D’Andrea, 2003;
see also Ponterotto, Casas, Suzuki, & Alexander, 2010). Thus, training cultur-
ally competent counselors is essential to ensure that the needs of culturally
diverse clientele are met. However, despite the increasing diversity of the
United States, mental health professionals who are sufficiently trained to meet
these needs are scarce (Penn & Post, 2012). Also, the counseling profession
increasingly relies on evidence-based treatments. However, it is difficult to
conduct research on the efficacy of treatment modalities for diverse racial/
ethnic communities (D’Andrea & Heckman, 2008; Dunn, Smith, & Mon-
toya, 2006; S. Sue, 2003) without being able to measure MCC as a covariate
(D’Andrea & Heckman, 2008; Fuertes, Bartolomeo, & Nichols, 2001). Treat-
ment efficacy studies need to take into account the MCC of the counselor
to ensure the clarity of the impact of a given treatment. In a recent study by
Owen, Tao, Leach, and Rodolfa (2011), for example, clients’ perceptions of
counselors’ multicultural orientation was associated with positive treatment
outcomes, with the relationship being potentially mediated by the develop-
ment of a strong therapeutic alliance.
Although the field is working on theoretical models that define and opera-
tionalize MCC, counselors still need standardized assessments to refine and
validate them (Cartwright, Daniels, & Zhang, 2008; Ponterotto, Gretchen, Utsey,
Rieger, & Austin, 2002). The gap between theory and research is particularly
problematic when it comes to assessing the effectiveness of multicultural
training programs (Worthington, Soth-McNett, & Moreno, 2007); counselors
have few ways to assess whether students and professionals are adequately
trained in multicultural counseling (Cartwright et al., 2008; Kim et al., 2003;
Manese, Wu, & Nepomuceno, 2001). Self-report measures often fall short
of accomplishing this (Dunning, Heath, & Suls, 2004); standardized tests of
MCC, such as our Multicultural Counseling and Psychotherapy Test (MCPT),
are sorely needed. Instead of asking counselors to report whether they think
they exhibit MCC in given areas, the realm of self-report measures (e.g., “I
am aware of specific skills that apply to clients of color”), the MCPT assesses
whether counselors actually know and can identify appropriate skills to use
with clients of color (e.g., “You are working with a client of color . . . Which
of the following four skills would be most appropriate to employ?” where only
one of the multiple-choice options offered is correct); the test taker passes
or fails a given test item according to an external predetermined standard.
Like the National Counselor Examination for Licensure and Certification,

210 Journal of Multicultural Counseling and Development • July 2016 • Vol. 44

the MCPT is a standardized “multiple-choice examination designed to assess
knowledge, skills and abilities determined to be important for providing
effective [multicultural] counseling services” (National Board for Certified
Counselors, 2016, para. 1).

current measures of mcc

The following are the most commonly used MCC scales: the Multicultural
Counseling Knowledge and Awareness Scale (MCKAS; Ponterotto et al.,
2002; Ponterotto & Potere, 2003), the Multicultural Counseling Inventory
(MCI; Sodowsky, Taffe, Gutkin, & Wise, 1994), the Multicultural Awareness
Knowledge Skills Survey–Counselor Edition–Revised (MAKSS-CE-R; Kim et al.,
2003), the Cross Cultural Counseling Inventory–Revised (CCCI-R; LaFrom-
boise, Coleman, & Hernandez, 1991), and the California Brief Multicultural
Competence Scale (CBMCS; Gamst et al., 2004). The MCKAS is a 32-item
instrument in which individuals rate themselves on multicultural awareness
and knowledge. The MCI, a 40-item self-report instrument, consists of four
subscales: Multicultural Counseling Skills, Multicultural Awareness, Multicul-
tural Counseling Relationship, and Multicultural Counseling Knowledge. The
33-item MAKSS-CE-R is a self-report survey designed to assess the effect of
training interventions on students’ acquisition of MCC. Originally intended as
an observer-rated method of assessment, the 20-item CCCI-R was later modi-
fied as a self-report measure by Ladany, Inman, Constantine, and Hofheinz
(1997). Finally, the CBMCS is a 21-item self-report measure that incorporated
items from the aforementioned scales.
Although widely used, self-report measures are affected by social desir-
ability; individuals tend to respond in perceived socially acceptable ways,
rather than in ways that reflect how they actually feel or behave (Constantine
& Ladany, 2000). Thus, they may assess anticipated responses or perceived
multicultural self-efficacy rather than true competence (Constantine &
Ladany, 2000; Pope-Davis & Dings, 1995); individuals may feel that they are
able to provide culturally competent counseling because they believe that
they possess multicultural knowledge, awareness, and skills even if they
do not (Constantine, Gloria, & Ladany, 2002). This tendency has resulted
in the need to control for social desirability in MCC research (Liu, Sheu, &
Williams, 2004; Neville, Spanierman, & Doan, 2006).
Lab research in social psychology has made this point as well. Across a
variety of domains, including health care, school outcomes, and job per-
formance, people have been found to consistently overrate their levels of
competence and accomplishment (see Dunning et al., 2004, for an extensive
review). Research has also indicated that people who perform poorly in a
particular domain are similarly deficient at identifying that their performance
is lacking (see Dunning, Johnson, Ehrlinger, & Kruger, 2003, for a review);
people can strongly believe that they know something that in fact they do

Journal of Multicultural Counseling and Development • July 2016 • Vol. 44 211

not know (e.g., Cartwright et al., 2008). For these reasons, measures of MCC
that depend on self-assessment may not provide accurate results.
The CCCI-R and multicultural case conceptualization are two promising
approaches to MCC assessment. The original observer-rated CCCI-R is the
only measure that assesses actual performance through observation. Con-
stantine (2001) found that a significant portion of the variance in CCCI-R
observer ratings was accounted for by the number of multicultural courses
taken by her participants, which suggests that this is a promising approach
to measuring MCC. Unfortunately, several researchers have found that self-
reported MCC does not strongly correlate with observer-rated MCC (Con-
stantine, 2001). Thus, as suggested earlier, “self-reported and observer-rated
multicultural counseling competence may be two theoretically divergent
constructs” (Constantine, 2001, p. 460).
Multicultural case conceptualization relies on a counselor’s ability to
examine a client’s presenting problem in the context of cultural factors
(Constantine & Ladany, 2000) and to recommend treatment on the basis of
this understanding (Neufeldt et al., 2006). Scores on MCC self-report assess-
ments have been unrelated to, and usually higher than, multicultural case
conceptualization scores even when social desirability is controlled, which
suggests that clinicians overestimate their MCC and that multicultural case
conceptualization is not as affected by social desirability. In fact, Constantine
and Ladany (2000) found that of four self-report MCC scales administered,
none accounted for a significant amount of the variance in multicultural
case conceptualization ability.
Although there are many benefits to multicultural case conceptualization,
there are a few limitations. In some studies (Constantine & Ladany, 2000;
Ladany et al., 1997), multicultural case conceptualization consisted of using
a single vignette. Although interrater agreement was high, it is possible that
reliability could change if participants were given multiple vignettes. This
is a common problem of random response error (Constantine & Ladany,
2000): Ratings could have been similar because of chance. Constantine and
Ladany (2000) suggested that, to assess MCC across a variety of settings,
researchers should use direct means to simulate counseling situations (e.g.,
videotape, direct observation). Although these direct means would increase
the objectivity of measuring MCC, they are time-consuming and costly; many
training programs and agencies do not have the resources to actualize this
recommendation.
Thus, it continues to be difficult to make substantial progress in evidence-
based MCC research and training until researchers are able to produce
simple, standardized measures of MCC that accurately reflect competence
(S. Sue, 2003). In the following paragraphs, we describe a two-phase process
by which we developed and validated such a measure. The MCPT addresses
concerns about social desirability, administration time and cost, and diver-
sity of stimulus content. Compared with self-report measures, test-based

212 Journal of Multicultural Counseling and Development • July 2016 • Vol. 44

assessment of MCC provides a more accurate measure of individuals’
competence because it is demand-free, is unbiased, and does not involve
self-evaluation of competence.

method
Measures
MCPT. The Phase 1 MCPT consisted of 451 multiple-choice and true–false
items about counseling diverse racial/ethnic groups. The test authors origi-
nally created 600 test items using professional guidelines and commonly used
graduate-level multicultural counseling texts (i.e., American Psychological
Association, 2003; Arredondo et al., 1996; Kivel, 2002; Ridley, 2005; D. W.
Sue & Sue, 2003). Each of the three test authors coded the texts indepen-
dently using the tripartite model of counseling competence: knowledge,
self-awareness, and skills. They then met to discuss the codes to consensus.
For each multicultural concept, they independently created several multiple-
choice and true–false items, each with only one keyable (correct) response,
which they discussed and revised to consensus. They attempted to balance
the items for inclusiveness of gender and race/ethnicity. Two outside con-
sultants edited the 600 items, which resulted in the 451-item version of the
MCPT used in Phase 1.
We assessed a wide array of multicultural topics. Knowledge items ad-
dressed topics such as key research findings, racial identity development,
cultural group norms and histories, culturally respectful language, and
concepts such as stereotype threat (i.e., anxiety about confirming negative
stereotypes about one’s social group; Steele, 1997) and microaggressions
(i.e., ordinary behaviors or speech that communicate stereotypical, demean-
ing, or invalidating messages to marginalized group members; D. W. Sue,
2010). Self-awareness items addressed such topics as awareness of personal
biases and values, limits of competence regarding cultural issues, color
blindness (i.e., negating or ignoring race, ethnicity, or culture as a factor in
understanding people’s experiences), and the impact of one’s own cultural
background on attitudes toward clients. Skill items included case examples
in which participants were asked to choose the best course of action when
multicultural assessment or multiculturally modified counseling skills were
essential (e.g., including extended family members in the counseling process
for cultural reasons, considering both religious or cultural background and
psychopathology when clients report unusual perceptual or sensory expe-
riences). We also included skill items that addressed social advocacy and
social justice (see Hays, 2008).
Demographic survey. The demographic survey used in both phases of the
study included items related to race/ethnicity, national origin, and multicul-
tural counseling training and experience, which established the diversity of

Journal of Multicultural Counseling and Development • July 2016 • Vol. 44 213

our sample and the level of their expertise based on their degree of multi-
cultural training and education. Research has suggested that multicultural
training predicts cultural knowledge and the ability to establish a productive
therapeutic alliance with clients of color (Castillo, Brossart, Reyes, Conoley,
& Phoummarath, 2007; Spanierman, Poteat, Wang, & Oh, 2008; Toporek &
Pope-Davis, 2005; Tummala-Narra, Singer, Li, Esposito, & Ash, 2012). This
finding supports our choice to validate the MCPT against training/educa-
tional indicators of MCC. Both phases of this study were approved by the
Arcadia University Institutional Review Board.

Phase 1
Participants. In Phase 1, we identified which items of the 451-item ver-
sion of the MCPT best discriminated between multicultural experts and
nonexperts. We recruited a national sample of 32 experts from among
professional colleagues of the first two authors and through snowballing,
a sampling method used to emulate as closely as possible a random sample
when the population to be sampled is not readily available (Goodman,
1961). The expert sample included professionals who taught or published
in the multicultural field, regularly attended multicultural conferences and
workshops, and incorporated multicultural principles in their practices.
We recruited 30 nonexperts from graduate programs in counseling at four
universities. The nonexpert sample included last-semester undergraduate
seniors pursuing counselor training upon graduation. Only students who
had never taken a multicultural counseling course were included. See Table
1 for sample characteristics.
Procedure. Participants filled out the demographic survey and answered
all 451 MCPT items. (The small sample size did not allow for assess-
ing order effects.) We expected them to take approximately 6 hours to
complete the MCPT because of its length; to minimize fatigue effects,
we instructed them to answer no more than 80 items in a sitting. To
assess the existing competence of participants and to maintain the
security of the test, we instructed them to refrain from discussing it
with others or consulting any resources. Participation was confidential;
their signed consent forms were separated from their assessments and
filed separately.
Results. We used nonparametric Wilcoxon methods to compare experts and
nonexperts on continuous measures (i.e., age, income, counseling experience,
academic experience, and MCPT score) and chi-square tests to compare the
distribution of responses in these two groups for categorical measures (i.e.,
socioeconomic status, gender, country of origin, disability, highest earned
degree, race/ethnicity, expertise, and licensure). We used a receiver-operating
characteristic (ROC) curve approach and reported the corresponding c statistic
(i.e., the area under the ROC curve) as our measure of discriminative power
between experts and nonexperts on the basis of participants’ scores on the MCPT.

214 Journal of Multicultural Counseling and Development • July 2016 • Vol. 44

 Table 1
Phase 1 Expert and Nonexpert Demographic and
Professional Characteristics
Expertsa Nonexpertsb
Variable n % n %
Gender
Female 26 81 28 93
Male 6 19 2 7
Has a disability 0 0 7 23
Race/ethnicity
Black/African American 2 6 3 10
Asian/Asian American 8 25 2 7
Hispanic/Latina(o) 4 13 0 0
White/European American 16 50 24 80
Multiracial/multicultural 2 6 1 3
Country of origin
United States 24 75 27 90
Other 8 25 3 10
Socioeconomic status
Middle class 22 69 21 70
Upper middle class 9 28 4 13
Upper class 0 0 1 3
Working class 1 3 4 13
Highest earned degree
Master’s degree 6 19 1 3
Doctoral degree 26 81 0 0
Licensed 23 72 0 0
Note. Percentages may not total 100 because of rounding.
a
For experts, n = 32; median age = 45 years; median income = $90,000; median years of coun-
seling experience = 13.5. bFor nonexperts, n = 30; median age = 23 years; median income =
$40,000; median years of counseling experience = 1.

We conducted a two-step item reduction process to explore how a subset

of the items would discriminate between experts and nonexperts, with the
aim of yielding a shorter, more user-friendly measure. First, we computed
univariate analyses predicting expert/nonexpert status as a function of
whether participants correctly responded to each item. We considered an
item significantly associated with expert/nonexpert status if the resulting p
value from the Fisher’s exact test was less than .05 for a two-sided hypoth-
esis test of association between expert/nonexpert status and frequency of
correct responses for that particular item. There were 191 items that met the
criteria. In the second step, we used the “score” automated model selection
procedure in the LOGISTIC procedure in SAS to find the best subset of items
predicting expert/nonexpert status. The best subset of items is the one that
optimizes the likelihood score statistic for all possible models combining
the 191 candidate items. The optimization uses branch and bound statistical
methods (Furnival & Wilson, 1974; Hand, 1981). The resulting best model
contained 49 of the 191 items.

Journal of Multicultural Counseling and Development • July 2016 • Vol. 44 215

 Experts and nonexperts were similar in terms of self-reported socioeconomic
status, country of origin, gender, and race/ethnicity. Experts were older
than nonexperts (medians of 45 and 23 years, respectively; p < .0001) and
reported more earned income (medians of $90,000 and $40,000, respectively;
p = .0003). More experts self-identified as having intermediate or advanced
levels of expertise compared with nonexperts (94% vs. 37%, respectively; p <
.0001), and more experts were licensed (72% vs. 0%, respectively; p < .0001).
Experts had more years of counseling experience (medians of 13.5 and 1,
respectively; p < .0001) and more academic experience (i.e., a composite score
calculated by adding the number of multicultural conferences and workshops
and graduate-level courses attended and taught, multicultural books and
articles written and read, and professional multicultural presentations done;
medians of 73 and 3.5, respectively; p < .0001). Experts’ median scores on
the original 451-item MCPT measure were significantly higher than those
of nonexperts (0.80 vs. 0.59, respectively; p < .0001).
The c statistic for the 451-item test was equal to .921, which indicated very
good discriminative ability to predict expert/nonexpert status (the higher the
c statistic, the better its discriminative ability; see Figure 1). The c statistic was

1.0 — 
Percentage Correctly Identified “Expert”

0.8 —

0.6 —

0.4 —

0.2 —
   = 191 items (c statistic = .967)
   = 49 items (c statistic = .981)

0.0 —
—
—

—
—
—

0.0 0.2 0.4 0.6 0.8 1.0

Percentage Incorrectly Identified “Nonexpert”

Figure 1
Multicultural Counseling and Psychotherapy Test (MCPT)
Receiver-Operating Characteristic Curve Analysis From Phase 1
Note. The 451-item version of the MCPT was not plotted (c statistic = .921).

216 Journal of Multicultural Counseling and Development • July 2016 • Vol. 44

.967 for the 191-item version and .981 for the 49-item version, thus indicating
no loss, but rather a slight gain, in discriminative ability in both instruments
with far fewer items. We also computed Hosmer–Lemeshow goodness-of-fit
statistics to assess overall model fit for the 451-, 191- and 49-item versions of
the measure. These statistics and corresponding p values were χ2(8) = 16.00, p
= .0424; χ2(8) = 5.50, p = .7029; and χ2(8) = 2.77, p = .9480, respectively. These
results indicated a significant lack of fit for the 451-item measure and no
lack of fit for the 191- and 49-item measures. The final 49 items addressed
a similar array of topics as the 451-item test, including awareness of biases,
color blindness, sensitivity toward religious beliefs, social justice and ad-
vocacy, cultural histories, the ability to identify the impact of oppression,
and racial identity.
Discussion. Phase 1 data support the validity of the MCPT. However, the expert
and nonexpert groups were different in several ways that may be confounded
with their expert/nonexpert status, including age, education, and counseling
experience. Also, it is likely that the final 49 items did so well because they
were chosen after the fact to fit with this set of data; that is, we deliberately
picked the items that discriminated between these experts and nonexperts.
Therefore, in Phase 2, we sought validation of the final 49 items as a measure
of MCC by comparing training/educational and professional indicators of
MCC with the MCPT scores of licensed mental health professionals.

Phase 2
Participants. We recruited, through professional electronic mailing lists, a
national sample of 353 master’s- and doctoral-level licensed counselors,
psychologists, social workers, and marriage and family therapists. We sent
an introductory e-mail that directed them to the research materials on Sur-
veyMonkey. Although 353 individuals responded, the final yield was 227
because of incomplete data. See Table 2 for Phase 2 sample characteristics.
Procedure. We described the study and informed participants that proceed-
ing with the completion of the assessments constituted consent. They com-
pleted the demographic survey and answered 50 MCPT items. (We added
the next highest discriminating item from Phase 1 to round out the number
of test items from 49 to 50.) As in Phase 1, participants were instructed not
to discuss the test or consult any resources. Participation was anonymous.
Results. We conducted an item analysis for the 50-item dichotomously scored
MCPT. Simple frequencies were calculated for each potential answer to the
50 items. We found that four items produced negative item discrimination
indices and item–total correlations; those items had two potentially keyable
(correct) answers. Therefore, the final version of the instrument had only 46
items with a maximum summary score of 46. Scores ranged from 18 to 45 (M
= 36.45, SD = 5.91). The average score was higher than the 50th percentile;
this negatively skewed distribution likely reflects the nonrandom selection
of the sample.

Journal of Multicultural Counseling and Development • July 2016 • Vol. 44 217

 Table 2
Phase 2 Sample Demographic and Professional Characteristics
Variable n %
Gender
Female 179 78.9
Male 47 20.7
Transgender 1 0.4
Has a disability 27 11.9
Race/ethnicitya
Black/African American 22 9.7
Asian/Asian American 16 7.0
Hispanic/Latina(o) 14 6.2
White/European American 178 78.4
Jewish 9 4.0
Native American/American Indian/Alaska Native 5 2.2
Multiracial/multicultural 10 4.4
Country of origin
United States 202 89.0
Other 25 11.0
Socioeconomic statusb
Middle class 128 56.4
Upper middle class 74 32.6
Upper class 2 0.9
Working class 18 7.9
Poor/lower income 3 1.3
Highest earned degree
Master’s degree 79 34.8
Doctoral degree 148 65.2
Licensed 192 84.6
Note. N = 227. Median age was 45 years. Median income was $103,000. Median years of
clinical experience was 12.5.
a
Some participants selected more than one race/ethnicity, and each selection was counted.
b
Two participants did not indicate their socioeconomic status.

Initial calculation of the Cronbach’s alpha produced strong evidence of

internal consistency reliability (α = .83; Pedhazur & Schmelkin, 1991). Be-
cause there is no gold-standard measure of MCC against which to validate
our scale, we validated the MCPT against activities that form the basis of an
individual’s professional reputation and that are commonly used measures
of expertise in academic and clinical hiring and in promotion and tenure
decisions. We expected that individuals who had published more research
on MCC, given more talks, and taught more courses would demonstrate a
higher level of knowledge, because these activities reflect the degree and
intentional nature of both exposure to multicultural material and multicul-
tural authorship and productivity. As shown in Table 3, MCPT scores were
positively and significantly correlated with six of the seven expertise areas
we measured. Specifically, MCPT scores were significantly correlated with
number of multicultural presentations (r = .30, p < .0001), number of mul-
ticultural texts read (r = .30, p < .0001), number of multicultural workshops
attended (r = .26, p < .0001), number of multicultural graduate courses taught

218 Journal of Multicultural Counseling and Development • July 2016 • Vol. 44

 Table 3
Correlations Between Multicultural Counseling and Psychotherapy
Test Scores and Multicultural and Professional Experiences
Professional Experience r
Number of multicultural conferences attended .18*
Number of multicultural courses taken .13
Number of multicultural workshops attended .26***
Number of multicultural texts read .30***
Number of multicultural publications .21*
Number of multicultural graduate courses taught .25**
Number of multicultural presentations .30***
*p < .01. **p < .001. ***p < .0001.

(r = .25, p < .001), number of multicultural conferences attended (r = .18, p <

.01), and number of multicultural publications (r = .21, p < .01). The correlation
between MCPT scores and number of multicultural courses taken fell just
short of reaching the traditional .05 significance threshold (r = .13, p = .06).
Discussion. We found evidence for both reliability and validity of the MCPT.
The final 46 items demonstrated strong internal consistency reliability, and
MCPT scores were positively correlated with real-world measures of mul-
ticultural expertise. All seven correlations had a positive sign, and six of
seven were statistically significant. Therefore, this study provides construct
validity for the MCPT in measuring MCC.
The weakest correlation was between MCPT scores and number of mul-
ticultural courses taken. This result is consistent with Ruelas’s (2003) find-
ing that educational course work was not related to the ability to identify
appropriate counseling strategies for working with clients of color. Our
finding is not surprising; the other six measures of expertise all depend on
voluntary choices by the clinician (to publish, present, attend, etc.). However,
high levels of course work are likely due to graduate program requirements
rather than personal interests or motivation. Hence, number of multicultural
courses taken may be a less powerful measure of expertise and competence
than the other indices. Indeed, it should be noted that of the seven real-world
behaviors we assessed, number of multicultural courses taken is one of two
measures least likely to be used elsewhere to rate professional expertise and
productivity (number of multicultural texts read is the other).
Two specific limitations should be noted. First, although correlations be-
tween MCPT scores and six of our seven real-world behaviors were positive
and significant, the magnitudes of the associations were modest (between
.18 and .30). This range of correlations is not surprising; instead of a single,
gold-standard measure of MCC, we had a set of self-reported activities,
none of which are solely influenced by MCC. For example, the number of
publications or presentations on multicultural topics could reflect in part the
settings in which clinicians were working (research oriented vs. treatment

Journal of Multicultural Counseling and Development • July 2016 • Vol. 44 219

or teaching oriented). Nevertheless, our data represent stronger evidence
for validity than those provided for any other MCC assessment.
Second, our results do not speak to whether the MCPT can provide accurate
information on within-subjects changes in MCC over time. One of our goals
in creating this measure was to assess the success of training programs in
producing multiculturally competent clinicians. We are currently collecting
data to examine whether the MCPT is sensitive to changes over time and can
be used to compare the efficacy of different teaching modalities.

conclusion
The validation of a standardized measure of MCC creates the possibility of
addressing a number of open questions in the field because it is demand-free,
unlike self-report measures. The MCPT can be used to assess the MCC of
students in a more standardized manner than has been available previously;
thus, the effectiveness of multicultural training can be evaluated. Faculty
can administer the MCPT along with self-report MCC scales to provide stu-
dents feedback on the validity of their MCC self-perceptions; discrepancies
between scores on self-report measures and the MCPT may help students
acknowledge deficits in their multicultural self-awareness and serve as mo-
tivation to seek further training. Community mental health agencies can use
the MCPT to assess the effectiveness of their multicultural orientation and
training for new hires. Professional organizations might use the measure to
promote the inclusion of a multicultural continuing education requirement
in state licensure.
The existence of a standardized measure of MCC may facilitate multicul-
tural counseling research, in that MCC can be used as an independent vari-
able to develop evidence-based practices for diverse populations (e.g., does
MCC allow for more effective implementation of evidence-based practices
or for the development of a stronger therapeutic alliance with racial/ethnic
minority clients?). Future studies might compare the MCPT with self-report
measures, multicultural case conceptualization, and demonstrated MCC
measures (see Sehgal et al., 2011) to determine how best to use the various
measures in complementary ways. Further research is needed to explore the
usefulness of the MCPT in a number of domains and its applicability not
only to clinical and academic settings, but also to policy issues. We hope that
the MCPT will be the first of several standardized MCC measures and that
additional versions of the MCPT will be developed to address other areas
of diversity (e.g., social class; lesbian, gay, bisexual, and transgender issues;
gender issues; ability and disability issues).
We recognize that the challenge of standardized measures rests in the dy-
namic nature of assessment, which requires test security, alternative forms,
and ongoing revisions based on developments in the field. Therefore, pro-
moting standardized measures inevitably implies promoting collaboration

220 Journal of Multicultural Counseling and Development • July 2016 • Vol. 44

among researchers and testing agencies. Standardized assessment of MCC
is not work that can be effectively sustained in isolation; however, it is work
that promises to have a powerful and much-needed impact on counseling
and psychotherapy.

references
American Psychological Association. (2003). Guidelines on multicultural education,
training, research, practice, and organizational change for psychologists. American
Psychologist, 58, 377–402. doi:10.1037/0003-066X.58.5.377
Arredondo, P., Toporek, R., Brown, S., Jones, J., Locke, D. C., Sanchez, J., & Stadler, H.
(1996). Operationalization of the Multicultural Counseling Competencies. Journal
of Multicultural Counseling and Development, 24, 42–78. doi:10.1002/j.2161-1912.1996.
tb00288.x
Cartwright, B. Y., Daniels, J., & Zhang, S. (2008). Assessing multicultural competence:
Perceived versus demonstrated performance. Journal of Counseling & Development,
86, 318–322. doi:10.1002/j.1556-6678.2008.tb00515.x
Castillo, L. G., Brossart, D. F., Reyes, C. J., Conoley, C. W., & Phoummarath, M. J.
(2007). The influence of multicultural training on perceived multicultural counsel-
ing competencies and implicit racial prejudice. Journal of Multicultural Counseling
and Development, 35, 243–254. doi:10.1002/j.2161-1912.2007.tb00064.x
Constantine, M. G. (2001). Predictors of observer ratings of multicultural counseling
competence in Black, Latino, and White American trainees. Journal of Counseling
Psychology, 48, 456–462. doi:10.1037/0022-0167.48.4.456
Constantine, M. G., Gloria, A. M., & Ladany, N. (2002). The factor structure underly-
ing three self-report multicultural counseling competence scales. Cultural Diversity
and Ethnic Minority Psychology, 8, 334–345. doi:10.1037/1099-9809.8.4.334
Constantine, M. G., & Ladany, N. (2000). Self-report multicultural counseling
competence scales: Their relation to social desirability attitudes and multicul-
tural case conceptualization ability. Journal of Counseling Psychology, 47, 155–164.
doi:10.1037/0022-0167.47.2.155
D’Andrea, M., & Heckman, E. (2008). A 40-year review of multicultural counsel-
ing outcome research: Outlining a future research agenda for the multicul-
tural counseling movement. Journal of Counseling & Development, 86, 356–363.
doi:10.1002/j.1556-6678.2008.tb00520.x
Dunn, T. W., Smith, T. B., & Montoya, J. A. (2006). Multicultural competency
instrumentation: A review and analysis of reliability generalization. Journal
of Counseling & Development, 84, 471–482. doi:10.1002/j.1556-6678.2006.
tb00431.x
Dunning, D., Heath, C., & Suls, J. M. (2004). Flawed self-assessment: Implications for
health, education, and the workplace. Psychological Science in the Public Interest, 5,
69–106. doi:10.1111/j.1529-1006.2004.00018.x
Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to rec-
ognize their own incompetence. Current Directions in Psychological Science, 12, 83–87.
doi:10.1111/1467-8721.01235
Fuertes, J. N., Bartolomeo, M., & Nichols, M. (2001). Future research directions in
the study of counselor multicultural competency. Journal of Multicultural Counseling
and Development, 29, 3–12. doi:10.1002/j.2161-1912.2001.tb00499.x

Journal of Multicultural Counseling and Development • July 2016 • Vol. 44 221

Furnival, G. M., & Wilson, R. W. (1974). Regression by leaps and bounds. Technometrics,
16, 499–511. doi:10.1080/00401706.1974.10489231
Gamst, G., Dana, R. H., Der-Karabetian, A., Aragon, M., Arellano, L., Morrow, G., &
Martenson, L. (2004). Cultural competency revised: The California Brief Multicul-
tural Competence Scale. Measurement and Evaluation in Counseling and Development,
37, 163–183.
Goodman, L. A. (1961). Snowball sampling. Annals of Mathematical Statistics, 32,
148–170. doi:10.1214/aoms/1177705148
Hand, D. J. (1981). Branch and bound in statistical data analysis. The Statistician, 30,
1–13.
Hays, D. G. (2008). Assessing multicultural competence in counselor trainees: A review
of instrumentation and future directions. Journal of Counseling & Development, 86,
95–101. doi:10.1002/j.1556-6678.2008.tb00630.x
Kim, B. S. K., Cartwright, B. Y., Asay, P. A., & D’Andrea, M. J. (2003). A revision of the
Multicultural Awareness, Knowledge, and Skills Survey–Counselor Edition. Measure-
ment and Evaluation in Counseling and Development, 36, 161–180.
Kivel, P. (2002). Uprooting racism: How White people can work for racial justice (Rev. ed.).
Gabriola Island, British Columbia, Canada: New Society.
Ladany, N., Inman, A. G., Constantine, M. G., & Hofheinz, E. W. (1997). Supervisee
multicultural case conceptualization ability and self-reported multicultural com-
petence as functions of supervisee racial identity and supervisor focus. Journal of
Counseling Psychology, 44, 284–293. doi:10.1037/0022-0167.44.3.284
LaFromboise, T. D., Coleman, H. L., & Hernandez, A. (1991). Development and factor
structure of the Cross-Cultural Counseling Inventory–Revised. Professional Psychology:
Research and Practice, 22, 380–388. doi:10.1037/0735-7028.22.5.380
Liu, W. M., Sheu, H., & Williams, K. (2004). Multicultural competency in research:
Examining the relationships among multicultural competencies, research training
and self-efficacy, and the multicultural environment. Cultural Diversity and Ethnic
Minority Psychology, 10, 324–339. doi:10.1037/1099-9809.10.4.324
Manese, J. E., Wu, J. T., & Nepomuceno, C. A. (2001). The effect of training on multi-
cultural counseling competencies: An exploratory study over a ten-year period. Journal
of Multicultural Counseling and Development, 29, 31–40. doi:10.1002/j.2161-1912.2001.
tb00501.x
National Board for Certified Counselors. (2016). National Counselor Examination for
Licensure and Certification. Retrieved from http://www.nbcc.org/Exam/National-
CounselorExaminationForLicensureAndCertification
Neufeldt, S. A., Pinterits, E. J., Moleiro, C. M., Lee, T. E., Yang, P. H., Brodie, R. E.,
& Orliss, M. J. (2006). How do graduate student therapists incorporate diversity
factors in case conceptualization? Psychotherapy: Theory, Research, Practice, Training,
43, 464–479. doi:10.1037/0033-3204.43.4.464
Neville, H., Spanierman, L., & Doan, B. (2006). Exploring the association between
color-blind racial ideology and multicultural counseling competencies. Cultural
Diversity and Ethnic Minority Psychology, 12, 275–290. doi:10.1037/1099-9809.12.2.275
Owen, J. J., Tao, K., Leach, M. M., & Rodolfa, E. (2011). Clients’ perceptions of their
psychotherapists’ multicultural orientation. Psychotherapy, 48, 274–282. doi:10.1037/
a0022065
Pedhazur, E. J., & Schmelkin, L. P. (1991). Measurement, design, and analysis: An inte-
grated approach. Hillsdale, NJ: Erlbaum.

222 Journal of Multicultural Counseling and Development • July 2016 • Vol. 44

Penn, S. L., & Post, P. B. (2012). Investigating various dimensions of play therapists’
self-reported multicultural counseling competence. International Journal of Play
Therapy, 21, 14–29. doi:10.1037/a0026894
Ponterotto, J. G., Casas, J. M., Suzuki, L. A., & Alexander, C. M. (Eds.). (2010). Hand-
book of multicultural counseling (3rd ed.). Thousand Oaks, CA: Sage.
Ponterotto, J. G., Gretchen, D., Utsey, S. O., Rieger, B. P., & Austin, R. (2002). A
revision of the Multicultural Counseling Awareness Scale. Journal of Multicultural
Counseling and Development, 30, 153–180. doi:10.1002/j.2161-1912.2002.tb00489.x
Ponterotto, J. G., & Potere, J. C. (2003). The Muticultural Counseling Knowledge and
Awareness Scale (MCKAS): Validity, reliability, and user guidelines. In D. B. Pope-
Davis, H. L. K. Coleman, W. M. Liu, & R. L. Toporek (Eds.), Handbook of multicultural
competencies in counseling and psychology (pp. 137–153). Thousand Oaks, CA: Sage.
Pope-Davis, D. B., & Dings, J. G. (1995). The assessment of multicultural counseling
competencies. In J. G. Ponterotto, J. M. Casas, L. A. Suzuki, & C. M. Alexander
(Eds.), Handbook of multicultural counseling (pp. 287–311). Thousand Oaks, CA: Sage.
Ridley, C. R. (2005). Overcoming unintentional racism in counseling and therapy (2nd ed.).
Thousand Oaks, CA: Sage.
Ridley, C. R., & Kleiner, A. J. (2003). Multicultural counseling competence: History,
themes, and issues. In D. B. Pope-Davis, H. L. K. Coleman, W. M. Liu, & R. L. Toporek
(Eds.), Handbook of multicultural competencies in counseling and psychology (pp. 3–20).
Thousand Oaks, CA: Sage.
Ruelas, S. (2003). Objectively measured multicultural counseling competencies. In
D. B. Pope-Davis, H. L. K. Coleman, W. M. Liu, & R. L. Toporek (Eds.), Handbook
of multicultural competencies in counseling and psychology (pp. 283–300). Thousand
Oaks, CA: Sage.
Sehgal, R., Saules, K., Young, A., Grey, M. J., Gillem, A. R., Nabors, N. A., . . . Jefferson,
S. (2011). Practicing what we know: Multicultural counseling competence among
clinical psychology trainees and experienced multicultural psychologists. Cultural
Diversity and Ethnic Minority Psychology, 17, 1–10. doi:10.1037/a0021667
Sodowsky, G. R., Taffe, R. C., Gutkin, T. B., & Wise, S. L. (1994). Development of
the Multicultural Counseling Inventory: A self-report measure of multicultural
competencies. Journal of Counseling Psychology, 41, 137–148. doi:10.1037/0022-
0167.41.2.137
Spanierman, L. B., Poteat, V. P., Wang, Y.-F., & Oh, E. (2008). Psychosocial costs of rac-
ism to White counselors: Predicting various dimensions of multicultural counseling
competence. Journal of Counseling Psychology, 55, 75–88. doi:10.1037/0022-0167.55.1.75
Steele, C. M. (1997). A threat in the air: How stereotypes shape intellectual identity
and performance. American Psychologist, 52, 613–629. doi:10.1037/0003-066X.52.6.613
Sue, D. W. (2010). Microaggressions in everyday life: Race, gender, and sexual orientation.
Hoboken, NJ: Wiley.
Sue, D. W., & Sue, D. (2003). Counseling the culturally diverse: Theory and practice (4th
ed.). New York, NY: Wiley.
Sue, S. (2003). In defense of cultural competency in psychotherapy and treatment.
American Psychologist, 58, 964–970. doi:10.1037/0003-066X.58.11.964
Toporek, R. L., & Pope-Davis, D. B. (2005). Exploring the relationships between
multicultural training, racial attitudes, and attributions of poverty among graduate
counseling trainees. Cultural Diversity and Ethnic Minority Psychology, 11, 259–271.
doi:10.1037/1099-9809.11.3.259

Journal of Multicultural Counseling and Development • July 2016 • Vol. 44 223

Tummala-Narra, P., Singer, R., Li, Z., Esposito, J., & Ash, S. E. (2012). Individual and
systemic factors in clinicians’ self-perceived cultural competence. Professional Psychol-
ogy: Research and Practice, 43, 165–174. doi:10.1037/a0025783
Vasquez, M. J. T. (2010). Ethics in multicultural counseling practice. In J. G. Ponterotto,
J. M. Casas, L. A. Suzuki, & C. M. Alexander (Eds.), Handbook of multicultural counsel-
ing (3rd ed., pp. 127–145). Thousand Oaks, CA: Sage.
Worthington, R. L., Soth-McNett, A. M., & Moreno, M. V. (2007). Multicultural
counseling competencies research: A 20-year content analysis. Journal of Counseling
Psychology, 54, 351–361. doi:10.1037/0022-0167.54.4.351

224 Journal of Multicultural Counseling and Development • July 2016 • Vol. 44