THE MACCAT-CA AND THE ECST-R: A CRITICAL EVALUATION AND

COMPARISON OF SECOND GENERATION COMPETENCY

TO STAND TRIAL INSTRUMENTS
____________________________________

A Thesis

Presented to the

Faculty of

California State University, Fullerton

____________________________________

In Partial Fulfillment

of the Requirements for the Degree

Master of Science

in

Clinical Psychology
_______________________________

By

Jael Huizar

Thesis Committee Approval:

Nancy Ryba Panza, Department of Psychology, Chair

Cheryl Crippen, Department of Psychology
Mindy Mechanic, Department of Psychology

Spring, 2017
 ABSTRACT

Among all the standardized tests available to assess competency to stand trial, the

MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT-CA)

and The Evaluation of Competence to Stand Trial-Revised (ECST-R) are unique in that

they are among the most extensively developed tools. While developed for the same

purposes, they differ in both the theoretical foundation and the approach to assess

competence. These differences have fostered questions regarding whether outcomes on

each test could potentially vary when used with the same defendant.

The purpose of this review was to provide an in-depth understanding of each test

by exploring what is known about the psychometric properties of each test; as well as

what is known about the way each instrument works across various populations. The

research available offers some basis to make tentative recommendations for the clinical

utility of these tests that are offered herein. Below follows a discussion of the limitations

and recommendations for use of each in clinical practice. While recommendations for

clinical use of the tests are provided, any suggestions offered should be considered

tentative, as there is a lack of consensus in the field regarding how the instruments should

be used.

ii
 TABLE OF CONTENTS

ABSTRACT ................................................................................................................... ii

Chapter
1. INTRODUCTION ................................................................................................ 1

The Dusky Standard .............................................................................................. 2

Assessing Competence to Stand Trial................................................................... 3
Introduction of MacCAT-CA and ECST-R .......................................................... 4
The Purpose of Literature Review ........................................................................ 6

2. MACARTHUR COMPETENCE ASSESSMENT TOOL-CRIMINAL

ADJUDICATION (MACCAT-CA) ..................................................................... 7

Test Overview ....................................................................................................... 8

Psychometric Properties........................................................................................ 10
Norms............................................................................................................. 10
Reliability....................................................................................................... 11
Validity .......................................................................................................... 12
Research with the MacCAT-CA ........................................................................... 17
Strengths and Weaknesses .................................................................................... 21

3. THE EVALUATION OF COMPETENCY TO STAND TRIAL-REVISED

(ECST-R) ............................................................................................................... 26

Test Overview ....................................................................................................... 26

Psychometric Properties........................................................................................ 30
Norms............................................................................................................. 30
Reliability....................................................................................................... 31
Validity .......................................................................................................... 32
The ATP Scales ............................................................................................. 35
Research with the ECST-R ................................................................................... 36
Strengths and Weaknesses .................................................................................... 37

4. RESEARCH COMPARING THE MACCAT-CA AND THE ECST-R .............. 40

5. CONCLUSIONS AND RECOMMENDATIONS FOR USE IN

CLINICAL PRACTICE ....................................................................................... 45

iii
APPENDIX: TABLES................................................................................................... 53

REFERENCES .............................................................................................................. 56

iv
 1

CHAPTER 1

INTRODUCTION

Competency to stand trial evaluations form a significant part of the legal process

with an estimated 60,000 referrals made to forensic psychologists each year

(Borum & Grisso, 1995). Competency to stand trial (CST) refers to an individual’s

capacity to function as a defendant in a court of law. The legal system can be very

intimidating for defendants, in part because legal jargon is not easily understood outside

the community of attorneys, lawmakers, and judges. Therefore, it can be difficult for

legal system participants to understand and effectively take part in the legal process.

Nowhere is it more important to understand the legal process than for individuals charged

with crimes because the outcome of a defendant’s case rests heavily on the actions taken

by attorneys, judges and jurors.

Although a law degree is not necessary to utilize the court system, it is important

for a defendant to be able to do certain things such as communicate with an attorney,

understand one’s civil rights, and possess enough basic knowledge to actively participate

in legal proceedings. In most cases the average person possesses the minimal level of

legal knowledge necessary for effectively working with an attorney and proceeding

through the legal system. However, individuals with cognitive or psychological

impairment are vulnerable in legal proceedings because symptoms of mental illness

 2

and/or cognitive deficits may hinder a defendant’s ability to think rationally or make

significant decisions about their legal defense.

The Dusky Standard

In Dusky v. United States (1960), the U.S. Supreme Court concluded that “it is not

enough . . . that ‘the defendant [is] oriented to time and place and [has] some recollection

of the events,’ but defined competency to stand trial as “whether [the defendant] has

sufficient present ability to consult with his lawyer with a reasonable degree of rational

understanding and a rational as well as factual understanding of the proceedings against

him” [362 U.S. 402, 403]. Since then, each state in the U.S. has adopted some variation

of the Dusky standard for CST evaluations. The justice system has relied on this standard

as a guide for determining whether defendants are competent and therefore, able to

advance in the adjudicative process.

In terms of clinical application, the Dusky standard has been challenging for

mental health practitioners to operationalize. No specific recommendations were

provided in the ruling as to how an examiner should define terms such as “rational

understanding” nor is it clear in terms of what degree of impairment is enough for a

defendant to be ruled incompetent to stand trial. Consequently, legal scholars and clinical

researchers have put forth great efforts to conceptualize Dusky in order to produce a more

well-defined illustration of the specific abilities and levels of those abilities that are

required to consider one competent to stand trial (Bonnie, 1992; Grisso, 2003; Otto, et al.,

1998; Poythress et al., 1999; Roesch, Hart, & Zapf, 1996; Roesch, Zapf, Golding,

& Skeem, 1999; Rogers et al., 2001; Rogers, Jackson, Sewell, Tillbrook, & Martin, 2003;

Zapf, 1999; Zapf et al., 2009). The Dusky standard has generally been conceptualized as
 3

having three central functions which include the ability to: consult with counsel, possess

sufficient rational understanding to formulate substantial decisions related to one’s trial,

and grasp adequate factual understanding about general courtroom proceedings (Grisso,

2003). Competent defendants are expected to understand their legal situation and how it

applies to them and to be able to assist their attorney in constructing a defense.

Assessing Competence to Stand Trial

Assessing competence to stand trial is not a simple or even a uniform process for

clinicians who conduct these evaluations. CST assessments require a careful investigation

of a defendant’s current mental state and level of functioning as well as thoughtful

consideration of how the defendant’s functioning might interfere with or impede one’s

ability to adequately participate in the legal proceedings (Grisso, 2003; Melton, Petrila,

Poythress, & Slobogin, 2007; Zapf & Roesch, 2009). Clinicians must consider

the expectations of the justice system, and whether the demands of the defendant’s case

may exceed the abilities the individual possesses. Following such an evaluation,

clinicians must then interpret and report their results in a format useful to the justice

system where a decision regarding the defendant's competence will be made, most

typically by the judge overseeing the case.

Despite recent improvements, the process of assessing competency is not without

criticism, especially given the wide variety of methods currently used by practitioners

who conduct CST evaluations (Archer, Buffington-Vollum, Stredny & Handel, 2006;

Borum & Grisso, 1995; Grisso, 1991, 2003; Holub, 1992; Neal & Grisso, 2014). In the

past, examiners tended to focus too heavily on mental illness and failed to address how

the illness directly related to the legal standard for competence in their evaluations
 4

(Grisso, 2003). However, there is now a clear consensus among experts that the presence

of mental illness alone does not automatically render a defendant incompetent to stand

trial (Bonnie, 1992; Golding & Roesch, 1988; Tillbrook, 2000). Instead, a careful

examination of not just the presence of impairments, but how those impairments intersect

with the demands of the legal case, is necessary when performing a CST evaluation.

Because traditional psychological tests do not address legal functioning, instruments or

tools that can help assess a defendant’s abilities as defined in Dusky have become highly

desirable. As such, forensic assessment instruments (FAIs), and specifically, Competence

Assessment Instruments (CAIs) have been developed to guide those who are conducting

CST evaluations to aid clinicians in their ability to evaluate both one’s mental status as

well as the effect any deficits that are present might have on the competence-related

abilities outlined in Dusky.

Introduction of MacCAT-CA and ECST-R

Since their creation, evaluators have regularly used CAIs to aid in forming their

opinions when conducting CST evaluations; however, the relation of such measures to

the Dusky standard has been debated. Among all of the standardized tests available to

assess competency, the MacArthur Competence Assessment Tool-Criminal Adjudication

(MacCAT-CA) and the Evaluation of Competence to Stand Trial-Revised (ECST-R) are

unique in that they are considered to be the most extensively developed tools, both of

which focus on the legally relevant concepts most related to the Dusky standard. While

both tests were created from thoughtful programs of research to address the functional

abilities drawn from Dusky, the MacCAT-CA and ECST-R are very distinct from each

other. These forensic instruments differ in the theoretical foundation on which they are
 5

based. For example, the MacCAT-CA is fashioned after Bonnie’s two-part conceptual

model on competence which includes both: basic and decisional competence; whereas

ECST-R is designed to directly parallel the three Dusky prongs, which includes: Consult

with Counsel, Factual Understanding of Court Proceedings, and Rational Understanding

of Courtroom Proceedings. In addition, these instruments differ in their approach to

assessing competence. The MacCAT-CA relies primarily on a series of structured

questions based on a hypothetical scenario to evaluate competence, while the ECST-R

takes a semi-structured approach to assess both competency and feigning while focusing

heavily on psychosis as the underlying basis for incompetence.

The expectation that one competency measure could be used by evaluators and

would have the flexibility and utility to fit every defendant’s unique case is unrealistic.

Indeed, all existing assessment tools do not operate uniformly and many approach the

assessment of CST differently, as is the case with the MacCAT-CA and the ECST-R.

Thus, it is important to know whether the findings from these forensic instruments are

equivalent and, if not, how might any variance across these tools affect the outcomes of

the defendants being evaluated. It is important to know: whether one test might hold

defendants to a higher standard of functioning than the others, whether one assessment

instrument is more suitable for use with a particular type of defendant, and whether

different instruments should be limited for use with certain groups of defendants. These

issues are important to understand given the differences between the ECST-R and the

MacCAT-CA. With these questions identified, a review of the available research would

help to provide insight as to the current state of knowledge regarding these two CAIs.
 6

The Purpose of Literature Review

For the present study, an extensive review of all the existing literature relating to

the MacCAT-CA and the ECST-R was conducted. The purpose of this literature review

was to provide an in-depth understanding of each test by exploring what is known about

the psychometric properties of each test; as well as what is known about the way in which

each instrument works across various populations. The main goals for this review were:

(a) to summarize the literature available for each test to provide a detailed overview of

the current knowledge of each instrument, including a critique of the strengths and

weaknesses of each; (b) to conduct a comparison of the two tests against other another;

and (c) to provide recommendations for use of each test in clinical practice for those

conducting CST evaluations.

 7

CHAPTER 2

MACARTHUR COMPETENCE ASSESSMENT TOOL-

CRIMINAL ADJUDICATION (MacCAT-CA)

The MacCAT-CA is a competence assessment instrument that resulted from two

main projects. The first project, which was supported by the MacArthur Foundation

Research Network on Mental Health and the Law, was designed to create a more

operationalized definition of decisional competence as it relates to individuals who are

criminal defendants (Bonnie et al., 1997; Poythress et al., 1999). As part of this project, a

lengthy research-based version of a CAI called the MacArthur Structured Assessment of

Competence-Criminal Defendants (MacSAC-CD; Hoge et al., 1997) was created to

measure abilities that were thought to be important when making decisions regarding

one’s legal case. The MacSAC-CD focused on four factors that included the abilities to:

comprehend important information, understand the personal nature of the situation,

reason with information related to one’s case, and make a decision. These factors were

thought to significantly influence a defendant’s ability to participate meaningfully in a

trial. However, the MacSAC-CD was not user-friendly because it was lengthy, difficult to

score, and the items were repetitive (Poythress et al., 1999). Therefore, this instrument

was not practical for everyday use in CST evaluations and eventually it was shortened

and simplified for use in CST evaluations. The more condensed, user-friendly, instrument

became known as the MacArthur Competence Assessment Tool-Criminal Adjudication

(MacCAT-CA) (Poythress et al., 1999).

 8

Test Overview

The MacCAT-CA contains 22 items from the original MacSAC-CD and is

intended for use with defendants who are 18 years of age or older, who have an IQ

greater than 60, and who speak English. The MacCAT-CA was based in part on the

MacSAC-CD’s foundation in decision-making and in part on Bonnie’s (1992) definition

of competence to stand trial. Bonnie (1992) proposed that the Dusky standard could be

operationalized into two types of competency: “basic competence” and “decisional

competence.” Basic competence refers to a defendant’s ability to understand the legal

accusations, the judicial process, and the defendant’s overall capacity to effectively work

with an attorney to prepare a defense; while decisional competence refers to aiding one’s

attorney by providing input on the case and possessing the ability to make rational and

reasonable decisions in conjunction with legal defense (Bonnie, 1992). The newer

MacCAT-CA was divided into three scales: Understanding, Reasoning, and

Appreciation. These scales were meant to align with Bonnie’s (1992) definition with the

Understanding and Reasoning scales measuring basic competence and the Appreciation

scale measuring decisional competence. The published version of the MacCAT-CA

contains standardized administration instructions and structured scoring guidelines to

provide ratings of patients’ abilities across the three areas assessed. Outcomes from the

MacCAT-CA are not meant to stand alone in determining competence or incompetence,

but are meant to be integrated with other relevant information from the defendant’s

specific legal case as well as a consideration of one’s overall level of functioning

(Poythress et al., 1999).

 9

The first domain of CST assessed by the MacCAT-CA is Understanding, which

evaluates a defendant’s knowledge of common information about the justice system

through questions about a fictional defendant. Theoretically, this domain aligns with the

“factual understanding” part of the Dusky standard because it measures a defendant’s

capacity to recognize general information related to the legal system. The Understanding

section allows retesting on many of the items when a defendant answers a question

wrong. The retest format is a benefit because it considers a defendant’s ability to learn

and make decisions (Poythress et al., 1999). The second domain of CST assessed by the

MacCAT-CA is Reasoning, which explores the defendant’s ability to detect the possible

importance of legal information, as well as the ability to process decisions that a

defendant faces throughout the legal process. Theoretically, this domain aligns with the

“rational understanding” part of the Dusky standard because it measures a defendant’s

ability to classify information based on the importance and relevance of the information

to a legal case. The last domain of the MacCAT-CA assesses Appreciation by probing

defendants about their attitudes and beliefs regarding the legal process related to the

defendant’s own legal case. Theoretically, this domain is the most connected to the

“rational understanding” part of the Dusky standard because of its emphasis on the

defendant’s ability to understand and appreciate the facts of the defendant’s legal case.

The responses made by defendants are rated on a three-point scale (0, 1, or 2) using

specific scoring criteria. Higher scores on the three-point scale indicate higher levels of

abilities. Scores are totaled for each domain, but no overall score is calculated.

The MacCAT-CA has been normed on English-speaking individuals over 18

years of age with an IQ higher than 60. Thus, the authors caution that the MacCAT-CA
 10

should only be used to assess individuals that speak English due to the absence of

available translations in other languages (Poythress et al., 1999). In fact, not only is the

MacCAT-CA only available to English speaking defendants, but there is also a scarcity

of research available regarding the impact of using the MacCAT-CA with non-English

speaking defendants and the use of a translator is contraindicated due to the potential for

variations in intended meaning of the item questions and subsequent responses (Poythress

et al., 1999).

Psychometric Properties

Norms

In the original standardization sample, the MacCAT-CA was administered to 729

defendants from three groups. The first group, known as “jail unscreened” (JU), was

assumed to be competent and was comprised of 197 randomly selected pretrial

defendants from the jail inmate population. The second group known as “jail treated”

(JT) was also assumed to be competent and comprised of 249 jail inmates that were

receiving mental health services not related to restoration of competency. The third group

known as “hospitalized incompetent” (HI) was comprised of 283 defendants who were

hospitalized for restoration of competency and who were considered legally incompetent.

The participants in the norming group were predominantly Caucasian (52.5%), males

(90%); although the sample did include a smaller number of individuals representing

minority groups (i.e., 12% black, less than 1% Asian, 36% “Nonwhite”). The authors of

the MacCAT-CA did report that race/ethnicity was significantly related to Understanding

and Reasoning subscales (Poythress et al., 1999). However, further statistical testing

indicated that the variation in scores between the Whites and Non-Whites was due to
 11

variances in intellectual abilities (Poythress et al., 1999). Only 10% of the 729

participants were female, but the manual does not provide any specific information on

how the female participants performed on the MacCAT-CA and no additional

information was reported on the presence or absence of gender differences.

Reliability

In terms of reliability, the original norms for the test showed good internal

consistency of the three abilities related to competency and found alpha coefficients of

.85 for Understanding, .81 for Reasoning, and .88 for Appreciation (Otto et al., 1998;

Poythress et al., 1999, 2002). The mean interitem correlations were .36 (Reasoning), .42

(Understanding), and .54 (Appreciation). These findings demonstrate good internal

consistency and acceptable similarity of item content for these measures. In a later study,

Rogers et al. (2001) also evaluated the reliability of the MacCAT-CA scales, and reported

good internal consistency with alphas of 0.81 (Understanding), 0.85 (Reasoning) and

0.75 (Appreciation). Additional studies conducted on the psychometric properties of the

MacCAT-CA have also shown good levels of internal consistency for the Understanding

and Reasoning domains of the MacCAT-CA (Jacobs, Ryba, & Zapf, 2008; Marcus,

Poythress, Edens, & Lilienfeld, 2010; Rogers, Grandjean, Tillbrook, Vitacco, & Sewell,

2001). However, some research has shown the appreciation scale to be weaker

psychometrically than the other two scales and it has been hypothesized that this

difference may stem from the differing format of the items on this scale from the other

two MacCAT-CA scales (Jacobs et al., 2008; McIlnay, 2010; Zapf, Skeem & Golding,

2005).
 12

Otto et al., (1998) reported that interscorer reliability for the three scales ranged

from good to excellent, with Intraclass Correlation Coefficients (ICC’s) of .75 for

Appreciation, .85 for Reasoning, and .90 for Understanding. Lexcen, Hawk, Herrick, and

Blank (2006) also reported ICCs for the subscales of the MacCAT-CA for administration

that occurred either in person or via video administration and the ICCs were excellent

across both conditions for all the scales, except for the appreciation subscale which

yielded a slightly lower ICC of .69 for both conditions. Overall, studies evaluating the

reliability of the MacCAT-CA scales have been favorable, although the Appreciation

scale does appear to be somewhat weaker than the Understanding and Reasoning scales.

Validity

In terms of validity, the original validation study conducted by the test developers

assessed the criterion-related validity by examining the variances between the average

scores for the three sample groups: HI, JT, and JU. The results indicated that the

MacCAT-CA appropriately distinguishes between competent and incompetent

defendants since the HI group scored significantly lower than the JT group on all

MacCAT-CA scales (Poythress et al., 1999). They also found that the MacCAT-CA

appeared to adequately differentiate between the different groups of defendants who were

hospitalized for being incompetent and defendants who were competent but receiving

mental health services in jail. Poythress et al. (1999) compared the performance of

defendants assessed with the MacCAT-CA and found that hospitalized incompetent

defendants, as well as jail inmates, demonstrated differences in abilities related to CST

that are consistent with Dusky. The authors concluded that these findings are similar to

those found for other forensic instruments such as Competence Screening Test (CST;
 13

Lipsitt, Lelos, & McGarry, 1971) and Georgia Court Competency Test-Mississippi State

Hospital Revision (GCCT-MSH) (Nicholson, Briggs, & Robertson, 1988; Otto et al.,

1998).

From the original norming studies, Otto et al. (1998) also provided support for the

MacCAT-CA’s construct validity, by showing that its scales appropriately correlated

with measures of clinical judgments of competence, with indicators of cognitive

functioning, and with higher levels of psychopathology. In terms of convergent validity,

all three domains of the MacCAT-CA were related to indicators of intellectual capacity

and psychopathology, as would be expected (Poythress et al., 1999; Redding, 1997; Zapf

et al., 2005) and an analysis of the relationships between the MacCAT-CA subscales and

the other measures suggest that the subscale scores converge with intelligence and

education, but diverge from measures of emotional distress, decisional temperance, risk

perception, and perspective which refers to a defendant’s ability to appreciate the short

and long term effects of a decision (Krause, 1998). Other studies conducted have also

provided evidence for construct validity (Boyd, 1999; Krause, 1998; Redding, 1997; Zapf

et al., 2005).

A number of studies have been conducted comparing the three competence

abilities measured by the MacCAT-CA with various measures of psychopathology,

specifically, by evaluating relations with the psychiatric symptoms assessed on the Brief

Psychiatric Rating Scale (BPRS) (Jacobs et al., 2008; Overall & Gorham, 1962; Ryba &

Zapf, 2011; Warren, Aaron, Ryan, Chauhan, & DuVal, 2003). The psychiatric symptoms

measured by the BPRS included four clusters: psychoticism, hostility, depression, and

withdrawal. Findings from these studies indicated a strong negative relationship between
 14

the Psychosis and Withdrawal symptoms on the BPRS with all three MacCAT-CA

scales, but no significant relationship with Depression symptoms (Jacobs et al., 2008;

Redding, 1997; Ryba & Zapf, 2011; Warren et al., 2003). There appears to be some

variability in the findings in relation to the Hostility cluster of the BPRS. Redding (1997)

reported that the hostility cluster was moderately negatively related to the Reasoning and

Appreciation scales, whereas Warren et al. (2003) found that the hostility cluster was

related to the Understanding and Reasoning. Ryba and Zapf (2011) found that the

hostility cluster was related to all three MacCAT-CA scales, but they note that variations

in cognitive abilities explained more differences in the scores of all three MacCAT-CA

scales than did the psychiatric symptoms (Ryba & Zapf, 2011).

In terms of concurrent validity, the initial validation studies showed clinician’s

ratings of competence were found to correlate with MacCAT-CA scores at moderate

levels of .36 (Understanding), .42 (Reasoning), and .49 (Appreciation) with all

correlations statistically significant (Poythress et al., 1999). Zapf and Roesch (2005)

conducted an analysis of the correlations between the MacCAT-CA, the Fitness Interview

Test (FIT), and the MacArthur Competence Assessment Tool-Treatment (MacCAT-T)

instruments to determine if these different types of competence share a common

underlying construct with the results indicating that they all correlate at a moderate to

high levels. The correlation between the FIT and the MacCAT-CA was .82 and the

correlation between the MacCAT-CA and the MacCAT-T was .86. These high

correlations provide good support for the construct validity of the MacCAT-CA-CA as

the FIT is another CAI and the MaCAT-T is a similarly designed instrument that assesses

one’s competence to consent to treatment, which is a separate, but related construct.

 15

Three studies have investigated the convergent validity of the MacCAT-CA in

comparison with another CAI, the Competence Assessment to Stand Trial for Defendants

with Mental Retardation (CAST-MR), which is designed specifically for use with

defendants diagnosed with an Intellectual Disability. Overall, these studies show a lack of

convergent validity between these two forensic instruments as there was no significant

relationship found between scores on the MacCAT-CA and those on the CAST-MR

(Bennett, 2005; Jurecska, 2010; Jurecska, Peterson, & Millkey, 2012). These findings

were somewhat surprising as both instruments are designed to measure competence, and

would be expected to yield similar outcomes (Bennett, 2005).

Four studies investigated the factor structure of the MacCAT-CA (Jacobs et al.,

2008; Krause, 1998; Rogers et al., 2001; Zapf, et al., 2005). The findings across these

studies generally revealed support for either a two- or three-factor model, with the three-

factor model generally receiving the most support. However, there was some variance

among the studies as to how to organize the items into a three-factor model. The two-

factor model tested by Rogers et al. (2001) combined the Understanding and Reasoning

sections into Factor One and maintained the Appreciation section as Factor Two and

while Rogers et al. (2001) noted that, on the surface, these results may be puzzling as two

of the Dusky criteria were merged into one factor, they hypothesized that the assessment

of the Dusky prongs is likely confounded by the nature of the questions that address the

hypothetical scenario as opposed to those addressing the defendant’s own case. The

three-factor model from Rogers’ study was broken down as: Appreciation section items

related to case-specific questions (Factor 1); Understanding items related to a

hypothetical case (Factor 2); and Reasoning items with the addition of two items from
 16

Understanding section (Factor 3). Zapf et al. (2005) later tested both the two and three

factor models proposed by Rogers et al. (2001) but did not find support for either.

Instead, Zapf et al.’s (2005) confirmatory factor analysis found evidence

supporting the utility of the original design of the MacCAT-CA scales as well as a

different three factor version that was a better fit and that was also later confirmed by

Jacobs et al. (2008). The new modified three-factor model placed together all the items in

the Understanding section with Items 14-16 from the Reasoning section (Factor 1); Items

9-13 from the Reasoning section (Factor 2); and all items in the Appreciation section

(Factor 3). In all, while the MacCAT-CA scales appear to hold up reasonably well

through factor analysis, although some questions remain as to how purely they fit with

the three aspects of Dusky and whether the nature of the questions (those related to the

hypothetical scenario versus those related to the defendant’s case) holds undue influence

over one’s performance on the different sections of the instrument.

A number of studies have called into question the differences between the types

of questions across the three scales on the MacCAT-CA. Zapf and Roesch (2005)

investigated the factor loadings of each of the scales of the MacCAT-CA and found less

support for the Appreciation scale than the other two scales, which they hypothesize is a

result of the defendant having to use abstract thinking related to their own circumstances,

whereas the other scales rely on responses to the hypothetical scenario. Zapf and Roesch

(2005) suggested that perhaps the complexity of measuring a defendant’s awareness and

understanding of his or her own situation using a standardized format resulted in weaker

measurement for the Appreciation scale. This idea is in line with the suggestions put forth

by Marcus et al., (2010) who proposed that rational understanding is taxonic and should
 17

not be measured as a continuous construct, and should instead result in a yes or no

determination. They posit that a defendant either has rational understanding or does not;

unlike the factual understanding scale, where defendants may possess varying degrees of

factual understanding and suggest modifications to the Appreciation section.

Overall, given the whole of the research available on the validity and usefulness

of the MacCAT-CA, it appears there is sufficient evidence to show that this instrument

can effectively differentiate between competent and incompetent defendants, that it is

related to constructs that one would expect it to relate to (i.e., intelligence, cognitive

ability, psychoticism, clinician ratings of competence, etc.) and it differs from constructs

one would expect it to (i.e. emotional distress, risk perception, depression, etc.). In

addition, the three scales hold up reasonably well under scrutiny, although questions have

been raised about the design and strengths of the Appreciation section.

Research with the MacCAT-CA

Beyond research that directly addresses the psychometric properties of the

MacCAT-CA, a substantial number of studies have investigated the use of this instrument

with additional populations, such as those with an intellectual disability (ID) and with

youthful defendants. Four studies have investigated the utility of the MacCAT-CA with

defendants diagnosed with intellectual disabilities (Bennett, 2005; Jurescka, 2010;

Jurescka, Peterson, & Millkey, 2012; Pinals, Tillbrook & Mumley, 2006). The results of

these studies have generally indicated that the MacCAT-CA was significantly more

challenging for defendants with ID. Many defendants were incapable of understanding

the MacCAT-CA’s hypothetical scenario and there were items that were answered

incorrectly by all participants with ID (Understanding items 7 and 8) or that were not
 18

answered at all (Appreciation item 22) (Pinals et al., 2006). Further, it was noted that

participants struggled to elaborate on their responses and many became discouraged with

the number of questions on the test thus resorted to responses of “I don’t know.” Another

problem observed was that examiners spent much longer amounts of time repeating

questions and attempting to make sense of incoherent responses that they were often

unable to complete the test in a reasonable amount of time (Pinals et al., 2006). Due to

the various obstacles with ID populations, many defendants were counted as incompetent

to stand trial. Thus, defendants with intellectual disabilities that are assessed using the

MacCAT-CA may be more likely to be found incompetent to stand trial than those who

tested using other CAIs such as the CAST-MR, which utilizes a multiple-choice format

that appears to be easier for those with ID to manage.

A total of thirteen studies have investigated the utility of the MacCAT-CA with

adolescent populations The findings of these studies indicate that juveniles between the

ages of 10-15 generally scored lower than the adults whereas the older juveniles between

the ages of 16-17 tend to perform similarly to adult samples (Boyd, 1999; Burnett, 1999;

Burnett, Noblin & Prosser, 2004; LaVelle Fick, Hart, & Deardorff, 2006; Grisso et al.,

2003; Grisso & Seagrave, 2005; Lyle, 2003; Viljoen, Odgers, Grisso, & Tillbrook, 2007;

Viljoen, Slaney, & Grisso, 2009; Warren et al., 2003). Only one study found that the

younger adolescent defendants performed similarly to adults (Krause, 1998); however,

the adults in this study had significantly low IQ levels in comparison to their adolescent

samples and conclusions based on these results should be made cautiously.

Some of these studies have also investigated specific characteristics of the

MacCAT-CA that contributed to lower performances by younger adolescents and

 19

whether the differences lie in lack of knowledge versus developmental factors (Viljoen et

al., 2007; Viljoen et al., 2009). For example, the MacCAT-CA’s unique retest format was

used to determine if an adolescent’s performance would improve with a bit of education

on legal concepts (Viljoen et al., 2007). However, younger adolescents did not perform

better even after receiving education suggesting that the obstacles for adolescents are

related more to capacities than to knowledge. An additional study showed age-related

measurement bias on several items from the Understanding and Reasoning scales,

although no items from the Appreciation scale showed evidence of bias (Viljoen et al.,

2009). Overall, the findings from studies on the use of the MacCAT-CA with youthful

defendants shows notable age differences that impact performance on the MacCAT-CA

and this literature calls into question how strongly a defendant’s age or level of maturity

can influence competence opinions when the MacCAT-CA is used with this group.

In a related area of study, it was found that participants with even small

impairments in frontal lobe abilities, such as attention and abstract reasoning, were

enough to cause individuals to perform poorly on the MacCAT-CA and the authors

conclude that these difficulties may stem from the rigidity of the instrument as well as the

wordiness of the questions (Pinals et al., 2006). They also noted that the MacCAT-CA

does not adequately detect a defendant’s delusional beliefs about one’s legal situation.

Pinals et al. (2006) reported that clinical factors such as psychotic symptoms, cognitive

deficits, or efforts to feign interfered with about 30% of defendants’ ability to complete

the MacCAT-CA. These authors suggest that evaluators should not use the MacCAT-CA

to assess competence with all defendants in a public forensic setting and that instead it

should be administered on a case by case scenario. However, Akinkunmi (2006) noted

 20

that perhaps it is significant to concede that the very reasons why defendants cannot

complete the MacCAT-CA are parallel to the reasons they may be found incompetent to

stand trial, such as the presence of severe thought disorganization or cognitive

limitations.

The original norms for the MacCAT-CA included only a limited number of

women and minorities and there is a notable lack of information on any gender or race

differences reported in the test manual (Grisso, 2003). Since its publication only a small

number of studies have discussed or focused on these issues. A few studies have been

conducted that included a small sample of female participants (10% to 14%) but no

specific analyses aimed at exploring gender differences were conducted (Burnett et al.,

2004; Edens et al., 1999; Jurecska, 2010; Jurecska et al., 2012; Mueller & Wylie, 2007).

Redding (1997) conducted one of the only empirical studies that directly included an

investigation of gender, but this research, which included a sample of incarcerated female

defendants who were mostly African American, had the primary focus of investigating

the relationship between depression and legal competence, which was measured using the

MacCAT-CA. Although, Redding (1997) did not address gender issues, specifically, in

his findings, he did note that female participants showed higher levels of depression, but

there were no significant correlations between gender and scores on the MacCAT-CA,

when controlling for psychopathology.

In terms of using the MacCAT-CA with various racial or ethnic groups, very few

studies have directly addressed this issue. In a study designed to examine the effect of

race on outcomes of the MacCAT-CA, a sample of diverse defendants included

defendants who were: 48% African American, 34% Caucasian, 10% Hispanic, 6% Asian
 21

and 2% came from other groups (Krause, 1998). Results of this study found that race was

a significant factor influencing the outcomes, but only on the Understanding scale.

Specifically, the results indicated that Caucasian defendants had the highest scores,

followed by African Americans, Hispanics, and then Asians; however, further analyses

revealed that these race/ethnic differences disappeared when the defendants level of

intelligence was controlled. The only other study conducted with defendants from

racial/ethnic minority groups, found that minority groups displayed reduced levels of

understanding before teaching (Viljoen et al., 2007). However, it was also shown that

minority groups demonstrated higher levels of improvement with teaching than the non-

Hispanic Caucasian defendants. To explain their findings, the authors note that perhaps

the minority groups in their sample were less acquainted with the process of the justice

system or even with legal jargon, but they did have the ability to learn and that the

difference in performance was not due to test bias or even to difference in intelligence,

but rather due to reduced exposure and familiarity with the court system (Viljoen et al.,

2007).

Strengths and Weaknesses

While the psychometric properties of the MacCAT-CA are generally quite strong,

the test is not without criticism (Rogers et al., 2001; Rogers & Johansson-Love, 2009;

Tillbrook, 2000). In particular, the MacCAT-CA has been criticized for its loose

adherence to the Dusky standard (Pinals et al., 2006; Rogers et al., 2001). Since the

theoretical foundation of the MacCAT-CA is based in part on Bonnie’s (1992) two-part

conceptual model on competence and in part on the three Dusky prongs, it has been

argued that the merging of these two theoretical approaches resulted in an instrument that
 22

does not align perfectly with either, nor does it fit well with the legal standard defined by

Dusky (Jacobs et al., 2008; Rogers et al., 2001; Zapf, 1999; Zapf, Skeem, & Golding,

2005). In order for an assessment instrument to adhere to Dusky, it must address the three

legally relevant abilities including: rational understanding, factual understanding and

ability to consult with an attorney. The Dusky standard requires the evaluation of abilities

specifically related to a defendant’s case as well as the ability to consult with an attorney;

therefore, it is unclear how well the three domains of the MacCAT-CA directly address

the defendants’ understanding and reasoning about their specific legal case since 16 of

the 22 test items refer to a hypothetical scenario rather than the defendant’s specific case.

The MacCAT-CA items do not directly address the ability to consult with counsel prong

and thus do not fully adhere to Dusky (Pinals et al., 2006; Rogers et al., 2001).

An additional critique of the MacCAT-CA is that it was not designed to

incorporate any screening or consideration of feigning. In fact, the authors of the

MacCAT-CA (Poythress et al., 1999) encourage clinicians who suspect an exaggeration

of symptoms to use other screens for malingering. While many would argue that CAIs

are not meant to assess both competence and feigning, others have noted the importance

and benefits of doing both simultaneously (Rogers et al., 2001) While the MacCAT-CA

does not include a formal screen for feigning built into the test, there is some preliminary

evidence that defendants who attempted to feign during the MacCAT-CA administration

were unable to maintain their exaggerated or faked symptoms throughout the testing

process (Pinals et al., 2006). The authors posit this may be due to the structured format of

the test providing a distraction to how they were coming across to evaluators.
 23

Concerns have also been noted that the questions addressing the hypothetical

scenario are too rigid. For example, the MacCAT-CA manual states that evaluators must

read the hypothetical scenario verbatim and cannot paraphrase any part of it even if a

defendant does not understand the vignette (Poythress, et al., 1999). An evaluator is only

permitted to reread the story at a slower pace but is not authorized to elaborate or clarify

any parts of the scenario. Even though the vignette format has been criticized, there are

still positive reviews about the MacCAT-CA’s ability to assess a defendant’s reasoning

(Acklin, 2012; Rogers et al., 2001). Zapf et al. (2005) pointed out that instead of a

hypothetical scenario, a clinician’s open-ended conversation with a defendant regarding

one’s case may be more useful in discovering any underlying irrational thought processes

that may not be obvious using highly structured questions used in the MacCAT-CA based

on a hypothetical scenario.

A notable strength of the MacCAT-CA is the ability of the tool to assess a

defendant’s capacity due to its retest format (Rogers et al., 2001). In particular, the

Understanding section allows retesting for many of the items when a defendant answers a

question wrong. The retest format is a benefit in that it considers a defendant’s ability to

learn any specific pieces of legal information of which the defendant may not have

initially been aware (Poythress et al., 1999). This education process, allows the examiner

to differentiate between defendants who perform poorly on items because they are

incompetent, simply because they lack sufficient knowledge of the justice system. Thus,

a retest format is useful in screening out competent defendants whose scores can improve

through brief education and many participants who received brief education demonstrated
 24

an improvement in legal understanding whereas less competent participants did not

improve their scores through teaching (Viljoen et al., 2007).

Another critique about the MacCAT-CA’s norming sample is that it was based on

a prison sample of defendants that may not adequately represent nonincarcerated pretrial

defendants undergoing CST evaluations (Golding, 2008; Rosenfeld, 2001). The argument

highlights the fact that the original sample relied on defendants whose competency was

not questioned and in turn compared them to a group of defendants that were found

legally incompetent to stand trial. The critique notes that this weakens the construct

validity of the instrument. Perhaps it would have been more appropriate for the MacCAT-

CA to have incorporated a more appropriate sample that represents its target population

that includes defendants of questionable competence. Further, Golding (2008) suggested

using a sample of defendants where clear-cut competent and incompetent cases have been

removed. Since publication, no other studies have been conducted that test Golding’s

(2008) suggestion to use a sample of competent defendants of previously questioned

competency.

In all, the MacCAT-CA is an extensively developed tool designed to assist

clinicians conducting CST evaluations by providing a standardized measure with strong

psychometric properties. The MacCAT-CA was specifically created to focus on abilities

related to competence by following Bonnie’s (1992) theory of competence as well as the

Dusky standard. Overall, the MacCAT-CA has been found to have good reliability and

validity for assessing competence (Otto et al., 1998; Zapf et al., 2005). The retest format

is viewed as a strength of the instrument, as it allows clinicians to assess for a defendant’s

capacity to learn new material. Although it is highly recommended for use as a tool,
 25

rather than a standalone test, critics also have commented on its loose adherence to

Dusky, the limited scope of practice due to the original norming sample focusing on

Caucasian males, with an IQ higher than 60 that speak English and its use of a

hypothetical scenario in the Appreciation section.

 26

CHAPTER 3

THE EVALUATION OF COMPETENCY TO STAND

TRIAL-REVISED (ECST-R)

Around the same time the MacCAT-CA was being developed, a second CAI was

also in the works. The original Evaluation of Competency to Stand Trial (ECST)

instrument was developed by Rogers in 1995 (Rogers & Tillbrook, 1998). Rogers,

Tillbrook, and Sewell (2004) introduced the revised version known as the Evaluation of

Competence to Stand Trial –Revised (ECST-R) in 2004. This revised instrument was

created to include scales that evaluated competency as well as scales that were designed

to detect feigning.

Test Overview

The ECST-R was designed as a hybrid interview with both semi-structured and

unstructured components that assess the three factors of competence as defined in Dusky

v. United States (1960). The ECST-R is designed for use with defendants who are 18

years of age or older, who have an IQ greater than 60, and who speak English. The

ECST-R has not been validated for use with juveniles, defendants with moderate mental

retardation, or with non-English speaking defendants (Rogers, Tillbrook & Sewell, 2004).

The ECST-R was validated on a sample of English-speaking adult defendants with a

range of cognitive abilities. Thus, the test manual recommends only using this instrument

when a defendant possesses the necessary English language skills to understand the

questions (Rogers et al., 2004). The authors stress caution must be taken when using the
 27

ECST-R with an interpreter as words can easily lose their intended meaning when

translated, unless using a certified court interpreter (Rogers, Tillbrook, & Sewell, 2004).

The authors also note that the task of interpreting the Atypical Presentation scales can be

particularly challenging (Rogers et al., 2004).

The ECST-R categorizes the 18 competency questions into three domains that

correspond with Dusky: Factual Understanding of the Courtroom Proceedings (FAC),

Rational Understanding of the Courtroom Proceedings (RAC), and Consult with Counsel

(CWC) (Rogers et al., 2004). The six items in the Factual Understanding of the

Courtroom Proceedings (FAC) domain assess a defendant’s basic understanding of how

the courtroom works as well as their knowledge of the legal process. Special attention is

placed on whether defendants can identify significant key players of the court such as the:

judge, jury, prosecutor, and defense attorney as well as the distinct roles of these

individuals in the defendant’s case (Rogers, Tillbrook & Sewell, 2004). The seven items

in the Rational Understanding of the Courtroom Proceedings (RAC) domain assess the

ability of a defendant to make decisions that are not severely compromised by mental

illness (Rogers, Tillbrook & Sewell, 2004). Special attention is paid to mental

deficiencies and how they may affect a defendant’s decision-making abilities in

reasoning, testifying, plea negotiating, as well as in their courtroom experiences and any

unmanageable courtroom behavior. The six items in the Consult with Counsel (CWC)

domain assess how well a defendant can maintain a working relationship with one’s

attorney (Rogers, Tillbrook & Sewell, 2004). The items in the CWC domain focus on a

defendant’s: methods of communication, ability to solve conflicts when differences arise,

and expectations of the attorney. Overall, the items explore general dynamics of the
 28

attorney-client relationship. The CWC scale is assessed using open-ended type questions

unlike the items on the other scales that are more structured.

The ECST-R’s hybrid format serves an important function. The structured part of

the instrument is designed to thoroughly measure the essential functional abilities related

to competency; however, the unstructured part of the instrument allows evaluators to

conduct more extensive questioning, as needed, on a case-by-case basis. For example,

clinicians can supplement the 18 competency questions provided with additional queries

when more information is necessary to evaluate a defendant. In short, ECST-R is unique

in that provides both the structure of a standardized measure as well as the flexibility to

act as an interview-based evaluation when necessary (Rogers et al., 2004).

The authors of the ECST-R had two goals in mind in deciding how to approach

the scoring method of this test. The first goal was to assess competence-related abilities

and the second goal was to avoid generating numerical scores that could complicate

important information about a defendant’s case. Thus, they produced a method in which

the ECST-R competency scales offer valuable information about an individual defendant

that is also combined information about his or her case. Therefore, the ECST-R uses

rating criteria (0 to 4) that measure the severity of psychotic symptoms (i.e. persecutory

delusions) and self-defeating motivation of impairment (Rogers et al., 2004). Each of the

18 items has several ratings that clinicians can make for each item. The clinician then

decides on the final score based on the ratings. In some instances, more than one rating is

applied for an item, thus the score would be the equivalent to the highest. In terms of

scoring, the items in each of the three ECST-R domains (CWC, FAC, and RAC) of the

ECST-R are quantified using a rating scale for psychotic symptoms and self-defeating
 29

motivation that range from: 0, not observed; 1, questionable clinical significance; 2, mild

impairment not related to competency; 3, moderate impairment that will influence but not

impair competency; and 4, severe impairment that impairs competency. The item scores

from each of the three domains are summed and then converted to t-scores that are

interpreted using the following categories: T = 60-69, moderate impairment; T = 70-79,

severe impairment; T = 80 – 89, extreme impairment; and T = 90 – 110, very extreme

impairment.

A distinct characteristic of the ESCT-R is its inclusion of the Atypical

Presentation scales, which are comprised of 28 items that assess feigning as it relates to

competency (Rogers, Tillbrook & Sewell, 2004). The authors of the ECST-R understand

that criminal defendants may fake incompetence and may not be truthful during the CST

investigation therefore it is very important to determine whether feigning is a factor in

one’s presentation. The Atypical Presentation scales can be broken down into five

different aspects of feigning that include: Realistic (ATP-R), Psychotic (ATP-P),

Nonpsychotic (ATP-N), Impairment (ATP-I), and Both Psychotic and Nonpsychotic

(ATP-B). The ATP-R scale assesses the level of common concerns that a defendant may

face, thus low scores in the ATP-R scale may indicate defensiveness. Defendants faking

psychotic symptoms will have high scores in both ATP-P and ATP-N, as reflected by a

high ATP-B score, while defendants faking nonpsychotic symptoms will only have high

scores in ATP-N. The ATP-I scale is determined by the sum of the ATP-P and ATP-N

which together produce the impairment ratings. The process of scoring the ATP scales is

designed to be simple with the items scored on scale of three points (0 = no,

1 = sometimes, and 2 = yes). Each ATP scale is then summed and the raw scores are
 30

converted to t-scores. The inclusion of these scales is significant not only because of their

ability to detect feigning, but also due to their focus on asking about the presence or

absence of symptoms related to specific competence abilities that may be affected by

psychosis.

Psychometric Properties

Norms

The validation of the ECST-R (Table 1) was based on four clinical studies that

together contained eight different samples of defendants and these groups were

supplemented by five additional samples (Rogers et al., 2001; 2003; Rogers, Martin, &

Sewell, 2003; Tillbrook, 2000). The first study used 100 male jail detainees divided into

two groups: genuine (nonfeigning) and feigning (probable faking group; Rogers et al.,

2001). The genuine group referred to defendants with valid reports of mental disorders

whereas the feigning group referred to the defendants with false reports of mental

disorders. The second study used 28 male pretrial defendants that were referred for CST

evaluations and 42 male defendants who had been found incompetent to stand trial

(Tillbrook, 2000). The third study used 56 male defendants hospitalized in a competency

restoration program in various stages of treatment. These defendants were divided into

two groups based on those with genuine mental illness and those believed to be feigning

(Rogers, Sewell, Martin & Jackson, 2003). The fourth study used 154 jailed defendants

randomly assigned to two groups: genuine (legitimate mental disorders) and feigning

(probable false mental disorders; Rogers, Martin & Sewell, 2003). An additional sample

drawn from competency cases administered in CST evaluations was collected, mainly for
 31

normative reasons and, to increase the overall sample for confirmatory factor analyses

(Rogers et al., 2004).

The authors of the ECST-R appear to have considered cultural diversity in their

validation study, as the norm group as a whole was made up of about 53% Caucasian,

40% African American, and 10% Hispanic individuals. In terms of gender, two studies

were all male defendants (Rogers et al., 2001; Tillbrook, 2000). The third validation

sample used 82.1% male and 17.9% female (Rogers et al., 2003), the fourth validation

sample used 52.1% male and 47.9% female (Rogers et al., 2003), and the ancillary data

used 85.4% male and 14.6% female (Rogers et al., 2004). Therefore, the norms for the

ECST-R, while reasonably racially/ethnically diverse, were primarily based on males.

Reliability

Test-retest reliability was assessed by the test authors using a sample of 29

defendants that were re-administered the ECST-R one week after an initial administration

(Rogers et al., 2004). The interviewers were unaware of the results from the first

administration when they conducted the second administration of the ECST-R. The

findings indicated that 90% of the defendants’ scores on the first administration of the

ECST-R did not vary significantly on the second administration (Rogers et al., 2004).

In terms of the internal consistency of the competency scales, values reported in

the test manual are listed in Table 2 along with the mean interitem correlations (Rogers et

al., 2001; 2003; Tillbrook, 2000). These findings generally demonstrate good internal

consistency and acceptable similarity of item content for these measures. In one study,

Rogers et al. (2001) elected to combine data used by Tillbrook’s (2000) and Rogers and

Grandjean (2000) studies and the results of this integration of data yielded alpha
 32

coefficients that were acceptable for CWC and excellent for factual and rational

understanding (Rogers et al., 2001). In later studies, Vitacco, Rogers, Gabel & Munizza

(2009) reported similar internal consistencies on two scales FAC, RAC, but a notably

higher CWC; while Norton and Ryba (2010) reported internal consistencies, which were

a bit lower than those reported in the normative studies (Table 2). Two additional studies

reported that the FAC showed good internal consistency, although in these studies the

RAC and CWC were not administered (Springman, 2007; Springman & Vandenberg,

2009). While some variance is seen across these findings, it is no more than might be

expected given the challenges in assessing strictly defined clinical constructs, and overall,

it appears the scales possess an acceptable level of consistency in terms of the constructs

measured.

Interrater reliability was initially established with either two (Rogers et al., 2001;

Rogers et al., 2003) or three (Tillbrook, 2000) independent examiners who showed a

fairly high level of agreement in CST decisions (89.3%) among the raters. The average

interrater reliabilities for the ECST-R scales are reported in Table 3, although some of the

raters reported difficulty in scoring semi-structured responses due to responses by

defendants that could be applied to more than one category (Tillbrook, 2000). The overall

results of these studies indicate that the ECST-R provides consistency in measurement

across the scales as well as the scoring conducted by various evaluators.

Validity

Empirical support has been established regarding the ECST-R’s utility with

individuals suffering from mental illness as well as with, criminal populations and

defendants who were suspected of feigning based on four clinical studies supplemented
 33

by five additional samples (Rogers, Jackson, Sewell, & Harrison, 2004; Rogers et al.,

2003; Rogers, Sewell, Grandjean, & Vitacco, 2002; Vitacco, Rogers, Gabel, & Munizza,

2007). The test authors used independent experts and applied bootstrapping procedures to

compare ECST-R conclusions to comprehensive CST evaluations instead of using legal

CST decisions which are confounded by clinical opinion. The ECST-R has shown

evidence of convergent validity in its theoretical relatedness to CST concepts measures of

psychopathology and competency related abilities (Rogers et al., 2001; Rogers et al.,

2004; Tillbrook, 2000).

Evidence for the construct validity of the ECST-R has been shown via

confirmatory factor analysis that found support for the three competency scales

applicability to the three prongs drawn from the Dusky standard (Rogers et. al., 2001;

Rogers et al., 2003). The first study conducted by Rogers et al. (2001) recommended the

use of a two-factor model which essentially focused on the factors that measure factual

understanding and rational understanding. In later research, Rogers et al. (2003) assessed

the factor structure of the ECST-R according to three competing competency models

referred to as: discrete abilities (three factor model), domains (two factor model) and

cognitive difficulty (two factor model). They found that the discrete abilities model was a

moderately good fit whereas the domains and cognitive difficulty models tested

previously produced less appropriate fits. The findings of the confirmatory factor analysis

provided support for the model using the three-factor discrete abilities which, indicated

the ECST-R’s competency scales are appropriately aligned with the abilities drawn from

Dusky (Rogers et. al., 2001; Rogers et al., 2003).

 34

Studies conducted with the ECST-R indicate excellent discriminant validity in

terms of differentiating feigned from real symptoms (Norton & Ryba, 2010; Springman

& Vandenberg, 2009; Vitacco et al., 2007, 2009). Norton and Ryba (2010) found that

honest responders varied from coached feigners demonstrating large effect sizes for

differences on all of the competency scales, with the strongest effects seen for the CWC

scale. Vitacco et al. (2009) found that a group of “possible feigners” had considerably

elevated scores in comparison to the genuine group and they also report large effect sizes

for RAC (d = 1.60) and CWC (d = 1.89), although for FAC they found only a moderate

effect size (d = 0.63).

In terms of using the ECST-R minorities and women, very little information has

been reported that specifically speaks to these issues. Two studies done using only the

FAC scale found no significant differences based on gender (Springman, 2007;

Springman & Vandenberg, 2009) as well as no significant effect of ethnicity on

competence (Springman, 2007; Springman & Vandenberg, 2009). Additionally, Norton

and Ryba (2004) found no significant differences between the control and feigned mental

illness groups in terms of ethnicity. However, the dearth of research in this area leave it

an open question regarding whether the ECST-R works equally well across a variety of

races/ethnicities as well as for both women and men.

Overall, the psychometric properties of the ECST-R indicate that this instrument

effectively differentiates between competent and incompetent defendants. The scales of

the ECST-R appropriately relate with measures of psychopathology and competency

related abilities (Rogers et al., 2001; Rogers et al., 2003; Tillbrook, 2000). In addition,

the competency scales of the ECST-R effectively differentiate between feigned mental
 35

illness and real symptoms (Norton & Ryba, 2010; Rogers et al., 2003; Rogers, et al.,

2004; Springman, 2007; Springman & Vandenberg, 2009). The ECST-R’s competency

scales demonstrated good internal consistency and inter-item correlations and appear to

have appropriate interrater reliability for the competency scales indicating high levels of

agreement among evaluators (Rogers et al., 2001; Rogers et al., 2003; Tillbrook, 2000).

The ATP Scales

Since the ECST-R also includes the ATP scales that are designed to detect

feigning, it is important to consider the psychometric properties and the utility of these

scales in detecting those who may try to falsely present as incompetent. In terms of

reliability for the ATP scales, moderate to high levels of internal consistency were

reported in the test manual: ATP-R = .63, ATP-P = .79, ATP-N = .70, ATP-I = .87 and

ATP-B = .86 (Rogers et al., 2004). These findings have generally been supported by

studies that have followed that also report good internal consistency for the overall ATP

scales (α = .85; Springman, 2007; Springman & Vandenberg, 2009). Norton and Ryba

reported appropriate levels of internal consistency: ATP-R = .50, ATP-P = .74, ATP-

N = .73, ATP-B = .82, although the ATP-I scale was not assessed. Vitacco et al., (2007)

found that the five ECST-R ATP scales showed good internal consistency (ranges of

α = .72 to .90) and mean interitem correlations (ranges from .22 to .42). No studies have

assessed the test-retest reliability of the ATP scales and, due to the objective and

simplified scores given on these scales, interrater reliability is irrelevant.

Regarding the validity of the ATP scales, there is evidence to show that these

scales can effectively differentiate between feigned mental illness and real symptoms as

well as feigned mental illness and honest/not mentally ill responders (Norton & Ryba,
 36

2010; Rogers et al., 2003; Rogers, et al., 2004; Springman, 2007; Springman &

Vandenberg, 2009) and that these scales appropriately relate with measures of

psychopathology and cognitive abilities (Rogers et al., 2004). The ECST-R ATP scales

also demonstrate good convergent validity with the scales of the Structured Interview of

Reported Symptoms (SIRS; Rogers, Bagby, & Dickens, 1992) with correlations between

.50 and .70 (Vitacco et al., 2009).

Two studies have investigated the effects of test coaching and whether a coached

feigner can elude detection using the ECST-R (Springman, 2007; Springman &

Vandenberg, 2009). The results of these studies indicated that, even with coaching, the

feigning groups appeared significantly more impaired on overall competency scores

compared to the control group and it was concluded that coaching had very little

influence on competency and feigning scale scores.

Research with the ECST-R

Even after an exhaustive literature search, the only empirical studies conducted

using the ECST-R are those reviewed above that relate to an evaluation of the test’s

psychometric properties. At present, no other research has been conducted with this

instrument. It is likely that the ECST-R is not used in research investigations of

competence due to the semi-structured nature of the test and the very case and symptom-

specific scoring system that is used in determining the outcomes across the three

competency scales. CST measures that use single scores and structured questioning are

more suitable for research purposes, but the ECST-R’s unique scoring is designed for

making psycholegal decisions about competency and is not easily adapted for use in

research investigations (Rogers et al., 2004). As such, it is important to note, that at the
 37

present time, there are only four studies that have been conducted by researchers who are

not authors of the test itself or who were not part of the test development team. Having so

few independent investigations of this test is currently a limitation to the overall

knowledge about its utility and effectiveness.

Strengths and Weaknesses

Like the MacCAT-CA, the ECST-R also has identified strengths and weaknesses.

One area of concern that has been noted is that the ECST-R focuses heavily on psychosis

as the source of impairment (Norton & Ryba, 2010; Rogers, Tillbrook & Sewell, 2004).

The ECST-R is most suitable in cases where a defendant’s competence is impaired due to

psychosis since the ECST-R is not designed to assess other types of impairments.

Although it is true that a diagnosis of psychosis is still the most common reason for

defendants to be ruled incompetent to participate in a trial (McIlnay, 2010; Nicholson &

Kugler 1991), is generally accepted that psychosis and legal incompetence are not one

and the same and the concern lies in the fact that the ECST-R may miss or underreport

other sources of impairment that are not due directly to psychosis.

Another critique of the ECST-R is that the semi-structured format complicates the

scoring process for evaluators (Tillbrook, 2000). For example, the FAC scale is simple to

score due to the explicit scoring criteria; however, some of the raters reported difficulty

in scoring the semi-structured responses of the CWC scale when a defendant’s answers

could be applied to more than one category or no category at all (Tillbrook, 2000). A

recommendation made to improve the clarity of scoring on this scale was to add more

rating categories for certain items and more clearly define the degree of impairment

meant for each rating (Tillbrook, 2000). Certainly, the semi-structured nature and flexible
 38

format of the questioning and scoring of the ECST-R has limited the ability for

researchers to use this test in tightly controlled settings that are often used to evaluate the

quality and effectiveness of tests or to replicate the results reported by the test authors

during development of the instrument. But, even more importantly, the question remains

as to whether the flexibility inherent in this instrument offers more harm, via inconsistent

scoring, than the good that stems from the examiner’s ability to adjust the questions

asked and the way that responses are scored.

In terms of strengths, the creators of the ECST-R have provided empirical support

for its use with forensic populations, suspected malingerers, mentally ill patients, and

correctional samples (Rogers et al., 2002; Rogers, Jackson, Sewell, Tillbrook, & Martin,

2003; Rogers, Jackson, et al., 2004; Vitacco et al., 2007). The ECST-R also has a strong

theoretical basis as it was created to parallel the three Dusky prongs related to the

psycholegal factors required by the justice system to determine CST (Golding, 2008;

Rogers et al. 2003; Rogers & Johansson-Love, 2009). The ECST-R’s strict alignment

with Dusky minimizes the incorporation of unnecessary information that may complicate

CST evaluations. Further, in terms of meeting the Daubert criteria, the test authors

maintain that one of the advantages of the ECST-R is its likelihood for admissibility

under the Daubert standard (Rogers, Tillbrook & Sewell, 2004; Rogers & Johansson-

Love, 2009).

Another advantage is that the ECST-R’s semi-structured format allows for

maximum relation with a defendant’s case because it allows the integration of questions

that can be specifically tailored to the defendant’s case (Rogers et al., 2004). The ECST-

R is intended to be flexible enough to relate directly to a defendant’s legal case such as

 39

the defendant’s relationship with an attorney. The flexible format provides a defendant

with an expansive opportunity to demonstrate the ability to assist an attorney and

meaningfully participate in trial.

Moreover, the ECST-R was designed to measure competency, but with a

supplementary feature to screen for feigning via the Atypical Presentation Scales which

have been shown to be effective in screening for feigning in competence evaluations

(Norton & Ryba, 2010; Rogers et al., 2001; Rogers et al., 2002; Rogers, Jackson, Sewell,

Tillbrook, & Martin, 2003; Rogers, Jackson, et al., 2004; Springman, 2007; Springman &

Vandenberg, 2009; Vitacco et al., 2007, 2009). Thus, the ECST-R effectively screens for

feigning and does so in a manner that conceals the strategies that it uses so that

defendants are limited in their ability to detect the screening techniques (Rogers et al.,

2004). Consequently, the ECST-R’s approach to feigning may provide evaluators with

useful and unrehearsed information from defendants who may be motivated to exaggerate

or lie about symptoms.

All in all, the ECST-R is an extensively developed forensic instrument designed

to simplify the process of evaluating one’s CST by providing a standardized approach

with strong psychometric properties. Research on the ECST-R has confirmed that it

possesses good reliability and validity for assessing competence while also providing a

screen for feigning. Although the ECST-R is highly recommended to forensic clinicians

in the evaluations of CST, critics have commented on its emphasis on psychosis, the

semi-hybrid format that complicates scoring, and the limited scope of practice due to the

original norming sample focusing primarily on males with an IQ higher than 60 who

speak English.
 40

CHAPTER 4

RESEARCH THAT COMPARES THE MACCAT-CA

AND THE ECST-R

Few articles have been published that directly compare the MacCAT-CA and the

ECST-R. One study compared the MacCAT-CA, the ECST-R, and the CAST-MR in

regard to their relevance to the Dusky standard as well as their error and classification

rates by using the Daubert standard as a guideline (Rogers & Johansson-Love, 2009). In

terms of the Dusky standard, the authors investigated items of each of the scales of the

MacCAT-CA and the ECST-R. The findings indicated that both instruments thoroughly

assessed the factual understanding prong in terms of roles of courtroom personnel and

understanding criminal charges. However, the MacCAT-CA also assesses a defendant’s

ability to understand convictions and plea--bargaining. The instruments differed slightly

in the way they measure the rational understanding prong of Dusky. The MacCAT-CA

focuses on a defendant’s beliefs about the outcome of their case and beliefs about the

fairness of the legal system as well as a defendant’s perception of one’s attorney.

Whereas, the ECST-R focuses on a defendant’s decision-making ability about one’s trial,

such as the ability to consider different outcomes of one’s trial as well as any challenging

courtroom experiences. In terms of the Consult with Counsel prong of Dusky, the

differences in the way these two instruments approach competency was most prominent

in this domain. The MacCAT-CA relies on a hypothetical scenario to assess how well a

defendant can discriminate between valuable and useless information to share with their
 41

attorney such as being able to understand as well as explain important mitigating issues.

The ECST-R relies on a defendant’s own criminal case to assess a defendant’s insight

and what they expect of their attorney. The CWC scale of the ECST-R uses open-ended

questions about a defendant’s case to assesses the individual’s ability to solve differences

with one’s attorney as well as any impairments that may interfere with a defendant’s

ability to communicate with an attorney.

The authors also concluded that the MacCAT-CA may have problems meeting the

Daubert standard due its reliance on a hypothetical scenario to assess a defendant’s

ability to consult with counsel; whereas ECST-R integrates a defendant’s case in its semi-

structured test format to achieve the same (Rogers & Johansson-Love, 2009). They note

that the MacCAT-CA may be useful in assessing defendants who want to represent

themselves due to the difficulty of the instrument’s material, which in turn provides a

useful benchmark in assessing a defendant’s ability to understand as well as process

complicated legal information. Rogers and Johansson-Love (2009) also investigated the

length and complexity of the questions in both instruments. The authors were interested

in the possibility that word length and complexity of items may impact the performance

of defendants with a lower level of intellectual functioning on instruments such as the

MacCAT-CA and the ECST-R. They found that in terms of word length on two of the

three scales on the MacCAT-CA contain an average of 40 to 45 words, whereas the

competency scales of the ECST-R typically had less than 10 (Rogers & Johansson-Love,

2009). The authors concluded that perhaps the MacCAT-CA would be less appropriate

for defendants of lower intelligence such as those with intellectual disabilities, whereas

the ECST-R did not evidence similar limitations.

 42

Tillbrook (2000) conducted a study that was used as part of the basis for the

development of the ECST-R, however, in his study he directly compared the

psychometric strengths and weaknesses of the MacCAT-CA and the ECST-R. The results

of this study revealed that even though the ECST-R and the MacCAT-CA are very

different in their approach to CST, there were several similarities in their overall

performance. Both tests showed good interrater reliability and internal consistency and

support was found in terms of convergent validity, concurrent validity, and hit rate

analyses (Tillbrook, 2000). Specifically, in terms of predictive validity, the ECST-R and

the MacCAT-CA appeared to perform similarly well in predicting determinations of

competency and two of the three competency scales (Understanding/FAC and

Reasoning/RAC) showed moderate convergent validity. However, the ECST-R’s CWC

scale produced a lower correlation with the Appreciation scale of the MacCAT-CA,

which the author attributes to the differences in the way these scales measure a

defendant’s ability to consult with an attorney.

Finally, Tillbrook (2000) also sought to investigate how evaluators rate, organize,

and associate items on the ECST-R and MacCAT-CA in CST evaluations. The findings

indicated that when using the ECST-R, FAC appears to have the most influence on an

evaluator’s decisions of CST followed by RAC and CWC. Similarly, when using the

MacCAT-CA the Understanding scale appears to have the most influence on an

evaluator’s decision, then Appreciation and Reasoning. The domains of the ECST-R and

the MacCAT-CA were also compared to the criterion but produced slightly different

results than those reported by the evaluators as being most influential in decisions of

CST. For the ECST-R, the scales that correlated with the criterion from highest to lowest:
 43

RAC, FAC, and CWC. For the MacCAT-CA, the scales that correlated with the criterion

from highest to lowest: Reasoning, Understanding, and Appreciation. Overall, Tillbrook

(2000) makes an equal recommendation for using either instrument due to their strong

psychometric properties and systemic approaches to gathering information that is relevant

to CST evaluations.

McIlnay (2010) conducted a meta-analysis to consolidate the research data

available from 1990 to 2009 on the MacCAT-CA and the ECST-R. The analyses

included all available studies that included participants with an IQ greater than 60 who

were over the age of 18 years old. Studies using participants with intellectual disabilities

or who were adolescents were excluded. The purpose of this meta-analysis was to

investigate the specific characteristics that differed between competent defendants and

legally incompetent defendants and the specific variables of interest included: psychosis,

withdrawal, hostility, and depression and IQ.

The findings indicated that psychosis was most related to incompetency and that

the Appreciation scale (MacCAT-CA) and the CWC scale (ECST-R) showed the largest

effect sizes in terms of the level of impairment due to psychosis. The Factual

Understanding scale (MacCAT-CA) and FAC scale (ECST-R) showed a significant

relationship with the overall competency score and further, all three of the MacCAT-CA

scales were significantly related to IQ. The findings also indicated that depression had a

small but significant positive relationship with each of the competency scales of the

MacCAT-CA; however, the author also noted the large variation in the findings among

the 13 studies in the meta-analysis in regards to what they reported on the relationship

between depression and the competency scales of the instruments. In terms of hostility,
 44

the author found a significant negative correlation between hostility and Reasoning

(MacCAT-CA). In examining the factors that possibly added to the considerable variance

among effect sizes, a consistent pattern arose in which the MacCAT-CA had greater

effect sizes than the ECST-R when comparing the performances of competent and

incompetent defendants. According to these findings, it appears that the MacCAT-CA

demonstrates more sensitivity on each domain of competency than the ECST-R which

contradicts the study conducted by Tillbrook (2000) also comparing these two tests.

Intelligence was found to be related to defendant scores on Reasoning (MacCAT-

CA) but only marginally related to defendant scores on Appreciation (MacCAT-CA).

Since Appreciation (MacCAT-CA) is based on a hypothetical scenario, critics such as

Rogers et al. (2001) say that it is immaterial to a defendant’s case and difficult for

defendants to identify with. The results of this meta-analysis agree with Rogers et al.

(2001) on the difficulty of a hypothetical scenario because it is difficult for everyone to

understand regardless of IQ.

Overall, the few articles that compare the MacCAT-CA and the ECST-R

demonstrate how these tests differ in their approach to CST. While research on the

psychometric properties of the ECST-R and the MacCAT-CA provide empirical support

for both tests in terms of reliability and validity, these are differences noted in terms of

the connection to the three Dusky prongs and in the likelihood of the tests meeting the

Daubert criteria. Comparative research also shows the tests have similarities in how they

assess competence abilities related to factual and rational understanding but that they

diverge in terms of assessing a defendant’s ability to consult with an attorney.

 45

CHAPTER 5

CONCLUSIONS AND RECOMMENDATIONS FOR USE

IN CLINICAL PRACTICE

The aim of this literature review was to summarize the literature that pertains to

the MacCAT-CA and the ECST-R, to report on the strengths and weaknesses of each test,

to compare the tests with one another, and to make recommendations for use in clinical

practice. A summary of the main characteristics of each test is provided in Table 4.

Above, the tests were summarized, compared, and the strengths and weaknesses were

discussed. Below follows a discussion of the limitations and recommendations for use of

each in clinical practice. While recommendations for clinical use of the tests are

provided, any suggestions offered should be considered tentative, as there is a lack of

consensus in the field regarding how the instruments should be used.

First, it is clear that, based on the research available, the MacCAT-CA and the

ECST-R both have strong psychometric properties, particularly in terms of reliability and

validity. Further, both tests include substantial numbers of individuals in their norm

groups; however, it is notable that both tests are somewhat limited in terms of the

inclusion of women and minorities in their norms which limits the applicability and

usefulness of the tests with these groups. While use with these populations is not directly

contraindicated, the tests results should be applied and interpreted with caution as

appropriate comparisons are limited and there is a noticeable lack of research that would
 46

be necessary to provide evidence that these tests work equally as well with women and

minority groups as they do with male Caucasian populations.

Second, the test manual for both the ECST-R and the MacCAT-CA state that

these tests are not optimal for defendants that do not speak English. Presently, no

literature is available that explores the use of these specific instruments with non-English

speakers and both test manuals caution against using interpreters as the nature of the

questions could easily get lost in translation. Therefore, neither test is appropriate for use

with non-English speaking defendants. Additionally, particular care should be taken

when using the test with individuals for whom English is their second language, as the

evaluator must ensure any impairments seen are a result of deficits in functioning and not

merely a lack of or misunderstanding of the test questions.

Third, neither test should be used with defendants who have substantial deficits in

cognitive functioning, and particularly not with those diagnosed with an Intellectual

Disability. The consensus among the existing literature is that the MacCAT-CA is not

suitable for use with defendants with ID because of its lengthy items, rigid

administration, and the indications that the hypothetical scenario is confusing to

cognitively impaired individuals. Due to these test characteristics, studies have shown

there is reason to be concerned that competent defendants with ID are at an elevated risk

to be found incompetent (Bennett, 2005; Jurescka, 2010; Jurescka et al., 2012; Pinals et

al., 2006). Further, research has shown that the MacCAT-CA’s word length and the

complexity of items can negatively impact the performance of defendants with a lower

level of intellectual functioning (Rogers & Johansson-Love, 2009). Although no literature

is currently available that explores the ECST-R’s utility with ID populations, the test
 47

authors note in the manual that the test is not recommended for use with those of limited

intellectual functioning (Rogers et al., 2004).

In terms of whom each test should be used with in clinical practice, the research

available allows for some tentative recommendations to be offered. It is clear that, even

though both the MacCAT-CA and the ECST-R were developed specifically for CST

evaluations, they differ substantially in terms of their underlying construction. The

MacCAT-CA is highly structured in relation to its administration and standardized

scoring, whereas the ECST-R is semi-structured in terms of administration with a more

flexible scoring format. As such, the hybrid format of the administration may complicate

scoring responses on the ECST-R and make it difficult to categorize particular ratings in

some instances. These differences in test construction cause the MacCAT-CA and the

ECST-R to work in different ways with different populations of defendants.

It appears the ECST-R is most likely to be useful in cases where defendants are

presenting with symptoms of a mental illness and particularly so in cases where there is

suspicion of feigning. The ECST-R is particularly well suited for cases where a

defendant’s competence is impaired due to psychosis because of its unique focus on the

impact of psychosis on competence abilities as well as the availability of the ATP Scales,

which can help to directly assess the veracity of psychotic symptoms. While the ECST-

R’s strength appears to be with cases in which defendant’s present with psychosis, there

is concern that it misses other sources of competence-related impairment that are not due

directly to psychosis and, as such, the test would be less useful in these types of cases.

The MacCAT-CA, on the other hand, appears to be most useful in cases where

defendants are presenting with symptoms of a mental illness and an average level of
 48

intellectual functioning. While the MacCAT-CA’s strength appears to be with cases of

defendants that present with some limitations, there is concern whether it is appropriate to

use with defendants who are severely impaired due to symptoms of psychosis or deficits

in cognitive functioning. In such cases, it is likely that these defendants are too impaired

to complete the test. Thus, the MacCAT may be best suited for any “grey area” cases

where the defendant has some limitations that are not too severe.

Research also suggests that, due to the nature of some of the questions, the

MacCAT-CA may be useful in situations where defendants want to represent themselves.

In such cases, the Rational and Factual Understanding scales are useful in providing the

courts with a benchmark for a defendant’s knowledge of the legal system, ability to plea

bargain, and reason with important legal information which are important factors in self-

representation cases. The MacCAT-CA is also likely to be a useful tool for defendants

who might improve their performance with the retest feature, such as defendants with

minimal education or limited familiarity of the legal system. C Finally, the MacCAT-CA

may be useful with defendants who are paranoid, mistrustful, guarded, or those who are

severely distressed about their criminal charges because of its reliance on a hypothetical

scenario rather than focusing on questions related to the defendant’s own case. On the

other hand, the MacCAT-CA may not be optimal for a busy inpatient public mental

health setting where evaluators have a higher need for flexibility in servicing a diverse

population of defendants such as defendants with intellectual deficits, neuropsychological

impairments, and cognitive impairments (Pinals et al., 2006).

Overall, the MacCAT-CA and the ECST-R provide two well-designed and

validated options for clinicians to use while conducting CST evaluations. While, on the
 49

surface, it would appear that two tests designed with the same purpose should be able to

be used interchangeably and in all situations in which such a test might be necessary, but

it is clear that is not the case with the ECST-R and the MacCAT-CA. The important

differences in the test structure and design, in the administration and scoring, and in the

interpretation, that can be drawn from the results all lead to many differences in the

clinical utility of each instrument. It is clear that both tests are useful, but it is also clear

that both tests have their limitations and it is very important that any forensic evaluator

who is seeking a test to assist in forming an opinion on competency must be informed

about the nature and limitations of these two instruments that are discussed herein. While

both tests are useful, the limitations of each is certainly important enough to warrant

caution in the use with certain groups either permanently, or until additional research is

available to justify such use.

On the whole, there have been a large number of studies conducted on or with the

MaCAT-CA, but far fewer that have included or evaluated the ECST-R. There are even

fewer studies that have directly compared the MacCAT-CA to the ECST-R to one

another; therefore, additional research is vital in gaining a better understanding of these

two instruments. Specifically, a study of CST evaluee performance on both tests that

directly compares scores should be conducted to investigate whether the MacCAT-CA

and the ECST-R hold people to the same standard of competence. In one unpublished

simulation study such a comparison was conducted and revealed large differences across

competency classifications across the two instruments (Panza, Celeya, Slavikova &

Preston, 2016). Further research along these lines that includes both competent and
 50

incompetent forensic patients, as well as those with a wide variety of impairments, would

give a much more detailed look at how each instrument functions.

Surveys of test usage, and specifically usage of FAIs, have been conducted in the

past and have generally revealed a low level of usage of CAIs in CST evaluations

(Archer, Buffington-Vollum, Stredny & Handel, 2006; Borum & Grisso, 1995; Grisso,

1991, 2003; Holub, 1992; Neal & Grisso, 2014). Since these surveys were done at least a

decade, if not two decades ago, it would certainly be useful to conduct an updated survey

of clinician use of these instruments. It would be of interest to know not only how many

examiners use the MacCAT-CA and the ECST-R, but also to know what types of cases

they use each test for and whether or not they use the tests in conjunction with any other

instruments.

Finally, there is significant need for more research in particular on the ECST-R’s

clinical utility with different populations. The vast majority of the research available on

the ECST-R was conducted by the test authors themselves, or by the authors in

conjunction with their colleagues. Very few of the published works on the ECST-R are

authored by individuals not directly involved in the test’s development. While this does

not imply that the research available is problematic or biased in any way, it is certainly a

boost to the instruments validity when those outside of the development team can

replicate or expand upon the original findings noted in the test manual. One explanation

for the dearth of literature available outside of that done by the test authors lies in the fact

that the ECST-R’s hybrid design makes it less than optimal for research purposes. The

unstructured interview and the flexible scoring options necessarily lead to inconsistencies

across examiners and examinees making it difficult to generalize any findings or draw
 51

any firm conclusions about the test. Further, these characteristics make the test very

challenging to employ in a tightly controlled research setting. Nevertheless, without

additional research that explores the clinical utility of the ECST-R in a variety of contexts

and populations, it is very difficult to comment on how this test would hold up.

In sum, the MacCAT-CA was published close to 18 years ago and the ECST-R

was published close to 13 years ago. Since their publications, research a fairly large

amount of knowledge has become available to help practitioners decide how best to use

these tests. Given that research which is available there is some foundation to make the

tentative recommendations for the use of these tests in clinical forensic practice that are

offered herein. Despite the evidence in support of using the MacCAT-CA and the ECST-

R, there still appears to be a wide range of practice among evaluators in the field of

forensic psychology and these inconsistencies can have a significant impact on the

outcomes of criminal cases (Neal & Grisso, 2014). Since the process of conducting

forensic assessments is not standardized, clinicians are free to arbitrarily select any

method they prefer when conducting CST evaluations and there does not appear to be a

current consensus among practitioners to use any tool. The question remains why

forensic evaluators would pass up using these tools specifically designed to simplify the

assessment process of CST. Given the high number of evaluators resorting to such a wide

range of practices, one must consider that perhaps they are not convinced of the utility of

standardized forensic instruments such as, the MacCAT-CA and the ECST-R. This is

especially true for forensic experts that view competency as highly dependent on context

whom consider the clinical interview as remaining superior to any standardized tool

(Golding, 2008; Roesch et al., 1996). Perhaps the present study can help guide clinicians
 52

who conduct CST evaluations in determining when the MaCAT-CA and the ECST-R

may be a useful forensic tool to use and in which cases the use of either test might be

indicated.
 53

APPENDIX

TABLES

Table 1. Validation Studies for ECST-R

Sample Setting N Gender (%) Ethnicity (%)

M F Black White Hispanic
Rogers, Grandjean,
Tillbrook, Vitacco Jail 100 100 0 23 67 6
& Sewell (2001)
Clinic/
Tillbrook (2000) 70 100 0 65.7 32.8 0
Hospital
Rogers, Sewell,
Martin, & Jackson Hospital 56 82.1 17.9 44.6 35.7 19.6
(2003)
Rogers, Martin, &
Jail 154 52.1 47.9 22.9 64.6 6.3
Sewell (2003)
Ancillary Data ---- 89 85.4 14.6 31.5 49.4 13.5
TOTALS ---- 469 84% 16% 38% 50% 9%
 54

Table 2. Internal Consistencies of the ECST-R Measure

Rogers, Rogers,
Grandjean, Jackson, Vitacco,
Tillbrook, Sewell, Rogers, Rogers,
Vitacco & Tillbrook Tillbrook, Gabel, & Norton
Tillbrook Sewell & Martin & Sewell Munizza & Ryba
(2000) (2001) (2003) (2004) (2009) (2010)
FAC .92 .90 .87 .87 .85 .76
RAC .75 .86 .89 .89 .91 .74
CWC .33 .72 .83 .83 .92 .87
Rational NA NA .99 .93 .95 .89
ATP-R NA NA 1.00 .63 NA .50
ATP-P NA NA 1.00 .79 NA .74
ATP-N NA NA 1.00 .70 NA .73
ATP-I NA NA 1.00 .87 NA NA
ATP-B NA NA 1.00 .86 NA .82

Table 3. Interrater Reliability among ECST-R Items

Rogers, Rogers,
Grandjean, Jackson, Vitacco,
Tillbrook, Sewell, Rogers, Rogers,
Vitacco & Tillbrook Tillbrook, Gabel, &
Tillbrook Sewell & Martin & Sewell Munizza
(2000) (2001) (2003) (2004) (2009)
FAC .95 .98 .98 .96 .97
RAC .88 1.00 .99 .91 .98
CWC .88 .95 .99 .91 .99
Rational .94 1.00 .99 .96 .98
ATP-R NA .99 1.00 1.00 NA
ATP-P NA 1.00 1.00 1.00 NA
ATP-N NA .96 1.00 .98 NA
ATP-I NA 1.00 1.00 1.00 NA
ATP-B NA .99 1.00 1.00 NA
 55

Table 4. Comparison of MacCAT-CA and ECST-R

MacCAT-CA ECST-R
Number of Items 22 Items 18 Items
Administration Time 25-55 minutes 25 to 45 minutes
Scoring Time 10 minutes 5 minutes
Age Range 18 to 80 years old 18 to 79 years old
IQ Level Greater than 60 Greater than 60
Language English English
Theoretical Basis Bonnie and Dusky standard Dusky standard
Understanding Factual Understanding
Domains and/or
Reasoning Rational Understanding
Competency Scales
Appreciation Consult with Counsel
Structural Format Standardized Semi-structured
Scoring format Structured Semi-structured
Assessment of Defendant’s
Hypothetical Scenario Defendant’s specific Case
ability to consult counsel
Rating Criteria 3-point scale rating 0 to 4 rating criteria
Overall Score No Yes
Type of Score Used Raw scores t-scores
Factor Structure Two or three Three
Adolescent Defendants 16 and 17 only No research available
Defendants with ID No No research available
Gender Differences Not significant Not significant
Race Differences No (controlling for IQ) Not enough research
Retest Format Yes No
Feigning Screen No Yes
Dusky Adherence Questionable Yes
Suitable for Research Yes No
Daubert Criteria Questionable Yes
Ease of Scoring Yes No
Psychosis No Yes
Cognitive Deficits No No research available
 56

REFERENCES

Acklin, M. W. (2012). The forensic clinician’s toolbox I: A review of competency to

stand trial (CST) instruments. Journal of Personality Assessment, 94(2), 220-222.

Akinkunmi, A. A. (2006). Commentary: Competently assessing competence to stand trial

can be trying. The Journal of the American Academy of Psychiatry and the Law,
34, 189-190.

Archer, R. P., Buffington-Vollum, J. K., Stredny, R. V., & Handel, R. W. (2006). A

survey of psychological test use patterns among forensic psychologists. Journal of
Personality Assessment, 87(1), 84-94.

Bennett, A. D. (2005). The measurement of adjudicative competence: A comparison of

three types of competencies in sample of individuals with mental retardation.
Dissertation Abstracts International: Section B: The Sciences and Engineering,
67 (1-B), 529.

Bonnie, R. J. (1992). The competence of criminal defendants: A theoretical reformation.

Behavioral Sciences and the Law, 10, 291-316.

Borum & Grisso (1995). Psychological test use in criminal forensic evaluations.
Professional Psychology: Research and Practice, 5, 465-473.

Boyd, J. C. (1999). The competence-related abilities of juveniles prosecuted in criminal

court. Dissertation Abstracts International, 60, 1294.

Burnett, D. R. (1999). Evaluation of competency to stand trial in a juvenile population.

Dissertation Archive. Paper 1841. http://aquila.usm.edu/thesesdissertations/1841

Burnett, D. M., Noblin, C. D., & Prosser (2004). Adjudicative competency in a juvenile
population. Criminal Justice Behavior, 31(4), 438-462.

Dusky v. United States, 362 U.S. 402, 80 S. Ct. 788 (1960).

Edens, Poythress, Nicholson, & Otto. (1999). Effects of state organizational structure and
forensic examiner training on pretrial competence assessments. The Journal of
Behavioral Health Services & Research, 26(2), 140-150.

Gabel, J. E. (2006). Consistency of screening measures of malingering among pre-trial

adjudicative competence defendants. Dissertation Abstracts International:
Section B: The Sciences and Engineering, 67(9-B), 5400.
 57

Golding, S. L. Evaluation of Adult Adjudicative Competency. Jackson, R. (Ed), (2008).

Learning forensic assessment. International perspectives on mental health, (pp.
75-108). New York, NY, US: Routledge/Taylor & Francis Group, xii, 618 pp.

Golding, S. L., & Roesch, R. (1988). Competency for adjudication: An international

analysis. In D. N. Weisstub (Ed.), Law and mental health: International
perspectives (Vol. 4, pp. 73–109). New York: Pergamon

Grisso, T. (2003). Competence to Stand Trial. In Evaluating competencies: Forensic

assessments and instruments (Second ed., Vol. 16, pp. 69-148). New York, New
York: Plenum Press.

Grisso, T. (1991). Clinical assessments for legal decision making: Research

recommendations. In S. A. Shah & B. D. Sales (Eds.), Law & mental health:
Major developments and research needs (pp. 49-80). DHHS Pub. No. (ADM) 91-
1875. Washington, DC: U.S. Government Printing Office.

Grisso, T., Steinberg, L., Woolard, J., Cauffman, E., Scott, E., Graham, S., Lexcen, F.,
Reppucci, N. D., & Schwartz, R. (2003). Juveniles’ competence to stand trial: A
comparison of adolescents’ and adults’ capacities as trial defendants. Law and
Human Behavior, 27(4), 333-363.

Grisso, T., Vincent, G. M., & Seagrave, D. (Eds.). (2005). Mental health screening and
assessment in juvenile justice. New York, US: The Guilford Press. Retrieved from
http://www.ebrary.com.

Hoge, S. K., Bonnie, R. J., Poythress, N., Monahan, J., Eisenberg, M., & Feucht-Havier,
T. (1997). The MacArthur adjudicative competence study: Development and
validation of a research instrument. Law and Human Behavior, 21, 141-182.

Holub, R. J. (1992). Forensic psychological testing: A survey of practices and beliefs.

Unpublished manuscript, Minnesota School of Professional Psychology,
Bloomington.

Jacobs, M. S., Ryba, N. L., & Zapf, P. A. (2008). Competence-related abilities and
psychiatric symptoms: An analysis of the underlying structure and correlates of
the MacCAT-CA and the BPRS. Law and Human Behavior, 32, 64-77.

Jurecska, D. E. (2010). Competence to stand trial: Special challenges for the populations
diagnosed with intellectual disabilities and borderline intellectual functioning.
Dissertation Abstracts International: Section B: The Sciences and Engineering,
72 (7-B), 4296.

Jurecska, D. E., Peterson, M. A. & Millkey, A. (2012). Comparative study of the

MacCAT-CA and CAST-MR for individuals with intellectual disabilities.
American Journal of Forensic Psychology, 30(2), 67-82.
 58

Krause, M. S. (1998). Methodological and developmental issues in the assessment of

adjudicative competence. (unpublished Ph.D. dissertation, University of
Virginia).

LaVelle Fick, S., Hart, K. J., & Deardorff, P. A. (2006). The performance of incarcerated
juveniles on the MacArthur Competence Assessment Tool-Criminal Adjudication
(MacCAT-CA). The Journal of the American Academy of Psychiatry and the
Law, 34, 360-373.

Lexcen, F. J., Hawk, G. L., Herrick, S., & Blank, M. (2006). The use of video
conferencing for psychiatric and forensic evaluations. Psychiatric Services, 57(5),
713-715.

Lipsitt, P. D., Lelos, D., & McGarry, A. L. (1971). Competency for trial: A screening
instrument. American Journal of Psychiatry, 128, 105–109.

Lyle, V. K. (2003). Emotional maturity and trial competence in urban male adolescents.
Dissertation Abstracts International: Section B: The Sciences and Engineering,
64(3-B), 1498. Publisher: ProQuest Information & Learning; [Dissertation]

Marcus, D. K., Poythress, N. G., Edens, J. F., & Lilienfield, S. O. (2010). Adjudicative
competence: Evidence that impairment in “rational understanding” is taxonic.
Psychological Assessment, 22(3), 716-722.

McIlnay, J. D. (2010). Correlates of defendants found incompetent to stand trial and

second-generation competency measures: A meta-analytic review. Dissertation
Abstracts International: Section B: The Sciences and Engineering, 71(10-B),
6445. Publisher: ProQuest Information & Learning; [Dissertation].

Melton, G. B., Petrila, J., Poythress, N. G., & Slobogin, C. (1997). Psychological
evaluations for the courts: A handbook for mental health professionals and
lawyers (2nd ed.). NY: The Guilford Press.

Mueller, C. & Wylie, A. M. (2007). Examining the effectiveness of an intervention

designed for the restoration of competency to stand trial. Behavioral Sciences and
the Law, 25, 891-900.

Mumley, D. E., Tillbrook, C. E., & Grisso, T. (2003). Five-year research update (1996–
2000): Evaluations for competence to stand trial. Behavioral Sciences and the
Law, 21, 329–350.

Neal, T. M. S. & Grisso, T. (2014). Assessment practices and expert judgment methods
in forensic psychology and psychiatry: An international snapshot. Criminal
Justice and Behavior, 1, 1–16. DOI: 10.1177/0093854814548449

Nicholson, R. A., Briggs, S. R., & Robertson, H. C. (1988). Instruments for assessing
competency to stand trial: How do they work? Professional Psychology: Research
and Practice, 19, 383-394
 59

Nicholson, R. A., & Kugler, K. E. (1991). Competent and incompetent criminal

defendants: A quantitative review of comparative research. Psychological
Bulletin, 3, 355–370.

Norton, K. A. & Ryba, N. L. (2010). An investigation of the ECST-R as a measure of

competence and feigning. Journal of Forensic Psychology Practice, 10, 91-106.

Otto, R. K., Poythress, N. G., Nicholson, R. A., Edens, J. F., Monahan, J., Bonnie, R. J.,
Hoge, S. K., & Eisenberg, M. (1998). Psychometric properties of the MacArthur
Competence Assessment Tool—Criminal Adjudication. Psychological
Assessment, 10, 435-443.

Overall, J. E., & Gorham, D. R. (1962). The Brief Psychiatric Rating scale. Psychological
Reports, 10, 799–812.

Panza, N. R., Slavikova, M., Celeya, K., & Preston, K. (2016). The MacCAT-CA and
ECST-R: Instrument review, additional validation data, and recommendations for
use in competency to stand trial evaluations. Unpublished Manuscript.

Pinals, D.A., Tillbrook, C.E., & Mumley, D.L. (2006). Practical application of the
MacArthur competence assessment tool-criminal adjudication (MacCAT-CA) in a
public sector forensic setting. The Journal of the American Academy of Psychiatry
and the Law, 32(2), 179-188.

Pirelli, G., Gottdiener, W. H., & Zapf, P. A. (2011). A meta-analytic review of

competency to stand trial research. Psychology, Public Policy, and Law, 17, 1-53.

Poythress, N. G., Nicholson, R., Otto, R. K., Edens, J. F., Bonnie, R. J. Monahan, J., &
Hoge, S. K. (1999) The MacArthur Competence Assessment Tool: Criminal
Adjudication: MacCAT-CA. Odessa, FL: PAR.

Redding, R. E. (1997). Depression in jailed women defendants and its relationship to

their adjudicative competence. The Journal of the American Academy of
Psychiatry and the Law, 25(1), 105-119.

Roesch, R., Hart, S. D., & Zapf, P. A. (1996). Conceptualizing and assessing competency
to stand trial: Implications and applications of the MacArthur treatment
competence model. Psychology, Public Policy, and Law, 2, 96-113.

Roesch, R., Zapf, P. A., Golding, S. L., & Skeem, J. L. (1999). Defining and assessing
competency to stand trial. In A.L. Hess & I.B. Weiner (Eds.), The handbook of
forensic psychology (pp. 327-367). New York: John Wiley & Sons, Inc.

Rogers, R., Grandjean, N., Tillbrook, C. E., Vitacco, M.J., & Sewell, K. W. (2001).
Recent interview-based measures of competency to stand trial: A critical review
augmented with research data. Behavioral Sciences and the Law 19, 503-518.
 60

Rogers, R., Jackson, R. L., Sewell, K. W., & Harrison, K. S. (2004). An examination of
the ECST-R as a screen for feigned incompetency to stand trial. Psychosocial
Assessment, 16, 139-145.

Rogers, R., Jackson, R., Sewell, K., Tillbrook, C., & Martin, M. (2003). Assessing
dimensions of competency to stand trial: Construct validation of the ECST-R.
Assessment, 101, 344-351.

Rogers, R. & Johansson-Love, J. (2009). Evaluating Competency to Stand Trial with

Evidence-Based Practice. The Journal of the American Academy of Psychiatry
and the Law, 37, 450–60.

Rogers, R., Sewell, K. W., Grandjean, N. R., & Vitacco, M. J. (2002). The detection of
feigned mental disorders on specific competency measures. Psychological
Assessment, 14, 177-183.

Rogers, R., Tillbrook, C., & Sewell, K. (2004). Evaluation of Competency to Stand Trial-
Revised (ESCT-R): Professional manual. Lutz, FL: Psychological Assessment
Resources.

Rosenfeld, B. (2001). Measure Development in Forensic Psychology. Hoboken, New

Jersey: Wiley.

Ryba, N. L. & Zapf, P. A. (2011). The influence of psychiatric symptoms and cognitive
abilities on competence-related abilities. International Journal of Forensic Mental
Health, 10, 29-40.

Springman, R. E. (2009). The detection of malingering on measures of competency to

stand trial: A study of coached and uncoached simulators. Dissertation Abstracts
International, 69, 5059.

Springman, R. E., & Vandenberg, B. R. (2009). The effects of test-strategy coaching on

measures of competency to stand trial. Journal of Forensic Psychology Practice,
9(3), 179-198.

Tillbrook, C. E. (2000). Competency to proceed: A comparative appraisal of approaches

to assessment. Dissertation Abstracts International: Section B: The Sciences and
Engineering, 61(9-B), 5009.

Viljoen, J. L., Odgers, C., Grisso, T., & Tillbrook, C. (2007). Teaching adolescents and
adults about adjudicative proceedings: A comparison of pre- and post- teaching
scores on the MacCAT-CA. Law and Human Behavior, 31, 419-432.

Viljoen, J. L., Slaney, K. L., & Grisso, T. (2009). The use of the MacCAT-CA with
adolescents: An item response theory investigation of age-related measurement
bias. Law and Human Behavior, 33, 283-297.
 61

Vitacco, M. J., Rogers, R., Gabel, J. & Munizza, J. (2009). An investigation of the ECST-
R in male pretrial patients evaluating the effects of feigning on competency
evaluations. Assessment, 16(3), 249-257.

Vitacco, M. J., Rogers, R., Gabel, J. & Munizza, J. (2007). An evaluation of malingering
screens with competency to stand trial patients: A known-groups comparison.
Law Human Behavior, 31, 249-260.

Warren, J. I., Aaron, J., Ryan, E., Chauhan, P., & DuVal, J. (2003). Correlates of
adjudicative competence among psychiatrically impaired juveniles. The Journal
of the American Academy of Psychiatry and the Law, 31, 299-309.

Zapf, P. A. (1999). An investigation of the construct of competence in a criminal and

civil context: A comparison of the FIT, the MacCAT-CA, and the MacCAT-T.
Dissertation Abstracts International: Section B: The Sciences and Engineering,
60(6-B), 3012. Publisher: ProQuest Information & Learning; [Dissertation]

Zapf, P. A., & Roesch, R. (2001). A comparison of the MacCAT-CA and the FIT for
making determinations of competency to stand trial. International Journal of Law
and Psychiatry, 24, 81-92.

Zapf, P. A., & Roesch, R. (2005). An investigation of the construct of competence: A

comparison of the FIT, the MacCAT-CA, and the MacCAT-T. Law and Human
Behavior,29(2), 229-252.

Zapf, P. A., & Roesch, R. (2009). Evaluation of competence to stand trial. Oxford, New
York: Oxford University Press.

Zapf, P. A., Skeem, J. L., & Golding, S. L. (2005). Factor structure and validity of the
MacArthur Competence Assessment—Criminal Adjudication. Psychological
Assessment, 17(4), 433-445.