Journal Pre-proof

Dynamic Evaluation of Motor Speech Skill: Adaptation for Brazilian

Portuguese

Marileda Barichelo Gubiani , Karina Carlesso Pagliarin ,

Rebecca J. McCauley , Márcia Keske-Soares

PII: S0021-9924(21)00037-X
DOI: https://doi.org/10.1016/j.jcomdis.2021.106114
Reference: JCD 106114

To appear in: Journal of Communication Disorders

Received date: 11 April 2019

Revised date: 7 May 2021
Accepted date: 15 May 2021

Please cite this article as: Marileda Barichelo Gubiani , Karina Carlesso Pagliarin ,
Rebecca J. McCauley , Márcia Keske-Soares , Dynamic Evaluation of Motor Speech Skill:
Adaptation for Brazilian Portuguese, Journal of Communication Disorders (2021), doi:
https://doi.org/10.1016/j.jcomdis.2021.106114

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition
of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of
record. This version will undergo additional copyediting, typesetting and review before it is published
in its final form, but we are providing this version to give early visibility of the article. Please note that,
during the production process, errors may be discovered which could affect the content, and all legal
disclaimers that apply to the journal pertain.

© 2021 Published by Elsevier Inc.

 Highlights

 A test for childhood apraxia of speech in Brazilian Portuguese is required

 The adaptation of the English language test was made for Brazilian

Portuguese

 Adapting a test from one language to another requires several steps

 These steps must preserve the content's validity and cultural/linguistic

adequacy

 Procedures and results are included in this new test in Brazilian Portuguese

1
Dynamic Evaluation of Motor Speech Skill: Adaptation for Brazilian Portuguese

Marileda Barichelo Gubiani¹

Karina Carlesso Pagliarin¹

Rebecca J. McCauley2

Márcia Keske-Soares¹*

¹ Universidade Federal de Santa Maria – UFSM – Santa Maria (RS), Brazil.

2
The Ohio State University – OSU – Columbus (Ohio), United States.

Research conducted in the Department of Speech-Language and Hearing of

Universidade Federal de Santa Maria – UFSM - Santa Maria (RS), Brazil.

Financial support: none

Conflicts of interest: none

*Correspondence address: Márcia Keske-Soares

RST 287, 900 – Bairro Cerrito - Santa Maria-RS-Brazil

Zip Code: 97060-448

Email: marcia-keske.soares@ufsm.br

Acknowledgements

The authors would like to thank to Dr. Edythe Strand and Dr. Rebecca J.

McCauley for agreeing to let us adapt the Dynamic Evaluation of Motor Speech

Skill (DEMSS) to Brazilian Portuguese (BP).

2
 The authors would like to thank all children and experts that contributed with

this study. In addition, we wish to thank the Foundation for Support to the Research

of Rio Grande do Sul State (Fundação de Amparo à Pesquisa do Estado do Rio

Grande do Sul – FAPERGS) for the doctoral scholarship (MBG) and postdoctoral

fellowship (KCP).

Abstract

Purpose: To describe the adaptation of verbal tasks (words) in the DEMSS (Strand et al . , 2013; Strand
& McCauley, 2019) for subsequent inclusion in an analogous instrument in BP.

Methods: The adaptation process consisted of six steps. Step 1: Three professionals carried out the
translation and back-translation of the test’s instructions and background content. Step 2: Two
speech-language pathologists (SLP) with expertise in speech-language selected new stimuli for the
instrument to make it appropriate for BP. Step 3: Seven expert judges determined the adequacy of
test stimuli. Step 4: Eight non-expert judges, children with typical speech development, indicated
whether the stimulus words were part of their vocabulary. Step 5: the instrument was administered
in 20 children with typical speech development (pilot sample). The results of steps 3, 4 and 5 were
examined using Content Validity Ratio. Step 6: Administration of the BP version of the DEMSS in one
case of CAS.

Results: 269 words were selected by the expert SLP (Step 2). These words were submitted to
evaluation for expert judges (Step 3) and 96 of them were considered adequate. These items were
then submitted to the child judges (Step 4) to evaluate their knowledge and use of the words, and in
the pilot sample (Step 5) to evaluate the production accuracy of a larger group of children. A total of
44 words were selected after analysis of the results of Steps 4 and 5. In Step 6 the patient completed
the final version of the BP version of the DEMSS to determine the feasibility of its use in young
children with CAS.

Conclusion: The translation, back-translation and evaluations by nativespeaking expert judges during
the cross-cultural adaptation process and the application in one children with CAS demonstrate the
content validity of the adapted instrument. Then, the BP version of the DEMSS has adequate content
validity for the assessment of motor speech skills.

1. Introduction

Childhood apraxia of speech (CAS) is a disorder characterized by impairments

in speech motor planning and/or programming (American Speech-Language-Hearing

Association [ASHA], 2007; Morgan & Murray, 2017; Strand et al., 2013). Its main

features include alterations in speech articulation and prosody, which can persist

3
throughout growth and development and cause impairments in academic, social and

occupational functioning (Murray et al., 2012). CAS can be caused by

neurodevelopmental (ASHA, 2007; Liégeois et al., 2016), genetic (ASHA, 2007,

Murray et al., 2015) or idiopathic factors (ASHA, 2007; Murray et al., 2012; Murray et

al., 2015).

The estimated prevalence of idiopathic CAS is 2.4% (Shriberg et al., 2019a).

However, the prevalence of CAS with a known neurodevelopmental cause varies

according to the associated disorder. In a recent study (Shriberg et al., 2019b), the

condition was identified in 11.1% of cases of Down Syndrome, 6.5% of patients with

galactosemia, 8.7% in individuals with idiopathic intellectual disability and 11.8% in

22q11.2 Deletion syndrome. The same study found a prevalence of 0% in patients

with autism spectrum disorder.

There is no consensus in the literature regarding the clinical characteristics of

CAS (ASHA, 2007; Forrest, 2003; Malmenholt et al., 2017). Nevertheless, the

identification and discussion of these characteristics are essential for treatment

planning and for the differential diagnosis of CAS and other common developmental

(phonological and neuromuscular) disorders (Ziegler et al., 2012). Expert opinion

remains the gold standard for the diagnosis of CAS (Forrest, 2003; Maas et al., 2012;

Murray et al., 2015). However, this method is inherently subjective and may result in

significant variability in diagnostic criteria for the disorder (Forrest, 2003; Murray et

al., 2015). To ensure an accurate diagnosis, the assessment of suspected CAS

should rely on both expert opinion as well as more objective methods (Gubiani et al.,

2015).

According to the American Speech-Language-Hearing Association (ASHA,

2007), the primary symptoms of CAS are as follows: inconsistent consonant and

4
vowel errors (during the production of both syllables and words), disrupted

coarticulatory transitions between sounds and syllables, as well as inappropriate

prosody, especially with regard to lexical and phrasal stress. These features should

receive special attention during individual assessments and are frequently cited in

studies of English-speaking samples (ASHA, 2007; Liégeois et al., 2015; Morgan &

Murray, 2017; Murray et al., 2012; Murray et al., 2015; Strand et al., 2013). Some

studies of CAS in Brazilian Portuguese (BP) speakers also rely on these diagnostic

features (Mezzomo et al., 2011; Navarro et al., 2018; Payão et al., 2010; Souza &

Payão, 2008).

Few instruments have shown satisfactory validity and reliability in the

assessment and diagnosis of CAS, as evidenced by the literature in English

(Blakeley, 2001; Hayden & Square, 1999; Kaufman, 1995; Murray et al., 2020; Power

et al., 2010; Strand & McCauley, 2019), Italian (Bearzotti et al., 2007) and BP

(Gubiani et al., 2015). This is especially problematic in Brazil, where no standardized

instruments are currently available for the assessment of CAS. Clinicians and

researchers working with English-speaking children with CAS are privileged to have

access to numerous standardized tests as well as measures predominately used in

research.

Further examination of measures available in English provides a basis for

selection of a measure to adapt for BP. In a 2008 review, McCauley and Strand used

operational definitions to examine six published tests of nonverbal oral and speech

motor performance in children for evidence of reliability and validity and adequacy of

norms or behavioral criteria on which to base test interpretation. Five of the

measures they evaluated focused primarily on speech motor performance: The

Apraxia Profile (Hickman, 1997), the Kaufman Speech Praxis Test (Kaufman, 1995),

5
the Screening Test for Developmental Apraxia of Speech-Second Edition (Blackeley,

2001); the Verbal Dyspraxia Profile (Jelm, 2001), and the Verbal Motor Production

Assessment for Children (VMPAC; Hayden & Square, 1999). Of these tests, only the

VMPAC fully met the operational definitions for any of the psychometric

characteristics considered; the others either provided no information or inadequate

detail needed to establish adequacy of reliability, validity, and test interpretation

methods. Further, the VMPAC only met the operational definitions for its norms and

evidence of content validity.

Another standardized test that was not included in the 2008 review is the

Diagnostic Evaluation of Articulation and Phonology – DEAP (Dodd et al., 2002), a

standardized instrument based on the theoretical model proposed by Dodd (1995,

2005). The DEAP is comprised of two screening measures--a diagnostic screen and

oral motor screens--as well as three assessments: articulation, phonology, and word

inconsistency. The word inconsistency assessment was developed to identify

children with greater than 40% inconsistency in errors on repeated production of

target words to aid in the diagnosis of either CAS or Inconsistent Disorder. Its

standardization was conducted first in the United Kingdom and Australia, then it was

adapted for use in the US where additional norms were obtained. Although the

DEAP has not yet included in a formal published review, its manual reports

considerable evidence of reliability and validity for the test as a whole and for several

of its components. However, it provides no evidence regarding reliability and validity

for the test of inconsistency, making its value to differential diagnosis doubtful.

In addition to measures published as standardized tests, some individual

measures and protocols have been developed and used primarily in the context of

research. In a 2015 review, Gubiani et al. examined five assessment tools for

6
possible use or adaptation for use with children speaking BP that included both

standardized tests and research measures. Specifically, the reviewed assessments

were the VMPAC, the Dynamic Evaluation of Motor Speech Skill (DEMSS, Strand et

al., 2013; Strand & McCauley, 2019), the Kaufman Speech Praxis Test, Orofacial

Praxis Test (Bearzotti et al., 2007) and the Madison Speech Assessment Protocol

(MSAP; Shriberg et al., 2010). Of these measures, the MSAP had primarily been

developed for research purposes, whereas the others had been intended for clinical

as well as possible research uses. Despite acknowledging positive psychometric

characteristics of the MSAP, Gubiani et al. identified the DEMSS as most suitable tool

for adaptation to BP because of its well-defined protocol and evidence of reliability

and validity (Strand et al., 2013).

The DEMSS was developed to examine articulatory accuracy, vowel accuracy,

prosodic accuracy (lexical stress), and consistency in children suspected of CAS,

characteristics that had been included in the ASHA list of discriminative features for

CAS (2007). Incorporation of dynamic assessment methods in the DEMSS was also

designed to facilitate evaluation of the severity of the child‘s speech disorder as well

as responsiveness to the use of tactile, gestural and temporal cues that could be

incorporated in intervention. Strand et al. (2013) reported the results of a study

examining the measure‘s intrajudge, interjudge and re-test reliability as well as its

construct validity for differentiating CAS and speech sound disorders for children from

3 to 6 years, 7 months. In addition to agreement measures of about 90% for each

type of reliability that was examined, positive and negative likelihood ratios as well as

measures of sensitivity and specificity suggested the diagnostic utility of the

measure. A manual for the DEMSS to expedite its clinical use was published (Strand

& McCauley, 2019).

7
 In a recent review, Murray et al. (2020) conducted a systematic review

examining measures for use in the differential diagnosis of childhood apraxia of

speech compared to other speech disorder. Unlike the reviews of McCauley and

Strand (2008) and Gubiani et al. (2015), Murray et al. focused their systematic review

only on articles reporting results of studies examining potential discriminative features

or diagnostic markers described in the research literature; test manuals were not

included among the reviewed sources. Nonetheless, the DEMSS was examined due

to the research by Strand et al. (2013). Although that reliability and validity study

received some criticism by the Murray and colleagues, it was the only measure that

has also been published for clinical use (Strand & McCauley, 2019) that was

mentioned by Murray et al. as a credible component of a diagnostic protocol for the

differential diagnosis of CAS from other speech sound disorders.

Given this literature suggesting the value of the DEMSS for use in the

assessment and differential diagnosis of severe speech disorders in English-

speaking children aged 3 years to 6 years, 7 months and the absence of similar

instruments for children speaking BP, the DEMSS was selected as a basis for the

meticulous process required for effective adaptation. The construction, adaptation

and validation of an assessment instrument begins with research-based item

development, followed by systematic observation and behavior analysis (Astepe &

Köleli, 2019; Pernambuco et al., 2017). This process must be rigorously followed and

consider factors such as the linguistic and cultural features of the country where the

instrument is being developed.

Throughout the adaptation process, the similarity between the original

instrument and its adapted version must be preserved to ensure its objectives and

methods are maintained (Chourdakis et al.¸ 2019; Pernambuco et al., 2017). At the

8
same time, it is crucial that the linguistic and cultural aspects of the instrument be

carefully considered, especially in the context of speech assessment (Chourdakis et

al.¸ 2019; Pernambuco et al., 2017). Given the influence of language and culture on

the evaluation of speech disorders, the cross-cultural adaptation of language

instruments must extend beyond the literal translation of the instrument in question.

In fact, translated stimuli would almost certainly be unsuitable for use since their

sound structure would fail to assess the target skills of the original instrument.

Language and culture influence the entire assessment process, from test

administration to the responses of participants and their interpretation by the

examiner (Chourdakis et al.¸ 2019; Pernambuco et al., 2017).

Therefore, the aim of this study was to describe the adaptation of procedures

and verbal tasks (words) in the DEMSS (Strand et al., 2013; Strand & McCauley,

2019) for subsequent inclusion in an analogous instrument in BP. The results of this

process will provide strong initial evidence of the content validity of this new

instrument.

2. Methods

2.1. Ethical concerns

The Research Ethics Committee of the University approved this study under

protocol number 437.023. All adult participants provided written informed consent,

while parents or guardians authorized their child‘s participation in the study.

Additionally, all children provided oral assent to participate in the procedures.

Since this study involved the adaptation of an assessment instrument

originally published in another country, its authors were contacted for permission to

translate the instrument into BP and invited to participate in every step of the
9
adaptation process. While only one of the authors (RM) was able to participate in the

study, both authorized the adaptation of the assessment instrument.

2.2. Participants

Different participants were involved in each step of the adaptation process.

The study involved samples of translators (Step 1), experts (Step 2), expert judges

(Step 3), child judges (Step 4), a pilot sample of children (Step 5), and one child with

CAS (Step 6).

2.3. Material

The English version of the DEMSS (Strand et al., 2013; Strand & McCauley,

2019) has eight subtests containing words with varying syllable structures and levels

of phonetic complexity. The structure of the original version of the test (Strand, et al.,

2013) was used as a basis for the selection of corresponding stimuli for the BP

version of the DEMSS. The subtests of the original version of DEMSS are presented

in Table 1.

INSERT TABLE 1

The original instrument evaluated motor speech skills based on word

repetition. The examiner says each word out loud then asks the child to repeat it, with

no additional cues. If the child produces the target word accurately twice, the

examiner continues to the next stimulus. If any errors are observed, the clinician may

repeat the word and elicit additional repetition attempts with various levels of visual,

temporal, tactile and kinesthetic cues. Each word can be repeated for a maximum of

four times to help the child achieve the best possible performance. After all cued trials

10
are completed, a final repetition is prompted. Each item is given a multidimensional

score based on the number of attempts and cues involved.

The following subscores are available in the DEMSS: overall articulatory

accuracy (precision), consistency, vowel production accuracy, and prosody (Strand et

al., 2013; Strand & McCauley, 2019). The scoring system for the original DEMSS

(Strand et al., 2013) is presented in Table 2.

INSERT TABLE 2

2.4. Adaptation Process

The adaptation of the DEMSS (Strand et al., 2013; Strand & McCauley, 2019)

into BP was conducted as described in Fonseca et al. (2011) and Maillart et al.

2012). The six steps of this process are described in the following sections.

Step 1. Translation and back translation of administration and scoring instructions

The DEMSS protocol (Strand et al., 2013; Strand & McCauley, 2019)

containing instructions regarding administration and scoring was independently

translated by two individuals familiar with the topic of study (one bilingual Speech-

Language Pathologist (SLP) and one bilingual SLP undergraduate student residing in

the United States). The two translations were compared to create a consensus

version, henceforth referred to as the BP version of the adapted instrument, or the

BP version of the DEMSS. The BP version of the protocol was then back-translated

into English by a bilingual English teacher unfamiliar with the topic of study. The

resulting translation was subsequently evaluated by one of the authors of the original

DEMSS. Throughout the translation and back translation processes, the researchers

sought to preserve the meaning of administration and scoring instructions of the

11
original DEMSS in the BP version of the instrument. This included addressing any

cultural differences that might have negatively affected the validity of the instrument if

a less stringent translation process had been conducted.

Step 2. Development of BP stimuli

This step involved two SLPs with doctoral degrees and expertise in speech

and language development. Both are authors of the present study. Since English and

BP differ sharply in terms of their word structure and order of phoneme acquisition,

these professionals developed new stimuli for the BP version of the DEMSS (Strand

et al., 2013; Strand & McCauley, 2019).

In BP there aren‘t words frequency studies, so the decision was to consider

the vocabulary and phoneme acquisition, considering different BP studies that

investigate these aspects on children‘s development. Candidate word for the BP

version of the DEMSS were selected based on previous research and available

instruments for vocabulary assessment in BP-speaking children (Bastos et al., 2004;

Capovilla et al., 2011; Pedromônico et al., 2002). To increase the range of consonant

sounds examined by the instrument, a BP dictionary was also used.

Potential stimuli were chosen based on the following criteria: order of

phoneme acquisition in BP (Ceron et al., 2017); ease of production;

representativeness (i.e., real words); and presence in young children‘s vocabularies

(Bastos et al., 2004; Capovilla et al., 2011; Pedromônico et al., 2002).

The syllable structures emphasized in each subtest of the DEMSS (Strand et

al., 2013; Strand & McCauley, 2019) were also examined to determine whether each

subtest should be retained in the instrument. In the original English version of the

DEMSS, the target words focus mostly on early-developing sounds, but also contain

12
a number of later-developing sounds. In the BP version of the DEMSS early-

developing sounds (e.g., /t/, /d/, /s/, /k/) (Ceron et al., 2017) were prioritized since

they can be sensitive measures to detected SSD Portuguese-speaking children.

Step 3. Expert Judges

This step involved seven expert judges, five doctoral level SLPs, one doctoral

level linguist as well as an author of the original instrument. Because the latter was

not fluent in BP, their assessment was based on the phonetic structure of each word

as reflected by its phonetic transcription. The sample of judges included the authors

of the present study as well as researchers with clinical and/or research experience

with the contents of the test, who were invited to participate in this step of the study

via e-mail. The invitation provided information regarding the goals of the study and

the procedures involved, in addition to a description of the evaluation required.

Participants who spoke BP were asked to rate 269 words based on the

syllable structures emphasized in each subtest of the BP version of the DEMSS.

Raters were asked to select the most appropriate words based on the following

variables: phoneme acquisition in BP, representativeness, and presence in young

children‘s vocabularies. Each word was to be given one of the following scores: 1

(appropriate), 2 (partially appropriate), 3 (inappropriate). Data from this step of the

study were analyzed using the Content Validity Ratio – CVR (Lawshe, 1975), as

described under the Data Analysis section. These results were used to select words

for the BP version of the DEMSS from the original set of candidate words.

Step 4. Child Judges

13
 This step consisted of eight child judges (3 female and 5 male; aged 3-10

years) with typical speech and language development. Children were recruited from

public and private schools in the city where the study was conducted. The absence of

speech and language difficulties was endorsed by parents and confirmed by a SLP

based on the observation of a spontaneous conversation.

Children were selected based on an initial interview with a parent or guardian

and an assessment session with one of the researchers. The interview examined

pre-, peri- and postnatal variables, as well as aspects of the children‘s motor and

language development. A skilled clinician informally assessed receptive language,

expressive language, and speech production skills in play, after ensuring the parent

didn‘t report challenges and there are no more risk factors.

After confirmation of eligibility, the analysis by child judges began. The

purpose of this procedure was to confirm that the words being considered for

inclusion in the BP version of the DEMSS were part of the children's vocabulary. The

authors believed that children‘s familiarity with the words, in addition to typical

phonological development, would be important requirements for accurate repetition.

For the analysis, the children were presented with the words rated as

appropriate in Step 3 and asked to indicate the meaning of each one as it was read

by the examiner. Answers were elicited using the following prompt: "What is (target

word)?". If the child provided an adequate answer using words or gestures, the

examiner would consider them to be familiar with the item in question. They would

indicate this by writing ―Yes‖ next to the item. These results were analyzed using

Content Validity Ratio (Lawshe, 1975), as described in the Data Analysis section.

The results of this process were used to select the words for subsequent

testing in a Pilot Sample.

14
Step 5. Pilot Study

The purpose of this step was to confirm the feasibility of the BP version of the

DEMSS and identify any additional words that should be omitted from the final

instrument. A pilot DEMSS in BP version was administered to 20 children (age 3;5-

6;7 years; 8 boys;12 girls) recruited and tested in a local public school. All

participants were typically developing, monolingual BP speakers. Although 34

children were initially recruited, only 20 were ultimately eligible for participation.

Initially, two questionnaires concerning each child were given to parents and

teachers, respectively. The parent questionnaire examined variables such as

pregnancy, birth, neuropsychomotor development, language development, and

learning. Items in the teacher questionnaire were divided into three domains: a)

classroom behavior; b) group participation; and c) attitudes toward authority. These

questionnaires were used to confirm children‘s eligibility for the pilot study. Both

instruments have been previously used for this purpose by the researchers (Ceron, et

al., 2018). Children with known neurodevelopmental disorders (Down Syndrome,

Autism Spectrum Disorder, etc.) were excluded from participation in the pilot study, as

were those described by parents or teachers as unwilling to comply with structured

tasks.

Additional data related to inclusion criteria were collected in two subsequent

assessment sessions, conducted individually with each child in a quiet environment

by an SLP (first author of this study). Each session lasted approximately 45 minutes.

The following tests were administered to each participant:

- Child Naming Test (Teste Infantil de Nomeação – TIN) (Seabra et al., 2012):

In this test, the participant is asked to name 60 sequentially presented images. The

test evaluates expressive language and long-term memory retrieval.

15
 - Phonological Assessment Instrument (Instrumento de Avaliação Fonológica -

INFONO) (Ceron et al., 2018): This computerized instrument evaluates the

production of all phonemes in BP, in all possible syllable and word positions, using

animated images.

- Auditory Vocabulary Test (AudVT 33o) (Capovilla et al., 2011): This test

evaluates the development of receptive vocabulary in children. The AudVT comprises

33 sets of images, each containing one target item and four distractors. For each set

of images, the child is asked to point to the one corresponding to a word read by the

examiner.

- Orofacial myofunctional evaluation protocol (Felício & Ferreira, 2008): This

protocol was used to screen for muscular and functional disorders, and to determine

the presence and degree of phonological and articulatory impairment.

- Auditory screening: A portable Interacoustics-AD 229 audiometer was used

to deliver pure tones of 500, 1000, 2000 and 4000-Hz to both ears. This evaluation

was used to calculate pure tone averages and ensure they fell within the normal

range.

All these procedures were performed to ensure that children in the final pilot

sample met inclusion criteria for the study. Eligibility was confirmed by assessments

of speech and language development. Participants found to be typically developing

remained in the sample, while 14 of the original 34 participants were excluded: 5

children due to atypical development according to the initial assessments; 3 children

due to hearing impairment; 2 children changed schools and were unavailable for

testing; and 4 children did not complete the assessment. This final group were 3

years old and refused to complete the tasks on three separate occasions.

16
 The final pilot sample included 12 girls and 8 boys who met the inclusion

criteria and therefore completed the pilot version of BP version of DEMSS. The aim

of this procedure was to determine whether the children would be able to produce the

words in the instrument, quickly and adequately, using the same guidelines as those

of the English version of the DEMSS (Strand et al., 2013; Strand & McCauley, 2019).

In the assessment, the SLP read each word aloud and asked the child to repeat it.

Next to each item, examiners wrote down ―Yes‖ or ―No‖ to indicate whether the child

was able to pronounce the word correctly. If the child did so correctly, with no cues of

any kind or any additional repetition by the examiner, the examiner would write down

―Yes‖. If cueing or additional repetitions of the stimulus word were required for the

child to reproduce the item correctly, the examiner would write down ―No‖. Any

substitutions or synonyms produced by the children, as well as any visual, auditory,

tactile and/or kinesthetic cues provided were also noted by the examiner next to each

item. Assessment sessions were filmed for later use during the selection of words to

compose the final instrument. These results were analyzed using Content Validity

Ratio (Lawshe, 1975), as described in the Data Analysis section.

Step 6. Administration of the BP version of the DEMSS in one case of CAS

A five-year-old native speaker of BP with suspected CAS was administered the

same instruments as the pilot sample (Step 5), in two one-hour sessions at the clinic.

He had experienced postnatal anoxia, but had no structural anomalies in speech

organs, dysarthric symptoms or visual/hearing impairments. The results of his

evaluation are presented in Table 3. His performance on all measures except for the

BP version of the DEMSS was used by two SLPs to diagnose the child with CAS

according to the criteria specified by Davis et al. (1989) and Shriberg et al. (2009).

17
Subsequently, the patient completed the final version of the BP version of the

DEMSS to determine the feasibility of its use in young children with CAS.

INSERT TABLE 3

2.5. Data Analysis

Different procedures were used to analyze the data from each step of the

study. In Step 1, qualitative methods were used to compare the original and back-

translated versions of the DEMSS protocol (administration and scoring instructions

(Strand et al., 2013; Strand & McCauley, 2019). The selection of new stimuli in Step 2

was based on the descriptive analysis of expert judgments.

In Steps 3 to 5, a Content Validity Ratio (Lawshe, 1975) was calculated for

each test stimulus to quantify the degree of consensus across each group of

participants (expert judges, child judges, and SLPs) (Steps 3-5) regarding the

adequacy of each item. The Content Validity Ratio has been used for this purpose in

previous studies and has proved to be effective in the assessment of reliability

(Bastilha et al., 2020; Bonini & Keske-Soares, 2018; Tohidast et al., 2019;). Content

validity‫‏‬ratio (CVR) (Cohen, 2014; Lawshe, 1975; Paccio & Hutz, 2015) is used to

select the most important and correct content in an instrument. In this way, the

experts are requested to specify whether an item is necessary for operating a

construct in a set of items or not. To this end, they are requested to score each item

from 1 to 3 with a three-degree range of ―not necessary, useful but not essential,

essential”, respectively. The higher score indicates further agreement of members of

the panel on the necessity of an item in an instrument

Content Validity Ratio was calculated using the following formula: CVR=(ne–

N/2)/(N/2), where ne corresponds to the number of judges who rated the item as

18
adequate, and N represents the total number of judges. The Content Validity Ratio

was preferred over simple percent agreement because the latter does not account for

measurement error or random chance and may therefore inflate estimates of

consistency. Simple percent agreement was only used when the Content Validity

Ratio could not be calculated.

Acceptable Content Validity Ratio values for different numbers of raters are

shown in Table 4. These values were initially proposed by Lawshe (1975). Values

equal to or greater than 0.8 were considered acceptable. A Content Validity Ratio of 1

was the main criterion for the inclusion of words in Steps 3 to 5.

INSERT TABLE 4

The Content Validity Ratio of candidate words selected in Steps 3 and 4 were

examined by the authors of the BP version of the DEMSS to decide on the final list of

stimuli for the test. When choosing between multiple words with the same Content

Validity Ratio, the following factors were used as tie-breakers: 1) place of articulation

of constituent phonemes; 2) presence of different types of sounds; and 3) nasal and

oral vowels. In other words, the researchers sought to ensure that the final set of

stimuli would contain sounds that varied in both place and manner of articulation,

showed a balance of oral and nasal sounds, and incorporate phonemes typically

acquired by age 3;0.

After the Content Validity Ratio analysis, the concordance between expert

judges (Step 3) was calculated for all words using Gwet‘s AC1 (Gwet, 2014). This

metric was interpreted based on the criteria provided by Landis and Kochs (1977):

>0.8 = excellent; 0.61–0.8 = substantial; 0.41–0.6 = moderate; 0.21 –0.4 = fair; and

<0.2 = poor.

3. Results
19
 The translation and back-translation processes carried out in Step 1 were

assisted by native speakers of both English and BP. This helped ensure that the

administration and scoring instructions of the original DEMSS were adequately

represented in the BP version of the DEMSS.

Step 2 involved the selection of word stimuli that fit the phonemic, syllable and

word structure of BP. The first step in this process was an assessment of the syllable

structures of each subtest in the original DEMSS (Strand et al., 2013; Strand &

McCauley, 2019), since the syllable structures produced by young speakers are often

similar across languages. The original DEMSS (Strand et al., 2013; Strand &

McCauley, 2019) contained eight subtests. Based on a careful analysis by expert

SLPs, six of the eight subtests were maintained in the BP version and one subtest

was added, as explained below.

Monosyllabic (Vowel-Consonant - VC) words, which are assessed in one

subtest from the English DEMSS, are not common in BP (e.g., ar /ar/, ―air‖; as /as/,

―definite article, female, plural‖). These words are also fairly rare in the vocabulary of

young children and cannot be easily represented as pictures. Lastly, the /r/ tap which

occur in monosyllabic words in BP in final word position (coda), have not been

acquired by age 3;0. This subtest was therefore replaced by one which evaluates

Monosyllabic Diphthongs (Vowel-Vowel, VV), for example, ai (/aj/; ―sad or pain

interjection‖), ui (/uj/; ―sad or pain interjection‖). This was suggested by one of the

authors of the original instrument based on the structure of words in BP. Similar

issues were observed in the monosyllabic (C1VC1) word subtest which assessed

words with a single vowel nucleus, with the same consonant sound in initial and final

word position (onset and coda). This is not a common word structure in BP, and as

such, this subtest was also excluded from the BP version of the DEMSS.

20
 The final list of possible word stimuli contained 269 words: 260 contained both

consonants and vowels and 9 contained only vowels. Some examples of the words

included in each subtest of the BP version of the instrument are as follows: (1)

Monosyllabic (CV), for example: mãe (/mãj/; ―mom‖), pai (/paj/; ―dad‖); (2) Bisyllabic-

Reduplicated Syllables (C1V1C1V1), for example: papa (/‘pap/; ―pope‖), vovô (/vo‘vo/;

―grandfather‖); (3) Monosyllabic (C1VC2) with different consonants, for example: dez

(/dƐs/; ―ten‖), pés (/pƐs/; ―foot‖); (4) Bisyllabic (C1V1C1V2) with same consonants and

different vowels, for example: pipa (/‘pip/; ―kite‖), fofa (/‘fof/; ―cute‖); (5) Bisyllabic

(C1V1C2V2) with different consonants and different vowels, for example: pano

(/‘pãno/; ―cloth‖), fica (/´fik/; ―stay‖); and (6) Multisyllabic, for example: batata

(/ba‘tat/; ―potato‖), peteca (/pe‘tƐk/; ―shuttlecock‖).

Table 5 presents the agreement rates for judges in Step 3, who assessed the

adequacy of the 269 candidate words for the BP version of the instrument. Content

Validity Ratio were calculated for each word and organized by subtest.

INSERT TABLE 5

The Monosyllabic Diphthongs (VV) subtest is not presented in Table 5

because the nine words in this section did not achieve a Content Validity Ratio of 1.0,

which was the original word selection criterion. The authors decided to retain this

subtest despite the lower agreement rates because it focuses on vowel production

and may therefore play an important role in the diagnosis of CAS (ASHA, 2007;

Namasivayan et al., 2013). Percent interrater percent agreement for words in this

subtest were also examined. The words that obtained the best agreement were

maintained in this first phase and also in the next phases of the instrument. Only one

of the nine words attained an 83% agreement rate, while five words had agreement
21
rates of 66%. The six remaining words in the Monosyllabic (VV) subtest continued to

Steps 4 and 5.

The words included in the BP version of the DEMSS are described in greater

detail in Table 6. The table shows the subtests of the original DEMSS (Strand et al.,

2013; Strand & McCauley, 2019) in order to point out the overlap between the two

measures. This table also provides information about the decisions, judgments and

observations made for each word during Steps 3 to 5. Finally, Table 6 includes

comments about the choices affecting stimuli selection for the BP version of the

DEMSS.

INSERT TABLE 6

Expert judges agreed on the inclusion of 96 of the original 269 words

proposed for the BP version of the DEMSS. These included 90 words with both

consonants and vowels, and six words containing only vowels. The 96 words were

evaluated in Step 4 and in Step 5.

A Content Validity Ratio of 1 was achieved by 67 of the 96 stimulus words in

Step 4, and 63 words in Step 5. The comparison of these findings revealed that 47

words had a Content Validity Ratio of 1 in both steps. These results are shown in

Table 6. The authors analyzed the results of Steps 3 and 4 simultaneously, looking for

the items with the highest agreement rates and closest alignment with the pre-

established criteria. After Step 5 (Pilot Study), the authors used these observations to

select 44 words for inclusion in the final BP version of the DEMSS.

In the Monosyllabic Diphthongs (VV) subtest, created during the adaptation

process, five words had CVR=1 and were therefore included in the subtest. The

Monosyllabic (CV) subtest contained 10 words (CVR=1), as does the corresponding

subtest in the English-language DEMSS. The words were selected based on the

22
place of articulation of constituent phonemes, use of different sound types, oral and

nasal vowels, and similarity to the simple word structures used in the original

DEMSS. The Monosyllabic (VC) subtest was excluded from the BP version because

this is not a common word structure in BP.

The Bisyllabic-Reduplicated Syllables (C1V1C1V1) subtest contained four

words, as does the corresponding subtest in the original DEMSS. The words for this

subtest were selected based on the same previously mentioned criteria.

In the Monosyllabic-Different consonants (C1VC2) subtest, only two words

achieved a Content Validity Ratio of 1. To increase the number of test stimuli and

preserve the similarity with the original DEMSS, the two words with the next highest

Content Validity Ratio values in Step 5 were also included (pés /pƐs/; ―feet‖,

CVR=0.8; mês /mes/; ―month‖, CVR=0.9).

In the Bisyllabic-Same consonants, different vowels (C1V1C1V2) subtest, five

words attained Content Validity Ratio values of 1. However, two of these words

shared the same phoneme (the /v/ in vivo /‗vivo/; ―alive‖; and viva /‗viv/; ―alive‖). One

of them was therefore excluded to avoid restricting phoneme variability. To maintain

the length of this subtest, the excluded word was replaced by another item (pipa

/‘pip/ - ―kite‖, CVR=0.71).

In the Bisyllabic-Different consonants, different vowels (C1V1C2V2) subtest, six

words obtained Content Validity Ratio values of 1. To collect additional data

concerning children‘s ability to produce this type of bisyllabic word, the authors

included two additional items that obtained relatively high Content Validity Ratio

values in Step 5 (copa /‗kp/; ―cup‖, CVR=0.8; mato /‗mato/; ―bush‖ or ―jungle‖,

CVR=0.9).

23
 In the Multisyllabic subtest, six words obtained Content Validity Ratio values of

1. One of these was excluded for sharing a place of articulation with another item.

Three words with a high Content Validity Ratio in Step 5 were also added to the

subtest (peteca /pe‘tƐk/; ―shuttlecock‖, CVR=0.8; banana /ba‘nãn/; ―banana‖,

CVR=0.9; tomada /to‘mad/; ―power plug‖, CVR=1).

The expert judges (Step 3) showed moderate agreement with regards to the

newly included stimuli, as demonstrated by the Gwet‘s AC1 value of 0.57 [CI=0.51 to

0.63]. This may be attributable to the low Content Validity Ratio for words in the

Monosyllabic Diphthongs (VV) subtest, which had a Gwet‘s AC1 value of 0.35

[CI=0.10 to 0.59].

Table 7 shows the performance of the participant with CAS in the final BP

version of the DEMSS (Step 6). His poor performance on this instrument (especially

in the precision, consistency and vowel scores) suggests that the BP version of the

DEMSS may have potential to contribute to the diagnosis of CAS in BP-speakers.

However, it is important to highlight that the measure would have to be tested with a

much larger sample of children, including those with CAS and speech delay, in order

to validate the measure as a diagnostic tool.

The child received a low precision score (94 out of 176) and made several

vowel errors, scoring 81 out of 88. A total of nine consistency errors were also

observed. These issues were noted in all subtests of the BP version of the DEMSS,

regardless of word complexity. These findings, combined with the ASHA (2007)

checklist, could assist the CAS diagnosis in this patient.

INSERT TABLE 7

24
4. Discussion

Several steps must be taken to ensure that an assessment instrument retains

its content validity as it is adapted from one language to another. These include

translation and back translation of procedural elements, the incorporation of expert

judgments and observations, and a pilot study. In addition, careful development of the

language-appropriate stimuli are crucial for a test involving speech production. All of

these steps were performed in the present study. According to the literature, these

procedures help ensure the validity of the adapted instrument (Beaton et al., 2000;

Chourdakis et al.¸ 2019; Fonseca et al., 2011; Maillart et al., 2012; Pernambuco et

al., 2017).

Adapted instruments must also be ―evaluated‖ by members of the target

population for its appropriateness to the target language and population. In the

present study, this was achieved by administering the instrument to a sample of child

judges, consisting of children who confirmed their familiarity with the target stimuli, as

well as a pilot sample. The authors of the original instrument must also approve of

and be involved in every step of the adaptation process (Astepe & Köleli, 2019;

Beaton et al., 2000; Pernambuco et al., 2017). This was ensured throughout the

adaptation of the DEMSS to BP. Every step in the process was evaluated by one of

the authors of the original instrument, facilitating the equivalence of both versions of

the test.

In Step 1, the instructions and scoring procedures for the instrument were

independently translated by two professionals, as recommended in the literature

(Astepe & Köleli, 2019; Beaton et al., 2000; Pernambuco et al., 2017). In addition to

being fluent in the language of the original instrument (in this case, English), the

translators must be familiar with the topic of study and the relevant technical-scientific
25
terminology, and perform more than one translation of the instrument (Astepe &

Köleli, 2019; Chourdakis et al.¸ 2019; Pernambuco et al., 2017). The creation of a

consensus translation is also essential (Astepe & Köleli, 2019; Beaton et al., 2000;

Pernambuco et al., 2017).

Back translation was performed to verify the similarity between the constructs

assessed by the original and adapted instruments, and to ensure that there were no

issues in the adaptation process as determined by an author of the original

instrument (Beaton et al., 2000; Pernambuco et al., 2017;). The similarity between

versions of an assessment instrument is the main goal of cross-cultural adaptation. In

the present study, translation and back-translation were important factors in achieving

this goal, as they helped ensure that the BP version of the instrument was as similar

as possible to the original.

Step 2 of the adaptation process, which involved the selection of BP stimuli,

was also very rigorously conducted as recommended by previous studies

(Pernambuco et al., 2017). Words were selected from BP studies (Bastos et al.,

2004; Pedromônico et al., 2002), BP instruments (eg.: Auditory Vocabulary Test -

Capovilla et al., 2011) and a BP dictionary with the same target population as the

DEMSS. However, those that were difficult to pronounce (phonetic factors), whose

phonemes did not exist in the phonological system of the target population

(phonological factors) (Ceron et al., 2017), or were absent from the vocabulary of the

children evaluated were not included in the instrument.

Throughout the selection of new test stimuli, the need to preserve the similarity

between both versions of the instrument but the integrity of the new measure to its

target language was continuously emphasized. Since English and BP have different

linguistic structures, two subtests had to be excluded from the adapted instrument. At

26
the same time, a subtest involving monosyllables with a VV structure was added to

the BP version of the instrument, since this is an important object of assessment in

CAS (Namasivayan et al., 2013) and VV represents a common feature in BP. In fact,

inconsistent vowel errors are a consensus diagnostic criterion for this disorder

(ASHA, 2007).

The analysis by expert judges (Step 3) allowed for the selection of the best

possible BP stimuli for the instrument. According to some studies (Astepe & Köleli,

2019; Beaton et al., 2000; Chourdakis et al., 2019; Pernambuco et al., 2017), expert

judges should suggest modifications to the test stimuli or administration procedures

to ensure that the instrument adequately represents the construct being evaluated. In

the present study, the judges confirmed the adequacy of test stimuli and suggested

modifications when appropriate (Fonseca et al., 2011).

The analysis by expert judges should be complemented by an evaluation

performed by members of the population with whom it will be used (Step 4). These

individuals are asked to indicate their familiarity with test stimuli. This procedure also

reveals possible responses to each stimulus (Astepe & Köleli, 2019; Beaton et al.,

2000; Chourdakis et al., 2019; Pernambuco et al., 2017). In the adaptation of the

DEMSS, eight child judges were involved in the evaluation of the new stimuli (Step

4). This was essential to determine the applicability of the selected items, in addition

to contributing to the content validity of the instrument.

In the pilot sample (Step 5), the BP version of the instrument was administered

to typically developing children in a real data collection scenario. Children were asked

to repeat the words and received articulatory cues if necessary. Most children had no

difficulties producing the stimuli and performed very well on the test. Overall accuracy

levels were high, as expected, given that the sample was known to have typically

27
developing speech and language skills. However, some children asked for the

stimuli to be repeated. The length of administration of BP version of the DEMSS was

at most ten minutes. The instructions for scoring and interpretation of the BP version

of the test were kept as similar as possible to those of the original version of the

instrument. These results reaffirm that the applicability of BP version of the DEMSS

to populations of BP-speaking children (Astepe & Köleli, 2019; Beaton et al., 2000;

Chourdakis et al.¸ 2019; Pernambuco et al., 2017).

The instrument evaluates the same motor speech skills as the original

DEMSS, with special attention paid to the vowels, prosody, and consistency of error

patterns — all of which are associated with CAS. Nevertheless, although the stimuli

in both instruments are similar in word and syllable structure, there are some

important differences between them. The BP version focused on early-developing

sounds and vocabulary recognizable to young children, while the original DEMSS

included some later-developing and more complex sounds, as well as words with a

varying frequency of occurrence in the English speaking children targeted by that

test. Some subtests were removed and others added in the BP version. However, all

of these changes were based entirely on the need to ensure that the BP version of

the instrument could provide as thorough an assessment of BP speakers as the

original DEMSS does of English speakers. As such, these differences between the

two versions of the instrument actually represent a strength of the adaptation

process. Step 6 showed that the BP version of the DEMSS was applicable to

individuals with CAS and may have potential to contribute to the diagnosis of this

condition.

In this study, we described the adaptation of verbal tasks from the English

DEMSS for use in BP populations, documenting the content validity of the newly

28
developed BP stimuli and the translated procedures and scoring instructions.

Currently, additional tasks are being developed (including nonverbal, diadochokinetic

and sentence stimuli) to complete the BP instrument and deliver a robust and reliable

tool for the evaluation of motor speech disorders in children. In their 2020 study,

Murray et al. recommended the English language DEMSS as part of a larger

protocol. Both development and study of the performance of such a larger protocol

will also be useful for this BP adaptation.

Empirical studies designed to provide additional evidence of the reliability and

validity of the adapted version of the instrument are also critically important,

especially those in which diagnostic accuracy of the BP measure can be evaluated

by a considerably larger sample than the single child included in this report whose

data served to demonstrate the potential for validity rather than strong evidence of

validity for use in differential diagnosis of this new instrument. We are currently in the

process of complementing the instrument by adding new tasks, administering it to a

larger sample, and investigating the validity and reliability of new tasks as well as the

protocol in this way. In this way, BP DEMSS needs to go through many additional

steps before it can be providing support for a differential diagnosis of CAS from other

severe speech sound disorders.

5. Conclusion

The use of techniques such as translation, back-translation, item development

and evaluations by native-speaking expert judges during the cross-cultural adaptation

process were crucial in providing preliminary evidence suggesting the content validity

of the adapted instrument. The results of the present study indicate that this

instrument may fill a significant gap in clinical practice and research for speech

29
therapists in Brazil, where few instruments are available to evaluate severe speech

disorders and contribute to differential diagnosis. Ongoing and additional research

designed to provide further evidence supporting the robustness of the BP version of

the DEMSS will further cement its place as an important part of best practice

standards for children with CAS among the approximately 220 million speakers of BP.

Credit Authors Statement

Marileda Barichelo Gubiani – Conceptualization; Methodology; Validation; Formal

Analysis; Investigation; Resources; Data Curation; Writing – Original Draft;

Visualization.

Karina Carlesso Pagliarin - Conceptualization; Methodology; Validation; Formal

Analysis; Investigation; Resources; Data Curation; Writing – Original Draft.

Rebecca J. McCauley - Methodology; Validation; Writing – Review & Editing.

Márcia Keske-Soares - Conceptualization; Methodology; Validation; Formal

Analysis; Writing – Review & Editing; Visualization; Supervision; Project

Administration.

References

American Speech-Language-Hearing Association. (2007). Childhood apraxia of

speech [Position statement]. Retrieved from www.asha.org/policy/

30
Astepe, S., & Köleli, S. (2019). Translation, cultural adaptation, and validation of

Australian pelvic floor questionnaire in a Turkish population. European Journal of

Obstetrics & Gynecology and Reproductive Biology, 234(1), 71-74.

https://doi.org/10.1016/j.ejogrb.2019.01.004

Bastilha, G. R., Paglarin, K. C., & Cielo, C. A. (2020). Development and evidence of

content validity of the Spectrographic Vocal Assessment Protocol (SVAP).

Journal of Voice, S0892-1997(19), 30450-30453.

https://doi.org/10.1016/j.jvoice.2019.12.008

Bastos, J. C., Ramos, A. P., & Marques, J. (2004). Study of children's vocabulary:

limitations of traditional collection methodologies. Revista da Sociedade

Brasileira de Fonoaudiologia, 9, 1-9.

Bearzotti, F., Tavano, A., & Fabbro, F. (2007). Developmental of orofacial praxis of

children from 4 to 8 years of age. Perceptual and Motor Skills, 104(3 Pt 2), 1355-

1366.https:// doi: 10.2466/pms.104.4.1355-1366. PMID: 17879670

Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for

the process of cross-cultural adaptation of selfreport measures. Spine, 16(2),

3186-3191. https://doi: 10.1097/00007632-200012150-00014

Blakeley, R. W. (2001). Screening Test for Developmental Apraxia of Speech. (2nd

ed.). Pro-Ed.

Bonini, J. B., & Keske-Soares, M. (2018). Pseudowords to phonological therapy

validated by expert judges. CoDAS, 30(5), e20170013.

https://doi.org/10.1590/2317-1782/20182017013

31
Capovilla, F.C., & Capovilla, A. G. S. (1997). Desenvolvimento linguístico da criança

dos dois aos seis anos: tradução e estandardização do Peabody Picture

Vocabulary Test de Dunn & Dunn, e da Language Development Survey de

Rescorla. Ciência Cognitiva: Teoria, Pesquisa e Aplicação, 1(1), 353-380.

Capovilla F. C., Negrão, V. B., & Damázio, M. (2011). Auditory Vocabulary Test and

Expressive Vocabulary Test - TVAud and TVExp: Validated and standardized for

the development and understanding of speech from 18 months to 6 years of age.

Memnon.

Ceron, M. I., Gubiani, M. B., Oliveira, C. R., & Keske-Soares, M. (2017). Factors

influencing consonant acquisition in Brazilian Portuguese-speaking children.

Journal of Speech, Language, and Hearing Research, 60(4), 759-771.

https://doi.org/10.1044 / 2016_JSLHR-S-15-0208

Ceron, M. I., Gubiani, M. B., Oliveira, C. R., & Keske-Soares, M. (2018). Evidence of

validity and reliability of a phonological assessment tool. CoDAS, 30(3),

e20170180. https://doi.org/10.1590/2317-1782/20182017180

Chourdakis, M., Grammatikopoulou, M. G., Poulia, K. A., Passakiotou, M., Pafili, Z.

K., Bouras, E., …, Heyland, D. K. (2019). Translation of the modified NUTRIC

score and adaptation to the Greek ICU setting. Clinical Nutrition ESPEN, 29(1),

72-76. https://doi.org/10.1016/j.clnesp.2018.12.003

Cohen, R. J. (2014). Testagem e avaliação psicológica: Introdução a testes e

medidas. (8ª ed). AMGH.

Davis, B. L. (1998). Differential Diagnosis of Developmental Apraxia. Perspectives on

Language Learning and Education, 5(2), 4-7. https://doi.org/10.1044/lle5.2.4

32
Dodd, B. (Ed.). (1995). Differential diagnosis and treatment of children with speech

disorder. Singular Publishing Group.

Dodd, B. (Ed.). (2005). Differential diagnosis and treatment of children with speech

disorder (2nd ed.). Wiley.

Dodd, B., Zhu H., Crosbie, S., Holm, A., & Ozanne, A. (2002) Diagnostic Evaluation

of Articulation and Phonology. Psychological Corporation.

Felício, C. M., & Ferreira, C. L. (2008). Protocol of orofacial myofunctional evaluation

with scores. International Journal of Pediatric Otorhinolaryngology, 72(3), 367-

375. https://doi.org/10.1016/j.ijporl.2007.11.012

Fonseca, R. P., Casarin, F. S., Oliveira, C. R., Gindri, G., Ishigaki, E. C. S. S., Ortiz,

K. Z., …, Scherer, L. C. (2011). Adaptation of verbal neuropsychological

instruments: A flowchart of procedures beyond translation. Interação em

Psicologia, 15(1), 59-69. https://doi.org/10.5380/psi.v15i0.25374

Forrest, K. (2003). Diagnostic criteria of developmental apraxia of speech used by

clinical speech-language pathologists. American Journal of Speech-Language

Pathology, 12(3), 376–380. https://doi.org/10.1044/1058-0360(2003/083)

Gubiani, M. B., Pagliarin, K. C., & Keske-Soares, M. (2015). Tools for the assessment

of childhood apraxia of speech. CoDAS, 27(6), 610-615.

http://dx.doi.org/10.1590/2317-1782/20152014152

Gwet, K. L. (2014). Handbook of inter-rater reliability: The definitive guide to

measuring the extent of agreement among multiple raters. (4th ed). Advanced

Analytics.

33
Hayden, D., & Square, P. (1999). Verbal Motor Production Assessment for Children.

The Psychological Corporation.

Kaufman, N. (1995). Kaufman Speech Praxis Test for Children. Wayne State

University Press.

Landis, J.R., & Koch, G. G. (1977). The measurement of observer agreement for

categorical data. Biometrics, 33(1), 159-174. https://doi.org/10.2307/2529310

Lawshe, C. H. (1975). The quantitative approach to content validity. Personnel

Psychology, 28(1), 563-575.

Liégeois, F. J., Hildebrand, M. S., Bonthrone, A., Turner, S. J., Scheffer, I. E, Bahlo,

M., …, Morgan, A. T. (2016). Early neuroimaging markers of FOXP2 intragenic

deletion. Scientific Reports, 6, 35192. https://doi.org/10.1038/srep35192

Maas, E., Butalla, C. E., & Farinella, K. A. (2012). Feedback frequency in treatment

for childhood apraxia of speech. American Journal of Speech-Language

Pathology, 21(3), 239-257. https://doi.org/10.1044/1058-0360(2012/11-0119)

Maillart, C., Parisse, C., & Tommerdahl, J. (2012). F-LARSP 1.0: an adaptation of the

LARSP language profile for French. Clinical Linguistics & Phonetics. 26(2), 188-

198. https://doi:10.3109/02699206.2011.602459

Malmenholt, A., Lohmander, A., & McAllister, A. (2017). Childhood apraxia of speech:

A survey of praxis and typical speech characteristics. Logopedics, Phoniatrics,

Vocology, 42(2),84–92. https://doi.org/10.1080/14015439.2016.1185147

Mezzomo, C. L., Vargas, D. Z., & Souza, A. P. R. (2011). The differences in the

correct production and use of the repair strategies in children with typical
34
 phonological development, atypical and with dyspraxia. Revista Distúrbios da

Comunicação, 23(3), 261-267.

Morgan, A., & Murray, E. (2017). Clinical management of motor speech disorders. In

B. Dodd, & A. Morgan (Orgs). Intervention case studies of child speech

impairment (75-94). J&R Press.

Murray, E., McCabe, P., & Ballard, K. J. (2012). A comparison of two treatments for

childhood apraxia of speech: methods and treatment protocol for a parallel group

randomised control trial. BMC Pediatrics, 12(112), 1-9.

https://doi.org/10.1186/1471-2431-12-112

Murray, E., McCabe, P., Heard, R., & Ballard, K. J. (2015). Differential diagnosis of

children with suspected childhood apraxia of speech. Journal of Speech,

Language, and Hearing Research: JSLHR, 58(1), 43–60.

Murray, E., Iuzzini-Seigel, J., Maas, E., Terband, H., & Ballard, K. J. (2020).

Differential diagnosis of childhood apraxia of speech compared to other speech

sound disorders: A systematic review. American Journal of Speech-Language

Pathology, 1-22. https://doi.org/10.1044/2020_AJSLP-20-00063

Namasivayam, A. K., Pukonen, M., Goshulak, D., Yu, V. Y., Kadis, D. S., Kroll, R.,

Pang, E. W., & De Nil, L. F. (2013). Relationship between speech motor control

and speech intelligibility in children with speech sound disorders. Journal of

Communication Disorders, 46(3), 264–280.

https://doi.org/10.1016/j.jcomdis.2013.02.003

Navarro, P. R., Silva, P. M. V., & Bordin, S. M. S. (2018). Speech apraxia in

childhood: beyond phonetic and phonological issues. Revista Distúrbios da

35
 Comunicação, 30(3), 475-489. https://doi.org/10.23925/2176-2724.2018v30i3p-

475-489

Pacico, J. C., & Hutz, C. S. (2015). Validade. In C. S. Hutz, D. R. Bandeira & C. M.

Trentini (Orgs.). Psicometria (71-84). Artmed.

Payão, L. M. C., Lavra-Pinto, B., Wolff, C. L., & Carvalho, Q. (2012). Clinical

characteristics of childhood apraxia of speech: literature review. Letras de Hoje,

47(1), 24-29.

Pedromônico, M. R., Affonso, L. A., & Sañudo A. (2002). Expressive vocabulary of

children between 22 and 36 months: exploratory study. Revista Brasileira de

Crescimento e Desenvolvimento Humano, 12(2), 13-22.

Pernambuco, L., Espelt, A., Junior, H. V. M., & Lima, K. C. de. (2017).

Recommendations for elaboration, transcultural adaptation and validation

process of tests in Speech, Hearing and Language Pathology. CoDAS, 29(3),

e20160217. https://doi.org/10.1590/2317-1782/20172016217

Power, E., Code, C., Croot, K., Sheard, C., & Gonzalez, R. L. J. (2010). Florida

Apraxia Battery-Extended and revised Sydney (FABERS): design, description,

and a healthy control sample. Journal of Clinical and Experimental

neuropsychology, 32(1), 1–18. https://doi.org/10.1080/13803390902791646

Seabra, A. G., Trevisan, B. T., & Capovilla, F. C. (2012). Teste infantil de nomeação.

In A. G. Seabra, & N. M. Dias (Orgs.), Cognitive neuropsychological assessment:

oral language (54-86). Memnon.

36
Shriberg, L. D., Lohmeier, H. L., Campbell, T. F., Dollaghan, C. A., Green, J. R., &

Moore, C. A. (2009). A nonword repetition task for speakers with misarticulations:

the Syllable Repetition Task (SRT). Journal of Speech, Language, and Hearing

Research, 52(5), 1189–1212. https://doi.org/10.1044/1092-4388(2009/08-0047)

Shriberg, L. D., Fourakis, M., Hall, S. D., Karlsson, H. B., Lohmeier, H. L., McSweeny,

J. L., Potter, N. L., Scheer-Cohen, A. R., Strand, E. A., Tilkens, C. M., & Wilson,

D. L. (2010). Extensions to the Speech Disorders Classification System

(SDCS). Clinical Linguistics & Phonetics, 24(10), 795–824.

https://doi.org/10.3109/02699206.2010.503006

Shriberg, L. D., Kwiatkowski, J., & Mabie, H. L. (2019a). Estimates of the prevalence

of motor speech disorders in children with idiopathic speech delay. Clinical

Linguistics & Phonetics, 33(8), 679–706.

https://doi.org/10.1080/02699206.2019.1595731

Shriberg, L. D., Strand, E. A., Jakielski, K. J., & Mabie, H. L. (2019b). Estimates of

the prevalence of speech and motor speech disorders in persons with complex

neurodevelopmental disorders. Clinical Linguistics & Phonetics, 33(8), 707–736.

https://doi.org/10.1080/02699206.2019.1595732

Souza, T. N. U., & Payão, L. M. C. (2008). Acquired and developmental apraxia of

speech: similarities and difference. Revista da Sociedade Brasileira de

Fonoaudiologia, 13(2), 193-202. https://doi.org/10.1590/S1516-

803420080002000015

Strand, E. A., & McCauley, R. J. (2019). Dynamic Evaluation of Motor Speech Skill

(DEMSS) Manual. Paul Brookes Publishing.

37
Strand, E. A., McCauley, R. J., Weigand, S. D., Stoeckel, R. E., & Baas, B. S. (2013).

A motor speech assessment for children with severe speech disorders: reliability

and validity evidence. Journal of Speech, Language, and Hearing

Research, 56(2), 505–520. https://doi.org/10.1044/1092-4388(2012/12-0094)

Tohidast, S. A., Ghelichi, L., Kamali, M., Ebadi, A., Shavaki, Y. A., Shafaroodi, N., &

Mansuri, B. (2019). Development and psychometric evaluation of speech and

language pathology evidence-based practice questionnaire (SLP-EBPQ).

Evidence Based Care Journal, 8(4), 61-74.

https://doi.org/10.22038/ebcj.2019.33473.1850

Wiethan, F.M., Mota, H. B., & Bragança, A. (2016). Correlations between vocabulary

and phonological acquisition: number of words produced versus acquired

consonants. CoDAS, 28(4), 379-387. https://doi.org/10.1590/2317-

1782/20162015108

Ziegler, W., Aichert, I., & Staiger, A. (2012). Apraxia of speech: concepts and

controversies. Journal of Speech, Language, and Hearing Research, 55(5),

S1485–S1501. https://doi.org/10.1044/1092-4388(2012/12-0128)

Table 1
Tasks of the original English DEMSS.
Syllable Task Example
structure
Monosyllabic Consonant-Vowel (CV) me, hi
words Vowel-Consonant (VC) up, eat
Consonant-Vowel- Consonant (C1VC1) pop, dad
Consonant-Vowel-Consonant (C1VC2) bus, cat
Bisyllabic words Reduplicated syllables (C1V1C1V1) Papa
Bisyllabic (C1V1C1V2) puppy, mommy
Bisyllabic (C1V1C2V2) bunny, happy

38
Multisyllabic Words banana, lemonade
Note: CV: Consonant-Vowel; VC: Vowel–Consonant; C1VC1: Consoant1–Vowel–Consoant1; C1VC2:
Consoant1-Vowel–Consoant2; C1V1C1V2: Consoant1–Vowel1-Consoant1-Vowel2; C1V1C2V2: Consoant1-
Vowel1–Consoant2-Vowel2

Table 2
Scores of the original English DEMSS.
Variable Description Scoring
Overall Score based on the 04 = correct on first attempt
articulatory number of attempts at 03 = consistent substitutions or distortions
accuracy each item. 02 = correct after one cued attempt
01 = correct after two to four additional attempts
00 = incorrect after all cued attempts
Consistency This score reflects 00 = consistent across all trials
changes in speech 01 = inconsistent across 2 or more trials.
production across
trials. The child‘s Regardless of whether the child‘s initial repetition is
production is correct, they are asked to repeat the utterance again
considered consistent so that consistency can be assessed.
when word production
does not vary across
trials.
Inconsistency refers to
variations in the way a
particular word is
pronounced over the
course of testing
Vowel This score reflects the 00 = correct
accuracy accuracy of vowel 01= mild distortion
production in the first 02 = frank distortion
attempted imitation of Vowel accuracy is always scored on the first attempt
the stimulus word at an utterance.

Prosodic This score is based on 00 = correct

accuracy the production of 01 = incorrect
lexical stress for
selected items. This Prosody is always scored on the first attempt at an
variable was only utterance.
analyzed in tasks
involving disyllabic
(reduplicated
syllables) and
polysyllabic words.

39
 Table 3
Summary of the results of the evaluation of the BP version of the DEMSS of a boy with CAS
aged 5 years and 8 months.
Assessment Results
Phonemic Discrimination 21/23 (average)
Child Naming Test 06 /60 (deficit)
Auditory Vocabulary Test 28/33 (deficit)
Phonological Assessment Instrument Phonemes acquired
Initial: /t, g, m, n/
Medial: /b, ʃ, ʒ/
Final: /n, l/

Table 4
Acceptable Content Validity Ratio (CVR) values according to Lawshe (1975).
Number of judges Minimum adequate CVR value
5-7 .99
8 .75
9 .78
10 .62
11 .59
12 .56
13 .54
14 .51
15 .49
20 .42

Note: CVR - Content Validity Ratio

Table 5
Agreement among expert judges (Step 3) of 260 words and number of words contained in each
subtest of the BP version of DEMSS.
Subtest
Bisyllabic -
Content Monosyllabic Reduplicated Monosyllabic Bisyllabic Bisyllabic Total
Multisyllabic
Validity (CV) Syllables (C1VC2) (C1V1C1V2) (C1V1C2V2) across
Ratio (C1V1C1V1) subtests
(CVR) (n
(n words) (n words) (n words) (n words) (n words) (n words)
words)
1 35 8 8 8 16 15 90
.86 24 7 6 9 9 9 64
.71 10 5 2 7 7 15 46
.57 14 2 3 3 7 8 37
.43 4 1 4 1 3 4 17
.28 1 1 1 2 0 0 5
.14 0 0 0 0 0 1 1
Total 88 24 25 30 42 52 260

40
 (Words)
Note: * This subtest was added in the BP adaptation and the agreement was by simple percentage.
VV: Vowel–Vowel; CV: Consonant-Vowel; C1V1C1V1: Consoant1–Vowel1–Consoant1–Vowel1; C1VC2:
Consoant1-Vowel–Consoant2; C1V1C1V2: Consoant1–Vowel1-Consoant1-Vowel2; C1V1C2V2: Consoant1-
Vowel1–Consoant2-Vowel2

Table 6
Words selected for the BP version of the DEMSS in different steps of this study.
Final
Step 4
BP
Non Comparison
Original Step 3 Step 5 version
Expert of
Subtest English DEMSS Expert Judges Pilot Study Comments of
Child Agreement
(n words) (n words) (n words) DEMSS
(n (n words)
(n
words)
words)
When
agreement
was
compared
between
Monosyllabic Step 4 and
No subtest with
Diphthong 6 5 5 5 Step 5, five 5
this syllable structure
(VV)* words had
CVR=1;
therefore,
all were
retained.

From the
final 16 BP
words
considered
acceptable,
ten words
Monosyllabic were
10 34 23 24 16 10
(CV) selected by
authors
according
to pre-
established
criteria.

No
appropriate
words with
Monosyllabic this
(VC) 10 Subtest excluded syllable 0
structure
exist in
Brazilian
Portuguese
From the
final seven
Bisyllabic - BP words,
Reduplicated four words
Syllables 4 8 8 7 7 were 4
(C1V1C1V1) selected by
authors
according
to pre-
41
 established
criteria.
No
appropriate
words with
this
Monosyllabic syllable
6 Subtest excluded 0
(C1VC1) structure
exist in
Brazilian
Portuguese

Only two
words had
CVR=1 for
steps 4
and 5 so
two
additional
words were
included by
Monosyllabic the authors
10 8 4 4 2 4
(C1VC2) based on
the Pilot
sample
(Step 5)
which
indicated
CVR >.9
for two
words.

From five
words, one
word was
excluded
(due to its
having the
Bisyllabic same
6 9 7 7 5 5
(C1V1C1V2) phonemes
as another
item), and
the authors
added one
word

Six words
had
CVR=1 for
both Step 4
and Step 5,
and the
Bisyllabic
9 16 10 7 6 authors 8
(C1V1C2V2)
add two
more
words
considering
Step 5
analysis.

42
 Six words
had
CVR=1 for
both Step 4
and Step 5.
The
authors
excluded
one word
(same
Multisyllabic 6 15 10 9 6 place of 8
articulation
as another
item) and
add three
words
considering
the Pilot
Sample
(Step 5)
analysis.
Total 44
61 96 67 63 47
(Words)
Note: CVR - Content Validity Ratio
VV: Vowel–Vowel; CV: Consonant-Vowel; C1V1C1V1: Consoant1–Vowel1–Consoant1–Vowel1; C1VC2:
Consoant1-Vowel–Consoant2; C1V1C1V2: Consoant1–Vowel1-Consoant1-Vowel2; C1V1C2V2: Consoant1-
Vowel1–Consoant2-Vowel2

43
Table 7
Scores for the child with CAS in the BP version of DEMSS.
Child‘s overall
Subscores
performance
Child‘s
Subtest Precision/TP Vowel/TP Consistency/TP Prosody/TP
score/TP
Monosyllabic Diphthong
(VV) 16/20 10/10 05/05 - 31/35

Monosyllabic
(CV) 26/40 20/20 10/10 - 56/70

Bisyllabic –
Reduplicated Syllables
16/16 08/08 04/04 04/04 32/32
(C1V1C1V1)

Monosyllabic
(C1VC2) 03/16 08/08 01/04 - 12/28

Bisyllabic
(C1V1C1V2) 10/20 03/10 05/05 05/05 23/40

Bisyllabic
(C1V1C2V2) 18/32 16/16 07/08 08/08 49/64

Multisyllabic
5/32 16/16 03/08 08/08 32/64
Child‘s performance on
each subscore 94/176 81/88 35/44 25/25 235/333

Note: TP - Total possible for each subscore

VV: Vowel–Vowel; CV: Consonant-Vowel; C1V1C1V1: Consoant1–Vowel1–Consoant1–Vowel1; C1VC2:
Consoant1-Vowel–Consoant2; C1V1C1V2: Consoant1–Vowel1-Consoant1-Vowel2; C1V1C2V2: Consoant1-
Vowel1–Consoant2-Vowel2

44