Perspectives of the ASHA Special Interest Groups

SIG 1, Vol. 3(Part 1), 2018, Copyright © 2018 American Speech-Language-Hearing Association

Using Language Sample Analysis to Assess Pragmatic Skills

in School-Age Children and Adolescents
Geralyn R. Timler
Department of Communication Disorders and Sciences, James Madison University
Harrisonburg, VA
Disclosures
Financial: Geralyn R. Timler has no relevant financial interests to disclose.
Nonfinancial: Geralyn R. Timler has no relevant nonfinancial interests to disclose.

The Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, addition of
social (pragmatic) communication disorder has underscored the need for development and
refinement of clinical tools to accurately identify pragmatic language deficits in clinical
populations. One of the 4 required social (pragmatic) communication disorder diagnostic
criteria is difficulty following rules for conversation and storytelling. As such, language
sample collection and analysis is an important assessment tool for identification of this
disorder. This tutorial describes a language sampling protocol and reviews analysis
techniques for the purpose of documenting pragmatic language skills in school-age
populations.

Case Example
Henry is a 9-year-old with attention-deficit/hyperactivity disorder (ADHD) whose family
volunteered to have him participate in a research study that focused on examination of social
pragmatic communication skills in school-age children. Henry’s testing protocol included norm-
referenced tests, a language sample, and several other nonstandardized measures. Henry’s
verbal and nonverbal IQ was above average. The Test of Pragmatic Language-2 (TOPL-2; Phelps-
Terasaki & Phelps-Gunn, 2007) and the Social Language Development Tests–Elementary
(Bowers, Huisingh, & LoGiudice, 2008) were administered. Henry’s standard score performance
of 100 and 98, respectively, placed him solidly within the average range on both measures.
Henry’s mother completed the Children’s Communication Checklist-2 (CCC-2; Bishop, 2006)—a
norm-referenced report measure for children aged 4;0–16;11 years;months. The CCC-2 yields one
composite score, the General Communication Composite (M = 100, SD = 15), reflecting overall
communication skills across eight scales representing language content (i.e., semantics scale),
form (i.e., speech and syntax scales), and use (i.e., coherence, initiation, scripted language,
context, and nonverbal communication scales). Henry’s General Communication Composite score
of 78 suggested that he had a communication disorder. A closer look at the individual scale scores
revealed low performance in the five language use or pragmatic scales (language use is referred to
as pragmatic language from this point forward). Whereas his content and form scale scores ranged
between 7 and 12 (M = 10, SD = 3), his pragmatic language scale scores ranged between 3 and 6.
Examples of individual CCC-2 items that his mother scored as problematic included “it’s hard to
make sense of what he is saying even though his words are clear” (coherence scale), “it’s difficult
to stop him from talking” (initiation scale), and “he doesn’t recognize when others are angry or
upset” (nonverbal communication scale).
Identification of Social (Pragmatic) Communication Disorder
Henry’s CCC-2 profile suggests that he may have social (pragmatic) communication disorder
(SPCD), a neurodevelopmental communication disorder introduced in the Diagnostic and Statistical
Manual of Mental Disorders, Fifth Edition (American Psychiatric Association, 2013, pp. 47–49). This
disorder of both verbal and nonverbal communication skills is characterized by four diagnostic
criteria including difficulties in:

23
 1. using appropriate communication for specific purposes;
2. changing communication style to match the situation (e.g., speaking differently with
adults than with a peer);
3. following rules for conversations and storytelling such as managing turns in the
conversation, rephrasing when misunderstood, reading facial expressions to understand
a communication partner’s intentions, and telling stories that others can follow; and
4. understanding what is not explicitly stated (e.g., poor comprehension of idioms,
figurative language, and multiple meanings of words).
Common co-occurring diagnoses with SPCD include ADHD, behavior disorders, and language
disorder.
Currently, a gold standard SPCD diagnostic tool does not exist. Best practice guidelines
for the identification of SPCD call for administration of comprehensive protocols consisting of
norm-referenced tests, caregiver report measures, and conversation and narrative samples including
samples of adult–child and peer-to-peer interactions (Adams, 2002, 2015; Izaryk, Skarakis-Doyle,
Campbell, & Kertoy, 2015; Norbury, 2014; Swineford, Thurm, Baird, Wetherby, & Swedo, 2014).
In my social communication laboratory, my students and I collect these various measures to
examine social communication skills in children and adolescents. Data from participants like
Henry reveal that currently available norm-referenced social language tests are of limited value for
uncovering problems with using communication for specific purposes and changing communication
style to match the situation (i.e., SPCD Criteria 1 and 2). Certainly, these tests fail to capture the
pragmatic language deficits parents and teachers have all too readily observed in a child’s daily
social interactions (i.e., SPCD Criterion 3, conversation and storytelling deficits). Although we
acknowledge that norm-referenced tests can be used to identify language content and form deficits,
including deficits in understanding figurative language (i.e., SPCD Criterion 4), our experience
aligns with expert opinion that language sampling remains the most promising tool for validating
parent and teacher concerns in pragmatic language skills (e.g., Adams et al., 2011; Norbury,
2014; Paul & Norbury, 2012). Unfortunately, language sampling and analysis do have some
perceived drawbacks. For example, speech-language pathologists (SLPs) report that language
sampling is time consuming and does not provide a score (see Heilmann, 2010; Pavelko, Owens,
Ireland, & Hahs-Vaughn, 2016; Schuele, 2010, for a review of other challenges). Although
norm-referenced tests do provide a score, these tests are also time consuming. In Henry’s case,
administration of the TOPL-2 required 50 min, which is slightly under the reported administration
time in the examiner’s manual of 60–90 min. Scoring of Henry’s TOPL-2 required another 10 min.
The test results provided evidence that Henry knew what to do when presented with a hypothetical
social situation, but the parent report measure revealed that Henry had difficulties implementing
his knowledge in everyday communication. As such, although we now had a standard score of his
performance, the 60 min of testing and scoring did not reveal the pragmatic language deficits
noted by his mother. A language sampling protocol designed to elicit discourse that occurs in
everyday conversations is more likely to reveal these deficits. In our view, the benefits of language
sampling far outweigh the barriers. Language sampling provides a baseline of the student’s verbal
and nonverbal behaviors. Baseline performance can then be used to develop intervention goals
that align with parent and teacher concerns. Unlike norm-referenced testing, language sampling
can be repeated as often as needed to document student progress or to identify needed modifications
in intervention goals. Finally, and this is worth repeating, for children like Henry who score within
the normal range on norm-referenced tests, language sampling may be the only tool to capture
concerns noted by parents and teachers. As such, this tutorial presents strategies and practical
tips for collecting and analyzing language samples when the goal is to identify problematic pragmatic
language behaviors.

24
What to Do? Tips for Language Sample Collection
Sampling Protocol for SLP–Child Interactions. Research has demonstrated the
importance of collecting samples of multiple discourse types including personal retells, exposition
(description of a procedure or game), story retell/narrative retell (description of a movie or book),
and persuasion for older school-age children and adolescents (e.g., see Miller, Andriacchi, &
Nockerts, 2016; Nippold, 2014; Paul & Norbury, 2012). Personal retells are usually collected
in traditional interview conversation samples when children are asked to talk about a birthday
party or favorite event. Conversation samples allow for views of turn-taking including responsiveness
and initiation, conversation repairs, and topic development and maintenance skills. Cohesion,
or use of linguistic devices such as pronouns, ellipsis, and conjunctions to tie meaning across
utterances, is best viewed in text level discourse (i.e., exposition, narratives, and persuasion
samples). Text level samples place more demands on children’s language abilities because children
must formulate and organize a series of consecutive utterances (Miller et al., 2016).
In our laboratory, we have adapted a protocol by Hadley (1998) to elicit multiple discourse
types within a single language sample. Hadley’s 12-min sample includes three mini-sample
segments to elicit personal retell, exposition, and narrative retell of a favorite book or movie.
Hadley’s protocol starts with general questions about the child’s family and favorite activities.
In the personal retell segment, the child is asked to share personal stories about a sibling and a
pet. The exposition segment begins with asking the child to “tell about a game/sport you like
and how you play it.” Finally, the story retell segment prompts the child to talk briefly about
the characters and plot of a favorite book or movie and why it is a favorite. Our adaptation to
this protocol includes the addition of brief scripts that model each of the desired discourse types.
Appendix A provides script examples. The personal retell scripts focus on “negative” past events
such as problems with a sibling or a lost pet because children are more likely to use emotion
words to describe these emotionally salient events (McCabe, Bliss, Barra, & Bennett, 2008;
McCabe & Rollins, 1994). Scripts are not read verbatim but are shared in a conversational tone.
All scripts include at least one complex sentence and a mental state verb or emotion word. We
find that only one script is needed per 4-min segment; however, if a child’s response is too brief,
a second script can be selected. The scripts frequently elicit child questions and comments about
our personal stories (i.e., the scripts), as illustrated in the language transcript excerpt below.
This sample and the others that follow were transcribed in the Systematic Analysis of Language
Transcripts (SALT) software (2017; Miller & Iglesias, 2016). In SALT conventions, A refers to the
adult and C refers to the child. An equal sign is used to provide comments. Slashes mark bound
morphemes, and an asterisk (*) precedes words that were omitted by the child. Speech disruptions
or mazes, characterized by word repetition or utterance reformulation, are placed within parenthesis
so that these words are omitted from total word counts and other analyses.
1. A: One time I had a birthday.
2. A: And no one wish/ed me a happy birthday.
3. A: And I thought all of my friend/s forgot about my birthday.
4. A: And then someone took me out to dinner.
5. A: And my friend/s were there.
6. A: And they yell/ed surprise.
7. A: So they did/n’t forget my birthday after all!
8. : Pause [.03]
9. = Adult looks expectantly at child.

25
 10. C: You thought they forgot!
11. A: I did think they forgot.
12. A: and I felt very sad.
13. C: They were just (try/ing to make) try/ing to make you so you think that they
forgot.
14. C: (then) then so they did it.
15. C: and that/’s how.
16. A: Yeah, they forgot they were do/ing it.
17. C: No they did/n’t forget.
18. C: They did it to trick you.
We have collected over 60 samples with this protocol. We find that many of our participants ask
a question or comment about our scripts. Children who do not comment or question now stand
out. Frequently, these are children with clinical diagnoses of autism spectrum disorder (ASD) or
ADHD. One of the benefits of using the same protocol across children is that we are more aware
of “typical” and “unusual” or “unexpected” standout pragmatic language behaviors.
SLP Behaviors. We were all taught in our graduate training that adult behaviors affect
the quality of child language samples. Our experience underscores this truth. We have documented
that several adult behaviors sabotage our ability to observe if a child can initiate and develop topics.
The language sample excerpt below was collected from one of our first participants, a 7-year-old
with ADHD. His CCC-2 scores revealed an overall communication deficit with relatively lower scores
in the pragmatic language scales. Unfortunately, we did not uncover these parent-reported deficits
in his language sample. His sample begins with the final sentence of the surprise birthday party
script (see Appendix A):
1. A: And it was a surprise party.
2. A: What happen/ed at your birthday party?
3. A: Do you have a party?
4. C: (um) one time I had a birthday party.
5. A: And what was it like?
6. C: It was ok.
7. A: Yeah?
8. A: now let me ask you about your little brother.
9. A: What/’s your little brother/z name?
10. C: My step little brother/z name is Maxim.
11. A: Maxim?
12. A: Does he ever do anything that bother/3 s you, like break any of your toy/s or
anything like that?
13. C: Mm, no sometimes *he just come/3 s right on me when I/’m play/ing with some
of his toy/s.
14. A: Is he cute?
15. A: Or is he annoying?

26
 16. C: Mm, he/’s a little baby.
17. A: So do you like playing with him?
18. A: Or does he bother you?
19. C: He/’s ok.
20. A: Do you have any pet/s?
21. C: Yeah.
22. A: What kind of pet/s do you have?
23. C: One turtle.
24. A: Tell me about that turtle.
25. C: (He/’s) he/’s good.
26. C: Michelangelo or Mikey or Mike.
27. A: What does he do?
28. C: All he does is swim in a dirty cage.
29. C: Mom never clean/3 s it.
30. A: Does it smell?
31. C: A little bit.
In hindsight, we believe that we did not give this child appropriate opportunities to show us how
he would develop a topic or to observe whether, as his mother reported, he had problems explaining
“a past event clearly” (CCC-2 coherence scale item). Lines 3, 12, 14, 15, 17, 18, 20, and 30 are
yes/no questions. We knew from graduate training that yes/no questions elicit single-word
responses, but we did not fully appreciate the need to monitor our use of these questions very
closely, particularly when faced with a child who was reluctant to speak with us. We now try to
limit yes/no questions to no more than three questions per 4-min segment. We avoid wh-questions
too (see Lines 2, 5, 9, 22, and 27). We have worked to replace yes/no and wh-questions with “I” +
mental state verb statements.” My undergraduate and graduate students practice this behavior
in the laboratory before collecting samples from child participants. For example, instead of “Does
he ever do anything that bothers you?” (Line 12), we practice saying statements such as “I wonder
what Maxim is like.” During our interactions with a child, we follow these statements with pauses
and a look of eager anticipation, sometimes for 15–30 s, to see if the child responds with a contingent
comment or question. In addition to using fewer questions and implementing pause time to facilitate
a child’s turn, we use more contingent comments ourselves. Now, instead of “Does it smell?” (Line
30), we would replace with “Ooh, I bet that’s stinky” with exaggerated emotion and wait for the child
to respond. We have found that, when we provide an authentic comment to something the child
has said, the child is more likely to provide additional information. We also attempt to stage at
least one communication breakdown per segment. For example, when the child states that he has
a pet turtle, we might repeat it back incorrectly such as “Oh wow you have a parrot!” This “repeat
it back incorrectly” strategy provides an opportunity to observe if the child is monitoring what we
say and if the child repairs the communication breakdown. Often, communication breakdowns
occur spontaneously especially in the text-level discourse segments of exposition and story retell.
During spontaneous naturally occurring breakdowns, we convey our confusion first by looking
puzzled. If the child does not respond to our puzzled expression, we produce an explicit statement
such as “I’m confused about what happened to the boy.” We are then able to judge whether the
child responds to our facial expression and if the child attempts to repair the communication
breakdown. “Rephrasing when not understood” is one of the conversation “rules” described in the
SPCD diagnostic criteria. Successful repair of a communication breakdown reflects a speaker’s
27
ability to read nonverbal cues and meet the needs of the listener, important skills for effective
conversations.
Recording
Live documentation of SLP–child language samples is not recommended when the purpose
of the sampling is to identify pragmatic language deficits. The normal to-and-fro of conversational
interactions is interrupted when the SLP looks away from the child to transcribe child utterances
as the child is speaking. In our experience, school-age children stop talking to look at what is
being written. As such, this practice may disrupt the child’s thinking and responsiveness to the
SLP’s questions and comments. Moreover, it is important that the SLP is pragmatically appropriate
herself as pragmatic analyses focus across speaker turns (e.g., expected answer to a partner’s
questions, ability to produce three consecutive or adjacent utterances to retell a past event).
Video recording of the language sampling is preferred to audio recording. Video recording
allows for capture of nonverbal behaviors such as eye gaze, gesture use, facial expressions, and
body positioning. Our early samples were audio-recorded only, and we regret the missing visual
information, particularly when parents have reported difficulty in several of the CCC-2 nonverbal
communication items such as “stands too close” or “looks blank where most children would
have a clear facial expression.” Recall that we recommend using a combination of pausing and
expectant waiting in sample collection. When a child does not respond to our question and
comment, we wish that we had the visual information to document whether the child was attentive,
appeared to be thinking of a response, or was distracted.
What to Look for? Strategies for Language Sample Analyses
Transcription and Language Analysis Software. We transcribe our language samples in
SALT (Miller & Iglesias, 2008), which can be purchased from the developers. Free language analysis
software is also available including Child Language Analysis (MacWhinney, 2000). The primary
advantage of using software to analyze a sample is that summaries of multiple language behaviors
are available with only a click. For example, in addition to standard semantic and syntactic
measures, such as the total number of different words and the average number of words per
utterance, SALT software provides a discourse summary that includes the percentage of child
utterances that were response to questions and the number of consecutive speaker turns (i.e., a
proxy for topic maintenance). Moreover, SLP-developed line-by-line codes, such as “off-topic” or
“unexpected response,” can be added to the transcript. These codes can also be automatically tallied
(Miller, Andriacchi, & Nockerts, 2015). The disadvantage is that line-by-line coding of language
samples is time consuming and may not be feasible for all clinical practice settings (Adams, Gaile,
Lockton, & Freed, 2011). One alternative to line-by-line coding is text level coding. Text level coding
summarizes child performance across multiple utterances. Thus, the listener reviews the entire
sample before selecting an overall rating or score (Miller et al., 2015). SALT software includes text-
level coding schemes for narrative and expository discourse samples. In our laboratory, we have
been using a text-level rating system to make judgments of a child’s pragmatic language behaviors
after reviewing all three discourse segments elicited from our 12-min language sampling protocol.
These ratings are then entered in SALT. If an SLP is not using language analysis software, ratings
can be documented with pen and paper using one of the pragmatic language analyses summarized
in the next section.
Pragmatic Language Analyses. Although SPCD is a relatively new diagnosis, SLPs have
been serving students with pragmatic language deficits for decades. Prutting’s 1982 classic paper,
“Pragmatics as Social Competence,” introduced the concept of pragmatics as an aspect of language
use and recommended that both verbal and nonverbal behaviors be examined. Prutting and
Kirchner’s (1987) Pragmatic Protocol continues to be used in research and clinical settings. Copies
of this protocol are readily available via a Google search. The protocol lists 30 verbal and nonverbal
behaviors. Each behavior is rated as appropriate, inappropriate, or no opportunity to observe.
Although this protocol provides comprehensive coverage of pragmatic behaviors, the rating scheme

28
of appropriate or inappropriate may not allow for capturing small but important changes in
pragmatic language skills as a result of intervention. For example, a child’s comments may be
inappropriate but less so than before an intervention; such changes would not be captured by the
original rating system.
The Pragmatic Rating Scale (PRS) is a 30-item scale focused on problematic pragmatic
language behaviors (Landa et al., 1992). For example, items include “confusing accounts,”
“overly talkative,” and “content out of sync with interlocutor.” Each item is rated as 0 = normal,
1 = moderately inappropriate, or 2 = absent or highly inappropriate; thus, higher scores indicate
more concern. The PRS was developed to capture the pragmatic language deficits of individuals
with ASD and their family members. This scale has been used successfully to discriminate the
language behaviors of adolescents with and without high-functioning ASD (Paul, Miles Orlovski,
Chuba Marcinko, & Volkmar, 2009). SLPs who serve individuals with ASD may find this scale
useful for documenting impairments in this population.
A relatively new pragmatic analysis tool is the Targeted Observation of Pragmatics in
Children’s Conversations (TOPICC; Adams, Gaile, Freed, & Lockton, 2010; a free download of this
tool is available online from The University of Manchester, n.d.). The TOPICC was developed to use
as an outcome measure for a series of studies investigating the efficacy of a social communication
intervention (see Adams, 2015, for details of the intervention). The TOPICC consists of six categories
of pragmatic language behavior, with several items focused on problematic behavior within each
category. For example, the reciprocity/turn-taking category includes three items to be rated:
difficulties responding to questions, interrupts speaker frequently or frequent pauses, and reticence
(defined as the child is hesitant to contribute to conversation). Items are rated using a 4-point Likert
scale from 0 = never observed and the behavior is typical of mature interaction style to 3 = marked
evidence of that behavior across conversation. The TOPICC was designed as an online observation
tool. Ratings are completed while administering the recommended TOPICC sampling protocol
consisting of a semistructured conversation supported by pictures (see Bishop & Adams, 1989,
for protocol details). Importantly, the content validity of the TOPICC has been examined by comparing
individual CCC-2 pragmatic language items with TOPICC item ratings. Acceptable agreement
between these measures was reported (i.e., 0.88% agreement; Adams et al., 2011).
We applied the Pragmatic Protocol, the PRS, and the TOPICC to a review of Henry’s 12-min
language sample. Below is one excerpt from his personal retell segment and two excerpts from his
exposition segment focused on how to play soccer and how to play his favorite video game, Modern
Warfare 2:
1. C: (My um) my Chinese Akita would chase squirrel/s up tree/s.
2. A: Oh gees.
3. C: Got one (got one) in (his life) in her life.
4. C: Yeah she got it kill/ed.
5. C: Like her eye was over here and yeah.
6. A: That/’s gross!
7. : Pause [.04]
8. C: (It was cu) He was just gone.

1. A: Ok, tell me about soccer.

2. C: Well I/’ve never play/ed it on a real team.
3. C: I/’ve play/ed it at school though.
4. C: And I think it/’s fun because you don’t use your hand/s.
29
 5. C: and it/’s kind of challenging that way.
6. C: And so like it/’s fun because you have to pass it.
7. C: and they/’ll sometimes steal it from you.
8. C: (And your) and you sometimes get hurt.
9. A: Oh no!
10. C: But I (I) like get/ing hurt.
11. C: I laugh at it.
12. C: I laugh at pain.
13. C: (So does my ma) so does (my um) my sister and the terrible two/s so does she.
14. C: Like (like um) she broke my brother/z nose twice.

1. A: Oh, Modern Warfare 2 sounds like a good one!

2. C: Well it/’s about (this) these people call/ed the task force.
3. C: And army ranger/s.
4. C: And they/’re try/ing to fight to get (um) to (de*) defeat a guy name/ed Macral.
5. C: And (um a) the captain of the task force that betray/ed the task force.
6. C: and he made his own (te) team fill/ed with thousand/s of people.
7. C: and he kill/ed almost two people.
8. C: Yeah it was two people.
9. C: (wha) And he got a DNS thing to get (um) headquarter/s where Macral was gonna be.
10. C: So there/’s like a big war that you had to go through.
11. A: I don’t know what a DNS is.
12. A: I/’m confused.
13. C: A DNS is like a thing.
14. C: You know thing/s that (you) give you online computer stuff.
15. A: Okay.
16. C: (One of them) but it/’s kind of like armor/ed.
17. C: And like he try/ed to get a DNS the whole entire time.
18. C: And he want/ed that so he can defeat Marcal.
19. C: And his only thing was to defeat Marcal in the task force.
20. A: Okay.
21. C: (So he was pretty much like not) he was American for a while.
22. C: but he betray/ed it.
The Pragmatic Protocol, PRS, and TOPICC all include items that identified areas of concern in
Henry’s pragmatic language behaviors. Moreover, items of concern aligned with some of the CCC-
2 items reported as problematic by his mother. For example, in these three excerpts, Henry

30
demonstrates problems with cohesion and his mother reported that “it’s hard to make sense of
what he is saying even though his words are clear.” In standard clinical practice, an SLP would
need to select only one of these protocols to analyze a sample. We recommend that SLPs review
these analysis protocols and select one that meets the needs of their clinical populations and
setting. Use of the same analysis protocol across children will facilitate the ability to quickly
identify problematic or unusual behaviors, the same benefit that we suggested for use of an
identical language sampling protocol. If an SLP’s assessment goal is to identify or rule out SPCD
in a school-age child or adolescent, pragmatic analysis of language samples is likely the only
consistent method for identifying the SPCD diagnostic criterion of conversation and storytelling
deficits.
Because language sampling and analysis appear so promising as the one assessment tool
most likely to validate parent concerns, we are developing a rating scale to be used for analyzing
samples collected with our 12-min sampling protocol. The rating scale incorporates items from
the Pragmatic Protocol, PRS, and TOPICC. Currently, the scale is composed of 11 pragmatic
categories: responding to questions, asking questions, balance in turn-taking, topic initiation,
topic details, topic maintenance, pronoun use/references, formal language use/stereotypic
language, communication breakdowns, proximity, and nonverbal behaviors. Each category is
rated on a Likert scale from 0 = no concerns/appears appropriate for age to 3 = significant/severe
concerns that would likely be always noted by communication partners and/or significantly
disrupt/distract interactions with the communication partner. Identical to the TOPICC, examples
of individual pragmatic behaviors are included within each category. The rater “checks” individual
behaviors that appear problematic or are of concern. We believe that identification of specific
behaviors can inform intervention planning. For example, behaviors for the category of responding
to questions are as follows: usually does not respond to the question, responds before partner
has finished asking the question, repeats question without responding, answers the question
but gives an unexpected or inappropriate response, and does not seem to understand questions.
We are completing scale ratings while listening/looking at a sample rather than from our language
transcripts. Raters review the sample up to three times, stopping as needed to complete the
ratings. Examination of scale reliability and validity is in progress. We are hoping that this tool
will yield a composite summary score that could be used to discriminate children with and without
pragmatic language concerns. We will also investigate the ability of this tool to serve as an
intervention outcome measure.
What to Remember? Best Practice Recommendations
If completed systematically, the results of language sampling and analysis appear to offer
the one ecologically valid assessment procedure most likely to align with and validate parent
concerns about a child’s pragmatic language skills. Best practices for collection and analysis
of these samples based on the experiences in our social communication laboratory include the
following:
1. Collect a parent and teacher report measure first and use the results to guide your
analysis and observation plans. We recommend the CCC-2 for parents of children
aged 4;0–16;11 years;months. See Appendix B for a list of other report measures.
2. Use the same sampling protocol for all assessments. Systematic use of the same
protocol increases your awareness of differences and similarities across children,
which will support your ability to identify problematic or standout pragmatic language
behaviors more readily. If time permits, collect samples from several typical children
to provide a preliminary benchmark of appropriate pragmatic language behaviors.
3. Monitor your sample elicitation techniques. Reduce number of yes/no and wh-questions.
Replace with contingent commenting yourself to facilitate continuation of the child’s
topic. Use pausing and expectant waiting to allow children time to develop a topic (or to

31
 show that they struggle with topic development). Stage communication breakdowns to
examine whether children notice breakdowns and attempt to repair them.
4. Video-record samples so that you can document both verbal and nonverbal behaviors.
5. Consider transcribing the sample to facilitate progress monitoring. Pragmatic
language analysis may be completed by listening to video recordings of the sample,
but transcription of the sample provides additional benefits. For example, transcribed
samples provide a means to conduct syntactic and semantic analysis such as average
number of words or morphemes per sentence or number of different words in the
sample. Moreover, transcription facilitates specific counts of particularly problematic
pragmatic language behaviors such as child interruptions or off-topic comments.
These counts may provide useful baseline data for comparison with data collected
after a period of intervention.
6. Review the three pragmatic language analyses presented in this article to determine
which protocol is the best match for your clinical practice. The use of the same analysis
protocol is likely to increase an SLP’s personal reliability (i.e., rating agreement). Rating
agreement is important so that the SLP feels confident about the validity or accuracy
of observed changes when rating the same child preintervention and postintervention.

References
Adams, C. (2002). Practitioner review: The assessment of language pragmatics. Journal of Child Psychology
and Psychiatry, 43(8), 973–987.
Adams, C. (2015). Assessment and intervention for children with pragmatic language impairment. In
D. Hwa-Froelich (Ed.), Social communication development and disorders (pp. 141–170). New York, NY:
Taylor & Francis.
Adams, C., Gaile, J., Freed, J., & Lockton, E. (2010). Targeted Observation of Pragmatics in Children’s
Conversation (TOPICC) Observation Scale.© Available at research.bmh.manchester.ac.uk/scip/topic.pdf
Adams, C., Gaile, J., Lockton, E., & Freed, J. (2011). TOPICCal applications: Assessing children’s
conversation skills: Turning a research instrument into a clinical profile. Speech & Language Therapy in
Practice, no. Spring, 7–9.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.).
Washington, DC: Author.
Bishop, D. (2006). Children’s Communication Checklist-2 (U.S. ed.). San Antonio, TX: Harcourt Assessment.
Bishop, D., & Adams, C. (1989). Conversational characteristics of children with semantic pragmatic
disorder. 2: What features lead to a judgement of inappropriacy? British Journal of Disorders of Communication,
24, 214–263.
Bowers, L., Huisingh, R., & LoGiudice, C. (2008). The Social Language Development Test–Elementary. East
Moline, IL: LinguiSystems.
Hadley, P. (1998). Language sampling protocols for eliciting text level discourse. Language, Speech, and
Hearing Services in Schools, 29, 132–147.
Heilmann, J. (2010). Myths and realities of language sample analysis. Perspectives on Language Learning
and Education, 17, 4–8.
Izaryk, K., Skarakis-Doyle, E., Campbell, W., & Kertoy, M. (2015). Social communication functioning: An
appraisal of current assessment tools through the lens of the International Classification of Functioning,
Disability, and Health-Child & Youth Version. Communication Disorders, Deaf Studies, and Hearing Aids,
3(3), 134. https://doi.org/10.4172/2375-4427.10000134
Landa, R., Piven, J., Wzorek, M., Gayle, J., Chase, G., & Folstein, S. (1992). Social language use in parents
of autistic individuals. Psychological Medicine, 22, 245–254. https://doi.org/10.1017/S0033291700032918
MacWhinney, B. (2000). The CHILDES project: Tools for analyzing talk (3rd ed.). Mahwah, NJ: Lawrence
Erlbaum Associates. Retrieved from http://talkbank.org/

32
McCabe, A., Bliss, L., Barra, G., & Bennett, M. (2008). Comparison of personal versus fictional narratives of
children with language impairment. American Journal of Speech-Language Pathology, 17, 194–206. https://
doi.org/10.1044/1058-0360(2008/019)
McCabe, A., & Rollins, P. R. (1994). Assessment of preschool narrative skills. American Journal of Speech-
Language Pathology, 3, 45–56. https://doi.org/10.1044/1058-0360.0301.45
Miller, J., Andriacchi, K., & Nockerts, A. (2015). Assessing language production using SALT software: A
clinician’s guide to language sample analysis (2nd ed.). Middleton, WI: SALT Software, LLC.
Miller, J., Andriacchi, K., & Nockerts, A. (2016). Using language sample analysis to assess spoken language
production in adolescents. Language, Speech, and Hearing Services in Schools, 47, 99–112.
Miller, J., & Iglesias, A. (2016). Systematic Analysis of Language Transcripts (SALT), Research Version 16)
[Computer software]. Madison, WI: SALT Software LLC.
Miller, J. F., & Iglesias, A. (2008). Systematic Analysis of Language Transcripts (SALT), English &
Spanish (Version 9) [Computer software]. Madison: University of Wisconsin—Madison, Waisman Center,
Language Analysis Laboratory.
Nippold, M. (2014). Language sampling with adolescents: Implications for intervention. San Diego, CA: Plural
Publishing.
Norbury, C. F. (2014). Practitioner review: Social (pragmatic) communication disorder conceptualization,
evidence, and clinical implications. Journal of Child Psychology and Psychiatry, 55(3), 204–216. https://
doi.org/10.111/jcpp.12154
Paul, R., Miles Orlovski, S., Chuba Marcinko, H., & Volkmar, F. (2009). Conversational behaviors in youth
with high-functioning ASD and Asperger syndrome. Journal of Autism and Developmental Disorders, 39,
115–125. https://doi.org/10.1007/s10803-008-0607-1
Paul, R., & Norbury, C. (2012). Language disorders from infancy through adolescence: Listening, speaking,
reading, writing, and communicating (4th ed.). St. Louis, MO: Mosby.
Pavelko, S. L., Owens, R., Ireland, M., & Hahs-Vaughn, D. L. (2016). Use of language sample analysis by
school based SLPs: Results of a nationwide survey. Language, Speech, and Hearing Services in Schools, 47,
246–258.
Phelps-Terasaki, D., & Phelps-Gunn, T. (2007). Test of Pragmatic Language–Second Edition (TOPL-2).
San Antonio, TX: The Psychological Corporation.
Prutting, C. A., & Kirchner, D. M. (1987). A clinical appraisal of the pragmatic aspects of language. Journal
of Speech and Hearing Disorders, 52, 105–119.
SALT Software. (2017). Retrieved from http://languageanalysislab.com/
Schuele, C. M. (2010). The many things language sample analysis has taught me. Perspectives on Language
Learning and Education, 17, 32–37.
Swineford, L. B., Thurm, A., Baird, G., Wetherby, A. M., & Swedo, S. (2014). Social (pragmatic) communication
disorder: A research review of this new DSM-5 diagnostic category. Journal of Neurodevelopmental Disorders,
6, 41. Retrieved from http://www.biomedcentral.com/content/pdf/1866-1955-6-41.pdf
The University of Manchester. (n.d.). Retrieved from http://research.bmh.manchester.ac.uk/scip

History:
Received November 22, 2017
Accepted December 07, 2017
https://doi.org/10.1044/persp3.SIG1.23

33
Appendix A. Scripts of Adult Models to Elicit Personal Retell,
Exposition, and Story Retell Segments
Personal Retell Scripts
Problem with sibling. “One time when I was younger I bought a CD player with my own
money. My brother was playing with it when I wasn’t home one day and he broke it! I was so
mad. I had to wait a whole year before I could get a new one.”
Problem with a pet. “My dog’s name is Pepper. One day, he was playing outside and he
saw a bunny. He jumped over the fence to chase the bunny. Then, I had to jump over the fence to
chase after him. Finally, the bunny went into a hole and Pepper came back to me. I was so relieved!”
Problem with birthday. “This year I thought all of my friends forgot my birthday because
no one called me to wish me a happy birthday- I was really sad. The day after my birthday, I
thought I was going out to dinner with my best friend, but when I got there all my friends were
there. It was a surprise party! My friends didn’t forget my birthday after all!”
Exposition Scripts
Board game. “I love to play monopoly. When I play monopoly, I like to be the banker. The
banker gets to pass out and collect all the money. I wait until someone takes a turn and rolls the
dice. Then, I see where they land. If they are supposed to get some money, then I give it to them.
For example, sometimes you get to pick a card and the card might say something like “Win a
prize, collect $100.00. Then I get to give them the money. If the card says that they owe money,
then I get to collect it! I think monopoly is fun because anything can happen and a different
person always wins. But when you’re the banker, you can’t lose!
Science experiment. “My favorite class in school was science class. I loved to do experiments.
One time we did an experiment to see if objects would float or sink in water. If the object floated,
then we had to write down what the object was made of. If it didn’t float, we had to guess why
it didn’t float. It was fun, but some people got wet because they were fooling around instead of
listening to the teacher. My teacher was annoyed with us!”
Book/Movie Retell Script
Movie: I like the Harry Potter movies! I’ll tell you about one of my favorite scenes. In Harry
Potter and the Chamber of Secrets, Harry and Ron went down into the chamber of Secrets where
they landed on old crunchy bones. It was dark and damp down in the chamber. Harry walked
through the slimy tunnel alone to find Ginny. He was scared because he heard the hissing of the
snake. I like the scary parts!

Appendix B. Parent and Teacher Communication Report Measures

Gilliam, J., & Miller, L. (2006). Pragmatic Language Skills Inventory. Austin, TX: Pro-Ed.
Teacher report measure; norm-referenced ages 5;0 to 12;0. Forty-five items across three subscales:
classroom interaction skills, social interaction skills, and personal interaction skills.
O’Neill, D. (2009). Language Use Inventory. Waterloo, Canada: Knowledge in Development.
Retrieved from http://www.knowledgeindevelopment.ca
Parent/primary caregiver report measure; ages 18–47 months. Includes 14 subscales to assess a
broad range of pragmatic functions including adapting communication to perspectives of others,
commenting about things and people, etc.). Provides percentile ranks by 1-month age bands.
Wiig, E., Semel, E., & Secord, W. (2013). Clinical Evaluation of Language Fundamentals–Fifth
Edition, Pragmatics Profile, Pragmatic Activities Checklist, and Observational Rating Scale.
San Antonio, TX: The Psychological Corporation.

34
Parent, teacher, and student self-report, ages 5;0–21;11. The Pragmatics Profile includes items
for nonverbal communication skills and conversational skills including asking for, giving, and
responding to information. Norm-referenced scaled scores are provided. The Pragmatic Activities
Checklist is a criterion referenced measure of negative behaviors. SLPs need to observe three
activities completed by the student and then identify the presence of undesirable behaviors. Clinical
cut-off for concern is a score of 10 or more negative behaviors (32 items total). The Observational
Rating Scale includes items for listening, speaking, reading, and writing and can be completed
by parents, teachers, and students as a screening or part of a comprehensive assessment. The
examiner’s manual recommends that raters circle 5-10 behaviors that are of most concern. This
is a descriptive measure; specific cut-off scores or criteria are not provided.
Wiig, E. (2008). Social Emotional Questionnaire. Part of the Social Emotional Evaluation assessment.
Retrieved from https://www.superduperinc.com/
Parent and teacher forms; criterion referenced screening tool, ages 6;0–12;11; 45 items

35
Copyright of Perspectives of the ASHA Special Interest Groups is the property of American
Speech-Language-Hearing Association and its content may not be copied or emailed to
multiple sites or posted to a listserv without the copyright holder's express written permission.
However, users may print, download, or email articles for individual use.