READING

AND AUDITORY-VISUAL EQUIVALENCES

MURRAY SIDMAN

Massachusetts General Hospital, Boston, Massachusetts

A retarded boy, unable to read printed words orally or with comprehension, could match spoken words to pictures and could name pictures. After being taught to match spoken to printed words, he was then capable of reading comprehension (matching the printed words to pictures) and oral reading (naming the printed words aloud). Reading may be regarded broadly as a type of stimulus-response relation in which the controlling stimuli are visual words-written or printed text. Within this general type of stimulus-response relation, several subcategories may be identified. One is oral reading. A simple example: If we show a child the word, bay, and he says boy, he indicates that he can read the word orally. Oral reading may or may not involve comprehension; for example, one can read words in a foreign language aloud without understanding them. Oral reading may, in fact, be more appropriately called "oral naming of words." As such, it may be no different than the oral naming of objects, or the pictures of objects. A common observation, however, is that children generally learn to name objects or pictures aloud before they learn to name the corresponding printed words. To demonstrate reading comprehension, we require a different kind of stimulus-response relation. A simple example: If we show a child the printed word, bay, and he is then able to select a picture of a boy out of several other pictures, we say that he understands the word. One simple kind of reading comprehension, then, may be demonstrated by the child's accurate matching of printed words to pictures. Defined this way, reading comprehension is a purely visual task. Note that one may be capable of this kind of reading comprehension without being able to read the words orally. A third stimulus-response relation, rarely discussed explicitly, might be termed, auditory-receptive reading. For example, we say the word "bay," to a child, and he is then to select the word boy out of several other printed words. This differers from oral reading in that the word is spoken to, and not by, the child. Nevertheless, discussions of the role of auditory-visual equivalences in reading often confuse oral reading and auditory-receptive reading under some such common heading as word recognition. Like oral reading, auditory-receptive reading may or may not involve comprehension, either of the auditory (spoken) or the visual (printed) word. As

noted before, simple visual comprehension can be tested by a visual word-picture matching task. Similarly, simple auditory comprehension can be tested by matching auditory words to visual pictures: We say the word boy, to a child, and he is then to select the picture of a boy out of several other pictures. Several lines of converging evidence and theory have led many writers to postulate that reading comprehension, a visual task, evolves from the previous learning of auditory-visual equivalences (Birch, 1962; Geschwind, in press; Wepman, 1962). First, there are certain common observations of normal developmental sequences: ( 1 ) Children normally understand words they hear before they learn to read with comprehension; auditory comprehension of words usually preceds visual comprehension. (2) Children usually name objects, or pictures of objects, before they learn to name the printed or written words that correspond to those objects; object naming precedes word naming (oral reading). Most children break through the "sound barrier" in the first or second grade, and learn to understand not just words they hear, but words they see. They also learn not just to name pictures, but to read words orally. A large group of retarded children and dyslexic children, however, have not made the transfer from auditory comprehension and picture naming to visual reading comprehension and oral reading. It is likely that this transfer marks a critical point in the development of behavior and the central nervous system. A second indication that auditory-visual equivalences and reading are closely linked comes from correlational studies by Birch and his coworkers (Birch and Belmont, 1964, 1965; Kahn and Birch, 1968). Using a test of auditory-visual integration (matching sound patterns to visual patterns), they found positive correlations between scores on this test and scores on standard reading achievement tests. A third set of considerations comes from the neuroanatomical theories of Geschwind (1965), who takes as his starting point the observations, like those noted above and others, that cross-modal equivalences and language are closely linked. He has proposed that cross-modal equivalences, particularly auditoryvisual, actually make language possible. Furthermore, he has suggested that the evolution of the angular gyrus region, strategically located at the junction of auditory, visual, and somesthetic association cortexes, makes that region the prime candidate as the central-nervous-system site for the mediation of crossmodal equivalences. As a consequence, the angular gyrus is held to be critical for language in general and for reading in particular. Geschwind has proposed that developmental dyslexia may be correlated with the slow maturation of the angular gyrus bilaterally, or perhaps even with its failure to develop. In spite of these empirical and theoretical considerations, and in spite of the educational practices (e.g., the "look-say" method of teaching reading) that are based on them, the question of whether auditory-visual learning is indeed a necessary or even a suflqeient prerequisite for the development of oral reading or reading comprehension seems not to have been studied experimentally. The experiment to be described now will demonstrate that certain learned auditoryvisual equivalences are indeed sufficient prerequisites for the emergence of
6 Journal of Speech and Hearing Research

14 5-13 1971

reading comprehension, even without explicitly teaching reading comprehension. Although the data raise a number of unanswered questions, the major finding is sufficiently provocative, and relevant to both theory and teaching practice, to warrant this report in advance of more extended studies. METHOD

Subiect
The subject of the experiment was an institutionalized 17-year-old boy, microcephalic and severely retarded. During the past two years, he had extensive experience with the apparatus and matching-to-sample procedures described below. The following findings are a relevant background for the present experiment: He was able to match pictures, colors, and printed numbers to picture names, color names, and number names that were spoken aloud to him. But he was unable to do the matching correctly when the names were presented to him visually rather than spoken. Also, he could name the pictures aloud, but not the corresponding printed words. Therefore, he showed good auditory comprehension and picture naming, but little if any reading comprehension or oral reading. He could not write.

Apparatus and Procedures

The subject sat before a panel of 9 translucent windows, each 2 inches square, arranged in a 3 3 matrix. Visual stimuli Were projected from the rear onto the windows (Rosenberger et al., 1968). Each trial began by presentation of a sample stimulus. Visual word or picture samples appeared on the center window of the matrix; auditory word samples, repeated at 2-second intervals, were dictated from tapes over a speaker (Figure 1, left column). In matching tests, the subject pressed the center window to bring choice stimuli, always visual, onto the outer windows of the matrix. Schematic examples of the displays are in the second column of Figure 1. On each trial, one choice, the correct one, corresponded to the sample; the other seven choices did not. The subject selected and pressed one of the choice windows. His correct choices were rewarded by chimes ringing and delivery of a candy and a penny. No rewards followed incorrect choices. The stimuli disappeared after each choice, and 1.5 seconds later a new sample began the next trial. In oral naming tests, the subject had simply to name the sample picture or word aloud. Reward procedures were the same as in the matching tests. Each test had 20 trials. The sample and choice stimuli, taken from a list of 20 pictures, or the printed (lower case) or spoken names of the pictures were:

axe, bed, bee, box, boy, bug, car, cat, cow, dog, ear, hat, hen, hut, ho~, man, pie, pig, saw, zoo.
Preliminary tests evaluated the subject's proficiency at simple comprehension and naming tasks; then, he was taught to match spoken to printed words; final SIDMAN: Reading and Auditory-Visual Equivalences 7

SAMPLE (AUDITORY OR V I S U A L )

RESPONSE ( N A M E OR MATC H) BASELINE CONTROL TESTS 10080TEACHING POSTTEACHING TESTS

oar
~CAT"

ear

key

(spoken t_.o

dot bet
kod

eat
eow

6040200., 10080, , I
, 0 , 0 <) , 0 , 0 t'q~p

subject)

9 .~

~&

NM

-I
I

Iq. k~ t~ t ~ l ~

"CAT"
eat

(spoken by

b040200.... 9

subject)

10080eat

6040-

,mr

omr

hol

~) t 0 0 *- 80v)

I
'

-a ~
hut

,,t
eow

~ 6oi40200-' 100~

hod

nn

~C AT m

60b_y 40-

(spoken

subject)

2oO.s i 100!

~' C A T "

(spoken to

subject)
*O1~ qr qr , 0 t~

[
qt~

FIGUItE 1. In the two left columns are examples of the sample stimuli and responses that comprised each type of test. Choice stimuli and correct window position in the matching tests varied from trial to trial. The three columns of bars represent scores in each depicted test during the three phases of the experiment. Absenco of a bar means no test on the indicated date. Letters identi~ the six auditory-visual word matching sets (uppermost row).

8 1ournal of Speech and Hearing Research

14

5-13

1971

tests evaluated the effects of this teaching on his reading comprehension and word naming. RESULTS Each row of bar graphs shows the subject's test scores on the task depicted at the left.

Baseline Control Tests

The results of preliminary tests are in the left column of bar graphs in Figure 1. Bars at the lower left show the subject's scores in tests that required him to match spoken word samples to picture choices. In four tests, administered from April 1967, to July 1969, he scored from 60 to 95% correct, demonstrating a fair proficiency at this type of auditory comprehension. He also scored 85% in naming the pictures (second row from bottom). In reading-all tests that involved printed words-the subject scored poorly. Continuing up the left column, these tests were: Matching picture samples to printed word choices; matching printed word samples to picture choices (3 tests over 2 years); naming printed words; and matching spoken word samples to printed word choices. The possibility that the subject could not distinguish the printed words from each other was ruled out by his score of 95% in matching printed word samples to printed word choices (not shown in Figure 1). The words in this test were the same 20 that comprised the other tests. Also, the two types of control tests at the bottom of Figure 1 (matching spoken words to pictures, and picture naming) show that the subject could already distinguish the pictures from each other, and the auditory words from each other, and that he could say the words aloud. His difficulties were neither with the discrimination of the stimuli used here, nor with the oral responses, but were specifically with the stimulusresponse relations that operationally define simple reading comprehension, oral reading, and auditory-receptive reading.

Teaching Auditory-Receptive Reading

Teaching the subject to match auditory to visual words was the critical experimental operation. Figure 2 will illustrate the logic of the experiment and will serve as a basis for later discussion. The three boxes at the left and center of Figure 2 represent the three types of stimuli, and the arrows represent stimulus equivalences as defined by the matching performance. The arrows connecting the center boxes to the right hand box represent the two naming performances, picture naming and oral reading. The subject came to the experiment knowing the equivalence of spoken words to pictures (Equivalence I). Would teaching him auditory-visual (Equivalence II), spoken words to visual words, suffice to establish reading comprehension, SIDMAN.' Reading and Auditory-Visual Equivalences 9

VISUAL PICTURES I I I

AUDITORY WOR D S

ORAL NAMING

1
III llV I I I I I
J i/ o

(SPOKEN TO
SUBJECT)

#
I I /V I I I

(SPOKEN

B,Y

ii\

"~[

SUBJECT)

VISUAL WORDS

FZCURE2. Schematic summary of the experiment. Of the stimulus equivalences, I-IV, the subject came to the experiment knowing I. Of the naming tasks, V and VI, he could do V. After being taught equivalence II, he could then do III, IV, and VI. the purely visual equivalence of printed words to pictures (Equivalences III and IV)? He also came to the experiment able to name the pictures (V); given this ability, would teaching him auditory-visual word matching suffice for oral reading (VI) to emerge? In the teaching procedure, sample stimuli were words spoken to the subject; choices were printed words (Figure 1, top row). Teaching differed from testing in several ways. (1) A correction procedure was used; when the subject chose a wrong printed word, the display remained unchanged until he pressed the correct window. (2) Errors had different consequences; if the child made one or more errors on a given trial, the chimes rang when he finally pressed the correct window, but he did not receive candy or a penny. (3) Each phase of the teaching procedure started with only two trials (sample-choice combinations); the two being repeated until the subject's first choices on both were correct. Then a third trial was added; when his first choices on all three were correct, a fourth was added. This progressive enlargement of the set continued as the subject attained each criterion of mastery, until his first choices were correct on the full set of twenty trials; (4) Six versions or sets of auditoryvisual word matching materials were used. Each set presented the same 20 sample words in different trial sequences, and displayed a different combination of seven wrong words along with each correct word. Set A was used for the preliminary control test. Then the subject was taught Set B until he scored 100%, and was tested on Set C. His low score on Set C (Figure 1, center section, first bar), suggested that his learning of Set B had been specific to the particular sequence of correct window positions and to the particular wrong words displayed along with each correct word. The subject then learned Set C, reviewed Set B to the same 100% criterion, and was tested on Set D. The process of learning, reviewing, and testing on a new set con10 ]oumal of Speech and Hearing Research 14 5-13 1971

tinued through Set F, and the center section of Figure 1 shows the gradual improvement on each new test. (The teaching process itself is not shown; only the test scores on each new set.) Finally, the subject was retested on Set A, which he had not seen since the preliminary test. The change from 20 to 8 0 ~ correct on Set A, one month after the preliminary test, demonstrated his new proficiency at the task.

Post-Teaching Tests
After the teaching, all comprehension and oral naming tests were administered once more. Scores are in the right column of Figure 1. The subject maintained his good performances on the first auditory-visual word matching set he had learned (upper right), in matching spoken words to pictures, and in picture naming (lower right). Of major interest are the subject's reading comprehension and oral reading tests (visual word-picture and picture-word matching; word naming). These improved greatly. Having learned to match spoken word samples to printed word choices, he was then able, without additional teaching, to match picture samples to the printed word choices, to match printed word samples to picture choices and to name printed words. Given the subject's initial ability to match spoken words to pictures, and to name the pictures, teaching him the second auditory-visual equivalence, spoken to printed words, sufficed for the emergency of purely visual reading comprehension and oral reading.

DISCUSSION The findings will be discussed with reference to Figure 2. A simple connectionistic interpretation of the emergence of reading comprehension might be that the visual words and pictures became equivalent to each other (III, IV) because each, independently, had become equivalent to the same auditory words (I, II). This would be entirely consistent with the theoretical suppositions of Geschwind (1965), particularly with respect to the integrating functions of the angular gyrus region. It is not clear from this experiment, however, whether equivalences I and II need be cross-modal. Suppose, for example, that visual nonsense syllables were substituted for the auditory words, and that arbitrary equivalences between these visual symbols and the words and pictures were taught to the subject. Would the words and pictures then emerge as equivalent to each other, even with a common intramodal, rather than a cross-modal linkage? That deaf children learn to read suggests an a~rmative answer. There is no need, however, to assume only a single mechanism for reading comprehension. The emergence of visual-word naming, or oral reading (VI)I complicates the simple connectionist view, since the equivalence of visual words to picSIDMAN': Reading and Auditory-Visual Equivalences 11

tures may have been mediated by naming (V, VI) rather than by auditory words. This, too, is testable. It should be emphasized, however, that even if the emergence of word naming permitted reading comprehension to develop, it did not do so through the auditory channel. The subject did not name the words or pictures aloud during the reading comprehension tests; the only auditory stimuli were the words spoken to him in previous tests and teaching sessions. Furthermore, whatever proves to be the role, if any, of the subject's ability to read orally in mediating the transfer from the cross-modal to the purely visual equivalences, the experiment has demonstrated that matching auditory words to pictures and to printed words are sufficient prerequisites for the emergence of both types of stimulus-response relation, reading comprehension, and oral reading. The identification of these sufficient prerequisites for reading comprehension suggests a most important practical consequence. Both auditory-visual equivalences (I, II) can be taught completely without the intervention of a teacher. Reading comprehension is usually taught by way of oral naming, and this does require that a teacher participate actively. Automated programs to teach reading comprehension via purely receptive auditory-visual training (Equivalences I and II) would permit a far larger number of children to be reached than is now possible. Furthermore, the technique provides a rapid method for determining whether a child who has not yet made the transfer from the auditory to the visual comprehension of words is actually incapable of passing through this apparently critical developmental stage. Independently of comprehension, one may ask whether oral reading (VI) will always emerge, in a child capable of speech, after the child has learned auditory-visual word matching. Guess (1969) has shown that receptive language training need not facilitate the learning of productive speech. The auditory stimuli Guess used were singular and plural object names, and he taught the children to match these spoken names to singular and plural objects, analogous to our Equivalence I. The children were unable, then, to use the correct singular and plural forms in naming the objects, analogous to our oral naming Task V. In addition to the differences in stimulus materials and responses (simple nouns versus singular and plural nouns), a likely reason for the discrepancy between Guess' experiment and the present experiment is that our subject was taught Equivalence II and tested on Task VI (word naming) after he had demonstrated his ability to do Task V (picture naming). If receptive training is to facilitate oral speech, it may be necessary for the child already to have the words in his own speech repertoire..

ACKNOWLEDGMENT This research was supported by Grant NS 03535 from the National Institute of Neurological Diseases and Stroke, and by the Joseph P. Kennedy, Jr., MemorialLaboratories, Neurology Service, Massachusetts General Hospital. I thank Osborne Cresson, Martha Wilson, and James Sidman for technical assistance. 12 Journal of Speech and Hearing Research 14 5-13 1971

REFERENCES BIaCH, H. G., Dyslexia and and the maturation of visual function. Ch. 10 in J. Money (Ed.), Reading Disability, Baltimore: Johns Hopkins (1962). BmCH, H. G., and BELMONT, L., Auditory-visual integration in normal and retarded readers. Amer. 1. Orthopsychiat., 34, 852-861 (1964). BllaCH, H. G., and BELMONT, L., Auditory-visual integration, intelligence, and reading ability in school children. Percept. Motor Skills, 20, 295-305 (1965). GESCHWIND, N., Aphasias and related disturbances. In R. W. Wilkins (Ed.), Textbook of Medicine, Boston: Little, Brown (in press). GESCHWXND, N., Disconnexion syndromes in animals and man: Part I. Brain, 88, 237-293 (1965). GuEss, D., A functional analysis of receptive language and productive speech: Acquisition of the plural morpheme. 1. Appl. Behav. Analysis, 2, 55-64 (1969). KAHN, D., and BmCH, H. G., Development of auditory-visual integration and reading achievement. Percept. Motor Skills, 27, 459-468 (1968). ROSENnEnCEa, P. B., MOHR, J. P., STODDAP, D, L. T., and SIDMAN,M., Inter- and intramodality matching deficits in a dysphasic youth. Arch. Neurol., 18, 549-562 (1968). WEPMAN, J. M. Dyslexia: Its relationship to language acquisition and concept formation. Ch. 12 in J. Money (Ed.), Reading Disability, Baltimore: Johns Hopkins (1962).

Received January 2, 1970.

SIDMAN: Reading and Auditory-Visual Equivalences

13

Reading and Auditory-Visual Equivalences Murray Sidman J Speech Hear Res 1971;14;5-13

This article has been cited by 1 HighWire-hosted article(s) which you can access for free at: http://jslhr.asha.org/cgi/content/abstract/14/1/5#otherarticles This information is current as of November 29, 2012 This article, along with updated information and services, is located on the World Wide Web at: http://jslhr.asha.org/cgi/content/abstract/14/1/5