Speech perception

with cochlear implants

and L2 acquisition

Laurianne Cabrera
UCL - 2016
 I. Introduction:
1. The peripheral auditory system

⇒ External auditory signal coded into

action potentials via ~20,000 hair cells

Basilar Membrane Tonotopy = spectral information

 I. Introduction:
2. Coding of speech
AMk(t)
Modulations = temporal information
-4
x 10
6

Envelope (Env
6
x 10
-4
Sk(t) 4

3
or Amplitude
FMk(t) modulations)
4 2

2 1

0
0
0 200 400 600 800 1000 1200

-2
0.1

-4

-6
0.08

0.06 Temporal
fine structure
0.04
0 200 400 600 800 1000 1200

0.02

(TFS or Frequency
-0.02

-0.04

-0.06

modulation)
-0.08

-0.1
0 200 400 600 800 1000 1200

From the signal to the cells:

Motion amplitude = Env
The Vocal system as a carrier circuit, Motion speed = TFS
and Speech as a mixed ENV-TFS signal REST EXCITATION

⇒ The Auditory System as a

demodulation system

Inner Cell
 I. Introduction:
2. Coding of speech

To sum-up:

Env

TFS ⇒ Speech = series of narrowband

signals from high to low frequency
Env composed of relatively slow
temporal envelope and a faster
TFS
temporal fine structure (which
oscillate at a rate close to the centre
Env frequency)

TFS

Waveforms at the outputs of simulated normal

auditory filters centered at 369, 1,499, and 4,803 Hz
in response to the sound “en” in “sense”.
See Moore, B., 2002
 I. Introduction:
3. Vocoders
Dudley ( 1939) = voice-coder
Aim = extract and manipulate spectral and temporal information of an original speech signal
Lowpass filtering
Narrowband
signal E(t)
Demodulation Temporal
80 1 Algorithm Env
1

Cf = 16 Hz
or half Output
bandwidth . signal
.
Original signal .
.
est .
.
. y(t)
.

x(t) Noise or
tone carrier
or
TFS
N N
8020 Hz

N = 4, 8, 16 or 32 bands
Analysis filters
Frequency resolution
 I. Introduction:
3. Vocoders
Dudley ( 1939) = voice-coder
Aim = extract and manipulate spectral and temporal information of an original speech signal
Lowpass filtering
Narrowband
signal E(t)
Demodulation Temporal
80 1 Algorithm Env
1

Cf = 16 Hz
Temporal or half Output
resolution bandwidth . signal
.
Original signal .
.
.
.
. y(t)
.

x(t) Noise or
tone carrier
or
TFS
N N
8020 Hz

N = 4, 8, 16 or 32 bands
Analysis filters
Frequency resolution
 I. Introduction:
3. Vocoders
Dudley ( 1939) = voice-coder
Aim = extract and manipulate spectral and temporal information of an original speech signal
Lowpass filtering
Narrowband
signal E(t)
Demodulation Temporal
80 1 Algorithm Env
1

Env degraded Cf = 16 Hz
or half Output
bandwidth . signal
. Env preserved
Original signal .
.
.
.
. y(t)
. TFS degraded
x(t) Noise or
tone carrier
or
TFS preserved TFS
N N
8020 Hz

N = 4, 8, 16 or 32 bands
Analysis filters
 I. Introduction:
3. Vocoders
Dudley ( 1939) = voice-coder
Aim = extract and manipulate spectral and temporal information of an original speech signal
Lowpass filtering
Narrowband
signal E(t)
Demodulation Temporal
80 1 Algorithm Env
1

Env degraded Cf = 16 Hz
or half
bandwidth . Vocoded signal
. Env preserved
Original signal .
.
.
.
. y(t)
. TFS degraded
x(t) Noise or
tone carrier
or
TFS preserved TFS
N N
8020 Hz

N = 4, 8, 16 or 32 bands
Analysis filters
 I. Introduction:
4. Cochlear Implants

http://www.hear2learn.org/CICSsim/index.htm l

⇒ CI processor = vocoder with:

- a limited number of electrodes/ frequency

bands (Frisen et al. 2000 ~ 8 active electrodes)

- only Env coded and applied to a click train (carrier)

 II. Adult’ speech perception with CIs
1. Consonants and vowels
a. Simulations with Normal Hearing (NH)

- In quiet: Shannon et al., 1995

1 to 4 frequency bands
Vocoder = TFS replaced by noise in each band
ENV filtered at >50 Hz or 16Hz

Consonants
>50 HZ
16 HZ

⇒ Near perfect Consonant/Vowel identification

when only Env cues extracted within ~ 4 bands
 II. Adult’ speech perception with CIs
1. Consonants and vowels
a. Simulations with Normal Hearing (NH)

- In noise: Zeng et al., 2005

Vocoder = 1 to 32 frequency bands
TFS preserved or replaced by noise in each band

Env+TFS Env+TFS

Env
Env

⇒ Benefit of TFS in noisy conditions

 II. Adult’ speech perception with CIs
1. Consonants and vowels
b. CI users’ performance: Zeng et al., 2005

quiet noise

Env+TFS
AM+FM Env+TFS
AM+FM
AM
Env AM
Env

⇒ High variability in performance in quiet, ~ NH listening to 8-band vocoder

Highly difficult in noise, ~ NH listening to 4-band vocoder
 II. Adult’ speech perception with CIs
2. Lexical Tones
a. Simulations:
- Role of TFS: Zeng et al., 2005 - Role of Fast Env: Fu, Zeng et al., 1998
1 to 32 frequency bands 1 to 4 frequency bands
Vocoder = Vocoder = TFS replaced by noise
TFS preserved or replaced
Env 500 Hz or 50 Hz

Mandarin 500 HZ
Mandarin 50 HZ
Percent Correct

Percent Correct
Word

Word
Env+TFS
Env

Number of bands

⇒Better performance when ⇒ Better performance when

TFS preserved (~F0-related) fast Env preserved (~ F0-related)

(see also Xu & Pfingst, 2007)

 II. Adult’ speech perception with CIs
2. Lexical Tones
b. CI users’ performance: Zeng et al., 2005

Percent Correct
Word

Env+TFS
Env

⇒ Poor lexical tone identification performance for CI users

~ NH listening to 1-band vocoder

Modulations related to F0 fluctuations (TFS and fast Env)

required for lexical tone perception
 II. Adult’ speech perception with CIs
3. Interaction with linguistic experience

Cabrera et al., 2014

– 8-bands Env
Vocoders= – click train mimicking F0 variations

8-band Env Pitch only

Intact
Discrimination score

Rising-Low Rising-Falling Falling-Low Rising-Low Rising-Falling Falling-Low Rising-Low Rising-Falling Falling-Low

⇒ Mandarin speakers are more impaired by reduction of TFS than French

⇒ Mandarin rely more on F0 information than French when discriminating
lexical-tones
⇒ Effect of linguistic experience on the use of acoustic cues
 III. Development of language using CIs
1. L1 acquisition
a. Non-tonal languages
- Prelingually vs Postlingually
Gantz et al., 1994
Phonetically Balanced Kindergarten Word Lists (PK-B- open set)
5 post-lingually hearing-impaired (HI) children (>5 years)
Prelingually HI
54 pre-lingually HI children (< 2 years) Post-lingually HI
Phoneme recognition (%)

Word recognition (%)

⇒ Prelingually HI children show lower speech perception

performance than postlingually HI children
⇒ Improvement with CI use
 III. Development of language using CIs
1. L1 acquisition
a. Non-tonal languages
Age at implantation
- Age at implantation - Long use of CIs
2 year-4 years 11 months
5 years-7years 11 months
Gantz et al., 1994 > 8years Eisenberg et al., 2004

2 years post implantation 3 years post implantation 7-year-olds, CIs at 2 years

~ 5 years post implantation
Percent correct

Percent correct
words
PK-B phonemes

N= N=

⇒ Early implantation more beneficial for speech perception capabilities

⇒ High variability but good performance with longer CI use
 III. Development of language using CIs
1. L1 acquisition
b. Tonal languages
- Discrimination vs identification - Factors influencing tone perception
Wong & Wong, 2004 Zhou et al., 2013
Age = 6 yr
Age at implantation: 2.67-6.4 yr Age range: 2.42-16.18 yr
Use of CI ~ 1-3 yr Age at implantation: 1.11-12.95 yr
discrimination Use of CI: 1.11-12.95 yr
identification

Tone perception
(% correct)
⇒ Lexical tone discrimination easier than
identification (like phoneme vs words)
⇒ High variability but increase in
performance with CI use Cf. Chapter by Xu & Zhou, Tonal Languages and Cochlear Implants
 III. Development of language using CIs
1. L1 acquisition
b. Tonal languages
- Consonants vs Tones
Cabrera et al, (ongoing)
N =16, Tested at 4.14-7 yr of age
Age at implantation: 1.44-3.38 yr
CI use: 1.16-5 yr

2.00

1.80

1.60

1.40
⇒ Lexical tone discrimination easier
than consonants for children using
discrimiation scrores

1.20

1.00 CIs and learning a tonal language

0.80
⇒ Place is more difficult (cf. Shannon
0.60
et al., 1995)
0.40

0.20

0.00
Manner Place Tone 1vs2 Tone 1vs3
 III. Development of language using CIs
2. L2 acquisition
- Is the presence of L2 detrimental for L1 in CI users?
McConkey Robbins et al., 2004

Age at implantation= 6mo-2yr 11mo

CI use = 10m-12yr
L1 perception L2 perception
5

Student Oral Language Obsevation matrix

4 N=1

NH’ scores
3 N=1

2 N=5

0
Year 1 Year 2

1: preproduction/early production
2: speech emergence
3: intermediate fluency
4: advanced fluency

⇒ Learning a 2nd language is not negative for the L1 proficiency

 III. Development of language using CIs
2. L2 acquisition
- Comparison between Monolingual and Bilingual CI users
- Teschendorf et al., 2011 - Thomas et al., 2008
Age at implantation ~ 3 yr Age at implantation < 6yr
CI use = 10m-12yr
Not age or SES matched Matched for age implantation,
cochlear anatomy, educational
setting, device type

!!!

CI use

⇒ When matched carefully, bilingual kids do not show lower language

proficiency than monolinguals when using CIs
 IV. Conclusions

What are the next challenges?