Brain Research 1664 (2017) 95–101

Contents lists available at ScienceDirect

Brain Research
journal homepage: www.elsevier.com/locate/bres

Research report

Visual stimuli approaching toward the body influence temporal

expectations about subsequent somatosensory stimuli
Tsukasa Kimura a,⇑, Jun’ichi Katayama a,b
a
Department of Psychological Science, Kwansei Gakuin University, Nishinomiya 662-8501, Japan
b
Center for Applied Psychological Science (CAPS), Kwansei Gakuin University, Nishinomiya 662-8501, Japan

a r t i c l e i n f o a b s t r a c t

Article history: The present study investigated whether visual stimuli approaching the body influence temporal expec-
Received 31 July 2016 tations about subsequent somatosensory stimuli. To examine this question, we recorded event-related
Received in revised form 13 March 2017 brain potentials (ERPs) during a simple reaction time task using somatosensory stimuli. Fourteen partic-
Accepted 29 March 2017
ipants were asked to place their arms on a desk, and three light-emitting diodes (LEDs) were placed at
Available online 4 April 2017
equal distances between their arms. Each trial was composed of three visual stimuli (i.e., LEDs), and
one subsequent electrical stimulus (i.e., somatosensory stimulus) to one wrist. The stimulus onset asyn-
Keywords:
chrony (SOA) between the visual stimuli was set to 1000 ms. The SOA between the third visual stimulus
Temporal expectation
Visuotactile processing
and the somatosensory stimulus was set to 1000 ms (standard; p = 0.75), 500 ms (early deviation;
Multimodality interaction p = 0.125), and 1500 ms (late deviation; p = 0.125). In the approach condition, the left, center, and right
ERP LEDs (or reverse) were turned sequentially toward the wrist to which the somatosensory stimulus was
presented. In the neutral condition, the center LED was flashed three times. The N1 amplitudes for early
deviations of stimuli were larger under the approach condition than under the neutral condition. These
results show that prior visual stimuli facilitate temporal expectations about subsequent somatosensory
stimuli, i.e., visual stimuli approaching toward the body facilitate the processing of early deviant stimuli.
The present study indicates the existence of a function of supramodal temporal expectation and detection
of deviation from this expectation using the approach of visual stimuli toward the body.
Ó 2017 Elsevier B.V. All rights reserved.

1. Introduction whether the prior approach of visual stimuli influences temporal

Although a main function of somatosensory modality is to iden-
tify ‘‘when”, ‘‘where” and ‘‘what” somatosensory events occur
(Gibson, 1962), expectation of the event is difficult, because
somatosensory sensation is evoked after physical contact occurred.
It is necessary for us to expect physical contact before dangerous
object (e.g., knife) touches our body to defend ourselves. The
expectation for somatosensory events is enabled by using other
sensory information. In particular, the number of studies reported
that visual information influences the processing of following
somatosensory sensation, e.g., facilitation for somatosensory target
detection in visually attended space (e.g., Spence et al., 1998; for a
review, see Spence, 2010). Moreover, a recent study reported that
the task irrelevant approach of visual stimuli toward the body
facilitates spatial expectations for subsequent somatosensory
events (Kimura and Katayama, 2015). However, it remains unclear
⇑ Corresponding author.
E-mail addresses: ts.kimura2011@kwansei.ac.jp (T. Kimura), jkatayama@
kwansei.ac.jp (J. Katayama).
expectations about subsequent somatosensory events.
In order to generate temporal expectation, i.e., when subse-
quent events will occur, people extract information about
frequency and temporal regularity of occurrence of, as well as
the context from, prior events. For example, it is known that the
response to a subsequent event is facilitated by regularity of prior
events. Specifically, judgment about when visual stimuli will arrive
at a goal becomes faster when the prior visual stimuli move at reg-
ular intervals (e.g., Doherty et al., 2005; Rohenkohl et al., 2011).
These studies suggest that temporal expectations about targets
are influenced by regularity of the stimulus onset asynchrony
(SOA) of prior visual information (e.g., for a review, see Nobre
et al., 2007).
This prior visual information influences temporal expectations
about subsequent somatosensory events. For example, the tempo-
ral order judgment (TOJ) for somatosensory stimuli is influenced
by prior visual information (e.g., Fujisaki and Nishida, 2009;
Spence et al., 2001); thereby, the existence of temporal interactions
between visual and somatosensory modalities assumed. Further-
more, TOJ for a somatosensory event is modulated by visual stimuli

http://dx.doi.org/10.1016/j.brainres.2017.03.030
0006-8993/Ó 2017 Elsevier B.V. All rights reserved.
96 T. Kimura, J. Katayama / Brain Research 1664 (2017) 95–101
presented near the hands and the same location as the somatosen-
sory event (e.g., Spence et al., 2003). Thus, it is possible that the
distance between visual stimuli and the body affects temporal
expectations about subsequent somatosensory events. In everyday
life, people can predict when an object will contact the body by
evaluating the speed and trajectory of the moving object (i.e.,
visual information), and we can choose adaptive behavior (e.g.,
avoid or catch) in response to expected events. Thus, it is possible
that visual stimuli approaching the body also influence temporal
expectations for subsequent somatosensory events.
In psychophysiological studies, electroencephalograms (EEGs)
and event-related brain potentials (ERPs) are often used as an
index for temporal expectations. Specifically, stimuli that deviate
from temporal contexts modulate ERPs not only voluntary compo-
nent (P3b and RON; e.g., Schwartze et al., 2011; for a review, see
Correa et al., 2006) but also involuntary (N1; e.g., Ford and
Hillyard, 1981) and preattentive components (MMN and P3a;
e.g., Alain et al., 1998; Takegata et al., 2001). Hence, we examined
whether visual stimuli approaching toward the body influence
temporal expectations about subsequent somatosensory stimuli,
using ERP.
As described above, although the expectation of somatosensory
event is necessary to defend our body from dangerous event, this
expectation is difficult using only somatosensory information and
visual information supports to create the expectation. Kimura
and Katayama (2015) shows that approaching visual stimuli
toward the body facilitates spatial expectations for subsequent
somatosensory events. To examine this effect on temporal expec-
tations, we recorded EEG while participants were performing a
simple reaction to somatosensory stimuli, and then analyzed
ERP components. Each trial was composed of three visual stimuli,
and then one somatosensory stimulus to only one wrist (left or
right) in a block. The SOA between visual stimuli was set to
1000 ms. The SOA between the third visual stimulus and the
somatosensory stimulus was set to either the same amount of
time (1000 ms; p = 0.75), shorter (500 ms; p = 0.125), or longer
(1500 ms; p = 0.125). In the approach condition, three visual stim-
uli were presented sequentially, moving toward the wrist where a
somatosensory stimulus was presented. In the neutral condition,
visual stimuli located at an equal distance from both wrists were
presented three times, the same number as under the approach
condition. Under the both conditions, somatosensory stimuli
invariably occurred after the third visual stimulus. The approach-
ing visual stimuli were irrelevant information in this simple reac-
tion time task because participants were told what the location of
the somatosensory stimulus would be (i.e., left or right wrist)
before each block.
We predicted that participants would expect the presentation
of somatosensory stimuli 1000 ms after the third visual stimulus
because this timing was the most frequent. Furthermore, we con-
sider that shorter and longer SOA stimuli modulate different ERPs.
Previous studies reported that stimulus repetition with the same
SOA produced habituation leading the decrement of N1 amplitude
for the stimulus (e.g., Budd et al., 1998). In the present study, prior
visual stimuli are presented repeatedly with the same SOA (i.e.,
1000 ms), thus shorter and longer SOA stimuli would elicit larger
N1 amplitude at the between frontal to central region because
these stimuli are deviant from temporal expectation. In particular,
shorter SOA stimuli would elicit a larger N1 amplitude response to
an early deviation from temporal expectations (e.g., Ford and
Hillyard, 1981) than other SOA stimuli. Moreover, another previous
study showed that shorter SOA stimuli elicit P3b and longer SOA
stimuli elicit P3a component at the between central and parietal
regions (Jongsma et al., 2007). They interpreted their results as that
the modulation of N1 and P3b amplitude to early deviant
stimuli reflected deviation from temporal expectation, whereas
modulation of P3a amplitude to later deviant stimuli reflected
stimulus omission from prior stimulus context. In the present
study, we examined which ERPs are affected by the temporal devi-
ation. In addition, these ERP amplitudes in the approach condition
would be larger than in the neutral condition, if visual stimuli
approaching toward the body facilitate temporal expectations
about subsequent somatosensory stimuli. The facilitation of tem-
poral expectations about subsequent somatosensory stimuli by
the approach of visual stimuli is demonstrated if these results
are obtained.
Finally, the previous study reported that the contingent nega-
tive variation (CNV) is elicited between the third visual stimulus
and the somatosensory stimulus at the between frontal to central
region (Kimura and Katayama, 2015). The modulation of the CNV
caused by the early and late deviation would be critical for evalu-
ation of the temporal expectation created by the preceding
context.
2. Results
2.1. Behavioral data
Trials with an incorrect response or with reaction time (RT)
shorter than 200 ms or longer than 1500 ms were discarded from
RT analysis. Table 1 shows the mean RTs of all participants. An
two-way repeated measures analysis of variance (ANOVA) of the
RTs revealed a main effect of stimulus timing (F(2, 26) = 45.31,
p < 0.001, e = 0.71, g2p = 0.78). Post hoc comparisons revealed that
the RTs for early deviant and late deviant stimuli were longer than
that for the standard stimuli (ps < 0.05). The main effect of
condition type was not significant (p > 0.10) and the interaction
between condition type and stimulus timing was weak trend
(F(2, 26) = 2.71, p = 0.10, e = 0.82, g2p = 0.17).
2.2. Electrophysiological data
2.2.1. N1
Fig. 1 illustrates the grand average of ERPs elicited by the
somatosensory stimuli during the approach and neutral condi-
tions, recorded from the FCz, Cz, and Pz. Fig. 2 shows (a) the topo-
graphic maps at the time range of N1 (100–180 ms), and (b) the
mean amplitude of N1 for both conditions. The early deviant stim-
uli of the approach condition elicited a larger N1 amplitude than
other conditions and other timing at FCz; N1 elicited maximum
amplitude at this electrode.
The two-way interaction of condition type and stimulus timing
was significant (F(2, 26) = 6.22, p = 0.009, e = 0.89, g2p = 0.32). Post
hoc comparison indicated that the early deviant stimuli elicited
larger N1 amplitude than the standard stimuli in the approach con-
dition (p < 0.05). Additionally, the early deviant stimuli under the
approach condition stimuli elicited larger N1 amplitudes than the
early deviant stimuli under the neutral condition (p < 0.05). In con-
trast, N1 amplitudes did not differ between stimulus timings in the
neutral condition (ps > 0.10). Moreover, in the both conditions, N1
amplitudes were not significant different for standard stimuli and
late deviant stimuli (ps > 0.10). These results revealed that the N1
Table 1
Mean RTs (ms) for somatosensory stimuli and standard errors of RTs in each
condition.
Standard (ms) Early deviant (ms) Late deviant (ms)
Approach 363 ± 18.52 452 ± 17.06 446 ± 19.41
Neutral 367 ± 20.26 455 ± 20.31 439 ± 19.60
 T. Kimura, J. Katayama / Brain Research 1664 (2017) 95–101 97

Fig. 1. Grand averages of ERP waveforms for each condition at FCz, Cz, and Pz electrode sites (N = 14). The light gray area denotes the time range of N1 (100–180 ms), and the
dark gray area denotes the time range of P3 (Standard and Late deviant: 220–360 ms; Early deviant: 260–400 ms).

Fig. 2. (a) The topographic map of the N1 time range (100–180 ms); and (b) mean N1 amplitude (N = 14). The error bars indicate the standard errors (SEs) of the means across
participants.

amplitude elicited by the early deviant timing was larger for the
approach condition than for the neutral condition. Furthermore,
all main effects were significant (condition: F (1, 13) = 6.53,
p = 0.02, g2p = 0.33; stimulus timings: F (2, 26) = 3.98, p < 0.05,
e = 0.76 g2p = 0.23).
2.2.2. P3
Fig. 3 illustrates (a) the topographic maps at the time range of
P3 (standard and late deviant: 220–360 ms; early deviant: 260–
400 ms), and (b) the mean amplitude of P3 for both conditions at
Pz; P3 elicited maximum amplitude at this electrode. The main
98 T. Kimura, J. Katayama / Brain Research 1664 (2017) 95–101

Fig. 3. (a) The topographic map of the P3 time range (Standard and Late deviant: 220–360 ms; Early deviant: 260–400 ms); and (b) mean P3 amplitude (N = 14). The error
bars indicate the standard errors (SEs) of the means across participants.

effect of stimulus timing (F(2, 26) = 8.83, p = 0.002, e = 0.90,
g2p = 0.40) was significant. Post hoc comparison indicated that
the P3 amplitudes were larger in standard and late deviant timing
than in early deviant timing (ps < 0.05). None of the interaction
(F (2, 26) = 0.70, p > 0.10) and the main effect of condition
(F (1, 13) = 2. 15, p > 0.10) were significant.
revealed that the all main effects and interactions were not
significant (condition: F (1, 13) = 2.15, p > 0.10; stimulus timings:
F (1, 13) = 0.66, p > 0.10; interaction of condition and stimulus
timings: F (1, 13) = 1.44, p > 0.10).
3. Discussion

The present study investigated whether the visual stimulus

2.2.3. CNV
Fig. 4 displays the grand average contingent negative variation
(CNV) elicited in both conditions recorded at Cz; CNV elicited max-
imum amplitude at this electrode. The gray area indicates the time
range of CNV amplitude evaluation (500–1000 ms) for standard
and late deviant timing. An ANOVA on the mean CNV amplitude
approaching toward the body influences temporal expectations
about subsequent somatosensory stimuli. For this purpose, we
recorded EEGs during simple reaction time tasks using somatosen-
sory stimuli, and then compared ERPs elicited by somatosensory
stimuli between a visual approach condition and a non-approach
condition.

Fig. 4. Grand averages of ERP waveforms for both conditions at Cz (N = 14). The 0 ms is third visual stimulus onset. The gray area indicates the time rage for CNV
(500–1000 ms).
 T. Kimura, J. Katayama / Brain Research 1664 (2017) 95–101
Our results show that RTs to low-probability stimuli were
longer than the RTs to high-probability stimuli. This result suggests
that the participants created temporal expectations about the tim-
ing of the subsequent somatosensory stimuli. Moreover, standard
and late deviant timings elicited similar CNV, which supports this
suggestion because CNV is related to a temporal expectation (e.g.,
Walter et al., 1964). No difference in CNVs between them would be
based on the fact that at the 1000 ms point, participant could not
discriminate standard and late deviant. In contrast, stimuli of early
deviant timing did not elicit the CNV because this stimulus was
presented before preparation for the temporal expectation began.
Furthermore, results of N1 indicate the influence of approaching
visual stimuli toward the body regarding this temporal expecta-
tion. Early deviation from the temporal regularity of approaching
visual stimuli enhanced the N1 amplitude of subsequent
somatosensory stimuli. It has been shown that the N1 amplitude
was enhanced by early deviation from the temporal regularity of
prior stimuli (Ford and Hillyard, 1981). In addition, the early stim-
ulus processing is relevant to rapid detection of events (Schwartze
et al., 2011). Thus, early deviation from the temporal expectations
created by temporal regularity was processed as stimuli that it
should be detected early. In addition, the emergence of stimuli
approaching the body and early stimuli deviating from temporal
expectation generate a need to process instantly. Therefore, we
consider that the approach of visual stimuli toward the body
facilitate processing of deviation to early deviant stimuli.
However, the P3 amplitude for stimuli of early deviant timing
was the smallest, and the P3 amplitudes for stimuli of standard
and late deviant timing were not different across the stimulus tim-
ings and the conditions. CNV was not elicited in the early deviant
timing, indicating that somatosensory stimuli were presented
before preparation of temporal expectations began in early devia-
tion timing. Additionally, stimuli of late deviation are predictable
in terms of presentation timing. When the third visual stimuli were
presented, the probabilities of subsequent somatosensory stimuli
were 75% for standard timing, and 12.5% for both deviant timings.
However, when the standard timing (SOA: 1000 ms) has passed,
the only remaining possibilities are a late deviant stimulus or no
stimulus (catch trial). In this case, participants only prepare for
whether stimuli present (late deviant trial) or not (catch trial). P3
is elicited by low frequency of occurrence of the stimulus, and
the lower frequency enhances these amplitudes (Duncan-Johnson
and Donchin, 1977). Thus, it is assumed that stimuli of late deviant
timing did not enhance the P3 amplitude.
Finally, it is possible that temporal expectations about subse-
quent somatosensory events elicited by visual stimuli approaching
toward the body occur automatically at an early stage. In the
present study, participants were instructed to perform a simple
reaction time task in response to somatosensory stimuli.
Somatosensory stimuli were presented to only one wrist (i.e., left
or right wrist), which was announced to participants before each
block. Thus, visual stimuli functioned as temporal clues for subse-
quent somatosensory stimuli; however, the element of approach-
ing was irrelevant information for the present task. However, this
information facilitated temporal expectations about somatosen-
sory stimuli of the approached location, and N1 amplitudes related
to early deviation from this temporal expectation were enhanced.
This result indicates that the approach of visual stimuli toward
the body facilitates processing of subsequent somatosensory
events automatically, and facilitates instant detection of early devi-
ation of somatosensory stimuli. A recent study indicated that the
approach of visual stimuli modulates spatial expectations about
subsequent somatosensory events (e.g., Kimura and Katayama,
2015). This study reported that the task-irrelevant approach of
visual stimuli toward the hand produced a greater spatial expecta-
tion than non-approach stimuli, and stimuli that deviate from this
99
expectation elicit a larger P3 amplitude than any other stimuli. In
other words, spatial expectation of a subsequent somatosensory
event elicited by visual stimuli approaching toward the body
influences late processing of the subsequent event. In contrast,
temporal expectation elicited by approach of visual stimuli toward
the body influences early processing of subsequent event, accord-
ing to the present results. These results suggest that the processing
affected in temporal prediction elicited by visual stimuli approach
toward the body is different from the processing involved in spatial
prediction.
3.1. Conclusion
In summary, the present study showed that the approach of
visual stimuli enhances the N1 amplitude of responses to subse-
quent somatosensory stimuli that deviate from temporal expecta-
tion by occurring early. In contrast, P3 and CNV amplitudes did not
change in response to deviation of stimulus timing and visual stim-
uli approaching toward the body. In other words, visual stimuli
approaching the body influence temporal expectations about sub-
sequent somatosensory stimuli and modulate early deviation of
stimuli from this expectation. This result indicates the existence
of a function of temporal expectation for subsequent somatosen-
sory stimuli using prior visual information and a function of
detection of stimuli that appear earlier than expected using the
approach of visual information toward the body.
4. Experimental procedure
4.1. Participants
Fourteen undergraduate and graduate students (9 females, 5
males; 21–24 years of age) participated in this experiment. Three
participants were left-handed and others were right-handed,
according to their self-report. All participants had normal or
corrected-to-normal vision. This experiment was approved by the
Kwansei Gakuin University (KGU) Institutional Review Board
under the KGU Regulations for Behavioral Research with Human
Participants. Written informed consent was obtained from all
participants.
4.2. Stimuli and procedures
Somatosensory stimuli were generated by an electrical stimulus
generator (Nihon Koden Corporation, SEN-7203) and were pre-
sented to the participants’ wrists via electric isolators (Nihon
Koden Corporation, SS-203 J) and Ag/AgCl electrodes with a diam-
eter of 1.0 cm. The anodes were placed on the participants’ wrists,
and the cathodes were placed 3.0 cm from the anodes toward the
elbow. The somatosensory stimuli were single block pulses of
0.2 ms in duration. The intensities were three times as high as
the threshold for each participant (never causing pain). The abso-
lute threshold was measured by six iterations of up-and-down
method per participant. The average intensity of the stimuli across
all participants was 3.5 mA. These stimuli were presented to only
one wrist (right or left) in each block. The order of the location (left
or right) of stimulus presentation was counterbalanced across
blocks.
Three white light-emitting diodes (LEDs) were used as visual
stimuli. Each LED was a square with 0.8 cm sides. Three LEDs were
placed at equal distances (8.0 cm each intervals) between the arms.
The visual stimuli were single block pulses of 200 ms in duration.
The light intensity was 25 cd.
Each trial was composed of three visual stimuli and one
somatosensory stimulus. The visual stimuli SOA were set to
100 T. Kimura, J. Katayama / Brain Research 1664 (2017) 95–101

1000 ms. The SOA between the third visual stimulus and the
somatosensory stimulus was set; (1) standard timing was set to
1000 ms (the same as the visual stimulus interval), (2) early devi-
ant timing was set to 500 ms, and (3) late deviant timing was set to
1500 ms. In each block, stimuli of standard timing were presented
with high probability (75%), and stimuli of early deviant and late
deviant timing were presented with low probability (12.5% for
each). Fig. 5 illustrates these stimulus timings. The interval
between presentations of somatosensory stimulus and first visual
stimulus was either 1000 or 1200 ms in random with equal
probability.
Each block was composed of 134 trials (including 6 catch trials),
which took approximately 10 min. A total of two blocks were pre-
sented for each condition. The interval between blocks was 2 min,
and after the second block, the participants rested for 10 min and
then started the remaining two blocks. The order in which the con-
ditions were performed was randomized across participants.
The approach condition and neutral condition were distin-
guished by the presentation patterns of the visual stimuli. In the
approach condition, the right, center, and left (or left, center, and
right) visual stimuli were sequentially presented, moving toward
the wrist to which the somatosensory stimulus was presented. In
the neutral condition, the center LED was stimulated three times
using the same timing, and then the somatosensory stimulus was
presented to the wrist.
The participants sat in front of a desk and gazed at the center
LED. They were asked to place their arms on the desk at a
32.0 cm interval and to avoid moving their bodies more than nec-
essary during the task. Fig. 6 shows the positioning of the visual
and somatosensory stimuli. The LEDs were placed between the

Fig. 5. The stimulus timings of Standard, Early deviant, and Late deviant. The gray line indicates the timing of visual stimuli, and the black lines indicate the timing of
somatosensory stimuli.

Fig. 6. The positions of stimulus presentations. The gray circles indicate the positions of somatosensory stimuli, and the white squares indicate the positions of visual stimuli.
 T. Kimura, J. Katayama / Brain Research 1664 (2017) 95–101
arms at equal intervals (8.0 cm each). Similarly, the distance
between the electrode for the somatosensory stimuli on the left
(right) arm and left (right) LED was 8.0 cm. Participants were
instructed to respond by pressing a button as quickly as possible
with the left (or right) foot whenever the somatosensory stimuli
were presented and to not respond when somatosensory stimuli
were not presented (i.e., the catch trials). Half of the participants,
selected at random, responded using the congruent foot on the side
of somatosensory stimuli location as the response foot, and the
other half responded using the incongruent foot.
4.3. Recording and analyses
EEG activity was recorded by Nuamps (Compumedics Neu-
roscan, USA) and an electrode cap (Easycap GmbH, Germany),
using Ag/AgCl electrodes at 32 sites (Fp1, Fp2, F7, F3, Fz, F4, F8,
FT7, FC3, FCz, FC4, FT8, T7, C3, Cz, C4, T8, TP7, CP3, CPz, CP4, TP8,
P7, P3, Pz, P4, P8, O1, Oz, O2, A1, and A2), according to the modified
10–20 System. All recordings were referenced to the nose tip, and
the ground electrode site was AFz. The electrode impedances were
maintained below 5 kX. The EEG signals were amplified using a
bandpass filter of 0.1–200 Hz and were digitized at 1000 Hz.
In the offline analysis, the data were analyzed with the EEGLAB
toolbox (Delorme and Makeig, 2004) in MATLAB (MathWorks Inc).
The data were digitally low–pass filtered at 30 Hz (6 dB/octave)
using an IIR Butterworth analog simulation filter. Artifacts derived
from eye movements and eye blinks were rejected using an auto-
matic EEG artifact detector based on the joint use of the spatial
and temporal features (ADJUST) of the EEGLAB toolbox (Mognon
et al., 2011).
ERP epochs for N1 and P3 were extracted within a time range of
900 ms, including 100 ms before the presentation of somatosen-
sory stimuli. Trials with errors or those in which the EEG signal
variation exceeded ±100 lV were automatically discarded. After
artifact rejection, the numbers of remaining trials ranged from
181 to 196 for standard, 26 to 32 for early deviant, and 28 to 32
for late deviant. The appropriate latency windows of N1
(100–180 ms) and P3 (220–360 ms for standard and late deviant;
260–400 ms for early deviant) were defined based on observation
of the resultant ERP waveforms.
In addition, epochs for CNV were extracted within a time win-
dow of 1900 ms, including 200 ms before the presentation of the
third visual stimulus. Trials with errors or those in which the
EEG signal variation exceeded ±100 lV were automatically dis-
carded. After artifact rejection, the numbers of remaining trials
were 152–196 for standard and 23–32 for late deviant. According
to the results of observations of ERP waveforms, early deviant
timing did not elicit CNV; therefore, CNV was analyzed only for
standard and late deviant timing. The mean CNV amplitude was
obtained from a latency window of 500–1000 ms for standard
and late deviant timing. The appropriate latency windows were
defined based on observation of the resultant ERP waveforms.
Two-way repeated measures ANOVA were conducted on the RT
to the onset of electrical stimuli, in which the two conditions
(approach and neutral) and the three stimulus timings (standard,
early deviant and late deviant) was considered the variables of
interest. Moreover, two-way repeated measures ANOVAs were
conducted on the mean amplitude of N1 at FCz and P3 at Pz
(2 conditions  3 stimulus timings). These electrodes showed the
maximum amplitude for each component. Finally, the CNV mean
amplitudes at Cz, where the CNV was elicited maximum ampli-
tude, were assessed with two-way repeated measures ANOVA
(2 conditions  2 stimulus timings (standard and late deviant)).
These ANOVAs were conducted by applying Greenhouse–Geisser
corrections to the degrees of freedom when appropriate
101
(Greenhouse and Geisser, 1959). The effect sizes have been indi-
cated in terms of partial eta squared (g2p ). Post hoc comparisons
were made using Shaffer’s modified sequentially rejective multiple
test procedure, which extends Bonferroni t tests in a stepwise fash-
ion (Shaffer, 1986). The significance level was set at p < 0.05.
Acknowledgement
Part of this study was supported by a Grant-in-Aid for Scientific
Research (B) (25285206) from the Japan Society for the Promotion
of Science (JSPS) and Special Research Fund A from Kwansei Gakuin
University, awarded to Jun’ichi Katayama. The experiment was
conducted as a part of the project supported by the Ministry of
Education, Culture, Sports, Science, and Technology (MEXT), JAPAN,
for the Strategic Research Foundation at Private Universities
(2015-2019; Project number S1511032) to the Center for Applied
Psychological Science (CAPS), Kwansei Gakuin University.
References
Alain, C., Cortese, F., Picton, W.T., 1998. Event-related brain activity associated with
auditory pattern processing. NeuroReport 9, 3537–3541.
Budd, T.W., Barry, R.J., Gordon, E., Rennie, C., Michie, P.T., 1998. Decrement of the N1
auditory event-related potential with stimulus repetition: habituation vs.
refractoriness. Int. J. Psychophysiol. 31, 51–68.
Correa, A., Lupiáñez, J., Madrid, E., Tudela, P., 2006. Temporal attention enhances
early visual processing: a review and new evidence from event-related
potentials. Brain Res. 1076, 116–128.
Delorme, A., Makeig, S., 2004. EEGLAB: an open source toolbox for analysis of single
trial EEG dynamics. J. Neurosci. Methods 134, 9–21.
Doherty, J.R., Rao, A., Mesulam, M.M., Nobre, C.A., 2005. Synergistic effect of
combined temporal and spatial expectations on visual attention. J. Neurosci. 25,
8259–8266.
Duncan-Johnson, C.C., Donchin, E., 1977. On quantifying surprise: the variation of
event-related potentials with subjective probability. Psychophysiology 14, 456–
467.
Ford, M.J., Hillyard, A.S., 1981. Event-related potentials (ERPs) to interruptions of a
steady rhythm. Psychophysiology 18, 322–330.
Fujisaki, W., Nishida, S., 2009. Audio–tactile superiority over visuo–tactile and
audio–visual combinations in the temporal resolution of synchrony perception.
Exp. Brain Res. 198, 245–259.
Gibson, J., 1962. Observation on active touch. Psychol. Rev. 69, 477–491.
Greenhouse, S.W., Geisser, S., 1959. On methods in the analysis of profile data.
Psychometrika 24, 95–112.
Jongsma, M.L., Meeuwissen, E., Vos, P.G., Maes, R., 2007. Rhythm perception:
speeding up or slowing down affects different subcomponents of the ERP P3
complex. Biol. Psychol. 75, 219–228.
Kimura, T., Katayama, J., 2015. Approach of visual stimuli modulates spatial
expectations for subsequent somatosensory stimuli. Int. J. Psychophysiol. 96,
176–182.
Mognon, A., Jovicich, J., Bruzzone, L., Buiatti, M., 2011. ADJUST: an automatic EEG
artifact detector based on the joint use of spatial and temporal features.
Psychophysiology 48, 229–240.
Nobre, C.A., Correa, A., Coull, T.J., 2007. The hazards of time. Curr. Opin. Neurobiol.
17, 465–470.
Rohenkohl, G., Coull, T.J., Nobre, C.A., 2011. Behavioural dissociation between
exogenous and endogenous temporal orienting of attention. PLoS ONE 6,
e14620.
Schwartze, M., Rothermich, K., Kassow-Schmidt, M., Kotz, A.S., 2011. Temporal
regularity effects on pre-attentive and attentive processing of deviance. Biol.
Psychol. 87, 146–151.
Shaffer, J.P., 1986. Modified sequentially rejective multiple test procedures. J. Am.
Stat. Assoc. 81, 826–831.
Spence, C., 2010. Crossmodal spatial attention. Ann. N. Y. Acad. Sci. 1191, 182–200.
Spence, C., Baddeley, R., Zampini, M., James, R., Shore, I.D., 2003. Multisensory
temporal order judgments: when two locations are better than one. Percept.
Psychophys. 65, 318–328.
Spence, C., Nicholls, M.E.R., Gillespie, N., Driver, J., 1998. Crossmodal links in
exogenous covert spatial orienting between touch, audition, and vision. Percept.
Psychophys. 60, 544–557.
Spence, C., Shore, I.D., Klein, M.R., 2001. Multisensory prior entry. J. Exp. Psychol.
Gen. 130, 799–832.
Takegata, R., Syssoeva, O., Winkler, I., Paavilainen, P., Näätänen, R., 2001. Common
neural mechanism for processing onset-to-onset intervals and silent gaps in
sound sequences. NeuroReport 12, 1783–1787.
Walter, W.G., Cooper, R., Aldridge, V.J., McCallum, W.C., Winter, A.L., 1964.
Contingent Negative Variation: an electric sign of sensorimotor association
and expectancy in the human brain. Nature 203, 380–384.