Neuroscience 312 (2016) 58–73
EVENT-RELATED BRAIN RESPONSES WHILE LISTENING TO ENTIRE
PIECES OF MUSIC
H. POIKONEN, a* V. ALLURI, b E. BRATTICO, a,c
O. LARTILLOT, d M. TERVANIEMI a,e AND
M. HUOTILAINEN a,e,f
a
Cognitive Brain Research Unit, Cognitive Science, Institute
of Behavioural Sciences, University of Helsinki, P.O. Box 9
(Siltavuorenpenger 1 B), FI-00014 University of Helsinki, Finland
b
Department of Music, University of Jyväskylä, P.O. Box 35, 40014
University of Jyväskylä, Finland
c
Center for Music in the Brain (MIB), Department of Clinical
Medicine, Aarhus University, Nørrebrograde 44, DK-8000 Aarhus C,
Denmark
d
Department of Architecture, Design and Media Technology,
University of Aalborg, Rendsburggade 14, DK-9000 Aalborg,
Denmark
e
Cicero Learning, P.O. Box 9 (Siltavuorenpenger 5 A),
FI-00014 University of Helsinki, Finland
f
Finnish Institute of Occupational Health, Haartmaninkatu 1 A,
00250 Helsinki, Finland
Abstract—Brain responses to discrete short sounds have
been studied intensively using the event-related potential
(ERP) method, in which the electroencephalogram (EEG) sig-
nal is divided into epochs time-locked to stimuli of interest.
Here we introduce and apply a novel technique which
enables one to isolate ERPs in human elicited by continuous
music. The ERPs were recorded during listening to a Tango
Nuevo piece, a deep techno track and an acoustic lullaby.
Acoustic features related to timbre, harmony, and dynamics
of the audio signal were computationally extracted from the
musical pieces. Negative deflation occurring around 100 mil-
liseconds after the stimulus onset (N100) and positive defla-
tion occurring around 200 milliseconds after the stimulus
onset (P200) ERP responses to peak changes in the acoustic
features were distinguishable and were often largest for
Tango Nuevo. In addition to large changes in these musical
features, long phases of low values that precede a rapid
increase – and that we will call Preceding Low-Feature
*Corresponding author. Address: Cognitive Brain Research Unit,
Institute of Behavioural Sciences, University of Helsinki, P.O. Box 9
(Siltavuorenpenger 1 B), FI-00014 University of Helsinki, Finland.
Mobile: +358-407348028.
E-mail addresses: hanna.poikonen@helsinki.fi (H. Poikonen), vinoo.
alluri@campus.jyu.fi (V. Alluri), elvira.brattico@helsinki.fi (E. Brattico),
olartillot@gmail.com (O. Lartillot), mari.tervaniemi@helsinki.fi
(M. Tervaniemi), minna.huotilainen@helsinki.fi (M. Huotilainen).
Abbreviations: EEG, electroencephalography; ERP, event-related
potential; FIR, finite impulse response; HG, Heschl’s gyrus; ICA,
independent component analysis; ISI, inter-stimulus interval; MoRI,
magnitude of the rapid increase; N100, negative deflation occurring
around 100 milliseconds after the stimulus onset; P200, positive
deflation occurring around 200 milliseconds after the stimulus onset;
PLFP, Preceding Low-Feature Phase; RMS, root mean square; STG,
superior temporal gyrus.
http://dx.doi.org/10.1016/j.neuroscience.2015.10.061
0306-4522/Crown Copyright Ó 2015 Published by Elsevier Ltd. on behalf of IBRO. All rights reserved.
58
Phases – followed by a rapid increase enhanced the ampli-
tudes of N100 and P200 responses. These ERP responses
resembled those to simpler sounds, making it possible
to utilize the tradition of ERP research with naturalistic para-
digms. Crown Copyright Ó 2015 Published by Elsevier Ltd.
on behalf of IBRO. All rights reserved.
Key words: event-related potentials, music, electroen-
cephalography, musical features, N100, P200.
INTRODUCTION
For centuries, music has been an important part of various
cultures from tribal drumming rites or performances of a
symphony orchestra to the urban underground electronic
music scene. Making music together as well as listening
to the music of a given culture assists in forming a sense
of community. At an individual level, music has versatile
effects, e.g. regulation of mood and emotions (Panksepp
and Bernatzky, 2002 for a review). Along with the technical
development of brain-imaging methods, the neural
dynamics underlying music perception, cognition, and
emotions started to fascinate researchers (Peretz and
Zatorre, 2003; Koelsch, 2014). The field of neurosciences
and music could offer explanations concerning the impor-
tance of music for humans as well as answer to questions
like: Why is music perceived differently than other auditory
stimuli like speech and environmental sounds? How are
the musical characteristics related to harmony, dynamics
and rhythm processed in the brain?
Traditionally, brain research of music with electro-
magnetic methods such as electroencephalography (EEG)
and magnetoencephalography (MEG) has focused on
understanding the neural processing of separated artificial
sounds designed to suit to the specification of each
particular experiment. This broad line of music-related
research includes different sequential sounds used as
stimuli – pure vs. complex tones (e.g., Pantev et al., 1995;
Tervaniemi et al., 2000), consonant vs. dissonant chords
(e.g. Brattico et al., 2010; Virtala et al., 2014), simple mono-
phonic melodies with and without harmony (Fujioka et al.,
2005; Brattico et al., 2006) and chordal cadences (Koelsch
and Jentschke, 2008). In addition to oddball paradigms
(Näätänen et al., 1978), multifeature paradigms have also
been established both in adults (Marie et al., 2012; Kühnis
et al., 2013; Tervaniemi et al., 2014) and in children
(Chobert et al., 2011, 2014; Putkinen et al., 2014). These
studies have offered precious information about the
 H. Poikonen et al. / Neuroscience 312 (2016) 58–73
processing of individual elements of music and paved the
way toward the research of natural listening, in which the
unique characters of music are perceived: spontaneity,
impurity, interaction and continuous flow of overlapping
notes.
Several research groups have taken the next step and
studied the brain processes evoked by long musical
excerpts with different EEG analysis approaches. For
example, Bhattacharya et al. (2001) showed that the
gamma-band synchrony increases over distributed corti-
cal areas with musical practice. This increase found in
professional musicians in comparison with laymen refers
to more advanced musical memory when dynamically
binding together several features of the intrinsic complex-
ity of music. In addition, professional training in music
refines emotional arousal, which was studied in a whole
musical piece by Mikutta et al. (2014). During high arou-
sal, professional musicians exhibited an increase of pos-
terior alpha, central delta, and beta rhythm. Also among
laymen, music is shown to be a powerful stimulus modu-
lating emotional arousal (Mikutta et al., 2012). To create
these modulations in emotional states and to transmit
esthetic experiences, confirmation and violation of expec-
tations are crucial in music perception. Pearce et al.
(2010) showed that low-probability notes, when compared
to high-probability notes, elicited larger late negative
event-related potential (ERP) component (at a time period
of 400–450 ms), and increased beta-band oscillation over
the parietal lobe and stronger long-range brain synchro-
nization between multiple brain regions. Meyer et al.
(2006a,b) investigated the perception of musical timbre
by choosing as stimuli instrument sounds and comparing
them to the sine wave sounds. In addition to the
enhanced N1/P2 responses, they revealed how instru-
ments with varying timbre activated also brain regions
associated with emotional and auditory imagery functions.
Grewe et al. (2005) studied the strong emotional experi-
ence of chills evoked by music noting that the peak emo-
tion of chills is a result of attentive, experienced and
conscious musical enjoyment. Furthermore, results by
Schaefer et al. (2011) suggest that recollecting an event
with emotional content involves multiple neural retrieval
subprocesses. These studies indicate that the immersive
sound space created by music, and its creation of strong
subjective experiences with vivid memories, emotions
and imagination, can indeed be investigated with multi-
faceted EEG analyses.
Several functional MRI (fMRI) studies have focused
on using natural continuous music. Typically, excerpts
from real musical pieces are used as stimuli (Morrison
et al., 2003; Koelsch et al., 2006; Pereira et al., 2011;
Brattico et al., 2011) and more recently even full musical
pieces (Alluri et al., 2012, 2013; Abrams et al., 2013;
Toiviainen et al., 2014). Due to the slower temporal
dynamics of hemodynamic reactions recorded with fMRI
compared to the electromagnetic brain research methods,
in the former fMRI studies brain activity is averaged
across several seconds, thus collapsing the fast dynamic
feature changes that occur in key moments in the musical
pieces. In addition, the fMRI device produces loud back-
ground noise which interacts with the auditory stimuli
59
and may distort the results (see Novitski et al., 2001,
2006). Alluri et al. (2012) took the research of natural
music further and studied the neural processing of individ-
ual musical features with fMRI during listening to a record
of real orchestral music. In their novel approach, the fMRI
data were correlated with computationally extracted musi-
cal features to study the brain activation relevant to each
particular musical feature. However, the sampling rate of
the data was 2 s, producing overlapping of the fast cranial
processing within each sample. Similarly, Alluri et al.
(2013) used two medleys, one comprising full songs by
Beatles and the other comprising instrumental pieces
belonging to the classical, jazz or pop/rock genres as
stimuli. In both studies by Alluri and colleagues, the musi-
cal features were chosen so that they depict the musically
and acoustically most relevant events and characteristics
of the musical pieces at two analysis window durations.
Both short-term features characterizing timbral properties
and long-term features related to context-dependent
aspects of music were shown to correlate with activation
in various brain regions, with the largest consistency
among features and musical genres for an anterior area
of the superior temporal gyrus (Alluri et al., 2013). When
investigating the neural activity for the low- or high-level
acoustic features, it was found that the timbral features
activated mainly the auditory cortex and the somatomotor
regions of the cerebral cortex, as well as the cerebellum,
whereas the tonal and rhythmic features activated limbic
and motor regions of the brain (Alluri et al., 2012).
To combine the development toward real musical
stimuli of EEG and the studies of individual musical
features of fMRI, we created a novel experimental
paradigm to reveal music-induced brain responses by
extracting several relevant individual musical features
from continuous musical pieces and studying the
electrophysiological brain responses evoked by changes
in these features. We decided to investigate the electric
brain activity elicited by the same acoustic features
extracted from musical pieces belonging to three very
different musical genres: a Tango Nuevo piece, a deep
techno track and an acoustic lullaby. The Tango Nuevo
piece, Adios Noniño by Astor Piazzolla was the same
piece used by Alluri et al. (2012, 2013) in their fMRI stud-
ies. In these studies, they observed that low-level musical
features, as used in our study, are mainly processed in
the auditory brain regions located in temporal cortices.
Previous knowledge from the processing of artificial
sounds was utilized in our study of continuous music, in
which the sounds are connected to each other in an
overlapping and dynamic manner. The single-trial ERP
method was considered the best option for studying the
immediate neural responses on auditory areas of
temporal cortices corresponding to rapid changes in
low-level musical features. We hypothesized that the
rapid changes in the musical features of real music
would elicit similar sensory components as revealed in
the conventional ERP studies using tone stimuli, and
that the amplitudes of the ERP components would be
dependent on the magnitude of the rapid increase in the
individual feature value (Picton et al., 1977; Polich et al.,
1996) as well as the duration of the preceding time period
60 H. Poikonen et al. / Neuroscience 312 (2016) 58–73

with low-feature values (Polich et al., 1987) in a similar 20 to 46 years (27.1 on average). No participants
way as the amplitudes of traditional ERP components reported hearing loss or history of neurological illnesses.
depend of stimulus presentation characteristics. All participants were musical laymen with no professional
Our main interests were the sensory negative musical education. However, many participants reported
deflation occurring around 100 milliseconds after the a background in different music-related interests such as
stimulus onset (N100) and positive deflation occurring learning to play an instrument, producing music with a
around 200 milliseconds after the stimulus onset (P200) computer, dancing or singing. Age and the non-
components. The N100 component reflects the acoustic professional musical background of each participant are
energy on stimulus onset and is largest in the fronto- reported in the Table 1. The experimental protocol was
central region (Woods, 1995). Generally, the N100 is conducted in accordance with the Declaration of Helsinki
thought to represent the initial extraction of the informa- and approved by the ethics committee of the Faculty of
tion from sensory analysis of the stimulus (Näätänen the Behavioural Sciences at the University of Helsinki.
and Picton, 1987), or the excitation relating to the
allocation of a channel for information processing out of
Stimuli
the auditory cortex (Hansen and Hillyard, 1980). The
N100-P200 complex is referred to as the vertex potential Three pieces of music from different genres were used as
because of its largest amplitude on the upper surface of stimuli: a modern tango (Adios Noniño by Astor
the brain (Hillyard and Kutas, 1983). Alluri et al. (2012) Piazzolla), an acoustic lullaby (Bless by Kira Kira) and a
suggested that timbre-related acoustic components of deep techno track (My Black Sheep by Len Faki and
continuous music correlate positively with activations in remixed by Radio Slave). The spectrograms of each
large areas of the temporal lobe. Since we wanted to song are shown in Fig. 1. The 8.5-min tango of Astor
study fast neural responses on auditory areas in the tem- Piazzolla was recorded in a concert in Lausanne,
poral cortices, we chose to focus on the timbral features Switzerland. This piece was chosen due to its large
of brightness, spectral flux, and zerocrossing rate. In addi- variation among several musical features related to
tion, the feature root mean square (RMS) related to loud- loudness, timbre, tonality and rhythm and to allow
ness was studied. The ERP method is shown to be comparison to the work of Alluri et al. (2012, 2013). The
adequate in the studies of musical timbre (Pantev et al., original duration of the acoustic lullaby Bless was 3 min
2001; Caclin et al., 2008). Also, Meyer et al. (2006a,b) and 17 s and it was prolonged with Audacity version
proposed that the N100 and P200 responses are 1.2.6 so that at the point of 2 min and 15 s the song
enhanced to instrumental tones when compared to sine was repeated again from the time point of 20 s until the
wave tones. In our study, the time period with low- end of the song, giving a total stimulus length of 5 min
feature values preceding the rapid increase in the value and 43 s. The acoustic lullaby had English lyrics pro-
of the same musical feature corresponds to the inter- nounced in an unclear way which made the singing sound
stimulus interval (ISI) of the previous literature. Naturally, more like humming. An excerpt was selected from the
with real music, regularly reoccurring silent intervals do
not exist. Thus, in this paper, the ISI-type of period is
Table 1. Age and musical background of each participant
called Preceding Low-Feature Phase (PLFP). In the same
way that prolonged ISI strengthens the early sensory ERP Code of Age Years of Instrument Years of Type
responses (Polich et al., 1987), we hypothesized that the participant musical activity
prolonged PLFP would increase the ERP amplitudes dur- activity in dance
ing listening to continuous music. Also, we expected the kh2 20 15 Piano/ None
magnitude of the rapid increase in each musical feature singing
correlate positively with the magnitude of the ERP ampli- kh3 23 13 Piano/flute None
tudes. The possibility to determine the neural correlates of kh4 23 16 Cello None
acoustic contrast and change in continuous music kh5 23 None 6 Ballet
enables one to study the brain with the sound material kh6 24 None None
we are exposed to in the everyday basis. Music is known kh7 20 2 Piano None
to have a strong emotional influence in the brain (Koelsch, kh8 42 15 Alto None
saxophone
2014; Sachs et al., 2015; Salimpoor et al., 2015). Thus, in
kh9 46 None None
addition to research, this method has a great potential for
kh11 22 7 Piano None
applications in therapy and rehabilitation. The efficacy of kh12 21 None None
different interventions, for example among patients suffer- kh13 34 6 Piano/ None
ing from stroke, dementia, disorders of consciousness or keyboards
mood disorders, could be estimated with applications kh14 31 5 Piano None
based on the novel method presented in this paper. kh20 25 7 Piano/ 7 Folk
violin dance/
street
EXPERIMENTAL PROCEDURES dance
kh23 25 None None
Participants kh25 24 3 Piano None
Sixteen right-handed native Finnish speakers took part in kh26 31 5 Computer None
music
the experiment; 10 female and 6 male, age ranged from
 H. Poikonen et al. / Neuroscience 312 (2016) 58–73 61

Fig. 1. Spectrograms of the musical pieces Adios Noniño by Astor Piazzolla (above), Bless by Kira Kira (middle) and My Black Sheep Radio Slave
by Len Faki (below).

whole piece of the electronic My Black Sheep Radio Slave spread over a wide-frequency spectrum with a predomi-
with Audacity. The excerpt started from the beginning of nant regular beat but without a melody. It was chosen
the electronic piece and was faded out after 5 min and due to the strong, even rhythmical structure and the lack
20 s. The techno piece consisted of rhythmical sound of harmony in contrast to the acoustic lullaby with
62 H. Poikonen et al. / Neuroscience 312 (2016) 58–73
lingering melody. The musical structure of Adios Noniño is
versatile whereas Bless and My Black Sheep Radio Slave
had more constant structure with repetitive musical pat-
terns. The last 25 s of Adios Noniño consists of the audi-
ence applause because of the live recording. Therefore,
we excluded the last 30 s of Adios Noniño from our data
analysis.
Equipment and procedure
The stimuli were presented to the participants with
Presentation 14.0 program in a random order via Sony
MDR-7506 headphones with and an intensity of 50
decibels above the individually determined hearing
threshold. This threshold was determined for each
participant by playing a recording of a Finnish children’s
poem, which was irrelevant to our study paradigm,
starting clearly above the hearing threshold, manually
attenuating the sounds, and asking the participant to
report when he/she did not hear the sound anymore.
Then, the test sounds were played below the hearing
threshold, manually augmenting the volume until the
participant reported hearing the sounds. The average of
these two reported volumes was considered as the
hearing threshold of the particular participant. The
amplifier was set to augment the stimuli of our study 50
decibels above this threshold, which was the standard
augmentation level of the amplifier used in our
experiment. The participants were advised to listen to
the music while sitting as still as possible with eyes
open. The playback of each piece of music was
launched by the researcher after a short conversation
with the participants via microphone.
The EEG data were recorded with 10–20 system
(Jasper, 1958) with BioSemi bioactive electrode caps with
64 EEG channels and 5 external electrodes placed at the
tip of the nose, left and right mastoids and around the right
eye both horizontally and vertically. The offsets of the
active electrodes were kept below 25 mV in the beginning
of the measurement and the data were collected with a
sampling rate of 2048 Hz. The beginning and the end of
each musical piece was marked with a trigger into the
EEG data.
Data processing and analysis
Feature extraction with MIRtoolbox. We used
MIRtoolbox (version 1.3.1) to computationally extract the
musical features. MIRtoolbox is a set of MATLAB
functions designed for the processing of audio files
(Lartillot and Toiviainen, 2007) and is used for the extrac-
tion of different musical features related to various musi-
cal dimensions identified in psychoacoustics and sound
engineering as well as traditionally defined in music the-
ory. In addition to the dimensions of dynamics, loudness,
rhythm, timbre and pitch, MIRtoolbox can process high-
level features related to meter and tonality, among others.
The short-term features selected in this study,
encapsulating loudness-related and timbral properties of
the stimulus, were RMS, brightness, spectral flux and
zero-crossing rate. They were obtained by employing
short-time analysis using a 25-ms window with a 50%
overlap, which is in the order of the commonly used
standard window length in the field of Music Information
Retrieval (MIR) (Tzanetakis and Cook, 2002). Overlap-
ping of windows is recommended in the analysis of musi-
cal features to detect fast changes in the features and
their possible inactive periods with a precise time resolu-
tion. Excluding RMS, these musical features are the same
as those found to activate the auditory areas in the tempo-
ral cortices in the study by Alluri et al. (2012).
Brightness is computed as the amount of spectral
energy above a threshold value fixed by default in
MIRtoolbox at 1500 Hz (Lartillot and Toiviainen, 2007)
for each analysis window. Therefore, high values in
brightness mean that a high percentage of the spectral
energy is concentrated in the higher end of the frequency
spectrum. The zero-crossing rate, known to be an indica-
tor of noisiness, is estimated by counting the number of
times the audio waveform crosses the temporal axis
(Lartillot and Toiviainen, 2007). This means that a higher
zero-crossing rate indicates that there is more noise in the
audio frame under consideration. The noise measured by
the zero-crossing rate refers to noise as opposed to har-
monic sounds rather than to noise as distortion of clean
signal. RMS is related to the dynamics of the song and
defined as the root average of the square of the amplitude
(Lartillot and Toiviainen, 2007). Louder sounds have high
RMS values whereas quieter ones have low RMS values.
Spectral flux represents the Euclidian distance between
the spectral distributions of successive frames (Lartillot
and Toiviainen, 2007). If there is a lot of variation in spec-
tral distribution between two successive frames, the flux
has high values. Spectral flux curves exhibit peaks at
transition between successive notes or chords.
Preprocessing. The EEG data of all the participants
were first preprocessed with EEGLAB (version 9.0.2.2b;
Delorme and Makeig, 2004). The external electrode of
the nose was set as a reference. The data were down-
sampled to 256 Hz, high-pass filtered with 1 Hz and low-
pass filtered with 30 Hz with finite impulse response
(FIR) filtering based on the firls (least square fitting of
FIR coefficients) MATLAB function. Visually detected
EEG channels with a noisy signal were removed from
the analysis.
Setting the triggers. The triggers related to the musical
features extracted with MIRtoolbox were added to the
preprocessed EEG data. In continuous speech, the best
ERP-related results are gained when the triggers are set
into the beginning of the word (Teder et al., 1993;
Sambeth et al., 2008). Long ISIs are shown to increase
the amplitude of the N100 component (Polich et al.,
1987). Additionally, strong stimulus intensity has been
shown to enhance ERP components (Picton et al.,
1977; Polich et al., 1996). Previous knowledge from the
individual sound processing was utilized in our study of
continuous music, in which the individual sounds are con-
nected to each other in an overlapping and dynamic
manner.
 H. Poikonen et al. / Neuroscience 312 (2016) 58–73
We designed an algorithm, implemented in MATLAB,
for the search of time points with rapid increase of a
musical feature. The algorithm was tuned using specific
parameter values adapted to each song, as shown in
Table 2. The length of the PLFP was modified and the
rapid increase was required to exceed a value called
magnitude of the rapid increase (MoRI), which was
tuned individually for each different song. The mean
values of all the segments of each song and each
feature were calculated and the magnitude of the
change was defined based on the mean value. The
largest change was from 20% of the mean value to
+20% of the mean value. Valid triggers were preceded
by a PLFP whose magnitude did not exceed the lower
threshold calculated of the mean value. The length of
PLFP was 1 s maximum. In all cases, valid triggers had
an increase phase that lasted less than 75 ms. Eight
triggers per each feature per each song were set in 10
cases out of the 12 combinations of the song and the
musical feature. However, in two cases (for RMS of
Bless and for brightness of My Black Sheep Radio
Slave), only 7 triggers were set because in these cases
only seven time points during the song matched with the
computationally set limits of the particular musical
feature. For each musical feature, the triggers were set
in the manner described in detail in Table 2.
Procedure of ERP analyses. After adding the triggers
into the preprocessed data, the data were treated with
Independent Component Analysis (ICA) decomposition
with the runica algorithm of EEGLAB (Delorme and
Makeig, 2004) trained with default settings (decomposi-
tion of input data using the logistic infomax ICA algorithm
of Bell and Sejnowski (1995) with the natural gradient
feature to detect and remove artifacts related to eye
movements and blinks as well as the heartbeat. ICA
Table 2. Presentation of the characteristics of the triggers for the features brightness, RMS, zero-crossing rate and spectral flux of songs Astor
Piazzolla: Adios Noniño, Kira Kira: Bless and Len Faki: My Black Sheep Radio Slave
Song and Number of Preceding Low-Feature
feature triggers Phase (PLFP) duration
Astor Piazzolla: Adios Noniño
Brightness 8 750 ms
RMS 8 500 ms
Zero-crossing 8 500 ms
rate
Spectral flux 8 1000 ms
Kira Kira: Bless
Brightness 8 500 ms
RMS 7 875 ms
Zero-crossing 8 500 ms
rate
Spectral flux 8 1000 ms
Len Faki: My Black Sheep Radio Slave
Brightness 7 500 ms
RMS 8 500 ms
Zero-crossing 8 500 ms
rate
Spectral flux 8 312.5 ms
63
decomposition gives as many spatial signal source com-
ponents as there are channels in the EEG data. Thus,
there were 68 components excluding the data of four par-
ticipants for whom one noisy channel each had been
removed in preprocessing. 67 ICA components were
decomposed. Typically 2 to 5 ICA components related
to the eye and heartbeat artifacts were removed. The
noisy EEG data channels of the abovementioned four
participants were interpolated. The continuous EEG data
were separated into epochs according to the triggers. The
epochs started 3000 ms before the trigger and ended
2000 ms after the trigger. The baseline was defined
according to the 500-ms time period before the trigger.
To double check the removal of the eye artifacts, the
epochs with amplitudes above ±100 lV were rejected.
In addition, all the epochs were visually inspected and
no artifacts were detected in the data.
Statistical analysis
The statistical analyses of the ERP data were conducted
with MATLAB version R2013a utilizing the Statistics
Toolbox. Shapiro–Wilk test showed that the data for
both N100 and P200 components were normally
distributed. Also, the data of each individual ERP
component were normally distributed except the N100
component of brightness in Bless by Kira Kira. T-tests
were calculated over the Cz electrode for each musical
feature of each musical piece with the MATLAB
command ttest(A), in which A refers to 16  1 array
including the peak value of each participant defined in
the following manner: For the N100 component, a
minimum value within a window from 80 ms to 150 ms
was searched for each participant and the mean value
over ±20 ms from the negative peak was calculated.
Similarly, for the P200 component, a maximum value
Mean value of the feature Magnitude of the rapid increase (MoRI) of the
across the whole song feature value, from X% to +X% of the mean
value
0.3272 20% to >+20%
0.04827 10% to >+10%
1089 15% to >+15%
13.30 15% to >+15%
0.3300 15% to >+15%
0.09500 15% to >+15%
987.3 10% to >+10%
24.25 20% to >+20%
0.2046 10% to >+10%
0.2846 15% to >+15%
471.1 15% to >+15%
85.93 10% to >+10%
64 H. Poikonen et al. / Neuroscience 312 (2016) 58–73

within a time window from 150 ms to 350 ms was
searched for each participant and the mean value over
±20 ms from the positive peak was calculated. The
command ttest performs a t-test of the null hypothesis
that data in the vector A are a random sample from a
normal distribution with mean 0 and unknown variance,
against the alternative that the mean is not 0. The test
result 1 indicates a rejection of the null hypothesis at the
5% significance level whereas test result 0 indicates a
failure to reject the null hypothesis at the 5%
significance level.
The two-way ANOVA for the factors Piece (Adios
Noniño by Astor Piazzolla, Bless by Kira Kira and My
Black Sheep Radio Slave by Len Faki) and Musical
feature (brightness, zero-crossing rate, spectral flux and
RMS) was calculated for the same peaks over the
electrode Cz as used in the t-tests with the MATLAB
command anova2 (M,number_of_participants), in which M
refers to the 64  3 matrix with peak values of participants
for four musical features in rows (4  16 = 64) according
to three musical piece defined in columns.
For the Figs. 2–6, the electrodes were pooled to
reduce noise in the plotted curves. For Figs. 2–5, three
frontal electrodes (F1, Fz, F2) were pooled as well as
central (C1, Cz, C2), parietal (P1, Pz, P2) and occipital
(O1, Oz, O2) electrodes. The Fig. 6 shows the brain
responses over the Cz electrode, which is the electrode
used in all the statistical analyses.
RESULTS
The temporal evolution of brightness in Astor Piazzolla’s
piece Adios Noniño, averaged across the triggers, and
the corresponding brain responses over the same time
window are shown in the Fig. 2. For Figs. 2–5, three
frontal electrodes (F1, Fz, F2), central electrodes (C1,
Cz, C2), parietal electrodes (P1, Pz, P2) and occipital
electrodes (O1, Oz, O2) were pooled together. The
evolutions in the musical features and their brain
responses can be seen in Figs. 3–5 in the following
order: Evolution of the musical feature spectral flux for
Kira Kira’s piece Bless in Fig. 3, zero-crossing for Len

Fig. 2. Brain response to the triggers related to increase of brightness in Adios Noniño by Astor Piazzolla. The absolute values of the amplitudes of
the EEG epochs are presented in the graph above over the frontal (F1, Fz and F2, which are averaged into one signal), central (C1, Cz and C2,
which are averaged into one signal), parietal (P1, Pz and P2, which are averaged into one signal) and occipital (O1, Oz and O2, which are averaged
into one signal) areas with the EEG epochs from 3 s to +2 s from the stimulus onset, and the temporal evolution of the musical feature brightness
for the same 5-s time window. The stimulus onset is defined by the end of the Preceding Low-Feature Phase (PLFP) period. The brain responses of
the EEG epochs are presented in the graph below with the same pooling of electrodes as in the graph above. The feature values present the values
of the feature brightness from 3 s to +2 s from the end of the PLFP period. The brightness curves for the different triggers are averaged into one
single curve over the time period from 3 to +2 s of the time points of the triggers.
 H. Poikonen et al. / Neuroscience 312 (2016) 58–73 65

Fig. 3. Brain response to the triggers related to increase of spectral flux in Bless by Kira Kira. The absolute values of the amplitudes of the EEG
epochs are presented in the graph above over the frontal (F1, Fz and F2, which are averaged into one signal), central (C1, Cz and C2, which are
averaged into one signal), parietal (P1, Pz and P2, which are averaged into one signal) and occipital (O1, Oz and O2, which are averaged into one
signal) areas with the EEG epochs from 3 s to +2 s from the stimulus onset, and the temporal evolution of the musical feature spectral flux for the
same 5-s time window. The stimulus onset is defined by the end of the Preceding Low-Feature Phase (PLFP) period. The brain responses of the
EEG epochs are presented in the graph below with the same pooling of electrodes as in the graph above. The feature values present the values of
the feature spectral flux from 3 s to +2 s from the end of the PLFP period. The curves of spectral flux for the different triggers are averaged into
one single curve over the time period from 3 to +2 s of the time points of the triggers.

Faki’s piece My Black Sheep Radio Slave in Fig. 4 and
RMS for Adios Noniño in Fig. 5.
The graphs reveal the dependence of the change in
the voltage measured with ERP signal on the change in
the musical feature value. The high value in the musical
feature is not alone sufficient to elicit an ERP
component but the high feature value needs to be
preceded by a relatively long time period with low-
feature values. The distribution of the increased
electrical activity over the cortex can be observed on
the graphs of frontal, central, parietal and occipital brain
regions. Alluri et al. (2012) revealed with fMRI that the
low-level musical features, as the ones used in our study,
increase activation mainly on the auditory regions located
on temporal cortices. In our study, the ERP components
indeed are the largest on the central region, which is
associated with increased activity originating from the
core auditory areas of temporal cortices. Thus, our results
suggest that the rapid changes in the low-level musical
features evoke neural responses in the temporal auditory
areas.
Fig. 6 reveals in detail the brain responses on the
central area for all musical features of brightness, zero-
crossing rate, spectral flux and RMS of all musical
pieces of Astor Piazzolla: Adios Noniño, Kira Kira: Bless
and Len Faki: My Black Sheep Radio Slave over the
electrode Cz. The conventional ERP responses are
shown for several features of several musical pieces.
The clearest sensory N100 and P200 components were
elicited for the brightness feature of Astor Piazzolla:
Adios Noniño which is illustrated over a longer time
window in the Fig. 2. The results of the t-tests
comparing the N100 amplitudes of the Cz electrode in
response to each feature and each musical piece
against the zero baseline are listed in Table 3.1 and for
the P200 amplitudes in Table 3.2.
At N100 amplitude, the following sound features elicited
responses which significantly differed from zero baseline:
spectral flux t(15) = 6.40, p < .0001; RMS t(15) =
2.97, p = .0095; brightness t(15) = 6.40, p<.0001
and zero-crossing rate t(15) = 7.35, p < .0001) of Adios
Noniño, zero-crossing rate t(15) = 2.76, p = .015 of
66 H. Poikonen et al. / Neuroscience 312 (2016) 58–73

Fig. 4. Brain response to the triggers related to increase of zero-crossing rate in My Black Sheep Radio Slave by Len Faki. The absolute values of
the amplitudes of the EEG epochs are presented in the graph above over the frontal (F1, Fz and F2, which are averaged into one signal), central
(C1, Cz and C2, which are averaged into one signal), parietal (P1, Pz and P2, which are averaged into one signal) and occipital (O1, Oz and O2,
which are averaged into one signal) areas with the EEG epochs from 3 s to +2 s from the stimulus onset, and the temporal evolution of the
musical feature zero-crossing rate for the same 5-s time window. The stimulus onset is defined by the end of the Preceding Low-Feature Phase
(PLFP) period. The brain responses of the EEG epochs are presented in the graph below with the same pooling of electrodes as in the graph above.
The feature values present the values of the feature zero-crossing rate from 3 s to +2 s from the end of the PLFP period. The curves of zero-
crossing rate for the different triggers are averaged into one single curve over the time period from 3 to +2 s of the time points of the triggers.

Bless and brightness t(15) = 3.06, p = .0079 and zero-
crossing rate t(15) = 2.53, p = .023 of My Black Sheep
Radio Slave. For all pieces and all musical features of the
P200 component, the elicited responses differed
significantly from zero baseline, as can be seen in detail in
Table 3.2. In Figs. 7–9 scalp maps for selected statistically
significant ERP components are presented. In these
figures the ERP components are the strongest in the
central region, which suggest the signal summation of the
left and right temporal cortices originating from the
auditory areas.
The electrode location factors (Laterality and Anterior-
posterior distribution) did not interact with any of the
factors of interest (Piece and Musical feature).
Therefore, we report here only the results from the two-
way ANOVA for the factors Piece (Adios Noniño by
Astor Piazzolla, Bless by Kira Kira and My Black Sheep
Radio Slave by Len Faki) and Musical feature
(brightness, zero-crossing rate, spectral flux and RMS)
over the electrode Cz.
For N100 component, the main effect of Piece was
significant: F(2,180) = 22.04, p < .0001, resulting from
larger N100 amplitudes for Adios Noniño by Astor
Piazzolla ( 4.17 lV) compared with My Black Sheep
Radio Slave by Len Faki ( 1.09 lV) and Bless by Kira
Kira ( 0.78 lV). We also found a significant main effect
of Musical feature: F(3,180) = 6.17, p = .0005, deriving
from larger N100 responses to the musical features of
zero-crossing rate ( 3.19 lV) and brightness
( 2.81 lV) compared to spectral flux ( 1.11 lV) and
RMS ( 0.96 lV). For brightness and RMS the largest
N100 was elicited in Adios Noniño and smallest in
Bless. For zero-crossing rate and spectral flux the
largest N100 was elicited with Adios Noniño and
smallest in My Black Sheep Radio Slave.
For the amplitudes of the P200 component, the two-
way ANOVA revealed a significant main effect of Piece:
F(2,180) = 11.51, p < .0001, deriving from larger P200
amplitudes for Adios Noniño (5.27 lV) compared with
Bless (4.19 lV) and My Black Sheep Radio Slave
 H. Poikonen et al. / Neuroscience 312 (2016) 58–73 67

Fig. 5. Brain response to the triggers related to increase of RMS in Adios Noniño by Astor Piazzolla. The absolute values of the amplitudes of the
EEG epochs are presented in the graph above over the frontal (F1, Fz and F2, which are averaged into one signal), central (C1, Cz and C2, which
are averaged into one signal), parietal (P1, Pz and P2, which are averaged into one signal) and occipital (O1, Oz and O2, which are averaged into
one signal) areas with the EEG epochs from 3 s to +2 s from the stimulus onset, and the temporal evolution of the musical feature RMS for the
same 5-s time window. The stimulus onset is defined by the end of the Preceding Low-Feature Phase (PLFP) period. The brain responses of the
EEG epochs are presented in the graph below with the same pooling of electrodes as in the graph above. The feature values present the values of
the feature RMS from 3 s to +2 s from the end of the PLFP period. The RMS curves for the different triggers are averaged into one single curve
over the time period from 3 to +2 s of the time points of the triggers.

(3.03 lV). For Feature, the main effect was not significant:
F(3,180) = 1.24, p = 0.30. For RMS and spectral flux the
P200 was largest in Bless and smallest in My Black
Sheep Radio Slave, and for brightness and zero-
crossing rate the P200 was largest in Adios Noniño and
smallest in My Black Sheep Radio Slave.
The relevance of magnitude of the rapid increase and
Preceding Low-Feature Phase
The MoRI and the PLFP between the musical features of
three musical pieces and the magnitude of the evoked
ERP components seem to be correlated. However, the
following speculation was not examined statistically. The
clearest ERP components were observed for brightness
in Astor Piazzolla’s piece Adios Noniño (Fig. 2 and
Fig. 6). In Table 2 PLFP preceding the trigger can be
seen between all three musical pieces. It is important to
note that the values of the PLFPs are the minimum
PLFP and not the exact PLFP. Brightness of Adios
Noniño had the longest PLFP with 750 ms in contrast to
brightness of Kira Kira: Bless and Len Faki: My Black
Sheep Radio Slave which both have the PLFP of
500 ms. Also MoRI of the feature value is larger for
Adios Noniño (from 20 per cent of the mean value to
+20 per cent of the mean value) compared to Bless
and My Black Sheep Radio Slave (from 15 per cent to
+15 percent and from 10 per cent to +10 per cent,
respectively). Again, these magnitudes of rapid changes
are the threshold magnitudes, not the exact magnitudes.
In addition, clear ERP components were also revealed
for zero-crossing rate of Adios Noniño but not for Bless or
My Black Sheep Radio Slave (Fig. 6) even though all the
songs have PLFP of 500 ms. Small P200 components
can be seen for My Black Sheep Radio Slave but not
for Bless. These differences could be explained with
different MoRI in the feature value (from 15 per cent to
+15 per cent for Adios Noniño and My Black Sheep
Radio Slave and from 10 per cent to +10 per cent for
Bless). Even though Adios Noniño and My Black Sheep
Radio Slave both have the same PLFP and magnitude
of increase in the feature value, only Adios Noniño had
68 H. Poikonen et al. / Neuroscience 312 (2016) 58–73

Fig. 6. Recapitulation of ERP responses for features spectral flux, RMS, brightness and zero-crossing rate of pieces Astor Piazzolla: Adios Noniño,
Kira Kira: Bless and Len Faki: My Black Sheep Radio Slave. The ERP responses are the values measured over the electrode Cz.

Table 3.1. T-tests of the N100 component (time window from 80 ms to

150 ms of the stimulus onset) over the Cz electrode for the features
brightness, RMS, zero-crossing rate and spectral flux of songs Astor Table 3.2. T-tests of the P200 component (time window from 150 ms to
Piazzolla: Adios Noniño, Kira Kira: Bless and Len Faki: My Black 350 ms of the stimulus onset) over the Cz electrode for the features
Sheep Radio Slave brightness, RMS, zero-crossing rate and spectral flux of songs Astor
Piazzolla: Adios Noniño, Kira Kira: Bless and Len Faki: My Black
t-test t15 p Sheep Radio Slave

Astor Piazzolla: Adios Noniño t-test t15 p

Brightness 6.40 0.000012
Astor Piazzolla: Adios Noniño
RMS 2.97 0.0095
Brightness 7.41 0.00000218
Zero-crossing rate 7.35 0.00000239
RMS 8.16 0.000000675
Spectral flux 4.68 0.000297
Zero-crossing rate 6.92 0.00000491
Kira Kira: Bless Spectral flux 7.52 0.00000184
Brightness 1.56 0.14
Kira Kira: Bless
RMS 0.56 0.58
Brightness 9.29 0.000000131
Zero-crossing rate 2.76 0.015
RMS 6.33 0.0000134
Spectral flux 0.39 0.70
Zero-crossing rate 4.43 0.000485
Len Faki: My Black Sheep Radio Slave Spectral flux 8.58 0.000000359
Brightness 3.06 0.0079
Len Faki: My Black Sheep Radio Slave
RMS 1.24 0.24
Brightness 3.73 0.0020
Zero-crossing rate 2.53 0.023
RMS 5.55 0.0000560
Spectral flux 0.56 0.59
Zero-crossing rate 3.83 0.0016
Spectral flux 6.02 0.0000234

a clear ERP response. The triggering is based on the

mean value of each musical piece and these two pieces
have very different mean values for zero-crossing rate
(1089 for Adios Noniño and 471.1 for My Black Sheep
Radio Slave; Table 2).
For the responses related to the values of spectral flux
and RMS (Fig. 6), the ERP components are evoked for
Adios Noniño and Bless but not My Black Sheep Radio
Slave. The latter has low values for PLFP and
magnitude of change of feature value for triggered time
points related to spectral flux (PLFP 312.5 ms and the
MoRI of the feature value from 10 per cent of the
mean value to +10 per cent of the mean value
compared to Adios Noniño and Bless; change in
magnitude of the feature value from 15 percent to
+15 per cent and 20 per cent to +20 percent,
respectively, and both have PLFP of 1000 ms). In
contrast, for RMS, My Black Sheep Radio Slave has the
same PLFP (500 ms) and change in magnitude (from
10 percent to +10 percent) as Adios Noniño.
However, the mean value varies a lot for RMS for all
 H. Poikonen et al. / Neuroscience 312 (2016) 58–73 69

Fig. 7. Scalp maps for selected ERP components of the pieces and the musical features, for which the ERP components differ significantly from the
zero baseline. The latencies of N100 component for Adios Noniño are: Brightness 131 ms and zero-crossing rate 122 ms.

Fig. 8. Scalp maps for selected ERP components of the pieces and the musical features, for which the ERP components differ significantly from the
zero baseline. The latencies of P200 component for Adios Noniño are: spectral flux 221 ms, RMS 210 ms, brightness 234 ms and zero-crossing rate
232 ms.

Fig. 9. Scalp maps for selected ERP components of the pieces and the musical features, for which the ERP components differ significantly from the
zero baseline. The latencies of P200 component for Bless are: spectral flux 203 ms, RMS 215 ms and brightness 237 ms.

songs (0.04827 for Adios Noniño, 0.09500 for Bless and
as high as 0.2846 for My Black Sheep Radio Slave).
DISCUSSION
In the present study, we investigated the ERP responses
elicited during rapid changes in low-level musical features
of three pieces from different music genres. We wanted to
study brain responses for music in general and keep the
category of music wide by using acoustically, digitally
and vocally produced material. A novel technique based
on automatic digital recognition of given sound features
enabled an interesting approach to study different
musical pieces by extracting the same musical features
from each musical piece. For this study we focused on
a few short-term musical features which were related to
the timbral and dynamic characters of the musical
pieces. To take the full advantage of the precise
temporal resolution of EEG and to be able to match the
rapid changes in music with the rapid changes in the
EEG data in a scale of milliseconds, we excluded time–
frequency analyses, such as wavelet and sliding
window, as possible methodological approaches. EEG
microstate analysis was excluded since, instead of the
global cognitive processes, we wanted to focus on the
sound processing on auditory regions of temporal
cortices, the importance of which was highlighted by
Alluri et al. (2012).
The results show a relationship between the
magnitude of the ERP components and the magnitude
of the rapid change in the feature value as well as the
length of the preceding time with low-feature values.
Therefore, the earlier results which coupled the intensity
of the sound and the length of the ISI with the
70 H. Poikonen et al. / Neuroscience 312 (2016) 58–73
magnitude of the ERP components seem to be valid also in
the context of the dynamics of the musical features. In
other words, the ERP components are elicited not only
by simple sound streams with precisely defined silent
ISIs but also by dynamic continuous natural stimuli such
as a musical piece. However, the musical stimulus needs
to include strong acoustical contrasts in the musical
features to be able to elicit conventional ERP components.
Carterette and Kendall (1999) describe the perceptual
process as that operating on the principle of contrast or
change, during which the sensory mechanisms look for
changes in order to make sense of the information in music.
In addition, Kluender et al. (2003) suggest that the percep-
tual systems of all sensory modalities respond mainly to
changes. Spectrotemporal modulations, represented by
spectral flux in general, and specifically its sub-bands
(Alluri and Toiviainen, 2012), is shown to capture this
aspect of perceivable change in polyphonic timbre. Alluri
et al. (2012) showed that the brain regions activated corre-
sponding to the musical features encapsulating brightness,
RMS, spectral flux and zero-crossing rate of the same
musical piece as the one used in our study (Adios Noniño)
are located in the auditory regions on the right and left tem-
poral cortex, namely superior temporal gyrus (STG) and
middle temporal gyrus (MTG) on the left hemisphere and
STG and Heschl’s gyrus (HG) on the right hemisphere.
Due to summation of the electric dipoles in the brain tissue,
the EEG signal originating from these regions is the stron-
gest on the frontal and central middle line. Therefore, it is
very likely that the ERP components measured in our study
are generated in the same brain regions as revealed in the
study of Alluri et al. (2012) and that these auditory-related
regions are sensitive in detecting rapid changes in these
musical features in particular.
Teder et al. (1993) highlighted the relevance of the
length of the ISI in the elicitation and magnitude of N100
component for continuous natural speech. Based on our
results, the length of the PLFP preceding the time point
of interest plays a role in the elicitation of N100 also in
continuous music. The prolonged PLFP seems to
increase the amplitude of N100. In addition, stimulus
intensity has been shown to influence the N100 amplitude
such that a stimulus with stronger intensity elicits a larger
N100 (Picton et al., 1977). Similarly, in our study the N100
and P200 amplitudes seem to correlate positively with the
increase of the magnitude of the rapid change in the musi-
cal feature of interest. It is important to note that in our
study at the time points of rapid change, both the intensity
and the feature value changed drastically. Further studies
need to be performed to extract the subcomponents
related to the intensity and the feature value from the
ERP components.
The clearest N100 and P200 responses were
observed for brightness in Astor Piazzolla’s piece Adios
Noniño. These findings reflect what was observed in the
fMRI cross-validation study using the same musical
piece (Alluri et al., 2013): Adios Noniño activated the brain
more robustly than the other pieces. In addition, clear
N100 and P200 components were also revealed for
zero-crossing rate of Adios Noniño but not for Bless or
My Black Sheep Radio Slave. These differences could
be explained with different magnitude of the increase in
the feature value. Adios Noniño and My Black Sheep
Radio Slave have very different mean values for zero-
crossing rate and since the triggering is based on the
mean value of each musical piece, the stimulus intensity
of zero-crossing-related time points may still be stronger
in Adios Noniño. Also, the musical features of brightness
and zero-crossing rate are strongly correlated in Adios
Noniño. The subjective listening to the triggered time
points related to brightness and zero-crossing rate reveal
similar elements of the sound across these time points: A
strong clear sound, either a single tone or a chord, is
played after a longer fading sound. This character of the
trigger-related time points matches well also with the
mathematical definition of brightness and zero-crossing
rate; both are large for the high pitches and small for
low pitches. This kind of unexpected sound, elicited while
all the preceding sound is fading, can be reasoned to form
sensory N100 and P200 components.
For the responses related to the values of spectral flux
and RMS, the N100 and P200 components are evoked for
Adios Noniño and Bless but not My Black Sheep Radio
Slave. The mean value of RMS calculated for each
musical piece varies a lot across three different pieces
used and reflects the principle of the lower the mean
value the stronger the N100 component. Based on the
mathematical definition of RMS, the PLFP with low RMS
values before the trigger-related time point involves
sounds with very low decibel levels. Novitski et al.
(2003) studied the effect of fMRI noise to ERP responses
and noticed that the N100 diminished in the presence of
noise. The existence of sound during PLFP can be con-
sidered as noise from the point of view of traditional
ERP research. Thus, higher RMS values during PLFP
may lead to an attenuated N100 response evoked by
the rapid increase of the RMS value.
In Bless, N100 responses for spectral flux and RMS
are smaller compared to Adios Noniño despite similar
PLFP and MoRI values. However, spectral flux and
RMS of Bless have higher values during PLFP
compared to Adios Noniño which might attenuate
the N100 responses. Since spectral flux indicates
the spectral difference of two consecutive segments, the
higher value indicates more varying spectral content
between segments. Again, the mean value for spectral
flux in Adios Noniño is significantly smaller than in
Bless. Thus, the difference in flux and RMS mean
values between Bless and Adios Noniño could explain
the smaller N100 component of Bless. However, the
sound in the background during PLFP does not affect
the P200 component. According to Novitski et al.
(2003), the P3 component and its latency increase due
to noise, which illustrates how differently each ERP com-
ponent reacts to noise. In any case, both latency and
magnitude of P200 amplitude remained unchanged for
Bless and Adios Noniño, which could indicate that the
N100 component is more sensitive to sound in the back-
ground than the later ERP components.
The current results are encouraging for further
research with this new realistic music paradigm for ERP
method.
 H. Poikonen et al. / Neuroscience 312 (2016) 58–73
Limitations
Due to pioneering and exploratory characteristics of the
current contribution, there are several issues which need
to be taken into account and, in some occasions, also
improved, when using the analysis method in future.
These issues, to be further specified below, deal with the
stimulation, ERP analysis, and the participants of the study.
First, regarding the stimulation, the musical pieces
used in our study differed acoustically from each other:
Human voice, instrumental sounds and digital sounds
are all known to evoke different brain processes (Belin
et al., 2000; Meyer et al., 2006a,b). However, according
to Levy et al. (2001), the difference between human vocal
and instrumental sounds affects only the later ERP com-
ponents starting from the 260 ms from the stimulus onset,
but not the early ones (N100 and P200 components)
which were in the focus in our study.
Second, regarding the analyses, since this study is
based on real music, the computationally defined
changes in the values for the musical features are
based on mathematically set boundaries. In the case of
such a specific threshold value, anything larger/smaller
than the threshold value is treated equally. Therefore,
the set of equally treated short musical excerpts may
include acoustically very different excerpts. These
excerpts are categorized into the same group based on
the values gained for a particular musical feature
meaning that all the other factors defining that specific
musical excerpt are ignored. The inter-stimulus variation
is an acknowledged deficit in single-trial ERP method
and with natural, non-repetitive stimuli this variation
becomes even larger. Thus, both mathematically and
philosophically, this straightforward categorizing evokes
uncertainty. However, when studying human perception,
emotion and cognition, especially with natural stimuli
such as music, we need to keep in mind the creative
and non-linear nature of the human brain in perception
of the external world, which has already been revealed
in such phenomena as the McGurk effect (McGurk and
MacDonald, 1976) and binaural beats (Dove, 1839;
Lane et al., 1998).
In addition, in the group comparisons of Musical
feature and Piece the results were not corrected for
multiple comparisons. These corrections would most
likely have reduced the number of the ERP components
which differed statistically significantly from each other.
Moreover, one of the challenges with the ERP method
consists in the unfavorable signal-to-noise ratio due to
artifacts and neural background activity. Typically 25–100
trials are averaged to form a clear ERP response (Luck,
2004). Here we had 8 trials per sound feature in each par-
ticipant. Also, the musical piece of Adios Noniño was
remarkably longer than Bless and My Black Sheep Radio
Slave, which could evoke differences in the habituation
processes in the brain (Seppänen et al., 2013). However,
Seppänen et al. (2013) discovered rapid perceptual
learning to the musical sounds only for professional musi-
cians but not for the musical laymen as used in our study.
Moreover, the number of repetitions in her study was
considerably higher than in our study. Since we had 16
71
participants and since the signal was very consistent
across the participants, it reached significance and can
be considered reliable. Yet, this analysis does not allow
us to study the individual responses of each participant
due to the small amount of averages. Another issue with
ERP analysis is high trial-to-trial variability, especially when
using natural non-repetitive stimulus such as music. How-
ever, this is an issue to consider in all time-locked analyses
of EEG. In contrast, long tradition and thus the extensive lit-
erature available for the ERP method is one of the remark-
able advantages of the method chosen. In addition, ERP
analysis is computationally light and technology is fully
mobile, which enables applications in clinical use with lower
technical demands in places such as hospitals, asylums,
therapy centers and refugee camps for example when
estimating the depth of coma (Fischer et al., 2008) or the
prognosis of the vegetative state (O’Kelly et al., 2013).
Third, regarding the participants, the unbalance of the
genders (10 females, 6 males) and relatively large age
range among participants (19–46 years) need also be
acknowledged. Age-related decline in auditory temporal
processing start to emerge in middle--aged adults and are
suggested to occur due to changes in auditory processing
in the central auditory system (Bertoli et al., 2002; Alain
et al., 2004). Thus, the large age range of the participants
may affect ERP responses in some participants in our
study, especially in the feature of brightness which captures
the rapid change from low frequencies to high.
Moreover, regarding the gender of the participants,
males are shown to have more pronounced right
hemispheric predominance in processing musical syntax
compared to female (Koelsch et al., 2003). Wager et al.
(2003) suggested that emotional activity is more lateral-
ized among males, whereas females showed more brain-
stem activity in affective paradigms. Moreover, females
tend to be more sensitive to emotional stimuli compared
to males, not due to increased emotional reactivity among
female, but due to more efficient emotional regulation
among male (McRae et al., 2008). However, to our knowl-
edge, there is no evidence about gender effects on corti-
cal auditory processes as measured in our study.
All the participants were native Finnish speakers but
their background in other languages was not
documented in this study. Yet, it should be noticed that
it is mandatory for all Finnish pupils to learn two foreign
languages as a part of their school curriculum after the
age of 7 years (in most cases English and Swedish, in
this order). This issue is of importance since the
profound knowledge in several languages is known to
shape the auditory areas. Ressel et al. (2012) showed
that the participants, who were bilingual since childhood,
had a larger HG bilaterally compared to the monolingual
participants. The enlarged HG is shown to improve per-
ception of sounds of an unfamiliar language (Golestani
et al., 2007). Therefore, differences in the language back-
ground might influence also to the perception of music
and therefore should be carefully screened in the future
studies. More importantly, the versatile non-professional
musical background of the participants may induce
individual differences in perception of the musical
features. The reported music-related interests varied from
72 H. Poikonen et al. / Neuroscience 312 (2016) 58–73
folk dance to singing and computer music. Many partici-
pants also had an experience of playing an instrument
over several years whereas some participants did not
report any music-related background. This heterogeneity
in the study sample may increase variance the ERP
components evoked in our study (Putkinen et al., 2014).
Further research is needed to specify the influence of
music and music-related background, such as dance, to
the ERP components evoked by a musical piece.
CONCLUSION
The aim of the study was to develop a novel paradigm
based on ERP method in the research of neural
correlates of continuous natural stimuli. Therefore,
continuous music was used as stimuli from which
changes in the individual musical features were
observed. The usage of continuous music as stimuli
creates a novel and uncharted test setting for the ERP
method. Studies investigating dynamics of musical
feature processing during continuous listening to entire
pieces of music have been conducted with fMRI (Alluri
et al., 2012, 2013; Toiviainen et al., 2014; Cong et al.,
2014), which gave an insight to both cortical and subcor-
tical processing of several high- and low-level musical
features. Since the temporal resolution of EEG is much
more precise compared to fMRI and since the two meth-
ods measure different kind of activity in the brain, the
extension of the research to EEG is important. With the
ERP method of EEG, temporal processing of natural stim-
uli in the cortex can be investigated in detail. The ERP
components N100 and P200 revealed for rapid changes
in the values related to spectral flux, RMS, brightness
and zero-crossing rate indicate the suitability of the realis-
tic, continuous stimuli in ERP research. Thus, elicitation of
ERP components is not restricted only to simplified, well-
controlled sequences of separated sounds presented with
silent ISI or with relatively repetitive musical passages.
Acknowledgments—We would like to thank Kone Foundation, The
Finnish Cultural Foundation and Academy of Finland for financial
support, Miika Leminen, Tommi Makkonen, Valtteri Wikström and
Tanja Linnavalli for their assistance during the EEG recordings
and data processing Caitlin Dawson for proofreading.
REFERENCES
Abrams DA, Ryali S, Chen T, Chordia P, Khouzam A, Levitin DJ,
Menon V (2013) Inter-subject synchronization of brain responses
during natural music listening. Eur J Neurosci 37(9):1458–1469.
Alain C, McDonald KL, Ostroff JM, Schneider B (2004) Aging: a
switch from automatic to controlled processing of sounds?
Psychol Aging 19:125–133.
Alluri V, Toiviainen P (2012) Effect of enculturation on the semantic
and acoustic correlates of polyphonic timbre. Music Percept 29
(3):297–310.
Alluri V, Toiviainen P, Jääskeläinen IP, Glerean E, Sams M, Brattico
E (2012) Large-scale brain networks emerge from dynamic
processing of musical timbre, key and rhythm. NeuroImage 59
(4):3677–3689.
Alluri V, Toiviainen P, Lund TE, Wallentin M, Vuust P, Nandi AK,
Ristaniemi T, Brattico E (2013) From Vivaldi to Beatles and back:
predicting lateralized brain responses to music. NeuroImage 83
(12):627–636.
Belin P, Zatorre RJ, Lafaille P, Ahad P, Pike B (2000) Voice-selective
areas in human auditory cortex. Nature 403:309–312.
Bell AJ, Sejnowski TJ (1995) An information maximisation approach
to blind separation and blind deconvolution. Neural Comput 7
(6):1129–1159.
Bertoli S, Smurzynski J, Probst R (2002) Temporal resolution in young
and elderly subjects as measured by mismatch negativity and
psychoacoustic gap detection task. Clin Neurophysiol 113:396–406.
Bhattacharya J, Petsche H, Pereda E (2001) Long-range synchrony in
the c band: role in music perception. J Neurosci 21(16):6329–6337.
Brattico E, Tervaniemi M, Näätänen R, Peretz I (2006) Musical scale
properties are automatically processed in the human auditory
cortex. Brain Res 1117(1):162–174.
Brattico E, Jacobsen T, De Baene W, Glerean E, Tervaniemi M
(2010) Cognitive vs. affective listening modes and judgements of
music – An ERP study. Biol Psychol 85(3):393–409.
Brattico E, Alluri V, Bogert B, Jacobsen T, Vartiainen N, Nieminen S,
Tervaniemi M (2011) A functional MRI study of happy and sad
emotions in music with and without lyrics. Front Psychol 2:308.
Caclin A, McAdams S, Smith BK, Giard MH (2008) Interactive
processing of timbre dimensions: an exploration with event-
related potentials. J Cogn Neurosci 20(1):49–64.
Carterette EC, Kendall RA (1999) Comparative music perception and
cognition. In: Deutsch D, editor. The Psychology of Music. San
Diego: Academic. p. 725–791.
Chobert J, Marie C, François C, Schön D, Besson M (2011)
Enhanced passive and active processing of syllables in
musician children. J Cogn Neurosci 23(12):3874–3887.
Chobert J, François C, Velay J-L, Besson M (2014) Twelve months of
active musical training in 8- to 10-year-old children enhances the
preattentive processing of syllabic duration and voice onset time.
Cereb Cortex 24(4):956–967.
Cong F, Puoliväli T, Alluri V, Sipola T, Burunat I, Toiviainen P, Nandi AK,
Brattico E, Ristaniemi T (2014) Key issues in decomposing fMRI
during naturalistic and continuous music experience with
independent component analysis. J Neurosci Meth 223(2): 74–84.
Delorme A, Makeig S (2004) EEGLAB: an open source toolbox for
analysis of single-trial EEG dynamics. J Neurosci Meth 134(1):9–21.
Dove HW (1839) Repertorium der Physik, Vol. III, p 494.
Fischer C, Dailler F, Morlet D (2008) Novelty P3 elicited by the
subject’s own name in comatose patients. Clin Neurophysiol
119:2224–2230.
Fujioka T, Trainor LJ, Ross B, Kakigi R, Pantev C (2005) Automatic
encoding of polyphonic melodies in musicians and nonmusicians.
J Cogn Neurosci 17(10):1578–1592.
Golestani N, Molko N, Dehaene S, LeBihan D, Pallier C (2007) Brain
structure predicts t learning of foreign speech sounds. Cereb
Cortex 17:575–582.
Grewe O, Nagel F, Kopiez R, Altenmüller E (2005) How does music
arouse ‘‘chills”? Ann NY Acad Sci 1060:446–449.
Hansen JC, Hillyard SA (1980) Endogenous brain potentials
associated with selective auditory attention. Electroencephalogr
Clin Neurophysiol 49:277–290.
Hillyard S, Kutas M (1983) Electrophysiology of cognitive processing.
Ann Rev Psychol 34:33–61.
Jasper HH (1958) The ten-twenty electrode system of the
international federation. Electroen Clin Neuro 10:371–375.
Kluender KR, Coady JA, Kiefte M (2003) Sensitivity to change in
perception of speech. Speech Commun 41:59–69.
Koelsch S (2014) Brain correlates of music-evoked emotions. Nat
Rev Neurosci 15:170–180.
Koelsch S, Jentschke S (2008) Short-term effects of processing
musical syntax: an ERP study. Brain Res 1212:55–62.
Koelsch S, Maess B, Grossmann T, Friederici AD (2003) Electric
brain responses reveal gender differences in music processing.
NeuroReport 14:709–713.
Koelsch S, Fritz T, v Cramon DY, Müller K, Friederici AD (2006)
Investigating emotion with music: an fMRI study. Hum Brain Mapp
27:239–250.
 H. Poikonen et al. / Neuroscience 312 (2016) 58–73
Kühnis J, Elmer S, Meyer M, Jäncke L (2013) The encoding of vowels
and temporal speech cues in the auditory cortex of professional
musicians: an EEG study. Neuropsychologia 51:1608–1618.
Lane JD, Kasian SJ, Owens JE, Marsh GR (1998) Binaural auditory
beats affect vigilance performance and mood. Physiol Behav 63
(2):249–252.
Lartillot O, Toiviainen P (2007) A Matlab toolbox for musical feature
extraction from audio. International Conference on Digital Audio
Effects, Bordeaux.
Levy DA, Granot R, Bentin S (2001) Processing specificity for human
voice stimuli: electrophysiological evidence. NeuroReport 12
(2653):2657.
Luck SJ (2004) An introduction to the event-related potential
technique. second edition. The MIT Press.
Marie C, Kujala T, Besson M (2012) Musical and linguistic expertise
influence pre-attentive and attentive processing of non-speech
sounds. Cortex 48:447–457.
McGurk H, MacDonald J (1976) Hearing lips and seeing voices.
Nature 264:746–748.
McRae K, Ochsner KN, Mauss IB, Gabrieli JJ, Gross JJ (2008) Gender
differences in emotion regulation: an fMRI study of cognitive
reappraisal. Group Process Intergroup Relat 11(2):143–162.
Meyer M, Baumann S, Jäncke L (2006a) Electrical brain imaging
reveals spatio-temporal dynamics of timbre perception in humans.
NeuroImage 32:1510–1523.
Meyer M, Baumann S, Jancke L (2006b) Electrical brain imaging
reveals spatio-temporal dynamics of timbre perception in humans.
NeuroImage 32:1510–1523.
Mikutta C, Altorfer A, Strik W, Koenig T (2012) Emotions, arousal,
and frontal alpha rhythm asymmetry during Beethoven’s 5th
symphony. Brain Topogr 25:423–430.
Mikutta CA, Maissen G, Altorfer A, Strik W, Koenig T (2014)
Professional musicians listen differently to music. Neuroscience
268:102–111.
Morrison SJ, Demorest SM, Aylward EH, Cramer SC, Maravilla KR
(2003) FMRI investigation of cross-cultural music comprehension.
NeuroImage 20(1):378–384.
Näätänen R, Picton T (1987) The N1 wave of the human electric and
magnetic response to sound: a reviw and an analysis of the
component structure. Psychophysiology 24:375–425.
Näätänen R, Gaillard AWK, Mäntysalo S (1978) Early selective-
attention effect on evoked potential reinterpreted. Acta Psychol
42:313–329.
Novitski N, Alho K, Korzyukov O, Carlson S, Martinkauppi S, Escera
C, Rinne T, Aronen HJ, Näätänen R (2001) Effects of acoustic
gradient noise from functional magnetic resonance imaging on
auditory processing as reflected by event-related brain potentials.
NeuroImage 14:244–251.
Novitski N, Anourova I, Martinkauppi S, Aronen HJ, Näätänen R,
Carlson S (2003) Effects of noise from functional magnetic
resonance imaging on auditory event-related potentials in
working memory task. NeuroImage 20(2):1320–1328.
Novitski N, Maess B, Tervaniemi M (2006) Frequency specific
impairment of automatic pitch change detection by fMRI
acoustic noise: an MEG study. J Neurosci Meth 155:149–159.
O’Kelly J, James L, Palaniappan R, Taborin J, Fachner J, Magee WL
(2013) Neurophysiological and behavioral responses to music
therapy in vegetative and minimally conscious states. Front Hum
Neurosci 12(7):884.
Panksepp J, Bernatzky G (2002) Emotional sounds and the brain: the
neuro-affective foundations of musical appreciation. Behav
Process 60:133–155.
Pantev C, Bertrand O, Eulitz C, Verkindt C, Hampson S, Schuierer G,
Elbert T (1995) Specific tonotopic organizations of different areas
of the human auditory cortex revealed by simultaneous magnetic
and electric recordings. Electroencephalogr Clin Neurophysiol 94
(1):26–40.
(Accepted 30 October 2015)
(Available online 7 November 2015)
73
Pantev C, Roberts LE, Schultz M, Engelien A, Ross B (2001) Timbre-
specific enhancement of auditory cortical representations in
musicians. NeuroReport 12:1–6.
Pearce MT, Herrojo Ruiz M, Kapasi S, Wiggins GA, Bhattacharya J
(2010) Unsupervised statistical learning underpins computational,
behavioural, and neural manifestations of musical expectation.
NeuroImage 50:302–313.
Pereira CS, Teixeira J, Figueiredo P, Xavier J, Castro SL, Brattico E
(2011) Music and emotions in the brain: familiarity matters. PLoS
One 6(11):e27241.
Peretz I, Zatorre RJ, editors. (2003) The cognitive neuroscience of
music. Oxford University Press.
Picton TW, Woods DL, Baribeau-Braun J, Healey TMG (1977)
Evoked potential audiometry. J Otolaryngol 6(2):90–116.
Polich J, Aung M, Dalessio DJ (1987) Long-latency auditory evoked
potentials: intensity, inter-stimulus interval and habituation.
Pavlovian J Biol Sci 23(1):35–40.
Polich J, Ellerson PC, Cohen J (1996) P300, stimulus intensity,
modality and probability. Int J Psychophysiol 23(1):55–62.
Putkinen V, Tervaniemi M, Saarikivi K, de Vent N, Huotilainen M
(2014) Investigating the effects of musical training on functional
brain development with a novel melodic MMN paradigm.
Neurobiol Learn Mem 110:8–15.
Ressel V, Pallier C, Ventura-Campos N, Diaz B, Roessler A, Ávila C,
Sebastián-Gallés N (2012) An effect of bilingualism on the
auditory cortex. J Neurosci 32(47):16597–16601.
Sachs ME, Damasio A, Habibi A (2015) The pleasure of sad music: a
systematic review. Front Hum Neurosci 24(9):404.
Salimpoor VN, Zald DH, Zatorre RJ, Dagher A, McIntosh AR (2015)
Predictions and the brain: how musical sounds become
rewarding. Trends Cogn Sci 19(2):86–91.
Sambeth A, Ruohio K, Alku P, Fellman V, Huotilainen M (2008)
Sleeping newborns extract prosody from continuous speech. Clin
Neurophysiol 119(2):332–341.
Schaefer A, Pottage CL, Rickart AJ (2011) Electrophysiological
correlates of remembering emotional pictures. NeuroImage
54:714–724.
Seppänen M, Hämäläinen J, Pesonen A-K, Tervaniemi M (2013)
Passive sound exposure induces rapid perceptual learning in
musicians: event-related potential evidence. Biol Psychol
94:341–353.
Teder W, Alho K, Reinikainen K, Näätänen R (1993) Interstimulus
interval and the selective-attention effect on auditory ERPs:
‘‘N100 enhancement” versus processing negativity.
Psychophysiology 30(1):71–81.
Tervaniemi M, Schröger E, Saher M, Näätänen R (2000) Effects of
spectral complexity and sound duration on automatic complex-
sound pitch processing in humans - a mismatch negativity study.
Neurosci Lett 290:66–70.
Tervaniemi M, Huotilainen M, Brattico E (2014) Melodic multi-feature
paradigm reveals auditory profiles in music-sound encoding.
Front Hum Neurosci 8:496.
Toiviainen P, Alluri V, Brattico E, Wallentin M, Vuust P (2014)
Capturing the musical brain with Lasso: dynamic decoding of
musical features from fMRI data. NeuroImage 88(3):170–180.
Tzanetakis G, Cook P (2002) Music genre classification of audio
signals. T-SAP 10:293–302.
Virtala P, Huotilainen M, Partanen E, Tervaniemi M (2014)
Musicianship facilitates the processing of Western music
chords-An ERP and behavioral study. Neuropsychologia 61:
247–258.
Wager TD, Phan KL, Liberzon I, Taylor SF (2003) Valence, gender
and lateralization of functional brain anatomy in emotion: a meta-
analysis of findings from neuroimaging. NeuroImage 19:513–531.
Woods DL (1995) The component structure of the N1 wave of the
human auditory evoked potential. Electroencephalogr Clin
Neurophysiol Suppl 44:102–109.