Neuroinform (2009) 7:245253

DOI 10.1007/s12021-009-9059-9

Fully Parametric Sleep Staging Compatible

with the Classical Criteria
Urszula Malinowska Hubert Klekowicz
Andrzej Wakarow Szymon Niemcewicz
Piotr J. Durka

Published online: 21 November 2009

Humana Press Inc. 2009

Abstract We present an open system for sleep staging,

based explicitly on the criteria used in visual EEG
analysis. Slow waves, theta and alpha waves, sleep spindles and K-complexes are parameterized in terms of
time duration, amplitude, and frequency of each waveform by means of the matching pursuit algorithm. It
allows the detection of these structures using mostly the
criteria from visual EEG analysis. For example, within
this framework for the first time we compute directly
the relative duration of slow waves, which is a basic
parameter in recognition of deep sleep stages. Performance of the system is evaluated on 20 polysomnographic recordings, scored by experienced encephalographers. Seven recordings were scored by more than
one expert. Proposed system gives concordance with visual staging close to the inter-expert concordance. The
algorithm is implemented in a user-friendly software
system for display and analysis of polysomnographic

U. Malinowska (B) H. Klekowicz P. J. Durka

Department of Biomedical Physics, Faculty of Physics,
University of Warsaw, ul. Hoz a 69, 00-681 Warszawa, Poland
e-mail: ula@fuw.edu.pl
H. Klekowicz
e-mail: hubert@fuw.edu.pl
P. J. Durka
e-mail: durka@fuw.edu.pl
A. Wakarow Sz. Niemcewicz
Department of Psychiatry, Medical University of Warsaw,
ul. Nowowiejska 27, 00-665 Warszawa, Poland
A. Wakarow
e-mail: awakar@psych.waw.pl
Sz. Niemcewicz
e-mail: szymon@psych.waw.pl

recordings, freely available with complete source code

from http://signalml.org/svarog.html.
Keywords Sleep EEG Stager Hypnogram
Matching pursuit Time-frequency

Introduction
Sleep stages scoring, as described in Rechtschaffen
and Kales (1968), is based upon recognition of several basic features of polysomnographic recordings.
These basic elements can be either structurescalled
transients or waveformspresent in the EEG, or
some general properties of the analyzed epoch of the
polysomnographic recording, relating also to the EMG
(electromyogram, that is muscle activity) and EOG
(electrooculogram, measuring eye movements).
EEG structure, relevant to staging, includes slow
waves, sleep spindles, K-complexes, vertex sharp waves
(VSW), alpha, theta and sawtooth waves. More or less
exact definitions of these phenomena were developed
in decades of visual analysis of EEG for the purpose
of standardization. They rely on terms like cycles per
second and time duration, which at a first glance
may appear to be easily translatable to the language
of computers methods used for signal analysis. This is
true for the frequency (cycles per second) since the
first application of Fourier Transform to EEG (Dietsch
1932). However, following decades of development of
computer methods brought surprisingly little progress
in reliable estimation of the other parameter, important
in classification of waveforms, that is their time duration. This stems from the fact that most of the methods
rely on a prior setting of this parameter, e.g. constant

246

length of the analysis window in spectrogram (shorttime Fourier transform) or constant ratio of frequency
to the time width of wavelets.
The first signal processing algorithm, which adopted
the window length to the local features of the analyzed
signal (which relates also to a reliable estimation of the
time width of detected waveforms) was the matching
pursuit (MP), proposed by Mallat and Zhang (1993).
It implements a suboptimal, iterative solution to the
problem of adaptive approximations in time-frequency
dictionaries of functions. Over 10 years of MP applications to various paradigms of EEG and MEG has
proven its sensitivity, selectivity, and high concordance
to visual detection of certain EEG waveforms, which
led to the thesis of unification of visual and computer
EEG analysis within this framework (Durka 2007b). In
particular, detection of sleep spindles and delta waves,
and its concordance with visual analysis, was presented in Durka et al. (2005), Malinowska et al. (2006),

Zygierewicz
et al. (1999).
This study gathers all these results into an open, user
friendly system for sleep staging, based upon a bottomup approach starting with detection and parameterization of relevant waveforms and patterns, and then
combining these results (together with information
from EMG and EOG signals) into final decision assigning sleep stages to epochsaccording to the R&K rules
(Rechtschaffen and Kales 1968).
Previous approaches to automatic sleep staging include hybrid systems (Park et al. 2000; Smith and
Karacan 1971), methods based on statistical pattern
recognition techniques (Martin et al. 1972; Stanus et al.
1987), neural networks (Schaltenbrand et al. 1996) and
expert systems (Ray et al. 1986; Kubat et al. 1994)
these examples include only a fraction of the studies on
digital sleep analyses reviewed in Penzel et al. (2007). In
addition to the automatization achieved so far, the procedure presented hereby can be fully controlled by the
user in any of its stages, from detection of waveforms
and patterns, through primary assignment of stages, to
final smoothing. All these steps can be verified separately by experts, who can adjust the parameters of
detection using terms and notions from the electroencepalographers rather than engineers dictionaryfor
example, minimum amplitude (microvolts) of delta
waves or sleep spindles. Primary assignment of stages
is a direct interpretation of R&K rules (Rechtschaffen
and Kales 1968), while the smoothing algorithm incorporates the classic rules and experiences published so
far in relevant literature. Given the above possibility
of interaction, and the clear foundations of the whole
algorithm, electroencephalographers can gain a full understanding of the procedures and their limitations, and

Neuroinform (2009) 7:245253

acquire a useful and reliable aid in one of the most

tedious routine tasks.
Methods and Materials
Parameterization of EEG Waveforms
Matching pursuit algorithm (MP) is an iterative heuristic proposed in Mallat and Zhang (1993) for optimal
representation of a signal in a redundant dictionary
of functions. We use time-frequency dictionaries composed of Gabor functions:
g (t) = e (

) cos (2 f (t u) + )

tu 2
w

(1)

with different n = {u, f, w, }. Examples of different

shapes of Gabor functions, which can be included in the
dictionary used for MP decomposition are presented in
Fig. 1.
MP provides a linear expansion, approximating the
analyzed signal:
s

M1


an gn

(2)

n=0

in terms of functions gn chosen from the dictionary.

Parameters u, f , w, and from Eq. 1 give time position,
frequency, time width and phase of each structure gn
detected in the signal; amplitudes are given (indirectly)
by an from Eq. 2see also Mallat and Zhang (1993),
Durka (2007a, b).
Classical criteria, defining some of the EEG waveforms in terms of frequency, time duration and amplitude, can be translated directly into the ranges of
parameters f , u and an , yielding automatic detection
and parameterization of relevant structures. This approach was successfully used for detection of slow

waves (Durka et al. 2005), sleep spindles (Zygierewicz

et al. 1999), epileptic spikes (Durka 2004), as well as

Fig. 1 Examples of different shapes of Gabor functions

Neuroinform (2009) 7:245253

247

selective estimation of powers in delta, theta and sigma

bands (Durka et al. 2002). Table 1 gives ranges of
parameters used in this study for automatic detection
of sleep spindles and alpha, delta, theta waves and
functions corresponding to K-complexes.
For compatibility with visual detection, amplitudes
of delta and theta waves were calculated as the actual
difference between maximum and minimum, contrary
to the mathematical amplitude of Gaussian envelope,
as in Durka (2004). Minimum amplitude of slow waves
ASWA was adjusted automatically for each recording,
to account for the inter-subject differences in EEG
amplitudes, using a linear dependence of the standard
deviation of EEG signals found in Durka et al. (2005).
For K-complexes, there was an additional constraint on
the phase, enforcing a negative deflection and condition
for the amplitude to exceed also the background amplitude by factor 1.5.
Detection of Body and Eye Movements
Movement Time (MT) is a separate stage assigned in
case of EEG and/or EMG signals obscured in more
than half the epoch by muscle tension and/or amplifier blocking artifacts associated with movement of the
subject (Rechtschaffen and Kales 1968). Movement
artifacts are usually reflected in a big increase of the
amplitude of EEG or EMG signals. In order to detect
these movement times, EEG in C3 and C4 derivations
was high-pass filtered above 15 Hz and EMG was highpass filtered above 5 Hz. From these filtered signals, a
trimmed root mean square (RMS) was calculated for
each 1-s epoch, after rejecting all values above the 75%
percentile in each second; giving 3 values (for C3, C4
and EMG). These values are compared to thresholds,
determined from the training data: Thr1EEG common
for C3 and C4, and Thr1EMG for EMG.
The tonic EMG activity differs across the sleep
stages and time of sleep. It almost always reaches its
lowest level during stage REMthis fact is used in the
scoring of sleep stages (Rechtschaffen and Kales 1968).
To determine relative level of tone EMG signal, for
each 20-s epochs mean of 1-s values (of trimmed RMS

Table 1 Parameters of functions (Eq. 1) from MP decomposition

(Eq. 2), used for automatic detection of relevant waveforms
Slow waves
Sleep spindles
Alpha waves
Theta waves
K-complexes

Frequency

Time duration

Min. amplitude

0.24 Hz
1115 Hz
812 Hz
48 Hz
0.052.5 Hz

0.5 s.
0.52.5 s.
1.5 s.
0.1 s.
0.31.5 s.

ASWA
13 V
5 V
15 V
100 V

of filtered EMG signal, as for MT above) were calculated, after omitting 10% of the smallest and 10% of
the largest values (Anderer et al. 2005). This procedure
was adopted to reject the burst-type high activities of
EMG occurred also in stage REM. Thresholds for the
value of tone EMG, detecting stage REM (Thr2), were
determined on the basis of the data collected to train
the system. These thresholds are computed for each
sleep recording as the average EMG tone of all the
epochs (with the additional constraint of not exceeding
50 V).
Rapid eye movement supports the information
needed to assign stage REM. Detection of rapid eye
movement is based upon correlation coefficients between two EOG derivations EOGL and EOGR, and
deflection between left and right EOG. The correlation
coefficients between EOGL and EOGR are computed
in 0.5-s epoch. Thresholds Thr3 for minimum deflection
between left and right EOG were also determined
based on the training data. If the correlation coefficients falls below 0.9, and EOGL-EOGR deflection is
lower than Thr3, a rapid eye movement event is scored.
Assigning Stages to Epochs
Automatic sleep stages classification is effectuated in
the hierarchical manner, presented in Fig. 2. In the
first step, each 20-s epoch is tested for muscle artifacts
in EEG or EMG derivation, which can indicate the
Movement Time (MT). Relevant parameter reflecting
increase in amplitude was calculated as described in
Section Detection of Body and Eye Movements. If
at least one of the analyzed derivations (C3, C4 or
EMG) exceeded a corresponding threshold (Thr1EEG
common for C3 and C4, and Thr1EMG for EMG) in
more than 50% of the 20-s epoch, the epoch is scored
as Movement Time.
In the second step, the algorithm detects slow wave
sleep (SWS) stages 3 and 4, by applying fixed 20%
and 50% thresholds to the amount of epochs time,
occupied by slow waves. Estimation of this parameter is
described in Section Parameterization of EEG Waveforms and in Durka et al. (2005).
In the next step, the algorithm detects sleep spindles
and K-complexes, which are related to stage 2. If at
least one sleep spindle or K-complex occurs in 20 s
epoch, and less than 75% of the epoch is occupied
by alpha activity (Section Parameterization of EEG
Waveforms) the epoch is scored as stage 2.
If alpha activity occupies above 75%, EOG and
EMG signals of this epoch are examined to distinguish stage REM from Wake. Stage REM is scored
in case of tone of EMG below Thr2, in presence of

248

Neuroinform (2009) 7:245253

stage 1. In the absence of theta wavesas stage Wake.

This is the alternative way of scoring stage Wake for
subjects who do not present clear alpha activity.
Smoothing the Hypnogram
According to Rechtschaffen and Kales (1968), the
epoch-by-epoch approach presented in the previous
section does not imply that each epoch is considered in
isolation. We explicitly implement the following, generally accepted smoothing rules: the 3-minute rule for
stage 2 without of any sleep spindles or K-complexes
(Rechtschaffen and Kales 1968), smoothing rules of
replacing consecutive epoch of REM S2 REM, S2 S1
S2, W REM W with REM REM REM, S2 S2 S2, W W
W respectively (Baumgart-Schmitt et al. 1998; Virkkala
et al. 2007). At the end of the procedure, hypnogram is reviewed for elimination of 1-epoch stages
(except MT).
Experimental Data

Fig. 2 Block diagram of the staging algorithm (Section

Assigning Stages to Epochs)

rapid eye movement events. Rapid eye movements

are detected from the EOG signal as described in
Section Detection of Body and Eye Movements,
the tone of EMG is determined as described in
Section Detection of Body and Eye Movements.
At this stage of the algorithm, we are dealing with
epochs which were not classified as stage 2, without
delta waves, and with possible alpha activity. Then, if
tone of EMG is below Thr2, and there are rapid eye
movements, epoch is scored as stage REM. If tone of
EMG is above Thr2, the epoch is scored as stage Wake.
Finally, if the epoch does not meet any of the above
rules, and theta waves are present, it is scored as

Whole night polysomnographic recordings of healthy

volunteers were archived in the Department of
Psychiatry, Medical University of Warsaw, as control groups for several other studies. Although EEG
was recorded in complete 1020 system, to maintain
the compatibility with rules of visual stage scoring
(Rechtschaffen and Kales 1968), we analyzed only C3
A2 derivation, in some cases using also information
from C4-A1.
In 7 of these recording, at least two (max 3) experts scored independently sleep stages. These data
were used for evaluation of inter-expert concordance (Table 2), and to estimate thresholds Thr1EEG ,
Thr1EMG , Thr2 and Thr3 (Sections Parameterization
of EEG Waveforms and Detection of Body and
Eye Movements), which were not defined explicitly
in Rechtschaffen and Kales (1968). The remaining 20
recordings were used to evaluate performance of the
system in routine applications, that is on datasets which
were not used for optimization of systems parameters
(Table 3).
Evaluation of Agreement in Sleep Scoring
Mathematical details of reporting agreement in sleep
scoring are presented in Durka et al. (2005) hereby we
briefly recall notions necessary for Section Results.
Basic form of presenting details of concordance between two sleep scores on the same dataset, created
either by two human experts or by an expert and an automatic system, is the contingency table, cross-classifies

Neuroinform (2009) 7:245253

249

Table 2 Inter-expert agreement in epoch-by-epoch scoring of sleep stages, evaluated on 7 overnight recordings (9128 epochs) scored
by 2 experts
Expert B
Expert A

Stage

S1

S2

S3

S4

SREM

MT

Selectivity
(%)

S1
S2
S3
S4
SREM
W
MT
Sensitivity (%)

126
85
1
0
12
18
13
49

36
3764
736
164
77
4
32
78

1
69
189
494
2
0
5
24

0
0
7
848
0
0
0
99

48
223
23
0
1663
2
29
83

36
34
0
0
3
188
27
65

7
17
0
3
9
4
129
76

49
89
20
56
94
87
55
76

Total concordance 76% (pair agreement range 7084%), Kappa coefficient 0.65, Stage 1 concordance 33%, S271%, S313%, S456%,
SREM80%, W60%, MT47%

data with respect to categories, i.e. sleep stages as in

Tables 2 and 3.
Total concordance is calculated by summing the
main diagonal of the contingency table and dividing by
the total number of epochs. In addition, the Cohens
Kappa is calculated to access the non-random component of this agreement. Kappa ranges from 0 to 1;
zero indicates that the agreement is not better than
produced by chance (Cohen 1960).
Independently of the Kappa, there is also a measure
of the proportion of agreement between two judges
within each of the categories separately (each assigned
sleep stage). This consists of the number of epochs,
scored by two experts as each stage, given that either of
the experts scored as these stages. Concordance for the
seven stages are given in the captions of Tables 2 and 3.
The last row of these tables indicates percentage of
the agreement between experts (or between system
and expert) for one category, with Expert B treated
as the referencethis is the sensitivity of Expert A.
The last column indicates percentage agreement between experts for one category, with expert A as the
referencethis is a selectivity of Expert A. Finally,

sensitivity and selectivity are understood as TP/(TP +

FN) and TP/(TP + FP) respectively, where TP, FP and
FN are the number of true positive, false positive, and
false negative detections.

Results
Staging criteria defined in Rechtschaffen and Kales
(1968) leave a significant margin for subjective interpretation; therefore, hypnograms constructed for the same
recording, even by experts from the same laboratory,
can be slightly different (Kim et al. 1992; Monroe 1967).
This effect is illustrated in Fig. 3, where hypnograms
by three human experts are displayed together with the
hypnogram constructed automatically by the system.
These differences in scoring of analyzed data are
summarized by statistical measures in Tables 2 and 3
tables of counts that cross-classify data. Table 2
presents a comparison of scoring sleep stages in 7
recordings by at least two experts.
As presented in Table 3, the total number of 20-s
epochs used in this study was 25316, corresponding to

Table 3 Concordance of the automatic detection of sleep stages, based upon MP parameterization of EEG (by rows) evaluated on 20
overnight recordings (25316 epochs), with hypnograms by human expert (by columns)
Human expert
MP Stager

Stage

S1

S2

S3

S4

SREM

MT

Selectivity
(%)

S1
S2
S3
S4
SREM
W
MT
Sensitivity (%)

279
168
96
3
235
90
24
31

125
9554
1294
175
370
45
102
82

59
664
1646
232
6
12
9
63

0
58
814
2170
0
0
6
71

181
701
64
6
4290
0
83
81

73
167
93
1
112
393
130
39

42
172
62
3
119
43
315
42

37
83
41
83
83
67
47
73

Total concordance 73% (range 6681%), Kappa coefficient 0.63, Stage 1 concordance 20%, S269%, S333%, S463%, SREM69%,
W33%, MT28%

250
Fig. 3 Example hypnograms
scored independently
for the same recording by
3 human experts (upper plots)
and presented algorithm
(bottom)

Neuroinform (2009) 7:245253

Expert1
M
W
S1
REM
S2
S3
S4
0

Expert2
M
W
S1
REM
S2
S3
S4
0

3
Expert3

M
W
S1
REM
S2
S3
S4
0

3
MP Stager

M
W
S1
REM
S2
S3
S4
0

about 140 h of polysomnographic recordings. Total concordance of the proposed automatic detection of sleep
stages with hypnograms by human experts, scored for
these epochs, is 73% with Cohens Kappa of about 0.63,
ranging from 66% to 81% for individual recordings.
Taking the stages separately, mean concordance between visual scoring and automated analysis was 21%
for sleep stage 1, 69% for stage 2, 33% for stage 3,
63% for stage 4, 69% for REM, and 33% and 28% for
stages Wake and Movement Time, respectively. These
concordances exhibit a clearly different pattern across
the stages for the system-expert and inter-expert cases
(Fig. 4). For stages 3 and 4 the algorithm achieves
on the averagebetter concordance with human experts (respectively 33% and 63%) than the inter-expert
concordance for these stages (13% and 56%). This is
not a paradox: if a system achieves a stable performance close to the average of experts decisions, then
the mean distance between any experts score and the
center of all the scores should be smaller than the
mean distance between random expert-expert pairs.
Similar result is reported in Anderer et al. (2005).
Location of the systems results close to the center of
the experts decisions is achieved owing to the adaptive
time-frequency parameterization by matching pursuit,
discussed in Section Parameterization of EEG Wave-

time [h]

forms. Significantly higher system-expert concordance

stems also from the large discrepancies between experts
in scoring stages 3 and 4.
Rules for the other stages are not as clearly and
strictly based upon time-frequency properties of well
defined waveforms, as it is for stages 3 and 4. Also, their
scoring depends on other factors, which are difficult

[%]

80
70
60
50
40
30
20
10
0
S1

S2

S3

S4

SREM

system-expert
expert-expert
W

MT

Fig. 4 The concordance (percentage) within each of the categories (sleep stages: S1, S2, S3, S4, SREM, W, MT) consists as the
number of epochs, scored by two experts (or system and expert)
as each stage, given that either of the experts (or system and
expert) scored as these stages

Neuroinform (2009) 7:245253

251

Fig. 5 Screenshot from the

Svarog system, implementing
the full procedure described
in this paper. In the
background part of one EEG
epoch from sleep recording,
in the upper part hypnogram
computed for the whole
recording. Window in the
foreground sets thresholds for
structures detectionBasic
config tab offers their
automatic setting. Complete
system is freely available
from http://signalml.org/
svarog.html

to translate a priori into clear-cut algorithmic rules.

Therefore, system-expert concordance for Movement
Time, stage 1 and Wake is lower than inter-expert
agreement. Finally, the concordance for stage 2 proves
similar to inter-expert agreements, owing mostly to relatively clear rules and good detection of sleep spindles.
The system used for all the computations, including the algorithm of automatic sleep stage classification, is freely available as the complete source code
(Matlab(R) version) from http://eeg.pl/stager. As a further step towards user-friendliness and ease of use,
these procedures were implemented in a user-friendly,
open and multisystem environment available from
http://signalml.org/svarog.html. This interface provides
users with possibility of manually adjusting the thresholds, crucial for the detection of all the relevant structures, which provide the basic input for the staging
algorithm. Basic config tab (Fig. 5) allows using the
automatic procedure of setting the thresholds based
upon the analyzed signal content. For delta waves, this
procedure was described in Durka et al. (2005) as mentioned in Section Parameterization of EEG Waveforms. For sleep spindles and alpha waves a similar
procedure was implemented. Advanced config offers
possibility to manually set the minimum and maximum
amplitudes for all the waveforms from Table 1 and
adopt thresholds for detection of movement artifacts
and muscle tension to analyzed signals.

Discussion
Results from the previous section indicate, that the concordance of the presented algorithm with visual staging
is similar to the inter-expert concordance. Although
similar results were already reported in literature, their
direct comparison to the performance of the presented
system, evaluated in details in Section Results, is very
difficult, because of different approaches to reporting
concordance. For example, Schaltenbrand et al. (1996)
reported sensitivity between 80 and 84.5%, Prinz et al.
(1994) mean proportion of agreement of 0.74 and a
mean kappa coefficient of 0.57, Hashizume et al. (2001)
total agreement ratio 85.8%, average agreement rate in
normal recordings 87.5% was reported by Park et al.
(2000), Stanus et al. (1987) found 7075% concordance,
Hasan et al. (1993) reported the agreements between
the computer and visual scores relatively good for 5
subjects having a prominent occipital alpha activity
during wakefulness (range 7079%) but less promising
(range 6470%) for the other 4 subjects with poor
occipital alpha activity. A review of these results is
given in Penzel et al. (2007).
These expert systems, tuned explicitly for maximizing concordance with visual staging, were usually based
on black-box approaches like e.g. artificial neural networks, which, according to Caffarel et al. (2006), are not
sufficiently accurate for sleep study analysis using the

252

R&K classification system. Parameters of such systems

are often difficult to relate to the physical properties of
the signal, as perceived by human experts. Also, most of
these systems are closed-source commercial solutions,
for which fine details of the algorithm are not disclosed.
For example, the Somnolyzer 24 x 7 (Anderer et al.
2005) provided 80% overall concordance (epoch-byepoch) with the human expert scoring and Morpheus
I Sleep Scoring System achieved mean concordance of
75% against 2 human scorers (Pittman et al. 2004).
On the contrary, bottom-up approach described in
this paper, together with an open implementation,
freely available from the Internet with complete source
code (Section Results), relies on basic notions which
are directly related to the language of visual EEG
analysis and physical properties of EEG waveforms.
Therefore, it is partly robust in respect to changing
criteria or environments: if we understand the nature of
the change, we only need to tune the parameters related
to the relevant structures or parts of the procedure,
without the need of repeated training of a black-box
expert system. Owing to this correspondence, adaptation of the proposed algorithm to the new AASM
criteria (Ibert et al. 2007) will be straightforward, once
a sample of reference hypnograms done by experts will
be available for verification.

Information Sharing Statement

All the software used in this study is freely available
from the Internet. Links to the matching pursuit signal
decomposition software, Matlab scripts computing the
hypnograms, and the Svarog system for EEG viewing and annotations, implementing also the mentioned
Matlab code via Matlab Builder for Java, are collected
on a dedicated page http://eeg.pl/stager. As for the
polysomnographic recordings used for the evaluation
of the presented system, please contact Coauthors from
the Medical University of Warsaw.
Acknowledgements This work was supported by Polish funds
for science 2006-2009, grant 3T11E02330 and NN518262933.

References
Anderer, P., Gruber, G., Parapatics, S., et al. (2005). An
E-health solution for automatic sleep classification according to Rechtschaffen and Kales: Validation study of the
Somnolyzer 24 7 utilizing the Siesta database. Neuropsychobiology, 51, 115133.
Baumgart-Schmitt, R., Herrmann, W., & Eilers, R. (1998). On
the use of neural network techniques to analyze sleep EEG
data. third communication: Robustification of the classifi-

Neuroinform (2009) 7:245253

cator by applying an algorithm obtained from 9 different
networks. Neuropsychobiology, 37, 4958.
Caffarel, J., Gibson, J. G., Harrison, J., Griffiths, C., &
Drinnan, M. (2006). Comparison of manual sleep staging
with automated neural network-based analysis in clinical
practice. Medical and Biological Engineering and Computing, 44(12), 105110.
Cohen, J. (1960). A coefficient of agreement for nominal scales.
Educational and Psychological Measurement, 20, 3746.
Dietsch, G. (1932). Fourier analyse von elektroenzephalogrammen des menschent. Pflgers Archiv fr die Gesamte Physiologie, 230.
Durka, P. J. (2004). Adaptive time-frequency parametrization of
epileptic EEG spikes. Physical Review E, 69(051914).
Durka, P. J. (2007a). Matching pursuit. Scholarpedia, 20910.
Durka, P. J. (2007b). Matching pursuit and unification in
EEG analysis. Engineering in medicine and biology. Artech
House. ISBN 978-1-58053-304-1.
Durka, P. J., Malinowska, U., Szelenberger, W., Wakarow, A., &
Blinowska, K. J. (2005). High resolution parametric description of slow wave sleep. Journal of Neuroscience Methods,
147(1), 1521.
Durka, P. J., Szelenberger, W., Blinowska, K., Androsiuk, W., &
Myszka, M. (2002). Adaptive time-frequency parametrization in pharmaco EEG. Journal of Neuroscience Methods,
117, 6571.
Hasan, J., Hirvonen, K., Varri, A., Hakkinen, V., & Loula, P.
(1993). Validation of computer analysed polygraphic patterns during drowsiness and sleep onset. Electroencephalography and Clinical Neurophysiology, 87, 117127.
Hashizume, Y., Kuwahara, H., Uchimura, N., Mukai, M.,
Shirakawa, S., Satomura, T., et al. (2001). Examination of
accuracy of sleep stages by means of an automatic sleep
analysis system, Sleep Ukiha. Psychiatry & Clinical Neurosciences, 55(3), 199200.
Ibert, C., Ancoli-Israel, S., Chesson, A., & Quan, S. (2007).
The AASM manual for the scoring of sleep and associated events: Rules, terminology and technical specification (1st ed.). Westchester: American Academy of Sleep
Medicine.
Kim, Y., Kurachi, M., Horita, M., Matsuura, K., & Kamikawa, Y.
(1992). Agreement in visual scoring of sleep stages among
laboratories in Japan. Journal of Sleep Research, 1, 5860.
Kubat, M., Pfurtscheller, G., & Flotzinger, D. (1994). AI-based
approach to automatic sleep classification. Biological Cybernetics, 70(5), 443448.
Malinowska, U., Durka, P. J., Blinowska, K., Szelenberger, W.,
& Wakarow, A. (2006). Micro- and macrostructure of sleep
EEG. IEEE BME Magazine, 25, 2631.
Mallat, S., & Zhang, Z. (1993). Matching pursuit with timefrequency dictionaries. IEEE Transactions on Signal Processing, 41, 33973415.
Martin, W., Johnson, L., Viglione, S. S., et al. (1972). Pattern
recognition of EEG-EOG as a technique for all-night sleep
stage scoring. Electroencephalography and Clinical Neurophysiology, 32(4):17427.
Monroe, L. (1967). Inter-rater reliability and the role of experience in scoring EEG sleep. Psychophysiol, 5, 376384.
Park, H.-J., Oh, J.-S., Jeong, D.-U., & Park, K.-S. (2000). Automated sleep stage scoring using hybrid rule- and casebased reasoning. Computers and Biomedical Research, 33,
330349.
Penzel, T., Hirshkowitz, H., Harsh, J., Chervin, R., Butkov,
N., Kryger, M., et al. (2007). Digital analysis and technical specifications. Journal of Clinical Sleep Medicine, 3(2),
109120.

Neuroinform (2009) 7:245253

Pittman, S., MacDonald, M., RB, R. F., Malhotra, A., Todros, K.,
Levy, B., et al. (2004). Assessment of automated scoring of
polysomnographic recordings in a population with suspected
sleep-disordered breathing. Sleep, 27(7), 13941440.
Prinz, P., Larsen, L., Moe, K., Dulberg, E., & Vitiello, M. (1994).
C stage, automated sleep scoring: Development and comparison with human sleep scoring for healthy older men and
women. Sleep, 17(8), 7177.
Ray, S., Lee, W., Morgan, C. D., & Airth-Kindree, W. (1986). Computer sleep stage scoringan expert system approach. International Journal of Bio-medical Computing, 19(1), 4361.
Rechtschaffen, A., & Kales, A., Eds. (1968). A manual of standardized terminology, techniques and scoring system for sleep
stages in human subjects. Number 204 in National Institutes
of Health Publications. Washington DC: US Government
Printing Office.
Schaltenbrand, N., Lengelle, R., Toussaint, M., Luthringer, R.,
Carelli, G., Jacqmin, A., et al. (1996). Sleep stage scor-

253
ing using the neural network model: Comparison between
visual and automatic analysis in normal subjects and patients. Sleep, 9(1), 2635.
Smith, J., & Karacan, I. (1971). EEG sleep stage scoring by an automatic hybrid system. Electroencephalography and Clinical
Neurophysiology, 31(3), 231237.
Stanus, E., Lacroix, B., Kerkhofs, M., & Mendlewicz, J. (1987).
Automated sleep scoring: a comparative reliability study of
two algorithms. Electroencephalography and Clinical Neurophysiology, 66(4), 448456.
Virkkala, J., Hasan, J., Vrri, A., Himanen, S.-L., & Mller,
K. (2007). Automatic sleep stage classification using twochannel electro-oculography. Journal of Neuroscience Methods, 166, 109115.

Zygierewicz,
J., Blinowska, K. J., Durka, P. J., Szelenberger, W.,
Niemcewicz, S., & Androsiuk, W. (1999). High resolution
study of sleep spindles. Clinical Neurophysiology, 110(12),
21362147.

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.