Toward a better Event-Related Potential based Brain-Computer Interface: Paradigm and Algorithm

BCI 37-107451, Yaming XU Yoshikazu NAKAJIMA, Associate Professor A Two-Level Predictive (TLP) paradigm was proposed based on neurophysiology and psychophysics. Compared with conventional Row/Column (RC) paradigm, the proposed one can evoke significantly better Event-Related Potential (ERP) results in significant better classification accuracy. In addition, a predictive language model was utilized to accelerate selection. Therefore, superior online accuracy, speed and information transfer rate was obtained. Furthermore, a Bayes fusion-based classifier was proposed. Offline analysis revealed its advantage. 1. Introduction 3.05 2.3 1 two decades. A common signal for ERP-based BCI is 8.14 6. 15

(B.2) 3x3 RC (step-2) (B.3) 3x3 SC (step-p) (A) 8x8 RC (B.1) 3x3 RC (step-1) Fig. (A) Conventional Row/Column (RC) paradigm for the 8x8 matrix, with one column flashing. (B) Proposed Two-Level Predictive (TLP) paradigm. The TLP has three different spelling steps (step-1, 2, p shown in B.1, B.2 and B.3). Horizontal and vertical adjacent distances of RC and TLP in this experimental setup are 3.05 and 2.31, 8.12 and 6.15 separately. Two kinds of Stimulus Types: RC and Single Character (SC, B.3) were applied separately, see text for details. Brain-Computer Interface (BCI) allows primates to directly interact with computers just using their brain signal [1]. This implies that BCI is probably the only resort for neuromuscular disorders, who do not have control of their peripheral nerve and muscles, to communicate with external world. Among various Neuroimaging means, electroencephalography (EEG) is still very popular because of its non-invasive nature, high temporal resolution, cheap and easy setup. Event-Related Potential (ERP) based BCI due to its relative high information throughput and no long-term training need is widely studied in the past the P300 which is a positive deflection over frontal, central and parietal cortex about 300ms after a rare and meaningful stimulus presentation according to oddball paradigm [2]. Row/Column (RC) random flashing matrix in Fig.1 (A) serves as an oddball paradigm is adopted by the classical P300 BCI as stimulus paradigm [2]. P300 can be evoked by attending to target flashing while ignoring others. So, such BCI can be served as a typing device, etc. However, such paradigm suffers adjacent distraction and refractory effects, more details in [3]. 2. Novelties: Paradigm and Algorithm

To mitigate these effects, a Two-Level Predictive (TLP) paradigm was proposed, shown in Fig.1 (B). 2.1 Adjacent Distraction Effect Spatial adjacent non-target stimulus may attract subjects attention while BCI operation will result in non-P300 evoked for target stimulus. Such attentional blink is called adjacent distraction effect. Compared with the 8x8 matrix (Fig.1 (A)), the 3x3 matrix with larger visual adjacent distance maybe reduce the distraction problem because visual acuity remarkably decreases with visual angle increases over 2 [4]. In addition, focusing attention on a target among fewer similar targets is much easier for human so that a larger P300 can be expected due to P300 amplitude is inversely correlated with mental workload [5]. Furthermore, relative larger visual stimulus probably evokes larger exogenous Visual Evoked Potential (VEP) for attended target flashing. Such discriminable information could contribute to the overall ERP classification. 2.2 Refractory Effect P300 amplitude is believed proportional to Target to Target temporal Interval (TTI) [5, 6]. In addition, TTI less than 500ms will distort P300 morphology [7]. Such refractory effect will affect P300 classification. Unlike row/column flashing in the RC paradigm, Single Character (SC) paradigm (Fig.1 (B.3)) flashes single item randomly. To facilitate comparison, we adjust sequence duration of 3x3 RC as same as 3x3 SC in current study. This results in larger average TTI for 3x3 SC, so that larger P300 amplitude is expected for 3x3 SC paradigm. 2.3 Level Design and Predictive Language Model In order to preserve same item amount, two-level scheme was designed (Fig.1 (B.1, B.2)). However, for one target selection, two steps were needed. To further improve selection efficiency, a predictive language model Partial Predictive-Match (PPM)

model [8] was incorporated to generate new matrix (Fig.1 (B.3)) filled by eight items with top-eight conditional probability given the typed history.. 2.4 Bayes fusion The predictive language model was also exploited to improve classifier. See sub-section 3.4. 3. Experiments 3.1 Subjects Fourteen healthy subjects (age: 25.291.77, four females) took part in the experiment. All except one were nave to BCI use. They were given written consent and the study was performed in accordance with the Declaration of Helsinki. 3.2 Experimental Procedure Each subject completed an 8x8 RC and a 3x3 TLP sessions counterbalanced. Each session consisted of a calibration phase and online test phase using the corresponding stimulus paradigm (Fig.1). In each phase, subjects were provided target items to select. Subjects just need to attend to the target stimuli and count flashing time. In calibration phase, constant 15 flashing sequences were used for one selection, while subject-specific sequences were applied in online test phase by WSR (see sub-section 3.5). Moreover, no result feedback in calibration phase due to no classifier was applied while result feedback existed in online test phase and subjects were asked to correct all selection errors. Note that the calibration phase of 3x3 TLP session includes two sub-phases: 3x3 TLP using RC and SC stimulus type separately. Which reached maximum WSR quicker was applied in online test phase. 3.3 Data Acquisition, Processing Subjects sat in a chair about 1m from a monitor that displayed a corresponding stimulus matrix (Fig.1) for current sessions. EEG was recorded with a 21channel electrode cap according to 10-20 system. All channels were referenced to the linked mastoid and

grounded to the forehead. The electrode impedances were reduced below 10k. The EEG signal were amplified and digitally sampled at 512Hz by NeXus32 System. Then, the signal were pass filtered at 0.5Hz~30hz and sectioned into overlapping epochs ranging from -100ms prestimulus to 800ms poststimulus. Before final classification, the epochs were moving averaged and downsampled to 20Hz. 3.4 Classification Given a sequence of EEG epochs X and the corresponding stimulus sequence C :

score

can

be

converted

into

probability

representative:

P ( X | ci ) =

1 (6) 1 + exp{ a [ L( X | ci ) + b]}

Finally, classification is performed according to (6), (3), (5) and (2). The Bayes fusion method was offline tested on online data collected during 3x3 TLP session. SWLDA was applied online classification in current study. 3.5 Performance Measures The golden standard measurements for BCIs are selection accuracy (especially online accuracy), speed and information transfer rate (ITR) [1]. These measurements were compared between conventional 8x8 RC and proposed 3x3 TLP in Section 4. Furthermore, written symbol rate (WSR) was used for determining optimal sequence number for online test, more details in [10]. 4. Results and Discussion 4.1 Matrix Size and Stimulus Type Effects As we expected, stimulus type SC was selected for online test for all ten subjects who finished both RC and SC sub-phase during calibration. Paired t-test revealed significant faster for SC to reach maximum WSR, t(9)=2.25, p=0.04. In addition, accuracy for 8x8 RC, 3x3 RC and 3x3 SC was analysed by a oneway Analysis of Variance (ANOVA) and post-hoc test, significant higher accuracy for 3x3 SC compare to 8x8 RC (p=0.038) and 3x3 RC (p= 0.047) was reported. Howerver, no significant effect was observed in ITR. To test whether P300 amplitude and latency differed across paradigms, we subjected the data to a two-way ANOVA with factors Paradigm (8x8 RC, 3x3 RC, 3x3 SC) and Electrode (Fz, Cz, Pz). Significant larger P300 amplitude for 3x3 SC compared with 8x8 RC, 3x3 RC was revealed, p<0.01. while significant larger P300 latency for 8x8 RC over other two, p < 0.05.

X = {x1 , x 2 ,..., x n }, C = {c1 , c 2 ,..., c n } (1) Likelihood L( X | ci ) for each stimulus can be simply obtained by calculating the output f ( xi ) of a
classification (or regression) program pre-trained by labeled epochs from calibration phase. So the target stimulus is indicated by:

i = arg max{L( X | ci )} = arg max{ f ( X i )} (2)

i i

However the prior probability of each stimulus has never been considered. Due to a statistical language model (here, PPM) was incorporated in stimulus paradigm, the prior probability can be easily obtained:

p (ci ) = PPPM (ci | prefix)

Then, Bayes fusion is very straightforward:

(3)

P (c i | X ) =

p (c i ) L ( X | c i ) P( X )

(4)

Where P ( X ) is independent with i , so it can be omitted:

P (c i | X ) p ( c i ) L ( X | c i )

(5)

However, current study applied Step-wised Linear Discriminant Analysis (SWLDA) which performed excellently for online classification, (more details in [9]) and is widely used. SWLDA can be seemed as Feature Selection plus Fishers Linear Discriminant (FLD), so the output is the margin distance to the discriminant plane (negative is possible). The margin

4.2 Online Accuracy and Information Transfer Rate As shown in Fig.3, online accuracy of 3x3 TLP (p=0.0003) and 3x3 SC (p=0.0005) was significantly higher than 8x8 RC. Online ITR of 3x3 TLP outperformed both 3x3 SC (p=0.005) and 8x8 RC (p=0.0005). 4.3 Time to Complete and Operational Behavior In order to complete the spelling task, a breakcontinue rule was made that if a subject could not
Cz
6

separately for each electrode, no significant effect was found (Fig.4). Such result was just expected by our design. Because P300 amplitude and latency also reflect mental workload. The result shown here means there is no significant difference in workload during switching among step-1, -2, -p so that spelling workload is smooth although prediction involved in step-p. 4.5 Offline Comparision between Bayes fusion and
O1
6 4 2 0 Target Non-target 8x8 RC 3x3 RC 3x3 SC

Amplitude [muV]

4 2 0 -2

-2 -4 0 100 200 300 400 500 600 700 800

Time [s]

100

200

300

400

500

600

700

800

Fig.2 Grand mean waveform of Target (solid) and Non-target (dashed) responses for 8x8 RC (black), 3x3 RC (cyan) and 3x3 SC (magenta) just over 10 subjects completed 3x3 RC session at electrode Cz and O1. correctly select an target item in five consequent selections, the investigator would break the current spelling, correct all error and continue the task after the subject rest enough. Two subjects couldnt finish the task under conventional 8x8 RC paradigm while they both 10 2
8 R x8 C finished under proposed 3x3 TLP. Exclude these two

pure SWLDA Offline accuracy for Bayes fusion proposed in current study was calculated using online 3x3 TLP data for each of 14 subjects. Paried t-test was conducted between SWLDA (88.461.09) and Bayes fusion (89.111.05), p=0.1719. Although no significant increase, 6 out of 14 subjects got better accuracy and 5 of rest kept accuracy unchanged. 5. Conclusion The proposed 3x3 TLP outperformed 8x8 conventional one in online accuracy, speed and ITR. Moreover, no significant different mental demanding among step-1, -2, -p was observed. References [1] Wolpaw et al, "Braincomputer interfaces for
7 8 9 communication p<0 0 .0 1

Online Accuracy [%]

subjects, we
8 0

10 0

3 S x3 C 3 a paired x3 L conducted T P

t-test to compare

consumed time between 8x8 RC and 3x3 TLP, as expected, significant faster for 3x3 TLP was revealed,
6 t(11)=3.97, 0 p=0.002. Meanwhile, significant less

break amount was found for 3x3 TLP, t(11)=2.40, p=0.04.

4 0

4.4 Wave 20 Morphology (Workload) for step-1, -2, -p One-way repeated ANOVA revealed no significant
1 2 3 5 6 difference for online accuracy bwteen4step-1, -2, -p, 0

1 and 1 0 1 1 2 1 3 1 Clinical M a 4 en control", S b c [# u je t ] F(2, 26)=2.25, p=0.12 (Fig.4). To investigate deeper 3x3 RC (light blue) Vol.113, (2002) pp.767-791. subjects and Fig.3 Online Accuracy of 8x8 RC (white), Neurophysiology, and 3x3 TLP (blue) for each Mean (with standard error ANOVA was reason, sample-wised one-way repeated bar). Significant higher accuracies for 3x3 SC off the top of your head:see text [2] Farwell et al, "Talking and TLP were achieved, for details. conducted for target and non-target ERP amplitude toward a mental prosthesis utilizing event-related

brain potentials", Clinical Neurophysiology, vol.70, (1988) pp.510-523. [3] Fazel-Rezai, "Human Error in P300 Speller Paradigm for Brain-Computer", Proceedings of the 29th Annual International Conference of the IEEE EMBS (2007). [4] Brunner et al, "Does the P300 speller depend on
Online Accuracy [%]

eye gaze?, J. Neural Eng., Vol.7, (2010) pp.1-9. 6

Amplitude [muV]
p > 0.05 100 [5] Polich, "Updating P300: An integrative theory of

Cz
6 4 2 0 -2 400 600 800 0 200

O1
1.00 Target Non-target Step-1 Step-2 Step-p 0.05 0.01 0.00 400 600 800

P3a and P3b", Clinical Neurophysiology, Vol.118, (2007) pp.21282148.

40 0 Correcting 60 2 0 [6]20 Hill et al, "Effects of Stimulus Type and of Error1

80

Code

Design p

on

Step [#] Performance", NIPS, (2008).

BCI Target -2 Speller Non-target 0 200

[7] Martens Online Accuracy forrefractory effects in a (light blue) and step-p (blue) averaging over all 14 subjects; Fig.4 Left: et al, "Overlap and step-1 (white), step-2 Center and Right: Grand speller based on the waveforms for step-1, 2, p of Target (solid) and Non-target (dashed) at brain-computer interface mean (all 14 subjects) visual electrodes Cz and O1. Sample-wised p-values are coded by color. No significance was revealed. P300 event-related potential", J. Neural Eng., Vol.6, (2009) pp.1-9. [8] Cleary et al, "Data Compression Using Adaptive Coding and Partial String Matching", IEEE Transactions of communications, Vol.COM-32(4), (1984) pp.396-402. [9] Krusienski et al, "A Comparison of Classification Techniques for the P300 Speller", J. Neural Eng., Vol.3, (2006) pp.299-305. [10] Furdea, et al, "An auditory oddball (P300) spelling system for brain-computer interfaces", Psychophysiology, Vol.46 (2009), pp.1-9.

Time [s]