Feature Extraction

Feature extraction algorithms might be linear or non-linear. Non-linear strategies

assume that the info of interest lie on an embedded non-linear manifold at intervals the
higher-dimensional space. If the manifold is of low enough dimension, we have a
tendency to visualize the info within the low-dimensional area. Thus, we have a
tendency to loosely classify non-linear strategies into two groups, those who offer
mapping and those that provides a visualization. Several of those non-linear spatial
property reduction ways are associated with the linear ways.

3.2.2.1 Principal Component Analysis

PCA is a common feature extraction method based on unsupervised dimension

reduction technique. In Figure 3.3 PCA uses an orthogonal transformation to convert
information into a lower-dimensional area whereas increasing the variance of the
information. The pre-processed dataset is taken as the input for feature extraction
mechanism for which PCA computes a covariance matrix which results in the selection
of optimum principle component. In the two-layer dimension reduction module PCA
transforms the n-dimensional random vector to the new features space consisting of
lower m dimensions. PCA will then sort the eigenvectors in descending order. At the
layer, Imbedded Error (IE) function is used to select the Principal Components (PC).
Cross validation is used to evaluate optimum principals with minimum errors.
Algorithm for PCA

Input: X- Pre-processed dataset

Output: Principal components

Step 1. Organize the data set

Step 2. Write x1…. xn as row vectors, each of which has p columns and place the
row vectors into a single matrix X of dimension n x p

Step 3. Calculate the empirical mean

1
3.1 m= ∑𝑛𝑘=1 𝑥𝑘
𝑛

Step 4. Find the covariance matrix

4.1 ∑𝑥 = ∑𝑛𝑘=1(𝑥𝑘 − 𝑚) (𝑥𝑘 − 𝑚)𝑇
Step 5. Eigenvector-eigenvalue decomposition
5.1 ∑ 𝑣 = λv
Step 6. Choose the principal components using IEF factor analysis measure

𝑙 ∑𝑚
𝑗=𝑙+1 𝜆 1⁄
6.1 IEF(l)= [ ] 2
𝑁𝑚(𝑚−𝑙)

Step 7. Compute the PCA transformed dataset

Linear Discriminant Analysis

LDA is another feature extraction method based on supervised dimension

reduction technique. The idea of LDA in Figure 3.4 given the training sample set, LDA
tries to project the sample onto a line so the projection points of interclass samples are
as close as attainable and therefore the projection points of the intraclass samples are as
far apart as attainable. The output obtained from PCA is further optimized using LDA
to apply the labelled data obtained from samples of PCA transformed the dataset to
achieve a reasonable speed in the proposed intrusion reduction system. There are two
scatter matrices that need to be obtained in LDA, the Between class Scatter matrix (SB)
and the Within class Scatter matrix (SW). The ratio between the projection matrix of
SB and SW is obtained for an ideal transformation of the processed dataset.

Algorithm for LDA

Input: PCA transformed dataset

Output: LDA transformed dataset

Step 1. The first step is to calculate the between-class variance

1.1 𝑆𝐵 = ∑𝑐(𝑢𝑐 − x)(𝑢𝑐 − 𝑥)𝑇

Step 2. Calculate the instance between the mean and the sample of each class
2.1 SW = ∑c ∑iεc(xi -uc )(xi − uc )T
Step 3. Find the mean value of class C
1
3.1 uc = ∑xεci x
ni

Step 4. Compute the ratio of SB an SW

WT
r SB Wr
4.1 J =
WT
r SW Wr

Step 5. Project the LDA transformed dataset.