LINEAR ALGEBRA

IN DATA
ANALYSIS
What do you see when you look at the image above?

flower, leaves -not too difficult.

Write that logic so that a computer can do the same for

you – it will be a very difficult task (to say the least).

This is achieved by storing the pixel intensities in a

construct called Matrix. Then, this matrix can be
processed to identify colours etc.
Representation of problems in Linear
Algebra
• Suppose that price of 1 ball & 2 bat or 2 ball and
1 bat is 100 units. Find price of a ball and a bat.
• 2x + y = 100 ...........(1)
• x + 2y = 100 ..............(2)
• (50,0) , (0,100), (100/3,100/3) and (30,40) satisfy our  equation (1) .
• (0,50),(100,0),(100/3,100/3) are some of the points that satisfy equation (2).
• Now in this situation, both of the conditions to be satisfied i.e. the point which lies
on both the lines. Intuitively, the intersection point of both the lines is the solution.
Visualise the problem
2x + y = 100 .............(1)
x + 2y = 100 ..........(2)
from equation (1)-
y = (100- x)/2
put value of y in equation (2)-
x + 2*(100-x)/2 = 100......(3)
Now, since the equation (3) is an
equation in single variable x, it can be
solved for x and subsequently y.
complicate the problem by increasing
variables
x+y+z=1.......(4)
2x+y=1......(5)
5x+3y+2z=4.......(6)
From equation (4),
z=1-x-y....(7)
Substituting value of z in equation (6),
5x+3y+2(1-x-y)=4
3x+y=2.....(8)
solve equations (8) and (5) as a case of two variables
to find the values of ‘x’ and ‘y’ in the problem of bat and ball.
Substitute ‘x’ and ‘y’, in (7) to find the value of ‘z’.
Conclusion : Matrix is used to solve a large set of linear equations
Planes

• A linear equation in 3 variables represents a plane which is a flat

geometric object which extends up to infinity.
• As in the case of a line, finding solutions to 3 variables linear
equation means to find the intersection of those planes.
• Imagine, in how many ways a set of three planes can intersect?
There are 4 possible cases –
1. No intersection at all.
2. Planes intersect in a line.
3. They can intersect in a plane.
4. All the three planes intersect at a point.
• Can you imagine the number of solutions in each case?
Visualisation
1. No intersection at all.
2. Planes intersect in a
line.
3. They can intersect in
a plane.
4. All the three planes
intersect at a point.
• Can you imagine the
number of solutions
in each case?

3 / 4D itself is so difficult / impossible to visualise

How 10000D
So, how do mathematicians deal with higher dimensional data so efficiently?
Matrices is one such trick employed by mathematicians to deal with higher dimensional data.
Terms related to Matrix
• Order of matrix – If a matrix has 3 rows and 4 columns, order of the matrix is 3*4 i.e.
row*column.
• Square matrix – The matrix in which the number of rows is equal to the number of columns.
• Diagonal matrix – A matrix with all the non-diagonal elements equal to 0 is called a diagonal
matrix.
• Upper triangular matrix – Square matrix with all the elements below diagonal equal to 0.
• Lower triangular matrix – Square matrix with all the elements above the diagonal equal to 0.
• Scalar matrix – Square matrix with all the diagonal elements equal to some constant k.
• Identity matrix – Square matrix with all the diagonal elements equal to 1 and all the
non-diagonal elements equal to 0.
• Column matrix –  The matrix which consists of only 1 column. Sometimes, it is used to represent
a vector.
• Row matrix –  A matrix consisting only of row.
• Trace – It is the sum of all the diagonal elements of a square matrix.
Basic operations on matrix

• Addition
• Scalar Multiplication
• Transposition
• Matrix Multiplication
Linear Regression
•Y — The dependent variable (Target prediction value)
•X — Independent variable(s)
•β — Regression coefficients: These refer to the relationship between the x variable
and the dependent variable y.
•ε — Error term: used to denote the a residual variable produced by a statistical or
mathematical model, which is created when the model does not fully represent the
actual relationship between the independent variables and the dependent variables.
As a result of this incomplete relationship, the error term is the amount at which the
equation may differ during empirical analysis.
The job of a linear regression model is essentially to find a linear relationship
between the input (X) and output (y).
Case study
• It describes the different variables of different baseball teams to predict whether it makes to
playoffs or not. But for right now to make it a regression problem, suppose we are interested in
predicting OOBP from the rest of the variables. So, ‘OOBP’ is our target variable. To solve this
problem using linear regression, we have to find parameter vector.

To find the final parameter vector(θ) assuming initial function parameterised by θ and X
, find the inverse of (XT X)
f θ (X)= θT X, where θ is the parameter to be calculated and X is the column vector of
features or independent variables.
 The main attributes that we need to be
concerned with are
.RS — Runs Scored
.RA — Runs Allowed
.W — Wins
.OBP — On Base Percentage
.SLG — Slugging Percentage
.BA — Batting Average
.Playoffs — Whether a team made it to playoffs or not
.OOBP — Opponentʼs On Base Percentage
.OSLG — Opponents Slugging Percentage

1. OBP — On Base Percentage: This is a statistic measuring how frequently a batter

reaches base, it does not take into account how far the batter runs i.e. third base.
2. SLG — Slugging Percentage: A measure of the batting ʻProductivityʼ, in simple
terms, it measures how far a batter makes it when they hit the ball.
3. BA — Batting Average: Defined by the number of hits divided by at bats,
measures how likely a batter is to hit a ball when pitched. Again, does not take
into account how far the batter runs.
Eigenvalues and Eigenvectors
• Applications in computer vision, physics and machine learning.
• In Principal component analysis algorithm, the concept of Eigenvectors is the
backbone.
• 2 types of vectors marked in red and yellow and the picture is showing the change in
vectors after a linear transformation.
• Note that on applying a linear transformation to yellow coloured vector, its direction
changes but the direction of the red coloured vector doesn’t change even after
applying the linear transformation.
• The vector coloured in red is an example of Eigenvector.
• Eigen vectors are only SCALED not changed
Use of Eigenvectors in Data Science
• In a high Dimensional (N) featured data, it is possible that there are
redundant features in that data, causing reduced efficiency and more disk
space.
• PCA algorithm
• Standardizing data by subtracting the mean and dividing by the standard deviation
• Calculate the Covariance matrix.
• Calculate eigenvalues and eigenvectors
• Merge the eigenvectors into a matrix and apply it to the data. This rotates and scales the
data. The principal components are now aligned with the axes of our features.
• Keep the new features which account for the most variation and discard the rest.
Linear Regression

•Y — The dependent variable (Target prediction value)

•X — Independent variable(s)
•β — Regression coefficients: These refer to the relationship between the x variable
and the dependent variable y.
•ε — Error term: used to denote the a residual variable produced by a statistical or
mathematical model, which is created when the model does not fully represent the
actual relationship between the independent variables and the dependent variables.
As a result of this incomplete relationship, the error term is the amount at which the
equation may differ during empirical analysis.
CASE STUDY - SPORTS

• IRIS -
https://towardsdatascience.com/eigenvalu
es-and-eigenvectors-378e851bf372
POLYNOMIAL REGRESSION

• fi’s are polynomial terms of the input variables, e.g., f1(x) = x1, f2(x) = x12 etc.
• Example: 2nd order polynomial regression of one predictor variable

Example: 2nd order polynomial regression of two predictor variables

xi is the linear terms, xi 2 is the quadratic term, and x1x2 is called interaction term between x1 and x2.

WEIGHT LOSS – Gym, Yoga, Walking, Jogging, Diet, Surgery, Genetics, etc
HOSPITAL – Gender, Illness, stay, contagious, seriousness, etc
PROs AND CONs

❖ ADVANTAGES OF POLYNOMIAL REGRESSION

• Non-linear relationships between variables can be modelled
• For curve fitting, a large range of different functions are available
• Good for exploration purposes: testing for the presence of curvature and its
inflections.
❖ DISADVANTAGES OF POLYNOMIAL REGRESSION
• Even a single outlier in the data plot can seriously mess up the results.
• PR models are prone to overfitting. If enough parameters are used, can fit anything.
As John von Neumann reportedly said: “with four parameters I can fit an elephant,
with five I can make him wiggle his trunk.”
• As a consequence of the previous, PR models might not generalize well outside of
the data used
regressions that would help in analyzing
and predicting the results accurate.
• Linear Regression - features as independent variables that depend on the target variable.
• Logistic Regression - explain the relationship between the ratio-level independent variable, binary independent variables
• Polynomial Regression
• Stepwise Regression - used to build a predictive model by adding and removing variables. 
• Lasso Regression - The features include a massive number of variables to fit into the model and enough computational challenges
• Regression Analysis - It is a statistics tool that allows analysts to calculate the mathematical relationship between the end result
and its effects. This technique applied for forecasting.

• Multiple Regression -  It is a powerful technique used to predict the unknown values of a variable from the available variables. The
known variables are classified as predictors.
• Ridge regression - shrinks the coefficients towards zero, it introduces some bias. But it can reduce the variance to a great extent which will
result in a better mean-squared error. The amount of shrinkage is controlled by λ which multiplies the ridge penalty. As large λ means more
shrinkage, we can get different coefficient estimates for the different values of λ.
Singular Value Decomposition
• SVD is some sort of generalisation of Eigen value decomposition. 
• SVD is used to remove the redundant features in a data set.
• Running an algorithm on the original data set ( of 1000 features..) will be time inefficient and
will require a lot of memory.
• Convert Colour Image to BW image based on Pixel Intensity
• different images, correspond to different ranks with different resolution. Higher the rank
implies the larger amount of information about pixel intensity.
• But the irony is pretty well image with 20 or 30 ranks instead of 100 or 200 ranks is possible
removing highly redundant data.
• Presence of redundant features causes multi co-linearity in linear regression.
• Also, some features are not significant for the model. Omitting these features helps to find a
better fit of algorithm along with time efficiency and lesser disk space. Singular value
decomposition is used to get rid of the redundant features present in data.
SVD