1

SIMPLE REGRESSION
LINEAR REGRESSION

Akhilesh Singh Rathore C-04

Ashish Srivastava C-17
Lt Col KN Sunil Kumar C-23
Lt Col Sanjeev Sharma C-27
Anurag Shukla C-09
Tejasvini Augustaya C-64
 2
Regression

 Relation between variables where changes in some variables may “explain” or

possibly “cause” changes in other variables.
 Explanatory variables are termed the independent variables and the variables to
be explained are termed the dependent variables.
 Regression model estimates the nature of the relationship between the
independent and dependent variables.
 Change in dependent variables that results from changes in independent
variables, ie. size of the relationship.
 Strength of the relationship.
 Statistical significance of the relationship.
 3
Examples

 Dependent variable is retail price of gasoline in Regina – independent variable is

the price of crude oil.
 Dependent variable is employment income – independent variables might be
hours of work, education, occupation, sex, age, region, years of experience,
unionization status, etc.
 Price of a product and quantity produced or sold:
 Quantity sold affected by price. Dependent variable is quantity of product
sold – independent variable is price.
 Price affected by quantity offered for sale. Dependent variable is price –
independent variable is quantity sold.
 4
Linear Regression

 x is the independent variable

 y is the dependent variable
 The regression model is
y   0  1 x  
 The model has two variables, the independent or explanatory variable, x, and the
dependent variable y, the variable whose variation is to be explained.
 The relationship between x and y is a linear or straight line relationship.
 Two parameters to estimate – the slope of the line β1 and the y-intercept β0
(where the line crosses the vertical axis).
 ε is the unexplained, random, or error component. Much more on this later.
 5
Scatter Plot

1. Plot of All (Xi, Yi) Pairs

SCATTER PLOT
2. Suggests How Well
Model Will Fit 160

140
Residual
Wt (X) Blood pressure(Y)
120
60 120
70 125 100
Regression line or best
80 128 fit line Y=a+bX
80
90 130
100 140 60 a
120 142
130 145 40 SLOPE b=(Y2-Y1)/(X2-X1)
140 145
20
150 148
155 150 0
40 60 80 100 120 140 160 180
 6
Assumptions

 Linear regression assumes that

 1. The relationship between X and Y is linear
 2. Y is distributed normally at each value of X
 3. The variance of Y at every value of X is the same (homogeneity of
variances)
 4. The observations are independent
 7
Limitations

 Linear Regression Is Limited to Linear Relationships

 Linear Regression Only Looks at the Mean of the Dependent Variable
 Linear Regression Is Sensitive to Outliers
 Data Must Be Independent