Handout Lec 10: Statistical Inference by Dr.

Javed Iqbal
Regression analysis: Simple Regression
Regression analysis is very widely used statistical method which aims at analyzing the
relationship between a dependent variable and one or more independent variables. When there is
only one independent variable the regression model is called a simple regression. With more than
one independent variables, the model is called a multiple regression model.
Alternative names of dependent variable are response or target variable, alternative names of
independent variables are explanatory variables, predictors or regressors.
Regression analysis has many applications in business, economics, finance and social and
computing sciences. For example in finance regression is used to estimate the riskiness of a risky
asset and classifying assets as aggressive or defensive investments. Regression is also used to
test asset pricing models e.g. the CAPM. In international finance, regression is used to test
theories of exchange rate determination e.g. to test the PPP theory. In microeconomics regression
is used to estimate the price elasticity and income elasticity of demand and to classify goods as
necessity or luxury. Social scientists use regression to conduct studies on social behavior e.g. to
assess whether academic success is related to socio economic class of students.
Scatterplot or scatter diagram: The first step in a simple regression is to examine visually the
nature or strength of relations between y (dependent variable) and x (independent variable).
Weiss Example 14.3, p-646:Suppose we are interested in estimating a regression model to
explain and predict the price (in hundreds of dollars) of particular make and model of car
(Orion)based on its age in years. Each pair (x,y) is plotted as point in the scatter plot. The scatter
plot indicates a negative and quite strong linear relationship between car’s price and its age.
Least Square Criterion: A method of arriving at the equation of straight line that best describes
the relationship between x and y. This criterion finds the best fitting line as the one which
minimizes the sum of squared residuals.
Regression model in parametric notation: y=β 0 + β 1 x +e

Estimated regression model: ^y =b0 +b1 x

The Least Square Estimators of the slope and intercept are:
S xy
b 1= , b 0= ý−b 1 x́ , where:
Sxx
2 2
(∑ x)( ∑ y ) 2 (∑ x ) 2 (∑ y )
S xy =∑ xy − S xx =∑ x − S yy =∑ y −
n n n
Weiss Example 14.6, p-650: Estimated regression line is:
^y =195.47−20.26 x
Interpretation of Intercept: For a brand new car (0 year age), the estimated price is $19546.
Note that in many of the cases the interpretation of intercept is often meaningless and unreliable
since the sample data may not contain x = 0.
Interpretation of Slope: As the age of car increases by one year, the price decreases by 20.26
hundred dollar or $2026 on average. Thus price depreciates $2026 every year (at least in the 2- to
7-year-old range)
Prediction from the regression: The predicted price of a 3 year old car is:
^y =195.47−20.26 ( 3 )=134.69 (hundreds of dollars)
Extrapolation from the regression model: It is recommended that the regression model is to be
used for predicting y only for the given range of x. To do so outside that range, which is called
extrapolation, may not be reasonable because the linear relationship between the predictor and
response variables may not hold there.

Non-linear regression:
The idea behind finding a regression line is based on the assumption that the data points are
scattered about a line. Sometimes the relationship is not linear. In such cases fitting a linear
equation is disastrous. The scatter plot is good way to judge whether a linear regression is
appropriate.
Anderson Ex 7, p-572:
x: Experience (years) y: sales ($000)
The Least Square Estimators of the slope and intercept are:
S xy
b 1= , b 0= ý−b 1 x́ , where:
Sxx
2 2
(∑ x)( ∑ y ) (∑ x )
2 (∑ y )
2
S xy =∑ xy − S xx =∑ x − S yy =∑ y −
n n n

∑x = 70, ∑y = 1080, ∑ x 2 = 632 ∑ xy = 8128, S xx = 142, S xy = 568

568
b 1= =4 b 0= ý−b 1 x́ = 108-4(7)=80
142
^y =80+4 x

160 Regression of Experience and Sales

140
120
100
Sales

80
60
40
20
0
0 2 4 6 8 10 12 14
Experience (years)