Regression Analysis

The Regression Analysis is a technique of studying the dependence of one variable

called the independent variable, on one or more variables called the dependent
variable, with a view to estimate or predict the value of the dependent variables in
terms of the known or fixed values of the independent variables.
 Regression Analysis
Regression Analysis is perhaps the most widely used statistical technique for investigating
or estimating relationship between dependent and a set of independent explanatory
variables. It is also used as a blanket item for a variety of data analysis techniques that are
utilized in a quantitative research method for modeling and analyzing numerous variables.
Regression Analysis is often used to model or analyze data. Majority of analysts use it to
understand the relationship between the variables, which can be further utilized to predict
the precise outcome.
Researchers often use this technique to examine and find a correlation between different
variables of interest. It provides an opportunity to gauge the influence of independent
variables on a dependent variable. Overall, regression analysis is a technique that saves
additional efforts of the researchers in arranging numerous independent variables in tables
and testing or calculating its effect on a dependent variable.
 Regression Analysis

The Use of Regression Analysis is to:

 Estimate the relationship that exists, on the average, between the dependent
variable and the independent variable.

 Determine the effect of each of the independent variables on the dependent

variable, controlling the effects of all the other independent variables.

 Predict the value of the dependent variable for a given value of the
independent variable.
 Regression Analysis

Types of Regression Analysis

Regression Analysis is generally classified into two types: Simple Regression and
Multiple Regression.

Simple Regression involves only two variables, one of which is the dependent
variable and the other is the independent variable. While this model is the easiest to
conceptualize, the model is severely limited in that not often is any criterion variable able
to be fully explained by one predictor variable.

Multiple Regression is an extension of simple regression. It is used when we want to

predict the value of a dependent variable based on the value of two or more independent
variables. Multiple regression encompasses all designs in which multiple predictor
variables are used to explain one dependent or criterion measure.
 Regression Analysis

Linear Regression Analysis

It is one of the most widely known modeling techniques, as it is amongst the first
elite regression analysis methods picked up by people at the time of learning predictive
modeling. The dependent variable is continuous and independent variable is more often
continuous or discrete with a linear regression line.
Please note, in a multiple linear regression, there is more than one independent
variable and in a simple linear regression, there is only one independent variable. Thus,
linear regression is best to be used only when there is a linear relationship between the
independent and dependent variable.
 Regression Analysis

Basic Assumptions for Linear Regression

 The relationship between the independent and dependent variables to be

linear (Linearity).

 The errors between observed and predicted values should be normally

distributed (Normality).

 Observations are independent of each other (Independence)

 The variance of error terms are similar across the values of the independent
variables (Homoscedasticity).
 Regression Analysis

Relationship of Correlation and Regression

Correlation measures the linear relationship that exists between two variables.
Once a correlation has been found between two variables the next logical question to ask
is whether or not that relationship can be used to explain one variable from the other. In
correlation we have no true independent or dependent variable, because we are simply
trying to measure the relationship between the two variables. However, in simple
regression one variable is defined as the dependent or criterion variable and the other
variable is the independent or predictor variable. The knowledge of independent variable
(i.e. PSE index) is used to predict or explain the dependent or criterion variable (MEG
price). Besides being a logical progression from correlation, regression has a great deal
of practical applications for practitioners.
 Regression Analysis

Example 1
As an example of simple regression, consider the correlation analysis about Juan
being an investor, PSE index will be used to predict MEG price. In this model, MEG
price is the criterion or dependent variable and PSE index is the predictor or independent
variable. The regression procedure from Excel Data Analysis will be used to calculate
test statistics.
 Regression Analysis
The Data Set

YEAR PSEI MEG Price YEAR PSEI MEG Price

1991 5671 3.02 2006 7750 4.65
1992 5781 3.07 2007 6789 3.87
1993 6701 3.98 2008 6905 4.20
1994 6789 4.01 2009 7289 4.37
1995 7234 4.45 2010 7890 4.75
1996 6578 3.85 2011 6987 4.13
1997 6567 3.75 2012 7780 4.72
1998 6800 4.03 2013 8450 5.67
1999 7800 4.85 2014 7780 4.78
2000 8200 5.30 2015 8200 5.27
2001 6789 3.93 2016 6589 3.80
2002 7820 4.80 2017 6987 4.10
2003 5789 3.09 2018 7489 4.44
2004 7890 4.77 2019 7654 3.99
2005 8656 5.79 2020 7900 4.90
 Regression Analysis
Using Excel Data Analysis
1. Press data. 2. Presss data
analysis.

4. Press
OK.

3. Look for Regression.

 Regression Analysis
5. Place the cursor
Using Excel Data Analysis on Input Y Range
(MEG price), then
highlight the data
set.
6. Place the cursor
on Input X Range 9. Press OK.
(PSE Index), then
highlight the data
set.
7. Make sure
the
Confidence
Level is at
95%.

8. Select output range

then place the cursor
on any vacant cell.
 Regression Analysis

The Output
 Regression Analysis

Analysis
Researchers may refer to two elements of the Excel Output in the interpretation of regression
analysis, the Significance F and R Square value. The significance F determine whether the
independent variable (PSE index) is a significant predictor of the criterion or dependent variable
(MEG price), The result indicates that the significance F is less than .001 (p<.001), so PSE index is
able to significantly contribute to the prediction of MEG price. R square represents represent the
total amount of dependent score variance that can be explained by the independent or predictor
variables. This means that 94.20% MEG price can be explained by the PSE index. The remaining
5.80% of MEG price is unexplained or due to other factors.
 Regression Analysis

Example 2
A mortgage department of a large bank is studying its recent loans. Of particular
interest is how such factors as the value of the home, age, and mortgage relate to family
income. Are these variables predictors of the income of the household?
Regression Analysis
Regression Analysis
Regression Analysis
Regression Analysis
Regression Analysis
Regression Analysis
Regression Analysis
Regression Analysis
 Regression Analysis
Table 1: Predictors of Household Income
 Regression Analysis

A multiple regression analysis was conducted to determine how well the independent variables
namely: value of home, age and mortgage predicted the dependent variable (family income). The
linear combination of value of home, age and mortgage was significantly related to family income
(𝑅 2 =0.626; p<.001). The value of the 𝑅 2 indicates that approximately 62.6% of the variance of
family income in the sample can be accounted for by the combination value of home, age and
mortgage. Table 1 also presented indices to indicate the relative strength of individual and
combination of predictors. For the individual predictors only value of home was found significant
(𝑅 2 =0.524; p<.001). The combination of value of home and age is also significantly related to
family income (𝑅 2 =0.609; p<.001). The same with the combination of value of home and mortgage
(𝑅 2 =0.560; p<.001). The combination of age and mortgage was found not significant (𝑅 2 =0.052;
p=.217). On the basis of this analyses, it is concluded that the combination of value of home and
age best predict family income.