Regression Modelling
Simple Linear Models
Lecture 1
Karanjah Anthony Ph.D.
Multimedia University
FoST-Department of Mathematics
2022@tkaranjah
Overview
Many problems in sciences involve a
study or analysis of the relationship
between two or more variables.
For example;
◼ Income and number of years of education
◼ Height and weight of people
◼ the pressure of a gas in a container is related
to the temperature,
◼ Dose of a drug and response
2022@tkaranjah
Overview
In both of these examples, the value of the
response of interest y (energy consumption,
fuel usage) cannot be predicted perfectly from
knowledge of the corresponding x.
It is possible for different automobiles to have
different fuel usage even if they weigh the
same, and it is possible for different houses to
use different amounts of electricity even if they
are the same size.
2022@tkaranjah
1. Introduction
2. Assumptions
3. Parameter Estimation
1 2022@tkaranjah 2
Overview
However, in many situations, the relationship
between variables is not deterministic.
For example, the electrical energy consumption
of a house (y) is related to the size of the house
(x), in square feet), but it is unlikely to be a
deterministic relationship.
Similarly, the fuel usage of an automobile (y) is
related to the vehicle weight (x), but the
relationship is not a deterministic one.
3 2022@tkaranjah 4
Overview
The collection of statistical tools that are used
to model and explore relationships between
variables that are related in a non-deterministic
manner is called regression analysis.
Because problems of this type occur so
frequently in many branches of science and
engineering, regression analysis is one of the
most widely used statistical tools
5 2022@tkaranjah 6
1
 Overview Simple Linear Regression
For example, in a chemical process, suppose that The case of simple linear regression considers a
the yield of the product is related to the process- single regressor variable or predictor variable x
operating temperature. and a dependent or response variable Y.

Regression analysis can be used to build a model

to predict yield at a given temperature level.
This model can also be used for process
optimization, such as finding the level of
temperature that maximizes yield, or for process
control purposes.
Suppose that the true relationship between Y and
x is a straight line and that the observation Y at
each level of x is a random variable. As noted
previously, the expected value of Y for each
value of x is

2022@tkaranjah 7 2022@tkaranjah 8

Simple Linear Regression Simple Linear Regression

◼ ◼

2022@tkaranjah 9 2022@tkaranjah 10

Simple Linear Regression Simple Linear Regression

◼ ◼

2022@tkaranjah 11 2022@tkaranjah 12

2
Simple Linear Regression
Assumptions Simple Linear Regression
◼

2022@tkaranjah 13 2022@tkaranjah 14

Simple Linear Regression

◼
Simple Linear Regression

2022@tkaranjah 15 2022@tkaranjah 16

Simple Linear Regression Simple Linear Regression

an and the sum of the squares of the deviations of
the observations from the true regression line is

2022@tkaranjah 17 2022@tkaranjah 18

3
 Simple Linear Regression Simple Linear Regression
The least squares estimates of the intercept and
slope in the simple linear regression model are;

2022@tkaranjah 19 2022@tkaranjah 20

Simple Linear Regression Simple Linear Regression

Later, we will use the residuals to provide
information about the adequacy of the fitted
model.

Deviations of the data from the estimated

regression model are can be represented
diagrammatically as follows

2022@tkaranjah 21 2022@tkaranjah 22

Simple Linear Regression Simple Linear Regression

Notationally, it is occasionally convenient to give
special symbols to the numerator and
denominator of above Equation.
Example
For example, in a chemical process, suppose that
the yield of the product y is related to the
process-operating temperature x.
Specifically the data in the table below table, y is
the purity of oxygen produced in a chemical
distillation process, and x is the percentage of
hydrocarbons present in the main condenser of
the distillation unit.

Required: Find the parameter estimates.

2022@tkaranjah 23 2022@tkaranjah 24

4
 Simple Linear Regression Simple Linear Regression

2022@tkaranjah 25 2022@tkaranjah 26

Simple Linear Regression Simple Linear Regression

We will fit a simple linear regression model to the
oxygen purity data in Table above.
The following quantities may be computed;

2022@tkaranjah 27 2022@tkaranjah 28

Simple Linear Regression Simple Linear Regression

Therefore, the least squares estimates of the This model is plotted in Fig. below, along with the
slope and intercept are; sample data

The fitted simple linear regression model (with the

coefficients reported to three decimal places) is;

2022@tkaranjah 29 2022@tkaranjah 30

5
 Simple Linear Regression Simple Linear Regression
Practical Interpretation:

The value of which 14.9475 is interpreted as follows:

When when the hydrocarbon level changes by1.00% then purity

increases by 14.9475 units.

2022@tkaranjah 31 2022@tkaranjah 32

Simple Linear Regression

Practical Interpretation:

What is the interpretation of ?

END

2022@tkaranjah 33 2022@tkaranjah 34