Download as pdf or txt
Download as pdf or txt
You are on page 1of 37

EN112 - Probability and statistics 2

Department of Mathematics and Computer Science


The Papua New Guinea University of Technology

Dr Dunstan
Room MCS213 Mathematics and Computer Science Building
- PNG Unitech, Lae

Semester 1, 2023

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 1 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Statistics

(Kreyszig 8th Ed, 2009) pg 1104


Recall the process of statistical inference from models:
sample mean,

n
1 1
x= ∑ x j = n (x1 + x2 + ... + xn )
n j=1

Where n is the sample size. the sample variance s2 is,

n n
1 1 1
s2 = ∑ xj = ∑ (x1 − x)2 = n − 1 [(x j − x)2 + ... + (xn − x)]
n−1 j=1 n−1 j=1

And the the positive square root of the variance, s, is the sample
standard deviation.

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 2 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Estimation of parameters

(Kreyszig 8th Ed, 2009) pg 1106

As an approximation of the mean µ of the population we may take the


mean x of a corresponding sample, that is,

1
µ = x = (x1 + x2 + ... + xn ) (1)
n
Where n is the sample size. Similarly an estimate σ̂ 2 for the variance of
a population is the variance s2 of a corresponding sample, that is,

n n
1 1
σ̂ 2 = s2 = ∑ (x j − x)2 = n − 1 ∑ (x j − x)2 (2)
n−1 j=1 j=1

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 3 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Estimation of parameters

(Kreyszig 8th Ed, 2009) pg 1106


(1) is a special case of the so-called method of moments. In this
method the parameters to be estimated are expressed in terms of the
moments of the distribution. In the resulting formulas, these moments
are replaced by the corresponding moments of the sample. This gives
the estimates. Here the kth moment of a sample x1 , ..., xn , is

n
1
mk =
n ∑ xkj
j=1

Maximum likelihood method (Kreyszig pg 1107).


We consider a discrete or continuous random variable X whose
probability density function f (x) depends on a single parameter θ . We
take a corresponding sample of n independent values x1 , · · · xn . Then in
the discrete case the probability that a sample of size n consists of
precisely those n values is l = f (x1 ) f (x2 ) · · · f (xn ).

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 4 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Maximum likelihood method

(Kreyszig 8th Ed, 2009) pg 1107

In the continuous case the probability that the sample consists of


values in the small intervals x j ≦ x ≦ x j + ∆x( j = 1, 2, · · · , n) is
f (x1 ) f (x2 ) · · · f (xn )∆x = l(∆x)n .
Since f (x j ) depends on θ , the function l depends on x1 , · · · , xn and θ .
We imagine x1 , · · · , xn to be given and fixed. Then l is a function of θ
which is called the likelihood function.
We choose that approximation for the unknown value of θ for which l
is as large as possible.
If l is a differentiable function of θ a necessary condition for l to have a
maximum in an interval (not at the boundary) is ∂∂θl = 0 (3).
We write a partial derivative because l depends also on x1 , · · · xn .
A solution to this depending on x1 , · · · , xn is called a maximum
likelihood estimate for θ .

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 5 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Maximum likelihood method

(Kreyszig 8th Ed, 2009) pg 1107


∂ ln l
We may replace (3) by, ∂θ = 0 (4).
because f (x j ) > 0, a maximum of l is in general positive, and ln l is a
monotone increasing function of l. This often simplifies the
calculations.
If the distribution of X involves several r parameters θ1 , · · · , θr , then
instead of (3) we have the r conditions ∂ l/∂ θ1 = 0, · · · , ∂ l/∂ θr = 0,
and instead of (4) we have, ∂∂ ln l ∂ ln l
θ1 = 0 · · · , ∂ θr = 0.

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 6 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Maximum likelihood method - example


(Kreyszig 8th Ed, 2009) pg 1108
Find the maximum likelihood estimates for µ and σ in the case of the
normal distribution.
From the normal distribution density equation, we obtain the likelihood
function,
 n 
1 n −h
l = √12π σ e where h = 2σ1 2 ∑nj=1 (x j − µ)2 .

Taking logarithms we have, ln l = −n ln 2π − n ln σ − h
Use the first equation above ∂ ln l/∂ µ = 0, written out,
∂ ln l 1
∂µ = − ∂∂ µh = σ2 ∑nj=1 (x j − µ) = 0, hence ∑nj=1 x j − nµ = 0.
The solution is the desired estimate µ̂ for µ; we find, µ̂ = n1 ∑nj=1 x j = x̄.
the second equation above is ∂ ln l/∂ σ = 0, written out
∂ ln l
∂σ = − σn − ∂∂σh = − σn + σ13 ∑nj=1 (x j − µ)2 = 0.
Replacing µ by µ̂ and solving for σ 2 , we obtain the estimate
σ̃ 2 = 1n ∑nj=1 (x j − x̄)2
EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 7 / 37
EN112 Engineering Mathematics I - Mathematical statistics

Confidence intervals

(Kreyszig 8th Ed, 2009) pg 1109


Confidence intervals for an unknown parameter θ of some distribution
(eg: θ = µ), are intervals θ1 ≦ θ ≦ θ2 , that contain θ not with
certainty, but with a high probability γ, which we can choose (95% and
99% are popular).
Such an interval are calculated from a sample.
γ = 95% means probability 1 − γ = 5% = 1/20 of being wrong - one of
about 20 such intervals will not contain θ .
Instead of writing θ1 ≦ θ ≦ θ2 , we denote this more distinctly by
writing,
CONFγ {θ1 ≦ θ ≦ θ2 },
where γ is called the confidence level, and θ1 and θ2 are the lower and
upper confidence limits. They depend on γ. The larger we choose γ the
smaller the error probability 1 − γ is, but the longer the confidence
interval will be; if γ → 1, then its length goes to infinity.

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 8 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Confidence intervals
(Kreyszig 8th Ed, 2009) pg 1110
θ1 and θ2 in (3) are calculated from a sample x1 , · · · , xn . These are n
observations of the random variable X. We can regard x1 , · · · , xn as a
single observations of n random variables, X1 , · · · , Xn (with the same
distribution, namely that two random variables Θ1 = Θ1 (X1 , · · · , Xn )
and Θ2 = Θ2 (X1 , · · · Xn ).
The condition involving γ can now be written, P(Θ1 ≦ θ ≦ Θ2 ) = γ.
Determination of a confidence interval for the mean µ of a normal
distribution with known variance σ 2 (TABLE 23.1):
1st step: Choose a confidence level γ (95%, 99%, or the like).
2nd step: Determine the corresponding c:
γ 0.90 0.95 0.99 0.999
c 1.645 1.960 2.576 3.291
3rd step: Compute the mean x of the sample x1 , ...xn .

4th step: Compute k = cσ / n. The confidence interval for µ is
CONFγ [x − k ≦ µ ≦ x + k]
EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 9 / 37
EN112 Engineering Mathematics I - Mathematical statistics

Confidence intervals - normal distribution with known variance

(Kreyszig 8th Ed, 2009) pg 1110


Determination of a confidence interval for the mean µ of a normal
distribution with known variance σ 2 (TABLE 23.1) - Example:
Determine a 95% confidence interval for the mean of a normal
distribution with variance σ 2 = 9, using a sample of n = 100 values
with mean x̄ = 5.
1st step: γ = 0.95 is required.
2nd step: The corresponding c equals 1.960 See Table 23.1:
3rd step: x̄ = 5 is given.

4th step: We need k = 1.960 · 3/ 100 = 0.588. Hence x̄ − k = 4.412
x̄ + k = 5.588 and the confidence interval is
CONF0.95 {4.412 ≦ µ ≦ 5.588}.

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 10 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Theory for Table 23.1

(Kreyszig 8th Ed, 2009) pg 1110


THEOREM: Sum of independent normal random variables;
Let X1 + · · · + Xn is normal with mean nµ and variance nσ 2 .
The following random variable X̄ is normal with mean µ and variance
σ 2 /n.
X̄ = n1 (X1 + · · · + Xn ).
The following random variable Z is normal with mean 0 and variance 1
Z = σX̄−µ
√ .
/ n

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 11 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Theory for Table 23.1 Example 2


(Kreyszig 8th Ed, 2009) pg 1111
Sample size needed for a confidence interval of prescribed length; How
large must n be in Example 1 if we want to obtain a 95% confidence
interval of length L = 0.4?

The confidence interval has the length L = 2k = 2xσ / n. Solving for n
we obtain n = (2cσ /L)2 . In the present case
n = (2 · 1.960 · 3/0.4)2 ≈ 870
The figure below shows the length of the confidence interval (in
multiples of σ ) as a function of the sample size n for γ = 95% and
γ = 99%

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 12 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Confidence intervals - normal distribution with unknown variance

(Kreyszig 8th Ed, 2009) pg 1110

In this case k differs from that in Table 23.1 and c now depends on n
and must be determined from Kreyszig′ s Table A9 in Appendix 5. That
table contains values z corresponding to given values of the (CDF)
Rz  2 −(m+1)/2

distribution function, F(z) = Km −∞ 1 + um du
This is the so-called t-distribution.
Here m(= 1, 2, · · · ) is a parameter, called the number of degrees of
freedom of the distribution.
The constant Km is such that F(∞) = 1.
 √
By integration it turns out that Km = Γ 12 m + 21 / mπΓ 12 m ,

R ∞ −t α−1
where Γ is the gamma function, given as Γ(α) = 0 e t dt (α > 0),
which is meaningful only if α > 0 (or, if we consider complex α, for
those α whose real part is positive). Read more in Kreyszig′ s Appendix
3 pg A54 and A55 (8th Edition).

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 13 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Confidence intervals - normal distribution with unknown variance

(Kreyszig 8th Ed, 2009) pg 1112

TABLE 23.2:
1st step: Choose a confidence level γ (95%, 99%, or the like).
2nd step: Determine the solution c of the equation:
1
F(c) = (1 + γ)
2
From the table of the t-distribution with n − 1 degrees of freedom.
(Table A9 n Appendix 5, n = sample size).
3rd step: Compute the mean x & the variance s2 of the sample
x1 , ..., xn .

4th step: Compute k = sc/ n. The confidence interval is
CONFγ [x − k ≦ µ ≦ x + k].

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 14 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Confidence intervals - normal distribution with unknown variance - example


(Kreyszig 8th Ed, 2009) pg 1113
Five independent measurements of the flash point (◦F) of Diesel oil
(D − 2) gave the values, 144 147 172 144. Assuming normality,
determine a 99%-confidence interval for the mean.
Solution: 1st Step. γ = 0.99 is required.
2nd Step. F(c) = 12 (1 + γ) = 0.995, and Table A9 in Appendix 5 with
n − 1 = 4 degrees of freedom gives c = 4.60.
3rd Step. x̄ = 144.6 s2 = 3.8
√ √
4th Step. k = 3.8 · 4.60/ 5 = 4.01. The confidence interval is
CONF0.99 {140.5 ≦ µ ≦ 148, 7}.
If the variance σ 2 were known and equal to the sample variance s2 ,
thus σ 2 √
= 3.8, then√Table√23.1 would give,
k = cσ / n = 2.576 3.8/ 5 = 2.25 and
CONF0.99 {142.35 ≦ µ ≦ 146.85}.
The length is 4.5 whearas the previous interval is almost twice as long,
8.1. Hence for small samples the difference is considerable.
EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 15 / 37
EN112 Engineering Mathematics I - Mathematical statistics

Table 23.2 example plots

(Kreyszig 8th Ed, 2009) pg 1113


Ratio of the lengths L′ and L of the confidence intervals of the normal
distribution with unknown variance to the normal distribution with
known variance, with γ = 95% and γ = 99% as a function of sample
size n for equal s and σ

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 16 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Theory for Table 23.2


(Kreyszig 8th Ed, 2009) pg 1114
For deriving CONFγ [x − k ≦ µ ≦ x + k], in Table 23.2, we need,
THEOREM: Let X1 · · · Xn be independent normal random variables with
the same mean µ and the same variance σ 2 . Then the random variable
X̄−µ
√ has a t-distribution with n − 1 degrees of freedom; here X̄ is
T = S/ n
1
the same as that for Table 23.1, and S2 = ∑nj=1 (X j − X̄)2 . n−1
We choose a number γ between 0 and 1 and determine a number c
from Kreyszig′ s Table A9 Appendix 5 (8th Edition), with n − 1 degrees
of freedom such that, P(−c ≦ T ≦ c) = F(c) − F(−c) = γ.
Since the t-distribution is symmetric, we have F(−c) = 1 − F(c), and
the above equation assumes the form F(c) = 12 (1 + γ).
Substituting the above and transforming the
√ result as before, we obtain
P(X̄ − K ≦ µ ≦ X̄ + K) = γ where K = cS/ n.
By inserting the observed values x̄ of X̄ and s2 of S2 into the above we
obtain the confidence interval of Table 23.2.
EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 17 / 37
EN112 Engineering Mathematics I - Mathematical statistics

t-distribution

(Kreyszig 10th Ed, 2009) pg 1072


The CDF and PDF of the t-distribution are shown below, left and right
respectively.

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 18 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Confidence intervals - for variance of the normal distribution whose mean


may not be known
(Kreyszig 8th Ed, 2009) pg 1115
Determination of a confidence interval for the variance σ 2 of the
normal distribution whose mean need not be known:
Table 23.3
Solution: 1st step - choose a confidence level γ(95%, 99% or the like).
2nd step - Determine solutions c1 and c2 of the equations,
F(c1 ) = 12 (1 − γ), F(c2 ) = 12 (1 + γ)
From the table of the chi-square distribution with n − 1 degrees of
freedom (Table A10 in Appendix 5; or use a graphing tool, n = sample
size).
3rd step - Compute (n − 1)s2 where s2 is the variance of the sample
x1 , ..., xn .
4th step - Compute k1 = (n − 1)s2 /c1 and k1 = (n − 1)s2 /c2 . The
confidence interval is,
CONFγ {k2 ≦ σ 2 ≦ k1 }
EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 19 / 37
EN112 Engineering Mathematics I - Mathematical statistics

Confidence intervals - normal distribution with possible unknown mean -


example

(Kreyszig 8th Ed, 2009) pg 1114


Determine a 95% confidence interval for the variance, using the sample
(tensile strength of sheet steel in kg/mm2 , rounded to an integer values.
89 84 87 81 89 86 91 90 78 89 87 99 83 89.
Solution, 1st step, γ = 0.95 is required.
2nd Step, For n − 1 = 13 we find c1 = 5.01 and c2 = 24.74.
3rd Step, 13s2 = 326.9.
4th Step, 13s2 /c1 = 65.25, 13s2 /c2 = 13.21. The confidence interval is
CONF0.95 {13.21 ≦ σ 2 ≦ 65.25}

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 20 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Theory for Table 23.3


(Kreyszig 8th Ed, 2009) pg 1115
In Table 23.1 we use the normal distribution, in Table 23.2 the
t-distribution and now we use the χ 2 -distribution.
Two numbers c1 and c2 have to be determined. Both are obtained
from Table A10 in Appendix 5, which contains values of z
corresponding to given values of the χ 2 -distribution (CDF) function
F(z) = 0 if z < 0 and,

 0 if z < 0
z
F(z) =
 cm e−u/2 u(m−2)/2 du if z ≧ 0
R
0
Here (m = 1, 2, ...), is a parameter, calledthe numberof degrees of
freedom of the distribution, and Cm = 1/ 2m/2 Γ 12 m
For deriving the confidence level in table 23.3 we need, under the
assumptions for the t-distribution in the given THEOREM.
2
The random variable, Y = (n − 1) σS 2
With S2 having a chi-square distribution with n − 1 degrees of freedom.
EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 21 / 37
EN112 Engineering Mathematics I - Mathematical statistics

χ 2 -distribution

(Kreyszig 8th Ed, 2009) pg 1115

The CDF and PDF of the χ 2 -distribution are shown below, left and
right respectively. The PDF is obviously not symmetric.

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 22 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Theory for Table 23.3 continued

(Kreyszig 8th Ed, 2009) pg 1116


the derivation of CONFγ {k2 ≦ σ 2 ≦ k1 }, in Table 23.3, is similar to the
previous two.
We choose a number γ between 0 and 1 and determine c1 and c2 from
Kreyszig′ s Table A10, Appendix 5, such that,
P(Y ≦ c1 ) = F(c1 ) = 12 (1 − γ), P(Y ≦ c2 ) = F(c2 ) = 12 (1 + γ).
Subtraction yields, P(c1 ≦ Y ≦ c2 ) = P(Y ≦ c2 ) − P(Y ≦ c1 ) = γ.
Transforming c1 ≦ Y ≦ c2 with Y into an inequality for σ 2 , we obtain
n−1 2 2 n−1 2
c2 S ≦ σ ≦ c1 S .
By inserting the observed value s2 of S2 , we obtain
CONFγ {k2 ≦ σ 2 ≦ k1 }, in Table 23.3.

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 23 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Confidence intervals for other distributions

(Kreyszig 8th Ed, 2009) pg 1116


The previous methods are designed for normal distributions. We now
show that the methods can be applied to other distributions if we use
large samples.
We know that X1 , · · ·n are independent random variables with the same
mean µ and the same variance σ 2 , then their sum Yn = X1 + · · · + Xn
has the following properties,
(A) Yn has the mean nµ and the variance nσ 2 .
(B) If those variables are normal then Yn is normal.
If those variables are not normal then (B) is not applicable. However
for large n the random variable Yn is still approximately normal.
This follows from the Central Limit theorem a fundamental theorem in
probability theory.

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 24 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Central limit theorem

(Kreyszig 8th Ed, 2009) pg 1116


THEOREM Central Limit Theorem:
Let X1 · · · Xn , · · · be independent random variables that have the same
distribution function and therefore the the same mean µ and the same
variance σ 2 . Let Yn = X1 + · · · + Xn . Then the random variable
Zn = Yσn −nµ

n
is asymptotically normal with mean 0 and variance 1:
That is the distribution function Fn (x) of Zn satisfies
R ∞ −u2 /2
limn→∞ Fn (x) = φ (x) = √1 e du
2π −∞
Hence when using Table 23.1 - 23.3, we must use sufficiently large
samples. As a rule of thumb, if the sample indicates that the skewness
of the distribution (the asymmetry) is small, use at least n = 20 for the
mean and at least n = 50 for the variance.

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 25 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Regression analysis
(Kreyszig 8th Ed, 2009) pg 1145
This is concerned with fitting a non-arbitrary straight lines to a cluster
of variables to ascertain trends.
In regression analysis the dependence of Y on x is a dependence of the
mean µ of Y on x so that µ(x) is a function in the ordinary sense. The
curve of µ(x) is called the regression curve of Y on x.
The simplest case is the case of a straight line µ(x) = κ0 + κ1 x
Method of least squares: Gauss least square method. The straight line
should be fitted through the given points so that the sum of the
squares of the distance of these points from the straight line is a
minimum, where the distance is measured in the vertical direction (the
y-direction).
General assumption 1 - the x- values x1 , ..., xn in the sample
(x1 , y1 ), ..., (xn , yn ) are not equal.
From a given sample (x1 , y1 ), ..., (xn , yn ) we can determine a straight
line by least squares.
.
EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 26 / 37
EN112 Engineering Mathematics I - Mathematical statistics

Method of least squared continued

(Kreyszig 8th Ed, 2009) pg 1145


We write the line as y = k0 + k1 x, and call it the sample regression line.
It will be the counterpart of the population regression line µ(x).
Now a sample point (x j , y j ) has the vertical distance (y-direction) from
y given by,
|y j − (k0 + k1 x j )|
Hence the sum of the squares of these distances is,
n
q= ∑ (y j − k0 − k1 x j )2 .
j=1

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 27 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Method of least squared continued

(Kreyszig 8th Ed, 2009) pg 1146


In the method of least squares we now have to determine k0 and k1
such that q is minimum. From calculus we know that a necessary
condition, for this is,
∂q
= 0 and ∂∂kq1 = 0
∂ k0
We see from this condition we obtain for the sample regression line the
formula y − y = k1 (x − x).
Where x and y are the means of the x- & the y-values in our sample.

1
x = (x1 + ... + xn ) and y = 1n (y1 + ... + yn )
n

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 28 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Method of least squared continued

(Kreyszig 8th Ed, 2009) pg 1146

The figure below shows the vertical distance of a point (x j , y j ) from a


straight line y = k0 + k1 x

the slope k1 called the regression coefficient is given by,


sxy
k1 = 2
sx

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 29 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Method of least squared continued

(Kreyszig 8th Ed, 2009) pg 1146


The sample covariance is given by,
" ! !#
n j n n
1 1 1
sxy =
n−1 ∑ (x j − x)(y j − y) = 1 − n ∑ x jy j − n ∑ xi ∑ yj
j=1 j=1 i=1 j=1

and
 !2 
n j n
1 1 1
s2x = ∑ (x j − x)2 = 1 − n  ∑ x2j − n ∑ xj 
n−1 j=1 j=1 i=1

From y − y we see that the sample regression line passes through the
point (x, y), by which it is determined, together with the regression
coefficient k1 .
We may call s2x the variance of the x-values in our sample.
Bearing in mind that x is an ordinary variable, not a random variable.

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 30 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Method of least squared continued


(Kreyszig 8th Ed, 2009) pg 1146
underlineDerivation of y − y0 and k1 .
First we obtain,

∂q
= −2 ∑(y j − k0 − k1 x j ) = 0
∂ k0

∂q
= −2 ∑ x j (y j − k0 − k1 x j ) = 0
∂ k1
Where we sum over j from 1 to n. We now divide by 2. Write each of
the two sums as three sums, & take those containing y j and x j y j over
to the right, to get the “normal equations”,

k0 n + k1 ∑ x j = ∑ y j
(3)
k0 ∑ x j + k1 ∑ x2j = ∑ x j y j .
This is a linear system of two equations in two unknowns k0 and k1 .
EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 31 / 37
EN112 Engineering Mathematics I - Mathematical statistics

Method of least squared continued

(Kreyszig 8th Ed, 2009) pg 1146


The coefficient determinant is,

n ∑ x j 2
2
∑ x j ∑ x2j = n ∑ x j −
∑xj = n(n − 1)s2x = n ∑(x j − x)2

This is not zero because of assumption 1. Hence the system has a


unique solution.
Dividing the first equation of (3) by n and using x and y, we get
k0 = y − k1 x.
Together with y = k0 + k1 x this gives y − y.
To get k1 we solve (3) by Cramers rule, or elimination, finding,
n ∑ x j y j − ∑ xi ∑ y j
k1 =
n(n − 1)s2x

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 32 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Regression line - Example

(Kreyszig 8th Ed, 2009) pg 1148


The decrease of volume y [%] of leather for certain fixed values of high
pressure x [atmospheres] was measured.
The results are shown in the first two columns of of the table
(Kreyszig′ s 8th Edition, Table 23.11) below

Find the regression line of y on x.

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 33 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Regression line - Example continued

(Kreyszig 8th Ed, 2009) pg 1148

Solution: We see that n = 4 and obtain the values x̄ = 28000/4 = 7000


ȳ = 19.0/4 = 4.75, and
 2

s2x = 13 216000000 − 28000
4 = 20000000
3 and
sxy = 13 148400 − 28000·19 = 15400

4 3 .
k1 = 15400/20000000 = 0.00077, and the regression line is
y − 4.75 = 0.00077(x − 7000) or (y = 0.00077x − 0.64).
Note that y(0) = −0.64, which is physically meaningless, but typically
indicates that a linear relation is merely an approximation valid on
some restricted interval.

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 34 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Confidence intervals in regression analysis

(Kreyszig 8th Ed, 2009) pg 1148

Assumption 2: for each fixed x the random variable Y is normal with


mean µ(x) = κ0 + κ1 x and variance σ 2 independent of x.
k1 is the population regression coefficient.
Assumption 3: the n performances of the experiment by which we
obtain a sample, (x1 , y1 ), (x2 , y2 ), ..., (xn , yn ) are independent.
Determination of a confidence interval for κ1 under assumption’s
1 to 3 TABLE 23.12.
1st step - Choose a confidence level γ (95%, 99%, or the like).
2nd step - Determine the solution c of the equation.

1
F(c) = (1 + γ)
2

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 35 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Table 23.12 continued

(Kreyszig 8th Ed, 2009) pg 1149


From the table of the t-distribution with n − 2 degrees of freedom
(Table A9 in Appendix 5 Kreyszig; n = sample size).
3rd step: Using a sample (x1 , y1 ), ..., (xn , yn ), compute (n − 1)s2x ,
(n − 1)sxy , k1 .

!2
n n n
1
(n − 1)s2y = ∑ y2j = ∑ y2j − ∑ yj and qo = (n − 1)(s2y − k12 s2x )
j=1 j=1 n j=1

4th step - compute


r
q0
K=c
(n − 2)(n − 1)s2x
The confidence interval is,
CONFγ {k1 − K ≦ κ1 ≦ k1 + K}

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 36 / 37


EN112 Engineering Mathematics I - Mathematical statistics

Confidence intervals for the regression coefficient

(Kreyszig 8th Ed, 2009) pg 1149

Example: using Table 23.11, determine a confidence interval for κ1 by


the method in Table 23.12.
Solution, 1st Step: We choose γ = 0.95
2nd Step: the equation F(c) = 12 (1 + γ), takes the form F(c) = 0.975
and (Kreyszig′ s) Table A9 in Appendix 5 with n − 2 = 2 degrees of
freedom gives c = 4.30
3rd Step: From Example 1 using the same table above, we have
3s2x = 20000000 and k1 = 0.00077. From Table 23.11 we compute
2
3s2y = 102.2 − 194 = 11.95, q0 = 11.95 − 20000000 · 0.000772 = 0.092.
p
4th Step: We thus obtain K = 4.30 0.092/(2 · 20000000) = 0.000206
and CONF0.95 {0.00056 ≦ κ1 ≦ 0.00098}.

EN112 Engineering Mathematics I Mathematics and Computer Science Semester 1, 2023 37 / 37

You might also like