BST 401 Probability Theory

Xing Qiu Ha Youn Lee

Department of Biostatistics and Computational Biology

University of Rochester

September 9, 2010

Qiu, Lee BST 401

Outline

1 Review of Calculus

Qiu, Lee BST 401

 The many faces of continuity

A real function f (x) : R → R is continuous at point x = a iff

Cauchy: For any number  > 0, there exist δ > 0 such that:

|f (x) − f (a)| < , ∀x ∈ (a − δ, a + δ). (1)

This is also called the -δ definition of limit.

Heine: For any sequence of real numbers (xn ) which converges to
a, we have
lim f (xn ) = f (a). (2)
n→∞

For those of you who have taken general topology, this

type of continuity is called sequential continuity.

Qiu, Lee BST 401

 The many faces of continuity

A real function f (x) : R → R is continuous at point x = a iff

Cauchy: For any number  > 0, there exist δ > 0 such that:

|f (x) − f (a)| < , ∀x ∈ (a − δ, a + δ). (1)

This is also called the -δ definition of limit.

Heine: For any sequence of real numbers (xn ) which converges to
a, we have
lim f (xn ) = f (a). (2)
n→∞

For those of you who have taken general topology, this

type of continuity is called sequential continuity.

Qiu, Lee BST 401

Reduce complexity of a function

f (x) : {a, b, c, d} → R. It is determined by four numbers:

f (a), f (b), f (c), f (d), which is equivalent to R4 .
Suppose f (x) has this property: f (a) = f (b), f (c) = f (d),
then it can be determined by just two numbers: f (a) and
f (d), or say R2 .
In general, a function has B A possible candidates, where A
is the domain of f and B is the range of f .
In general, we certainly want to reduce the complexity of a
function by its mathematical properties.

Qiu, Lee BST 401

Reduce complexity of a function

f (x) : {a, b, c, d} → R. It is determined by four numbers:

f (a), f (b), f (c), f (d), which is equivalent to R4 .
Suppose f (x) has this property: f (a) = f (b), f (c) = f (d),
then it can be determined by just two numbers: f (a) and
f (d), or say R2 .
In general, a function has B A possible candidates, where A
is the domain of f and B is the range of f .
In general, we certainly want to reduce the complexity of a
function by its mathematical properties.

Qiu, Lee BST 401

Reduce complexity of a function

f (x) : {a, b, c, d} → R. It is determined by four numbers:

f (a), f (b), f (c), f (d), which is equivalent to R4 .
Suppose f (x) has this property: f (a) = f (b), f (c) = f (d),
then it can be determined by just two numbers: f (a) and
f (d), or say R2 .
In general, a function has B A possible candidates, where A
is the domain of f and B is the range of f .
In general, we certainly want to reduce the complexity of a
function by its mathematical properties.

Qiu, Lee BST 401

Reduce complexity of a function

f (x) : {a, b, c, d} → R. It is determined by four numbers:

f (a), f (b), f (c), f (d), which is equivalent to R4 .
Suppose f (x) has this property: f (a) = f (b), f (c) = f (d),
then it can be determined by just two numbers: f (a) and
f (d), or say R2 .
In general, a function has B A possible candidates, where A
is the domain of f and B is the range of f .
In general, we certainly want to reduce the complexity of a
function by its mathematical properties.

Qiu, Lee BST 401

Real functions

Notation: countable infinity (ℵ0 ); continuum infinity (ℵ1 ),

Number of possible real valued functions: RR , or
equivalently, ℵ1 different points are required to determine a
real function.
A simple line f (x) = b0 + b1 x is a real function.
It is determined by just two points. So lines are much
easier objects than arbitrary real functions.
What about continuous functions?
Claim: the number of continuous real functions is RQ . I.e.,
ℵ0 points (Q) are enough to determines a continuous real
function f .

Qiu, Lee BST 401

Real functions

Notation: countable infinity (ℵ0 ); continuum infinity (ℵ1 ),

Number of possible real valued functions: RR , or
equivalently, ℵ1 different points are required to determine a
real function.
A simple line f (x) = b0 + b1 x is a real function.
It is determined by just two points. So lines are much
easier objects than arbitrary real functions.
What about continuous functions?
Claim: the number of continuous real functions is RQ . I.e.,
ℵ0 points (Q) are enough to determines a continuous real
function f .

Qiu, Lee BST 401

Real functions

Notation: countable infinity (ℵ0 ); continuum infinity (ℵ1 ),

Number of possible real valued functions: RR , or
equivalently, ℵ1 different points are required to determine a
real function.
A simple line f (x) = b0 + b1 x is a real function.
It is determined by just two points. So lines are much
easier objects than arbitrary real functions.
What about continuous functions?
Claim: the number of continuous real functions is RQ . I.e.,
ℵ0 points (Q) are enough to determines a continuous real
function f .

Qiu, Lee BST 401

Real functions

Notation: countable infinity (ℵ0 ); continuum infinity (ℵ1 ),

Number of possible real valued functions: RR , or
equivalently, ℵ1 different points are required to determine a
real function.
A simple line f (x) = b0 + b1 x is a real function.
It is determined by just two points. So lines are much
easier objects than arbitrary real functions.
What about continuous functions?
Claim: the number of continuous real functions is RQ . I.e.,
ℵ0 points (Q) are enough to determines a continuous real
function f .

Qiu, Lee BST 401

Real functions

Notation: countable infinity (ℵ0 ); continuum infinity (ℵ1 ),

Number of possible real valued functions: RR , or
equivalently, ℵ1 different points are required to determine a
real function.
A simple line f (x) = b0 + b1 x is a real function.
It is determined by just two points. So lines are much
easier objects than arbitrary real functions.
What about continuous functions?
Claim: the number of continuous real functions is RQ . I.e.,
ℵ0 points (Q) are enough to determines a continuous real
function f .

Qiu, Lee BST 401

Real functions

Notation: countable infinity (ℵ0 ); continuum infinity (ℵ1 ),

Number of possible real valued functions: RR , or
equivalently, ℵ1 different points are required to determine a
real function.
A simple line f (x) = b0 + b1 x is a real function.
It is determined by just two points. So lines are much
easier objects than arbitrary real functions.
What about continuous functions?
Claim: the number of continuous real functions is RQ . I.e.,
ℵ0 points (Q) are enough to determines a continuous real
function f .

Qiu, Lee BST 401

Continuity and Approximation
Sketch of proof:
For any a ∈ R, there exists a sequence of rational numbers
q1 , q2 , . . . such that limn→∞ qn = a.
By sequential continuity, f (a) = limn→∞ f (qn ). In other
words, f (q1 ), f (q2 ), . . . determines f (a).
Since every f (a), a ∈ R is determined by some sequence
of rational numbers, collectively, {f (q)} , q ∈ Q determines
all possible values of f .
This theorem, together with Cauchy’s continuity principle,
allows us to approximate a real function with finite many
steps:

∀ > 0, ∃n ∈ N, s.t. |f (qn ) − f (a)| < . (3)

Qiu, Lee BST 401

Continuity and Approximation
Sketch of proof:
For any a ∈ R, there exists a sequence of rational numbers
q1 , q2 , . . . such that limn→∞ qn = a.
By sequential continuity, f (a) = limn→∞ f (qn ). In other
words, f (q1 ), f (q2 ), . . . determines f (a).
Since every f (a), a ∈ R is determined by some sequence
of rational numbers, collectively, {f (q)} , q ∈ Q determines
all possible values of f .
This theorem, together with Cauchy’s continuity principle,
allows us to approximate a real function with finite many
steps:

∀ > 0, ∃n ∈ N, s.t. |f (qn ) − f (a)| < . (3)

Qiu, Lee BST 401

Continuity and Approximation
Sketch of proof:
For any a ∈ R, there exists a sequence of rational numbers
q1 , q2 , . . . such that limn→∞ qn = a.
By sequential continuity, f (a) = limn→∞ f (qn ). In other
words, f (q1 ), f (q2 ), . . . determines f (a).
Since every f (a), a ∈ R is determined by some sequence
of rational numbers, collectively, {f (q)} , q ∈ Q determines
all possible values of f .
This theorem, together with Cauchy’s continuity principle,
allows us to approximate a real function with finite many
steps:

∀ > 0, ∃n ∈ N, s.t. |f (qn ) − f (a)| < . (3)

Qiu, Lee BST 401

Continuity and Approximation
Sketch of proof:
For any a ∈ R, there exists a sequence of rational numbers
q1 , q2 , . . . such that limn→∞ qn = a.
By sequential continuity, f (a) = limn→∞ f (qn ). In other
words, f (q1 ), f (q2 ), . . . determines f (a).
Since every f (a), a ∈ R is determined by some sequence
of rational numbers, collectively, {f (q)} , q ∈ Q determines
all possible values of f .
This theorem, together with Cauchy’s continuity principle,
allows us to approximate a real function with finite many
steps:

∀ > 0, ∃n ∈ N, s.t. |f (qn ) − f (a)| < . (3)

Qiu, Lee BST 401

The Squeeze (or Sandwich) theorem

Functional version. If f (x) 6 g(x) 6 h(x) in a

neighborhood of x0 (what does that mean?) with possible
exception at x0 , and

lim f (x) = lim h(x) = L,

x→x0 x→x0

then we have
lim g(x) = L.
x→x0

Sequence version. Replace functions by three sequences

and the functional convergence when x approaches x0 by
sequence convergence when n → ∞.

Qiu, Lee BST 401

The Squeeze (or Sandwich) theorem

Functional version. If f (x) 6 g(x) 6 h(x) in a

neighborhood of x0 (what does that mean?) with possible
exception at x0 , and

lim f (x) = lim h(x) = L,

x→x0 x→x0

then we have
lim g(x) = L.
x→x0

Sequence version. Replace functions by three sequences

and the functional convergence when x approaches x0 by
sequence convergence when n → ∞.

Qiu, Lee BST 401

Differentiation and linear approximation

The tangent problem. Algebraic definition:


0 df (x)  f (x + h) − f (x)
f (x0 ) := = lim .
dx x=x0 x→x0
 h

Differentiation requires continuity, but not the other way.

Show students three examples: the step function, the
absolute value function, and a differentiable function.
Linear approximation: Near a neighborhood of x0 , we have

f (x0 + ∆x) ≈ f (x0 ) + f 0 (x0 )∆x.

Example: linear approximation of 182 near 20.

Approximation: 320. True value: 324.

Qiu, Lee BST 401

Differentiation and linear approximation

The tangent problem. Algebraic definition:


0 df (x)  f (x + h) − f (x)
f (x0 ) := = lim .
dx x=x0 x→x0
 h

Differentiation requires continuity, but not the other way.

Show students three examples: the step function, the
absolute value function, and a differentiable function.
Linear approximation: Near a neighborhood of x0 , we have

f (x0 + ∆x) ≈ f (x0 ) + f 0 (x0 )∆x.

Example: linear approximation of 182 near 20.

Approximation: 320. True value: 324.

Qiu, Lee BST 401

Differentiation and linear approximation

The tangent problem. Algebraic definition:


0 df (x)  f (x + h) − f (x)
f (x0 ) := = lim .
dx x=x0 x→x0
 h

Differentiation requires continuity, but not the other way.

Show students three examples: the step function, the
absolute value function, and a differentiable function.
Linear approximation: Near a neighborhood of x0 , we have

f (x0 + ∆x) ≈ f (x0 ) + f 0 (x0 )∆x.

Example: linear approximation of 182 near 20.

Approximation: 320. True value: 324.

Qiu, Lee BST 401

Differentiation and linear approximation

The tangent problem. Algebraic definition:


0 df (x)  f (x + h) − f (x)
f (x0 ) := = lim .
dx x=x0 x→x0
 h

Differentiation requires continuity, but not the other way.

Show students three examples: the step function, the
absolute value function, and a differentiable function.
Linear approximation: Near a neighborhood of x0 , we have

f (x0 + ∆x) ≈ f (x0 ) + f 0 (x0 )∆x.

Example: linear approximation of 182 near 20.

Approximation: 320. True value: 324.

Qiu, Lee BST 401

Differentiation and linear approximation

The tangent problem. Algebraic definition:


0 df (x)  f (x + h) − f (x)
f (x0 ) := = lim .
dx x=x0 x→x0
 h

Differentiation requires continuity, but not the other way.

Show students three examples: the step function, the
absolute value function, and a differentiable function.
Linear approximation: Near a neighborhood of x0 , we have

f (x0 + ∆x) ≈ f (x0 ) + f 0 (x0 )∆x.

Example: linear approximation of 182 near 20.

Approximation: 320. True value: 324.

Qiu, Lee BST 401

Power series of real numbers

A series is a sequence of partial sums of a sequence.

∞ ,b =
Pn k
(bn
P∞ n) n=1 n k =1 a . Alternative notation:
2 n
n=1 a = limn→∞ (a + a + . . . + a ). For simplicity, we
assume −1 < a < 1.

(1 − a)(1 + a + a2 + . . . + an ) = 1 − an+1
1 − an+1
an = a + a2 + . . . + an = −1
1−a
1 a
lim bn = −1= .
n→∞ 1−a 1−a
If a ≥ 1 or a ≤ −1: an doesn’t converge to zero, so bn must
diverge. Generalization to complex numbers: |a| < 1 ⇔
convergence of power series bn .

Qiu, Lee BST 401

Power series of real numbers

A series is a sequence of partial sums of a sequence.

∞ ,b =
Pn k
(bn
P∞ n) n=1 n k =1 a . Alternative notation:
2 n
n=1 a = limn→∞ (a + a + . . . + a ). For simplicity, we
assume −1 < a < 1.

(1 − a)(1 + a + a2 + . . . + an ) = 1 − an+1
1 − an+1
an = a + a2 + . . . + an = −1
1−a
1 a
lim bn = −1= .
n→∞ 1−a 1−a
If a ≥ 1 or a ≤ −1: an doesn’t converge to zero, so bn must
diverge. Generalization to complex numbers: |a| < 1 ⇔
convergence of power series bn .

Qiu, Lee BST 401

Power series of real numbers

A series is a sequence of partial sums of a sequence.

∞ ,b =
Pn k
(bn
P∞ n) n=1 n k =1 a . Alternative notation:
2 n
n=1 a = limn→∞ (a + a + . . . + a ). For simplicity, we
assume −1 < a < 1.

(1 − a)(1 + a + a2 + . . . + an ) = 1 − an+1
1 − an+1
an = a + a2 + . . . + an = −1
1−a
1 a
lim bn = −1= .
n→∞ 1−a 1−a
If a ≥ 1 or a ≤ −1: an doesn’t converge to zero, so bn must
diverge. Generalization to complex numbers: |a| < 1 ⇔
convergence of power series bn .

Qiu, Lee BST 401

Approximation and Taylor Expansion

Taylor expansion is another way of approximating a smooth

function with finite steps. Rather than approximating f (a)
by information provided by nearby points, we approximate
it with the knowledge of its derivatives. It is closely related
to the concept of the moment generating function and the
characteristic function Dr. Lee will cover later.

Qiu, Lee BST 401

Approximations in the real world

Finite step approximation is more than “approximation”. It

is pretty much the only way we, as animals equipped with
finite step logic calculation ability, can deal with the real
world, which is infinitely complex.
Philosophical implications. Almost all engineering solutions
assumes continuity of the real world. Think: why you even
dare to drive a car? Predictability.
No perfect predictability in this world. In fact there is no
perfect measurement of any sort: time, length, force,
thickness of your car, smoothness of the road, etc.
Tolerance of small errors is of crucial importance.
Ever close precision.

Qiu, Lee BST 401

Approximations in the real world

Finite step approximation is more than “approximation”. It

is pretty much the only way we, as animals equipped with
finite step logic calculation ability, can deal with the real
world, which is infinitely complex.
Philosophical implications. Almost all engineering solutions
assumes continuity of the real world. Think: why you even
dare to drive a car? Predictability.
No perfect predictability in this world. In fact there is no
perfect measurement of any sort: time, length, force,
thickness of your car, smoothness of the road, etc.
Tolerance of small errors is of crucial importance.
Ever close precision.

Qiu, Lee BST 401

Approximations in the real world

Finite step approximation is more than “approximation”. It

is pretty much the only way we, as animals equipped with
finite step logic calculation ability, can deal with the real
world, which is infinitely complex.
Philosophical implications. Almost all engineering solutions
assumes continuity of the real world. Think: why you even
dare to drive a car? Predictability.
No perfect predictability in this world. In fact there is no
perfect measurement of any sort: time, length, force,
thickness of your car, smoothness of the road, etc.
Tolerance of small errors is of crucial importance.
Ever close precision.

Qiu, Lee BST 401

Approximations in the real world

Finite step approximation is more than “approximation”. It

is pretty much the only way we, as animals equipped with
finite step logic calculation ability, can deal with the real
world, which is infinitely complex.
Philosophical implications. Almost all engineering solutions
assumes continuity of the real world. Think: why you even
dare to drive a car? Predictability.
No perfect predictability in this world. In fact there is no
perfect measurement of any sort: time, length, force,
thickness of your car, smoothness of the road, etc.
Tolerance of small errors is of crucial importance.
Ever close precision.

Qiu, Lee BST 401

Approximations in the real world

Finite step approximation is more than “approximation”. It

is pretty much the only way we, as animals equipped with
finite step logic calculation ability, can deal with the real
world, which is infinitely complex.
Philosophical implications. Almost all engineering solutions
assumes continuity of the real world. Think: why you even
dare to drive a car? Predictability.
No perfect predictability in this world. In fact there is no
perfect measurement of any sort: time, length, force,
thickness of your car, smoothness of the road, etc.
Tolerance of small errors is of crucial importance.
Ever close precision.

Qiu, Lee BST 401

Qualitative properties of a real function (I)

For a given differentiable function f (x), many qualitative

properties can be obtained by computing its first and second
order derivatives.
f 0 (a) > 0: f (x) is increasing at point x = a.
f 0 (a) < 0: f (x) is decreasing at point x = a.
f 0 (a) = 0: a is a critical point of f (x).
For a critical point a∗ :
If f 00 (a∗ ) > 0, a∗ is a local minimizer of f (x).
If f 00 (a∗ ) < 0, a∗ is a local maximizer of f (x).
If f 00 (a∗ ) = 0, higher derivatives are needed to determine
the property of this point.

Qiu, Lee BST 401

Qualitative properties of a real function (I)

For a given differentiable function f (x), many qualitative

properties can be obtained by computing its first and second
order derivatives.
f 0 (a) > 0: f (x) is increasing at point x = a.
f 0 (a) < 0: f (x) is decreasing at point x = a.
f 0 (a) = 0: a is a critical point of f (x).
For a critical point a∗ :
If f 00 (a∗ ) > 0, a∗ is a local minimizer of f (x).
If f 00 (a∗ ) < 0, a∗ is a local maximizer of f (x).
If f 00 (a∗ ) = 0, higher derivatives are needed to determine
the property of this point.

Qiu, Lee BST 401

Qualitative properties of a real function (I)

For a given differentiable function f (x), many qualitative

properties can be obtained by computing its first and second
order derivatives.
f 0 (a) > 0: f (x) is increasing at point x = a.
f 0 (a) < 0: f (x) is decreasing at point x = a.
f 0 (a) = 0: a is a critical point of f (x).
For a critical point a∗ :
If f 00 (a∗ ) > 0, a∗ is a local minimizer of f (x).
If f 00 (a∗ ) < 0, a∗ is a local maximizer of f (x).
If f 00 (a∗ ) = 0, higher derivatives are needed to determine
the property of this point.

Qiu, Lee BST 401

Qualitative properties of a real function (I)

For a given differentiable function f (x), many qualitative

properties can be obtained by computing its first and second
order derivatives.
f 0 (a) > 0: f (x) is increasing at point x = a.
f 0 (a) < 0: f (x) is decreasing at point x = a.
f 0 (a) = 0: a is a critical point of f (x).
For a critical point a∗ :
If f 00 (a∗ ) > 0, a∗ is a local minimizer of f (x).
If f 00 (a∗ ) < 0, a∗ is a local maximizer of f (x).
If f 00 (a∗ ) = 0, higher derivatives are needed to determine
the property of this point.

Qiu, Lee BST 401

Qualitative properties of a real function (I)

For a given differentiable function f (x), many qualitative

properties can be obtained by computing its first and second
order derivatives.
f 0 (a) > 0: f (x) is increasing at point x = a.
f 0 (a) < 0: f (x) is decreasing at point x = a.
f 0 (a) = 0: a is a critical point of f (x).
For a critical point a∗ :
If f 00 (a∗ ) > 0, a∗ is a local minimizer of f (x).
If f 00 (a∗ ) < 0, a∗ is a local maximizer of f (x).
If f 00 (a∗ ) = 0, higher derivatives are needed to determine
the property of this point.

Qiu, Lee BST 401

Qualitative properties of a real function (I)

For a given differentiable function f (x), many qualitative

properties can be obtained by computing its first and second
order derivatives.
f 0 (a) > 0: f (x) is increasing at point x = a.
f 0 (a) < 0: f (x) is decreasing at point x = a.
f 0 (a) = 0: a is a critical point of f (x).
For a critical point a∗ :
If f 00 (a∗ ) > 0, a∗ is a local minimizer of f (x).
If f 00 (a∗ ) < 0, a∗ is a local maximizer of f (x).
If f 00 (a∗ ) = 0, higher derivatives are needed to determine
the property of this point.

Qiu, Lee BST 401

Qualitative properties of a real function (II)

Alternative interpretation through Taylor expansion:

f (x) = b0 + b1 (x − a) + b2 (x − a)2 + . . .
f 00 (a)  
= f (a) + f 0 (a)(x − a) + (x − a)2 + O (x − a)3
2

T1 (x) = f (a) + f 0 (a)(x − a) is the best first order

approximation of f (x) near x = a (which is the tangent
line). f 0 (a) is the slope of this line. Apparently,
positive/negative slope implies T1 (x) being
increasing/decreasing near x = a.
00
T2 (x) = f (a) + f 0 (a)(x − a) + f 2(a) (x − a)2 is the best
second order approximation of f (x) near x = a.

Qiu, Lee BST 401

Qualitative properties of a real function (II)

Alternative interpretation through Taylor expansion:

f (x) = b0 + b1 (x − a) + b2 (x − a)2 + . . .
f 00 (a)  
= f (a) + f 0 (a)(x − a) + (x − a)2 + O (x − a)3
2

T1 (x) = f (a) + f 0 (a)(x − a) is the best first order

approximation of f (x) near x = a (which is the tangent
line). f 0 (a) is the slope of this line. Apparently,
positive/negative slope implies T1 (x) being
increasing/decreasing near x = a.
00
T2 (x) = f (a) + f 0 (a)(x − a) + f 2(a) (x − a)2 is the best
second order approximation of f (x) near x = a.

Qiu, Lee BST 401

Qualitative properties of a real function (III)

Usually the first order term dominates the second order

term when x ≈ a =⇒ (x − a)2  |x − a|.
One notable exception: f 0 (a) = 0. Since this “nukes” the
first order term completely.
When the first order term is absent (f 0 (a) = 0), the second
order term becomes important. That’s why we need to
00
know the sign of f 2(x) in order to classify critical points.
00
The sign of f 2(x) determines the concavity of f (x). Draw
two figures: b0 + (x − a)2 and b0 − (x − a)2 .
Back to the quiz: Taylor expansion of sin(2x) at: a) π2 ; b) 0;
c) π4 .

Qiu, Lee BST 401

Qualitative properties of a real function (III)

Usually the first order term dominates the second order

term when x ≈ a =⇒ (x − a)2  |x − a|.
One notable exception: f 0 (a) = 0. Since this “nukes” the
first order term completely.
When the first order term is absent (f 0 (a) = 0), the second
order term becomes important. That’s why we need to
00
know the sign of f 2(x) in order to classify critical points.
00
The sign of f 2(x) determines the concavity of f (x). Draw
two figures: b0 + (x − a)2 and b0 − (x − a)2 .
Back to the quiz: Taylor expansion of sin(2x) at: a) π2 ; b) 0;
c) π4 .

Qiu, Lee BST 401

Qualitative properties of a real function (III)

Usually the first order term dominates the second order

term when x ≈ a =⇒ (x − a)2  |x − a|.
One notable exception: f 0 (a) = 0. Since this “nukes” the
first order term completely.
When the first order term is absent (f 0 (a) = 0), the second
order term becomes important. That’s why we need to
00
know the sign of f 2(x) in order to classify critical points.
00
The sign of f 2(x) determines the concavity of f (x). Draw
two figures: b0 + (x − a)2 and b0 − (x − a)2 .
Back to the quiz: Taylor expansion of sin(2x) at: a) π2 ; b) 0;
c) π4 .

Qiu, Lee BST 401

Qualitative properties of a real function (III)

Usually the first order term dominates the second order

term when x ≈ a =⇒ (x − a)2  |x − a|.
One notable exception: f 0 (a) = 0. Since this “nukes” the
first order term completely.
When the first order term is absent (f 0 (a) = 0), the second
order term becomes important. That’s why we need to
00
know the sign of f 2(x) in order to classify critical points.
00
The sign of f 2(x) determines the concavity of f (x). Draw
two figures: b0 + (x − a)2 and b0 − (x − a)2 .
Back to the quiz: Taylor expansion of sin(2x) at: a) π2 ; b) 0;
c) π4 .

Qiu, Lee BST 401

Qualitative properties of a real function (III)

Usually the first order term dominates the second order

term when x ≈ a =⇒ (x − a)2  |x − a|.
One notable exception: f 0 (a) = 0. Since this “nukes” the
first order term completely.
When the first order term is absent (f 0 (a) = 0), the second
order term becomes important. That’s why we need to
00
know the sign of f 2(x) in order to classify critical points.
00
The sign of f 2(x) determines the concavity of f (x). Draw
two figures: b0 + (x − a)2 and b0 − (x − a)2 .
Back to the quiz: Taylor expansion of sin(2x) at: a) π2 ; b) 0;
c) π4 .

Qiu, Lee BST 401

Compact/Non-compact set

Whether a continuous function f (x) has global min/max

depends on its domain
If a continuous function f (x) is defined on a bounded,
closed interval [a, b], then there exist x1 and x2 in [a, b]
such that f (x1 ) is the global maximum, f (x2 ) is the global
minimum.
Generalization: as long as its domain is a) bounded; b)
closed. Or simply put: a compact set.
Counter examples
1 Domain is [a, ∞). Escape to ∞.
2 Domain is (a, b). Stat. example: MLE of the variance
problem.
3 Domain is compact, but f (x) is not continuous.

Qiu, Lee BST 401

Compact/Non-compact set

Whether a continuous function f (x) has global min/max

depends on its domain
If a continuous function f (x) is defined on a bounded,
closed interval [a, b], then there exist x1 and x2 in [a, b]
such that f (x1 ) is the global maximum, f (x2 ) is the global
minimum.
Generalization: as long as its domain is a) bounded; b)
closed. Or simply put: a compact set.
Counter examples
1 Domain is [a, ∞). Escape to ∞.
2 Domain is (a, b). Stat. example: MLE of the variance
problem.
3 Domain is compact, but f (x) is not continuous.

Qiu, Lee BST 401

Compact/Non-compact set

Whether a continuous function f (x) has global min/max

depends on its domain
If a continuous function f (x) is defined on a bounded,
closed interval [a, b], then there exist x1 and x2 in [a, b]
such that f (x1 ) is the global maximum, f (x2 ) is the global
minimum.
Generalization: as long as its domain is a) bounded; b)
closed. Or simply put: a compact set.
Counter examples
1 Domain is [a, ∞). Escape to ∞.
2 Domain is (a, b). Stat. example: MLE of the variance
problem.
3 Domain is compact, but f (x) is not continuous.

Qiu, Lee BST 401

Compact/Non-compact set

Whether a continuous function f (x) has global min/max

depends on its domain
If a continuous function f (x) is defined on a bounded,
closed interval [a, b], then there exist x1 and x2 in [a, b]
such that f (x1 ) is the global maximum, f (x2 ) is the global
minimum.
Generalization: as long as its domain is a) bounded; b)
closed. Or simply put: a compact set.
Counter examples
1 Domain is [a, ∞). Escape to ∞.
2 Domain is (a, b). Stat. example: MLE of the variance
problem.
3 Domain is compact, but f (x) is not continuous.

Qiu, Lee BST 401

Compact/Non-compact set

Whether a continuous function f (x) has global min/max

depends on its domain
If a continuous function f (x) is defined on a bounded,
closed interval [a, b], then there exist x1 and x2 in [a, b]
such that f (x1 ) is the global maximum, f (x2 ) is the global
minimum.
Generalization: as long as its domain is a) bounded; b)
closed. Or simply put: a compact set.
Counter examples
1 Domain is [a, ∞). Escape to ∞.
2 Domain is (a, b). Stat. example: MLE of the variance
problem.
3 Domain is compact, but f (x) is not continuous.

Qiu, Lee BST 401

Compact/Non-compact set

Whether a continuous function f (x) has global min/max

depends on its domain
If a continuous function f (x) is defined on a bounded,
closed interval [a, b], then there exist x1 and x2 in [a, b]
such that f (x1 ) is the global maximum, f (x2 ) is the global
minimum.
Generalization: as long as its domain is a) bounded; b)
closed. Or simply put: a compact set.
Counter examples
1 Domain is [a, ∞). Escape to ∞.
2 Domain is (a, b). Stat. example: MLE of the variance
problem.
3 Domain is compact, but f (x) is not continuous.

Qiu, Lee BST 401

Homework

Homework will be sent to you via email.

Homework is always due on the next Thursday.

Qiu, Lee BST 401

Homework

Homework will be sent to you via email.

Homework is always due on the next Thursday.

Qiu, Lee BST 401