LECTURER: ROBERT GRAF

ADVANCED MATHEMATICS
TOPIC OUTLINE

Calculus 1

Integral Transformations 2

Vector Algebra 3

Vector Calculus 4

Matrices and Vector Spaces 5

Information Theory 6
UNIT 2

INTEGRAL TRANSFORMATIONS
STUDY GOALS

− Integral transformations
− How to combine the effects of two functions using a
convolution integral
− How to use convolutions in concrete problems
− How to express periodic signals as Fourier series
− How to relate time domain and frequency domain
functions using Fourier transformations
EXPLAIN SIMPLY

1. Explain the concept of intrinsic resolution of a

measuring instrument.
2. Explain the main idea of a Fourier series and why
such series are used to approximate periodic
functions/signals.
3. Recall Euler’s formula. Why is it important?
CONVOLUTIONS:
DEFINITION

Convolution is an important integral transformation that takes two functions 𝑓 and 𝑔 as

an input and produces a third function ℎ as an output. This operation is usually denoted
by ℎ = 𝑓 ∗ 𝑔.
As such, the convolution of 𝑓 and 𝑔, is a so-called mathematical operator, i.e. a function
that transforms input functions 𝑓, 𝑔 into a new output function ℎ = 𝑓 ∗ 𝑔.
(Compare this to the notion of a functional introduced in Unit 1 — how are these concepts
related?)

Convolution is defined as the integral of the product of the two functions after one
function is reversed and shifted:
∞
ℎ(𝑧) = (𝑓 ∗ 𝑔)(𝑧) = න 𝑓 𝑥 𝑔 𝑧 − 𝑥 d𝑥
−∞
CONVOLUTIONS:
MEASURING A PHYSICAL QUANTITY

We rely on a measurement device to measure any physical quantity,

For example, to measure temperature, we use a thermometer.

The function 𝑓 𝑥 represents the “true” values of the quantity being

measured.
Usually, the underlying physical process is stochastic and hence 𝑓 𝑥 is a
probability distribution. Hence, we can calculate the probability of getting a
true value in the interval 𝑥, 𝑥 + d𝑥 .
CONVOLUTIONS:
MEASURING A PHYSICAL QUANTITY

The actual measurement is a potentially distorted observation of the “true” physical

quantity. The distortion is determined by the intrinsic resolution of the measuring
instrument.
The device is represented by a (stochastic) resolution function 𝑔 𝑦 that determines how
the true values are observed. The measurement instrument will generally shift the true
value to the observed value, hence we do not observe 𝑥 but rather a shifted value 𝑧 = 𝑥
+ 𝑦, in which 𝑦 denotes the systematic error introduced by the device.

The measurements themselves are then represented by a third function ℎ 𝑧 , i.e. the
convolution of 𝑓 and 𝑔 :
∞
ℎ(𝑧) = (𝑓 ∗ 𝑔)(𝑧) = න 𝑓 𝑥 𝑔 𝑧 − 𝑥 d𝑥
−∞
CONVOLUTIONS:
RESOLUTION

− A device can have a high or low

resolution.
− It can return an unbiased or a biased
measurement. Thus it can be faithful
or not faithful to the „true" values.

Explain these different

concepts via the image on
the right.

Source of the text: Course book DLMDSAM01, p. 49.

CONVOLUTIONS:
MEASURING A PHYSICAL QUANTITY

In case of an unbiased instrument, the resolution will “smear” the true values so that the resulting
distribution is broader than the true physical one. In case of a biased instrument, this step will also include
a further shift in one direction.
As an example, assume the “true” values are distributed according to a Γ function (black solid line) while
the measurement device is represented by an unbiased Gaussian resolution function with mean zero and
standard deviation one. The observed values are then distributed according to the convolution of both
functions as shown in the graph on the right.

Note that although we know that the true

values are strictly positive, the observed values
can be negative due to the resolution of the
measurement instrument.
This needs special attention depending on the
concrete problem!

Source of the text: Course book DLMDSAM01, p. 52.

CONVOLUTIONS:
DISCRETE CONVOLUTION IN IMAGE ANALYSIS

Convolutions play an important role in image processing in which 𝑓 represents a digital image
while 𝑔 represents a so-called kernel that is used to operate on the image.
A suitable kernel defines a filter that can be used for a wide range of applications such as
blurring, sharpening, and edge detection.

Illustration of Gaussian Blurring: Original, Medium, and Strong Blurring

The kernel function is convolved with the

digital image to get a new image, e.g. a
blurred version of the original image.
Importantly, a digital image is a discrete
object and so is the function 𝑓
representing it!

Source of the text: Course book DLMDSAM01, p. 54.

CONVOLUTIONS:
DISCRETE CONVOLUTION IN IMAGE ANALYSIS

We represent a digital image as a collection of individual pixels of the form 𝑥, 𝑦, 𝑟, 𝑔, 𝑏 , in

which 𝑥 and 𝑦 give the position of the pixel in the image and 𝑟, 𝑔, 𝑏 give the levels of red, green
and blue in that pixel. For simplicity, we can also just work with grey value images of the form
𝑥, 𝑦, 𝑔 .

The continuous convolution integral is discretised and becomes a summation

𝑎 𝑏

𝐾 ∗ 𝑓 𝑥, 𝑦 = ෍ ෍ 𝐾 𝑖, 𝑗 𝑓 𝑥 − 𝑖, 𝑦 − 𝑗 ,
𝑖=−𝑎 𝑗=−𝑏
in which 𝐾 denotes a discrete kernel function represented as 𝑛 × 𝑚-matrix with 𝑛 = 2𝑎 + 1 and
𝑚 = 2𝑏 + 1.
CONVOLUTIONS:
DISCRETE CONVOLUTION IN IMAGE ANALYSIS

An example of a Gaussian blurring kernel is:

1 1 2 1
𝐾= 2 4 2
16
1 2 1

In this case the convolution becomes

1 1

𝐾 ∗ 𝑓 𝑥, 𝑦 = ෍ ෍ 𝐾 𝑖, 𝑗 𝑓 𝑥 − 𝑖, 𝑦 − 𝑗
𝑖=−1 𝑗=−1
1
= ൫𝑓 𝑥 + 1, 𝑦 + 1 + 2𝑓 𝑥 + 1, 𝑦 + 𝑓 𝑥 + 1, 𝑦 − 1 + 2𝑓 𝑥, 𝑦 + 1 + 4𝑓 𝑥, 𝑦 + 2𝑓 𝑥, 𝑦 − 1
16
+ 𝑓 𝑥 − 1, 𝑦 + 1 + 2𝑓 𝑥 − 1, 𝑦 + 𝑓 𝑥 − 1, 𝑦 − 1 ൯

We will see more examples in today’s transfer task.

FOURIER TRANSFORMATION
FOURIER SERIES – DIRICHLET CONDITIONS

The main idea of Fourier series is to express a signal as a sum of sine and cosine
components of varying strength and frequency. Hence, Fourier series are used to
approximate periodic signals.
In order to approximate a function with a Fourier series, the signal must satisfy the so-
called Dirichlet conditions:

1. The function must be periodic, i.e. 𝑓(𝑥) = 𝑓(𝑥 + 𝐿) = 𝑓(𝑥 + 𝑛𝐿) for every 𝑥 and
every integer 𝑛, where 𝐿 is the length of the period.
2. The function must have at most a finite number of discontinuities within a period.
3. The function must have a finite number of maxima and minima within each period.
4. The integral of the function over a single period must exist and be finite.
FOURIER TRANSFORMATION
FOURIER SERIES – DEFINITION OF REAL FOURIER SERIES

The real Fourier series of a function 𝑓 satisfying the Dirichlet conditions is given by
∞
𝑎0 2𝜋
𝑓(𝑥) = + ෍ 𝑎𝑛 cos 𝜔𝑛 𝑥 + 𝑏𝑛 sin 𝜔𝑛 𝑥 with 𝜔𝑛 = 𝑛.
2 𝐿
𝑛=1

Calculating or estimating the coefficients of this series is usually the main challenge in
practice. In principle there are the following formulas for the coefficients of the real
Fourier series:
2 𝐿/2
𝑎𝑛 = න 𝑓 𝑥 ⋅ cos 𝜔𝑛 𝑥 d𝑥 for 𝑛 ≥ 0,
𝐿 −𝐿/2

2 𝐿/2
𝑏𝑛 = න 𝑓 𝑥 ⋅ sin 𝜔𝑛 𝑥 d𝑥 for 𝑛 ≥ 1.
𝐿 −𝐿/2
FOURIER TRANSFORMATION
FOURIER SERIES – DEFINITION OF COMPLEX FOURIER SERIES

The Fourier series of a function is often stated in terms of complex exponentials instead
of trigonometric functions.
The main link between the two representations is Euler’s formula
e𝑖𝜑 = cos𝜑 + 𝑖 sin 𝜑.
Using this identity, we get
1 1
sin(𝜔𝑛 𝑥) = e 𝑖𝜔𝑛 𝑥
−e −𝑖𝜔𝑛 𝑥
and cos(𝜔𝑛 𝑥) = e𝑖𝜔𝑛 𝑥 + e−𝑖𝜔𝑛 𝑥 .
2𝑖 2

As a result, the complex Fourier series can also be expressed as

∞
1 𝐿/2
𝑓(𝑥) = ෍ 𝑐𝑛 e𝑖𝜔𝑛 𝑥 with 𝑐𝑛 = න 𝑓 𝑥 e−𝑖𝜔𝑛 𝑥 d𝑥 .
𝐿 −𝐿/2
𝑛=−∞
FOURIER TRANSFORMATION
FOURIER SERIES – RELATION BETWEEN REAL AND COMPLEX COEFFICIENTS

Relation between real and complex Fourier series coefficients:

1
𝑐𝑛 = 𝑎𝑛 − 𝑖𝑏𝑛 for 𝑛 ≥ 1,
2
𝑎0
𝑐0 = ,
2
1
𝑐−𝑛 = 𝑎𝑛 + 𝑖𝑏𝑛 for 𝑛 ≥ 1.
2
FOURIER TRANSFORMATION
FOURIER SERIES – EXAMPLES

The first example shows a sinusoidal signal to which only one frequency contributes, namely
sin 𝑥 .
The second example shows a more complicated signal in which the signal of the first example is
overlaid with a signal that is ten times faster.

Source of the images: Course book DLMDSAM01, p. 55f.

FOURIER TRANSFORMATION
FOURIER SERIES – EXAMPLES

The third example shows a more realistic signal that is frequently encountered in
electrical engineering. During each period the sawtooth signal rises linearly between the
minimal and maximal value and drops to the minimum when the signal reaches the
maximum. The right part of the figure shows that many coefficients are needed in the
Fourier series to approximate
this signal. However, the weight
is exponentially dampened, which
makes it easy to control the
approximation error when we
truncate the Fourier series after
finitely many terms!

Source of the image: Course book DLMDSAM01, p. 57.

FOURIER TRANSFORMATION
DEFINITION

Fourier transforms are motivated by the study of Fourier series.

The Fourier transform is an extension of the Fourier series that results when the period of the
represented function is lengthened and allowed to approach infinity as outlined in the course
book.
A Fourier Transform is a mathematical operation that decomposes functions that depend on
time or space into functions that depend on spatial or temporal frequencies.
ሚ
The Fourier transform 𝑓(𝑥) of a function 𝑓(𝑥) is defined as
∞
1
𝑓ሚ 𝜔 = න 𝑓 𝑥 e−𝑖𝜔𝑥 d𝑥 .
2𝜋 −∞
The inverse of the Fourier transform is then given as
∞
1
𝑓(𝑥) = න 𝑓ሚ 𝜔 e𝑖𝜔𝑥 d𝜔 .
2𝜋 −∞
FOURIER TRANSFORMATION
APPLICATION

Using the Fourier series/Fourier transform we can switch between two equivalent
representations of a signal!
The main advantage of this potential switch is that some operations might be very complicated in
one domain, while being simple in the other.
As an example, we consider a so-called frequency filter which is very difficult to apply in the time
domain, but simple in the frequency domain.

A low-pass filter is used to extract the part of the

signal with low frequency. Intuitively, recalling
Fourier series, applying a low pass filter would
mean to discard or dampen all contributions with
an index 𝑛 larger than a certain threshold.

Source of the image: Course book DLMDSAM01, p. 61.

FOURIER TRANSFORMATION
APPLICATION

Fourier Fourier
transform inversion

Source of the image: Course book DLMDSAM01, p. 56, 57, 61.

REVIEW STUDY GOALS

− Integral transformations
− How to combine the effects of two functions using a
convolution integral
− How to use convolutions in concrete problems
− How to express periodic signals as Fourier series
− How to relate time domain and frequency domain
functions using Fourier transformations
SESSION 2

TRANSFER TASK
TRANSFER TASK

We have seen that kernel functions can be used to define filters for different
tasks in image processing.
Give an appropriate kernel 𝐾1, i.e. a 3 × 3 matrix, that leaves a digital image
unchanged.
Find a kernel 𝐾2, i.e. another 3 × 3 matrix that can be used to detect
horizontal and vertical edges in an image.
TRANSFER TASKS
PRESENTATION OF THE RESULTS

Please present your

results.
The results will be
discussed in
plenary.
TRANSFER TASK – HINTS

Recall that the discrete convolution operation, i.e. applying a kernel function 𝐾 to a
digital image 𝑓 𝑥, 𝑦 , is defined as
𝑎 𝑏

𝐾 ∗ 𝑓 𝑥, 𝑦 = ෍ ෍ 𝐾 𝑖, 𝑗 𝑓 𝑥 − 𝑖, 𝑦 − 𝑗 .
𝑖=−𝑎 𝑗=−𝑏
Moreover, for simplicity, we assume that the digital image 𝑓 has a single colour value at
𝑥, 𝑦 .
Part 1: We need to find a kernel 𝐾 such that 𝐾 ∗ 𝑓 𝑥, 𝑦 = 𝑓 𝑥, 𝑦 . Using the above
formula, it is easy to see that the kernel
0 0 0
𝐾 = 0 1 0 does the trick.
0 0 0
TRANSFER TASK – HINTS

Part 2: We need to find a matrix that can detect edges, i.e. changes between areas of
different colours. Assuming that the edge goes through the point 𝑥, 𝑦 , this means that
we either have different colours left and right of 𝑥, 𝑦 or above and below of 𝑥, 𝑦 . On
the other hand, we wish that the kernel classifies all pixels in 3 × 3 patches of the same
colour as “non-edge” pixels. Formalising these thoughts naturally leads to solutions of
the form:
0 1 0
𝐾 = 1 −4 1
0 1 0

This kernel is also known as Laplace filter.

LEARNING CONTROL QUESTIONS

1. True or false?
a) When applying Gaussian blurring to an image, the rgb-
values of a given pixel in the output are influenced by all
the neighbouring pixels in the input image.
b) When applying a discrete kernel in image processing the
rgb-values of a given pixel in the output are influenced
by all the neighbouring pixels in the input image.
LEARNING CONTROL QUESTIONS

2. Suppose 𝑓(𝑥) is the probability distribution of the actual

temperature we wish to measure and 𝑔(𝑦) is the resolution of the
thermometer. Then, the convolution
ℎ(𝑧) = (𝑓 ∗ 𝑔)(𝑧) …
a) … is the probability distribution of the temperature measurements.
b) … is the temperature observed on the thermometer.
c) … corresponds to a very narrow distribution when the thermometer
has low resolution
d) … is skewed towards higher values if the thermometer tends to
underestimate the true temperature.
LEARNING CONTROL QUESTIONS

3. Describe the Fourier transform

𝑔(𝜔)
෤ of 𝑔(𝑥) = 3cos(𝑥) + 7cos(10𝑥).
LEARNING CONTROL QUESTIONS – SOLUTIONS

1. True or false?
a) True, because the matrix on p. 54 of course book contains no 0s.
b) In general this is not true — look at the matrices used in the transfer task.
2. Suppose 𝑓(𝑥) is the probability distribution of the actual temperature we
wish to measure and 𝑔(𝑦) is the resolution of the thermometer. Then, the
convolution ℎ(𝑧) = (𝑓 ∗ 𝑔)(𝑧) …
a) … is the probability distribution of the temperature measurements.
3. Describe the Fourier transform 𝑔(𝜔) ෤ of 𝑔(𝑥) = 3cos(𝑥) + 7cos(10𝑥).
1 1
Since cos 𝑥 = 𝑒 𝑖𝑥 + 𝑒 −𝑖𝑥 and cos 10𝑥 = 𝑒 𝑖⋅10𝑥 + 𝑒 −𝑖⋅10𝑥 , the Fourier
2 2
transform is given by four spikes (delta functions) at 𝜔 = ±1 and 𝜔 = ±10.