Image Registration

Geometrical adjustment of images

Bj orn Svensson, Johanna Pettersson, Anders Eklund, Hans Knutsson
Department of Biomedical Engineering, Link oping University
October, 2008
1 Problem description
Find a movement eld v(x) : R
2
R
2
, x = [x, y], such that the morphed
image I
2
will t as good as possible wth the original image I
1
, i.e. nd the v that
minimizes given by

2
= I
2
(x +v(x)) I
1
(x)
2
(1)
The movement eld describes how each point x has moved from I
1
to I
2
.
v(x)
I
2
I
1
Figure 1: The image registration problem is about nding the mapping v(x) that
best describes the global movement between image I
1
and image I
2
The method described in these notes rests on the following 2 assumptions.
1
I. The motion can locally be described as movement x.
I(x, t) = I(x + x, t + 1) (2)
II. The image can locally be described as a leaning plane
I(x + v(x), t + 1) = I(x, t + 1) +I
T
v (3)
where I = [
x
I,
y
I]
T
.
Assumption I says that the difference between I
1
and I
2
is only a movement. Spa-
tial image information such as intensity will not change in corresponding points.
Assumption II says that a local neighbourhoud in the image can be approximated
with a Taylor expansion of order 1.
2 Solution
Assumption I and II together gives the equation
I
T
v
t
= 0 (4)
where
t
= I(x, t) I(x, t + 1). The equation describes the ow v that is called
the Optical ow.
In each image point there will be a lot of solutions for v, see [1] section 6.6 for
more details. Therefore you need a model for what the movement eld should look
like and search for a solution that will t as good as possible for all image points.
The solution is found by nding the ow v that minimizes in the least squares
expression (5).

2
=

i
(I(x
i
)
T
v(x
i
)
t
(x
i
))
2
(5)
2
Let the movement eld be translation only, i.e.
v(x
i
) = v
0
. This gives

2
=

i
(I(x
i
)
T
v
0

t
(x
i
))
2
(6)
The minimum is found by studying the derivative of with respect to v
0
which
gives equation (7). The dependence of x
i
will from now only be written with index
i, i.e. v(x
i
) = v
i
.

2
v
0
= 2

i
I
i
(I
T
i
v
0

ti
) = 0 (7)
This gives

i
I
i
I
T
i
. .
A
v
0
=

i
I
i

ti
. .
h
(8)
In matrix form this can be written as the matrix A (size n n) and the vector
h (n 1), where n is the number of pixels in the image. The solution v
0opt
that
minimizes can then be written as v
0opt
= A
1
h.
3 Parametrisation
Translation only is normally not an adequate description and a more general ap-
proach is to model a movement eld v(x), expressed with help of the parameter
vector p.
Some commonly used models include the following
Translation
Rotation
Scaling, zooming
Local stretching
3
The model can be written as v(x) = B(x)p, where B(x) is the base matrix for
the parameter vector p.
Earlier only translation was assumed, but this could have been written as a para-
metrical movement eld according to:
v
0
= v(x) =
_
1 0
0 1
_ _
p
x
p
y
_
An afne movement eld is modeled with the help of a 6 dimensional parameter
vector. p = [p
1
, p
2
, p
3
, p
4
, p
5
, p
6
] according to:
v(x) =
_
p
1
p
2
_
+
_
p
3
p
4
p
5
p
6
_ _
x
y
_
=
_
1 0 x y 0 0
0 1 0 0 x y
_
p
A base for second degree adaptation in 2D, a Fourier base, a base of nite elements
or a Wavelet base are additional examples of possible bases for a parametrical
model.
3.1 Solution I
Replace v(x
i
) in equation (5) with v(x
i
) = B(x
i
)p, which gives the expression
to minimize for a parametrical model.

2
=

i
(I
T
i
B
i
p
ti
)
2
(9)
Now we search for the parameter vector p that minimizes . The derivative of
with respect to p then gives equation (10).

2
p
= 2

i
B
T
i
I
i
(I
T
i
B
i
p
ti
) = 0 (10)
This gives

i
B
T
i
I
i
I
T
i
B
i
. .
A
p =

i
B
T
i
I
i

ti
. .
h
(11)
and the solution can as before be written as p
opt
= A
1
h.
4
3.2 Solution II
Exactly the same thing can be written more compactly from the equation that de-
scribes Optical ow.
I
T
v
t
= 0 (12)
In terms of I
1
, I
2
, B, p it can be rewritten as
I
T
2
B(x)
. .

A
p I
1
(x) I
2
(x)
. .
b
x (13)
y
x
x
1
x
2
x
3
x
n
x
n1
x
n2
Figure 2: The positions of the pixels can be described as n vectors x
1
x
n
.
The matrix

A (size n p) and the vector b (size n 1), where n is the amount
of pixels in the image and p is the number of parameters in the parameter vector
p, will have the following expression if we number the image points according to
gure 2.
5

A =
_

_
I
2
(x
1
)
T
B(x
1
)
I
2
(x
2
)
T
B(x
2
)
.
.
.
I
2
(x
n
)
T
B(x
n
)
_

_
b =
_

_
I
1
(x
1
) I
2
(x
1
)
I
1
(x
2
) I
2
(x
2
)
.
.
.
I
1
(x
n
) I
2
(x
n
)
_

_
Over determined equation systems of the type

Ax b can be recognized from
linear algebra. The least squares solution is given by
p
opt
= (

A
T

A
. .
=A
)
1

A
T
b
..
=h
(14)
and A, h can be identied from the previous solution.
4 Phase based estimation of movement
There are many cases when assumption I (local movement) and II (locally leaning
plane) are not valid. Images where the contrast and mean value varies with time
(assumption I) are some simple examples of this. For neighbourhoods with local
tops and valleys (assumption II) the performance is also poor.
If you reformulate these assumptions to instead be valid for the quadrature lter
phase, you will get a more robust estimation of the movement that can handle
the cases described above in a better way. Quadrature lters are described in [1],
chapter 7.
The quadrature lter phase describes similarities between the signal and a line, and
the signal and an edge. The lter response is completely imaginary when the signal
corresponds to a line and completely real when the signal corresponds to an edge.
This is illustrated in gure 3. The advantage with the quadrature lter phase is that
it is independent of the signal energy.
The quadrature lter phase (x) is given by arg(q), where q represents the lter
response. From gure 3 you see how this corresponds to the similarity between the
signal and an edge and a line respectively.
The equation for Optical ow can therefore be formulated for the quadrature lter
phase according to:

T
v
t
= 0 (15)
6
Figure 3: The quadrature lter phase is given as the argument of the lter response,
and describes the similarity between the signal and an edge and a line respectively.
where
t
= (x, t + 1) (x, t) =
1

2
= arg(q
2
q

1
)
4.1 Phase gradient
The phase gradient is approximated with help of the quadrature lter response
q. Compare with local frequency, chapter 8 in [1]. The following estimation can be
seen as a rst order approximation and can be found in [2].
=
_

_
=
_
arg[q
1
(x + 1, y)q

1
(x 1, y) + q
2
(x + 1, y)q

2
(x 1, y)]
arg[q
1
(x, y + 1)q

1
(x, y 1) + q
2
(x, y + 1)q

2
(x, y 1)]
_
(16)
4.2 Security measure
Quadrature lters only describe a signal in different directions and therefore more
lters are needed to cover all the possible directions of the movement eld.
7
Since different lters are not equally good at describing signals in different direc-
tions you need a security measure that tells you which lter(s) that you should trust
in each point.
Since the phase is periodic with 2, and since the uncertainty increases with the
phase difference, you also want to suppress large phase differences. One suggestion
of security measure is
c =
_
|q
1
q
2
| cos (
t
/2)
2
(17)
The factor
_
|q
1
q
2
| demands that you have a strong signal in both I
1
and I
2
and
the cos
2
-factor suppresses large phase differences.
Even when you dont use phase based estimation, you can benet from using se-
curity measures. Assumption I and II doesnt have to be valid for all the points in
the image, if you can identify the points where the estimation is bad, , so called
outliers, and suppress their inuence on the solution.
4.3 Solution
The new equation valid for the quadrature lter phase can be written to t the
previous approach.

T
v
t
= 0 (18)
Expressed in
1
,
2
, B, p you get

2
T
B(x)p
1
(x)
2
(x) x, q (19)
Equation 19 shall now be valid as much as possible for all image points and all
quadrature lter responses. Therefore we need to include our security measure in
equation (19). For m quadrature lters, k = 1 m you then get
c(x, k)
. .
W
(
2
T
B(x))
. .

A
p c(x, k)
. .
W
(I
1
(x) I
2
(x))
. .
b
x, q (20)
8
In matrix form you get

A(size mnmn), diagonal matrix W (size mnmn)
and the vector b (size m n 1). Since m quadrature lters are used the number
of equations will increase with a factor m.

A =
_

_
(x
1
)
T
B(x
1
)
(x
2
)
T
B(x
2
)
.
.
.
(x
n
)
T
B(x
n
)
_

_
b =
_

1
(x
1
)
2
(x
1
)

1
(x
2
)
2
(x
2
)
.
.
.

1
(x
n
)
2
(x
n
)
_

_
W =
_

_
c(x
1
, 1) 0 0
0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. 0
0 0 c(x
n
, m)
_

_
The parameter vector p
o
pt that minimizes is given by the weighted least squares
solution
p
opt
= (

A
T
W
T
W

A)
1

A
T
W
T
Wb
5 Image adjustment
To create an image

I that is moved according to the movement eld v the following
operation is performed.

I(x) = I(x v(x)) x Z

2
, v R
2
(21)
But v is real numbers and each pixel position x is discrete. The solution is to
interpolate, i.e. to approximate the signal value with neighbouring pixel values.
In gure 4 bilinear interpolation is shown where the signal value I is estimated
with help of the pixels I
11
, I
12
, I
21
, I
22
. In the x-direciton the signal values I
1
, I
2
are given by
x-direction
_
I
1
= xI
21
+ (1 x)I
11
I
2
= xI
22
+ (1 x)I
12
From I
1
, I
2
the interpolation in the y-direction can be calculated and the expression
that describes bilinear interpolation can be derived easily.
9
y
I
x I
21
I
22
I
12
I
11
I
1
I
2
Figure 4: Bilinear interpolation to calculate I, with the help of 4 neighbouring pixel
values. The centers of the pixels, with signal values I
11
, I
12
, I
21
, I
22
, are marked.
I = yI
2
+ (1 y)I
1
=
= y(xI
22
+ (1 x)I
12
) + (1 y)(xI
21
+ (1 x)I
11
) =
= I
11
+ x(I
21
I
11
) + y(I
12
I
11
) + xy(I
22
+ I
11
I
12
I
21
)
(22)
References
[1] G. H. Granlund and H. Knutsson. Signal Processing for Computer Vision.
Kluwer Academic Publishers, 1995. ISBN 0-7923-9530-1.
[2] C-J. Westelius. Focus of Attention and Gaze Control for Robot Vision. PhD
thesis, Link oping University, Sweden, SE-581 83 Link oping, Sweden, 1995.
Dissertation No 379, ISBN 91-7871-530-X.
10