In this class

• 2D and 3D images
• Data format
H06W8a: Medical image analysis • Basic intensity transformations
– Histogram
Class 2: Basic concepts – Window/level
– Maximum intensity projection
– Filtering
Prof. Frederik Maes • Spatial transformations
– Coordinate systems
frederik.maes@esat.kuleuven.be
– Coordinate transformations
– Image resampling
– Image interpolation

2D images 3D images
• 2D matrix of picture elements (= pixels) of size Nc x Nr (= dimensions), e.g. 256 x 256 • 3D matrix of volume elements (= voxels), i.e. a stack of 2D image planes (= slices)
• Pixel size: each pixel ‘covers’ an area in the plane of vc x vr mm2, depending on field of • Image dimensions = Nc x Nr x Np, e.g. 256 x 256 x 160
view (FOV) and dimensions, e.g. FOV = 25cm, Nc = 256 è Vc = 250/256 = 0.97mm • Voxel size: vc x vr x vp (in mm3) depends on FOV and slice distance
• Pixel value: each pixel holds a number which is converted into greyscale value when • Caveat: voxel size >< resolution
displaying the image on screen (“image intensity”, “grey value”) – Voxel size = geometric property vc
• Image data: pixel values stored in 1D array row-by-row (= row-major) or column-by- Plane p vr
– Resolution = physical property v vp
p
column (= column-major, e.g. Matlab)

Row major Plane (p+1)

Nc • Data stored plane by plane: vp

Nr
...
1st plane 2nd plane last plane

1D data array (Nc x Nr x Np elements)

4D images: multi-temporal 4D images: multi-spectral

• 3 spatial dimensions + time as 4th dimension (= 3D images at different time points) • 3 spatial dimensions + channel as 4th dimension (e.g. RGB)
• Image dimensions = ncols x nrows x nplanes x ntimepoints • Image dimensions = ncols x nrows x nplanes x nchannels
• Created by acquisition, e.g. multi-slice cine cardiac MR images, 256 x 256 x 10 x 20 • Created by acquisition, e.g. PD and T2-weighted brain images, 256 x 256 x 40 x 2
• Generated by construction, e.g. aligned (= registered and resampled) follow-up images • Generated by construction, e.g. aligned (= registered and resampled) multi-
parametric MRI images

Figure 1 Serial MRI in a patient with relapsing–remitting multiple sclerosis

Wattjes, M. P. et al. (2015) MAGNIMS consensus guidelines on the use of MRI in multiple sclerosis—establishing disease prognosis and
monitoring patients, Nat. Rev. Neurol. doi:10.1038/nrneurol.2015.157
 Image data indexing: 2D Image data indexing: 3D
• Pixel at column c and row r in image I: I(c,r) (row-major) or I(r,c) (column-major)
• First pixel at index 0 or 1 by convention: I(0,0) (e.g. C++) or I(1,1) (e.g. Matlab) • I(c,r,p) = voxel at column c, row r and plane p in image I
• Index of element (c,r) in the data array: i = c + r*Nc (if row-major, first index = 0) • I(c,r,p) = I(i) with i = c + r*Nc + p*Nc*Nr (if row-major, first index = 0)
• Indexing neighbouring pixels: the index of each neighbour differs by a constant offset • Indexing neighboring voxels:
from that of the central pixel, independently of the position of that pixel in the image Absolute:
• Application: image filtering using local neighbourhood operators (e.g. c-1 c c+1 i1 = I(c-1,r+1,p-1) = (c-1)+(r+1)*Nc+(p-1)*Nc*Nr
convolution) r-1 i2 = I(c+1,r-1,p) = (c+1)+(r-1)*Nc+(p)*Nc*Nr
r
r+1
c Relative to voxel i = I(c,r,p):
c-1 c c+1 c-1 c c+1 p-1 i1 i1 = i + (-1 + Nc - Nc*Nr)
i i2
i2 = i + (1 - Nc)
r-1 i3 r-1 -1-n -n 1-n p in = i + Δc * Oc + Δr * Or + Δp * Op
r (Δc , Δr , Δp = offsets of in relative to i)
r i i2 r -1 0 1 p+1 Neighbour index offsets (row-major):
column direction: Oc = 1
row direction: Or = Nc
r+1 i1 r+1 -1+n n 1+n plane direction: Op = Nc* Nr
Index offsets
3x3 neighbourhood (row-major, n = Nc)

Data encoding Data types

• Numerical values in a computer are represented by a sequence of bytes: NAME REPRESENTATION RANGE
1 byte = sequence of 8 bits = the elementary computer unit, e.g. [10001111]
unsigned char, uint8 1 byte: no sign bit, 8 data bits [0…28-1] = [0…255]
• Decimal numbers are encoded in bytes using base 2 representation:
base 10: digits 0,1,2,3,4,5,6,7,8,9 ➔ 143 = 3 * 100
+4* +1* 101 102 char, int8 1 byte: 1 sign bit + 7 data bits -128,...,127
base 2: digits 0,1 ➔ 10001111 = 1*20+1*21+1*22+1*23+0*24+0*25 +0*26+1*27 = 143
e.g. 143 in base 10 = 10001111 in base 2 unsigned short, uint16 2 bytes: no sign bit, 16 data bits [0…216-1] = [0…65535]
e.g. 3445 in base 10 = 110101110101 in base 2 = [00001101] [01110101] in bytes
signed short, int16 2 bytes: 1 sign bit + 15 data bits -32268, ..., 32267
• Image data is encoded as an array of bytes:
number of bytes = (number of pixels) x (number of bytes per pixel) unsigned integer, uint32 4 bytes: 32 data bits [0…232-1] = [0…4294967295]
e.g. 256x256, 2 bytes per pixel = 256 x 256 x 2 = 131072 bytes = 1048575 bits
signed integer, int16 4 bytes: 1 sign bit + 31 data bits -231,...,231-1
• Number of bytes per pixel depends on the data type:
– Signed / unsigned, integer / floating point data types single precision floating 4 bytes: 1 sign bit, 8 bits exponent, e* = 2-23 = 1.19209289 x 10-7
– Limited number of bits ➔ discretisation ➔ limited floating point precision point (float) 23 bits mantissa
double precison floating 8 bytes: 1 sign bit, 11 bits e* = 2-52 = 2.22044604 x 10-16
point (double) exponent, 52 bits mantissa
* ε = relative accuracy = the distance from 1 to the next largest floating point number

Medical image data types Image file formats

• Native images (as acquired by the scanner): usually provided as 16 bit (= 2 byte) short
integer, i.e. [0…65535] (unsigned) or [-32768…+32767] (signed)
• Examples:
– 3D brain MRI scan: 256 x 256 x 160, uint16 è 20MB
– 4D cardiac MRI scan: 256 x 256 x 13 x 60, uint16 è 100MB
– 3D thorax CT scan: 512 x 512 x 600, uint16 è 300MB
– 3D micro-CT scan: 1000 x 1000 x 1000, uint16 è 2 GB
• Derived images resulting from intensity transformations, arithmetic, filtering,
interpolation, … would be floating point (single or double precision)
è often rounded to int16 or uint8 to save memory
e.g. If = int16(I1*w1+I2*w2+I3*w3+…+In*wn) (I = original, int16; If = filtered)
• Segmented image:
– Binary image: 0 (= background), 1 (= foreground) è uint8: 0, 255
– Label image: 0 (background), 1, 2, …, n (label assigned to each voxel) è uint8
– Probabilistic map: p ∈ [0..1] (floating point) = likelihood to belong to the object
è y = uint8(round(255*p)) ∈ [0,1,…,255]: 1 byte instead of 4, reduced precision
• Other quantities: floating point
– E.g. deformation field (non-rigid registration) = 3D vector field, e.g. MRI brain
image: 256 x 256 x 160 x 3 (X,Y,Z component), double è 250 MB
• In Radiology: DICOM standard, vendor independent (medical.nema.org)
– header with variable length + data array, int16
– stores information on patient, examination, image type, scan parameters, …
– public information (according to Nema standard) + private vendor-specific items
– facilitates communication of image information within PACS (Picture Archiving
and Communication System) between different modalities and workstations
– originally a 2D image format, each slice stored as separate file, requires sorting
the image slices into 3D stacks, not handy for 3D image analysis
• For medical image analysis:
– typically: separate header and data in 2 files (e.g. Analyze: .img, .hdr; Mevislab:
*.dcm, *.tif)
– alternatively: header followed by data in 1 file (e.g. NIfTI: .nii)
– header contains information on dimensions, data type, spatial location and
orientation, …
– 3D or 4D data stored in a single file
– conversion tools to convert from stack of DICOM files to 3D/4D image files
DICOM standard DICOM image elements

Tag: identifies data element

Tag = (group number, element number) = (2 bytes, 2 bytes) = 4 bytes
Even group number = public elements
Odd group number = private elements
VR = Value Representation, e.g. string, integer, double, sequence, … (2 bytes)
Value Length = length in bytes of Value Field

DICOM in Matlab Selecting a 3D region of interest

I(0,0,0)

Image “cropping”
c1 c2 I2 = I1(c1:c2,r1:r2,p1:p2)
I2(c,r,p) = I1(c1+c,r1+r,p1+p) (if first index = 0)
r1
r2 Image “downsampling”
p1 I2 = I1(c1:2:c2,r1:2:r2,p1:2:p2) (factor 2x2x2)

Image “upsampling” ?
I2 = I1(c1:0.5:c2,r1:0.5:r2,p1:0.5:p2)
p2 è Requires interpolation !
I(C-1,R-1,P-1)
 Orthogonal re-slicing of 3D images Anatomical orientations in 3D images
I(0,0,0) I(0,0,0) I(0,0,0) c
dimensions = C x R x P
r

I(:,:,p) I(:,r,:)
I(c,:,:)
p

I(C-1,R-1,P-1) I(C-1,R-1,P-1) I(C-1,R-1,P-1)

Radiological
convention

DICOM standard:
X: right to left (RàL) Plane X-Y = transversal / axial
Y: anterior to posterior (AàP) Plane Y-(—Z) = sagittal
Z: inferior (foot) to posterior (head) Plane X-(—Z) = coronal
(FàH or IàS)
Picture from www.vtk.org

Example: CT thorax Example: brain MRI

S P L

I A R
First index (c or X axis) Second index (r or Y axis) Third index (p or Z axis) R/A R/S A/S
R L Anterior Superior

L/P L/I P/I

axial / transverse cross-section coronal cross-section sagittal cross-section
Inferior
Posterior (coronal re-slice)
Native MRI slices can be acquired with arbitrary orientation: transverse,
coronal, sagittal, oblique (è orientation specified in DICOM header)
Native CT slices are "always/usually" axial slices (X = RàL, Y = AàP) Left side of the image = patient’s right side (radiological convention) in
Left side of the image = patient’s right side (radiological convention) transverse and coronal images
Slices sorted from patient’s inferior to superior (Z = IàS) Slices usually sorted from patient’s I to S (transverse slices), A to P (coronal
slices), R to L (sagittal slices)

Example: cardiac MRI DICOM world coordinates

• ImagePositionPatient: position of first voxel in DICOM world space (p)

• ImageOrientationPatient: orientation vectors of image row and column
direction (oc,or)
– Relative to the patient (invariant for supine or prone positioning)
– Supine or prone recorded in PatientPosition (e.g. HFS = head first supine)

Native MRI slices acquired with oblique orientation, aligned with the main
cardiac axes (long axis view, short axis view)
Orientation of the image planes can be retrieved from the DICOM header
(origin + direction vectors of column and row axis)
 Slice orientation and location Intensity operations
Op P = ImagePositionPatient (DICOM)
Oc,Or = ImageOrientationPatient (DICOM) • Histogram
z2 Op (Oc,Or = unit vectors)
Op = Oc x Or (cross product) • Colormap
P2 (Op = orthogonal to image plane) • Intensity windowing
z1 zi = Pi.Op (dot product)
z1 < z2 è slice 1 more inferior than slice 2 • Intensity thresholding
P1 Sort slices based on z • Intensity filtering
• Linear filtering (‘convolution’)
• Examples:
Origin – median filtering, smoothing, edge detection
Oc

Or Voxel size: vc, vr = PixelSpacing (DICOM)

Slice spacing: vp = (P2-P1).Op (dot product)
Note: P2-P1 needs not to be parallel with Op

Image histogram Colormap

h(I) = the number of voxels in the image with the same intensity value I, or with • Image display requires conversion from image values to screen intensity values
intensity in the range [I – ΔI/2, I + ΔI/2] (ΔI = binsize) • Image intensity range [Imin, Imax] is usually much larger than screen intensity range [0-255] ➔
Voxels with the same or similar intensity show up as a cluster in the histogram ‘colormap’ maps image values to screen ‘colors’ (or shades of gray)
Spatial information is lost… • Typically: linear mapping, specified by window width & center values
(1) I1 = Imin → black (0), I2 = Imax → white (255)
Window
(2) I1 = Center – Width/2 → black (0)
255 Center (2)
2 I2 = Center + Width/2 → white (255)
Intermediate image values → gray [1...254]: Gray
3 G = max(min(floor(255*(I – I1)/(I2 – I1)),255),0) scale
or (mapping I2+1 onto 255+1): (1) Width
2
1 3 G = max(min(floor(256*(I – I1)/(I2+1 – I1)),255),0) 0 I1 I2
• This will usually map different image values onto the same Imin Image value Imax
1
screen intensity value è quantisation è not all image information
can be visualised simultaneously G+2
1 = background Ig = (1 – G/256)*I1 + G/256*(I2+1)
Gray G+1
2 = white matter ΔI = Ig+1 – Ig = (I2+1 – I1)/256 = (Width+1)/256
scale G
3 = csf
ΔI
Ig Ig+1 Ig+2

Intensity windowing Example: thorax CT

Visualize / enhance the relevant objects in the image by manipulating the colormap,
without changing the image data itself
Lung
Assigns the full display range (256 values) to the intensity range of interest
Appropriate window/level selected interactively or by inspection of the image histogram
Colormap 1 Colormap 2
255 255

gray Fixed window settings

value used in radiology to
increase consistency
0 0 in visual interpretation
Imin Imax Imin Imax of the images
image value image value

Mediastinum
 Maximum intensity projection Maximum intensity projection
R L R L
S S

I I

A P A P
Example: MR angiography of the upper legs Example: MR angiography of the upper legs
First image: before MR contrast agent injection Second image: after MR contrast agent injection ➔ vessels enhanced

Maximum intensity projection Maximum intensity projection

x
R L
S

stenosis
Orthogonal MIP:
MIP(x,y) = max z I(x,y,z)

Oblique MIP:
MIPa(u,y) = max t I(x,y,z)
x = cos(a).u — sin(a). t
z = sin(a).u + cos(a). t
I

A P
Example: MR angiography of the upper legs
Subtraction image: non-enhancing structures suppressed
MIP: maximum intensity along the third dimension
➔ mapping 3D image onto 2D plane
➔ facilitates interpretation of connectivity of 3D structures

Maximum intensity projection Intensity thresholding

Histogram

Select all pixels with intensity larger than some

threshold It
➔ creates binary image J: J(I>It) = 1
= operator acting on each voxel independently

MIP: maximum intensity along the third dimension

➔ 3D perspective is lost It
➔ can be restored by animating projections in different directions Example: bone in CT
 Intensity thresholding Intensity filtering
Histogram From the image I, obtain a new image J by computing the value J(c,r) at pixel (c,r) in J as ‘some
function’ of I(c,r) and its (nearest) neighbors, ∀c,r (in 2D, or analogous in 3D)
= local operator, acting on each pixel (or voxel) neighborhood (‘filter running over the image’)
Neighbors that fall outside the image? ➔ ignore, zero-padding, mirroring, …

c-1 c c+1 c-1 c c+1

Select all pixels with intensity larger than a lower
threshold It1 and smaller than an upper threshold r-1 i1 i2 i3 r-1
It2
➔ J(I>It1 & I<It2) = 1 r i4 i i5 r j

r+1 i6 i7 i8 r+1

N(i)

Neighborhood N around pixel i in original image I Filtered image: j = J(c,r) = f(i, ik ∈ N(i))
It1 It2

Example: median filtering Linear filtering (‘convolution’)

Original c-1 c c+1 From the image I, obtain a new image J by computing the value J(c,r) at pixel (c,r) in J
as a weighted sum of I(c,r) and its nearest neighbors
r-1 i1 i2 i3 = discrete convolution operator (‘convolution filter’)
Neighbors that fall outside the image? ➔ ignore, zero-padding, mirroring, …
r i4 i i5 c-1 c c+1
c-1 c c+1

r+1 i6 i7 i8 r-1 i1 i2 i3 w1 w2 w3 r-1

Filtered
r i4 i0 i5 * w4 w0 w5 r j

3x3 neighborhood r+1 i6 i7 i8 w6 w7 w8 r+1

j = median(i,i1,i2,i3,…,i8)
E.g.: 3x3 neighborhood filter W filtered image J
Effect: noise suppression
The basic operation in
Deep Learning using CNN!

Example: Gaussian smoothing Edge enhancement

W = 5x5 filter, approximating a 2D Gaussian kernel with variance σ=2
è weighted average ‘Edges ‘ = local maxima of the gradient magnitude
è smoothing effect (‘blurring’) = difference in image intensity in row or
-1 -2 -1 -1 0 1 column direction
0.0232 0.0338 0.0383 0.0338 0.0232 W = 3 x3 Sobel edge filter = 0 0 0 or -2 0 2 = discrete approximation of X and Y
0.0338 0.0492 0.0558 0.0492 0.0338 1 2 1 -1 0 1 derivative of image I ➔ ‘gradient’
W= 0.0383 0.0558 0.0632 0.0558 0.0383 (analogous operators in 3D)
0.0338 0.0492 0.0558 0.0492 0.0338
0.0232 0.0338 0.0383 0.0338 0.0232 horizontal vertical
edges edges
Gradient magnitude
Original Filtered (‘edge strength’)
X

Original Filtered (‘smoothed’) Fy ≈ dI/dy Fx ≈ dI/dx F = sqrt(Fx2 + Fy2)

 Example: edge enhancement Edge enhancement
Differentiation also enhances the noise…
Practical ‘edge’ operators: smooth the image first before differentiation

W = Sobel ⊗ Gaussian smoothing filter

Example: 0.0232 0.0338 0.0383 0.0338 0.0232
-1 0 1 0.0338 0.0492 0.0558 0.0492 0.0338
W= -2 0 2 ⊗ 0.0383 0.0558 0.0632 0.0558 0.0383
0.0338 0.0492 0.0558 0.0492 0.0338
-1 0 1 0.0232 0.0338 0.0383 0.0338 0.0232
(W = 7x7 filter)

Can do both simultaneously è ‘derivative of Gaussian’ smoothing filter

Example:
0.0073 0.0053 0 -0.0053 -0.0073
0.0106 0.0077 0 -0.0077 -0.0106
Original image Gradient magnitude image Idem, different window 0.0121 0.0088 0 -0.0088 -0.0121
(W = 5x5 filter, s = 2)
è only strong edges visible è image noise results in 0.0106 0.0077 0 -0.0077 -0.0106
in this window many spurious edges 0.0073 0.0053 0 -0.0053 -0.0073

Higher order derivatives Example: image enhancement

1st order derivatives of Gaussian filter (for a certain s)
è d/dx(s), d/dy(s), d/dz(s)

2nd order derivatives of Gaussian filter

è d2/dx2, d2/dy2, d2/dz2, d2/dxdy, d2/dxdz, d2/dydz

Laplacian filter: L(s) = d2/dx2 + d2/dy2 + d2/dz2

Hessian matrix of second order derivatives: H(s)

d2/dx2 d2/dxdy d2/dxdz
d2/dxdy d2/dy2 d2/dydz
d2/dxdz d2/dydz d2/dz2

Eigenvalues of Hessian: l1, l2, l3

è Analysis of local intensity variation (‘curvature’)
è Discriminate between blob-like, plate-like, tubular-like structures
Application: vesselness filter: 0 ≈|l1| << |l2| and l2 ≈ l3 (= local tubularity)

Multi-scale: different values for s è ‘scale-space’ analysis (X,Y,Z,s)

Image coordinate system World coordinate system

X (mm) 0 vx 2*vx
c=0 1 2 X
X (pixels) x (mm)
x
0
r=0
Y

vy
(x,y)
Y (pixels)

1
(x,y)
2*vy
2 Z

y
y
Y (mm)

(mm) X

Y
(c,r,p start counting from 0,
x,y,z start counting from 0)
Pixel representation: pixel is tile on a plane, voxel is cube in 3D space
Spatial representation: pixel / voxel is centered around point in 2D / 3D space (c,r,p start counting from 0,
x,y,z start counting from 0)
➔ pixel / voxel coordinates = center of each pixel / voxel
(vx,vy,vz = voxel size)
Z
 Application: distance measurement 2D coordinate transformation (1)
X

(x0,y0) X’
p1(x1,y1)

p2(x2,y2)

Y
Y’

Y
translation
(move the origin of (x,y) from (0,0)
to (x0,y0))

2D coordinate transformation (2) 2D coordinate transformation (3)

Y’ Y Y’

X’

(x0,0) X (sx.x0,0) X’ X θ

(0,y0) (0,sy.y0)

scaling (sx,sy = scale factors) rotation

(switch axis orientation if scale (rotation over an angle of θ
factor is negative, e.g. sy=-1) radians from X to Y)

2D coordinate transformation (4) 2D coordinate transformation (5)

Y Y’ Y’

γ
X
X’ X’

(g = tan(γ)) affine transformation = translation + scale + rotation + skew

skew
(axes are no longer orthogonal) (putting it all together...)
 2D affine transformation 3D affine transformation

• The 2D affine coordinate transformation from (x,y) to (x’,y’) can be written in matrix
notation:

(‘Euler angles’,
different choices possible)
skew G rotation R scaling S translation T

A = G*R*S*T = 3x3 affine transformation matrix, defined by 6 parameters:

translation (x0,y0), scaling (sx,sy), rotation θ and skew g
p’ = A*p with p= (x,y,1) and p’ = (x’,y’,1) (‘homogeneous coordinates’)

• Alternative decompositions for A , e.g. A = T*R*G*S (no fixed definition)

Linear transformation

dp = [1 0 0 0]T ⇒ dqx = [A00 A10 A20 0] T

dp = [0 1 0 0]T ⇒ dqy = [A01 A11 A21 0] T po po+dp po+2*dp
X
dp = [0 0 1 0]T ⇒ dqz = [A02 A12 A22 0] T q0
qo+dq qo+2*dq
➔Allows for fast evaluation
using additions only X’
(no multiplications needed):
p0 = [0 0 0 1]T ⇒ q0 = [A03 A13 A23 1] T
+dqx when incrementing x (column nr)
+dqy when incrementing y (row nr)
Inverse formulas can be derived as well: (tx,ty,tz,sx,sy,sz,θx,θy,θz,gx,gy,gz) = f(Aij)
+dqz when incrementing z (plane nr)
Y’
Inverse transformation: A- 1 = (T*R*G*S)-1 (= matrix inversion)
Y

Linear transformations: special cases Image resampling (1)

X X’

• Translation: A = T (x0,y0)
2 parameters in 2D, 3 parameters in 3D θ
• Rigid body transformation: A = T* R
translation & rotation X’
3 parameters in 2D, 6 parameters in 3D Y’

• Similarity transformation: A = T*R*S with s = sx = sy = sz

translation & rotation & isotropic scaling
?
4 parameters in 2D, 7 parameters in 3D I’(x’,y’)
I(x,y)
Y Y’
• Full affine transformation: A = T*R*G*S
translation & rotation & scaling & skew
6 parameters in 2D, 12 parameters in 3D Problem:
given the image I(x,y), find the image I’(x’,y’) with p’(x’,y’,1) = A*p(x,y,1), i.e.
resample I along the axes X’, Y’ defined by the affine transformation A
 Image resampling (2) Image resampling (3)
X X’ X

column
A-1 c c+1
p(x’,y’)
p1 p3
r
p(x,y) p(x,y) row
p
r+1
! p2 p4

I’(x’,y’) p1, p2, p3, p4 are the 4 nearest neighbors of p on the

I(x,y) I(x,y) image grid of I
Y Y’ Y (8 nearest neighbors in 3D)

Solution: use the inverse transformation Problem: p(x,y) in general does not coincide with a pixel position of I
I’(x’,y’) = I(x,y) with p(x,y,1) = A-1 * p(x’,y’,1), ∀(x’,y’ ) ⇒ resampling requires interpolation to determine intensity of p
with A-1 the inverse transformation of A ⇒ spatial transformation implies intensity transformation
➔ run over I’, (inverse) transform (x’,y’) into (x,y), get I(x,y)

Image interpolation Linear interpolation

How to assign an intensity to p ?
The value at p is computed as a linear function of the intensities of its nearest neighbors

p1 p3 p1 p3 p1 p3

p(x,y) p(x,y) p(x,y) c = floor(x)

p1(r,c) p3(r,c+1) r = floor(y)
dx 1-dx
p2 p4 p2 p4 p2 p4 dx = x-c
w4 w2 dy = y-r
dy
p(x,y) w1 = (1-dx).(1-dy)
1-dy w3 w1 w2 = (1-dx).dy
nearest neighbor: linear: higher order w3 = dx.(1-dy)
(cubic, B-spline, sinc,…) p2(r+1,c) p4(r+1,c+1)
I(x,y) = I(p2) I(x,y) = Σ wk I(pk) w4 = dx.dy
use more neighbors, I(p) = w1.I(p1) + w2.I(p2) + w3.I(p3) + w4.I(p4)
different interpolation kernel
Ubiquitous in image processing and analysis: Analogous in 3D (8 NN; dx,dy,dz)
interpolation of image intensities, of gradient magnitude, of deformation fields, …

Application: image registration After image registration and resampling:

points at identical coordinates in both images refer to the same anatomical
positions
I1 I2

A?

(x,y,z) (x’,y’,z’)
(x,y,z) (x,y,z)

Problem: find the transformation A that maps points (x,y,z) in I1 to their

anatomically corresponding points (x’,y’,z’) in I2 I1(x,y,z) I2r(x,y,z) = I2(A(x,y,z))
 Application: arbitrary reslicing 3D connected cursor
3D image volume 2D slice

x x’
y

z y’
A

(A is 4x3 2D to 3D affine
3D interpolation transformation)

Application: curvilinear reslicing Non-linear coordinate transformations

MRA or
(MIP) extracted vessel

Deformation field (u,v,w) = 3D vector field on (x1,y1,z1)

= displacement from (x1,y1,z1) to (x2,y2,z2)
define new views along
vessel and orthogonal
to vessel axis X
(f,g,h)
(x1,y1,z1)

(u,v,w) X
crossection
(x1,y1,z1)
Y
(x2,y2,z2)

rectified vessel Graphical representation of a 2D deformation field,

Y deforming a uniform grid into a non-uniform grid

Deformation field Application: non-rigid matching

• Local, non-linear transformation (‘non-rigid’, ‘elastic’, ‘free form’), i.e. no global
transformation matrix Find the deformation field (u,v,w) that maps structures in image I1 onto the
➔ large number of degrees of freedom (= number of parameters) corresponding structures in image I2 : (x2,y2,z2) = (x1,y1,z1) + (u(.),v(.),w(.))
• Representations: see class on Non-rigid image registration
– discrete vector field = deformation vector in each voxel If (u,v,w) is known, I2 can be ‘deformed’ to I1 (= interpolated onto the grid of I1)
➔ 3 x N degrees of freedom (N = #voxels)
– continuous basis functions, e.g. B-spline or thin-plate spline function Image 1 Image 2 deformed to Image 1 Image 2

➔ degrees of freedom determined by the mesh on which the spline is defined

• Local behavior can be characterized by Jacobian matrix J (= local linearization)
– computed numerically (finite difference approximation) or analytically in case
of basis functions
• |det(J)| = determinant of Jacobian matrix
➔ measure of local stretching or shrinkage (‘volume change’)
det(J) > 1: local volume increase
det(J) < 1: local volume decrease
det(J) < 0: local folding è undesired…
• Physically plausible deformations require det(J)>0 I1(x1,y1,z1) I2’(x1,y1,z1) I2(x2,y2,z2)
➔ ‘one-to-one’, ‘diffeomorphic’ transformations
I2’(x1,y1,z1) = I2(x1+u(x1,y1,z1),y1+v(x1,y1,z1),z1+w(x1,y1,z1))
 Conclusion
• Image = N-D matrix (2D, 3D, 3D+time, …)
– Dimensions
– Data types
– Indexing
– Image format: DICOM
• Intensity transformations
– Histogram
– Color map
– Thresholding
– Filtering
– Convolution
• Smoothing
• Gradient
• Derivative operators
• Spatial transformations
– Image coordinates (in pixels / voxels)
– World coordinates (in mm)
– Linear coordinate transformations (‘affine transformation matrix’)
– Non-linear coordinate transformations (‘deformation field’)
– Resampling
– Interpolation