Máster Universitario en Ingeniería Biomédica

Advanced Methods In Medical Signals and Images

Segmentation of Medical Images

Juan Ortuño Fisac
je.ortuno@upm.es

Biomedical Imaging Technology (BIT) www.die.upm.es/im

Departamento de Ingeniería Electrónica, ETSIT-UPM

TSIM (2022-2023) - Segmentation 1

of medical images
 Outline

• Outline
– Segmentation of medical images
– Classical methods
– Morphological methods
– 1st exercise, image segmentation
– Active contours
– Graph-based methods
– Active shape models (ASM)
– Machine learning: Clustering
– 2nd exercise, clustering segmentation

TSIM (2022-2023) - Segmentation of medical images 2

 Model-based segmentation

• Segmentation by matching a model which contains information about the

expected shape and appearance of the structure of interest
– Most structures of clinical interest have a characteristic shape and relative
location
– Can we use that information to properly segment the organ, even with noise or
missing boundaries?

Manual segmentation (red)

Model-based segmentation (green)

TSIM (2022-2023) - Segmentation of medical images 3

 Model-based segmentation

• Proposed methods:
– Active shape models
– Statistical shape models
– Appearance shape models
– Oriented active shape models
– Morphable models ….

T. Heimann, and H. P. Meinzer, “Statistical shape models for 3D medical image

segmentation: A review,” Medical Image Analysis, vol. 13, no. 4, pp. 543-563,
2009.

TSIM (2022-2023) - Segmentation of medical images 4

 Model-based segmentation

• Shape:
• Representation of an object or its external boundary, outline, or external
surface
• In model-based segmentation, shapes are defined using connected set of
points (Explicit definition)

2-D Surface mesh

(e.g., face recognition) 3-D
(mesh)

TSIM (2022-2023) - Segmentation of medical images 5

 Model-based segmentation

• The shape of an organ can be characterized as being a transformation of

some template
• Build a statistical boundary shape model that consists of a mean shape and
its allowable variations

TSIM (2022-2023) - Segmentation of medical images 6

 Model-based segmentation

Steps of the model-based segmentation workflow

a) Model definition
b) Training step
c) Segmentation (matching) step

TSIM (2022-2023) - Segmentation of medical images 7

 Active Shapes Models (ASM) - Model definition

• A shape q is represented by a chosen set of points p

(features) each of which is a k-dimensional vector
• N points form a vector of length k*N:
q = [ p1 ,..., pN ]T pi = (xi , yi ) (2 dimensions)
• Training set of M shapes  q1 , q2 ,..., qM 

q1 q2 q3
TSIM (2022-2023) - Segmentation of medical images 8
 Active Shape Models (ASM) - Training

1) Alignment
– Keep the model as specific as possible, Invariant against translations
and rotations
– Procrustes alignment
• Removes translational component, scaling component, and
rotational component.
• Minimizes the mean squared distance between two shapes
• Run iteratively to align a group of shapes, resulting in the GPA

TSIM (2022-2023) - Segmentation of medical images 9

 Active Shape Models (ASM) - Training

2) Dimensionality reduction
Principal component analysis (PCA)
1 M
Mean shape: q = 
M i =1
qi
1 M
 ( i )( i )
T
Covariance matrix: S = q - q q - q
M − 1 i =1

• Compute the eigenvectors u and eigenvalues λ of S, sorting them in

decreasing order of eigenvalues
• Remove small eigenvalues, retaining most of the “variation” with L
eigenvalues of M
L M

   0.98   
i =1
i
i =1
i

TSIM (2022-2023) - Segmentation of medical images 10

 Active Shape Models (ASM) - Training

3) Model variation:
• Approximate any instance of the shape by projecting onto the
first L eigenvectors u L
q = q +  bi ui b =  b1 ,...,bL 
T

i =1

• The vector b identify the characteristic of the shape instance.

• The weights bi enables us to explore the variations in the
shape.

– L eigenvectors u: Principal modes of variation

– L Eigenvalues: Variances

TSIM (2022-2023) - Segmentation of medical images 11

 Modes of variation

• Examples:

Brain structures

TSIM (2022-2023) - Segmentation of medical images 12

 Modes of variation

Lung shapes in x-rays/CTs

TSIM (2022-2023) - Segmentation of medical images 13

 Model matching

• ASM fitting algorithm

Model points

Strongest edges

Searching for local match for each point of the model:

• Strongest edge
• Correlation
• Statistical model of profile

TSIM (2022-2023) - Segmentation of medical images 14

 Model matching

Not all points have to fit

properly for the ASM to
work!

A subset that fits well

with the learned model
will be suffice

TSIM (2022-2023) - Segmentation of medical images 15

 Model matching

Suppose we have identified a set of points y in the image:

The goal is to minimize the squared distance of:
L 2

y − T (q +  bi ui )
i =1

Algorithm:
1. Initialize b=0 L

2. Generate model instance q = q +  bi ui

i =1
3. Find transformation T (e.g., rigid) that best aligns q to y
−1
4. Invert T to project y into model frame: m y = T (y)
5. Update the model parameters: b = UT (y m − q)
6. Repeat from step 2 until convergence
TSIM (2022-2023) - Segmentation of medical images 16
 Model matching

Rigid transformation using Update modes

point-based-registration

TSIM (2022-2023) - Segmentation of medical images 17

 Heart segmentation using ASM

Grbic S, et al. (2012) 'Complete valvular heart apparatus model from 4D cardiac CT', Medical
Image Analysis, 16(5), 1003-1014.

TSIM (2022-2023) - Segmentation of medical images 18

 Liver segmentation using ASMs and active contours

• An additional shape term

(using ASM) guides the
evolution of the active
contour

• Red: Active contour

• Blue: shape model

M. Rousson, N. Paragios, R. Deriche. Implicit Active Shape Models for 3D Segmentation in

MR Imaging. MICCAI 2004, 3216:209-216, 2004

TSIM (2022-2023) - Segmentation of medical images 19

 ASMs and Deep Learning

TSIM (2022-2023) - Segmentation of medical images 20