Goals

Understand the fundamentals of digital image

Image Segmentation and Pre- processing.
processing Define image enhancement and the different types
commonly used in medical/surgical applications.
UUnderstand
de sta d the
t e process
p ocess of
o image
age segmentation
seg e tat o and
a d its
ts
relevance in manipulation of medical images.
Vincent Luboz
Enumerate the most commonly used image
Department of Surgery and Cancer segmentation techniques indicating their main
p
Imperial College
g London characteristics and advantages/disadvantages.
Mention some possible applications of image
segmentation in surgery.
MSc Surgical Technology Slide 1 MSc Surgical Technology Slide 2
Manipulation Contents

Selection of region of interest Pre-processing

Pre processing
Image resampling Segmentation
Greyscale contrast enhancement Applications
Pre-processing Summary and Conclusion
Segmentation

MSc Surgical Technology Slide 3 MSc Surgical Technology Slide 4

Pre-processing Pre-processing

Goal: Image resampling

- Enhance the visual appearance of images. Greyscale contrast enhancement
- Improve the manipulation of datasets. Noise removal
Mathematical operations
C
Caution:
i enhancement
h techniques
h i can emphasize
h i Manual correction
image artefacts, or even lead to a loss of information
if not correctly used.

MSc Surgical Technology Slide 5 MSc Surgical Technology Slide 6

Pre-processing Noise removal

Image resampling: Several techniques:

- Reduce or increase the number of pixels of the dataset. - Low-pass, high-pass, band-pass spatial filtering
- Mean filtering
Greyscale contrast enhancement: - Median filtering
- Improve
I the
th visualisation
i li ti by b brightening
b i ht i theth dataset.
d t t

MSc Surgical Technology Slide 7 MSc Surgical Technology Slide 8

Noise removal Noise removal

Low-pass
Low pass filtering replaces all pixels of intensity High-pass
High pass filtering replaces all pixels of intensity
higher than a specified value. lower than a specified value.
Example: Band-pass filtering replaces all pixels of intensity
lower than a specified value and higher than another
one.
one

Low, high-pass, band-pass spatial filtering are

Mask efficient only in specific cases.
Noisy image Low-pass filtered Most of the time, blur the image
image
MSc Surgical Technology Slide 9 MSc Surgical Technology Slide 10
Noise removal Noise removal

Mean filtering and median filtering work on a n x n Mean filtering:

sub-region of the image. - The 3x3 sub-region is scanned over the entire image
n is usually 3 or 5. - At each position the centre pixel is replaced by the average
value.
Example on a 4x4 sub-image:

Raw sub-image 3x3 average value

MSc Surgical Technology Slide 11 MSc Surgical Technology Slide 12
Noise removal Noise removal

Median filtering: Mean filtering applied to the image with a 3x3 sub
sub-
- The 3x3 sub-region is scanned over the entire image region:
- At each position the centre pixel is replaced by the median
value.

Raw sub-image 3x3 average value

MSc Surgical Technology Slide 13 MSc Surgical Technology Slide 14 Mean filtered
Noise removal Noise removal

Median filtering applied to the image with a 3x3 sub

sub- Mean filtering:
region: - Fast to compute.
- Blurs edges.
- Smears noise specks.

Median filtering:
- Slower to compute.
- Preserves edges.
- Can remove noise.

MSc Surgical Technology Slide 15 Median filtered MSc Surgical Technology Slide 16
Mathematical operations Mathematical operations

It is possible to apply to images: Addition is not very helpful

- Arithmetic operations (addition, subtraction). Subtraction can be used to eliminate confusing
- And morphological operations (dilation, erosion). background detail which has remained unchanged
between the two images
Goal:
G l to enhance
h particular
i l features
f done
d pixel-by
i l b pixel
i l
Operations between two images are only useful if the
images can be aligned closely enough
Often used for x-rayy contrast angiography
g g p y to highlight
g g
occluded arteries
Can also be used to show changes over time
MSc Surgical Technology Slide 17 MSc Surgical Technology Slide 18
Mathematical operations Mathematical operations

Example of subtraction: cerebral volume changes in Dilation is used to connect features in an image
dementia

Structural element:

MSc Surgical Technology Slide 19 MSc Surgical Technology Slide 20

Mathematical operations Mathematical operations

Erosion is used to disconnect features in an image and It is possible to change the structural element to adjust
remove small ones the operators:
- Different shapes.
- Different sizes.
It is possible to combine dilation and erosion to
combine their effects:
- Dilation followed by erosion = Closing.
- Erosion followed by dilation = Opening.
- Both
B h tendd to smoothh the
h images
i features.
f
Structural element:

MSc Surgical Technology Slide 21 MSc Surgical Technology Slide 22

Manual correction Manual correction

Goal: fine tune an image by editing it.

it Example of line editing: separating the liver from the
Editing can be done: ribs using a 3D spline
- Pixel by pixel.
- Using lines or splines.
- Using
U i predefined
d fi d 2D or 3D shapes
h ((rectangle,
t l brick,
bi k
sphere).

MSc Surgical Technology Slide 23 MSc Surgical Technology Slide 24

Contents Segmentation

Pre-processing
Pre processing Needed for:
Segmentation - Improving the analysis of an image when there is no direct
correspondence between the image pixel properties and the
Applications type of tissue.
Summary and Conclusion p
- Separatingg (labelling)
( g) the pixels
p of an image
g accordingg to
semantic content (studied structure).
- Facilitating the manipulation and visualization of the data
with a computer.

MSc Surgical Technology Slide 25 MSc Surgical Technology Slide 26

Segmentation
Involves the partitioning of an image or volume into
distinct (usually) non-overlapping regions in a
meaningful way.
Can also be thought of as a labelling operation: a
label corresponding to tissue type/anatomical
structure is assigned to each pixel or voxel in the
image.
MSc Surgical Technology Slide 27
Segmentation
Identifies separate objects within an image.
image
Finds regions of connected pixels with similar
properties.
Finds boundaries between regions.
Removes unwanted regions.
MSc Surgical Technology Slide 28
Segmentation Segmentation

Simple example: segmentation of rice grains Types of image segmentation

- Image domain:
Manual.
Thresholding.
Region growing
growing.
Hierarchical.
- Feature domain:
Original image Segmented (binary) image Supervised segmentation.
Each pixel is assigned a label: Unsupervised
U i d segmentation.
t ti
0 = not rice grain pixel
1 = rice grain pixel
MSc Surgical Technology Slide 29 MSc Surgical Technology Slide 30
Segmentation Segmentation

Manual segmentation Manual segmentation

- Outlines the studied structure in each slice. - Example of aorta segmentation with a spline:
- Only on the contour or on the whole object.
- Lines or splines can be used.
- Usually time consuming
consuming.

The spline delineates Once applied, the

the contour contour pixels are
MSc Surgical Technology Slide 31 MSc Surgical Technology Slide 32
highlighted
Segmentation Segmentation

Thresholding Thresholding
- Relies on intensity differences between structures in an - Example of aorta segmentation in CTA.
image. - Big intensity difference between bone and soft-tissue, easy
- Can be extended to multiple threshold levels. to partition into:
- Advantage: simple to implement Bones
Bones,
- Disadvantages: Vessels,
Low tolerance to intensity rescaling, Other soft
Difficult to set threshold, tissues.
Little use of spatial information
information.

MSc Surgical Technology Slide 33 MSc Surgical Technology Slide 34

Segmentation Segmentation

Region growing Region growing

- Relies on intensity differences, but includes the notion of - Example of aorta segmentation in CTA:
spatial proximity of pixels, and a seed point for the region. First, a probability map
- Advantages: is built to separate roughly
Simple to implement,
implement the structures.
structures
Human interaction is easy to provide (via seed point). Then seeds are placed
- Disadvantages: in the studied structure.
Low tolerance to intensity rescaling, Finally, the region is
Difficult to set growing criteria and stopping criteria,
criteria growing
i to t fit the
th structure.
t t
Needs human intervention for defining seed point.

MSc Surgical Technology Slide 35 MSc Surgical Technology Slide 36

Segmentation Segmentation

Hierarchical segmentation Hierarchical segmentation

- Clusters image pixels into regions of similar intensity to - Advantages:
create an intensity hierarchy. Fast,
- Marking seeds inside and outside the desired structure Reasonably easy to implement.
g g of the hierarchy.
starts the merging y - Disadvantages:
- Iteratively separates the inside and outside of the structure. Medium tolerance to intensity rescaling,
Needs human intervention for defining seed points.

MSc Surgical Technology Slide 37 MSc Surgical Technology Slide 38

Segmentation Segmentation

Hierarchical segmentation Feature domain segmentation

- Example of aorta segmentation in CTA:
First, the intensity hierarchy is built to pre-separate the
structures.
Then seeds are placed in and out the studied structure.
structure

MSc Surgical Technology Slide 39 MSc Surgical Technology Slide 40 Courtesy of Dr. F. Bello, Dept. of Biosurgery
Segmentation Segmentation

Two types of feature domain segmentation: Feature domain segmentation:

- Supervised: a set of training data is given, a learning - Advantages:
algorithm uses this to determine a classification rule for . Very powerful,
new data. . Tremendously flexible.
p
- Unsupervised: algorithms
g attempt
p to discover clusters (or
( - Disadvantages:
groups of data points) in feature space.
. Generates increased computation (because each pixel is
mapped to N pixels),
. Not obvious what features should be used,
. Large feature spaces require lots of data (for automated
learning) or training examples (for supervised learning).

MSc Surgical Technology Slide 41 Courtesy of Dr. F. Bello, Dept. of Biosurgery MSc Surgical Technology Slide 42 Courtesy of Dr. F. Bello, Dept. of Biosurgery
Contents Applications

Pre-processing
Pre processing
Quantitative, or semi-quantitative diagnostic image
Segmentation analysis.
Applications Surgical planning.
Summary and Conclusion Computer assisted surgery.

MSc Surgical Technology Slide 43 MSc Surgical Technology Slide 44

Applications Applications

Diagnostic analysis Diagnostic analysis

- Patient come with headache, visual troubles, and speech - CT scan of the brain shows a tumour:
difficulties.
- Diagnosis?

Tumour

MSc Surgical Technology Slide 45 MSc Surgical Technology Slide 46

Applications Applications

Diagnostic analysis Surgical planning

- Segmentation and 3D rendering reveals the tumour size - Diagnosis: aortic aneurisms
and influence:

Tumour

MSc Surgical Technology Slide 47 MSc Surgical Technology Slide 48

Applications Applications

Surgical planning Surgical planning

- Diagnosis: aortic aneurisms - Interventional Radiology to deploy stents to stabilise the
- How to treat the patient? aneurysms.
- First, need to know the exact size of the aneurysms and
g instruments.
choose the right

MSc Surgical Technology Slide 49 MSc Surgical Technology Slide 50

Applications Applications

Surgical planning Surgical planning

Inner diameter: Outer diameter:

31mm 60mm

Original CTA with superimposed segmentation

MSc Surgical Technology Slide 51 Original CTA with superimposed segmentation
MSc Surgical Technology Slide 52
Applications Applications

Computer assisted surgery Computer assisted surgery

- Da Vinci robot heart surgery - Da vinci robot heart surgery

Organ segmentation

Augmented surgery (real surgery with

an overlay of the virtual organs)
MSc Surgical Technology Slide 53 MSc Surgical Technology Slide 54
Contents Summary

Pre-processing
Pre processing We have seen:
Segmentation - Key points of digital image processing.
Applications - Definition of image enhancement and some
medical/surgical applications.
Summary and Conclusion - Overview of image segmentation.
segmentation
- Introduction to the most common image segmentation
techniques.
- Three possible applications in surgery.

MSc Surgical Technology Slide 55 MSc Surgical Technology Slide 56

Conclusion

Medical imaging is very powerful on its own, but not

always intuitive. Image Segmentation and Pre-
Pre-processing and segmentation are key techniques: processing
- To improve the various imaging modalities.
p
- To allow interpretation for better diagnosis.
g
- To integrate in planning and training software.
Segmentation is a fast evolving field but there is still a
Vincent Luboz
lot to do: Department of Surgery and Cancer
- Completely automatic. p
Imperial College
g London
- Motion compensation.
-

MSc Surgical Technology Slide 57 MSc Surgical Technology Slide 58