6.

8 Noise In Color Images

6.9 Color Image
Compression

Noise In Color
Images

Limitation of Imaging Technology

Two plagues in image acquisition
Noise interference
Blur (motion, out-of-focus, hazy weather)
Difficult to obtain high-quality images as

imaging goes
Beyond visible spectrum
Micro-scale (microscopic imaging)
Macro-scale (astronomical imaging)

What is Noise?
Wiki definition: noisemeans any unwanted

signal
One persons signal is another ones noise
Noise is not always random and
randomness is an artificial term
Noise is not always bad (see stochastic
resonance example in the next slide)

Stochastic Resonance

no noise

light noise

heavy noise
5

Image Denoising

Where does noise come from?

Sensor (e.g., thermal or electrical

interference)
Environmental conditions (rain, snow
etc.)

Why do we want to denoise?

Visually unpleasant
Bad for compression
Bad for analysis
6

Noisy Image Examples

thermal imaging

electrical interference

physical interference
ultrasound imaging

(Ad-hoc) Noise Modeling

Simplified assumptions
Noise is independent of signal
Noise types
Independent of spatial location
Impulse noise
Additive white Gaussian noise

Spatially dependent
Periodic noise

Noise Removal
Techniques
Linear filtering
Nonlinear filtering

Recall

Linear system

Image Denoising

Introduction
Impulse noise removal

Additive white Gaussian noise

removal

Median filtering

2D convolution and DFT

Periodic noise removal

Band-rejection and Notch filter

10

Impulse Noise (salt-pepper

Noise)
Definition
Each pixel in an image has the probability of p/2 (0<p<1) being
contaminated by either a white dot (salt) or a black dot (pepper)

Y (i, j )

255
0
X (i, j )

1 i H ,1 j W

with probability of p/2

with probability of p/2
with probability of 1-p

noisy pixels
clean pixels

X: noise-free image, Y: noisy image

Note: in some applications, noisy pixels are not simply black or white,
which makes the impulse noise removal problem more difficult
11

Numerical Example
P=0.1
128 128 128 128 128 128 128 128 128
128 128 255
0 128 128 128 128 128
128
128
128 128 128 128 128 128 128 128 128
128 128 128 128
0 128 128 128 128
128
0
128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128
128
128
128 128 128 128 128 128 128 128 128
128 128
0 128 128 128 128 128 128
128
128
128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128
128
128
128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 128 128
128
128
128 128 128 128 X128 128 128 128 128
128 128 128 128 Y128 128 128 128 128
128
128
128
128
128
128
128
128
128
128
128
0
128
128of128
128 are
255 contaminated
128 128 128
Noise level p=0.1 means that approximately
10%
pixels
128
128
salt
pepper
noise
by red 128
color)
128or128
128 128
128 (highlighted
128 128 128 128
128 128 128 128 128 128 128 128
128
255
128 128 128 128 128 128 128 128 128
128 128 128 128 128 128 128 255 128
128
128
12

MATLAB Command
>Y = IMNOISE(X,'salt & pepper',p)
Notes:
The intensity of input images is assumed to be normalized to [0,1].
If X is double, you need to do normalization first, i.e., X=X/255;
If X is uint8, MATLAB would do the normalization automatically
The default value of p is 0.05 (i.e., 5 percent of pixels are contaminated)
imnoise function can produce other types of noise as well (you need to
change the noise type salt & pepper)

13

Impulse Noise Removal Problem

filtering
algorithm

Noisy image Y

denoised ^
X
image

^
Can we make the denoised image X as close
to the noise-free image X as possible?

14

Median Operator
Given a sequence of numbers {y1,,yN}
Mean: average of N numbers
Min: minimum of N numbers
Max: maximum of N numbers
Median: half-way of N numbers

y
[50,0,52,255,54,55,56]
Example

y
sorted [0,50,52,54,55,56,255]

median( y ) 54
15

1D Median Filtering
y(n)

W=2T+1

x (n) median[ y ( n T ),..., y (n),..., y (n T )]

MATLAB command: x=median(y(n-T:n+T));
Note: median operator is nonlinear

16

Numerical Example

y [50,T=1:
0,52,255,54,55,56]
Boundary
Padding

y [50,50,0,52,255,54,55,56,56]

x [50,50,52,54,54,55,55]

17

2D Median Filtering
x(m,n)

W: (2T+1)-by-(2T+1) window

x (m, n) median[ y (m T , n T ),..., y (m T , n T ),...,

y (m, n),..., y (m T , n T ),..., y (m T , n T )]
MATLAB command: x=medfilt2(y,[2*T+1,2*T+1]);

18

Numerical Example
225
226
225
226
226
255
255
226
225
255
255
226
226
228
226
226

225 225 226 226 226 226

225 255 226 226 226 225
226 225 226
226 225
255

0 226 226

0 226 226 226

0 225 226 226 226

225 224

Y
226

226

0 225

0
225
225
226
226
225
225
225
226

225
225
226
226

225
226
226
225

226
226
226
225

226
226
226
226

226
226
226
226

226 226
226 226
226 226
226

225
225
225
226

225
225
225
226

225
226
226
226

226
226
226
226

226
226
226
226

226
226
226
226

^
X

226
226
226
226

225 225 226 255 226 226

226 225 226 226 226 226

Sorted: [0, 0, 0, 225, 225, 225, 226, 226, 226]

19

Image Example
P=0.1

Noisy image Y

denoised ^
X
image
3-by-3 window
20

Image Example (Cont)

noisy
(p=0.2)

clean

3-by-3 window

5-by-5 window
21

Reflections
What is good about median operation?
Since we know impulse noise appears as black
(minimum) or white (maximum) dots, taking
median effectively suppresses the noise
What is bad about median operation?
It affects clean pixels as well
Noticeable edge blurring after median filtering

22

Idea of Improving Median Filtering

Can we get rid of impulse noise without

affecting clean pixels?

Yes, if we know where the clean pixels are

or equivalently where the noisy pixels are

How to detect noisy pixels?

They are black or white dots

23

Median Filtering with Noise

Detection
Noisy image Y

Median filtering
x=medfilt2(y,[2*T+1,2*T+1]);

Noise detection
C=(y==0)|(y==255);
Obtain filtering results
xx=c.*x+(1-c).*y;
24

Image Example

clean

w/o
noise
detection

noisy
(p=0.2)

with
noise
detection
25

Image Denoising

Introduction
Impulse noise removal

Additive white Gaussian noise

removal

Median filtering

2D convolution and DFT

Periodic noise removal

Band-rejection and Notch filter

26

Additive White Gaussian Noise

Definition
Each pixel in an image is disturbed by a Gaussian random variable
With zero mean and variance 2

Y (i, j ) X (i, j ) N (i, j ),

N (i, j ) ~ N (0, 2 ),1 i H ,1 j W
X: noise-free image, Y: noisy image
Note: unlike impulse noise situation, every pixel in the image contaminated
by AWGN is noisy
27

Numerical Example
128
128
128
128
128
128
128
128
128
128
128
128
128
128
128
128

128 128 128 128 128 128

128 128 128 128 128 128
128 128 128 128 128 128
128 128 128 128 128 128
128 128 128 128 128 128

2 =1

128
126
128
128
127
129
129
128

128
128
128
129
127
130
128
128

129
128
128
127
128
127
129
130

127
129
129
126
127
129
128
129

129
129
129
129
129
127
128
128

126
128
127
129
127
129
128
127

126
128
127
129
129
130
129
127

128
127
128
128
128
128
129
126

128 128 128 128 128 128

128 128 128 128 128 128
128 128 128 128 128 128

28

MATLAB Command
>Y = IMNOISE(X,gaussian',m,v)
or

>Y = X+m+randn(size(X))*v;
rand() generates random numbers uniformly distributed over [0,1]
Note:

randn() generates random numbers observing Gaussian distribution

N(0,1)
29

Image Denoising

filtering
algorithm

Noisy image Y

denoised ^
X
image

Question: Why not use median filtering?

Hint: the noise type has changed.

30

1D Linear Filtering
f(n)

g ( n)

h(n)

g(n)
See review section

h( k ) f ( n k ) h( n) f ( n) f ( n) h( n)

Linear convolution

- Linearity a1 f1 (n) a2 f 2 (n) a1 g1 (n) a2 g 2 ( n)

- Time-invariant property

f (n n0 ) g (n n0 )

31

Fourier Series

forward

F ( w) f (n)e jwn

inverse

1
f ( n)
2

time-domain convolution

f ( n) h( n)

F ( w)e jwn dw
frequency-domain multiplication

F ( w) H ( w)

Note that the input signal is a discrete sequence

while its FT is a continuous function

32

Filter Examples
Low-pass (LP)

|H(w)|
2

h(n)=[1,1]

1.8

HP

LP

1.6
1.4

|h(w)|=2cos(w/2)

1.2
1

High-pass (LP)

0.8
0.6

h(n)=[1,-1]

0.4
0.2
0

|h(w)|=2sin(w/2)

0.5

1.5

2.5

3.5

w
33

1D Discrete Fourier Transform

forward transform
N 1

y (k ) x(n)WNkn
n0

Properties
- periodic
Proof:

inverse transform
N 1

x(n) y (k )WN kn
n 0

y (k N ) y (k )
N 1

N 1

n 0

n 0

WN exp{

j 2
}
N

y (k N ) xnWN( k N ) n xnWNkn y (k )
*
y
(
N

k
)

y
(k )
- conjugate symmetric

Proof:

N 1

N 1

n 0

n 0

y ( N k ) xnWN( N k ) n xnWN kn y * (k )
34

Matrix Representation of 1D
DFT
A NN

a11
...

... ... a1N

... ... ...

... ... ... ...

a
...
...
a
NN
N1

1
akl
WNkl ,
N
WN e

2
N

, WNN 1

yk akl xl

Im

l 1

DFT:

1
yk
N

Re

xW
l 1

kl
N

WN

35

Fast Fourier Transform

(FFT)*
Invented by Tukey and Cooley in 1965
Basic idea: divide-and-conquer
N 1

yk xnWNkn
n 0

N-point DFT

k 2m
x
W
2m N

n2m

km
k
x
W

W
2m N / 2 N

n2m

N/2-point DFT

k ( 2 m 1)
x
W
2m1 N

n 2 m 1

km
x
W
2m1 N / 2

n 2 m 1

N/2-point DFT

Reduce the complexity of N-point DFT from

O(N2) to O(Nlog2N)

36

Filtering in the Frequency

Domain
f(n)

h(n)

g(n)

g ( n) f ( n) h( n)

F(k)

DFT

H(k)

G(k)

G (k ) F (k ) H (k )

convolution in the time domain is equivalent to

multiplication in the frequency domain

37

2D Linear Filtering
f(m,n)

g (m, n)

h(m,n)

g(m,n)

h(k , l ) f (m k , n l ) h(m, n) f (m, n)

k ,l

2D convolution
MATLAB function: C = CONV2(A, B)

38

2D Filtering=Two
Sequential 1D Filtering
Just as we have observed with 2D

transform, 2D (separable) filtering can be

viewed as two sequential 1D filtering
operations: one along row direction and the
other along column direction
The order of filtering does not matter

h(m, n) h1 (m) h1 (n) h1 (n) h1 (m)

h1 : 1D filter
39

Numerical Example
1D filter

h1(m)=[1,1], h1(n)=[1,-1]

h1 (m) h1 (n)

h1 (n) h1 (m)

1 1

1 1

MATLAB command:
>h1=[1,1];h2=[1,-1];
>conv2(h1,h2)
>conv2(h2,h1)

40

Fourier Series (2D case)

F ( w1 , w2 )

f (m, n)e j ( w1m w2 n )

m n

spatial-domain convolution

f (m, n) h(m, n)

frequency-domain multiplication

F ( w1 , w2 ) H ( w1 , w2 )

Note that the input signal is discrete

while its FT is a continuous function

41

Filter Examples
|h(w1,w2)|
Low-pass (LP)
h1(n)=[1,1]
1D
|h1(w)|=2cos(w/2)

h(n)=[1,1;1,1]
2D
|h(w1,w2)|=4cos(w1/2)cos(w2/2)

w2
w1
42

Image DFT Example

Original ray image X

choice 1: Y=fft2(X)
43

Image DFT Example

(Cont)

choice 1: Y=fft2(X)

Low-frequency at four corners

choice 2: Y=fftshift(fft2(X))
Low-frequency at the center

FFTSHIFT Shift zero-frequency component to center of spectrum.

44

Gaussian Filter
FT

h(m, n) exp(

m2 n2
2

w12 w22
H ( w1 , w2 ) exp(
)
2
2

MATLAB code:>h=fspecial(gaussian, HSIZE,SIGMA);

45

Image Example
noisy

PSNR=20.2dB
(=25)

denoised

denoised

PSNR=24.4dB
(=1)

PSNR=22.8dB
(=1.5)

Matlab functions: imfilter, filter2

46

Gaussian Filter=Heat
Diffusion
Linear Heat Flow Equation:

I ( x, y, t )
2 I ( x, y , t ) 2 I ( x, y , t )
I ( x, y, t )

2
t
x
y 2

Isotropic diffusion:I ( x, y, t ) I ( x, y,0) G (t )

scale

A Gaussian filter
with zero mean
and variance of t

47

Basic Idea of Nonlinear Diffusion*

y I(x,y)

x
image I
Diffusion should be anisotropic
instead of isotropic
image I viewed as a 3D surface (x,y,I(x,y))
48

Experimental Results
noisy

PSNR=20.2dB
(=25)

linear diffusion

PSNR=24.4dB
(Gaussian filtering)

nonlinear diffusion

PSNR=27.5dB
(TV filtering)
49

Image Denoising

Introduction
Impulse noise removal

Additive white Gaussian noise

removal

Median filtering

2D convolution and DFT

Periodic noise removal

Band-rejection and Notch filter

50

Periodic Noise
Source: electrical or electromechanical

interference during image acquistion

Characteristics
Spatially dependent
Periodic easy to observe in frequency domain

Processing method
Suppressing noise component in frequency

domain

51

Image Example

spatial

Frequency (note the four pairs of bright dots)

52

Band Rejection Filter

W
W
2
2
0 D w1 w2 D
H ( w1 , w2 )
2
2
1
otherwise
w2

w1

53

Image Example

Before filtering

After filtering

54

Advanced Denoising Techniques*

N I
W I

E I

s I

N I I i 1, j I i , j
S I I i 1, j I i , j
E I I i , j 1 I i , j
W I I i , j 1 I i , j
cd g (|| d I ||), d N , S , E , W

I it,j1 I it, j [c N N I cS S I cE E I cW W I ]
Basic idea: from linear diffusion (equivalent to Gaussian filtering)
to nonlinear diffusion (with implicit edge-stopping criterion)
55

Image Compression

Such a large amount of data has brought the issue

of data transfer and storage

Purpose of Image Compression

Saving storage space
Saving transfer time
Easy processing & reduce cost

Image Compression Coding

Huffman Coding
lossless compression
Run-length encoding
Image Compression
Predictive Coding
lossy compression
Transform Coding

General compression system model

Output image

Input image

Source encoder

Source decoder

Encoder

Decoder

Channel encoder

Channel decoder
Channel

Three basic data redundancies

Coding redundancy
Interpixel redundancy
Psychovisual redundancy

Data Redundancies

Coding redundancy

Interpixel redundancy

Psychovisual redundancy

Objective fidelity criteria

Objective fidelity criteria :root-mean-square (rms) error

between an input and output image.

Let f(x, y) represent an input image and let g(x,y) denote an

approximation
of f(x, y), x, y = 0, 1, 2,,N-1,
(1)The root-mean-square
error
(2)The rms value of the
signal-to-noise ratio

1
MSE 2
N

N 1 N 1

[ g( x , y ) f ( x , y )]
x 0 y0

N 1 N 1

1/ 2

g ( x, y )

x0 y 0

( SNR) rms

N 1 N 1

x 0 y 0

2
[
g
(
x
,
y
)

f
(
x
,
y
)]

Subjective fidelity criteria

Huffman Coding
Huffman coding is an entropy encoding algorithm

used for lossless data compression.

Huffman coding uses a specific method for choosing
the representation for each symbol, resulting in a
prefix code that expresses the most common
characters using shorter strings of bits than are used
for less common source symbols. A method was later
found to do this in linear time if input probabilities are
sorted.

Huffman Coding Algorithm

The simplest construction algorithm uses a priority queue

where the node with lowest probability is given highest

priority:

(1)Create a leaf node for each symbol and add it to the priority
queue. While there is more than one node in the queue, remove
the node of highest priority (lowest probability) twice to get two
nodes.
(2)Create a new internal node with these two nodes as children
and with probability equal to the sum of the two nodes'
probabilities. Add the new node to the queue.
(3)Repeat step 2 until the remaining node is the root and the tree
is complete.

Example of Huffman Coding

For example, a source generates 5 different symbols
{A, B, C, D, E} with probability {0.16; 0.51; 0.09; 0.13;
0.11}.

Run Length Encoding

Run-length encoding (RLE) is a very simple form of

data compression in which runs of data (that is,

sequences in which the same data value occurs in
many consecutive data elements) are stored as a single
data value and count, rather than as the original run.
This is most useful on data that contains many such
runs: for example, relatively simple graphic images
such as icons, line drawings, and animations.

Example of RLE
Let us take a hypothetical single scan line, with B

representing a black pixel and W representing white:

WWWWWWWWWWWWBWWWWWWWWWWWWBBBWWWW
WWWWWWWWWWWWWWWWWWWWBWWWWWWWWWW
WWWW
If we apply the run-length encoding (RLE) data compression
algorithm to the above hypothetical scan line, we get the
following:12W1B12W3B24W1B14W

Predictive Coding
DM (Delta Modulation)
Common Predictive
Coding

DPCM (Differential Pulse

Code Modulation)

The principle of Predictive Coding

The system consists of an encoder and a decoder,

each containing an identical predictor. As each

successive pixel of the input image, is introduced to
the encoder, the predictor generates the anticipated
value of that pixel based on some number of past
inputs. The output of the predictor is then rounded to
the nearest integer.

Predictive coding model I

Predictive coding
model
II

The predictor :

f n F ( f n 1 , f n 2 , , f n k )
n f n fn

The symbol encoder : generate the next element of the

compressed data stream
Decoder : perform the inverse of encoding
The linearity predictor :
fn F ( f n 1 , f n 2 , , f n k )

n 1

a
k l

f k , ak 1

Delta Modulation I
Delta modulation (DM or -modulation) is an analog-todigital and digital-to-analog signal conversion technique
used for transmission of voice information where quality
is not of primary importance. DM is the simplest form of
differential pulse-code modulation (DPCM) where the
difference between successive samples is encoded into nbit data streams. In delta modulation, the transmitted
data is reduced to a 1-bit data stream.

Analog signal input range

Delta Modulation II
Quantify the bands

Slope overload

Granular noise

Coding output

Differential Pulse Code Modulation

Differential Pulse Code Modulation (DPCM) compares

two successive analog amplitude values, quantizes and

encodes the difference, and transmits the differential
value.

The principle of DPCM

A natural refinement of this general approach is to predict the current

sample based on the previous M samples utilizing linear prediction
(LP), where LP parameters are dynamically estimated. Block diagram
of a DPCM encoder and decoder is shown below. Part (a) shows DPCM
encoder and part (b) shows DPCM decoder at the receiver.

Transform Coding I
Transform coding is a type of data compression for

"natural" data like audio signals or photographic images.

The transformation is typically lossy, resulting in a lower
quality copy of the original input.

Transform Coding II

Transform Coding III

A transform coding system

Joint
Picture
Expert
Group

The name "JPEG" stands for Joint Photographic Experts Group,

the name of the committee that created the standard. The group
was organized in 1986, issuing a standard in 1992, which was
approved in 1994 as ISO 10918-1. JPEG is a commonly used
method of compression for photographic images. The degree of
compression can be adjusted, allowing a selectable tradeoff
between storage size and image quality.

Steps of JPEG
The image is subdivided into pixel blocks of size 8 * 8, which are
processed left to right, top to bottom. As each 8 * 8 block or subimage is
encountered, its 64 pixels are level shifted by subtracting the quantity
2n-1, where 2n is the maximum number of gray levels.
1.
The 2-D discrete cosine transform of the block is then computed,
quantized and reordered.
2.
Use the zigzag pattern, to form a 1-D sequence of quantized
coefficients.
3.
Use the DPCM (differential pulse code modulation) code the DC
coefficients.
4.
The nozero AC coefficients are coded using a run-length encoding.
5.
Entropy coding

JPEG Model
Image block

DCT-based encoder
Quantizer

Entropy coder
Compressed image data

The original image data

Quantization table

Entropy coding table

DCT-based compression encoding steps

DCT-based encoder
Entropy coder Inverse quantizer
Compressed image data
Entropy coding table

Reconstruction of image data

Quantization table
DCT-based compression encoding steps

JPEG2000
JPEG 2000 is a wavelet-based image compression standard. It was

created by the Joint Photographic Experts Group committee in the

year 2000 with the intention of superseding their original discrete
cosine transform-based JPEG standard (created about 1991).