1

Color Image Processing

in the block DCT Space
Jayanta Mukhopadhyay
Department of Computer Science & Engineering
Indian Institute of Technology, Kharagpur, 721302, India
jay@cse.iitkgp.ernet.in



2
What is COLOR?
Selective emission/reflectance of different wavelengths

3
Color Spectrum
Illumination Reflectance
Spectrum: Intensity as a function of wavelength.
4
Color Stimuli
Illumination Reflectance
The colour of an object: is the product of the spectrum of
the incident light with the light absorption and/or reflection
properties of the object.

5
What is Perceived Color?
The response generated by a stimulus in the cones
gives the perceived color
Three responses

6
Human color perception
For human eye
Approximately 65% of all cone are
sensitive to red light
33% are sensitive to green light
2% are sensitive to blue light
But blue cones are the most sensitive.


7
Tri-stimulus Values

Integration over wavelength

X = C()x() d = C()x()
Y = C()y() d = C()y()
Z = C()z() d = C()z()

Real colors span a subset of the XYZ space.
Two different stimuli can have same XYZ values.
Metameris
Additive color mixtures modeled by addition in XYZ
space.


8
Amounts of three primaries needed
to match all wavelengths of the
spectrum
The curves represented by the cones
reception are not simple peaks. They are,
instead, quite complex curves.
They even go negative!
RGB is not capable of reproducing every
single color we can see.
9
Perceived Color Features
Intensity
Sum of the spectrum
Energy under the spectrum
Hue
Mean wavelength of the spectrum
What wavelength sensation is
dominant?
Saturation
Standard deviation of the spectrum
How much achromatic/gray
component?
Chrominance Hue and saturation

10
Limitation of Tri-Stimulus Model
No physical feel as to how colors are
arranged.

How do brightness change?

How does hue change?

Subtractive like paint cannot be
modeled by XYZ space.


11
CIE XYZ Space
Intensity (I) X+Y+Z
Chrominance (x,y) - (X/I, Y/I)
Chromaticity chart
Projection on a plane with normal (1,1,1)
Reduction of dimension
Similar to 3D to 2D in geometry
Each vector from (0,0,0) is an iso-
chrominance line
Each vector maps to a point in the
chromaticity chart
12
RGB-to-XYZ Space
13
CIE Chromaticity Chart
Shows all the visible colors
Achromatic Colors are at
(0.33,0.33)
Called white point
The saturated colors at the
boundary
Spectral Colors
14
Chromaticity Chart: Hue
All colors on straight line
from white point to a
boundary has the same
spectral hue
Dominant wavelength
15
Chromaticity Chart: Saturation
Purity (Saturation)
How far shifted towards
the spectral color
Ratio of a/b
Purity =1 implies spectral
color with maximum
saturation
16
Color Reproducibility
Only a subset of the 3D
CIE XYZ space called 3D
color gamut
Projection of the 3D
color gamut
Triangle
2D color gamut
Large if using more saturated
primaries
Cannot describe
brightness range
reproducibility
17
Standard Color Gamut
18
Color spaces
RGB (CIE), RnGnBn (TV NTSC)
XYZ (CIE)
UVW (UCS de la CIE), U*V*W* (UCS modified
by the CIE)
YUV, YIQ, YCbCr
HSV, HLS, IHS
Munsel colour space (cylindrical representation)
CIELuv
CIELab

19
RGB-to-YCbCr
20


Color Enhancement


21
Color Processing in the
Compressed Domain
Computation with reduced storage.

Avoid overhead of inverse and
forward transform..

Exploit spectral factorization for
improving the quality of result and
speed of computation.
22
Basic Approaches
Modify the DC coefficient for increasing brightness.
Aghaglzadeh and Ersoy (1992), Opt.Engg.

Modify AC coefficients for increasing contrast.
Tang, Peli and Acton (2003), IEEE SPL

A combination of both.
S. Lee (2007), IEEE CSVT

Preserve also colors by processing DCT of chromatic
components.
23
Different methods
Multi-Contrast Enhancement with Dynamic
Range Compression (S. Lee (2007), IEEE CSVT)

Modification of DC coefficients and AC coefficients
(following similar strategy of multi-contrast
enhancement).

Normalized DC coefficients (x) are modified as follows:
24
Proposed Approach
Adjust background illumination.
Use DC coefficients of the Y component.

Preserve Local Contrast.
Scale AC coefficients of the Y component appropriately.

Preserve Colors.
Preserve Color Vectors in the DCT domain.
DCT coefficients of Cb and Cr components.
25

Let and denote the mean and standard deviation of
an image. Contrast of an image is defined here as: .

Contrast : Definition
o
,

=
Weber Law:

where is the difference in luminance between a
stimulus and its surround, and L is the luminance of the
surround
L
L
,
A
=
L A
26
Theorem on Contrast Preservation in
the DCT Domain
Let k
d
be the scale factor for the DC coefficient and k
a

k
a
be the scale factor for the AC coefficients of a DCT
block Y. The processed DCT block Y
e
is given by:

( , ), 0
( , ),
( , ) {
d
a
Y i j i j
e Y i j otherwise
Y i j
k
k
= =
=
The contrast of the processed image then becomes k
a
/ k
d

times of the contrast of the original image.

In this algorithm k
d
= k
a
= k for preservation of the contrast.
27
Preservation of Colours in the DCT
Domain
Let U and V be the DCT coefficients of the C
b
and C
r

components, respectively. If the luminance component Y
of an image is uniformly scaled by a factor k , the colors
of the processed image with Y
e
, U
e
and V
e
are preserved
by the following operations:

( , )
( ( 128)) 128, 0
( , ),
( , ) {
U i j
N i j
N
e U i j otherwise
U i j
k
k
+ = =
=
( , )
( ( 128)) 128, 0
( , ),
( , ) {
V i j
N i j
N
e V i j otherwise
V i j
k
k
+ = =
=
28
Enhancement by Scaling Coefficients
Find the scale factor by mapping the DC
coefficient with a monotonically increasing
function.

Apply scaling to all other coefficients in all the
components.

For blocks having greater details, apply block
decomposition and re-composition strategy.
29
Mapping functions for adjusting the
local background illumination
( (0, 0))
(0, 0)
f Y
Y
k =
(TW)

Mitra and Yu , CVGIP87
(DRC)

Lee, CSVT07
(SF)

De, TENCON89
30
Monotonic Mapping Functions
31
Scaling only DC coefficients
32
Scaling both DC and AC
coefficients
33
Preservation of Contrast and
Color
original
34



Block
Decompos.
Smaller DCT
blocks
8x8 block
Apply CES
on smaller
blocks
Block
Composition
Enhanced Block
Enhancement of Blocks with more details
35
Removal of Blocking Artifacts
original
36
Some Results
original
AR MCE
MCEDRC
TW-CES-BLK
MSR
37
Enhancement near Edges
AR MCE MCEDRC
TW-CES-BLK DRC-CES-BLK
SF-CES-BLK
38
Some Results
original
AR
MCE
MCEDRC
TW-CES-BLK
MSR
39
Enhancement near edges
AR MCE MCEDRC
TW-CES-BLK
DRC-CES-BLK SF-CES-BLK
40
Some Results
original
AR MCE
MCEDRC TW-CES-BLK
MSR
41
Enhancement near edges
AR
MCE
MCEDRC
TW-CES-BLK DRC-CES-BLK SF-CES-BLK
42
Metrics for Comparison
( ) ( )
2
2 2
4
xy
x y
xy
QM
xy
o
o o
=
+
Wang and Bovic (SPL, 2002)
JPEG Quality Metric (JPQM)
Wang and Bovic (ICIP,2002)
2 2 2 2
0.3 CM
o | o |
o o = + + + Susstrunk and Winkler (SPIE, 2004)
R G o =
2
R G
B |
+
| |
=
|
\ .
43
Approaches under consideration
Alpha Rooting (AR) :
Aghaglzadeh and Ersoy (1992), Opt.Engg.
Multi-Contrast Enhancement (MCE):
Tang, Peli and Acton (2003), IEEE SPL
Multi-Contrast Enhancement with Dynamic Range
Compression (MCEDRC):
S. Lee (2007), IEEE CSVT
Contrast Enhancement by Scaling (CES):
Proposed work
Multi-Scale Retinex (MSR) (a reference spatial
domain technique):
Jobson, Rahman and Woodell (1997), IEEE IP

44
Average Performance Measures
Techniques JPQM CEF Y-
QM
C
b
-
QM
C
r
-
QM
AR
8.58 0.97 0.80 0.67 0.67
MCE
7.00 0.94 0.76 0.67 0.67
MCEDRC
7.92 0.97 0.86 0.67 0.67
TW-CES-BLK
7.79 1.50 0.90 0.82 0.81
DRC-CES-
BLK
8.16 1.18 0.86 0.76 0.76
SF-CES-BLK
8.13 1.25 0.89 0.78 0.77
45
Computational Complexities
Techniques Per Pixel Operations
AR
1E + 1M
MCE
2.19M+1.97A
MCEDRC
0.03E+3.97M+2A
TW-CES
0.02E+4.02M+1.05A
DRC-CES
0.05E+4M+1.08A
SF-CES
0.03E+4.02M+1.06A
MSR
18E+1866378M+8156703A
aE+bM+cA implies a Exponentiation, b Multiplication and c Addition operations.
46
Iterative Enhancement
original
Iteration no.=1
Iteration no.=2
Iteration no.=3
Iteration no.=4
47
Problem of Color Constancy
Three factors of image formation:

Objects present in the scene.
Spectral Energy of Light Sources.
Spectral Sensitivity of sensors.
Spectral Power Distribution Surface Reflectance Spectrum
Spectral Response of a Sensor
48
Same Scene Captured under
Different Illumination
Can we transfer colors from one illumination to another one?
49
Computation of Color
Constancy
Deriving an illumination independent
representation.

- Estimation of SPD of Light Source.

Color Correction
- Diagonal Correction.
E() <R, G, B>
To perform this computation with DCT coefficients.
50
Different Spatial Domain
Approaches
Gray World Assumption (Buchsbaum
(1980), Gershon et al. (1988))

<R, G, B> <R
avg
, G
avg
, B
avg
>

White World Assumption (Land (1977))

<R, G, B> <R
max
, G
max
, B
max
>

51
Select from a set of Canonical
Illuminants
Observe distribution of points in 2-D
Chromatic Space.
Assign SPD of the nearest illuminant.
Gamut Mapping Approach (Forsyth (1990),
Finlayson (1996))
- Existence of chromatic points.
Color by Correlation (Finlayson et. al. (2001))
- Relative strength over the distribution.
Nearest Neighbor Approach (Proposed)
- Mean and Covariance Matrix.
- Use of Mahalanobis Distance.

52
Processing in the Compressed
Domain
Consists of non-overlapping DCT blocks
(of 8 x 8).

Use DC coefficients of each block.

The color space used is Y-Cb-Cr instead
of RGB.

Chromatic Space for Statistical
Techniques is the Cb-Cr space.
53
Different Algorithms under
consideration
54
List of Illuminants
55
Images Captured at Different
Illumination
Source: http://www.cs.sfu.ca/ colour/data.

56
Performance Metrics
Estimated SPD: E=<R
E
,G
E
,B
E
>
True SPD: T= <R
T
,G
T
,B
T
>
62
Average

0
5
10
15
20
25
30
spatial DCT
GRW
MXW
GMAP
COR
NN
MXW-Y
63
Average
rg

0
0.05
0.1
0.15
0.2
0.25
spatial DCT
GRW
MXW
GMAP
COR
NN
MXW-Y
64
Average
RGB

0
50
100
150
200
250
300
spatial DCT
GRW
MXW
GMAP
COR
NN
MXW-Y
65
Average
L

0
50
100
150
200
250
300
350
400
450
spatial DCT
GRW
MXW
GMAP
COR
NN
MXW-Y
66
n
l
: number of illuminants.

n
c:
size of the 2-D chromaticity space

n: number of image pixels

f: Fraction of chromaticity space covered.

aM+bA a number of Multiplications and
b number of Additions.



Time and Storage Complexities
67
Time and Storage Complexities
68
Equivalent No. of Additions
per pixel (1 M= 3 A)
0
5
10
15
20
25
30
35
spatial DCT
GRW
MXW
GMAP
COR
NN
MXW-Y
n=512, n
c
=32, n
l
=12, f=1
69
Color Correction: An Example
Target Ref. Image
(syl-50mr16q)
Image captured with
(solux-4100)

MXW-DCT-Y COR
COR-DCT
70
Color Restoration
Original Enhanced w/o
Color Correction
Enhanced with
Color Correction
71
Conclusion-I
Color-constancy computation in the
compressed domain :
- requires less time and storage.
- comparable quality of results.

Both NN and NN-DCT perform well
compared to other existing statistical
approaches.

Color constancy computation is useful in
restoration of colors.
72
Direct filtering in the 8x8 block DCT
space using convolution multiplication
properties.

Approximate and exact computations
by block DCT composition and
decomposition.

Demonstration of its applications in
removing blocking artifacts and image
enhancement.
Conclusion-II
73
Thanks