Image Analysis: Assignment 2

Ali Kadhem

1 Task 1:
An image can be represented by different colormaps. The most known is RGB. An image that has
not so vivid colours as seen in figure 1, can be made to have more vivid colours by using a different
colormap.

Figure 1: Image with little saturation.

The colormap HSV is instead used to increase saturation. This results in the image seen in figure
2.

1
 Figure 2: Image with corrected saturation.

The implementation of this in MATLAB can be seen in listing 1

Listing 1: Code for more vivid image.

im1=imread("arcimboldo_low.jpg");
hsvIm1=rgb2hsv(im1);
hsvIm1(:,:,2)=6.*hsvIm1(:,:,2);
im1=hsv2rgb(hsvIm1);
imshow(im1)
saveas(gcf,’im1-vivid’,’epsc’)

Images with wrong white balance can be corrected in a couple of ways. One of the more simple
ways to correct the whitebalance of an image is to scale each channel in RGB so that the maximum
value is set to 1. In figure 3 you can see the correction of whitebalance using the code in lisitng 2.

2
 (a) Image with wrong whitebalance.

(b) Corrected whitebalance.

Figure 3: Whitebalance correction.

Listing 2: Code for whitebalance.

im2=double(imread(’michelangelo_colorshift.jpg’));
for i=1:size(im2,3)
im2(:,:,i)=255/max(max(im2(:,:,i))).*im2(:,:,i);
end
figure
imshow(uint8(im2))
saveas(gcf,’im2-whitebalance’,’epsc’)

3
2 Task 2: Segmentation with graph cuts
The image in figure 4 is to be segmented into chamber and background. Using ground truth data
for chamber and background with the assumption that the intensities have a Gaussian distribution,
each pixel can be classified as being of the chamber class or the background class. Mean intensity for
chamber is µ1 = 0.3542 with standard deviation σ1 = 0.0992 and for the background µ2 = 0.1065
with standard deviation σ2 = 0.0936. The likelihood is given by the Gaussian
(fi − µk )2
 
1
P (fi | classk ) = √ exp − , (1)
2πσ 2 2σ 2
for class k and pixel intensity fi . The negative logarithm of the likelihood is

1
(fi − µk )2
− log(P (fi | classk )) = log 2πσk2 + . (2)
2 2σk2
The first term is basically constant (the standard deviations are almost the same) and is therefore
ignored, so the negative logarithm of the likelihood is
(fi − µk )2
− log(P (fi | classk )) = . (3)
2σk2

10

20

30

40

50

60

70

80

90

10 20 30 40 50 60 70 80 90

Figure 4: Image of heart.

To segment the heart into chamber and background, ground truth values for these two classes were
used to find the mean and standard deviation of the classes. The method used was implemented as
shown in listing 3. Segmentation of a given image can be done using max–flow/min–cut method.

4
 Listing 3: Matlab code for segmentation of heart.
load heart_data % load data
%
M = size(im,1); % height of image, change this!
N = size(im,2); % width of image, change this!

n = M*N; % Number of image pixels

% create neighbour structure

Neighbours = edges4connected(M,N); % use 4-neighbours (or 8-neighbours with

edges8connected)

i=Neighbours(:,1);
j=Neighbours(:,2);
A = sparse(i,j,2,n,n); % create sparse matrix of connections between pixels

% Choose weights:
mu1 = mean(chamber_values);
std1=std(chamber_values);
mu2 = mean(background_values);
std2=std(background_values);
for i=1:size(im,1)
for k=1:size(im,2)
pix=im(i,k);
kk=normpdf(pix,mu1,std1)+normpdf(pix,mu2,std2);
tTs(i,k)=(pix-mu1)^2/(2*std1^2);
tTt(i,k)=(pix-mu2)^2/(2*std2^2);
end
end

Ts = sparse(tTs(:)); % set weights to source, according to assignment!

Tt = sparse(tTt(:)); % set weights to sink, according to assignment!

% create matrix of the full graph, adding source and sink as nodes n+1 and
% n+2 respectively

F = sparse(zeros(n+2,n+2));
F(1:n,1:n) = A; % set regularization weights
F(n+1,1:n) = Ts’; % set data terms
F(1:n,n+1) = Ts; % set data terms
F(n+2,1:n) = Tt’; % set data terms
F(1:n,n+2) = Tt; % set data terms

Fg = graph(F); % turn F into a graph Fg ’lower’

[MF,GF,CS,CT] = maxflow(Fg,n+1,n+2); % run maxflow on graph with source node (n+1) and
sink node (n+2)

5
% CS contains the pixels connected to the source node (including the source
% node n+1 as final entry (CT contains the sink nodes).

% We can construct out segmentation mask using these indices

seg = zeros(M,N);
seg(CS(1:end-1)) = 1; % set source pixels to 1

figure
imagesc(im)
figure
imagesc(seg);
saveas(gcf,’P2_heart’,’epsc’)

The segmentation obtained is shown in figure 5.

Figure 5: Segmentation of heart into chamber and background.

3 Task 3: Computer Vision

Two projections of camera matrices are givn by
   
1 0 0 0 1 2 3 3
P1 = 0 1 0 0 , P2 = 1 2 0 −2 .
0 0 1 0 1 1 2 0

6
The fundamental matrix is given by
 
2 2 4
F = 3 3 6 .
−5 −10 −6

If points x in image 1 and x̄ in image 2 satisfies x̄T F x = 0. then there is a 3D point X that projects
to x in P1 and x̄ in P2 . The points x and x̄ is said to be corresponding. Three points are detected
in image 1:
a1 = (−4, 4, 1), a2 = (−10, 5, 1), a3 = (3, −7, 1),
and three points in image 2:

b1 = (2, 3, 1), b2 = (2, −2, 1), b3 = (6, −1, 1).

The points that satisfies bT

i F aj = 0 are (a1 , b1) and (a2 , b2 ). So points that can be in correspondence
are a1 with b1 and a2 with b2 .

4 Task 4: OCR system construction and system testing

The hitrate results of the system are:
short1: 62 %
short2: 62 %
home1: 26.6 %
home2: 29 %
home3: 25.5 %.

It is believed that the main flaw with this OCR system is the segmentation and features. The
Jaccard scores are:
short1: 0.87951
short2: 0.90341
home1: 0.82604
home2: 0.8253
home3: 0.79859.
An attempt to improve the segmentations of images in home folders was done, but without any
improvement. However, feature extraction was altered by scaling the features to achieve the hitrate
results:
short1: 52 %
short2: 52 %
home1: 50 %
home2: 49.9 %
home3: 50.1 %.
This system is still not optimal and better features should be considered. Segmentation is also
lacking and and needs improvement to get at least above 0.9 in Jaccard score. The functions for
this OCR system are presented below.

7
 Listing 4: Functions for segmentation.
function S = im2segment(im)
im=padarray(im,[1 1],0,’both’);%Padding with zeros around
m = size(im,1);
n = size(im,2);
%Filter
im=filterim(im);

obj=zeros(m,n);
objnum=1;% number of bjects
objects=[];%Array for objects

while(sum(sum(im))~=0)
[a b]=find(im==1,1);%find first non-zero pixel
ind=[a b];
oldLeng=length(ind(:,1));
newLeng=0;
while(newLeng~=oldLeng)
oldLeng=length(ind(:,1));
for i=1:length(ind(:,1))
nei=hasNeigh(im,ind(i,1),ind(i,2));
ind=[ind;nei];
end
ind=unique(ind,’rows’);
newLeng=length(ind(:,1));
end
obj=zeros(size(im,1),size(im,2));
for i=1:length(ind(:,1))
obj(ind(i,1),ind(i,2))=1;
end
im=im-obj;
% Remove padding
obj(:,1)=[];
obj(:,end)=[];
obj(1,:)=[];
obj(end,:)=[];
objects(:,:,objnum)=obj;
objnum=objnum+1;
end

nrofsegments = length(objects(1,1,:));
S = cell(1,nrofsegments);
for kk = 1:nrofsegments
S{kk}= objects(:,:,kk);
end
while length(S)>5
for i=1:length(S)
sums(i)=sum(sum(S{i}));
end

8
[s,ind]=min(sums);
sums=[];
S(ind)=[];
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function im=filterim(im)
im=imgaussfilt(im,0.3);
im=medFilt(im);
im=thresh(im,1);
im=threshpix(im,3);
im=threshpix(im,1);
im=fillpix(im,7);
end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function image=medFilt(im)
m=size(im, 1);
n=size(im,2) ;
image=zeros(m,n);
thr=20;
for i=2: m-1
for k=2:n - 1
med=median(im(i-1:i+1,k-1:k+1) , ’all’ );
if(im(i,k)<thr)
image(i,k)=0;
else
image(i, k) =med;
end
end
end
end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function image=thresh(im,thr)
m=size(im,1);
n=size(im,2) ;
image=zeros(m,n);
for i=1:m
for k=1:n
if(im(i,k)<=thr)
image(i,k)=0;
else
image(i,k)=1;
end
end
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function im=threshpix(im,pix)
m=size(im,1);
n=size(im,2) ;

9
for i=2:m-1
for k=2:n-1
if(sum(im(i-1:i+1,k-1:k+1),’all’)<=pix)
im(i,k)=0;
end
end
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function im=fillpix(im,pix)
m=size(im,1);
n=size(im,2) ;
for i=2:m-1
for k=2:n-1
if(pix<=sum(im(i-1:i+1,k-1:k+1),’all’))
im(i,k)=1;
end
end
end
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

function index=hasNeigh(f,m,n)
i=[m-1 m m+1];%row
k=[n-1 n n+1];%column
h=1;
index=[0 0];
for r=i
for q=k
if(f(r,q)==1)
index(h,1)=r;
index(h,2)=q;
h=h+1;
end
end
end

Listing 5: Functions used to extract features from segments.

function f = segment2features(B)
B=resizeIm(B);
area=sum(sum(B));
[width height]=numDim(B);
[r c]=size(B);
[stdW stdH]=stdNum(B);

f(1)=area/(r*c)*8.5;% relative area

f(2)=width/height*1.31;% ratio between width and height
f(3)=massPos(B)*1.13;

10
f(4)=stdW/91;
f(5)=stdH/29;
f(6)=perimeterIm(B)/6000;
f(7)=noHoles(B)/2;% Number of holes
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

function zB=resizeIm(B)
[r c]=size(B);
nB=B(find(sum(B,2),1):find(sum(B,2),1,’last’),...
find(sum(B,1),1):find(sum(B,1),1,’last’));
[rows columns]=size(nB);
scale=(r-2)/rows;
rB=imresize(nB,scale,’nearest’);
zB = zeros(size(B));
zB(1:size(rB,1),1:size(rB,2)) = rB;
zB=shiftIm(zB);
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function shiftedImage=shiftIm(B)

[cx cy]=centerPoints(B);
[rows columns] = size(B);

Ix=columns/2;Iy=rows/2;
cx=round(cx);
cy=round(cy);
shiftX=Ix-cx;
shiftY=Iy-cy;
shiftedImage=zeros(rows,columns);
for i=1:rows
for k=1:columns
if(B(i,k)==1&&i+shiftY>=1&&k+shiftX>=1)
shiftedImage(i+1,k+shiftX)=1;%+shiftY
end
end
end
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

function [cx,cy]=centerPoints(B)

[m n] = size(B);
y = 1:m;
x = 1:n;
[X Y] = meshgrid(x,y);

cx = mean(X(B==1));
cy = mean(Y(B==1));
end

11
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function [width, height]=numDim(B)
%Dimension of number
width=find(sum(B,1),1,’last’)-find(sum(B,1),1);
height=find(sum(B,2),1,’last’)-find(sum(B,2),1);
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

function [stdW stdH]=stdNum(B)

sW=sum(B,1);
sH=sum(B,2);
stdW=std(sW);
stdH=std(sH);
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function pos=massPos(B)
[r c]=size(B);
[cx cy]=centerPoints(B);
cy=round(cy);
pos=0.5;
if (r/2-cy>=1)
pos=0.6;
end
if (r/2-cy<=-1)
pos=0.4;
end
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function p=perimeterIm(B)
p = 0;
for i = 1:size(B,1)
for k=1:size(B,2)
if (B(i,k)==1)
p=p+(4-noNeigh(B,i,k));
end
end
end
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function number=noNeigh(B,i,k)%Number of neighbours
number = 0;
if (B(i-1,k)==1)
number=number+1;
end
if (B(i,k - 1)==1)
number=number+1;
end
if (B(i + 1,k)==1)
number=number+1;
end

12
if (B(i,k + 1)==1)
number=number+1;
end

Listing 6: Functions for classification.

function classification_data = class_train(X, Y)

classification_data{1}=X;
classification_data{2}=Y;
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

function y = classify(x, classification_data)

for i=1:size(classification_data{1},2)
dist(i)=min(norm(classification_data{1}(:,i)-x));
end
[val,ind]=min(dist);
y=classification_data{2}(ind);
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function y=features2class(x, classification_data)
y = classify(x, classification_data);
end

13