Image 4
Image 4
Image 4
Ali Kadhem
1 Task 1:
An image can be represented by different colormaps. The most known is RGB. An image that has
not so vivid colours as seen in figure 1, can be made to have more vivid colours by using a different
colormap.
The colormap HSV is instead used to increase saturation. This results in the image seen in figure
2.
1
Figure 2: Image with corrected saturation.
Images with wrong white balance can be corrected in a couple of ways. One of the more simple
ways to correct the whitebalance of an image is to scale each channel in RGB so that the maximum
value is set to 1. In figure 3 you can see the correction of whitebalance using the code in lisitng 2.
2
(a) Image with wrong whitebalance.
3
2 Task 2: Segmentation with graph cuts
The image in figure 4 is to be segmented into chamber and background. Using ground truth data
for chamber and background with the assumption that the intensities have a Gaussian distribution,
each pixel can be classified as being of the chamber class or the background class. Mean intensity for
chamber is µ1 = 0.3542 with standard deviation σ1 = 0.0992 and for the background µ2 = 0.1065
with standard deviation σ2 = 0.0936. The likelihood is given by the Gaussian
(fi − µk )2
1
P (fi | classk ) = √ exp − , (1)
2πσ 2 2σ 2
for class k and pixel intensity fi . The negative logarithm of the likelihood is
1 (fi − µk )2
− log(P (fi | classk )) = log 2πσk2 + . (2)
2 2σk2
The first term is basically constant (the standard deviations are almost the same) and is therefore
ignored, so the negative logarithm of the likelihood is
(fi − µk )2
− log(P (fi | classk )) = . (3)
2σk2
10
20
30
40
50
60
70
80
90
10 20 30 40 50 60 70 80 90
To segment the heart into chamber and background, ground truth values for these two classes were
used to find the mean and standard deviation of the classes. The method used was implemented as
shown in listing 3. Segmentation of a given image can be done using max–flow/min–cut method.
4
Listing 3: Matlab code for segmentation of heart.
load heart_data % load data
%
M = size(im,1); % height of image, change this!
N = size(im,2); % width of image, change this!
i=Neighbours(:,1);
j=Neighbours(:,2);
A = sparse(i,j,2,n,n); % create sparse matrix of connections between pixels
% Choose weights:
mu1 = mean(chamber_values);
std1=std(chamber_values);
mu2 = mean(background_values);
std2=std(background_values);
for i=1:size(im,1)
for k=1:size(im,2)
pix=im(i,k);
kk=normpdf(pix,mu1,std1)+normpdf(pix,mu2,std2);
tTs(i,k)=(pix-mu1)^2/(2*std1^2);
tTt(i,k)=(pix-mu2)^2/(2*std2^2);
end
end
% create matrix of the full graph, adding source and sink as nodes n+1 and
% n+2 respectively
F = sparse(zeros(n+2,n+2));
F(1:n,1:n) = A; % set regularization weights
F(n+1,1:n) = Ts’; % set data terms
F(1:n,n+1) = Ts; % set data terms
F(n+2,1:n) = Tt’; % set data terms
F(1:n,n+2) = Tt; % set data terms
[MF,GF,CS,CT] = maxflow(Fg,n+1,n+2); % run maxflow on graph with source node (n+1) and
sink node (n+2)
5
% CS contains the pixels connected to the source node (including the source
% node n+1 as final entry (CT contains the sink nodes).
figure
imagesc(im)
figure
imagesc(seg);
saveas(gcf,’P2_heart’,’epsc’)
6
The fundamental matrix is given by
2 2 4
F = 3 3 6 .
−5 −10 −6
If points x in image 1 and x̄ in image 2 satisfies x̄T F x = 0. then there is a 3D point X that projects
to x in P1 and x̄ in P2 . The points x and x̄ is said to be corresponding. Three points are detected
in image 1:
a1 = (−4, 4, 1), a2 = (−10, 5, 1), a3 = (3, −7, 1),
and three points in image 2:
It is believed that the main flaw with this OCR system is the segmentation and features. The
Jaccard scores are:
short1: 0.87951
short2: 0.90341
home1: 0.82604
home2: 0.8253
home3: 0.79859.
An attempt to improve the segmentations of images in home folders was done, but without any
improvement. However, feature extraction was altered by scaling the features to achieve the hitrate
results:
short1: 52 %
short2: 52 %
home1: 50 %
home2: 49.9 %
home3: 50.1 %.
This system is still not optimal and better features should be considered. Segmentation is also
lacking and and needs improvement to get at least above 0.9 in Jaccard score. The functions for
this OCR system are presented below.
7
Listing 4: Functions for segmentation.
function S = im2segment(im)
im=padarray(im,[1 1],0,’both’);%Padding with zeros around
m = size(im,1);
n = size(im,2);
%Filter
im=filterim(im);
obj=zeros(m,n);
objnum=1;% number of bjects
objects=[];%Array for objects
while(sum(sum(im))~=0)
[a b]=find(im==1,1);%find first non-zero pixel
ind=[a b];
oldLeng=length(ind(:,1));
newLeng=0;
while(newLeng~=oldLeng)
oldLeng=length(ind(:,1));
for i=1:length(ind(:,1))
nei=hasNeigh(im,ind(i,1),ind(i,2));
ind=[ind;nei];
end
ind=unique(ind,’rows’);
newLeng=length(ind(:,1));
end
obj=zeros(size(im,1),size(im,2));
for i=1:length(ind(:,1))
obj(ind(i,1),ind(i,2))=1;
end
im=im-obj;
% Remove padding
obj(:,1)=[];
obj(:,end)=[];
obj(1,:)=[];
obj(end,:)=[];
objects(:,:,objnum)=obj;
objnum=objnum+1;
end
nrofsegments = length(objects(1,1,:));
S = cell(1,nrofsegments);
for kk = 1:nrofsegments
S{kk}= objects(:,:,kk);
end
while length(S)>5
for i=1:length(S)
sums(i)=sum(sum(S{i}));
end
8
[s,ind]=min(sums);
sums=[];
S(ind)=[];
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function im=filterim(im)
im=imgaussfilt(im,0.3);
im=medFilt(im);
im=thresh(im,1);
im=threshpix(im,3);
im=threshpix(im,1);
im=fillpix(im,7);
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function image=medFilt(im)
m=size(im, 1);
n=size(im,2) ;
image=zeros(m,n);
thr=20;
for i=2: m-1
for k=2:n - 1
med=median(im(i-1:i+1,k-1:k+1) , ’all’ );
if(im(i,k)<thr)
image(i,k)=0;
else
image(i, k) =med;
end
end
end
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function image=thresh(im,thr)
m=size(im,1);
n=size(im,2) ;
image=zeros(m,n);
for i=1:m
for k=1:n
if(im(i,k)<=thr)
image(i,k)=0;
else
image(i,k)=1;
end
end
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function im=threshpix(im,pix)
m=size(im,1);
n=size(im,2) ;
9
for i=2:m-1
for k=2:n-1
if(sum(im(i-1:i+1,k-1:k+1),’all’)<=pix)
im(i,k)=0;
end
end
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function im=fillpix(im,pix)
m=size(im,1);
n=size(im,2) ;
for i=2:m-1
for k=2:n-1
if(pix<=sum(im(i-1:i+1,k-1:k+1),’all’))
im(i,k)=1;
end
end
end
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function index=hasNeigh(f,m,n)
i=[m-1 m m+1];%row
k=[n-1 n n+1];%column
h=1;
index=[0 0];
for r=i
for q=k
if(f(r,q)==1)
index(h,1)=r;
index(h,2)=q;
h=h+1;
end
end
end
function f = segment2features(B)
B=resizeIm(B);
area=sum(sum(B));
[width height]=numDim(B);
[r c]=size(B);
[stdW stdH]=stdNum(B);
10
f(4)=stdW/91;
f(5)=stdH/29;
f(6)=perimeterIm(B)/6000;
f(7)=noHoles(B)/2;% Number of holes
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function zB=resizeIm(B)
[r c]=size(B);
nB=B(find(sum(B,2),1):find(sum(B,2),1,’last’),...
find(sum(B,1),1):find(sum(B,1),1,’last’));
[rows columns]=size(nB);
scale=(r-2)/rows;
rB=imresize(nB,scale,’nearest’);
zB = zeros(size(B));
zB(1:size(rB,1),1:size(rB,2)) = rB;
zB=shiftIm(zB);
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function shiftedImage=shiftIm(B)
[cx cy]=centerPoints(B);
[rows columns] = size(B);
Ix=columns/2;Iy=rows/2;
cx=round(cx);
cy=round(cy);
shiftX=Ix-cx;
shiftY=Iy-cy;
shiftedImage=zeros(rows,columns);
for i=1:rows
for k=1:columns
if(B(i,k)==1&&i+shiftY>=1&&k+shiftX>=1)
shiftedImage(i+1,k+shiftX)=1;%+shiftY
end
end
end
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function [cx,cy]=centerPoints(B)
[m n] = size(B);
y = 1:m;
x = 1:n;
[X Y] = meshgrid(x,y);
cx = mean(X(B==1));
cy = mean(Y(B==1));
end
11
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
function [width, height]=numDim(B)
%Dimension of number
width=find(sum(B,1),1,’last’)-find(sum(B,1),1);
height=find(sum(B,2),1,’last’)-find(sum(B,2),1);
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
12
if (B(i,k + 1)==1)
number=number+1;
end
13