MATLAB Programs and Exercises 79

p(x2 |x1 ) 5 p(x2 ) (2.146)

Write the respective sets Ai , i 5 1, 2, . . . , 6, and construct the corresponding

DAG.
2.40 In the DAG defined in Figure 2.29, assume that the variable z is measured to
be z0. Compute P(x1|z0) and P(w0|z0).
2.41 In the example associated with the tree-structured DAG of Figure 2.28,assume
that the patient undergoes the medical test H1 and that this turns out to be
positive (True). Based on this test, compute the probability that the patient
has developed cancer. In other words, compute the conditional probability
P(C 5 True|H1 5 True).

MATLAB PROGRAMS AND EXERCISES

Computer Exercises
A number of MATLAB functions are provided,which will help the interested reader to
experiment on some of the most important issues discussed in the present chapter.
Needless to say that there may be other implementations of these functions. Short
comments are also given along with the code. In addition,we have used the symbols
m and S to denote the mean vector (given as a column vector) and the covariance
matrix, respectively, instead of the symbols m and , which are used in the text.
In the following, unless otherwise stated, each class is represented by an integer in
{1, . . . , c} where c is the number of classes.

2.1 Gaussian generator. Generate N l-dimensional vectors from a Gaussian dis-

tribution with mean m and covariance matrix S, using the mvnrnd MATLAB
function.

Solution
Just type
mvnrnd(m,S,N)

2.2 Gaussian function evaluation. Write a MATLAB function that computes the
value of the Gaussian distribution N (m, S), at a given vector x.

Solution
function z=comp_gauss_dens_val(m,S,x)
[l,q]=size(m); % l=dimensionality
z=(1/((2*pi)^ (l/2)*det(S)^ 0.5) )...
*exp(-0.5*(x-m)'*inv(S)*(x-m));
80 CHAPTER 2 Classifiers Based on Bayes Decision Theory

2.3 Data set generation from Gaussian classes. Write a MATLAB function that
generates a data set of N l-dimensional vectors that stem from c different
Gaussian distributions N (mi , Si ), with corresponding a priori probabilities
Pi , i 5 1, . . . , c.

Solution
In the sequel:
■ m is an l ! c matrix, the i-th column of which is the mean vector of the
i-th class distribution.
■ S is an l ! l ! c (three-dimensional) matrix, whose ith two-dimensional
l ! l component is the covariance of the distribution of the ith class.
In MATLAB S(:, :, i) denotes the i-th two-dimensional l ! l matrix of S.
■ P is the c dimensional vector that contains the a priori probabilities of
the classes. mi , Si , Pi , and c are provided as inputs.
The following function returns:
■ A matrix X with (approximately) N columns, each column of which is
an l-dimensional data vector.
■ A row vector y whose ith entry denotes the class from which the ith
data vector stems.
function [X,y]=generate_gauss_classes(m,S,P,N)
[l,c]=size(m);
X=[];
y=[];
for j=1:c
% Generating the [p(j)*N)] vectors from each distribution
t=mvnrnd(m(:,j),S(:,:,j),fix(P(j)*N));
% The total number of points may be slightly less than N
% due to the fix operator
X=[X t];
y=[y ones(1,fix(P(j)*N))*j];
end

2.4 Plot of data. Write a MATLAB function that takes as inputs: (a) a matrix X and
a vector y defined as in the previous function, (b) the mean vectors of c class
distributions. It plots: (a) the data vectors of X using a different color for each
class, (b) the mean vectors of the class distributions. It is assumed that the data
live in the two-dimensional space.

Solution
% CAUTION: This function can handle up to
% six different classes
 MATLAB Programs and Exercises 81

function plot_data(X,y,m)
[l,N]=size(X); % N=no. of data vectors, l=dimensionality
[l,c]=size(m); % c=no. of classes
if(l ~ =2)
fprintf('NO PLOT CAN BE GENERATED\n')
return
else
pale=['r.'; 'g.'; 'b.'; 'y.'; 'm.'; 'c.'];
figure(1)
% Plot of the data vectors
hold on
for i=1:N
plot(X(1,i),X(2,i),pale(y(i),:))
end
% Plot of the class means
for j=1:c
plot(m(1,j),m(2,j),'k+')
end
end

2.5 Bayesian classifier ( for Gaussian Processes). Write a MATLAB function that
will take as inputs: (a) the mean vectors, (b) the covariance matrices of the
class distributions of a c-class problem, (c) the a priori probabilities of the c
classes,and (d) a matrix X containing column vectors that stem from the above
classes. It will give as output an N -dimensional vector whose ith component
contains the class where the corresponding vector is assigned,according to the
Bayesian classification rule.

Solution
Caution: While inserting the following function, do not type the labels (A),
(B) and (C). They are used to serve as references, as we will see later on.

(A) function z=bayes_classifier(m,S,P,X)

[l,c]=size(m); % l=dimensionality, c=no. of classes
[l,N]=size(X); % N=no. of vectors
for i=1:N
for j=1:c
(B) t(j)=P(j)*comp_gauss_dens_val(m(:,j),...
S(:,:,j),X(:,i));
end
% Determining the maximum quantity Pi*p(x|wi)
(C) [num,z(i)]=max(t);
end
82 CHAPTER 2 Classifiers Based on Bayes Decision Theory

2.6 Euclidean distance classifier. Write a MATLAB function that will take as inputs:
(a) the mean vectors, and (b) a matrix X containing column vectors that stem
from the above classes. It will give as output an N -dimensional vector whose
ith component contains the class where the corresponding vector is assigned,
according to the minimum Euclidean distance classifier.

Solution
The requested function may be obtained by the bayes_classifier function by
replacing (A), (B), and (C) with

■ function z=euclidean_classifier(m,X)
■ t(j)=sqrt((X(:,i)-m(:,j))'*(X(:,i)-m(:,j)));
(computation of the Euclidean distances from all class representatives)

■ [num,z(i)]=min(t);

(determination of the closest class mean),

respectively.
2.7 Mahalanobis distance classifier. Write a MATLAB function that will take as
inputs: (a) the mean vectors,(b) the covariance matrix of the class distributions
of a c-class problem, and (c) a matrix X containing column vectors that stem
from the above classes. It will give as output an N -dimensional vector whose
ith component contains the class where the corresponding vector is assigned
according to the minimum Mahalanobis distance classifier.

Solution
The requested function may be obtained by the bayes_classifier function by
replacing (A), (B) and (C) with

■ function z=mahalanobis_classifier(m,S,X)
■ t(j)=sqrt((X(:,i)-m(:,j))'*inv(S(:,:,j))*...
(X(:,i)-m(:,j)));
(computation of the Mahalanobis distances from all class representatives)

■ [num,z(i)]=min(t);

(determination of the closest class mean), respectively.

2.8 k-nearest neighbor classifier. Write a MATLAB function that takes as inputs:
(a) a set of N1 vectors packed as columns of a matrix Z, (b) an N1 -dimensional
vector containing the classes where each vector in Z belongs, (c) the value for
the parameter k of the classifier,(d) a set of N vectors packed as columns in the
 MATLAB Programs and Exercises 83

matrix X. It returns an N -dimensional vector whose ith component contains

the class where the corresponding vector of X is assigned, according to the
k-nearest neighbor classifier.

Solution
function z=k_nn_classifier(Z,v,k,X)
[l,N1]=size(Z);
[l,N]=size(X);
c=max(v); % The number of classes
% Computation of the (squared) Euclidean distance
% of a point from each reference vector
for i=1:N
dist=sum((X(:,i)*ones(1,N1)-Z).^ 2);
%Sorting the above distances in ascending order
[sorted,nearest]=sort(dist);
% Counting the class occurrences among the k-closest
% reference vectors Z(:,i)
refe=zeros(1,c); %Counting the reference vectors per class
for q=1:k
class=v(nearest(q));
refe(class)=refe(class)+1;
end
[val,z(i)]=max(refe);
end

2.9 Classification error evaluation. Write a MATLAB function that will take
as inputs: (a) an N -dimensional vector, each component of which contains
the class where the corresponding data vector belongs and (b) a similar N -
dimensional vector each component of which contains the class where the
corresponding data vector is assigned from a certain classifier. Its output
will be the percentage of the places where the two vectors differ (i.e., the
classification error of the classifier).

Solution
function clas_error=compute_error(y,y_est)
[q,N]=size(y); % N= no. of vectors
c=max(y); % Determining the number of classes
clas_error=0; % Counting the misclassified vectors
for i=1:N
if(y(i)~ =y_est(i))
clas_error=clas_error+1;
end
end