Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
35 views

Detection and Pattern Recognition: Matlab: 1 Supervised Classification

This document provides a collection of problems related to supervised classification algorithms for exercises in detection and pattern recognition using MATLAB. It includes problems on implementing classifiers like the mean classifier, Mahalanobis distance classifier, support vector machines (SVM), and exploring concepts like feature normalization, feature transforms, and margins. The problems provide documentation, example datasets, and evaluation approaches to test and compare different classifiers.

Uploaded by

ace.kris8117
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
35 views

Detection and Pattern Recognition: Matlab: 1 Supervised Classification

This document provides a collection of problems related to supervised classification algorithms for exercises in detection and pattern recognition using MATLAB. It includes problems on implementing classifiers like the mean classifier, Mahalanobis distance classifier, support vector machines (SVM), and exploring concepts like feature normalization, feature transforms, and margins. The problems provide documentation, example datasets, and evaluation approaches to test and compare different classifiers.

Uploaded by

ace.kris8117
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

UNIVERSITT STUTTGART

LEHRSTUHL FR SYSTEMTHEORIE UND SIGNALVERARBEITUNG


Prof. Dr.-Ing. B. Yang
Collection of Problems of the Exercises in
Detection and Pattern Recognition: Matlab
Additional Problems are marked with
Version: July 5, 2012

Contents
1 Supervised Classification
1.1 Mean Classifier . . . . .
1.2 Mahalanobis distance . .
1.3 XOR Feature Transform
1.4 Feature Normalization .
1.5 Hard Margin SVM . . .

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

1
1
1
2
2
3

Collection of Problems of the Exercises in Detection and Pattern Recognition: Matlab

1 Supervised Classification
1.1 Mean Classifier
Use the protoype provided to implement a mean classifier using the euqlidian distance.
(a) Compute the mean i for every class i , i = 1, . . . , C by using the training examples.
(b) Compute the distance from each sample to each class mean.
(c) Asign the class label with the nearest mean as the prediction for every sample.
Apply the created mean classifer to the first Set of the lecture which uses two different gaussians
for the classes.
(d) Create a dataset of type 1 by the call DATA = Set1_data(N) with N = 1e3.
(e) Compute the error rate of the mean classifier for different training set sizes, i.e N =
10, 20, 30, . . . , 200, by using the extract_subset and generate_error_rate functions.
(f) Repeat the provious task 100 times to compute the mean and standart deviation of the error
rate for every training set size.
(g) Plot the mean and standart deviation of the training set size using the errorbar function and
discuss the results.

1.2 Mahalanobis distance


Use the mean classifier with the euqlidian distance as a template and extend it to use the Mahalanobis distance.
(a) Additionaly compute the covariance matrix Ci for every class i , i = 1, . . . , C by using the
training examples.
(b) Compute the Mahalanobis distance from each sample to each class mean.
(c) Asign the class label with the nearest mean as the prediction for every sample.
Apply the created mean classifer to the first Set of the lecture which uses two different gaussians
for the classes.
(d) Create a dataset of type 1 by the call DATA = Set1_data(N) with N = 1e3.
(e) Compute the error rate of the mean classifier for different training set sizes, i.e N =
10, 20, 30, . . . , 200, by using the extract_subset and generate_error_rate functions.
(f) Repeat the provious task 100 times to compute the mean and standart deviation of the error
rate for every training set size.
(g) Plot the mean and standart deviation of the training set size using the errorbar function.
(h) Compare the results to the mean classifier using only the euqlidian distance.

Collection of Problems of the Exercises in Detection and Pattern Recognition: Matlab


rs
rs

rs
sr rs
rs
rs
rs

rs rs

rs sr
rs

bc
bc

rs

bc
bc

bc

bc
bc
bc
bc

bc

bc
bc

bc
bc

rs
rs

bc
bc

bc
rs

bc

bc
bc

bc
bc

bc
bc

rs

bc
rs

rs
bc
bc

bc

bc
bc
bc

bc

bc
bc
bc

bc

bc

bc bc

bc
bc

rs
rs

rs
rs

rs

rs
rs

rs

rs
rs
rs sr
rs rs

rs

rs

bc
rs

Figure 1: XOR problem with gaussian mixtures.

1.3 XOR Feature Transform


An example of the provided XOR Dataset is given in Figure 1. Both classes have the same probability, the gaussian means are = [1, 1]T , = [1, 1]T , = [1, 1]T , = [1, 1]T and
1,1
1,2
2,1
2,2
all gaussian share the same covariance matrix C.
(a) Create the XOR dataset by the call DATA = XOR_data.
(b) Give the theoretical limit of the class means and for the mean classifier, i.e number of
1
2
training samples N .
(c) Is it possible to find a different linear classifier which solves this problem using these features?
(d) Find and implement a simple feature transformation y = (x) which makes a linear sepration
possible, i.e DATA_NEW = xor_transform(DATA_OLD).
(e) Test your feature transformation and compare the results using the mean classifier.
(f) Plot the resulting decision threshold with the call
visualize_descision(DATA,@(D) mean_classifier(xor_transform(D)));

1.4 Feature Normalization


(a) Create a margin dataset by the call DATA = Margin_data.
(b) Visualize the k-NN decision by calling visualize_decision(DATA,@knn_classifier);
(c) Why is the performance of the k-NN classifier so bad for this example?
(d) Write a function normalize_matrix which normalizes each feature to a new feature with zero
mean and standard deviation of one.
(e) Visualize the normalized decision with
visualize_decision(DATA,@(D) knn_classifier(normalize_matrix(D)));

Collection of Problems of the Exercises in Detection and Pattern Recognition: Matlab

1.5 Hard Margin SVM


This exercise will use the provided SVM Classifier to introduce the concept of margin and support
vectors using the margin dataset.
(a) Create a margin dataset by the call DATA = Margin_data.

(b) Visualize the support vector decision by calling visualize_descision(DATA,@(D) svm_classifier(D));


(c) Create a TRAIN dataset containing the training samples as classification samples, i.e
TRAIN=DATA;TRAIN.CLASSIFICATION = TRAIN.TRAINING;.
(d) Compute the discriminant values with [EM f] = svm_classifier(TRAIN);.
Now the vector f containts the fi = wT xi + w0 for the trainings samples xi . This creates the
relationship f = Xw 1b, where X = [x1 , . . . , xN ]T .
(e) Find the support vectors based on the computed fi , take into account that the computed discriminant values are only correct to a precision of = 1e 3.
(f) Highlight the support vectors X sv by the call
hold on;plot(X_sv(:,1),X_sv(:,2),ok,MarkerSize,10);.

(g) Compute the normal w and offset b by using the realationship f = Xw + 1w0 .
(h) Compute the margin using the support vectors.
(i) Draw the hyperplanes wT x + w0 = 0, wT x + w0 = 1 and wt x + w0 = 1 into the previously
created figure using the hold on; option.

You might also like