COMP3308/COMP3608 Artificial Intelligence Week 10 Tutorial Exercises Support Vector Machines. Ensembles of Classifiers
COMP3308/COMP3608 Artificial Intelligence Week 10 Tutorial Exercises Support Vector Machines. Ensembles of Classifiers
COMP3308/COMP3608 Artificial Intelligence Week 10 Tutorial Exercises Support Vector Machines. Ensembles of Classifiers
Regarding Assignment 2: Please do not underestimate the report! It is worth 12/24 marks = 50% of your
mark for this assignment. Sometimes students spend too much time on the code and then there is not enough
time left to write a good quality report.
Exercise 1. (Homework)
This is a set of 5 multiple choice questions to be answered online on Canvas. To complete the homework,
please go to Canvas -> Quizzes->w10-homework-submission. Remember to press the “Submit” button at
the end. Only 1 attempt is allowed.
Q2. After SVM learning, each Lagrange multiplier lambda_i takes either zero or non-zero value. Which
one is correct:
a) A non-zero lambda_i indicates that example i is a support vector.
b) A zero lambda_i indicates that example i is a support vector.
c) A non-zero lambda_i indicates that the learning has not yet converged to a global minimum.
d) A zero lambda_i indicates that the learning process has identified support for example i.
Q4. Bagging is only applicable to classification problems and cannot be applied to regression problems.
a) True
b) False
Q5. Boosting is guaranteed to improve the performance of the single classifier it uses.
a) True
b) False
b) What are the 2 main ideas that are combined in Random Forest?
COMP3308/3608 Artificial Intelligence, s1 2021
Exercise 3. Boosting
Continue the example from slides 24-25. The current probabilities of the training examples to be used by
AdaBoost are given below:
p(x1) p(x2) p(x3) p(x4) p(x5) p(x6) p(x7) p(x8) p(x9) p(x10)
0.07 0.07 0.07 0.07 0.07 0.07 0.07 0.17 0.17 0.17
From these, a training set T2 has been created and using T2 a classifier C2 has been generated. Suppose that
C2 misclassifies examples x2 and x9. Show how the probabilities of all training examples are recalculated
and normalized.
Choose 10-fold cross validation. Run the SVM (SMO located under “functions”). This implementation of
SVM uses John Platt's sequential minimal optimization algorithm to solve the optimization problem (i.e. to
train SVM classifier) and shows the computed decision boundary.
b) Let’s try non-linear SVM. Edit the properties of SMO (by clicking on “SMO”). Keep the same type of
kernel (Poly Kernel, it is a general type of kernel and we can use it to create both linear and no-linear SVMs)
but change the exponent from 1 to 2 (by clicking on “Poly kernel”). Now the kernel function is nonlinear:
Poly Kernel: K(x,y) = <x,y>^2.0. Run the classifier, observe the new equation of the decision boundary
and the new accuracy. Compare the accuracy of the linear and non-linear SVM, which one is better?
Experiment with other types of kernels (e.g. other polynomial, by increasing and exponent, or RBF).
3) Compare SVM’s accuracy with the backpropagation neural network and the other classifiers we have
studied.
a) Examine the output. Not 1 but 3 SVM are generated. Do you know why?
b) Examining again the output, which of the 2 methods does Weka use?
2. Run bagging of decision trees (J48). It is under “Meta”. The default base classifier is REPTree, you need
to change it to j48. Compare the performance with the single DT classifier from 1).
COMP3308/3608 Artificial Intelligence, s1 2021
3. Run boosting (AdaBoost) of decision trees (J48). It is also under “Meta” and you need to change the
default base classifier. Compare performance with the single classifiers from 1) and also with the bagged
classifier from 2).
4. Run Random Forest of decision trees (J48). It’s under “Trees” and you need to change the default base
classifier. Compare performance with the previous classifiers.
5. Use AdaBoost with OneR as a base classifier. Experiment with different number of hypotheses M. What
happens with the accuracy on the training data as M increases? What about the accuracy on test data? Is
this consistent with the AdaBoost theorem?
Exercise 7. Do you have any questions about Assignment 2? Ask your tutor!