Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
23 views

ML3 Some Supervised

This document discusses different machine learning algorithms including supervised learning techniques like K-nearest neighbors (KNN), decision trees, random forests, linear regression, and logistic regression as well as unsupervised techniques like Bayesian models and hidden Markov models. It then focuses on explaining KNN in more detail including how it works, requirements, advantages, and disadvantages.

Uploaded by

HariHara Shankar Sbz
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
23 views

ML3 Some Supervised

This document discusses different machine learning algorithms including supervised learning techniques like K-nearest neighbors (KNN), decision trees, random forests, linear regression, and logistic regression as well as unsupervised techniques like Bayesian models and hidden Markov models. It then focuses on explaining KNN in more detail including how it works, requirements, advantages, and disadvantages.

Uploaded by

HariHara Shankar Sbz
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

Supervised Learning

Statistical – KNN, Regression, SVM,


Decision Trees
Probabilistic – Bayesian, HMM
Neural Networks – MLP, CNN, RNN

N. Sudha, SASTRA University 1


Statistical Machine Learning

K Nearest Neighbours
Decision Trees
Random Forest
Linear Regression
Logistic Regression
Support Vector Machine

N. Sudha, SASTRA University 2


K Nearest Neighbours

N. Sudha, SASTRA University 3


K Nearest Neighbours
Requirements
• Feature space (Training data)
• Distance metric
– Euclidean, Hamming etc
• Value of K
– The number of nearest neighbours to consider for
obtaining the class information

N. Sudha, SASTRA University 4


K Nearest Neighbours
How to classify an unknown data?

N. Sudha, SASTRA University 5


K Nearest Neighbours
How to classify an unknown data?
1. Compute distance to training data
2. Identify K nearest neighbours
3. Take the majority vote of class labels of K
nearest neighbours
4. Weighting factor of 1/d may be used for each
neighbour
5. Class giving maximum weighted vote is assigned

N. Sudha, SASTRA University 6


K Nearest Neigbours

K=7
 K = 1:
Belongs to square class
K=3
K=1  K = 3:
? Belongs to triangle class

 K = 7:
Belongs to square class

Choice of K:
– If K is too small, sensitive to noise points
– If K is too large, neighborhood may include points from other classes

N. Sudha, SASTRA University 7


K Nearest Neighbours
Advantages
• Simple technique
• Building model is inexpensive
• Suitable for multi-class data

Disadvantages
• Classifying unknown data is expensive
– Requires distance computation with all training data
– Computationally intensive when the training set is large
• Accuracy can be severely degraded by the presence of
noisy or irrelevant features

N. Sudha, SASTRA University 8


Linear Regression

N. Sudha, SASTRA University 9


Linear equation
Y
Y = mX + b
Change
m = Slope in Y
Change in X
b = Y-intercept
X

N. Sudha, SASTRA University 10


Line fitting

(X5,Y5) (X6,Y6)

(X2,Y2)

(X1,Y1) (X4,Y4)
(X3,Y3)

N. Sudha, SASTRA University 11


Linear regression model

Relationship between Variables in a Linear Function

Population Population Random


Y-Intercept Slope Error

Yi   0 +1 Xi   i
Dependent Independent
(Response) (Explanatory)
Variable Variable
N. Sudha, SASTRA University 12
Linear regression model

Y Yi   0  1 X i   i

i = Random error

Y   0  1 X

X
N. Sudha, SASTRA University 13
Estimating parameters
– Least Squares method

‘Best Fit’ means difference between actual Y values &


predicted Y values is a minimum.
But positive differences off-set negative ones.
So square errors!

N. Sudha, SASTRA University 14


Least squares
LS minimizes the Sum of the Squared Differences
(errors) (SSE)

    ˆ
n 2 n
Yi  Yˆi 2
i
i 1 i 1

N. Sudha, SASTRA University 15


Least squares graphically
n
LS minimizes  ˆ  ˆ  ˆ  ˆ  ˆ
i
2 2
1
2
2
2
3
2
4
i 1
Y Y2  ˆ0  ˆ1 X 2  ˆ2

^4
^2
^1 ^3
Yˆi  ˆ0  ˆ1 X i
X
N. Sudha, SASTRA University 16
Derivation of parameters
n n

    y    1 xi 
2 2
Minimize squared error i i 0
i 1 i 1

  2    i2
i
0 0
 0 1

Cov ( x, y )  x  x  y  y 
1 
ˆ  ˆ0  y  ˆ1x
i i

 x  x 
2
Var ( x) i

N. Sudha, SASTRA University 17

You might also like