Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
60 views

KNN Algorithm

The K-Nearest Neighbors (K-NN) algorithm is a simple supervised machine learning algorithm that stores all available data and classifies new data based on similarity. It finds the K closest training examples in the feature space and assigns the new data to the most common class among its K neighbors. K-NN is a non-parametric algorithm that makes no assumptions about the distribution of data and can be used for both classification and regression tasks.

Uploaded by

Megha Sahu
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
60 views

KNN Algorithm

The K-Nearest Neighbors (K-NN) algorithm is a simple supervised machine learning algorithm that stores all available data and classifies new data based on similarity. It finds the K closest training examples in the feature space and assigns the new data to the most common class among its K neighbors. K-NN is a non-parametric algorithm that makes no assumptions about the distribution of data and can be used for both classification and regression tasks.

Uploaded by

Megha Sahu
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 4

o K-Nearest Neighbour is one of the simplest Machine Learning algorithms

based on Supervised Learning technique.

o K-NN algorithm assumes the similarity between the new case/data and

available cases and put the new case into the category that is most similar to
the available categories.

o K-NN algorithm stores all the available data and classifies a new data point

based on the similarity. This means when new data appears then it can be
easily classified into a well suite category by using K- NN algorithm.

o K-NN algorithm can be used for Regression as well as for Classification but

mostly it is used for the Classification problems.

o K-NN is a non-parametric algorithm, which means it does not make any

assumption on underlying data.

o It is also called a lazy learner algorithm because it does not learn from the

training set immediately instead it stores the dataset and at the time of
classification, it performs an action on the dataset.

o KNN algorithm at the training phase just stores the dataset and when it gets

new data, then it classifies that data into a category that is much similar to
the new data

Suppose there are two categories, i.e., Category A and Category B, and we
have a new data point x1, so this data point will lie in which of these
categories. To solve this type of problem, we need a K-NN algorithm. With
the help of K-NN, we can easily identify the category or class of a particular
dataset. Consider the below diagram:
How does K-NN work?
The K-NN working can be explained on the basis of the below algorithm:

o Step-1: Select the number K of the neighbors

o Step-2: Calculate the Euclidean distance of K number of neighbors

o Step-3: Take the K nearest neighbors as per the calculated Euclidean

distance.

o Step-4: Among these k neighbors, count the number of the data points in

each category.

o Step-5: Assign the new data points to that category for which the number of

the neighbor is maximum.

o Step-6: Our model is ready.

Suppose we have a new data point and we need to put it in the required
category. Consider the below image:
o Firstly, we will choose the number of neighbors, so we will choose the k=5.

o Next, we will calculate the Euclidean distance between the data points. The

Euclidean distance is the distance between two points, which we have


already studied in geometry. It can be calculated as

o By calculating the Euclidean distance we got the nearest neighbors, as three

nearest neighbors in category A and two nearest neighbors in category B.


Consider the below image:
o As we can see the 3 nearest neighbors are from category A, hence this new

data point must belong to category A.

How to select the value of K in the K-NN Algorithm?


Below are some points to remember while selecting the value of K in the K-
NN algorithm:

o There is no particular way to determine the best value for "K", so we need to

try some values to find the best out of them. The most preferred value for K
is 5.

o A very low value for K such as K=1 or K=2, can be noisy and lead to the

effects of outliers in the model.

o Large values for K are good, but it may find some difficulties.

Advantages of KNN Algorithm:


o It is simple to implement.

o It is robust to the noisy training data

o It can be more effective if the training data is large.

Disadvantages of KNN Algorithm:


o Always needs to determine the value of K which may be complex some time.

o The computation cost is high because of calculating the distance between the

data points for all the training samples.

You might also like