Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 39 views

    ML Application in Signal Processing and Communication Engineering

    Uploaded by

    aniruddh nain
    This document summarizes a lecture on Bayesian learning and clustering algorithms. It introduces unsupervised learning and the clustering algorithm K-means. K-means partitions unlabeled data into k clusters by minimizing the distance between data points and their closest cluster center. It works by iteratively reassigning points to clusters and recalculating cluster centers until convergence. The document also briefly discusses competitive learning algorithms like self-organizing maps and challenges in clustering like determining the optimal number of clusters and handling outliers.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    ML Application in Signal Processing and Communication Engineering

    Uploaded by

    aniruddh nain
    39 views27 pages
    This document summarizes a lecture on Bayesian learning and clustering algorithms. It introduces unsupervised learning and the clustering algorithm K-means. K-means partitions unlabeled data into k clusters by minimizing the distance between data points and their closest cluster center. It works by iteratively reassigning points to clusters and recalculating cluster centers until convergence. The document also briefly discusses competitive learning algorithms like self-organizing maps and challenges in clustering like determining the optimal number of clusters and handling outliers.

    Original Title

    Lecture-7_Clustering

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document summarizes a lecture on Bayesian learning and clustering algorithms. It introduces unsupervised learning and the clustering algorithm K-means. K-means partitions unlabeled data into k clusters by minimizing the distance between data points and their closest cluster center. It works by iteratively reassigning points to clusters and recalculating cluster centers until convergence. The document also briefly discusses competitive learning algorithms like self-organizing maps and challenges in clustering like determining the optimal number of clusters and handling outliers.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    39 views27 pages

    ML Application in Signal Processing and Communication Engineering

    Uploaded by

    aniruddh nain
    This document summarizes a lecture on Bayesian learning and clustering algorithms. It introduces unsupervised learning and the clustering algorithm K-means. K-means partitions unlabeled data into k clusters by minimizing the distance between data points and their closest cluster center. It works by iteratively reassigning points to clusters and recalculating cluster centers until convergence. The document also briefly discusses competitive learning algorithms like self-organizing maps and challenges in clustering like determining the optimal number of clusters and handling outliers.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 27

    Joint Faculty Development Programme on

    ML Application in Signal Processing


    and Communication Engineering

    Lecture 07: Bayesian Learning

    Presented by
    Dr. Irshad Ansari
    (Indian Institute of Information Technology, Design and Manufacturing, Jabalpur,)
    Machine Learning Application in Signal Processing and Communication Engineering

    Clustering

    By- Dr. Irshad Ahmad Ansari


    Email: irshad@iiitdmj.ac.in
    Outline
    • Unsupervised Learning
    • Introduction to Clustering
    • K-means
    • Competitive Learning (SOM):
    Winners takes all*

    *No Error Correction: Like in BP


    What is clustering?
    • The organization of unlabeled data into
    similarity groups called clustering.
    • A cluster is a collection of data items which
    are “similar*” and “dissimilar” to data items
    in other clusters.

    *Color, approach, group etc.


    Historic application of clustering

    • Covid-19 Clusters in
    India

    https://howindialives.com/gram/coronadistricts/
    Computer vision application:
    Image segmentation
    Cluster evaluation (a hard problem)
    • Intra-cluster cohesion (compactness):
    – Cohesion measures how near the data points
    in a cluster are to the cluster centroid.
    – Sum of squared error (SSE) is a commonly
    used measure.

    • Inter-cluster separation (isolation):


    – Separation means that different cluster
    centroids should be far away from one
    another.
    How many clusters?
    K-Means clustering
    • K-means (MacQueen, 1967) is a partitional
    clustering algorithm
    • Let the set of data points D be {x1, x2, …, xn},
    where xi = (xi1, xi2, …, xir) is a vector in X  Rr, and r is
    the number of dimensions.
    • The k-means algorithm partitions the given data
    into k clusters:
    – Each cluster has a cluster center, called centroid.
    – k is specified by the user
    K-means algorithm

    • Given k, the k-means algorithm works as follows:


    1. Choose k (random) data points (seeds) to be the
    initial centroids, cluster centers
    2. Assign each data point to the closest centroid
    3. Re-compute the centroids using the current
    cluster memberships
    4. If a convergence criterion is not met, repeat steps 2
    and 3
    K-means convergence (stopping)
    criterion
    • no (or minimum) re-assignments of data
    points to different clusters, or
    • no (or minimum) change of centroids, or
    • minimum decrease in the sum of squared error
    (SSE),
    SSE    xC
    2
    k d (x, m j )
    j
    j1
    – Cj is the jth cluster,
    – mj is the centroid of cluster Cj (the mean vector of all
    the data points in Cj),
    – d(x, mj) is the (Eucledian) distance between data
    point x and centroid mj.
    K-means clustering example: step 1
    K-means clustering example – step 2
    K-means clustering example – step 3
    K-means clustering example
    K-means clustering example
    K-means clustering example
    Why use K-means?
    • Strengths:
    – Simple: easy to understand and to implement
    – Efficient: Time complexity: O(tkn),
    where n is the number of data
    points, k is the number of
    clusters, and
    t is the number of iterations.
    – Since both k and t are small. k-means is considered a
    linear
    algorithm.
    • K-means is the most popular clustering algorithm.
    • Note that: it terminates at a local optimum if SSE is
    used. The global optimum is hard to find due to
    complexity.
    Outlier
    s
    Competitive learning

    You might also like