Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 45 views

    Support Vector Machine (SVM) PDF

    Uploaded by

    Dhivya S - 67
    SVM searches for the optimal hyperplane that separates classes with the largest margin. It does this by formulating the problem as a quadratic optimization problem with linear constraints that can be solved using Lagrangian multipliers. SVMs can handle non-linear classification using kernel functions to transform data into higher dimensions where a linear separator may exist. They are effective for high dimensional data because the complexity is determined by support vectors rather than dimensionality, and support vectors do not increase with dimensionality. SVMs are commonly used for classification and regression tasks.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Support Vector Machine (SVM) PDF

    Uploaded by

    Dhivya S - 67
    45 views15 pages
    SVM searches for the optimal hyperplane that separates classes with the largest margin. It does this by formulating the problem as a quadratic optimization problem with linear constraints that can be solved using Lagrangian multipliers. SVMs can handle non-linear classification using kernel functions to transform data into higher dimensions where a linear separator may exist. They are effective for high dimensional data because the complexity is determined by support vectors rather than dimensionality, and support vectors do not increase with dimensionality. SVMs are commonly used for classification and regression tasks.

    Original Title

    SUPPORT VECTOR MACHINE(SVM).pdf

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    SVM searches for the optimal hyperplane that separates classes with the largest margin. It does this by formulating the problem as a quadratic optimization problem with linear constraints that can be solved using Lagrangian multipliers. SVMs can handle non-linear classification using kernel functions to transform data into higher dimensions where a linear separator may exist. They are effective for high dimensional data because the complexity is determined by support vectors rather than dimensionality, and support vectors do not increase with dimensionality. SVMs are commonly used for classification and regression tasks.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    45 views15 pages

    Support Vector Machine (SVM) PDF

    Uploaded by

    Dhivya S - 67
    SVM searches for the optimal hyperplane that separates classes with the largest margin. It does this by formulating the problem as a quadratic optimization problem with linear constraints that can be solved using Lagrangian multipliers. SVMs can handle non-linear classification using kernel functions to transform data into higher dimensions where a linear separator may exist. They are effective for high dimensional data because the complexity is determined by support vectors rather than dimensionality, and support vectors do not increase with dimensionality. SVMs are commonly used for classification and regression tasks.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 15

    SUPPORT VECTOR

    MACHINE(SVM)
    SVM—History

    • groundwork from Vapnik & Chervonenkis’ statistical learning


    theory in 1960s
    • Vapnik and colleagues (1992)

    2
    2
    SVM

    3
    4
    SVM—Definition

    Let data D be (X1, y1), …, (X|D|, y|D|), where Xi is the set of training tuples
    associated with the class labels yi
    There are infinite lines (hyperplanes) separating the two classes but we want to
    find the best one (the one that minimizes classification error on unseen data)
    SVM searches for the hyperplane with the largest margin, i.e., maximum
    marginal hyperplane (MMH)
    5
    5
    MMH & Support Vector

    6
    SVM—Linearly Separable
    ■ A separating hyperplane can be written as
    W●X+b=0
    where W={w1, w2, …, wn} is a weight vector and b a scalar (bias)
    ■ For 2-D it can be written as
    w0 + w1 x1 + w2 x2 = 0
    ■ The hyperplane defining the sides of the margin:
    H1 : w 0 + w 1 x 1 + w 2 x 2 ≥ 1 for yi = +1, and
    H2: w0 + w1 x1 + w2 x2 ≤ – 1 for yi = –1
    ■ Any training tuples that fall on hyperplanes H1 or H2 (i.e., the
    sides defining the margin) are support vectors
    ■ This becomes a constrained (convex) quadratic optimization
    problem: Quadratic objective function and linear constraints 🡪
    Quadratic Programming (QP) 🡪 Lagrangian multipliers
    7
    7
    Specialty of SVM

    8
    Specialty of SVM

    9
    SVM—Linearly Inseparable

    10
    SVM—Linearly Inseparable

    11
    11
    SVM—Linearly Inseparable

    ■ Transform the original input data into a higher dimensional


    space

    ■ Search for a linear separating hyperplane in the new space


    12
    12
    SVM: Different Kernel functions

    ■ Instead of computing the dot product on the transformed


    data, it is math. equivalent to applying a kernel function
    K(Xi, Xj) to the original data, i.e., K(Xi, Xj) = Φ(Xi) Φ(Xj)
    ■ Typical Kernel Functions

    ■ SVM can also be used for classifying multiple (> 2) classes


    and for regression analysis (with additional parameters)
    13
    13
    Why Is SVM Effective on High Dimensional Data?

    ■ The complexity of trained classifier is characterized by the # of


    support vectors rather than the dimensionality of the data
    ■ The support vectors are the essential or critical training examples
    —they lie closest to the decision boundary (MMH)
    ■ If all other training examples are removed and the training is
    repeated, the same separating hyperplane would be found
    ■ The number of support vectors found can be used to compute an
    (upper) bound on the expected error rate of the SVM classifier, which
    is independent of the data dimensionality
    ■ Thus, an SVM with a small number of support vectors can have good
    generalization, even when the dimensionality of the data is high
    14
    14
    SVM— Applications
    • classification and numeric prediction
    • handwritten digit recognition, object recognition,
    speaker identification, benchmarking time-series
    prediction tests

    15
    15

    You might also like