Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 73 views

    Preceptron

    Uploaded by

    eng_kmm
    The document discusses the perceptron, which is a single artificial neuron that uses a weighted sum of inputs and threshold activation function to classify inputs into two categories. It describes how the perceptron learns by adjusting its weights to find a decision boundary that separates the two classes. However, perceptrons can only learn linearly separable problems. The document notes that multilayer networks with sigmoid activations and the backpropagation learning algorithm can learn non-linear problems like XOR, which perceptrons cannot.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Preceptron

    Uploaded by

    eng_kmm
    73 views17 pages
    The document discusses the perceptron, which is a single artificial neuron that uses a weighted sum of inputs and threshold activation function to classify inputs into two categories. It describes how the perceptron learns by adjusting its weights to find a decision boundary that separates the two classes. However, perceptrons can only learn linearly separable problems. The document notes that multilayer networks with sigmoid activations and the backpropagation learning algorithm can learn non-linear problems like XOR, which perceptrons cannot.

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document discusses the perceptron, which is a single artificial neuron that uses a weighted sum of inputs and threshold activation function to classify inputs into two categories. It describes how the perceptron learns by adjusting its weights to find a decision boundary that separates the two classes. However, perceptrons can only learn linearly separable problems. The document notes that multilayer networks with sigmoid activations and the backpropagation learning algorithm can learn non-linear problems like XOR, which perceptrons cannot.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    73 views17 pages

    Preceptron

    Uploaded by

    eng_kmm
    The document discusses the perceptron, which is a single artificial neuron that uses a weighted sum of inputs and threshold activation function to classify inputs into two categories. It describes how the perceptron learns by adjusting its weights to find a decision boundary that separates the two classes. However, perceptrons can only learn linearly separable problems. The document notes that multilayer networks with sigmoid activations and the backpropagation learning algorithm can learn non-linear problems like XOR, which perceptrons cannot.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 17

    Artificial Neural Networks

    Part 2/3 – Perceptron

    Slides modified from Neural Network Design


    by Hagan, Demuth and Beale

    Berrin Yanikoglu
    Perceptron
    •  A single artificial neuron that computes its weighted input and
    uses a threshold activation function.

    •  It effectively separates the input space into two categories by


    the hyperplane:
    wTx + b = 0
    Decision Boundary

    The weight vector is orthogonal to the decision boundary

    The weight vector points in the direction of the + class


    (where an output of 1 is expected)
    –  if w pointed in the opposite direction, the dot products of all input
    vectors would have the opposite sign
    –  would result in same classification but with opposite labels

    The bias determines the position of the boundary


    •  solve for wTp+b = 0 using one point on the decision boundary to
    find b.
    Two-Input Case

    +
    a = hardlim(n) = [1 2]p + -2

    w1 , 1 = 1 w1 , 2 = 2 -
    Decision Boundary: all points p for which wTp + b =0
    If we have the weights and not the bias, we can take a point on
    the decision boundary, p=[2 0]T, and solving for [1 2]p + b = 0,
    we see that b=-2.
    p
    w wT.p = ||w||||p||Cosθ
    θ
    proj. of p onto w
    Decision Boundary
    proj. of p onto w
    T T = ||p||Cosθ
    1w p + b = 0 1w p = -b
    œb
    = wT.p/||w||
    • All points on the decision boundary have the same inner
    product (= -b) with the weight vector
    • Therefore they have the same projection onto the weight
    vector; so they must lie on a line orthogonal to the weight
    vector

    ADVANCED
    An
    Illustrative
    Example
    Boolean OR

    ⎧ 0 , t = 0 ⎫ ⎧ 0 , t = 1 ⎫ ⎧ 1 ,t = ⎫ ⎧ 1 , t = 1 ⎫
    p
    ⎨ 1 = 1 ⎬ p
    ⎨ 2 = 2 ⎬ ⎨ p3 = 3 1 ⎬ p
    ⎨ 4 = 4 ⎬
    ⎩ 0 ⎭ ⎩ 1 ⎭ ⎩ 0 ⎭ ⎩ 1 ⎭

    Given the above input-output pairs (p,t), can you find (manually)
    the weights of a perceptron to do the job?
    Boolean OR Solution

    1) Pick an
    admissable decision
    boundary

    2) Weight vector should be orthogonal to the decision boundary.

    w = 0.5
    1
    0.5

    3) Pick a point on the decision boundary to find the bias.

    T 0 + b = 0.25 + b = 0
    1 w p + b = 0.5 0.5 ⇒ b = œ0.25
    0.5
    Multiple-Neuron Perceptron: Matrix Form
    weights of one neuron
    in one row of W. w 1, 1 w 1, 2 … w 1, R
    w w … w 2, R
    W = 2, 1 2, 2
    3x2
    w S, 1 w S, 2 … w S, R

    T
    1w
    w i, 1
    i T
    W = 2w w i, 2
    iw =
    2x1
    T
    Sw w i, R

    ADVANCED
    T
    ai = har dlim(ni ) = hardlim( iw p + bi)
    Multiple-Neuron Perceptron

    Each neuron will have its own decision boundary.


    T
    iw p + bi = 0

    A single neuron can classify input vectors into


    two categories.

    An S-neuron perceptron can potentially


    classify input vectors into 2S categories.
    Perceptron Limitations
    Perceptron Limitations
    •  A single layer perceptron can only learn linearly separable
    problems.
    –  Boolean AND function is linearly separable,
    whereas Boolean XOR function is not.

    Boolean AND Boolean XOR


    Perceptron Limitations

    Linear Decision Boundary


    T
    1w p + b = 0

    Linearly Inseparable Problems


    AND Network

    x1
    W1=0.5

    Σ
    W2=0.5

    x2 W0 = -0.8

    X0=1
    Perceptron Limitations
    For a linearly not-separable problem:
    –  Would it help if we use more layers of neurons?
    –  What could be the learning rule for each neuron?

    Solution: Multilayer networks


    and the backpropagation
    learning algorithm
    •  More than one layer of perceptrons (with a hardlimiting
    activation function) can learn any Boolean function.

    •  However, a learning algorithm for multi-layer perceptrons has


    not been developed until much later
    –  backpropagation algorithm
    –  replacing the hardlimiter in the perceptron with a sigmoid
    activation function
    Summary
    •  So far we have seen how a single neuron with a threshold
    activation function separates the input space into two.

    •  We also talked about how more than one nodes may indicate
    convex (open or closed) regions

    •  The next slides = Backpropagation algorithm to learn the


    weights automatically

    You might also like