19 - Introduction To Neural Networks

19 - Introduction to Neural Networks
1. Artificial Neural Network (ANN) is an efficient computing system whose

central theme is borrowed from the analogy of biological neural
networks.
2. ANNs are also named as “artificial neural systems,” or “parallel
distributed processing systems,” or “connectionist systems.”
3. ANN acquires a large collection of units that are interconnected in some
pattern to allow communication between the units.
4. These units, also referred to as nodes or neurons, are simple processors
which operate in parallel.
5. Every neuron is connected with other neuron through a connection
link.
6. Each connection link is associated with a weight that has information
about the input signal. This is the most useful information for neurons
to solve a particular problem because the weight usually excites or
inhibits the signal that is being communicated.
7. Each neuron has an internal state, which is called an activation signal.
Output signals, which are produced after combining the input signals
and activation rule, may be sent to other units.
20 - The Perceptron
Perceptron
Developed by Frank Rosenblatt by using McCulloch and Pitts model,
perceptron is the basic operational unit of artificial neural networks. It
employs supervised learning rule and is able to classify the data into two
classes.
Operational characteristics of the perceptron: It consists of a single neuron
with an arbitrary number of inputs along with adjustable weights, but the
output of the neuron is 1 or 0 depending upon the threshold. It also consists
of a bias whose weight is always 1. Following figure gives a schematic
representation of the perceptron.
Perceptron thus has the following three basic elements −
 Links − It would have a set of connection links, which carries a weight
including a bias always having weight 1.
 Adder − It adds the input after they are multiplied with their respective
weights.
 Activation function − It limits the output of neuron. The most basic
activation function is a Heaviside step function that has two possible
outputs. This function returns 1, if the input is positive, and 0 for any
negative input.
Training Algorithm
Perceptron network can be trained for single output unit as well as multiple
output units.
Training Algorithm for Single Output Unit
Step 1 − Initialize the following to start the training −
 Weights
 Bias
 Learning rate α
For easy calculation and simplicity, weights and bias must be set equal to 0
and the learning rate must be set equal to 1.
Step 2 − Continue step 3-8 when the stopping condition is not true.
Step 3 − Continue step 4-6 for every training vector x.
Step 4 − Activate each input unit as follows −
xi=si(i=1ton)
Step 5 − Now obtain the net input with the following relation −
yin=b+∑(i se n tak ) xi.wi
Here ‘b’ is bias and ‘n’ is the total number of input neurons.
Step 6 − Apply the following activation function to obtain the final output.
Step 7 − Adjust the weight and bias as follows −

Case 1 − if y ≠ t then,
wi(new)=wi(old)+αtxi
b(new)=b(old)+αt
Case 2 − if y = t then,
wi(new)=wi(old)
b(new)=b(old)
Here ‘y’ is the actual output and ‘t’ is the desired/target output.
Step 8 − Test for the stopping condition, which would happen when there is
no change in weight.
21 - The Backpropagation Algorithm
Backpropagation is a training method used for a multi layer neural network. It

is also called the generalized delta rule. It is a gradient descent method which
minimizes the total squared error of the output computed by the net.
The training of a neural network by back propagation takes place in three

stages
1. Feedforward of the input pattern
2. Calculation and Back propagation of the associated error
3. Adjustments of the weights.

Concept:
During feedforward, each input unit (Xi) receives an input signal and sends this
signal to each of the hidden units Z1, Z2, …Zj. Each hidden unit computes its
activation and sends its signal to each output unit. Each output unit computes
its activation to compute the output or the response of the neural net for the
given input pattern.
Algorithm
Step1: Initialize weights
Step 2: While stopping condition is false, do steps 3 to 10
Step 3: For each training pair, do steps 4 - 9
Feed forward
Step 4: Input unit receives input signal and propagates it to all units in the
hidden layer
Step 5: Each hidden unit sums its weighted input signals
Step 6: Each output unit sums its weighted input signals and applied its
activation function to compute its output signal.
Backpropagation
Step 7: Each output unit receives a target pattern corresponding to the input
training pattern, computes its error information term
δk = ( tk – yk) f’ (y_ink)
Calculates its bias correction term ∆Wok = αδk
And sends δk to units in the layer below
Step 8: Each hidden unit sums its delta inputs ∑ =− = m k jkkj win 1 δδ
Multiplies by the derivative of its activation function to calculate its error

information term
)_('_ jjj inzfin δδ =
Calculates its weight correction term ∆vij = αδjxi And calculates its bias
correction term ∆voj = αδj Update weights and biases
Step 9: Each output unit updates its bias and weights
Wjk(new) = wjk(old) + ∆ wjk
Each hidden unit updates its bias and weights
Vij (new) = vij (old) + ∆vij
Step 10: Test stopping condition
22 - Training Procedures
Convolutional Neural Network

1. A convolutional neural network (CNN) is a type of artificial neural
network used in image recognition and processing that is specifically
designed to process pixel data.
2. A CNN uses a system much like a multilayer perceptron that has been
designed for reduced processing requirements.
3. The layers of a CNN consist of an input layer, an output layer and a

hidden layer that includes multiple convolutional layers, pooling layers,
fully connected layers and normalization layers.
4. A CNN typically has three layers: a convolutional layer, pooling layer,

and fully connected layer.
Convolution Layer
The convolution layer is the core building block of the CNN. It carries the
main portion of the network’s computational load. This layer performs a
dot product between two matrices, where one matrix is the set of
learnable parameters otherwise known as a kernel, and the other matrix is
the restricted portion of the receptive field. The kernel is spatially smaller
than an image, but is more in-depth. This means that, if the image is
composed of three (RGB) channels, the kernel height and width will be
spatially small, but the depth extends up to all three channels.
Pooling Layer
The pooling layer replaces the output of the network at certain locations
by deriving a summary statistic of the nearby outputs. This helps in
reducing the spatial size of the representation, which decreases the
required amount of computation and weights. The pooling operation is
processed on every slice of the representation individually.
Fully Connected Layer

Neurons in this layer have full connectivity with all neurons in the
preceding and succeeding layer as seen in regular FCNN. This is why it can
be computed as usual by a matrix multiplication followed by a bias effect.
The FC layer helps map the representation between the input and the
output.
Applications
1. Object detection: With CNN, we now have sophisticated models like R-

CNN, Fast R-CNN, and Faster R-CNN that are the predominant pipeline for
many object detection models deployed in autonomous vehicles, facial
detection, and more.
2. Semantic segmentation: Semantic segmentation describes the process of

associating each pixel of an image with a class label, (such as flower, person,
road, sky, ocean, or car). Applications for semantic segmentation include:
Autonomous driving.
3. Image captioning: CNNs are used with recurrent neural networks to

write captions for images and videos.

19 - Introduction To Neural Networks

Uploaded by

Copyright:

Available Formats

19 - Introduction To Neural Networks

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

19 - Introduction To Neural Networks

Uploaded by

Copyright:

Available Formats

19 - Introduction to Neural Networks

1. Artificial Neural Network (ANN) is an efficient computing system whose

Step 7 − Adjust the weight and bias as follows −

Backpropagation is a training method used for a multi layer neural network. It

The training of a neural network by back propagation takes place in three

1. Feedforward of the input pattern

2. Calculation and Back propagation of the associated error

3. Adjustments of the weights.

Step1: Initialize weights

Step 2: While stopping condition is false, do steps 3 to 10

Step 3: For each training pair, do steps 4 - 9

Step 5: Each hidden unit sums its weighted input signals

Calculates its bias correction term ∆Wok = αδk

And sends δk to units in the layer below

Multiplies by the derivative of its activation function to calculate its error

)_('_ jjj inzfin δδ =

Step 9: Each output unit updates its bias and weights

Wjk(new) = wjk(old) + ∆ wjk

Each hidden unit updates its bias and weights

Vij (new) = vij (old) + ∆vij

Step 10: Test stopping condition

Convolutional Neural Network

3. The layers of a CNN consist of an input layer, an output layer and a

4. A CNN typically has three layers: a convolutional layer, pooling layer,

Fully Connected Layer

1. Object detection: With CNN, we now have sophisticated models like R-

2. Semantic segmentation: Semantic segmentation describes the process of

3. Image captioning: CNNs are used with recurrent neural networks to

You might also like