ARTIFICIAL NEURAL NETWORK

Machine Learning

Benedict C. Borja Engr. Kristoffer Jan B. Notario

BSEE VA Instructor
Brief History of Artificial Intelligence
Ever since the 1950’s scientist has been trying to mimic
the functioning of neuron ang use it to build smarter
robots.

After a lot of trial and error, humans finally designed a

computer that can recognized human speech.
Brief History of Artificial Intelligence

It was only 2000 that humans were

able to give birth to Deep Learning
that was able to see and distinguish
between different images and videos.
 What is Deep Learning ?
Deep Learning is a machine learning technique
that teaches computers to do what comes
naturally to humans: learn by example
 To understand how an Artificial Neuron works, we need
to understand how the real one works

BIOLOGICAL NEURAL NETWORK

ARTIFICIAL NEURAL NETWORK
What is the Difference Between Computer and
Human Brain?
Artificial Neural Networks (ANN) and Biological Neural Networks
(BNN) – Difference
Artificial Neural Networks with Biological Neural
Network – Similarity
Neural Networks resemble the human brain in
the following two ways –

o A neural network acquires knowledge through

learning.
o A neural network’s knowledge is stored within
inter-neuron connection strengths known as
synaptic weights.
Artificial Neural Networks with Biological Neural
Network – Similarity
The dendrites in the Biological Neural Network are analogous
to the weighted inputs based on their synaptic
interconnection in the Artificial Neural Network.

Artificial Neural Network (ANN) With Biological Neural

Network (BNN) – Comparison
The cell body is comparable to the artificial neuron unit
in the Artificial Neural Network which also comprises of
summation and threshold unit.

Axon carries output that is analogous to the output unit

in the case of an Artificial Neural Network. So, ANN is
modeled using the working of basic biological neurons.

Artificial Neural Network (ANN) With Biological Neural

Network (BNN) – Comparison
 ARTIFICIAL NEURAL NETWORK
A Neural Network is a system or hardware
and/or software patterned after the
operation of neurons in the human brain.
Neural Networks also called Artificial
Neural Network is a way of achieving Deep
Learning.
How Does Artificial Neural Network Works?

Artificial Neural Networks can be viewed as weighted directed

graphs in which artificial neurons are nodes, and directed edges
with weights are connections between neuron outputs and
neuron inputs.
How Does Artificial Neural Network Works?

The Artificial Neural Network receives information from the

external world in the form of pattern and image in vector form.
These inputs are mathematically designated by the notation x(n)
for n number of inputs.

Each input is multiplied by its corresponding weights. Weights

are the information used by the neural network to solve a
problem. Typically weight represents the strength of the
interconnection between neurons inside the Neural Network.
How Does Artificial Neural Network Works?
The sum corresponds to any numerical value
ranging from 0 to infinity. To limit the response to
arrive at the desired value, the threshold value is set
up. For this, the sum is passed through an
activation function.

The activation function is set to the transfer

function used to get the desired output. There are
linear as well as the nonlinear activation function.
Types of Neural Networks in Artificial Intelligence
Neural Network Architecture Types
 Perceptron Model in Neural Networks
Neural Network is having two input units and one output
unit with no hidden layers. These are also known as ‘single-
layer perceptrons’.

Radial Basis Function Neural Network

These networks are similar to the feed-forward Neural
Network except radial basis function is used as the activation
function of these neurons.

Neural Network Architecture Types

 Multilayer Perceptron Neural Network
These networks use more than one hidden layer of neurons, unlike
single-layer perceptron. These are also known as Deep Feedforward
Neural Networks.

Recurrent Neural Network

Type of Neural Network in which hidden layer neurons have self-
connections. Recurrent Neural Networks possess memory. At any
instance, hidden layer neuron receives activation from the lower layer
as well as its previous activation value.

Neural Network Architecture Types

 Long Short-Term Memory Neural Network (LSTM)
Type of Neural Network in which memory cell is incorporated into hidden layer neurons is
called LSTM network.

Hopfield Network
A fully interconnected network of neurons in which each neuron is connected to every other
neuron. The network is trained with input patterns by setting a value of neurons to the desired
pattern. Then its weights are computed. The weights are not changed. Once trained for one or
more patterns, the network will converge to the learned patterns. It is different from other
Neural Networks.

Boltzmann Machine Neural Network

These networks are similar to the Hopfield network except some neurons are input, while
others are hidden in nature. The weights are initialized randomly and learn through the
backpropagation algorithm.

Neural Network Architecture Types

 Modular Neural Network
It is the combined structure of different types of a neural networks like
multilayer perceptron, Hopfield Network, Recurrent Neural Network,
etc. which are incorporated as a single module into the network to
perform independent subtask of whole complete Neural Networks.

Physical Neural Network

In this type of Artificial Neural Network, electrically adjustable
resistance material is used to emulate the function of synapse instead
of software simulations performed in the neural network.

Neural Network Architecture Types

Learning Techniques in Artificial Neural Networks
 Training Algorithms For Artificial Neural Networks
Gradient Descent Algorithm
This is the simplest training algorithm used in the case of a supervised
training model. In case, the actual output is different from the target
output, the difference or error is find out. The gradient descent
algorithm changes the weights of the network in such a manner to
minimize this mistake.

Back Propagation Algorithm

It is an extension of the gradient-based delta learning rule. Here, after
finding an error (the difference between desired and target), the error
is propagated backward from the output layer to the input layer via the
hidden layer. It is used in the case of Multilayer Neural Network.
 Learning Data Sets in Artificial Neural Networks
Training Data Set
A set of examples used for learning, that is to fit the parameters [i.e., weights]
of the network. One approach comprises of one full training cycle on the
training set.

Validation Set Approach

A set of examples used to tune the parameters [i.e., architecture] of the
network. For example to choose the number of hidden units in a Neural
Network.

Making Test Set

A set of examples used only to assess the performance [generalization] of a
fully specified network or to apply successfully in predicting output whose
input is known.
 Five Algorithms to Train a Neural Network

 Hebbian Learning Rule

 Self – Organizing Kohonen Rule
 Hopfield Network Law
 LMS algorithm (Least Mean Square)
 Competitive Learning
A typical Neural Network contains a large number of
artificial neurons called units arranged in a series of layers.
Artificial Neural Network Architecture
Input layer – It contains those units (Artificial Neurons)
which receive input from the outside world on which the
network will learn, recognize about or otherwise process.

Output layer – It contains units that respond to the

information about how it’s learned any task.

Hidden layer – These units are in between input and output

layers. The job of the hidden layer is to transform the input
into something that the output unit can use in some way.
Artificial Neural Network Architecture
 Learning Techniques in Neural Networks
Supervised Learning
In supervised learning, the training data is input to the network, and the desired output is known weights are
adjusted until production yields desired value.

Unsupervised Learning
The input data is used to train the network whose output is known. The network classifies the input data and
adjusts the weight by feature extraction in input data.

Reinforcement Learning
Here the value of the output is unknown, but the network provides feedback on whether the output is right
or wrong. It is Semi-Supervised Learning.

Offline Learning
The adjustment of the weight vector and threshold is made only after all the training set is presented to the
network. It is also called Batch Learning.

Online Learning
The adjustment of the weight and threshold is made after presenting each training sample to the network.
 4 Different Techniques of Neural Networks
Classification Neural Network
A Neural Network can be trained to classify a given pattern or dataset into a predefined class. It uses
Feedforward Networks.

Prediction Neural Network

A Neural Network can be trained to produce outputs that are expected from a given input. E.g., – Stock
market prediction.

Clustering Neural Network

The Neural network can be used to identify a unique feature of the data and classify them into different
categories without any prior knowledge of the data.

Association Neural Network

A Neural Network can be trained to remember the particular pattern so that when the noise pattern is
presented to the network, the network associates it with the closest one in the memory or discard it. E.g. ,
Hopfield Networks which performs recognition, classification, and clustering, etc.
Neural Network for Deep Learning

Feed-forward neural networks

Recurrent neural network
Multi-layer perceptrons (MLP)
Convolutional neural networks
Recursive neural networks
Deep belief networks
Convolutional deep belief networks
Self-Organizing Maps
Deep Boltzmann machines
Stacked de-noising auto-encoders
 Neural Network for Machine Learning

Multilayer Perceptron (supervised classification)

Back Propagation Network (supervised classification)

Hopfield Network (for pattern association)

Deep Neural Networks (unsupervised clustering)

Applications of Neural Networks
 Limitations of Neural Networks

• The neural network needs the training to operate.

• The architecture of a neural network is different
from the architecture of microprocessors,
therefore, needs to be emulated.
• Requires high processing time for large neural
networks.
Neural Networks in 5 Minutes