Artificial

Intelligence
Introduction to Neural Networks and
Deep Learning Frameworks

Copyright Intellipaat. All rights reserved.

 Agenda
Topology of Neural
01 Networks 02 Perceptrons

Activation Functions Perceptron Training

03 and Their Types
04 Algorithm

Deep Learning 06 What Are Tensors?

05 Frameworks

Program Elements
07 Computational Graph 08 in TensorFlow

Copyright Intellipaat. All rights reserved.

Topology of a Neural Network

Typically, artificial neural networks have a layered structure. The Input Layer picks up the input signals and passes them on to the next layer, also
known as the ‘Hidden’ Layer (there may be more than one Hidden Layer in a neural network). Last comes the Output Layer that delivers the result

Input layer Hidden layer Output layer

Copyright Intellipaat. All rights reserved.
Well, everyone has heard about AI, but how
many of you know that the inspiration behind
artificial neural networks came from the
biological neurons that are found within
human brains?

Copyright Intellipaat. All rights reserved.

 Let us first understand the architecture of
our biological neurons which is very similar to
that of artificial neurons

Copyright Intellipaat. All rights reserved.

Neurons: How Do They Work?

A neural network is a computer simulation of the way biological neurons work within a human brain

Dendrites: These branch-like structures extending away from the cell body
receive messages from other neurons and allow them travel to the cell body

Cell Body: It contains a nucleus, smooth and rough endoplasmic reticulum,

Golgi apparatus, mitochondria, and other cellular components

Axon: An axon carries an electrical impulse from the cell body to another
neuron

Copyright Intellipaat. All rights reserved.

Now, let us understand about artificial
neurons in detail!

Copyright Intellipaat. All rights reserved.

Artificial Neurons

▪ The most fundamental unit of a deep neural network is

called as an artificial neuron

▪ It takes an input, processes it, passes it through an

activation function, and returns the output

▪ Such type of artificial neurons are called as perceptrons

▪ A perceptron is a linear model used for binary

classification

Schematic Representation of a Neuron in a Neural Network

Copyright Intellipaat. All rights reserved.

Perceptron: How Does It Work?

▪ The three arrows correspond to the three inputs coming into the network

▪ Values [0.7, 0.6, and 1.4] are weights assigned to the corresponding input

▪ Inputs get multiplied with their respective weights and their sum is taken

▪ Consider the three inputs as x1, x2, and x3

▪ Let the three weights be w1, w2, and w3

Sum = x1w1 + x2w2+x3w3

Sum=x1(0.7) + x2(0.6) + x3(1.4)

▪ An offset is added to this sum. This offset is called Bias

▪ It is just a constant number, say 1, which is added for scaling purposes

New_Sum = x1(0.7) + x2(0.6) + x3(1.4) + bias

Copyright Intellipaat. All rights reserved.

Why Do We Need Weights?

▪ Statistically, weights determine the relative importance of input

▪ Mathematically, they are just the slope of the line

Copyright Intellipaat. All rights reserved.

Why Do We Need Weights?

Will it rain, if I
wear a blue
shirt?
Humidity x1
Output

Will it rain? (0/1)

Blue shirt x2

w2 is assigned a lower value because significance of the

input ‘blue shirt’ is less than ‘humidity’

Copyright Intellipaat. All rights reserved.

Why Do We Need Activation Functions?

We have two classes. One set is

represented with triangles and
the other with circles

Copyright Intellipaat. All rights reserved.

Why Do We Need Activation Functions?

Draw me a linear decision

boundary which can separate
these two classes

Copyright Intellipaat. All rights reserved.

Why Do We Need Activation Functions?

We will have to add a third

dimension to create a linearly
separable model which is easy to
deal with

Copyright Intellipaat. All rights reserved.

Activation Functions

▪ They are used to convert an input signal of a node in an

artificial neural network to an output signal

▪ That output signal now is used as an input in the next layer in

the stack

▪ Activation functions introduce non-linear properties to our

network

▪ A neural network without an activation function is essentially

just a linear regression model

▪ The activation function does non-linear transformation to the

input making it capable to learn and perform more complex

tasks

Copyright Intellipaat. All rights reserved.

 Identity Types of Activation Functions

Binary Step

Sigmoid

Tanh

ReLU

Leaky ReLU

Softmax
Copyright Intellipaat. All rights reserved.
Identity Function
• A straight line function where activation is proportional to input

• No matter how many layers we have, if all of them are linear in nature, the final activation function of the last

layer will be nothing but just a linear function of the input of the first layer

• We use a linear function to solve a linear regression problem

• Range: (−∞,∞)

𝒇 𝒙 =𝒙

Copyright Intellipaat. All rights reserved.

 Binary Step Function
• It is also known as the Heaviside step function, or the unit step function, which is usually denoted by H or θ, is a

discontinuous function

• Its value is 0 for the negative argument and 1 for the positive argument

• It depends on the threshold value we define

• We use the binary step function to solve a binary classification problem

𝟎 𝒇𝒐𝒓 𝒙 < 𝟎 • Range: (0,1)
𝒇 𝒙 =ቐ
𝟏 𝒇𝒐𝒓 𝒙 = > 𝟎

Copyright Intellipaat. All rights reserved.

 Sigmoid Function
• The sigmoid function is an activation function where it scales values between 0 and 1 by applying a threshold

• When we apply the weighted sum in the place of x, the values are scaled in between 0 and 1

• Large negative numbers are scaled toward 0, and large positive numbers are scaled toward 1

• Range: (0,1)

𝟏
𝒇 𝒙 =
𝟏 + ⅇ−𝒙

Copyright Intellipaat. All rights reserved.

 Tanh Function
• It is a hyperbolic trigonometric function

• The Tanh activation works almost always better than sigmoid functions as optimization is easier in this method

• The advantage of Tanh is that it can deal more easily with negative numbers

• It is actually a mathematically shifted version of the sigmoid function

• Range: (−1,1)

𝟐
𝒇 𝒙 : 𝒕𝒂𝒏 𝒉 𝒙 = 𝒙 −𝟏
𝟏+ⅇ−𝟐

Copyright Intellipaat. All rights reserved.

 ReLU Function
• ReLU stands for rectified linear unit

• It is the most widely used activation function

• It is primarily implemented in Hidden Layers of the neural network

• This function allows only the maximum values to pass during the front propagation as shown in the graph below

• Range: (0,∞)
𝟎 𝒇𝒐𝒓 𝒙 < 𝟎
𝒇 𝒙 =ቐ
𝒙 𝒇𝒐𝒓 𝒙 = > 𝟎

Copyright Intellipaat. All rights reserved.

 Leaky ReLU Function
• Leaky ReLU allows a small negative value to pass during the back propagation if we have a dead ReLU problem

• This eventually activates the neuron and brings it down

• Range: (−∞,∞)

𝟎. 𝟎𝟏𝒙 𝒇𝒐𝒓 𝒙 < 𝟎

𝒇 𝒙 =ቐ
𝒙 𝒇𝒐𝒓 𝒙 = > 𝟎

Copyright Intellipaat. All rights reserved.

 Softmax Function
• The Softmax function is used when we have multiple classes

• It is useful for finding out the class which has the max. probability

• The Softmax function is ideally used in the Output Layer of the classifier where we are actually trying to attain the

probabilities to define the class of each input

• Range: (0,1)

ⅇ𝒛 𝒋
𝜎 𝒛 𝒋 = σ𝑲 𝒛 , 𝒋 = 𝟏, 𝟐, . . 𝑲
𝒌=𝟏 ⅇ 𝒌

Copyright Intellipaat. All rights reserved.

 Got bored again? Let us get back to
perceptrons and try to understand them in a
better way!

Copyright Intellipaat. All rights reserved.

Training a Perceptron

By training a perceptron, we try to find a line, plane, or some hyperplane which can accurately separate two
classes by adjusting weights and biases

Error = 2 Error = 1 Error = 0

Copyright Intellipaat. All rights reserved.

Perceptron Training Algorithm

Input

W1
Bias
X1

Activation Initialize weights,

X2 W2 Function bias, and threshold

෍ Output
Calculate the sum
Update weights
X3 W3 and pass through an
activation function 𝜕𝐸
Wnew = Wold – LR*( )
𝜕𝑤

Error Produce the output

(Error)
Xn Wn
(Correct)

Stop

Copyright Intellipaat. All rights reserved.

Benefits of Using Artificial Neural Networks

Organic Non-linear Data

Fault Tolerance Self-repairing
Learning Processing

Copyright Intellipaat. All rights reserved.

Let us now move toward Deep
Learning frameworks!

Copyright Intellipaat. All rights reserved.

Deep Learning Frameworks

These Deep Learning libraries help in implementing artificial neural networks

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J
TensorFlow is an open-source software library for high-performance numerical computations
MXNet

Developed by Google

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J

MXNet Natural language

Forecasting
processing

Text classification Tagging

Google Translate

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J

MXNet
Tensor
Used for visualizing TensorFlow computations and graphs
Board

TensorFlow Used for rapid deployment of new algorithms/experiments

Serving while retaining the same server architecture and APIs

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J

MXNet

A high-Level API which can run on top of

TensorFlow, Theano, or CNTK

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J

MXNet
A recurrent neural network

A convolutional neural network

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J

MXNet

Stacks layers on top

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J

MXNet

A scientific computing framework developed by

Facebook

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J

MXNet

‘Pythonic’ in nature

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J

MXNet

Offers dynamic computational

graphs

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J

MXNet

A Deep Learning programming

library written for Java

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J

MXNet

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J

MXNet
Image recognition Fraud detection

Parts of speech
Text mining
tagging

Natural language
processing

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J

MXNet

Developed by Apache
Software Foundation

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J

MXNet

Copyright Intellipaat. All rights reserved.

TensorFlow

Keras

PyTorch

DL4J

MXNet
Speech
Imaging
recognition

Forecasting NLP

Copyright Intellipaat. All rights reserved.

What Are Tensors?

A tensor is a multi-dimensional array in which data is stored

Tensor is given
as an input to a
neural network

Tensor

Copyright Intellipaat. All rights reserved.

Tensor Rank

Tensor rank represents the dimension of the n-dimensional array

Rank Math Entity Example

0 Scalar (magnitude only) s = 483

1 Vector (magnitude and v = [1.1, 2.2, 3.3]

direction)
2 Matrix (table of numbers)
m = [1, 2, 3], [4, 5, 6], [7, 8, 9]]
3 3-Tensor (cube of numbers) t=
[[[2], [4], [6]], [[8], [10], [12]], [
[14], [16], [18]]]
n n-Tensor ……

Copyright Intellipaat. All rights reserved.

Computational Graph

Computation is done in the form of a graph

a = 10
b = 20 Addition
c = 30 c
ℎ = 𝑎∗𝑏 +𝑐
Multiplication

a b

Copyright Intellipaat. All rights reserved.

Computational Graph

The computational graph is executed inside a session

h h

a = 10
b = 20 Addition Addition
c = 30 c c
ℎ = 𝑎∗𝑏 +𝑐
Multiplication Multiplication

a b a b

Session

Copyright Intellipaat. All rights reserved.

Computational Graph

The computational graph is executed inside a session

a = 10 Node -> Mathematical operation

b = 20 Addition
c = 30 c
ℎ = 𝑎∗𝑏 +𝑐
Multiplication
Edge -> Tensor

a b

Copyright Intellipaat. All rights reserved.

Program Elements in TensorFlow

Constant Placeholder Variable

Copyright Intellipaat. All rights reserved.

 Constant Constants are program elements whose values do not change

Placeholder

Variable a=tf.constant(10) b=tf.constant(20)

Copyright Intellipaat. All rights reserved.

 Constant A placeholder is a program element to which we can assign data at a later
time

Placeholder

Variable x=tf.placeholder(tf.float32) y=tf.placeholder(tf.string)

Copyright Intellipaat. All rights reserved.

 Constant A variable is a program element which allows us to add new trainable
parameters to the graph

Placeholder

Variable W=tf.Variable([3],tf.float32) b=tf.Variable([0.4],tf.float32)

Copyright Intellipaat. All rights reserved.

Quiz

Copyright Intellipaat. All rights reserved.

Quiz 1

A tensor is a single-dimensional array in

which data is stored

A True

B False

Copyright Intellipaat. All rights reserved.

Answer 1

A tensor is a single-dimensional array in

which data is stored

A True

B False

Copyright Intellipaat. All rights reserved.

Quiz 2

How many layers does a standard Neural

Network has?

A 1

B 2

C 3

D 4 or more

Copyright Intellipaat. All rights reserved.

Answer 2

How many layers does a standard Neural

Network has?

A 1

B 2

C 3

D 4 or more

Copyright Intellipaat. All rights reserved.

Quiz 3

Perceptron is other name of Neurons

A Yes

B No

Copyright Intellipaat. All rights reserved.

Answer 3

Perceptron is other name of Neurons

A Yes

B No

Copyright Intellipaat. All rights reserved.

Thank you!

Copyright Intellipaat. All rights reserved.

India: +91-7847955955

US: 1-800-216-8930 (TOLL FREE)

sales@intellipaat.com

24/7 Chat with Our Course Advisor

Copyright Intellipaat. All rights reserved.