High Impact Skills Development Program

in Artificial Intelligence, Data Science, and Blockchain

Module 1: AI Fundamentals
Lecture 3: Introduction to Deep Learning

Instructor: Dr Syed Imran Ali

Assistant Professor, SEECS, NUST

Courtesy: Dr. Faisal Shafait and Dr. Adnan ul Hasan 1

What is Machine
Learning?

2
Example
• Handwritten Digit Recognition

3
Goal of Machine Learning
• Consider a 28 x 28 pixel image
• Represented by a 784-dimensional vector x
• Goal: build a machine that takes the vector x as input
and produces the identity of digit 0,…,9 as the output
• Tom Mitchel’s definition of Machine Learning:
A computer program is said to learn from experience E
with respect to some class of tasks T and performance
measure P, if its performance at tasks in T, as
measured by P, improves with experience E.

4
Important Terminology
• Training data
• Data samples
• Target vectors
• Learning / Training
• Machine takes training data and automatically learns mapping
from data samples to target vectors
• Test data
• Target vectors are concealed from the machine
• Machine predicts the target vectors based on previously
learned model
• Accuracy can be evaluated by comparing the predicted
vectors to the actual vectors

5
What is AI, Machine (Deep) Learning

6
What is AI, Machine (Deep) Learning

7
What is AI, Machine (Deep) Learning

8
Types of Machine
Learning Problems

9
Types of Machine Learning Problems
• Supervised learning
• Regression
• Classification
• Unsupervised learning
• Clustering
• Density estimation

10
Machine Learning vs Deep Learning

11
The “deep” in Deep Learning
• Stands for this idea of successive layers of
representations
• How many layers contribute to a model of the data is
called the depth of the model
• Modern deep learning often involves tens or even
hundreds of successive layers of representations —
and they’re all learned automatically from exposure to
training data
• Meanwhile, other approaches to machine learning tend
to focus on learning only one or two layers of
representations of the data; hence, they’re sometimes
called shallow learning
12
Achievements
• Near-human-level image classification, speech
recognition, handwriting transcription, autonomous
driving
• Improved machine translation, text-to-speech
conversion, search results on the web, ad targeting
(Google, Baidu, and Bing)
• Digital assistants such as Google Now and Amazon
Alexa
• Ability to answer natural-language questions
• Superhuman Go playing
• Artificial Arts and game playing

13
 Memory Lane

Backpropagation, Deep Learning, Transformers,

Neural Networks RBM, RNN DBN GANs GPT GPT 3.5

1957 1970
1979 2016 2017
1943 – – 2006 2009 2014 2021 2022
-98 -17 -19
1962 1986

Perceptrons CNN, LSTM, ImageNet, AlphaGo, DALL-E

MNIST AlexNet AlphaZero

Deep Learning Boom

14
Why Deep Learning is so successful?

15
Why Deep Learning is so successful?

16
 Why Deep Learning is so successful?

• We provide the raw data of images to the first layer of the input layer.
• After then, these input layer will determine the patterns of local contrast that
means it will differentiate on the basis of colors, luminosity, etc.

17
 Why Deep Learning is so successful?

• Then the 1st hidden layer will determine the face feature
• For example, it will fixate on eyes, nose, and lips, etc.
• And then, it will fixate those face features on the correct face template.

18
 Why Deep Learning is so successful?

• So, in the 2nd hidden layer, it will actually determine the correct face here as
it can be seen in the above image,
• after which it will be sent to the output layer.

19
 Why Deep Learning is so successful?

• Likewise, more hidden layers can be added to solve more complex

problems
• for example, if you want to find out a particular kind of face having light
complexions.
• So, as and when the hidden layers increase, we are able to solve complex
20
problems.
Typical Cycle Design

Performan
Data Feature Model Model
ce
Collection Choice Selection Training Evaluation

21
 Typical Cycle Design

Performan
Data Feature Model Model
ce
Collection Choice Selection Training Evaluation

• Training Set: Example used for learning | Target Value is known

• Validation Set: Architecture Tuning and Error Estimation

• Test Set: Assess the performance | Never seen before

22
Typical Cycle Design

Performan
Data Feature Model Model
ce
Collection Choice Selection Training Evaluation

• Easy to extract
• Highly discriminative
• Noise robust
• Invariant to irrelevant transformations
23
Typical Cycle Design

Performan
Data Feature Model Model
ce
Collection Choice Selection Training Evaluation

• Choose model whose assumption fits your data

• Comes with experiences

24
Typical Cycle Design

Model Performan
Data Feature Model
ce
Collection Choice Selection Training Evaluation

• Training constraints
• Parameters
• Best practices

25
Typical Cycle Design

Data Feature Model Model Performance

Collection Choice Selection Training Evaluation

• What measures to use?

• Different measures can make your results great or
terrible
• Always be a bit skeptical if you get great performance
very early on

26
Happy
Learning!