CSC701 Introduction to

Machine Learning
Introduction to Machine Learning
Course objectives and outcomes
Syllabus: MODULE 1
● Machine Learning, Types of Machine Learning- Supervised,
unsupervised and reinforcement, Issues in Machine Learning,
Application of Machine Learning, Steps in developing a
Machine Learning Application.

● Training Error, Generalization error, Overfitting, Underfitting,

Bias
TEXTBOOKS
Learning from Data
• Prediction
The world is driven by data.
• what can we predict about this
• Germany’s climate research
phenomenon?
centre generates 10
• Description
petabytes per year
• how can we describe/understand
• Google processes 24
this phenomenon in a new way?
petabytes per day
• The Large Hadron Collider
• How can we extract knowledge
produces 60 gigabytes per
from data to help humans take
minute (~12 DVDs)
decisions?
• There are over 50m credit
card transactions a day in • How can we automate decisions
the US alone. from data?
• Data is recorded from some
real-world phenomenon. • How can we adapt systems
• What might we want to do dynamically to enable better user
with that data? experiences?
 ML: Where does it fit? What is it not?

Artificial Intelligence Data Statistics /

Mining Mathematics

Computer Vision
Machine
Learning
Robotics
 Machine Learning
Two definitions of Machine Learning are offered. Arthur Samuel described it as: "the field of study that gives
computers the ability to learn without being explicitly programmed." This is an older, informal definition.
Tom Mitchell provides a more modern definition: "A computer program is said to learn from experience E
with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as
measured by P, improves with experience E."
Example: playing checkers.
E = the experience of playing many games of checkers
T = the task of playing checkers.
P = the probability that the program will win the next game.
In general, any machine learning problem can be assigned to one of two broad classifications:
Supervised learning and Unsupervised learning.
A branch of artificial intelligence, concerned with the design and development of algorithms that allow
computers to evolve behaviors based on empirical data.
As intelligence requires knowledge, it is necessary for the computers to acquire knowledge.
What is Machine Learning?
A simpler way
ML in a Nutshell Evaluation
Representation • Accuracy
●Tens of thousands • Decision trees • Precision and recall
of machine • Sets of rules / Logic programs • Squared error
learning • Instances • Likelihood
algorithms • Graphical models • Posterior probability
• Cost / Utility
(Bayes/Markov nets)
●Hundreds new • Neural networks • Margin
every year • Support vector machines
• Entropy
• K-L divergence
●Every machine • Model ensembles • Etc.
learning • Etc.
algorithm has Optimization
three • Combinatorial optimization
components: • E.g.: Greedy search
○ Representation • Convex optimization
○ Evaluation • E.g.: Gradient descent
○ Optimization • Constrained optimization
• E.g.: Linear programming
Types of Machine Learning
• Supervised (inductive) learning
• Training data includes desired outputs-
• Ex. Regression, Decision Tree, Random Forest, KNN,
Logistic Regression, back propagation neural network
• Unsupervised learning
• Training data does not include desired outputs-
• Ex. clustering, dimensionality reduction and
association rule learning..
• Semi-supervised learning
• Training data includes a few desired outputs:
Speech Analysis,Internet Content Classification
• Reinforcement learning
• Rewards from sequence of actions
Input [temperature=20] -> Model -> Output = [visitors=high]

Input [temperature=20] -> Model -> Output = [visitors=300]

Issues in Machine Learning
https://www.forbes.com/sites/bernardmarr/2018/04/30/2
7-incredible-examples-of-ai-and-machine-learning-in-
practice/?sh=7d66b9657502

https://towardsdatascience.com/data-types-from-a-
machine-learning-perspective-with-examples-
111ac679e8bc
https://www.cogitotech.com/blog/what-are-the-various-
types-of-data-sets-used-in-machine-learning
Application of Machine Learning
• Fraud detection.
• Web search results.
• Real-time ads on web pages and mobile devices.
• Text-based sentiment analysis.
• Credit scoring and next-best offers.
• Prediction of equipment failures.
• New pricing models.
• Network intrusion detection.
• Pattern and image recognition.
● Using machine learning to detect spam emails.
To: you@gmail.com
GET YOUR DIPLOMA TODAY!
If you are looking for a fast and cheap way to
get a diploma, this is the best way out for
you. Choose the desired field and degree
and call us right now: For US:
1.845.709.8044 Outside US:
+1.845.709.8044 "Just leave your NAME &
PHONE NO. (with CountryCode)" in the
voicemail. Our staff will get back to you in
next few days!

ALGORITHM
Naïve
Bayes
Rule
mining
● Using machine learning to recommend books.

ALGORITHMS
Collaborative
Filtering
Nearest Neighbour
● Using machine learning to identify faces and
expressions.

ALGORITHMS
Decision
Trees
Adaboost
• Using machine learning to identify vocal
patterns

ALGORITHMS
Feature Extraction
Probabilistic
Classifiers
Support Vector
Machines
+ many more….
● ML for working with social network data:
detecting fraud, predicting click-thru
patterns, targeted advertising, etc etc etc .

ALGORITHMS
Support Vector
Machines
Collaborative filtering
Rule mining
algorithms
Many many more….
Driving a car
Recognising spam emails
Recommending books
Reading handwriting
Recognising speech, faces, etc

How would you write these programs?

Would you want to?!?!?!?
Many applications are immensely hard to program directly.
These almost always turn out to be “pattern recognition” tasks.

1. Program the computer to do the pattern recognition task

directly.

1. Program the computer to be able to learn from examples.

2. Provide “training” data.
2000s
ML merging with statistics continues. Other subfields continue in
parallel.
First commercial-strength applications: Google, Amazon, computer
games, route-finding, credit card fraud detection, etc…
Tools adopted as standard by other fields e.g. biology
The future?
http://www.youtube.com/watch?v=NS_L3Yyv2RI

Microsoft has a MAJOR worldwide

investment in Machine Learning
Programming language : “Matlab”

MATrix LABoratory

• Interactive scripting language

• Interpreted (i.e. no compiling)
• Objects possible, not
compulsory
• Dynamically typed
• Flexible GUI / plotting
framework
• Large libraries of tools
• Highly optimized for maths
Steps in developing a Machine Learning Application
Bias Variance
Bias is formally defined as the expectation of : (the Predicted value- the actual value=error);
it is the expectation of error.
If high, model hasn’t captured patterns in the training data and hence cannot perform well on the testing
data too.
On the other hand, if our model is allowed to view the data too many times, it will learn very well for only
that data. It will capture most patterns in the data, but it will also learn from the unnecessary data present,
or from the noise.
Variance is the model’s sensitivity to fluctuations in the data. The model will perform really well on
testing data and get high accuracy but will fail to perform on new, unseen data.
Training Error, Generalization error
The training error is the error of our model as calculated on the training dataset, while
generalization error is the expectation of our model’s error were we to apply it to an infinite stream
of additional data examples drawn from the same underlying data distribution as our original
sample.
Overfitting
Overfitting and Underfitting are two crucial concepts in machine learning and are the prevalent causes for
the poor performance of a machine learning model.

OVERFITTING: the machine learning model learns the details and noise in the training data such
that it negatively affects the performance of the model on test data.
Overfitting can happen due to low bias and high variance.

● Data used for training is not cleaned and contains noise (garbage values) in it
● The model has a high variance
● The size of the training dataset used is not enough
● The model is too complex

HOW TO OVERCOME
● Using K-fold cross-validation
● Using Regularization techniques such as Lasso and Ridge
● Training model with sufficient data
● Adopting ensembling techniques
Underfitting
When a model has not learned the patterns in the training data well and is unable to
generalize well on the new data and will result in unreliable predictions, it is known as
underfitting.
● Data used for training is not cleaned and contains noise (garbage values) in it
● The model has a high bias
● The size of the training dataset used is not enough
● The model is too simple

HOW TO OVERCOME

● Increase the number of features in the dataset

● Increase model complexity
● Reduce noise in the data
● Increase the duration of training the data

In order to achieve a good fit, you need to stop training at a point where the error starts to increase.
 Validation error

Training error
Crossvalidation
 How well a model trained on the Fitting a sixth-order polynomial to
training set predicts the right output noisy data sampled from a
for new generalization instances is overfitting third-order polynomial, is
called —---------------------------- called —-------------

As the amount of training data

increases, the —---- —----
decreases.

To summarise,

● A model with a high bias error —------- data and makes

very simplistic assumptions on it
● A model with a high variance error —------------ the data and
learns too much from it
● A good model is where both —------------------------ errors are
balanced
Bias Variance trade-off
For any model, we have to find the perfect balance between Bias and Variance. This just ensures that we
capture the essential patterns in our model while ignoring the noise present it in
● when bias is high, the error in both testing and training set is also high.If we have a high variance,
the model performs well on the testing set, we can see that the error is low, but gives high error on
the training set. We can see that there is a region in the middle, where the error in both training and
testing set is low and the bias and variance is in perfect balance.
The center i.e. the bull’s eye is the model
result we want to achieve that perfectly
predicts all the values correctly. As we move
away from the bull’s eye, our model starts to
make more and more wrong predictions.

A model with low bias and high variance

predicts points that are around the center
generally, but pretty far away from each
other. A model with high bias and low
variance is pretty far away from the bull’s
eye, but since the variance is low, the
predicted points are closer to each other.