Unit 1: Introduction of Machine

Learning
Asst.Prof. S.B.Mehta
Computer Science and Engineer

Presented by : Prof. S.B.Mehta

 Machine Learning

Presented by : Prof. S.B.Mehta

 Machine Learning
 Basics :
AI vs ML vs Deep Learning
• Artificial Intelligent : 1956 John McCorthy : Machine to mimic human
behaviour
• Machine Learning : 1959 Arthur Samuel : Machine to Learn by itself form its
experience like human learn i.e. Machine to learn
1. Training Time (Execution Time )for particular Model required less than
Deep Learning .
2. Testing Time is Large .
3. Machine Learning deals with small amount of data
• Deep Learning : 2000 Igor Aizenberg : Algo. Inspired by Structure & function
of Human brain
1. Training Time (Execution Time )for particular Model required More than
Machine Learning bcoz it require deals with huge amount of data .
2. Testing Time is Less
3. Deep Learning deals with huge amount of data like structure or
Presented by : Prof.uSn.B
st.M
ruechttua red data
 Introduction of Machine Learning Cont..

 Basics :

• Machine learning is a branch of the field called Artificial

intelligence which aims to create intelligent systems which do
human like jobs by learning from a lot of relevant data.
• To give computers the ability to learn without being explicitly
programmed.

Presented by : Prof. S.B.Mehta

 Introduction of Machine Learning
Conti..
 Basics :

Data (Input)
Traditional Output
Program Program

Data (Input)
Program
Machine Learning
Output

Presented by : Prof. S.B.Mehta

 How do Machines learn?

 Basics :

• It work , just like humans do! First, we receive the knowledge about a
certain thing and then keeping this knowledge in mind, we are able to
identify the thing in the future. Also, past experiences help us in
taking decisions accordingly in the future. Our brain trains itself by
identifying the features and patterns in knowledge/data received, thus
enabling itself to successfully identify or distinguish between various
things.

Presented by : Prof. S.B.Mehta

 Machine Learning Process

Presented by : Prof. S.B.Mehta

 How Does Machine Learning Work
 There are basic steps used to perform a machine learning task:
1. Collecting and preparing data:

2. Training a model:
3. Evaluating the model:
4. Improving the performance

Presented by : Prof. S.B.Mehta

 Advantages of ML
• Easily identifies trends and patterns
 Machine Learning can review large volumes of data and discover specific
trends and patterns that would not be apparent to humans.
• No human intervention needed (automation)
 A very powerful utility of Machine Learning is its ability to automate
various decision-making tasks. Because of the machine learning technique,
we don’t need to assist our system or give it commands to follow certain
instructions.
• Continuous Improvement
 Machine Learning algorithms are capable of learning from the data we
provide helps the system to continuously understand the errors and resulted
rectification for that errors. Hence this increases efficiency and accuracy

• Handling multi-dimensional and multi-variety data :managing and improving

the multi-dimensional and large amount of data

• Wide Applications: ML can be helpful for those who are in the field of e-
commerce or the healthcare providers
Presented by : Prof. S.B.Mehta
 Disadvantages of ML
• Data acquisition: In the process of machine learning, a large amount
of data is used in the process of training and learning. Many times we face
a situation where we find an imbalance in data which leads to poor
accuracy of models
• Time-consuming: Larger the volume of data, the time to learn from
data and process it also increases
• Algorithm Selection: A Machine Learning problem can implement
various algorithms to find a solution. It is a manual and tedious task to run
models with different algorithms and identify the most accurate algorithm
based on the results
• High error susceptibility

Presented by : Prof. S.B.Mehta

 •Real World Application of Machine Learning

Presented by : Prof. S.B.Mehta

 •Real World Application of Machine Learning

1. Virtual Personal Assistants:

2. Traffic Prediction:
3. Email Spam filtering:
4. Social Media services:
5. Image Recognition:
6. Product recommendations:
7. Automatic Language Translation:
8. Online Fraud Detection:
9. Self-driving cars:
10. Medical Diagnosis:

Presented by : Prof. S.B.Mehta

 •Life Cycle of Machine Learning

Presented by : Prof. S.B.Mehta

 •Life Cycle of Machine Learning
Machine learning life cycle is a cyclic process to build an
efficient machine learning project.

Machine learning life cycle involves seven major steps

1. Gathering Data
2. Data preparation
3. Data Wrangling
4. Analyze Data
5. Train the model
6. Test the model
7. Deployment

Presented by : Prof. S.B.Mehta

Types of Machine Learning Algorithms

Presented by : Prof. S.B.Mehta

 Supervised Learning Algorithms
In supervised learning, we use known or labeled data for the training data.
Since the data is known, the learning is supervised, i.e., directed into
successful execution. The input data goes through the Machine Learning
algorithm and is used to train the model. Once the model is trained based on
the known data, you can use unknown data into the model and get a new
response

Presented by : Prof. S.B.Mehta

Example Supervised Learning Algorithms

Presented by : Prof. S.B.Mehta

Type of Supervised Learning Algorithms

Presented by : Prof. S.B.Mehta

 Supervised Learning Algorithms Conti..
Supervised Machine Learning Algorithms can be broadly divided into
two types of algorithms

1. Classification techniques: Classification algorithms are used when the

output variable is categorical, which means there are two classes such as
Yes-No, Male-Female, True-false, etc.
Algorithms
• Random Forest
• Decision Trees
• Logistic Regression
• Support vector Machines
• K-NN Algorithms

1. Regression techniques :These algorithms are used to determine the

mathematical relationship between two or more variables and the level of
dependency between variables
Regression Algorithms:
• Linear Regression
• Regression Trees
• Non-Linear Regression
Presented by : •ProfB
. Sa.B a Linear Regression
y.eMseihatn
 Supervised Learning Algorithms Conti..
Advantages:
oWith the help of supervised learning, the model can predict the output on the basis of
prior experiences.
oIn supervised learning, we can have an exact idea about the classes of objects.

oSupervised learning model helps us to solve various real-world problems such

as fraud detection, spam filtering,

Disadvantages:

oSupervised learning models are not suitable for handling the complex tasks.
oSupervised learning cannot predict the correct output if the test data is different from

the training dataset.

oTraining required lots of computation times.

oIn supervised learning, we need enough knowledge about the classes of object.

Presented by : Prof. S.B.Mehta

 Unsupervised Learning Algorithms

Unsupervised learning is a type of machine learning in which models are trained

using unlabeled dataset and are allowed to act on that data without any supervision

Presented by : Prof. S.B.Mehta

Working of Unsupervised Learning
Algorithms

Presented by : Prof. S.B.Mehta

 Type of Unsupervised Learning Algorithms

Presented by : Prof. S.B.Mehta

 Type of Unsupervised Learning Algorithms
Unsupervised Learning uses technique:
Clustering : Clustering is a method of grouping the objects into clusters
such that objects with most similarities remains into a group and has less or no
similarities with the objects of another
Example:

Presented by : Prof. S.B.Mehta

 Type of Unsupervised Learning Algorithms
Unsupervised Learning uses technique:
Association: An association rule is an unsupervised learning method which
is used for finding the relationships between variables in the large database. It
determines the set of items that occurs together in the dataset.

Example: Super Market Analysis

Presented by : Prof. S.B.Mehta

 Unsupervised Learning Algorithms

o K-means clustering
o Hierarchal clustering
o Anomaly detection
o Neural Networks
o Principle Component Analysis
o Independent Component Analysis
o Apriori algorithm
o Singular value decomposition

Presented by : Prof. S.B.Mehta

 Un Supervised Learning Algorithms
Advantages:
oUnsupervised learning is used for more complex tasks as compared to

supervised learning because, in unsupervised learning, we don't have labeled

input data.
oUnsupervised learning is preferable as it is easy to get unlabeled data in

comparison to labeled data.

Disadvantages:

oUnsupervised learning is intrinsically more difficult than supervised learning

as it does not have corresponding output.
oThe result of the unsupervised learning algorithm might be less accurate as

input data is not labeled, and algorithms do not know the exact output in
advance.

Presented by : Prof. S.B.Mehta

 Reinforcement learning Algorithms

Reinforcement learning is a type of learning that is based on interaction with the

environment. Reinforcement learning is a type of machine learning method
where an intelligent agent (computer program) interacts with the
environment and learns to act within that.

Presented by : Prof. S.B.Mehta

Example Reinforcement learning Algorithms

Presented by : Prof. S.B.Mehta

Presented by : Prof. S.B.Mehta
Difference between Supervised Learning, Unsupervised
and Reinforcement Learning Algorithms
Criteria Supervised Learning Unsupervised Reinforcement Learning
Learning

Definition The Machine learns The Machine is trained An agent interacts with its
by using labeled data on unlabeled data environment by
without aby guidance performing actions and
learning from errors or
rewards
Type of Regression and Association and Reward based
Problems Classification Clustering
Type of Data Labeled Data Unlabeled data No Predefined data
Training External supervision No Supervision No Supervision
Approach Maps the labeled to Understand patterns & Fallow the trial and error
inputs to the known discovers the output method
outputs
Application Fraud Detection, Text mining, Face Gaming, Inventory
Email Spam Recognition, Image management, Finance
Detection, Image Recognition, Big Data Sector, Robot navigation
Classification, Visualization
Diagnostics, Score
Prediction
Presented by : Prof. S.B.M ehta
 Hypothesis Space:
Hypothesis space is the set of all the possible legal hypothesis.
This is the set from which the machine learning algorithm would
determine the best possible (only one) which would best describe the
target function or the outputs.

A hypothesis in machine learning:

1. Covers the available evidence: the training dataset.

2.Is falsifiable (kind-of): a test harness is devised beforehand and
used to estimate performance and compare it to a baseline model to
see if is skillful or not.
3. Can be used in new situations: make predictions on new data.

Presented by : Prof. S.B.Mehta

 Inductive Bias:
The inductive bias (also known as learning bias) of a learning
algorithm is the set of assumptions that the learner uses to predict
outputs given inputs that it has not encountered. In machine learning,
one aims to construct algorithms that are able to learn to predict a
certain target output.

Presented by : Prof. S.B.Mehta

 Cross-validation:
Cross-validation is a technique in which we train our model using
the subset of the data-set and then evaluate using the complementary
subset of the data-set.

All cross-validation methods follow the same basic procedure:

(1) Divide the dataset into 2 parts: training and testing

(2) Train the model on the training set
(3) Evaluate the model on the testing set
(4) Optionally, repeat steps 1 to 3 for a different set of data points

Presented by : Prof. S.B.Mehta

 Cross-validation Technique:

1. Holdout
2. K-Fold Cross Validation
3. LOOCV (Leave One Out Cross Validation):

Presented by : Prof. S.B.Mehta

 Holdout Cross-validation Technique:
Holdout cross validation is the simplest and most common. We
simply split the data into two sets: train and test.

Presented by : Prof. S.B.Mehta

 K-Fold Cross-validation Technique:
In this method, we split the data-set into k number of
subsets(known as folds) then we perform training on the all the
subsets but leave one(k-1) subset for the evaluation of the trained
model. In this method, we iterate k times with a different subset
reserved for testing purpose each time.

Presented by : Prof. S.B.Mehta

 K-Fold Cross-validation Technique:
Example:
Total instances: 9
K=3
Test dataset Training dataset Estimate
Iteration: Error :
1 E1

Train dataset Train dataset

2 E2

3 E3

Training dataset Test dataset

Leave one out Cross-validation
Technique(LOOVE):
In this method, we perform training on the whole data-set but
leaves only one data-point of the available data-set and then
iterates for each data-point
 Example Leave one out Cross-validation
Technique(LOOVE):

Presented by : Prof. S.B.Mehta

 Linear Regression:
 It is a statistical method that is used for predictive analysis. Linear
regression makes predictions for continuous/real or numeric variables
such as sales, salary, age, product price, etc.
 Linear regression algorithm shows a linear relationship between a
dependent (y) and one or more independent (y) variables, hence called
as linear regression. Since linear regression shows the linear
relationship, which means it finds how the value of the dependent
variable is changing according to the value of the independent variable.

Presented by : Prof. S.B.Mehta

 Linear Regression Conti..:
 The linear regression model provides a sloped straight line representing the
relationship between the variables.
Consider the below image:

Presented by : Prof. S.B.Mehta

 Linear Regression Conti..:

Presented by : Prof. S.B.Mehta

 Linear Regression Conti..:
 The linear regression model provides a sloped straight line representing
the relationship between the variables.
Mathematically, we can represent a linear regression as:

Y= b0+b1X+ ε
Or Y = mX+b

Here,
Y= Dependent Variable (Target Variable or label to data i.e output data)
X= Independent Variable (predictor Variable or input Training Data)
b0= intercept of the line (Gives an additional degree of freedom)
b1 = Linear regression coefficient (scale factor to each input value).
ε = random error (For a good model it will be negligible)

Presented by : Prof. S.B.Mehta

 Type of Linear Regression :
1. Simple Linear Regression: When there is a single input variable (x),
the method is referred to as simple linear regression
2. Multiple Linear Regression : When there are multiple input
variables, literature from statistics often refers to the method as
multiple linear regression.

Presented by : Prof. S.B.Mehta

 Linear Regression Line : A linear line
showing the relationship between the dependent and
independent variables is called a regression line.
A regression line can show two types of relationship:
1. Positive Linear Relationship:

Presented by : Prof. S.B.Mehta

 Type of Linear Regression Line Conti…

2. Negative Positive Linear Relationship:

Presented by : Prof. S.B.Mehta

 Linear Regression Use Cases:

1. Evaluating Trends & Sales Estimate(Sales Forecasting)

2. Analyzing the impact of price changes
3. Housing Applications To Predict the prices and other factors
4. Finance Applications To Predict Stock prices, investment evaluation

Presented by : Prof. S.B.Mehta

 Decision Trees:
 It is a graphical representation for getting all the possible
solutions to a problem/decision based on given
conditions.
 The decisions or the test are performed on the basis of
features of the given dataset.
 In a Decision tree, there are two nodes, which are the
Decision Node and Leaf Node. Decision nodes are used
to make any decision and have multiple branches,
whereas Leaf nodes are the output of those decisions and
do not contain any further branches.

Presented by : Prof. S.B.Mehta

 How does the Decision Tree algorithm Work:

 Step-1: Begin the tree with the root node, says S, which contains the
complete dataset.
 Step-2: Find the best attribute in the dataset using Attribute

Selection Measure (ASM).

 Step-3: Divide the S into subsets that contains possible values for

the best attributes.

 Step-4: Generate the decision tree node, which contains the best

attribute.
 Step-5: Recursively make new decision trees using the subsets of

the dataset created in step

-3. Continue this process until a stage is reached where you cannot
further classify the nodes and called the final node as a leaf node.

Presented by : Prof. S.B.Mehta

 Example of Decision Tree algorithm :

Presented by : Prof. S.B.Mehta

 Type of Decision Tree Algorithms:

• Iterative Dichotomiser 3 (ID3): This algorithm uses Information Gain to

decide which attribute is to be used classify the current subset of the data. For
each level of the tree, information gain is calculated for the remaining data
recursively. uses Entropy function and Information gain as metrics

• C4.5: This algorithm is the successor of the ID3 algorithm. This algorithm uses
either Information gain or Gain ratio to decide upon the classifying attribute. It
is a direct improvement from the ID3 algorithm as it can handle both
continuous and missing attribute values.

• Classification and Regression Tree(CART): It is a dynamic learning

algorithm which can produce a regression tree as well as a classification tree
depending upon the dependent variable. Classification trees (Yes/No types)
Here the decision variable is Categorical. Regression trees (Continuous data
types) Here the decision or the outcome variable is Continuous, e.g. a number
like 123
Presented by : Prof. S.B.Mehta
Entropy:
Entropy, also called as Shannon Entropy is denoted by H(S) for a
finite set S, is the measure of the amount of uncertainty or
randomness in data. In machine learning sense and especially in this
case Entropy is the measure of homogeneity in the data.

Here pi represents the proportion of the data with i th

classification and c represents the different types of
classification

Presented by : Prof. S.B.Mehta

Information Gain
Information gain is also called as Kullback-Leibler divergence denoted by
IG(S,A) for a set S is the effective change in entropy after deciding on a
particular attribute A. Information gain is the measurement of changes in entropy
after the segmentation of a dataset based on an attribute.

The formula to calculate Gain by splitting the data on Dataset

‘S’and on the attribute ‘A’is :

Alternatively,

Here Entropy(S) represents the entropy of the dataset and the second
term on the right is the weighted entropy of the different possible
classes obtain after the split.
Presented by : Prof. S.B.Mehta
 Iterative Dichotomiser 3 (ID3) Algorithm:

 Create root node for the tree

 If all examples are positive, return leaf node ‘positive’
 Else if all examples are negative, return leaf node ‘negative’
 Calculate the entropy of current state H(S)
 For each attribute, calculate the entropy with respect to the attribute ‘x’
denoted by H(S, x)
 Select the attribute which has maximum value of IG(S, x)
 Remove the attribute that offers highest IG from the set of attributes
 Repeat until we run out of all attributes, or the decision tree has all leaf nodes.

Presented by : Prof. S.B.Mehta

 Iterative Dichotomiser 3 (ID3) Algorithm:

 Create root node for the tree

 If all examples are positive, return leaf node ‘positive’
 Else if all examples are negative, return leaf node ‘negative’
 Calculate the entropy of current state H(S)
 For each attribute, calculate the entropy with respect to the attribute ‘x’
denoted by H(S, x)
 Select the attribute which has maximum value of IG(S, x)
 Remove the attribute that offers highest IG from the set of attributes
 Repeat until we run out of all attributes, or the decision tree has all leaf nodes.

Presented by : Prof. S.B.Mehta

 Advantages and Disadvantages of Decision Tree
Advantages:
 It is simple to understand as it follows the same process which a

human follow while making any decision in real-life.

 It can be very useful for solving decision-related problems.

 It helps to think about all the possible outcomes for a problem.

 There is less requirement of data cleaning compared to other

algorithms.

Disadvantages:

 The decision tree contains lots of layers, which makes it complex.

 It may have an overfitting issue, which can be resolved using the
Random Forest algorithm.
 For more class labels, the computational complexity of the decision
tree may increase.

Presented by : Prof. S.B.Mehta

 Overfitting:
Overfitting refers to a model that models the training data too well. Overfitting
happens when a model learns the detail and noise in the training data to the extent
that it negatively impacts the performance of the model on new data.

To Avoid Overfitting
1. Cross-Validation
2. Training With More Data
3. Removing Features
4. Early Stopping
5. Regularization
6. Ensembling

Presented by : Prof. S.B.Mehta