Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
23 views

Decision Tree - ML Class

The document discusses decision tree learning methods including ID3, CART, and C4.5 algorithms. It describes the basic steps and characteristics of each algorithm including calculating information gain, Gini index, and gain ratio. Numerical examples are provided for each algorithm.

Uploaded by

Pratyansh Vaibhav
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
23 views

Decision Tree - ML Class

The document discusses decision tree learning methods including ID3, CART, and C4.5 algorithms. It describes the basic steps and characteristics of each algorithm including calculating information gain, Gini index, and gain ratio. Numerical examples are provided for each algorithm.

Uploaded by

Pratyansh Vaibhav
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 58

Decision Tree Learning

Decision Tree Learning


• The decision tree learning is a method for approximating discrete valued target function in
which the learned function is represented by a decision tree.
• In machine learning, decision tree is a supervised learning method.
• It is used for regression and classification.

Performance matrices for Decision Tree

• Entropy E(x): The "average amount of information" contained by a random variable (x) is
called Entropy. It is denoted by (E) or (H).
• In other words, entropy is the "measure of randomness of information" of a variable.
Entropy (E) is the measure of impurity or uncertainty associated with a random variable
(X).
In the above fig, the entropy H(x) is zero when probability Pr(x) is 0 or 1. The entropy is
maximum (i.e ., 1) when probability is 0.5, because at this point randomness or impurity in
data is very high.
Basic Characteristic of Decision Tree Algorithms
ID3 Algorithm

ID3 Steps
1. Calculate the Information Gain of each feature.
2. Considering that all rows don’t belong to the same class, split the dataset S into subsets
using the feature for which the Information Gain is maximum.
3. Make a decision tree node using the feature with the maximum Information gain.
4. If all rows belong to the same class, make the current node as a leaf node with the class as
its label.
5. Repeat for the remaining features until we run out of all features, or the decision tree has
all leaf nodes.
Here, nine “Yes”, Five “No” i.e. 9 (+ve) and 5(-ve) example in this Table 1
CART Algorithm

Gini index

• Gini Index, also known as Gini impurity, measures the probability of a specific feature that
is classified incorrectly when selected randomly.

• The value of 0.5 of the Gini Index shows an equal distribution of elements over some
classes.

• It is a metric for classification tasks in CART. It stores sum of squared probabilities of


each class. We can formulate it as illustrated below.

Gini = 1 – Σ (Pi)2 for i=1 to number of classes


C 4.5 Algorithm

Gini ratio

Gain Ratio(A) = Gain(A) / SplitInfo(A)

SplitInfo(A) = -∑ |Dj|/|D| x log2|Dj|/|D|


Solved Numerical URLs:

• For ID3: https://sefiks.com/2017/11/20/a-step-by-step-id3-decision-tree-example/


• For CART: https://sefiks.com/2018/08/27/a-step-by-step-cart-decision-tree-example/
• For c4.5 : https://sefiks.com/2018/05/13/a-step-by-step-c4-5-decision-tree-example/

You might also like