Decision Tree Learning
•Download as PPTX, PDF•
7 likes•2,713 views
Basic of Decision Tree Learning. This slide includes definition of decision tree, basic example, basic construction of a decision tree, mathlab example
Report
Share
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Decision Tree Learning
Similar to Decision Tree Learning (20)
More from Md. Ariful Hoque
Recently uploaded
Recently uploaded (20)
Decision Tree Learning
- 1. Md. Ariful Hoque Roll: MSC 140202 Khulna University
- 2. Introduction. Decision Tree Terms. Example. Constructing A decision Tree. Calculation Of Entropy. Information Gain. Gini Impurity. Termination Criteria. Mathlab Example. Implementations. Advantage Limitation Conclusion
- 3. Decision tree learning is the construction of a decision tree from class- labeled training tuples. A decision tree is model of decisions and their possible consequences. It Includes chance event outcomes, resource costs, and utility. Its follow top down approach. Decision trees classify instances by sorting them down the tree from the root to some leaf node, which provides the classification of the instance
- 4. Root Node Condition Check Leaf Node(Decision Point) Leaf Node(Decision Point) Condition Check
- 5. Yes Color = Green ? Size = Big ? Color = Yellow ? Size =medium?Watermelon Apple Grape Shape = Round? Size = Small ? Taste = sweet? Apple Yes No Yes No Cherry Grape Size = Big? Grape Lemon Banana Yes YesYes Yes Yes No No No No No
- 6. There are many specific decision-tree algorithms- ID3 C4.5 CART CHAID MARS
- 7. Which attribute to choose? ◦ Information Gain ENTROPY Where to stop? ◦ Termination criteria
- 8. Different algorithms use different metrics for measuring best. These generally measure the homogeneity of the target variable within the subsets. Some examples are given in the next slides.
- 9. ◦ Entropy is a measure of uncertainty in the data Entropy(S) = ∑(i=1 to l)-|Si|/|S| * log2(|Si|/|S|) S = set of examples Si = subset of S with value vi under the target attribute l = size of the range of the target attribute
- 10. Used by the ID3, C4.5 and C5.0 tree-generation algorithms. Information gain is based on the concept of entropy from information theory. Here , fi = fraction of items m = Items
- 11. Used by the CART (classification and regression tree). It measure incorrectly labeled of randomly chosen element. Gini impurity can be computed by summing the probability of each item being chosen times the probability of a mistake in categorizing that item. It reaches its minimum (zero) when all cases in the node fall into a single target category. To compute Gini impurity for a set of items, suppose- Let f be the fraction of items labeled with value i in the set.
- 12. All the records at the node belong to one class A significant majority fraction of records belong to a single class The segment contains only one or very small number of records The improvement is not substantial enough to warrant making the split.
- 13. Create a classification decision tree for Fisher's iris data: load fisheriris; t = classregtree(meas,species,... 'names',{'SL' 'SW' 'PL' 'PW'}) view(t)
- 14. t = Decision tree for classification if PL<2.45 then node 2 elseif PL>=2.45 then node 3 else setosa class = setosa if PW<1.75 then node 4 elseif PW>=1.75 then node 5 else versicolor if PL<4.95 then node 6 elseif PL>=4.95 then node 7 else versicolor class = virginica if PW<1.65 then node 8 elseif PW>=1.65 then node 9 else versicolor class = virginica class = versicolor class = virginica
- 16. In data mining software. Several examples include Salford Systems CART, IBM SPSS , KNIME, Microsoft SQL Server, and scikit-learn.
- 17. Decision-tree learners can create over-complex trees. There are concepts that are hard to learn because decision trees do not express them easily, such as XOR, parity or multiplexer problems. When there are more records and very less number of attributes/features.
- 18. Simple to understand and interpret. Requires little data preparation. Able to handle both numerical and categorical data. Performs well with large datasets.
- 19. Decision tree learning is one of the predictive modeling approaches used in statistics, data mining and machine learning. In our example section we saw a classification tree. Where the target variable can take a finite set of values. In Mathlab example section we saw regression trees. Where the target variable can take continuous values (typically real numbers).
- 20. 1. Decision tree learning[Online]. Available:http://en.wikipedia.org/wiki/Decision_tree_learning 2. Classregtree[Online]. Available:http://www.mathworks.com/help/stats/classregtree.html 3. Richard O.Duda, Peter E. Hart, David G. Stok. Pattern Classification. Second Edition 4. Breiman, L., J. Friedman, R. Olshen, and C. Stone. Classification and Regression Trees. Boca Raton, FL: CRC Press, 1984.
- 21. Thanks