Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

An Overview of Machine Learning

•
•
T
Tanvir Moin

Machine learning is important because it gives enterprises a view of trends in customer behaviour and business operational patterns, as well as supports the development of new products. Many of today's leading companies, such as Facebook, Google and Uber, make machine learning a central part of their operations.

1 of 23
An Overview of Machine Learning TANVIR SIDDIKE MOIN UNIVERSITY OF DHAKA
Outline & Content What is machine learning? Learning system model Training and testing Performance Algorithms Machine learning structure What are we seeking? Learning techniques Applications Conclusion
What is machine learning? A branch of artificial intelligence, concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data. As intelligence requires knowledge, it is necessary for the computers to acquire knowledge.
Learning system model Input Samples Learning Method System Training Testing
Training and testing Training set (observed) Universal set (unobserved) Testing set (unobserved) Data acquisition Practical usage
Training is the process of making the system able to learn. No free lunch rule: ◦ Training set and testing set come from the same distribution ◦ Need to make some assumptions or bias Training and testing
Performance There are several factors affecting the performance: • Types of training provided • The form and extent of any initial background knowledge • The type of feedback provided • The learning algorithms used Two important factors: • Modeling • Optimization
Algorithms The success of machine learning system also depends on the algorithms. The algorithms control the search to find and build the knowledge structures. The learning algorithms should extract useful information from training examples.
Supervised learning ( ) Prediction Classification (discrete labels), Regression (real values) Unsupervised learning ( ) Clustering Probability distribution estimation Finding association (in features) Dimension reduction Semi-supervised learning Reinforcement learning Decision making (robot, chess machine) Algorithms
10 Algorithms Supervised learning Unsupervised learning Semi-supervised learning
Machine learning structure S U P E RV I S E D L EA R N I N G
Machine learning structure U N S U P E RV I S E D L EA R N I N G
Supervised: Low E-out or maximize probabilistic terms Unsupervised: Minimum quantization error, Minimum distance, MAP, MLE(maximum likelihood estimation) What are we seeking? E-in: for training set E-out: for testing set
Under-fitting VS. Over-fitting (fixed N) What are we seeking? error (model = hypothesis + loss functions)
Supervised learning categories and techniques ◦ Linear classifier (numerical functions) ◦ Parametric (Probabilistic functions) ◦ Naïve Bayes, Gaussian discriminant analysis (GDA), Hidden Markov models (HMM), Probabilistic graphical models ◦ Non-parametric (Instance-based functions) ◦ K-nearest neighbors, Kernel regression, Kernel density estimation, Local regression ◦ Non-metric (Symbolic functions) ◦ Classification and regression tree (CART), decision tree ◦ Aggregation ◦ Bagging (bootstrap + aggregation), Adaboost, Random forest Learning techniques
Techniques: ◦ Perceptron ◦ Logistic regression ◦ Support vector machine (SVM) ◦ Ada-line ◦ Multi-layer perceptron (MLP) Learning techniques , where w is a d-dim vector (learned) • Linear classifier
Learning techniques Using perceptron learning algorithm(PLA) Training Testing Error rate: 0.10 Error rate: 0.156
Learning techniques Using logistic regression Training Testing Error rate: 0.11 Error rate: 0.145
Support vector machine (SVM): ◦ Linear to nonlinear: Feature transform and kernel function Learning techniques • Non-linear case
Learning techniques Unsupervised learning categories and techniques ◦ Clustering ◦ K-means clustering ◦ Spectral clustering ◦ Density Estimation ◦ Gaussian mixture model (GMM) ◦ Graphical models ◦ Dimensionality reduction ◦ Principal component analysis (PCA) ◦ Factor analysis
Applications Face detection Object detection and recognition Image segmentation Multimedia event detection Economical and commercial usage
Conclusion We have a simple overview of some techniques and algorithms in machine learning. Furthermore, there are more and more techniques apply machine learning as a solution. In the future, machine learning will play an important role in our daily life.
Reference [1] W. L. Chao, J. J. Ding, “Integrated Machine Learning Algorithms for Human Age Estimation”, NTU, 2011.

More Related Content

An Overview of Machine Learning

  • 1. An Overview of Machine Learning TANVIR SIDDIKE MOIN UNIVERSITY OF DHAKA
  • 2. Outline & Content What is machine learning? Learning system model Training and testing Performance Algorithms Machine learning structure What are we seeking? Learning techniques Applications Conclusion
  • 3. What is machine learning? A branch of artificial intelligence, concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data. As intelligence requires knowledge, it is necessary for the computers to acquire knowledge.
  • 5. Training and testing Training set (observed) Universal set (unobserved) Testing set (unobserved) Data acquisition Practical usage
  • 6. Training is the process of making the system able to learn. No free lunch rule: ◦ Training set and testing set come from the same distribution ◦ Need to make some assumptions or bias Training and testing
  • 7. Performance There are several factors affecting the performance: • Types of training provided • The form and extent of any initial background knowledge • The type of feedback provided • The learning algorithms used Two important factors: • Modeling • Optimization
  • 8. Algorithms The success of machine learning system also depends on the algorithms. The algorithms control the search to find and build the knowledge structures. The learning algorithms should extract useful information from training examples.
  • 9. Supervised learning ( ) Prediction Classification (discrete labels), Regression (real values) Unsupervised learning ( ) Clustering Probability distribution estimation Finding association (in features) Dimension reduction Semi-supervised learning Reinforcement learning Decision making (robot, chess machine) Algorithms
  • 10. 10 Algorithms Supervised learning Unsupervised learning Semi-supervised learning
  • 11. Machine learning structure S U P E RV I S E D L EA R N I N G
  • 12. Machine learning structure U N S U P E RV I S E D L EA R N I N G
  • 13. Supervised: Low E-out or maximize probabilistic terms Unsupervised: Minimum quantization error, Minimum distance, MAP, MLE(maximum likelihood estimation) What are we seeking? E-in: for training set E-out: for testing set
  • 14. Under-fitting VS. Over-fitting (fixed N) What are we seeking? error (model = hypothesis + loss functions)
  • 15. Supervised learning categories and techniques ◦ Linear classifier (numerical functions) ◦ Parametric (Probabilistic functions) ◦ Naïve Bayes, Gaussian discriminant analysis (GDA), Hidden Markov models (HMM), Probabilistic graphical models ◦ Non-parametric (Instance-based functions) ◦ K-nearest neighbors, Kernel regression, Kernel density estimation, Local regression ◦ Non-metric (Symbolic functions) ◦ Classification and regression tree (CART), decision tree ◦ Aggregation ◦ Bagging (bootstrap + aggregation), Adaboost, Random forest Learning techniques
  • 16. Techniques: ◦ Perceptron ◦ Logistic regression ◦ Support vector machine (SVM) ◦ Ada-line ◦ Multi-layer perceptron (MLP) Learning techniques , where w is a d-dim vector (learned) • Linear classifier
  • 17. Learning techniques Using perceptron learning algorithm(PLA) Training Testing Error rate: 0.10 Error rate: 0.156
  • 18. Learning techniques Using logistic regression Training Testing Error rate: 0.11 Error rate: 0.145
  • 19. Support vector machine (SVM): ◦ Linear to nonlinear: Feature transform and kernel function Learning techniques • Non-linear case
  • 20. Learning techniques Unsupervised learning categories and techniques ◦ Clustering ◦ K-means clustering ◦ Spectral clustering ◦ Density Estimation ◦ Gaussian mixture model (GMM) ◦ Graphical models ◦ Dimensionality reduction ◦ Principal component analysis (PCA) ◦ Factor analysis
  • 21. Applications Face detection Object detection and recognition Image segmentation Multimedia event detection Economical and commercial usage
  • 22. Conclusion We have a simple overview of some techniques and algorithms in machine learning. Furthermore, there are more and more techniques apply machine learning as a solution. In the future, machine learning will play an important role in our daily life.
  • 23. Reference [1] W. L. Chao, J. J. Ding, “Integrated Machine Learning Algorithms for Human Age Estimation”, NTU, 2011.