Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
17 views

ml_cheatsheet

Uploaded by

Lenara
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
17 views

ml_cheatsheet

Uploaded by

Lenara
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Machine Learning Algorithms Cheat Sheet

Table of Contents

1. Supervised Learning
Linear Regression
Logistic Regression
Decision Trees
Random Forests
Support Vector Machines (SVM)
k-Nearest Neighbors (k-NN)
Naive Bayes
Gradient Boosting Machines (GBM)
Neural Networks
2. Unsupervised Learning
k-Means Clustering
Hierarchical Clustering
Principal Component Analysis (PCA)
Independent Component Analysis (ICA)
Association Rules
Autoencoders
3. Reinforcement Learning
Q-Learning
Deep Q-Networks (DQN)
Policy Gradients
Actor-Critic Methods
4. Semi-Supervised and Self-Supervised Learning
Self-Training
Co-Training
5. Ensemble Methods
Bagging
Boosting
Stacking

Supervised Learning

1. Linear Regression

Purpose: Predict continuous target variables.


Key Concept: Models the relationship between input features and output as a linear combination.
Equation: ( y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \dots + \beta_nx_n + \epsilon )

2. Logistic Regression

Purpose: Binary classification.


Key Concept: Uses the logistic function to model the probability of a class.
Equation: ( P(Y=1) = \frac{1}{1 + e^{-(\beta_0 + \beta_1x_1 + \dots + \beta_nx_n)}} )

3. Decision Trees

Purpose: Classification and regression.


Key Concept: Splits data into subsets based on feature values.
Advantages: Easy to interpret, handles both numerical and categorical data.
Disadvantages: Prone to overfitting.

4. Random Forests

Purpose: Classification and regression.


Key Concept: Ensemble of decision trees using bagging.
Advantages: Reduces overfitting, handles large datasets well.
Key Parameters: Number of trees, max depth.

5. Support Vector Machines (SVM)

Purpose: Classification and regression.


Key Concept: Finds the hyperplane that best separates classes with maximum margin.
Kernel Trick: Enables handling non-linear relationships.
Common Kernels: Linear, Polynomial, RBF.

6. k-Nearest Neighbors (k-NN)

Purpose: Classification and regression.


Key Concept: Assigns the output based on the majority label of the k closest training examples.
Advantages: Simple, no training phase.
Disadvantages: Computationally intensive during prediction, sensitive to irrelevant features.

7. Naive Bayes

Purpose: Classification.
Key Concept: Based on Bayes' Theorem with the assumption of feature independence.
Variants: Gaussian, Multinomial, Bernoulli.

8. Gradient Boosting Machines (GBM)

Purpose: Classification and regression.


Key Concept: Builds models sequentially, each new model correcting errors of the previous ones.
Popular Implementations: XGBoost, LightGBM, CatBoost.
Advantages: High predictive performance, handles missing data.

9. Neural Networks

Purpose: Various tasks including classification, regression, and more.


Key Concept: Composed of layers of interconnected nodes (neurons) that can capture complex patterns.
Types: Feedforward Neural Networks, Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN).

Unsupervised Learning

1. k-Means Clustering

Purpose: Partition data into k distinct clusters.


Key Concept: Minimizes within-cluster variance.
Parameters: Number of clusters (k), distance metric.

2. Hierarchical Clustering

Purpose: Create a hierarchy of clusters.


Key Concept: Either agglomerative (bottom-up) or divisive (top-down).
Linkage Criteria: Single, complete, average, ward.

3. Principal Component Analysis (PCA)

Purpose: Dimensionality reduction.


Key Concept: Transforms data to a new coordinate system with orthogonal principal components.
Uses: Feature reduction, visualization.

4. Independent Component Analysis (ICA)

Purpose: Separate a multivariate signal into additive, independent components.


Key Concept: Maximizes statistical independence.

5. Association Rules

Purpose: Discover interesting relations between variables in large databases.


Key Concepts: Support, Confidence, Lift.
Algorithms: Apriori, Eclat.

6. Autoencoders

Purpose: Learn efficient codings of input data.


Key Concept: Neural network architecture with encoder and decoder parts.
Uses: Dimensionality reduction, anomaly detection.

Reinforcement Learning

1. Q-Learning

Purpose: Learn the value of actions in states to derive an optimal policy.


Key Concept: Off-policy temporal difference learning.
Equation: ( Q(s, a) \leftarrow Q(s, a) + \alpha [r + \gamma \max_{a'} Q(s', a') - Q(s, a)] )

2. Deep Q-Networks (DQN)

Purpose: Combine Q-Learning with deep neural networks.


Key Concept: Uses neural networks to approximate Q-values.
Features: Experience replay, target networks.

3. Policy Gradients

Purpose: Optimize the policy directly.


Key Concept: Uses gradient ascent on expected rewards.
Algorithms: REINFORCE, Proximal Policy Optimization (PPO).

4. Actor-Critic Methods

Purpose: Combine value-based and policy-based methods.


Key Concept: Actor updates policy, Critic evaluates it.
Examples: A3C, DDPG.

Semi-Supervised and Self-Supervised Learning

1. Self-Training

Purpose: Utilize unlabeled data to improve model performance.


Key Concept: Iteratively label unlabeled data using the current model.

2. Co-Training

Purpose: Use multiple views of data to train models.


Key Concept: Each model trains on different feature sets and labels unlabeled data for each other.

Ensemble Methods

1. Bagging (Bootstrap Aggregating)

Purpose: Reduce variance and prevent overfitting.


Key Concept: Train multiple models on different bootstrap samples and aggregate predictions.
Example: Random Forest.

2. Boosting

Purpose: Reduce bias and build strong predictive models.


Key Concept: Sequentially train models, each focusing on errors of the previous ones.
Examples: AdaBoost, Gradient Boosting, XGBoost.

3. Stacking

Purpose: Combine multiple models to improve performance.


Key Concept: Use a meta-model to aggregate predictions from base models.

Additional Algorithms and Techniques

Support Vector Regression (SVR)

Purpose: Regression using SVM principles.


Key Concept: Fits the best line within a predefined margin.

Elastic Net

Purpose: Regularized regression combining L1 and L2 penalties.


Key Concept: Balances between feature selection and coefficient shrinkage.

Gaussian Mixture Models (GMM)

Purpose: Probabilistic clustering.


Key Concept: Assumes data is generated from a mixture of several Gaussian distributions.

t-Distributed Stochastic Neighbor Embedding (t-SNE)


Purpose: Data visualization.
Key Concept: Reduces dimensions while preserving local structure.

Hidden Markov Models (HMM)

Purpose: Model sequential data.


Key Concept: States are hidden and emit observable events.

Key Concepts and Terms

Overfitting: Model performs well on training data but poorly on unseen data.
Underfitting: Model is too simple to capture underlying patterns.
Bias-Variance Tradeoff: Balance between model complexity and generalization.
Cross-Validation: Technique to assess model performance by partitioning data.
Regularization: Techniques to prevent overfitting (e.g., L1, L2).
Feature Scaling: Standardizing features to improve model performance.

Resources and Libraries

Python Libraries:

Scikit-learn: Comprehensive ML algorithms.


TensorFlow & Keras: Deep learning frameworks.
PyTorch: Flexible deep learning library.
XGBoost, LightGBM, CatBoost: Gradient boosting implementations.

Books:

"Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow" by Aurélien Géron
"Pattern Recognition and Machine Learning" by Christopher M. Bishop
"The Elements of Statistical Learning" by Trevor Hastie, Robert Tibshirani, and Jerome Friedman

Online Courses:

Coursera's Machine Learning by Andrew Ng


edX's MicroMasters in Statistics and Data Science
Udacity's Machine Learning Nanodegree

You might also like