| |
| |
| List of Figures | |
| |
| |
| List of Tables | |
| |
| |
| Preface | |
| |
| |
| Acknowledgments | |
| |
| |
| |
| Introduction | |
| |
| |
| |
| Machine Learning Roots | |
| |
| |
| |
| Generative Learning | |
| |
| |
| |
| Generative Models in AI | |
| |
| |
| |
| Generative Models in Perception | |
| |
| |
| |
| Generative Models in Tracking and Dynamics | |
| |
| |
| |
| Why a Probability of Everything? | |
| |
| |
| |
| Discriminative Learning | |
| |
| |
| |
| Objective | |
| |
| |
| |
| Scope and Organization | |
| |
| |
| |
| Online Support | |
| |
| |
| |
| Generative Versus Discriminative Learning | |
| |
| |
| |
| Two Schools of Thought | |
| |
| |
| |
| Generative Probabilistic Models | |
| |
| |
| |
| Discriminative Classifiers and Regressors | |
| |
| |
| |
| Generative Learning | |
| |
| |
| |
| Bayesian Inference | |
| |
| |
| |
| Maximum Likelihood | |
| |
| |
| |
| The Exponential Family | |
| |
| |
| |
| Maximum Entropy | |
| |
| |
| |
| Expectation Maximization and Mixtures | |
| |
| |
| |
| Graphical Models | |
| |
| |
| |
| Conditional Learning | |
| |
| |
| |
| Conditional Bayesian Inference | |
| |
| |
| |
| Maximum Conditional Likelihood | |
| |
| |
| |
| Logistic Regression | |
| |
| |
| |
| Discriminative Learning | |
| |
| |
| |
| Empirical Risk Minimization | |
| |
| |
| |
| Structural Risk Minimization | |
| |
| |
| |
| VC Dimension and Large Margins | |
| |
| |
| |
| Support Vector Machines | |
| |
| |
| |
| Kernel Methods | |
| |
| |
| |
| Averaged Classifiers | |
| |
| |
| |
| Joint Generative-Discriminative Learning | |
| |
| |
| |
| Maximum Entropy Discrimination | |
| |
| |
| |
| Regularization Theory and Support Vector Machines | |
| |
| |
| |
| Solvability | |
| |
| |
| |
| Support Vector Machines and Kernels | |
| |
| |
| |
| A Distribution over Solutions | |
| |
| |
| |
| Augmented Distributions | |
| |
| |
| |
| Information and Geometry Interpretations | |
| |
| |
| |
| Computing the Partition Function | |
| |
| |
| |
| Margin Priors | |
| |
| |
| |
| Bias Priors | |
| |
| |
| |
| Gaussian Bias Priors | |
| |
| |
| |
| Non-Informative Bias Priors | |
| |
| |
| |
| Support Vector Machines | |
| |
| |
| |
| Single Axis SVM Optimization | |
| |
| |
| |
| Kernels | |
| |
| |
| |
| Generative Models | |
| |
| |
| |
| Exponential Family Models | |
| |
| |
| |
| Empirical Bayes Priors | |
| |
| |
| |
| Full Covariance Gaussians | |
| |
| |
| |
| Multinomials | |
| |
| |
| |
| Generalization Guarantees | |
| |
| |
| |
| VC Dimension | |
| |
| |
| |
| Sparsity | |
| |
| |
| |
| PAC-Bayes Bounds | |
| |
| |
| |
| Extensions to Med | |
| |
| |
| |
| Multiclass Classification | |
| |
| |
| |
| Regression | |
| |
| |
| |
| SVM Regression | |
| |
| |
| |
| Generative Model Regression | |
| |
| |
| |
| Feature Selection and Structure Learning | |
| |
| |
| |
| Feature Selection in Classification | |
| |
| |
| |
| Feature Selection in Regression | |
| |
| |
| |
| Feature Selection in Generative Models | |
| |
| |
| |
| Kernel Selection | |
| |
| |
| |
| Meta-Learning | |
| |
| |
| |
| Transduction | |
| |
| |
| |
| Transductive Classification | |
| |
| |
| |
| Transductive Regression | |
| |
| |
| |
| Other Extensions | |
| |
| |
| |
| Latent Discrimination | |
| |
| |
| |
| Mixture Models and Latent Variables | |
| |
| |
| |
| Iterative MED Projection | |
| |
| |
| |
| Bounding the Latent MED Constraints | |
| |
| |
| |
| Latent Decision Rules | |
| |
| |
| |
| Large Margin Mixtures of Gaussians | |
| |
| |
| |
| Parameter Distribution Update | |
| |
| |
| |
| Just a Support Vector Machine | |
| |
| |
| |
| Latent Distributions Update | |
| |
| |
| |
| Extension to Kernels | |
| |
| |
| |
| Extension to Non Gaussian Mixtures | |
| |
| |
| |
| Efficiency | |
| |
| |
| |
| Efficient Mixtures of Gaussians | |
| |
| |
| |
| Structured Latent Models | |
| |
| |
| |
| Factorization of Lagrange Multipliers | |
| |
| |
| |
| Mean Field for Intractable Models | |
| |
| |
| |
| Conclusion | |
| |
| |
| |
| A Generative and Discriminative Hybrid | |
| |
| |
| |
| Designing Models versus Designing Kernels | |
| |
| |
| |
| What's Next? | |
| |
| |
| |
| Appendix | |
| |
| |
| |
| Optimization in the MED Framework | |
| |
| |
| |
| Constrained Gradient Ascent | |
| |
| |
| |
| Axis-Parallel Optimization | |
| |
| |
| |
| Learning Axis Transitions | |
| |
| |
| Index | |