Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
27 views

EM Algorithm

The document discusses the EM algorithm, which is an iterative method used to find maximum likelihood estimates of parameters in probabilistic models with latent variables. It consists of expectation and maximization steps, where the expectation step estimates the latent variables given current parameters, and the maximization step updates the parameters given current estimates of the latent variables. The process repeats until convergence of the parameter values is reached.

Uploaded by

bhavana vashishtha
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
27 views

EM Algorithm

The document discusses the EM algorithm, which is an iterative method used to find maximum likelihood estimates of parameters in probabilistic models with latent variables. It consists of expectation and maximization steps, where the expectation step estimates the latent variables given current parameters, and the maximization step updates the parameters given current estimates of the latent variables. The process repeats until convergence of the parameter values is reached.

Uploaded by

bhavana vashishtha
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 5

EM Algorithm (Expectation and Maximization)

What is an EM algorithm?
(It is a basic for for many unsupervised clustering algorithms)
It is used to find the latent variables)
The Expectation-Maximization (EM) algorithm is defined as the combination of various unsupervised
machine learning algorithms, which is used to determine the local maximum likelihood estimates
(MLE) or maximum a posteriori estimates (MAP) for unobservable variables in statistical models.
Further, it is a technique to find maximum likelihood estimation when the latent variables are present. It is
also referred to as the latent variable model.

A latent variable model consists of both observable and unobservable variables where observable can be
predicted while unobserved are inferred from the observed variable. These unobservable variables are
known as latent variables.

Key Points:

o It is known as the latent variable model to determine MLE and MAP parameters for latent variables.
o It is used to predict values of parameters in instances where data is missing or unobservable for
learning, and this is done until convergence of the values occurs.
EM Algorithm

The EM algorithm is the combination of various unsupervised ML algorithms, such as the k-means
clustering algorithm. Being an iterative approach, it consists of two modes. In the first mode, we estimate
the missing or latent variables. Hence it is referred to as the Expectation/estimation step (E-step). Further,
the other mode is used to optimize the parameters of the models so that it can explain the data more clearly.
The second mode is known as the maximization-step or M-step.

o Expectation step (E - step): It involves the estimation (guess) of all missing values in the dataset so
that after completing this step, there should not be any missing value.
o Maximization step (M - step): This step involves the use of estimated data in the E-step and updating
the parameters.
o Repeat E-step and M-step until the convergence of the values occurs.

The primary goal of the EM algorithm is to use the available observed data of the dataset to estimate the
missing data of the latent variables and then use that data to update the values of the parameters in the M-
step.

What is Convergence in the EM algorithm?

Convergence is defined as the specific situation in probability based on intuition, e.g., if there are two
random variables that have very less difference in their probability, then they are known as converged. In
other words, whenever the values of given variables are matched with each other, it is called convergence.

Steps in EM Algorithm

The EM algorithm is completed mainly in 4 steps, which include Initialization Step, Expectation Step,
Maximization Step, and convergence Step. These steps are explained as follows:
o 1st Step: The very first step is to initialize the parameter values. Further, the system is provided with
incomplete observed data with the assumption that data is obtained from a specific model.

o 2nd Step: This step is known as Expectation or E-Step, which is used to estimate or guess the values of
the missing or incomplete data using the observed data. Further, E-step primarily updates the variables.
o 3rd Step: This step is known as Maximization or M-step, where we use complete data obtained from
the 2nd step to update the parameter values. Further, M-step primarily updates the hypothesis.
o 4th step: The last step is to check if the values of latent variables are converging or not. If it gets "yes",
then stop the process; else, repeat the process from step 2 until the convergence occurs.

You might also like