Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
101 views

Neural Network Unsupervised Machine Learning: What Are Autoencoders?

An autoencoder is an unsupervised machine learning algorithm that applies backpropagation to learn compressed representations of input data. It consists of an encoder that compresses the input into a latent space representation and a decoder that reconstructs the input from this encoding. Autoencoders are useful for tasks like dimensionality reduction, denoising images, and generating representations of data for other models. There are different types of autoencoders like convolutional and denoising autoencoders that are tailored for specific applications and data types.

Uploaded by

shubh agrawal
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
101 views

Neural Network Unsupervised Machine Learning: What Are Autoencoders?

An autoencoder is an unsupervised machine learning algorithm that applies backpropagation to learn compressed representations of input data. It consists of an encoder that compresses the input into a latent space representation and a decoder that reconstructs the input from this encoding. Autoencoders are useful for tasks like dimensionality reduction, denoising images, and generating representations of data for other models. There are different types of autoencoders like convolutional and denoising autoencoders that are tailored for specific applications and data types.

Uploaded by

shubh agrawal
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 22

What are Autoencoders?

An autoencoder neural network is an Unsupervised Machine learning algorithm that


applies backpropagation, setting the target values to be equal to the inputs. Autoencoders
are used to reduce the size of our inputs into a smaller representation. If anyone needs the
original data, they can reconstruct it from the compressed data.
•An autoencoder can learn non-
linear transformations with a non-linear
activation function and multiple layers.
•It doesn’t have to learn dense layers. It can
use convolutional layers to learn which is
better for video, image and series data.
•It is more efficient to learn several layers with
an autoencoder rather than learn one huge
transformation with PCA.
•An autoencoder provides a representation of
each layer as the output.
•It can make use of pre-trained layers from
another model to apply transfer learning to
enhance the encoder/decoder.
Now let’s have a look at a few Industrial
Applications of Autoencoders.
Applications of Autoencoders

• Image Coloring
• Autoencoders are used for converting any black and white picture
into a colored image. Depending on what is in the picture, it is
possible to tell what the color should be
Applications of Autoencoders

• Feature variation
• It extracts only the required features of an image and generates the
output by removing any noise or unnecessary interruption.
Applications of Autoencoders
• Dimensionality Reduction
• The reconstructed image is the same as our input but with reduced
dimensions. It helps in providing the similar image with a reduced
pixel value.
Applications of Autoencoders
• Denoising Image
• The input seen by the autoencoder
is not the raw input but a
stochastically corrupted version.
A denoising autoencoder is thus
trained to reconstruct the original input from the noisy version.
Applications of Autoencoders

• Watermark Removal
• It is also used for removing watermarks from images or to remove any
object while filming a video or a movie.
• Now that you have an idea of the different industrial applications of
Autoencoders, let’s continue our Autoencoders Tutorial Blog and
understand the complex architecture of Autoencoders.
Architecture of Autoencoders
Architecture of Autoencoders
• Encoder: This part of the network compresses the input into a latent
space representation. The encoder layer encodes the input image as
a compressed representation in a reduced dimension. The
compressed image is the distorted version of the original image.
• Code: This part of the network represents the compressed input
which is fed to the decoder.
• Decoder: This layer decodes the encoded image back to the original
dimension. The decoded image is a lossy reconstruction of the
original image and it is reconstructed from the latent space
representation.
Architecture of Autoencoders

The layer between the encoder and


decoder, ie. the code is also known
as Bottleneck. This is a well-
designed approach to decide which
aspects of observed data are
relevant information and what
aspects can be discarded. It does
this by balancing two criteria :
•Compactness of representation,
measured as the compressibility.
• It retains some behaviourally
Hyperparameters of Autoencoders:

There are 4 hyperparameters that we need to set before training an autoencoder:


•Code size: It represents the number of nodes in the middle layer. Smaller size results in more
compression.
•Number of layers: The autoencoder can consist of as many layers as we want.
•Number of nodes per layer: The number of nodes per layer decreases with each subsequent
layer of the encoder, and increases back in the decoder. The decoder is symmetric to the
encoder in terms of the layer structure.
•Loss function: We either use mean squared error or binary cross-entropy. If the input values
are in the range [0, 1] then we typically use cross-entropy, otherwise, we use the mean squared
error.
 
Types of Autoencoders

Convolution Autoencoders
Autoencoders in their traditional formulation does not take into account the fact that a signal can be seen as a
sum of other signals. Convolutional Autoencoders use the convolution operator to exploit this observation.
They learn to encode the input in a set of simple signals and then try to reconstruct the input from them,
modify the geometry or the reflectance of the image.
Use cases of CAE:
•Image Reconstruction
•Image Colorization
•latent space clustering
•generating higher resolution images
Deep Autoencoders
The extension of the simple Autoencoder is the Deep Autoencoder. The first layer of the Deep
Autoencoder is used for first-order features in the raw input. The second layer is used for second-order
features corresponding to patterns in the appearance of first-order features. Deeper layers of the Deep
Autoencoder tend to learn even higher-order features.
A deep autoencoder is composed of two, symmetrical deep-belief networks-
1.First four or five shallow layers representing the encoding half of the net.
2.The second set of four or five layers that make up the decoding half.
Use cases of Deep Autoencoders
•Image Search
•Data Compression
•Topic Modeling & Information Retrieval (IR)
 
Denoising Autoencoders
• There is another way to force the autoencoder to learn useful
features, which is adding random noise to its inputs and making it
recover the original noise-free data.
• This way the autoencoder can’t simply copy the input to its output
because the input also contains random noise.
• We are asking it to subtract the noise and produce the underlying
meaningful data. This is called a denoising autoencoder.
The top row contains the original images. We add random Gaussian noise to them
and the noisy data becomes the input to the autoencoder. The autoencoder doesn’t
see the original image at all. But then we expect the autoencoder to regenerate the
noise-free original image.
 Sparse Autoencoders

• We introduced two ways to force the autoencoder to learn useful features:


keeping the code size small and denoising autoencoders. The third method
is using regularization.
• We can regularize the autoencoder by using a sparsity constraint such that
only a fraction of the nodes would have nonzero values, called active nodes.
• In particular, we add a penalty term to the loss function such that only a
fraction of the nodes become active. This forces the autoencoder to
represent each input as a combination of small number of nodes, and
demands it to discover interesting structure in the data.
• This method works even if the code size is large, since only a small subset of
the nodes will be active at any time.
Contractive Autoencoders
A contractive autoencoder is an unsupervised deep learning technique that helps a neural
network encode unlabeled training data. This is accomplished by constructing a loss
term which penalizes large derivatives of our hidden layer activations with respect to the
input training examples, essentially penalizing instances where a small change in the
input leads to a large change in the encoding space.
Sparse Autoencoders
Sparse autoencoders offer us an alternative method for introducing an information bottleneck without
requiring a reduction in the number of nodes at our hidden layers. Instead, we’ll construct our loss
function such that we penalize activations within a layer.

You might also like