Auto-Encoder

An Autoencoder is a neural network that's trained on compressing and reconstructing some data.

It has two parts, encoder and decoder and model's loss is defined as: $L = ||x - \hat{x}||^2$ which is also called reconstruction error.

The bottleneck in the middle forces model to learn what's identifying about the information. This identification part allows it to be used in:

• Anomaly detection: When trained on "normal" data, passing anormal data to an autoencoder results with a high reconstruction error.
• Denoising and Data Cleaning: When trained on noisy data and target is the clean version, model learns to see through the noise and recover the actual data.
• Transfer learning or Representation learning: When we have limited labeled data, we can train an auto-encoder on it and let it learn its features and then only use the encoder part to have a feature extractor.