Materials

cs231n Lecture 12 slides

Variational Auto-encoder Architecture

• $\mathbf X$: Training Data
• $\mathbf X'$: Generated Sample
• Green: encoding / real data (training)
• Blue: generation
• Red: latent variable, simple/tractable distribution

Autoencoder

Unsupervised approach for learning a lower-dimensional feature representation from unlabeled training data

• $\mathbf z$ usually smaller than $\mathbf x$ (dimensionality reduction)

  Why? we want features to capture meaningful factors of variation in data

• How to learn this feature representation?

  L2 Loss function $||x-\hat x||^2$ → doesn’t use labels

• After training, throw away decoder

• Transfer from large and unlabeled dataset to small and labeled dataset
• Encoder can be used to initialize a supervised model
• Can we use autoencoder for image generation?
  • (X) We do not know the distribution of $\mathbf z$
  • (X) We do not know how to draw a sample from $p(z)$
  • Then… How do we make autoencoder a generative model?