Generative Models vs Discriminative Models

• Generative Models:

  do not need to have labels

  learn all the probability density

  Find $f(x), f(x,y)$ (estimate distributions)

• Discriminative Models

  Input - Label matching

  do not need to know distributions

  Find $f(y|x)$

  ex) Classification, Regression

Applications of generative models

• Content generation
  • DALLE, Imagen by Google, Make-A-Video from Meta, Copilot by OpenAI
• Representation learning
• Data Augmentation
• Image editing & Art
• Classification
• Anomarly detection

  • Anormarly = Low Likelihood (density)

  • Distance to the k-nearest neighbors

  • One-Class Classification

  • Reconstruction error

  • Voting / Teacher-students

  • Variance / Uncertainty

  • Auto-encoder based AD

    $\mathbf z = \phi_e(\mathbf x; \Theta_e), \hat {\mathbf x}=\phi_d(\mathbf z; \Theta_d)$

    $\{\Theta_e^, \Theta_d^\}=\argmin_{\Theta_e, \Theta_d}\sum_{\mathbf x \in \mathcal X}||\mathbf x-\phi_d(\phi_e(\mathbf x;\Theta_e);\Theta_d)||^2$

    $s_{\mathbf x}=||\mathbf x-\hat{\mathbf x}||^2 = ||\mathbf x-\phi_d(\phi_e(\mathbf x;\Theta_e^);\Theta_d^)||^2$