Generative Adversarial Networks (GANs)

The coolest idea in Machine Learning in the last twenty years - Yann Lecun

Generative Adversarial Networks (GANs) 1 / 28

Introduction

Generative Adversarial Networks (GANs) 2 / 28

Supervised Learning

Find deterministic function f: y = f(x), x:data, y:label

Generative Adversarial Networks (GANs) 3 / 28

Unsupervised Learning

More challenging than supervised learning. No label or curriculum.

Some NN solutions:
Boltzmann machine
AutoEncoder
Generative Adversarial Networks

Generative Adversarial Networks (GANs) 4 / 28

Unsupervised Learning

"Most of human and animal learning is unsupervised learning. If

intelligence was a cake, unsupervised learning would be the cake, supervised
learning would be the icing on the cake, and reinforcement learning would
be the cherry on the cake. We know how to make the icing and the cherry,
but we do not know how to make the cake. We need to solve the
unsupervised learning problem before we can even think of getting to true
AI." - Yann Lecun
"What I cannot create, I do not understand" - Richard P. Feynman

Generative Adversarial Networks (GANs) 5 / 28

Generative Models
Introduction

What does "generative" mean?

You can sample from the model and the distribution of samples
approximates the distribution of true data points
To build a sampler, do we need to optimize the log-likelihood?
Does this sampler work?

Generative Adversarial Networks (GANs) 6 / 28

Generative Adversarial Networks
Desired Properties of the Sampler

What is wrong with the sampler? Why is not desired?

Desired Properties
We don’t want to sample the same existing data points
Build a generative model that understands structure and underlying
distribution of data points
Output similar but not the same points in the training dataset
Example: Faces with different poses, Different handwriting etc

Generative Adversarial Networks (GANs) 7 / 28

Generative Adversarial Networks

Implicit Models

Given samples from data distribution pdata : x1 , x2 , . . . , xn

Given a sampler qφ (z) = DNN(z; φ) where z ∼ p(z)
Let pmodel be the density function of x = qφ (z)
We do not have an explicit form for pdata or pmodel
Objective: Make pmodel as close as possible to pdata by learning φ

Generative Adversarial Networks (GANs) 8 / 28

Generative Adversarial Networks
Moment Matching

Given samples X = x1 , x2 , . . . , xn from PX and samples

Y = y1 , y2 , . . . , yn from PY , how do we compare PX and PY ?
Match moments of both PX and PY to bring them closer
Not feasible to compute higher order moments in higher dimensions

Generative Adversarial Networks (GANs) 9 / 28

Generative Adversarial Networks

Moment Matching

Generative Adversarial Networks (GANs) 10 / 28

Generative Adversarial Networks

Generative Adversarial Networks (GANs) 11 / 28

Generative Adversarial Networks

Generative Adversarial Networks (GANs) 12 / 28

Generative Adversarial Networks

Generative Adversarial Networks (GANs) 13 / 28

Generative Adversarial Networks

What is going on?

What is the optimal discriminator given generated and true

distributions?

Generative Adversarial Networks (GANs) 14 / 28

Generative Adversarial Networks
What is going on?

Generative Adversarial Networks (GANs) 15 / 28

Generative Adversarial Networks

What is going on?

What is the objective of the generator given the optimal discriminator?

Generative Adversarial Networks (GANs) 16 / 28

Generative Adversarial Networks
Discriminator Saturation

Generative Adversarial Networks (GANs) 17 / 28

Generative Adversarial Networks

Optimization

Generative Adversarial Networks (GANs) 18 / 28

Generative Adversarial Networks
What is going on?

Generative Adversarial Networks (GANs) 19 / 28

Generative Adversarial Networks

Live Action

GAN LAB

Generative Adversarial Networks (GANs) 20 / 28

Generative Adversarial Networks
Result

Generative Adversarial Networks (GANs) 21 / 28

Generative Adversarial Networks

DCGAN Results

Generative Adversarial Networks (GANs) 22 / 28

Evaluation of GANs

Still an open research problem

Some Measures:
Inception Score
Frechet Inception Distance

Generative Adversarial Networks (GANs) 23 / 28

Inception Score

How do we say generated images are good?

Good generators generate semantically diverse samples
Use a trained model (Inception v3) to model p(y | x)
Each image should have a recognizable object from a class → p(y | x)
should have low entropy
There should be many classes generated → p(y) should have high
entropy

Generative Adversarial Networks (GANs) 24 / 28

Inception Score

Generative Adversarial Networks (GANs) 25 / 28

What GAN you do?

Generative Adversarial Networks (GANs) 26 / 28

What GAN you do?

Generative Adversarial Networks (GANs) 27 / 28

References

CS294-158 Deep Unsupervised Learning Spring 2019 - UC Berkeley

CS231n Deep Learning for Computer Vision - Stanford University
Tutorial on GANs, Ian Goodfellow

Generative Adversarial Networks (GANs) 28 / 28