Header logo is

The Numerics of GANs


Conference Paper


In this paper, we analyze the numerics of common algorithms for training Generative Adversarial Networks (GANs). Using the formalism of smooth two-player games we analyze the associated gradient vector field of GAN training objectives. Our findings suggest that the convergence of current algorithms suffers due to two factors: i) presence of eigenvalues of the Jacobian of the gradient vector field with zero real-part, and ii) eigenvalues with big imaginary part. Using these findings, we design a new algorithm that overcomes some of these limitations and has better convergence properties. Experimentally, we demonstrate its superiority on training common GAN architectures and show convergence on GAN architectures that are known to be notoriously hard to train.

Author(s): Lars Mescheder and Sebastian Nowozin and Andreas Geiger
Book Title: Proceedings Neural Information Processing Systems
Year: 2017

Department(s): Autonomous Vision
Bibtex Type: Conference Paper (inproceedings)

Event Name: Neural Information Processing Systems (NIPS 2017)
Event Place: Long Beach, USA

Links: pdf


  title = {The Numerics of GANs },
  author = {Mescheder, Lars and Nowozin, Sebastian and Geiger, Andreas},
  booktitle = {Proceedings Neural Information Processing Systems},
  year = {2017}