Three ploys for robust co-generation with generative adversarial nets

Generative adversarial nets (GANs) and variational auto-encoders enable accurate modeling of high-dimensional data distributions by forward propagating a sample drawn from a latent space. However, an often overlooked shortcoming is their inability to find an arbitrary marginal distribution, which is useful for completion of missing data in tasks like super-resolution, image inpainting, etc., where we don’t know the missing part ahead of time. To address such applications it seems intuitive at first to search for that latent space sample which ‘best’ matches the observations. However, irrespective of the GAN loss, unexpected challenges arise: we find that the energy landscape of well trained generators is extremely hard to optimize, exhibiting ‘folds’ that are very hard to overcome. To address this issue, in this thesis, three ploys are proposed which help to address the challenge for all investigated GAN losses and which yield more accurate reconstructions, quantitatively and qualitatively