Stochastic backpropagation and approximate inference in deep generative models

We marry ideas from deep neural networks and approximate Bayesian inference to derive a gen-eralised class of deep, directed generative mod-els, endowed with a new algorithm for scalable inference and learning. Our algorithm introduces a recognition model to represent an approximate posterior distribution and uses this for optimisa-tion of a variational lower bound. We develop stochastic backpropagation – rules for gradient backpropagation through stochastic variables – and derive an algorithm that allows for joint op-timisation of the parameters of both the genera-tive and recognition models. We demonstrate on several real-world data sets that by using stochas-tic backpropagation and variational inference, we obtain models that are able to generate realistic samples of data, allow for accurate imputations of missing data, and provide a useful tool for high-dimensional data visualisation. 1