Variational Inference with Normalizing Flows Danilo Jimenez Rezende

The choice of approximate posterior distribution is one of the core problems in variational infer-ence. Most applications of variational inference employ simple families of posterior approxima-tions in order to allow for efficient inference, fo-cusing on mean-field or other simple structured approximations. This restriction has a signifi-cant impact on the quality of inferences made using variational methods. We introduce a new approach for specifying flexible, arbitrarily com-plex and scalable approximate posterior distribu-tions. Our approximations are distributions con-structed through a normalizing flow, whereby a simple initial density is transformed into a more complex one by applying a sequence of invertible transformations until a desired level of complex-ity is attained. We use this view of normalizing flows to develop categories of finite and infinites-imal flows and provide a unified view of ap-proaches for constructing rich posterior approxi-mations. We demonstrate that the theoretical ad-vantages of having posteriors that better match the true posterior, combined with the scalability of amortized variational approaches, provides a clear improvement in performance and applica-bility of variational inference. 1