End-to-end Learning of Latent Dirichlet Allocation by Mirror-Descent Back Propagation

We develop a fully discriminative learning approach for supervised Latent Dirich-let Allocation (LDA) model, which maximizes the posterior probability of the pre-diction variable given the input document. Different from traditional variational learning or Gibbs sampling approaches, the proposed learning method applies (i) the mirror descent algorithm for exact maximum a posterior inference and (ii) back propagation with stochastic gradient descent for model parameter estimation, leading to scalable learning of the model in an end-to-end discriminative manner. As a byproduct, we also apply this technique to develop a new learning method for the traditional unsupervised LDA model. Experimental results on two real-world regression and classification tasks show that the proposed methods significantly outperform the previous supervised/unsupervised LDA learning methods.