Augmenting Neural Encoder-Decoder Model for Natural Language Generation Tasks

Neural Encoder-Decoder model has been widely adopted for grounded language generation tasks. Such tasks usually require translation of data from one domain to a different domain, including machine translation (language to language), image captioning (image to language), and text summarization (long article to short summary). In this thesis, we aim to improve Neural Encoder-Decoder model for two different generation tasks: text summarization and image captioning. For summarization, we aim to improve the encoder of a popular pointer-generator model by adding a ‘closed-book’ decoder without attention and pointer mechanism. We argue that such a decoder forces the encoder to be more selective on the information encoded in its memory state since it can’t rely on the extra information provided by the attention and copy modules, and hence improves the entire model. On the CNN/Daily Mail dataset, our model outperforms the baseline significantly in terms of ROUGE and METEOR metrics and receives higher saliency scores, for both cross-entropy and reinforced setups. For image captioning, we propose a framework based on Conditional Generative Adversarial Network (CGAN), which jointly trains a a generator that produces captions conditioned on the image, and a discriminator that evaluates the probability of the caption being generated or not. We present a series of experiments to show that our CGAN-based model consistently and significantly outperforms strong baseline trained with maximum likelihood, which is not achieved by previous work that adopted a similar approach.Bachelor of Scienc