Fear the REAPER: A System for Automatic Multi-Document Summarization with Reinforcement Learning

This paper explores alternate algorithms, reward functions and feature sets for per-forming multi-document summarization using reinforcement learning with a high focus on reproducibility. We show that ROUGE results can be improved using a unigram and bigram similarity metric when training a learner to select sentences for summarization. Learners are trained to summarize document clusters based on various algorithms and reward functions and then evaluated using ROUGE. Our ex-periments show a statistically significant improvement of 1.33%, 1.58%, and 2.25% for ROUGE-1, ROUGE-2 and ROUGE-L scores, respectively, when compared with the performance of the state of the art in automatic summarization with re-inforcement learning on the DUC2004 dataset. Furthermore query focused exten-sions of our approach show an improve-ment of 1.37 % and 2.31 % for ROUGE-2 and ROUGE-SU4 respectively over query focused extensions of the state of the art with reinforcement learning on the DUC2006 dataset.