Measuring the Impact of Neural Machine Translation on Easy-to-Read Texts: An Exploratory Study

Over the last decade, translation research has experienced an increased interest in the study of controlled languages (CLs). The vast majority of studies carried out during that time have focused on technical documentation, with a strong emphasis on the implications of combining CL approaches with machine translation (MT) for the quality of the final translation product. However, few research efforts have been devoted to exploring the impact of MT on texts that follow the guidelines of other forms of controlled language, such as plain language (PL) or Easy to Read (EtR). The most recent example can be found in the work by Rossetti (2018), who measured the machine translatability of PL summaries of health-related complex systematic reviews through the estimation of number and severity of errors in the MT output. Yet, the focus was still on target text fluency and adequacy rather than usability of the resulting PL text. Similarly, target users have rarely included people with special needs, as most studies of this nature are aimed at non-native speakers of a given language. By shifting the focal point towards the end user, our project aims to explore the impact of neural machine translation (NMT) on the usability of EtR texts by people with intellectual disabilities. The main goal of the planned study is to understand whether the EtR guidelines followed in the source text are respected in the resulting NMT output for the English-French language combination. The rationale behind the tests is that, as opposed to statistical machine translation (SMT), output is less predictable and undesired grammar and vocabulary changes can be introduced by the system (Neubig, Morishita, and Nakamura 2015). The study will consist of two phases: first, a textual analysis will be carried out to compare the outputs of two SMT and NMT systems in terms of translation quality and EtR guidelines violation. Second, a user evaluation stage will be conducted to assess the resulting translations for comprehensibility and satisfaction. We expect that study findings will provide further insight into whether NMT is a viable option for translating EtR documents for accessible communication in this particular language pair