Neural machine translation of literary texts from English to Slovene

Raziskave so pokazale, da dosegajo nevronski strojni prevajalniki obetavne rezultate pri prevajanju literarnih besedil. Strojno prevajanje literature iz angleščine v slovenščino pa še ni bilo raziskano, zato je namen tega dela razvoj nevronskega strojnega prevajalnika v več različicah, prilagojenega za prevajanje literature, in primerjava lastnih prevajalnikov s splošnim prevajalnikom Google Translate. Da bi ugotovila, ali specializirani modeli dosežejo boljše rezultate kot splošni prevajalnik Google Translate in ali je prevajalnik, specializiran za prevajanje romanov ene avtorice, uspešnejši kot prevajalnik, specializiran na različnih literarnih delih, primerjam prevajalnike z metrikama BLEU in METEOR, analizo napak strojnih prevodov, ocenjevanjem berljivosti in ustreznosti ter merjenjem zahtevnosti popravljanja. Analiza je pokazala, da je popravljanje strojnih prevodov vseh treh prevajalnikov hitrejše od prevajanja od začetka. Vse metode evalvacije razen ocene berljivosti so potrdile, da je prevajalnik, specializiran za prevajanje romanov ene avtorice, uspešnejši kot prevajalnik, naučen na korpusu različnih literarnih del. Kljub vsemu pa pri vseh metodah evalvacije dosega najboljše rezultate prevajalnik Google Translate. Raziskava razkriva tudi velike nekonsistentnosti med udeleženci pri ocenjevanju berljivosti in ustreznosti ter pri merjenju zahtevnosti popravljanja. Rezultati nakazujejo na to, da morda te metode niso zanesljive, zato bi bilo dobro te ugotovitve nadaljnje raziskati.Neural Machine Translation (NMT) has shown a promising performance on literary texts. Since the literary machine translation has not yet been researched for the English-to-Slovene translation direction, this Master’s thesis aims to bridge this gap by presenting a comparison among bespoke NMT models, tailored to novels, and Google Translate. To determine whether models, adapted to literary texts, perform better than Google Translate, and to establish whether an adaptation to a specific author further improves the performance of the NMT system, models were evaluated by the BLEU and METEOR metrics, error analysis of machine translation output, assessment of fluency and adequacy, and measurement of the post-editing (PE) effort. The findings show that all evaluated NMT approaches resulted in increases in translation productivity. The model, tailored to a specific author, performs better than the model, trained on a literary corpus, based on all scores except the scores for fluency. However, Google Translate still outperforms all bespoke models. The evaluation reveals a very low inter-rater agreement on fluency and adequacy, based on the kappa coefficient values, and significant discrepancies between post-editors. This suggests that these methods are not reliable, which should be addressed in future studies