An evaluation of machine translation quality of news headlines / Luwaytha Salah Habeeb

The present study is concerned with the analysis and evaluation of the Machine Translation (MT) of Arabic news headlines into English. The study aims at examining the translation output against translation techniques to find out which of them are used in the translation process, and what influence they have on conveying the accurate meaning of the news headlines. Besides, the study intends to evaluate the machine translated versions of selected news headlines produced by the used machine translators, namely Google and Babylon, in terms of clarity, accuracy and style through a questionnaire distributed to experienced professionals working in different academic institutions. The data for the study consists of 40 Arabic news headlines with their manually translated versions which were selected from three online sources, namely Aljazeera, Al-Hayat, and Al-Sharqalawsat, and which were retranslated electronically into English using two machine translators, Google and Babylon. The outputs of the machine translations are analysed to determine the structure of each translated headline in terms of Swan’s model (1996) and the influence of that structure on the quality of the original meaning of the news headline. Besides that, the selected data was also examined to find the types of translation techniques that are available in both machine outputs. Furthermore, this study will make use of Hutchins and Somers‘s evaluation criteria (1992), namely on clarity, accuracy and style. The study reveals that most of the distinctive features of Swan’s (1996) model are found in the structure of the machine translated outputs, that is, there is a list of vocabulary, noun phrases with no verbs, phrases with simple present tense of verbs, passives with no auxiliaries, and phrases with infinitives. In addition to that, the punctuation marks which are listed by Swan were also found in the outcome. Among the translation techniques used by the two machine translators in the translation process are additions, alterations and subtractions. Sometimes, three techniques are iii available in one translated output. The translated outputs contain additions of articles and prepositions, alteration of nouns into verbs, plural into singular, and various types of subtractions, such as subtractions of articles, prepositions, and pronouns. The results of the distributed questionnaire reveal that both Google and Babylon have 80% of Clarity. However, Google scored a higher value for Accuracy(77.5%) than Babylon (75%), whereas Babylon scored a higher value for Style(72.5%) than Google (70%). In comparison with the results of previous studies, the results of this study indicate that MT is undergoing improvement; MT today is better than yesterday and MT tomorrow will be better than today. Keywords: MT, News Headlines, Google and Babylon translate, Quality, and MT Evaluatio