Text-based language identification for the South African languages

We investigate the performance of text-based language identification systems on the 11 offi-cial languages of South Africa, when n-gram statistics are used as features for classification. In particular, we compare support vector ma-chines (SVMs) and likelihood-based classifiers on different amounts of input text, both from a closed domain and an open domain. With as few as 15 words of input text, reliable language identification is possible. Although the SVM is generally more accurate a classifier, the addi-tional computational complexity of training this classifier may not be justified in light of the im-portance of using a large value for n. 1