Evaluation of an English-Chinese Cross-Lingual Retrieval Experiment

We conducted retrieval experiments using a collection of 170MB Chinese text from TREC-5. The supplied queries (topics)are both English and Chinese, the latter to a large extent may be considered as very good translation of the former. Another set of Chinese queries were created by translating the English via a simple dictionary look-up procedure. Retrieval effectiveness for both types of queries were evaluated, essentially giving upper and lower bounds of how machine translation may contribute to this cross-lingual retrieval experiment. Results show that naive translation returns effectiveness about 30 % to 50 % worse than good translation.