Compression of Persian Text for WebBased Applications, Without Explicit Decompression

Abstract: The increasing importance of Unicode for text files implies a possible doubling of data storage space and data transmission time, with a corresponding need for data compression. The approach presented in this paper aims to reduce the storage and the transmission time for Farsi text files in web-based applications and Internet. The basic idea here is to compute the most repetitive n-grams in the Farsi text and replace them by a single character. The new characters then will be placed in the user-defined sections of the Unicode and proper glyphs will be created for them. In this approach the compression will be done on the server side once and the decompression process is eliminated completely. The rendering process in the browser will do the decompression. There is no need for any additional program or add-ins for decompression to be installed on the browser or client side. The user needs only to download the proper Unicode font once. A genetic algorithm is utilized to select the most appropriate n-grams of different sizes in order to maximum the amount of compression. In the best case, we have achieved 52.26 % reduction of the file size. The method is general, and applies equally well to English and other languages