Distributional context generalisation and normalisation as a mean to reduce data sparsity: evaluation of medical corpora

International audienceDistributional hypothesis relies on the recurrence of information in the contexts of words to associate. However, these Vector space models implementing the approach suffer from data sparsity, and then from a high number of dimensions in the matrix of contextual vectors. If the data sparseness reduction is an important aspect with general corpora. it is also a major issue with specialised corpora that are of much smaller size and with much lower context frequencies. We tackle the problem of data sparsity on specialised texts and we propose a method that allows to make the matrix denser. To achieve this goal, we use synonymy and hypernymy relations acquired on the corpora to normalise and then generalise the distributional contexts. In addition, we experiment various configuration as the size of graphical window, similarity measures, threshold on the distributional parameters. Evaluation has been performed on a French biomedical corpus. On one hand, Generalisation gives better results with Jaccard index, narrow graphical windows and relations of lexical inclusion. On the other hand, normalisation improves the results with large windows, but thresholds on the distributional parameters need to be tuned