Analyzing the Sense Distribution of Concordances Obtained by Web As Corpus Approach

Abstract. In corpus-based lexicography and natural language processing fields some authors have proposed using the Internet as a source of corpora for obtain-ing concordances of words. Most techniques implemented with this method are based on information retrieval-oriented web searchers. However, rankings of concordances obtained by these search engines are not built according to lin-guistic criteria but to topic similarity or navigational oriented criteria, such as page-rank. It follows that examples or concordances could not be linguistically representative, and so, linguistic knowledge mined by these methods might not be very useful. This work analyzes the linguistic representativeness of concord-ances obtained by different relevance criteria based web search engines (web, blog and news search engines). The analysis consists of comparing web con-cordances and SemCor (the reference) with regard to the distribution of word senses. Results showed that sense distributions in concordances obtained by web search engines are, in general, quite different from those obtained from the reference corpus. Among the search engines, those that were found to be the most similar to the reference were the informational oriented engines (news and blog search engines).