Human-Computer Interaction as a basis for assessing Geographic Information Retrieval Systems.

In recent years, researches related to Geographic Information Retrieval Systems as a specific field of Information Retrieval has continued to attract the attention of the research community by holding several assessing forums. However, these forums provide sets of tests comprised of text documents and queries that are ready to evaluate non‐interactive systems. This framework reduces the possibilities of carrying out a more thorough evaluation of these systems because it is not considering several important features such as the diversity provided by different information sources or the human‐computer interaction. The aim of this paper is to describe a new approach to evaluate interactive Geographic Information Retrieval Systems, which main novelty is to consider the user’s knowledge generated by the human‐computer interaction as well as the spatial information provided by different data sources. The proposed method will require generating a set of tests from three main data sources (Geonames, Wikipedia, and OpenStreetMap), as well as a set of queries that will consist of a tuple of three components: the object type, the spatial relationship and the geographic object. As a result, the proposed evaluation approach integrates the two most commonly used strategies to evaluate IR systems, which are focused on the system and the end user, by applying several user satisfaction techniques and usability tests. As a main conclusion, we pointed out that the evaluation process of Geographic Information Retrieval systems should consider the user’s knowledge generated by the human-computer interaction as well as the spatial information provided by different and heterogeneous data sources