A realistic evaluation of indoor positioning systems based on Wi-Fi fingerprinting: The 2015 EvAAL–ETRI competition

Pre-print versionThis paper presents results from comparing different Wi-Fi fingerprinting algorithms on the same private dataset. The algorithms where realized by independent teams in the frame of the off-site track of the EvAAL-ETRI Indoor Localization Competition which was part of the Sixth International Conference on Indoor Positioning and Indoor Navigation (IPIN 2015). Competitors designed and validated their algorithms against the publicly available UJIIndoorLoc database which contains a huge reference- and validation data set. All competing systems were evaluated using the mean error in positioning, with penalties, using a private test dataset. The authors believe that this is the first work in which Wi-Fi fingerprinting algorithm results delivered by several independent and competing teams are fairly compared under the same evaluation conditions. The analysis also comprises a combined approach: Results indicate that the competing systems where complementary, since an ensemble that combines three competing methods reported the overall best results.We would like to thank Francesco Potortì, Paolo Barsocchi, Michele Girolami and Kyle O’Keefe for their valuable help in organizing and spread the EVAALETRI competition and the off-site track. We would also like to thank the TPC members Machaj Juraj, Christos Laoudias, Antoni Pérez-Navarro and Robert Piché for their valuable comments, suggestions and reviews. Parts of this work were funded in the frame of the Spanish Ministry of Economy and Competitiveness through the “Metodologiías avanzadas para el diseño, desarrollo, evaluación e integración de algoritmos de localización en interiores” project (Proyectos I+D Excelencia, código TIN2015-70202-P) and the “Red de Posicionamiento y Navegación en Interiores” network (Redes de Excelencia, código TEC2015-71426- REDT). Parts of this work were funded in the frame of the German federal Ministry of Education and Research programme "FHprofUnt2013" under contract 03FH035PB3 (Project SPIRIT).