The CLEF-2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News

Nakov, Preslav
Da San Martino, Giovanni
Elsayed, Tamer
Barrón-Cedeño, Alberto
Míguez, Rubén
Shaar, Shaden
Alam, Firoj
Haouari, Fatima
Hasanain, Maram
Babulkov, Nikolay
Nikolov, Alex
Shahi, Gautam Kishore
Struß, Julia Maria
M
l, Thomas

Publication date

January 2021

Publisher

Springer International Publishing

Abstract

We describe the fourth edition of the CheckThat! Lab, part of the 2021 Cross-Language Evaluation Forum (CLEF). The lab evaluates technology supporting various tasks related to factuality, and it is offered in Arabic, Bulgarian, English, and Spanish. Task 1 asks to predict which tweets in a Twitter stream are worth fact-checking (focusing on COVID-19). Task 2 asks to determine whether a claim in a tweet can be verified using a set of previously fact-checked claims. Task 3 asks to predict the veracity of a target news article and its topical domain. The evaluation is carried out using mean average precision or precision at rank k for the ranking tasks, and F1 for the classification tasks