Dr. Preslav Nakov and Dr. Giovanni Da San Martino from QCRI took part in providing an overview of the second version of CheckThat! Lab.
They presented two papers on tasks aimed at the automatic identification and verification of claims. The first paper, titled “Automatic Identification and Verification of Claims. Task 1: Check-Worthiness”, focused on developing a system for automatically identifying claims that are worth checking in political debates. The second paper, “Automatic Identification and Verification of Claims. Task 2: Evidence and Factuality”, aimed to provide a full evaluation framework that used data in Arabic that was evaluated based on normalized discounted cumulative gain (nDCG) for ranking and F1 for classification.
The system that the researchers are trying to build has the potential to be a valuable tool for fact-checkers and journalists, who often need to quickly verify the accuracy of claims made in news stories. In the age of the internet and rapid access to information, it is easier than ever to spread misinformation. However, it is harder than ever to identify that misinformation. Hence, it becomes the job of researchers to develop the necessary tools that can first identify the news most worthy of fact-checking, thus, reducing the time and effort needed to fact-check the never-ending ocean of news. The CheckThat! Lab 2019 was held as a part of CLEF (Conference and Labs of the Evaluation Forum) 2019, the 10th edition of the CLEF campaigns. It brings independent peer-reviewed conferences related to issues in the field of multilingual and multimodal information access evaluation. CLEF aims to contribute to the systematic evaluation of information access systems using, primarily, experimentation on shared tasks. The 2019 edition of CLEF was held in Lugano, Switzerland, September 9-12, 2019.
- Overview of the CLEF-2019 CheckThat! Lab: Automatic Identification and Verification of Claims. Task 1: Check-Worthiness
Pepa Atanasova, Preslav Nakov, Georgi Karadzhov, Mitra Mohtarami, Giovanni Da San Martino
- Overview of the CLEF-2019 CheckThat! Lab: Automatic Identification and Verification of Claims. Task 2: Evidence and Factuality
Maram Hasanain, Reem Suwaileh, Tamer Elsayed, Alberto Barrón-Cedeño, Preslav Nakov