9:00-9:05 Opening Remarks
FEVER Organizers
9:05-9:35 Detecting the "Fake News" Before It Was Even Written, Media Literacy, and Flattening the Curve of the COVID-19 Infodemic
Preslav Nakov
9:35-10:05 Personalised Longitudinal Natural Language Processing
Maria Liakata
10:05-10:20 Live Q&A Session
Q&A with Preslav Nakov and Maria Liakata
10:20-10:50 Oral Presentations
10:20-10:35 Evidence Selection as a Token-Level Prediction Task
Dominik Stammbach
10:35-10:50 Graph Reasoning with Context-Aware Linearizations for Fact Extraction and Verification
Neema Kotonya, Thomas Spooner, Daniele Magazzeni and Francesca Toni
10:50-11:00 Live Q&A Session
Q&A with Oral Presentation speakers
11:00-11:15 Break
11:15-11:45 Extractive Opinion Summarization in Quantized Transformer Spaces
Mirella Lapata
11:45-12:15 Answering Complex Questions Over Knowledge Bases and Natural Language Corpora At Scale
Pasquale Minervini
12:15-12:30 Live Q&A Session
Q&A with Mirella Lapata and Pasqualle Minervini
12:30-13:30 Lunch Break
13:30-14:00 Facts and myths about (mis)information exposure and beliefs
Brendan Nyhan
14:00-14:10 The FEVEROUS Shared Task
FEVER Organizers
14:10-14:30 Shared Task System Presentations
14:10-14:20 FaBULOUS: Fact-checking Based on Understanding of Language Over Unstructured and Structured information
Mostafa Bouziane, Hugo Perrin, Amine Sadeq, Thanh Nguyen, Aurélien Cluzeau and Julien Mardas
14:20-14:30 Team Papelo at FEVEROUS: Multi-hop Evidence Pursuit
Christopher Malon
14:30-14:45 Live Q&A Session
Q&A with Brendan Nyhan and Shared Task System presenters
14:45-16:15 Poster Session [show/hide details]
Modeling Entity Knowledge for Fact Verification
Yang Liu, Chenguang Zhu and Michael Zeng
NCU: Verdict Inference with Claim and Retrieved Elements Using RoBERTa
In-Zu Gi, Ting-Yu Fang and Richard Tzong-Han Tsai
Stance Detection in German News Articles
Laura Mascarell, Tatyana Ruzsics, Christian Schneebeli, Philippe Schlattner, Luca Campanella, Severin Klingler and Cristina Kadar
Combining sentence and table evidence to predict veracity of factual claims using TaPaS and RoBERTa
Martin Funkquist
Automatic Fact-Checking with Document-level Annotations using BERT and Multiple Instance Learning
Aalok Sathe and Joonsuk Park
Neural Re-rankers for Evidence Retrieval in the FEVEROUS Task
Mohammed Adel Saeed, Giulio Alfarano, Khai nguyen, Duc Pham, Raphael Troncy and Paolo Papotti
FANG-COVID: A New Large-Scale Benchmark Dataset for Fake News Detection in German
Justus Mattern, Yu Qiao, Elma Kerz, Daniel Wiechmann and Markus Strohmaier
A Fact Checking and Verification System for FEVEROUS Using a Zero-Shot Learning Approach
Orkun Temiz, Özgün Ozan Kılıç, Arif Ozan Kızıldağ and Tuğba Taşkaya Temizel
RelDiff: Enriching Knowledge Graph Relation Representations for Sensitivity Classification
Hitarth Narvala, Graham McDonald and Iadh Ounis
Knowledge-Enhanced Evidence Retrieval for Counterargument Generation
Yohan Jo, Haneul Yoo, JinYeong Bak, Alice Oh, Chris Reed and Eduard Hovy
REBEL: Relation Extraction By End-to-end Language generation
Pere-Lluís Huguet Cabot and Roberto Navigli
MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets
Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md. Shad Akhtar, Preslav Nakov and Tanmoy Chakraborty
Detecting Frames in News Headlines and Lead Images in U.S. Gun Violence Coverage
Isidora Tourni, Lei Guo, Taufiq Daryanto, Fabian Zhafransyah, Edward Edberg Halim, Mona Jalal, Boqi Chen, Sha Lai, Hengchang Hu, Margrit Betke, Prakash Ishwar and Derry Tanti Wijaya
ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts
Yuta Koreeda and Christopher Manning
Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Society
Firoj Alam, Shaden Shaar, Fahim Dalvi, Hassan Sajjad, Alex Nikolov, Hamdy Mubarak, Giovanni Da San Martino, Ahmed Abdelali, Nadir Durrani, Kareem Darwish, Abdulaziz Al-Homaid, Wajdi Zaghouani, Tommaso Caselli, Gijs Danoe, Friso Stolk, Britt Bruntink and Preslav Nakov
Neural Unification for Logic Reasoning over Natural Language
Gabriele Picco, Thanh Lam Hoang, Marco Luca Sbodio and Vanessa Lopez
16:15-16:45 The Misinformation Trifecta: What are the tactics used by those spreading misinformation and false narratives online?
Steven Novella
16:45-17:15 Multimodally-Grounded Information Retrieval and Verification
Mohit Bansal
17:15-17:30 Live Q&A Session
Q&A with Steven Novella and Mohit Bansal
17:30-17:45 Closing Remarks
FEVER Organizers

Invited Talks

Multimodally-Grounded Information Retrieval and Verification
Mohit Bansal

In this talk, we will discuss how to address the tasks of fact/information extraction, retrieval, verification, and inference when grounded in multimodal information, such as dynamic spatio-temporal videos and the dialogue-based language in them. We will look at the domain of compositional, multi-hop understanding by combining information across videos and dialogue, and based on the diverse multimodal tasks of temporal moment retrieval (including multilingual), spatial grounding, question answering, future next-event inference, and retrieval highlighting/saliency., as well as a multi-task benchmark for video-and-language understanding evaluation to promote generalizable methods. Lastly, we will briefly discuss some other modalities and cross-media consistency checking for fake news detection.

Extractive Opinion Summarization in Quantized Transformer Spaces
Mirella Lapata

Online reviews play an integral role in modern life, as we look to previous customer experiences to inform everyday decisions. The need to digest review content has fueled progress in opinion mining, whose central goal is to automatically summarize people's attitudes towards an entity (e.g., a hotel or a restaurant). In this talk I will present the Quantized Transformer (QT), an unsupervised system for extractive opinion summarization. QT is inspired by Vector Quantized Variational Autoencoders, which we repurpose for popularity-driven summarization. It uses a clustering interpretation of the quantized space and a novel extraction algorithm to discover popular opinions among hundreds of reviews, a significant step towards opinion summarization of practical scope. In addition, QT enables faithful and controllable summarization by utilizing properties of the quantized space to extract aspect summaries (focusing on specific properties of an entity such as the location, or cleanliness of a hotel).

Personalised Longitudinal Natural Language Processing
Maria Liakata

In most of the tasks and models that we have made great progress with in recent years, such as text classification and natural language inference, there isn't a notion of time. However many of these tasks are sensitive to changes and temporality in real world data, especially when pertaining to individuals, their behaviour and their evolution over time. I will present a new programme of work on personalised longitudinal natural language processing. This consists in developing natural language processing methods to: (1) represent individuals over time from their language and other heterogenous and multi-modal content (2) capture changes in individuals' behaviour over time (3) generate and evaluate synthetic data from individuals' content over time (4) summarise the progress of an individual over time, incorporating information about changes. I will discuss progress and challenges this far as well as the implications of this programme of work for downstream tasks such as mental health monitoring, moderation of interactions & protection of individuals on online platforms as well as source credibility and verification of information.

Answering Complex Questions Over Knowledge Bases and Natural Language Corpora At Scale
Pasquale Minervini

We present a framework for efficiently answering complex queries on incomplete Knowledge Graphs. We translate each query into an end-to-end differentiable objective, where the truth value of each atom is computed by a pre-trained neural link predictor; we then analyse two solutions to the optimisation problem, including gradient-based and combinatorial search. The proposed approach produces more accurate results than state-of-the-art methods --- black-box neural models trained on millions of generated queries --- without the need for training on a large and diverse set of complex queries. Using orders of magnitude less training data, we obtain relative improvements ranging from 8% up to 40% in Hits@3 across different Knowledge Graphs containing factual information. Finally, we demonstrate that it is possible to explain the outcome of our model in terms of the intermediate solutions identified for each of the complex query atoms. This work was presented at ICLR 2021, where it was awarded an Outstanding Paper Award. We then introduce PAQ/RePAQ, a method to answer natural language questions over large document corpora (such as Wikipedia) extremely efficiently, allowing users to answer thousands of queries per second and improving the explainability properties of the model. This work was presented during the Efficient Open-Domain Question Answering Challenge at NeurIPS 2020, where it ranked first in two tracks out of three and was recently published in TACL.

Detecting the "Fake News" Before It Was Even Written, Media Literacy, and Flattening the Curve of the COVID-19 Infodemic
Preslav Nakov

Given the recent proliferation of disinformation online, there has been growing research interest in automatically debunking rumors, false claims, and "fake news". A number of fact-checking initiatives have been launched so far, both manual and automatic, but the whole enterprise remains in a state of crisis: by the time a claim is finally fact-checked, it could have reached millions of users, and the harm caused could hardly be undone. An arguably more promising direction is to focus on analyzing entire news outlets, which can be done in advance; then, we could fact-check the news before it was even written: by checking how trustworthy the outlet that has published it is (which is what journalists actually do). We will show how we do this in the Tanbih news aggregator (, which aims to limit the impact of "fake news", propaganda and media bias by making users aware of what they are reading, thus promoting media literacy and critical thinking, which are arguably the best way to address disinformation in the long run. In particular, we develop media profiles that show the general factuality of reporting, the degree of propagandistic content, hyper-partisanship, leading political ideology, general frame of reporting, stance with respect to various claims and topics, as well as audience reach and audience bias in social media. Another important observation is that the term "fake news" misleads people to focus exclusively on factuality, and to ignore the other half of the problem: the potential malicious intent. Thus, we detect the use of specific propaganda techniques in text, e.g., appeal to emotions, fear, prejudices, logical fallacies, etc. We will show how we do this in the Prta system (, another media literacy tool, which got the Best Demo Award (Honorable Mention) at ACL-2020; an associated shared task got the Best task award (Honorable Mention) at SemEval-2020. Finally, at the time of COVID-19, the problem of disinformation online got elevated to a whole new level as the first global infodemic. While fighting this infodemic is typically thought of in terms of factuality, the problem is much broader as malicious content includes not only "fake news", rumors, and conspiracy theories, but also promotion of fake cures, panic, racism, xenophobia, and mistrust in the authorities, among others. Thus, we argue for the need of a holistic approach combining the perspectives of journalists, fact-checkers, policymakers, social media platforms, and society as a whole, and we present our recent research in that direction (

The Misinformation Trifecta: What are the tactics used by those spreading misinformation and false narratives online?
Steven Novella

Misinformation is becoming ubiquitous, either due to honest reporting mistakes or, increasingly as a deliberate method to achieve political, corporate or ideological goals. In this talk I will discuss three strategies that purveyors of misinformation take advantage of - the "misinformation trifecta": the spreading the misinformation itself, the failure of the "knowledge deficit" approach, and the use of free speech as an attempt to evade quality control. I will go over examples for each strategy and discuss possible mitigation methods.

Facts and myths about (mis)information exposure and beliefs
Brendan Nyhan

Observers fear that people are trapped in digital “echo chambers” that amplify misinformation and extremism. However, these claims overstate the prevalence and severity of these patterns, which at most capture the experience of a minority of the public. Similarly, the ineffectiveness of fact-checks at countering misinformation has been overstated in prior research and media coverage. Exposure to corrective information reduces belief in false claims, though these effects do not persist over time. I will present the findings from several recent studies on these points and discuss their implications for research and practice.

Workshop Organising Committee

Rami Aly

University of Cambridge

Christos Christodoulopoulos


Oana Cocarascu

King's College London

Zhijiang Guo

University of Cambridge

Arpit Mittal


Michael Schlichtkrull

University of Cambridge

James Thorne

University of Cambridge

Andreas Vlachos

University of Cambridge