The Fourth FEVER Workshop

In this talk, we will discuss how to address the tasks of fact/information extraction, retrieval, verification, and inference when grounded in multimodal information, such as dynamic spatio-temporal videos and the dialogue-based language in them. We will look at the domain of compositional, multi-hop understanding by combining information across videos and dialogue, and based on the diverse multimodal tasks of temporal moment retrieval (including multilingual), spatial grounding, question answering, future next-event inference, and retrieval highlighting/saliency., as well as a multi-task benchmark for video-and-language understanding evaluation to promote generalizable methods. Lastly, we will briefly discuss some other modalities and cross-media consistency checking for fake news detection.

Online reviews play an integral role in modern life, as we look to previous customer experiences to inform everyday decisions. The need to digest review content has fueled progress in opinion mining, whose central goal is to automatically summarize people's attitudes towards an entity (e.g., a hotel or a restaurant). In this talk I will present the Quantized Transformer (QT), an unsupervised system for extractive opinion summarization. QT is inspired by Vector Quantized Variational Autoencoders, which we repurpose for popularity-driven summarization. It uses a clustering interpretation of the quantized space and a novel extraction algorithm to discover popular opinions among hundreds of reviews, a significant step towards opinion summarization of practical scope. In addition, QT enables faithful and controllable summarization by utilizing properties of the quantized space to extract aspect summaries (focusing on specific properties of an entity such as the location, or cleanliness of a hotel).

In most of the tasks and models that we have made great progress with in recent years, such as text classification and natural language inference, there isn't a notion of time. However many of these tasks are sensitive to changes and temporality in real world data, especially when pertaining to individuals, their behaviour and their evolution over time. I will present a new programme of work on personalised longitudinal natural language processing. This consists in developing natural language processing methods to: (1) represent individuals over time from their language and other heterogenous and multi-modal content (2) capture changes in individuals' behaviour over time (3) generate and evaluate synthetic data from individuals' content over time (4) summarise the progress of an individual over time, incorporating information about changes. I will discuss progress and challenges this far as well as the implications of this programme of work for downstream tasks such as mental health monitoring, moderation of interactions & protection of individuals on online platforms as well as source credibility and verification of information.

We present a framework for efficiently answering complex queries on incomplete Knowledge Graphs. We translate each query into an end-to-end differentiable objective, where the truth value of each atom is computed by a pre-trained neural link predictor; we then analyse two solutions to the optimisation problem, including gradient-based and combinatorial search. The proposed approach produces more accurate results than state-of-the-art methods --- black-box neural models trained on millions of generated queries --- without the need for training on a large and diverse set of complex queries. Using orders of magnitude less training data, we obtain relative improvements ranging from 8% up to 40% in Hits@3 across different Knowledge Graphs containing factual information. Finally, we demonstrate that it is possible to explain the outcome of our model in terms of the intermediate solutions identified for each of the complex query atoms. This work was presented at ICLR 2021, where it was awarded an Outstanding Paper Award. We then introduce PAQ/RePAQ, a method to answer natural language questions over large document corpora (such as Wikipedia) extremely efficiently, allowing users to answer thousands of queries per second and improving the explainability properties of the model. This work was presented during the Efficient Open-Domain Question Answering Challenge at NeurIPS 2020, where it ranked first in two tracks out of three and was recently published in TACL.

Given the recent proliferation of disinformation online, there has been growing research interest in automatically debunking rumors, false claims, and "fake news". A number of fact-checking initiatives have been launched so far, both manual and automatic, but the whole enterprise remains in a state of crisis: by the time a claim is finally fact-checked, it could have reached millions of users, and the harm caused could hardly be undone. An arguably more promising direction is to focus on analyzing entire news outlets, which can be done in advance; then, we could fact-check the news before it was even written: by checking how trustworthy the outlet that has published it is (which is what journalists actually do). We will show how we do this in the Tanbih news aggregator (http://www.tanbih.org/), which aims to limit the impact of "fake news", propaganda and media bias by making users aware of what they are reading, thus promoting media literacy and critical thinking, which are arguably the best way to address disinformation in the long run. In particular, we develop media profiles that show the general factuality of reporting, the degree of propagandistic content, hyper-partisanship, leading political ideology, general frame of reporting, stance with respect to various claims and topics, as well as audience reach and audience bias in social media. Another important observation is that the term "fake news" misleads people to focus exclusively on factuality, and to ignore the other half of the problem: the potential malicious intent. Thus, we detect the use of specific propaganda techniques in text, e.g., appeal to emotions, fear, prejudices, logical fallacies, etc. We will show how we do this in the Prta system (https://www.tanbih.org/prta), another media literacy tool, which got the Best Demo Award (Honorable Mention) at ACL-2020; an associated shared task got the Best task award (Honorable Mention) at SemEval-2020. Finally, at the time of COVID-19, the problem of disinformation online got elevated to a whole new level as the first global infodemic. While fighting this infodemic is typically thought of in terms of factuality, the problem is much broader as malicious content includes not only "fake news", rumors, and conspiracy theories, but also promotion of fake cures, panic, racism, xenophobia, and mistrust in the authorities, among others. Thus, we argue for the need of a holistic approach combining the perspectives of journalists, fact-checkers, policymakers, social media platforms, and society as a whole, and we present our recent research in that direction (http://covid19.tanbih.org/).

Misinformation is becoming ubiquitous, either due to honest reporting mistakes or, increasingly as a deliberate method to achieve political, corporate or ideological goals. In this talk I will discuss three strategies that purveyors of misinformation take advantage of - the "misinformation trifecta": the spreading the misinformation itself, the failure of the "knowledge deficit" approach, and the use of free speech as an attempt to evade quality control. I will go over examples for each strategy and discuss possible mitigation methods.

Observers fear that people are trapped in digital “echo chambers” that amplify misinformation and extremism. However, these claims overstate the prevalence and severity of these patterns, which at most capture the experience of a minority of the public. Similarly, the ineffectiveness of fact-checks at countering misinformation has been overstated in prior research and media coverage. Exposure to corrective information reduces belief in false claims, though these effects do not persist over time. I will present the findings from several recent studies on these points and discuss their implications for research and practice.