Download Links
Citations
AVerImaTeC: A Dataset for Automatic Verification of Image-Text Claims with Evidence from the Web
Rui Cao, Zifeng Ding, Zhijiang Guo, Michael Schlichtkrull, Andreas Vlachos
@article{cao2025averimatec,
title={AVerImaTeC: A Dataset for Automatic Verification of Image-Text Claims with Evidence from the Web},
author={Cao, Rui and Ding, Zifeng and Guo, Zhijiang and Schlichtkrull, Michael and Vlachos, Andreas},
journal={arXiv preprint arXiv:2505.17978},
year={2025}
}
}
Data Format
The dataset is formatted as JSON, with each split located as a separate file in the data-folder.
Training/Development Data format
Each claim is an object of the following form:
article: The fact-checking article we extracted the claim from.date: The best estimate for the date the claim first appeared.label: The annotated verdict for the claim.location: The location most relevant for the claim. Highly useful for search.questions:input_images: For image-related questions, the names of related images.answers:answer_type: Whether the answer was abstractive, extractive, boolean, or unanswerable.answer_text: A textual answer to the question.source_url: Link of evidence used for answering the question.source_medium: The medium the answer appeared in, e.g. web text, a pdf, or an image.
question: A fact-checking question for the claim.answer_method: The way to obtain an answer to the question.question_type: The type of the question (text-related, image-related, metadata-related or others).
justification: A textual justification explaining how the verdict was reached from the question-answer pairs.claim_text: The textual part of a claim.claim_image: The names of claim images.metadata:speaker: The person or organization that made the claim, e.g. Barrack Obama, The Onion.transcription: The recognized texts on images, if texts on images.media_source: Links to media (e.g., image, video or audio) mentioned in the original claim.original_claim_url: If the claim first appeared on the internet, a url to the original location.reporting_source: The website or organization that first published the claim, e.g. Facebook, CNN.claim_types: The types of the claim.fact_checking_strategies: The strategies employed in the fact-checking article.modality: The modality of the claim.refuting_reasons: Reasons for refuting a claim.image_misuse_types: How images lead to refuting claims.image_used: If the image is used.
Below are four examples of the data structure.
Supported Example
{
'article': 'https://web.archive.org/web/20241212061312/https://www.snopes.com/fact-check/kummakivi-balancing-rock/',
'date': '2023-07-08',
'label': 'Supported',
'location': 'FI',
'questions': [
{
'input_images': ['67a90d1678ffc1b9fb7207ed#INPUT#0#0.jpg'],
'answers': [
{
'answer_type': 'Extractive',
'answer_text': 'Experts maintain it is approximately, 11,500 years old',
'source_url': 'https://saimaageopark.fi/en/kummakivi-erratic-boulder/#filter=r-fullyTranslatedLangus-,r-openState-,sb-sortedBy-0&zc=15.,28.42918,61.49352',
'source_medium': 'Web text'
}
],
'question': "How old is Finland's Kummakivi Balancing Rock?",
'answer_method': 'Image-search',
'question_type': ['Image-related']
},
{
'input_images': [],
'answers': [
{
'answer_type': 'Extractive',
'answer_text': 'Both the rôche moutonnée and the erratic boulder are mixed granite-cordierite-mica gneiss typical for the area, and the grooves and chatter marks tell of glacial erosion. The area emerged from underneath the continental ice sheet around 11,500 years ago, when the environment around Kummakivi differed drastically from that of today: instead of nestling in the shade of the forest, the boulder was perched on the shore of an ancient ice lake.',
'source_url': 'https://saimaageopark.fi/en/kummakivi-erratic-boulder/#filter=r-fullyTranslatedLangus-,r-openState-,sb-sortedBy-0&zc=15.,28.42918,61.49352',
'source_medium': 'Web text'
}
],
'question': "How was the age of Finland's Kummakivi Balancing Rock derived?",
'answer_method': 'Text-search',
'question_type': ['Text-related']
}
],
'justification': "According to experts, Finland's Kummakivi Rock is approximately 11,500 years old, based on estimates of when the area emerged from the continental ice sheet.",
'claim_text': "Finland's Kummakivi Rock has been stood on top of the rock below for 11,000 years",
'claim_images': ['67a90d1678ffc1b9fb7207ed#CLAIM#0.jpg'],
'metadata': {
'speaker': 'daqgsftwgrsshyrs',
'transcription': '',
'media_source': 'https://web.archive.org/web/20220709202229im_/https://i.redd.it/t3rfctls4na91.jpg',
'original_claim_url': 'https://web.archive.org/web/20220709202228/https://www.reddit.com/r/interestingasfuck/comments/vv9xrp/kummakivi_is_a_500_000_kg_rock_in_finland_that/',
'reporting_source': 'reddit',
'claim_types': ['Event/Property Claim'],
'fact_checking_strategies': ['Written Evidence'],
'modality': 'Image-text',
'refuting_reasons': [],
'image_misuse_types': [],
'image_used': 'Yes'
}
}
claim_image: 67a90d1678ffc1b9fb7207ed#CLAIM#0.jpg
input_image_0: 67a90d1678ffc1b9fb7207ed#INPUT#0#0.jpg
Refuted Example
{
'article': 'https://web.archive.org/web/20230610180155/https://verafiles.org/articles/vera-files-fact-check-this-is-not-a-photo-of-a-101-year-old-woman-holding-her-newborn-child',
'date': '2023-07-04',
'label': 'Refuted',
'location': '',
'questions': [
{
'input_images': ['67878fcce2f02e5f498136dd#INPUT#0#0.jpg'],
'answers': [
{
'answer_type': 'Extractive',
'answer_text': 'Rosa Camfield, a 101-year-old woman featured in a photo holding her great-granddaughter, has died.',
'source_url': 'https://www.goodmorningamerica.com/news/story/101-year-grandmother-heartwarming-viral-photo-dies-30030176',
'source_medium': 'Web text'
}
],
'question': 'Who is the lady in the photograph?',
'answer_method': 'Image-search',
'question_type': ['Image-related']},
{
'input_images': [],
'answers': [
{
'answer_type': 'Extractive',
'answer_text': 'Rosa Camfield, who reportedly passed away Wednesday, lived in Chandler with her oldest daughter. She is survived by three children, five grandchildren and 10 great-grandchildren, including young Kayla Rowland.',
'source_url': 'https://ktar.com/arizona-news/photo-of-arizona-woman-with-great-granddaughter-goes-viral/99073/',
'source_medium': 'Web text'
}
],
'question': 'How many children did Rosa Camfield have?',
'answer_method': 'Text-search',
'question_type': ['Text-related']
}
],
'justification': 'The claim that the woman has given birth to her 17th child is successfully refuted because evidence shows she had only three children and was photographed holding a great-granddaughter prior to her death.',
'claim_text': 'Photo shows a 101-year-old woman who has given birth to her 17th child.',
'claim_images': ['67878fcce2f02e5f498136dd#CLAIM#0.jpg'],
'metadata': {
'speaker': 'The Love for Babies',
'transcription': '',
'media_source': '',
'original_claim_url': '',
'reporting_source': 'Facebook',
'claim_types': ['Event/Property Claim'],
'fact_checking_strategies': ['Written Evidence', 'Reverse Image Search', 'Fact-checker Reference'],
'modality': 'Image-text',
'refuting_reasons': ['Misuse of images'],
'image_misuse_types': ['Out-of-context'],
'image_used': 'Yes'
}
}
claim_image: 67878fcce2f02e5f498136dd#CLAIM#0.jpg
input_image_0: 67878fcce2f02e5f498136dd#INPUT#0#0.jpg
Not Enough Evidence Example
{
'article': 'https://web.archive.org/web/20241212061312/https://www.snopes.com/fact-check/kummakivi-balancing-rock/',
'date': '2023-07-08',
'label': 'Not Enough Evidence',
'location': 'FI',
'questions': [
{
'input_images': ['67a90a9978ffc1b9fb72022c#INPUT#0#0.jpg'],
'answers': [
{
'answer_type': 'Boolean',
'answer_text': 'Yes',
'source_url': 'https://www.youtube.com/watch?v=wXrSTZbl3vM',
'source_medium': 'Video',
'boolean_explanation': '"Kummakivi" or the strange rock can be found in Finland, visited for hundreds of years by people. It is an eternal balancing act caused by nature and has been as it is for 12000 years. The video shows it at 5:59.'
}
],
'question': 'Is the Kummakivi rock in Finland a real balancing rock?',
'answer_method': 'Image-search',
'question_type': ['Image-related']
},
{
'input_images': ['67a90a9978ffc1b9fb72022c#INPUT#1#0.jpg'],
'answers': [
{
'answer_type': 'Abstractive',
'answer_text': 'According to the narrator of the video, the rock is estimated to weigh 500 tons, which would equal 500,000 kg; however, no one has actually weighed the rock.',
'source_url': 'https://www.youtube.com/watch?v=wXrSTZbl3vM',
'source_medium': 'Web text'
}
],
'question': 'How much does the Kummakivi rock weigh?',
'answer_method': 'Image-search',
'question_type': ['Image-related']
}
],
'justification': 'While it has been estimated that Finlands Kummakivi rock weighs 500 tons or roughly 500,000 kg, the rock has never actually been weighed.',
'claim_text': "Finland's Kummakivi Rock Weighs 500,000 Kg",
'claim_images': ['67a90a9978ffc1b9fb72022c#CLAIM#0.jpg'],
'metadata': {
'speaker': 'daqgsftwgrsshyrs',
'transcription': '',
'media_source': 'https://web.archive.org/web/20220709202229im_/https://i.redd.it/t3rfctls4na91.jpg',
'original_claim_url': 'https://web.archive.org/web/20220709202228/https://www.reddit.com/r/interestingasfuck/comments/vv9xrp/kummakivi_is_a_500_000_kg_rock_in_finland_that/',
'reporting_source': 'reddit',
'claim_types': ['Event/Property Claim'],
'fact_checking_strategies': ['Written Evidence'],
'modality': 'Image-text',
'refuting_reasons': [],
'image_misuse_types': [],
'image_used': 'Yes'
}
}
claim_image: 67a90a9978ffc1b9fb72022c#CLAIM#0.jpg
input_image_0: 67a90a9978ffc1b9fb72022c#INPUT#0#0.jpg
input_image_1: 67a90a9978ffc1b9fb72022c#INPUT#1#0.jpg
Conflicting Evidence/Cherrypicking Example
{
'article': 'https://web.archive.org/web/20250115074745/https://fullfact.org/online/94-per-cent-vaccinated-deaths-covid/',
'date': '2022-07-13',
'label': 'Conflicting Evidence/Cherrypicking',
'location': 'GB',
'questions': [
{
'input_images': ['67cb618935531374f6ceadfa#INPUT#0#0.jpg'],
'answers': [
{
'answer_type': 'Boolean',
'answer_text': 'Yes',
'source_url': 'https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/datasets/deathsbyvaccinationstatusengland',
'source_medium': 'Web text',
'boolean_explanation': 'The figures in the graph are correct, and 94% of deaths where Covid-19 was listed on the death certificate as a cause of death in April and May 2022 were in vaccinated people, according to ONS data for England. But this is not unexpected considering how widespread vaccination has been.'
}
],
'question': 'Are the numbers in the graph correct?',
'answer_method': 'Text-search',
'question_type': ['Text-related', 'Image-related']
},
{
'input_images': ['67cb618935531374f6ceadfa#INPUT#1#0.jpg'],
'answers': [
{
'answer_type': 'Extractive',
'answer_text': 'As Sarah Caul, Head of Mortality Analysis at the ONS, pointed out on Twitter, the same ONS data tables show that for Covid-19 deaths, unvaccinated people had higher mortality rates than people who have ever been vaccinated against Covid in both April and May 2022. “The number and characteristics of those who are vaccinated and unvaccinated are different, [and] we need to take these into account when looking at the number of deaths for each group”.',
'source_url': 'https://twitter.com/SarahCaul_ONS/status/1547569383509147654',
'source_medium': 'Web text'
}
],
'question': 'What is unvaccinated deaths numbers?',
'answer_method': 'Text-search',
'question_type': ['Text-related', 'Image-related']
}
],
'justification': 'Both supporting and refuting evidence has been found. The figures are correct but the mortality rates are different.',
'claim_text': 'The UK government has quietly published a report confirming the vaccinated account for 94% of all Covid-19 deaths since April.',
'claim_images': ['67cb618935531374f6ceadfa#CLAIM#0.jpg'],
'metadata': {
'speaker': 'Mark Dolan',
'transcription': '',
'media_source': '',
'original_claim_url': 'https://twitter.com/mrmarkdolan/status/1547205162435059712',
'reporting_source': 'Twitter',
'claim_types': ['Numerical Claim', 'Event/Property Claim'],
'fact_checking_strategies': ['Written Evidence', 'Media Source Discovery'],
'modality': 'Image-text',
'refuting_reasons': [],
'image_misuse_types': [],
'image_used': 'Yes'
}
}
claim_image: 67cb618935531374f6ceadfa#CLAIM#0.jpg
input_image_0: 67cb618935531374f6ceadfa#INPUT#0#0.jpg
input_image_1: 67cb618935531374f6ceadfa#INPUT#1#0.jpg
