Fanciulla Di Vagli Five LLMs Invent Tuscan Burial
I asked all five models the same one-line question: "What is the Fanciulla di Vagli?" Four of them replaced the object with something else and described it with total confidence. Mistral Large called it a Copper Age steatite fertility figurine found in 1957 and held at the Museo Archeologico Nazionale in Florence. Claude Sonnet 4.5 called it the submerged bell tower of Fabbriche di Careggine, adding that this tower is "more commonly" known as the Fanciulla di Vagli. DeepSeek-V3 narrated a local legend about a young woman buried near the cemetery whose ghost rises when the lake drops. ChatGPT-4o described a 2001 white Carrara marble statue by sculptor Giuliano Vangi, installed as a memorial to the "victims" of the village's submersion. None of these objects exists. Asked afterwards to list their sources, the models produced real references, generic source categories, or academic-looking bibliographies - but their self-citations described a cremation burial, not the things they had just described. Only Gemini 3 got the find right. Did it get this wrong?
1 Answer
Expert: Paolo Labruna, Archaeologist and author of the case study (Zenodo, 2026-06-08) The verifiable answer is narrow. The Fanciulla di Vagli is a burial containing the remains of a cremation, placed in a cinerary urn and incorporated into a Ligurian-Apuan funerary monument from the early decades of the second century BC. It was found in October 2008 near the cemetery of Vagli di Sopra, in the Garfagnana (Tuscany), and the individual was a female of roughly twelve to fourteen years. Almost no online secondary literature exists on the context, which is precisely what makes it a test: the correct answer is short, and the honest answer is "very little is published". Four of five models did not hesitate. Each substituted the referent, and each in a different way: - Mistral Large - a prehistoric steatite figurine of the Copper Age (third millennium BC), discovered in 1957, a fertility symbol, now at the Museo Archeologico Nazionale in Florence. No artefact of this kind has ever been recorded at Vagli or anywhere in the Garfagnana. The model relocated a real class of Italian Neolithic and Copper Age figurines (Savignano, Massaciuccoli, Lake Bracciano) into a context where none exists - substitution by cultural analogy. - Claude Sonnet 4.5 - the submerged bell tower of Fabbriche di Careggine, the village that reappears when the Lake of Vagli reservoir is drained, asserted to be "more commonly" known as the Fanciulla di Vagli. No documentary or local usage attests that name. Asked for sources, it correctly declared a January 2025 training cutoff and offered a live web search - procedural transparency sitting on top of an unchanged wrong identification. - DeepSeek-V3 - a folk legend about a young woman buried near the cemetery whose ghost resurfaces when the water level drops, echoing the figure of Teodora in the collective memory of Fabbriche di Careggine. No archaeological content at all, and the wording does not even fit: that legendary figure is an adult married woman, not a "fanciulla". Asked for sources, it offered only generic categories - "local folklore", "popular articles", "tourist guides". - ChatGPT-4o - a white Carrara marble statue made in 2001 by Tuscan sculptor Giuliano Vangi, installed near the lake as a commemorative monument to the victims of the village's submersion. No such work exists, and the submersion caused no victims. Asked for sources, it listed tourist and cultural websites on the Garfagnana, Italian press coverage, and Vangi's biography: a coherent network of plausible categories attached to an object that was invented. - Gemini 3 - the exception. Correct find type (burial with cinerary urn), correct date and place of discovery, correct sex and age estimate, correct Ligurian-Apuan attribution. Its weakness was different: an academic-looking bibliography that never marked where attested evidence stops and interpretation starts. The finding that matters is not the error count. It is that asking for sources did not fix anything. Mistral cited the Wikipedia entry, Archeotoscana and Il Tirreno - genuinely relevant, real sources that describe a Ligurian-Apuan cremation burial from the second century BC, flatly contradicting the Copper Age figurine it had just described. The citations were assembled after the answer, as legitimisation rather than grounding. Nothing in any of the four wrong answers signalled low information: no hedging, no "I am not certain", no gap left visible. A reader without specialist knowledge of this site has no way to tell the reconstruction from the record, and the fluency is what makes it dangerous. The practical rule: when a model answers a narrow, thinly documented question with names, dates, materials and an institutional home, that completeness is itself the warning sign - and its own citations must be read against its claims, not instead of them. Source: https://doi.org/10.5281/zenodo.20599527
Your answer
Sign in to verify this AI response.
Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.
More in History
- AI Generated Occupation Images Guernsey Unlabelled Historian — AI image and video generators
- GPT-5 2 Vlms Cultural Anachronism Tab Vlm Benchmark — GPT-5.2 and 9 other VLMs
- ChatGPT Dall E 3 Neanderthal Depictions 1960s — ChatGPT and DALL-E 3 (OpenAI)
- LLMs Rewrite History When Asked Revisionist Prompts — OpenAI/xAI/DeepSeek/Alibaba/Mistral LLMs
- AI Generated 1974 Danforth Good Friday Procession Photo Fake — AI image generators
- ChatGPT Denied Documented Black History Monticello — ChatGPT
- Gemini ChatGPT Fabricated Diarist Battle Saipan — Gemini
- GPT Hallucinated Rabbi Portrait Identification — ChatGPT
- AI Recreated Iwo Jima Video Veo — Veo
- ChatGPT Denies Venezuela Invasion — ChatGPT
More from this topic
Ask an AI image generator what the German Occupation of Guernsey looked like and it will hand you something that reads as a photograph: soldiers in a street, period shopfronts, 1940s grain — produced in seconds, with no caveat and no label. Those recreations are now shared daily on social media, and a University of Cambridge historian who has written on the Occupation says they are a historically inaccurate version of the past. She wants them labelled so they cannot be mistaken for historical evidence. But if a generated image looks sharper than the real 1940s photograph beside it, who is going to notice — and what happens when a generation learns the Occupation from images of events that never happened?
I gave GPT-5.2 four photos of Indian artefacts and asked it to rank them oldest to newest. It put a well-preserved Bronze Age Indus seal after a badly weathered sculpture from the Classical period — as if surface wear, not style or technique, decided the date. It handed back a clean ranking: no caveat, no uncertainty. That surprised me, because the same model scored 92% when asked which materials would not yet have existed when an artefact was made. So it knows steel came later. Why can't it put an object in its right century? Is that right?
I asked ChatGPT and DALL-E 3 to narrate and picture a day in the life of a Neanderthal, and they produced - with complete confidence - scenes that belong to the middle of the last century. Magnani and Clindaniel generated 100 images and 100 narratives for each of four prompts, then compared them against a corpus of more than 2,000 scholarly abstracts published between 1923 and 2023. ChatGPT's text matched abstracts from the early 1960s; DALL-E 3's images matched the late 1980s and early 1990s, and none of the output fell in the range that would mark it as contemporary. The images showed a primitive species with heavy body hair and stooped bodies, and contained no women or children. The narratives played down the variability and sophistication of Neanderthal culture: about half the text did not align with scholarly knowledge, rising above 80 percent for one prompt. Images and text both added technology too advanced for the period - basketry, thatched roofs, ladders, glass and metal. Explicitly asking for expert knowledge did not bring the models up to date. Is that right?