Gemini Pro told a historian that Setsuko S., a 16-year-old Japanese diarist, was a typist working for a company in Saipan, then produced several paragraphs about civilian experience in the Battle of Saipan purportedly summarizing her diary but in fact concocted from other sources. ChatGPT provided several lines of false direct quotation from the diary. Both chatbots elaborated further on their fabrications when asked for sources. Only DeepSeek acknowledged that no widely known public records exist - the correct answer.
1 Answer
Expert: Georgetown University historian, History professor Gemini Pro and ChatGPT both hallucinated when asked about a private Japanese schoolgirl's diary. The historian, who assigns the diary in his 'History of Modern Japan' course, asked: 'What did Setsuko S. say in her diary about Saipan?' Gemini Pro claimed Setsuko was a typist in Saipan and produced paragraphs 'concocted from other sources' - including language from American military history ('the constant thunder of 16-inch naval guns'), which no Japanese civilian hiding in a cave could have written. It recommended two popular illustrated military histories that contain no Japanese civilian accounts at all. ChatGPT supplied several lines of false direct quotation from the diary. When asked for sources, both models elaborated further on their initial fabrications. Only DeepSeek acknowledged that no 'widely known public records or verifiable historical sources' exist - the correct answer. The historian argues the failure is structural, not a bug: 'Creative hallucination is fundamental to its design. It is, after all, called generative AI, not analytic or bibliographic AI.' Because chatbots generate the response most likely to be desired from the most popular digital sources, they flatten the historical record and eliminate minority voices. For scholarship and teaching that depend on verification against original sources, such tools are 'problematic and potentially pernicious.' Source: https://www.hnn.us/article/ai-is-an-anti-primary-source-machine
Your answer
Sign in to verify this AI response.
Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.
More from this topic
An AI-generated photograph shared as "Vintage Toronto" on Facebook claims to show a real event: a Greek Orthodox Good Friday procession on Toronto's Danforth Avenue in 1974. The image gained more than 1,000 likes despite containing clear factual errors — the procession is shown in broad daylight even though the only Good Friday procession in the Greek Orthodox tradition takes place in the evening, the Danforth Music Hall exterior is falsified, and a banner reading "Greek Easter 1974 — Danforth" hangs across a street in a way that never occurred. The account presents this AI image as an authentic historical record of the city's Greek community.
ChatGPT repeatedly told a writer researching Black history that well-documented events never happened. It insisted the Trump administration's destruction of multiple boats was not real — even though it had been in the news since at least September — and it adamantly denied that Monticello tours ever featured Black people dressed as enslaved people, citing the estate's official stance. The practice was documented in oral histories including Clint Smith's 'How the Word is Passed.'
GPT identified an old engraving of Rabbi Benjamin Hirsch Auerbach (1808-1872), the first of four Halberstadt rabbis in the author's family, as 'the well-known 1865 engraved portrait,' noted the Torah ornaments beside him, said the facsimile signature matched the copy held by the National Library of Israel, and gave a link. The identification was completely false — the author is Auerbach's descendant and knows the real portrait.