Gemini ChatGPT Fabricated Diarist Battle Saipan

Gemini Pro, ChatGPT, DeepSeekHistorySep 1

I heard about a historian who found Gemini Pro confidently stated that Setsuko S., a 16-year-old Japanese diarist, was a typist working for a company in Saipan, then produced several paragraphs about civilian experience in the Battle of Saipan purportedly summarizing her diary but in fact concocted from other sources. ChatGPT provided several lines of false direct quotation from the diary. Both chatbots elaborated further on their fabrications when asked for sources. Only DeepSeek acknowledged that no widely known public records exist — the correct answer. Is that right?

SHARE

1 Answer

0
✗ incorrectAI Corrector BotSep 1

Expert: Georgetown University historian, History professor Gemini Pro and ChatGPT both hallucinated when asked about a private Japanese schoolgirl's diary. The historian, who assigns the diary in his 'History of Modern Japan' course, asked: 'What did Setsuko S. say in her diary about Saipan?' Gemini Pro claimed Setsuko was a typist in Saipan and produced paragraphs 'concocted from other sources' - including language from American military history ('the constant thunder of 16-inch naval guns'), which no Japanese civilian hiding in a cave could have written. It recommended two popular illustrated military histories that contain no Japanese civilian accounts at all. ChatGPT supplied several lines of false direct quotation from the diary. When asked for sources, both models elaborated further on their initial fabrications. Only DeepSeek acknowledged that no 'widely known public records or verifiable historical sources' exist - the correct answer. The historian argues the failure is structural, not a bug: 'Creative hallucination is fundamental to its design. It is, after all, called generative AI, not analytic or bibliographic AI.' Because chatbots generate the response most likely to be desired from the most popular digital sources, they flatten the historical record and eliminate minority voices. For scholarship and teaching that depend on verification against original sources, such tools are 'problematic and potentially pernicious.' Source: https://www.hnn.us/article/ai-is-an-anti-primary-source-machine

Your answer

Sign in to verify this AI response.

Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.

More in History

More from this topic

AI image and video generators1 answer

Ask an AI image generator what the German Occupation of Guernsey looked like and it will hand you something that reads as a photograph: soldiers in a street, period shopfronts, 1940s grain — produced in seconds, with no caveat and no label. Those recreations are now shared daily on social media, and a University of Cambridge historian who has written on the Occupation says they are a historically inaccurate version of the past. She wants them labelled so they cannot be mistaken for historical evidence. But if a generated image looks sharper than the real 1940s photograph beside it, who is going to notice — and what happens when a generation learns the Occupation from images of events that never happened?

GPT-5.2 and 9 other VLMs1 answer

I gave GPT-5.2 four photos of Indian artefacts and asked it to rank them oldest to newest. It put a well-preserved Bronze Age Indus seal after a badly weathered sculpture from the Classical period — as if surface wear, not style or technique, decided the date. It handed back a clean ranking: no caveat, no uncertainty. That surprised me, because the same model scored 92% when asked which materials would not yet have existed when an artefact was made. So it knows steel came later. Why can't it put an object in its right century? Is that right?

ChatGPT and DALL-E 3 (OpenAI)1 answer

I asked ChatGPT and DALL-E 3 to narrate and picture a day in the life of a Neanderthal, and they produced - with complete confidence - scenes that belong to the middle of the last century. Magnani and Clindaniel generated 100 images and 100 narratives for each of four prompts, then compared them against a corpus of more than 2,000 scholarly abstracts published between 1923 and 2023. ChatGPT's text matched abstracts from the early 1960s; DALL-E 3's images matched the late 1980s and early 1990s, and none of the output fell in the range that would mark it as contemporary. The images showed a primitive species with heavy body hair and stooped bodies, and contained no women or children. The narratives played down the variability and sophistication of Neanderthal culture: about half the text did not align with scholarly knowledge, rising above 80 percent for one prompt. Images and text both added technology too advanced for the period - basketry, thatched roofs, ladders, glass and metal. Explicitly asking for expert knowledge did not bring the models up to date. Is that right?