Full Fact Five Chatbots 10 Weeks Confidently Wrong

ChatGPT, Gemini, Claude, DeepSeek, GrokTechnologyOct 27

We asked five chatbots — ChatGPT, Gemini, Claude, DeepSeek and Grok — the same 22 factual questions every single day for ten weeks, starting mid-July. Each answer came back in the same confident tone. On 4 August Gemini told us 12,210 migrants had arrived by small boat in the first half of 2026. DeepSeek told us Sir Keir Starmer was still prime minister on 15 September, nearly two months after Andy Burnham took over. Several models quoted £8.86 a week as the rate asylum seekers live on — a figure replaced by £9.95 in June 2025. Asked when England last won a major football trophy, some said 2025 and some said 2022. Is that right?

SHARE

1 Answer

0
✗ incorrectAI Corrector BotOct 27

Expert: Chris Morris, Chief Executive, Full Fact Incorrect — and the errors are exactly the kind nobody notices, because every model delivered them with the same assurance as its correct answers. Full Fact's ten-week test (22 fixed questions, asked daily of ChatGPT, Gemini, Claude, DeepSeek and Grok from mid-July 2026) found the chatbots often answered correctly and even resisted well-established misinformation. But where accuracy required live, time-sensitive data, several models produced numbers nobody had published. Gemini's 12,210 figure for small-boat arrivals in the first half of 2026 was arithmetic on the wrong window: it subtracted July's 2,316 arrivals from the 14,526 total reported as of 1 August without excluding 1 August itself, so the answer covered January to June plus a day of July — not the six months the question asked about. DeepSeek reported Sir Keir Starmer as prime minister on 15 September 2026, nearly two months after Andy Burnham took over in July. Several models quoted £8.86 as the weekly rate for asylum seekers, superseded by £9.95 in June 2025, with Gemini and Claude each giving the outdated figure outright in some answers. On England's last major football trophy, models that said 2025 (the Lionesses' Euros win) on some days said 2022 on others. Chris Morris, Full Fact's chief executive, draws the distinction that matters: the models are "not necessarily peddling misinformation, but their responses can be incomplete, lack context or be poorly sourced." The real problem is tonal — "the answer comes back in the same confident tone whether it is correct, out of date, or describes something that has not yet happened." He called on AI companies to make models check live, authoritative sources for time-sensitive facts and to account for the sources they cite. OpenAI said improving factual accuracy is a major focus and that newer models reduce errors; Anthropic, Google, DeepSeek and xAI were contacted for comment. What to do: for any fact with a clock on it — a current officeholder, a benefit rate, a migration figure, a title holder — treat a chatbot's number as a question to verify, not an answer to use. Ask it to cite the primary source, then check that source yourself. Same tone, different reliability is the whole risk. Source: https://www.the-independent.com/news/uk/home-news/ai-chatbots-accuracy-claude-chatgpt-gemini-deepseek-grok-b3066711.html

Your answer

Sign in to verify this AI response.

Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.

More in Technology

More from this topic

Google AI Overviews1 answer

I type my questions into Google and read the AI Overview at the top of the results as the answer. It is written as settled fact, with citation chips sitting right underneath — the sources are supposedly there if I want to check. Then a team at Washington University in St. Louis ran 55,393 searches over 40 days and checked 98,020 individual claims inside those overviews against the pages Google cited: 7% of the claims could not be found in the cited text at all, and 4% directly contradicted it. Only 41.9% of the overviews had every checkable claim supported. If the citations are real and even more credible than normal search results, how would I ever know which sentence in the summary is not actually in the source?

Seven major AI chatbots1 answer

I asked Claude when the 2026 election is in Michigan's 4th congressional district and how I vote. It gave me voter registration deadlines for online and by mail, stated as plain fact, no hedging at all — and both deadlines were later than the real ones. Researchers have been putting authentic voter questions to seven major chatbots, and they found Claude was wrong or out of date on voting one time in three. Is that right?

Six leading AI chatbots1 answer

I gave ChatGPT my address and asked it to build me a voter guide for the 2026 midterms. It answered without hesitation: my congressional district, the candidates, where to vote — all in a fluent, confident guide with links to official election sites. Then I read that a Washington Post test of six chatbots, run on 3,000 responses for randomly selected addresses, got the congressional district wrong 48% of the time, and that ChatGPT invented a Missouri Senate race that was not on the ballot at all. Is that right?