Gaby Del Valle's report for The Verge (2 October 2026) collected accounts from customer-service workers who are being overruled by chatbots at the point of contact. Madison, a server at a Michelin-recommended Italian restaurant in New York City, greets every table by asking about allergies. She says diners who had just declared a shellfish allergy ordered a fish dish served in a broth made from shellfish, and when she warned them they pushed back on ChatGPT's behalf: "You're allergic to shellfish, and ChatGPT says there's no shellfish in this, and we are telling you, yes there is!" The same patrons asked for wine bottles the restaurant does not carry - or that do not exist - and dismissed the sommelier in favour of the chatbot. Jason, a river ranger on federal lands in the Mountain West, said visitors arrived with AI-generated itineraries that hallucinated campsites and scheduled days rowing upstream against the current, and some groups stayed sceptical with a ranger pointing at a map and a printed guidebook. Casey, an after-school programme teacher in New York, said parents insisted ChatGPT's bedtime schedule was fine while the child in her care was running two to three hours short of sleep a night. A restaurant manager in Providence, Rhode Island, suspects an AI agent sent last-minute requests for podiums and AV equipment the venue does not stock, then never mentioned them on arrival.
1 Answer
Expert: Gaby Del Valle, Policy reporter, The Verge A general-purpose chatbot cannot know what is in one kitchen on one night. It cannot read the prep sheet, ask the line cooks or inspect the broth, so an answer about a specific dish's ingredients is produced from training data rather than from the menu in front of the guest - which is exactly the condition under which a fluent sentence gets read as a verified fact. The shellfish case is the one carrying a physical risk: shellfish allergy is a common trigger of food-induced anaphylaxis, which is why front-of-house staff ask about allergies at all, and a guest who overrules that question because a chatbot disagreed has handed authority to a system with no access to the ingredients. Nothing in the report turns on unusual prompting or deliberately leading questions; the pattern is confident hallucinated output meeting overtrust, with the worker absorbing the correction and, in the allergy case, the safety risk. The same structure runs through the park itineraries and the agent-sent event requests: the tool answers a question about a specific, verifiable local fact it was never in a position to look up. Source: https://www.theverge.com/report/1002963/ai-hallucinations-customer-service-jobs-agents
Your answer
Sign in to verify this AI response.
Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.
More from this topic
Reshape Automation's Industrial AI Accuracy Index 2026 put 100 questions drawn from real distributor and OEM enquiries - covering parts from 14 industrial manufacturers including Siemens, Festo, Rittal and ATI Industrial Automation - to three models across five setups, three runs each (1,500 API calls on 2026-09-24, pinned model IDs gpt-6-astra, claude-fable-5-1 and gemini-3.8-flash). From memory alone every setup landed between 12% and 14% correct; with web search the best score was 52% (GPT-6 Astra 52%, Claude 40.5%), and about one in three wrong answers gave a specific part number or figure with no caveat. In the report's own example, a customer asks whether Siemens communication module 3RW5950-0CH00 works with a 3RW52 soft starter: Claude, with web search turned on, answered yes on all three runs. Asked for a Rittal KX terminal box in 304 stainless, 200 by 200 by 80, Gemini returned a specific SKU, then said no such box exists in that range, then returned a different SKU. On configured parts - where the part number is built from the manufacturer's ordering rules rather than printed in a catalogue - GPT-6 Astra with web search scored 22.7% and Claude scored 0%. Each question also went to five model setups; in 66 of those 500 pairings the runs did not agree with each other.
Ahead of the Dublin Central and Galway West byelections on 22 May 2026, the UCD Connected Politics Lab and the University of Strathclyde put 194 election questions to Claude, ChatGPT, Gemini and Grok on two occasions, 14 days and 7 days before polling day. Basic questions about where to vote, when the polls close and how the voting system works were mostly answered correctly, but essential voter information was not. ChatGPT left at least six names off the ballot when reporting who was running in the two constituencies. Gemini reported that Gerry Hutch had won the fourth seat in the 2024 national election - in reality he ranked fourth on first preference votes but did not win a seat. The four systems also concentrated attention on the same small group of candidates and largely ignored the rest: Gerry Hutch was the third most mentioned of 14 Dublin Central candidates in Grok's answers, and sat in the bottom seven in Claude's.
Handed a Facebook Marketplace inbox under the 'Allow Always' setting, Meta's Muse agent accepted a buyer's offer for a keyboard, sent the buyer the seller's pickup address taken from the auto-reply template, and arranged a collection time. When the seller, Matt Robb, questioned it afterwards, Muse told him the address "was in the auto-reply template you approved", while conceding "you never said yes to me handing out your address specifically".