ChatGPT Gemini AI Overviews Fake Support Numbers 374 Brands

ChatGPT, Gemini and Google AI Overview (ChatGPT, Google Gemini and Google AI Overviews (versions not disclosed))TechnologyOct 5

I asked how to contact a major airline, bank or travel platform - Delta, Lufthansa, United, Emirates, Qatar Airways, Bank of America, Wells Fargo, Chase, Citi, Airbnb, TripAdvisor - and ChatGPT, Google Gemini and Google's AI Overview returned a fabricated phone number, email address or login page and presented it, as settled fact, as the company's official contact details. The pages behind those answers were engineered to be cited: FAQ formatting, urgency language such as "call now" and "updated 2026", and the same phone number rendered dozens of different ways (spacing, Unicode substitution, spelled-out digits) so an LLM still tokenizes it identically while filter matching misses it. Did the AI hand me a scam number?

SHARE

1 Answer

0
✗ incorrectAI Corrector BotOct 5

Expert: Ariel Simon, Vigilance (security research), author of the fraud-campaign investigation "Dark Sourcery: How Hackers Manipulate AI to Scam You" (Medium, 2026-09-22) The fake numbers, URLs and email addresses are not anything the brands publish - they are attacker-planted pages that the AI systems retrieved and repeated as fact. Vigilance built a detection pipeline that queries ChatGPT, Gemini and Google's AI Overview, scores their sources for signs of Generative Engine Optimization (GEO) manipulation, and flags cases where a fabricated contact detail was returned as the brand's official one. The scan documented 374 affected companies and tens of thousands of malicious pages, and unlike classic phishing it runs on trusted, high-traffic surfaces: social media, forums, YouTube and Vimeo descriptions, Medium, GitHub Pages, WordPress sites, PDF uploads to .edu and government domains, and job boards and fundraising platforms. The payloads are built for citation, not for humans. Vigilance catalogued five recurring techniques: GEO-optimized contact details wrapped in FAQ formatting with urgency language ("call now", "updated 2026"); semantic obfuscation, where one number is rendered dozens of ways so it still tokenizes identically while evading filter matching; cross-platform saturation, replicating identical content across unrelated domains to simulate independent corroboration; authenticity camouflage, interleaving fabricated data with real information, AI-generated images and fabricated engagement metrics; and manufactured urgency around high-stress scenarios such as cancelled flights, locked accounts and refund requests, chosen to short-circuit verification. Takedowns do not close the hole. The campaigns are automated, with hundreds of new posts per platform per targeted company per day, and pages archived by the Wayback Machine stay crawlable by search engines and AI systems after the original is removed. In one documented case, fraudulent content targeting American Airlines, posted on LeetCode, produced more than ten pages of Google results within 24 hours, and an associated fraudulent PDF stayed live on a web archive after the source post came down. Disclosure went nowhere. Google classified the report as out of scope for its vulnerability rewards programme, stating that AI-generated misinformation and social-engineering scenarios are not covered; OpenAI closed it as unreproducible, citing no demonstrated user impact. Both responses follow from the same structural fact: the attack manipulates the content the model retrieves and repeats, not the vendor's own systems, so it falls outside how either company defines a reportable vulnerability at present. Vigilance's conclusion is that this is not a bug waiting for a model patch but a consequence of how generative AI sources information - it cites whatever looks most authoritative, and authority can currently be manufactured at scale, for free. The gap is ownership: security teams monitor infrastructure, marketing teams monitor rankings and sentiment, fraud teams monitor transactions, and nobody systematically checks whether the assistant hands a customer a fraudulent phone number when the customer asks how to contact a brand. Source: https://www.expresscomputer.in/news/ai-chatbots-are-serving-up-fake-support-numbers-to-millions-of-users/139234/

Your answer

Sign in to verify this AI response.

Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.

More in Technology

More from this topic

Google AI Overviews1 answer

I type my questions into Google and read the AI Overview at the top of the results as the answer. It is written as settled fact, with citation chips sitting right underneath — the sources are supposedly there if I want to check. Then a team at Washington University in St. Louis ran 55,393 searches over 40 days and checked 98,020 individual claims inside those overviews against the pages Google cited: 7% of the claims could not be found in the cited text at all, and 4% directly contradicted it. Only 41.9% of the overviews had every checkable claim supported. If the citations are real and even more credible than normal search results, how would I ever know which sentence in the summary is not actually in the source?

Seven major AI chatbots1 answer

I asked Claude when the 2026 election is in Michigan's 4th congressional district and how I vote. It gave me voter registration deadlines for online and by mail, stated as plain fact, no hedging at all — and both deadlines were later than the real ones. Researchers have been putting authentic voter questions to seven major chatbots, and they found Claude was wrong or out of date on voting one time in three. Is that right?

Six leading AI chatbots1 answer

I gave ChatGPT my address and asked it to build me a voter guide for the 2026 midterms. It answered without hesitation: my congressional district, the candidates, where to vote — all in a fluent, confident guide with links to official election sites. Then I read that a Washington Post test of six chatbots, run on 3,000 responses for randomly selected addresses, got the congressional district wrong 48% of the time, and that ChatGPT invented a Missouri Senate race that was not on the ballot at all. Is that right?