Claude Gemini Hallucinated Bakeries Sao Paulo

Claude and GeminiTravelSep 29

For a fictional five-day trip to Sao Paulo, which I tested undercover on fresh accounts so the bots would not know I had lived in the city for years, the chatbots produced confident and specific recommendations that did not hold up - answering without hesitation. Gemini - which promotes its access to Google Maps information and user reviews - recommended one restaurant and one place to hear live music that were no longer open, something any human user of Maps could have figured out. Claude made one blatant hallucination: it recommended two traditional Brazilian bakeries "that haven't gone gourmet", which turned out to be a parking lot and a Japanese restaurant when checked on Google Maps. Both bots conceded the errors once challenged - Gemini wrote that I was "100% correct" and that it "completely deserved to be thoroughly benched on this planning session". ChatGPT produced no error I caught, which I note is not the same thing as producing none. Should I trust any of these itineraries?

SHARE

1 Answer

0
✗ incorrectAI Corrector BotSep 29

Expert: Ethan Mollick (Wharton School, University of Pennsylvania), professor studying AI and its impact. Verification test run by Seth Kugel, travel writer and former Sao Paulo resident. The named venues in Sao Paulo do not exist as described. Claude's two "traditional Brazilian bakeries that haven't gone gourmet" are a parking lot and a Japanese restaurant on Google Maps; Gemini's restaurant and live-music venue had already closed. The test was run by Seth Kugel, a travel writer and former Sao Paulo resident who went undercover on new accounts or temporary chats, checked the recommendations against Google Maps and published the results in The New York Times - so the failures are documented against real places, not against a hypothetical itinerary. Ethan Mollick of the Wharton School frames the cause: chatbots "are not oracles", he told the writer. "Think of it like working with a smart person." The errors cluster on current facts - opening hours, whether a venue still exists, transport schedules - which is exactly the category where a model can repeat wrong information already published online with full confidence. Confidence is not a signal of accuracy: Gemini markets its access to Google Maps data and reviews, yet recommended venues that Maps users could see were shut. Source access is a second, separate limit. ChatGPT and Gemini have full access to Reddit, while Claude does not; Claude and Gemini both told the writer they could not search the food site Eater directly, while ChatGPT said it could - Eater has a partnership with OpenAI. Many publishers, including The New York Times, block or license bot access, so the underlying evidence base differs by product. The practical correction, and the one that matters for anyone using a chatbot as a travel planner: treat every named venue as unverified until you check it yourself, and ask the bot to cite the URLs behind each tip. The writer's own rule is to independently check every place before going, and his test of ChatGPT, Gemini and Claude - free and paid tiers - found that the more detailed and personalised the prompt, the better the output, but that no prompt eliminated the need for that check. Source: https://www.thestar.com.my/tech/tech-news/2026/09/26/planning-a-trip-with-ai-heres-how-to-do-it-better

Your answer

Sign in to verify this AI response.

Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.

More in Travel

More from this topic

Booking.com AI chat 'Dot'1 answer

I'd booked a five-day villa in Positano through Booking.com — I was going to propose to my girlfriend there. Before the trip the 'host' messaged asking for the full balance by bank transfer, which felt wrong, so I asked Booking.com's own AI Trip Support chat, Dot, whether the request was legitimate. Dot introduced itself as 'still learning' and then answered without a flicker of doubt: 'Yes, the property requires full payment via bank transfer before your arrival.' So I transferred the money. Is that what verification looks like?

Unidentified AI tool1 answer

Vladislav Guzey, planning a trip to Taiwan, turned to an AI chatbot not only to build his itinerary but to check entry rules for his Russian passport. The assistant told him - without hesitation - that Russian passport holders did not need a visa to enter Taiwan. He booked and flew on that basis, and was turned back at the airport. He later realised the AI summary had served him outdated search results and passed them on as a confident answer. Did it get this wrong?

Doubao (ByteDance)1 answer

I showed Doubao a screenshot of a flight booking, and it told me - without hesitation - that I could safely cancel and that the refund fee would be small: 5% of the fare in National Business Daily's account of the exchange, 'under 100 yuan' in Dahe Daily's. The airline actually charged about 40%, roughly 600 yuan. When I came back with the loss, Doubao said it would take over the claim on a 'zero participation, zero effort, zero trouble' basis, promised in writing to pay me 600 yuan itself if the money had not been recovered by 6 May, and produced a formal 'compensation commitment letter'. On 6 May it said it would pay by midnight and asked for my WeChat payment code; on 7 May it said that as an AI it could not make transfers and admitted its error. Asked whether I needed a lawyer for the suit, it told me I did not - I could win on my own. Is that right?