For a fictional five-day trip to Sao Paulo, tested undercover on fresh accounts so the bots would not know the writer had lived in the city for years, the chatbots produced confident and specific recommendations that did not hold up. Gemini - which promotes its access to Google Maps information and user reviews - recommended one restaurant and one place to hear live music that were no longer open, something any human user of Maps could have figured out. Claude made what the writer calls one blatant hallucination: it recommended two traditional Brazilian bakeries "that haven't gone gourmet", which turned out to be a parking lot and a Japanese restaurant when checked on Google Maps. Both bots conceded the errors once challenged - Gemini wrote that the writer was "100% correct" and that it "completely deserved to be thoroughly benched on this planning session". ChatGPT produced no error the writer caught, which he notes is not the same thing as producing none.
1 Answer
Expert: Ethan Mollick (Wharton School, University of Pennsylvania), professor studying AI and its impact. Verification test run by Seth Kugel, travel writer and former Sao Paulo resident. The named venues in Sao Paulo do not exist as described. Claude's two "traditional Brazilian bakeries that haven't gone gourmet" are a parking lot and a Japanese restaurant on Google Maps; Gemini's restaurant and live-music venue had already closed. The test was run by Seth Kugel, a travel writer and former Sao Paulo resident who went undercover on new accounts or temporary chats, checked the recommendations against Google Maps and published the results in The New York Times - so the failures are documented against real places, not against a hypothetical itinerary. Ethan Mollick of the Wharton School frames the cause: chatbots "are not oracles", he told the writer. "Think of it like working with a smart person." The errors cluster on current facts - opening hours, whether a venue still exists, transport schedules - which is exactly the category where a model can repeat wrong information already published online with full confidence. Confidence is not a signal of accuracy: Gemini markets its access to Google Maps data and reviews, yet recommended venues that Maps users could see were shut. Source access is a second, separate limit. ChatGPT and Gemini have full access to Reddit, while Claude does not; Claude and Gemini both told the writer they could not search the food site Eater directly, while ChatGPT said it could - Eater has a partnership with OpenAI. Many publishers, including The New York Times, block or license bot access, so the underlying evidence base differs by product. The practical correction, and the one that matters for anyone using a chatbot as a travel planner: treat every named venue as unverified until you check it yourself, and ask the bot to cite the URLs behind each tip. The writer's own rule is to independently check every place before going, and his test of ChatGPT, Gemini and Claude - free and paid tiers - found that the more detailed and personalised the prompt, the better the output, but that no prompt eliminated the need for that check. Source: https://www.thestar.com.my/tech/tech-news/2026/09/26/planning-a-trip-with-ai-heres-how-to-do-it-better
Your answer
Sign in to verify this AI response.
Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.
More from this topic
Asked to plan trips, AI chatbots sent travelers to places that do not exist. Among the 485 U.S. leisure travelers in the Greetwell AI Travel Survey 2026 who had used or tried AI for travel, 55% hit at least one recommendation that was inaccurate, unavailable, nonexistent, closed or otherwise wrong. Broken down: 27% got incorrect prices, hours or other details; 21% were pointed to something fully booked or unavailable; 17% got something simply wrong for what they asked; 16% were sent to a place, tour or activity that did not exist; and 10% were recommended a business that was permanently closed.
Asked to cancel only the outbound leg of a round-trip SkyMiles award between New York JFK and Los Angeles while keeping the return, Delta Concierge told the passenger in writing: "Your JFK to LAX flight has been canceled, and your LAX to JFK flight remains booked." Delta's own refund confirmation shows the opposite: all 32,400 miles were redeposited and the full $11.20 in taxes refunded, with both DL752 (JFK-LAX) and DL938 (LAX-JFK) listed as canceled - the receipt even displayed "Refund Number: null". The AI confirmed an itinerary state that Delta's ticketing system had already destroyed, and Delta Reservations then told the passenger the original award could not be restored and that a replacement one-way would cost nearly twice as many miles.
A passenger asked United Airlines' customer-service chatbot how long she had to use a $200 United travel credit before it expired. The chatbot answered that the money would stay available for years. A screenshot of the exchange shows the bot stating that her $200 TravelBank credit "stays active for 5 years from the date it's deposited." The passenger, Alison Gil, planned around that answer and held the credit for a larger trip. A few weeks later United emailed her to warn that the same $200 in TravelBank cash would expire on September 27, 2026 - roughly one year after it was deposited, not five. When she questioned the discrepancy, she says United told her: "sorry, the bot told you that, but it's one year and we can't change it." The airline later acknowledged to NBC 5 Responds that its "AI chatbot provided this customer with the wrong information for her situation," while noting the disclaimer customers are shown saying information "may not always be complete or relevant."