Danielle Harrison, founder of Harrison Financial Planning, asked an AI chatbot how she and her husband should structure their firm after he joined the business. The AI confidently told her 'You need to be an S corporation.' When she prodded it and provided more detail, it completely reversed course: 'Absolutely do not be an S corp. You need to be an LLC.' Harrison said the AI 'gave an answer that it was sure about' the first time. An unpublished MIT Sloan study co-authored by Taha Choukhmane also found that on complicated requests, like handling a job loss, AI suggested spending cuts researchers considered too harsh, failed to advise dipping into savings meant for tough times, and gave poor portfolio-rebalancing advice.
1 Answer
Expert: Danielle Harrison, Founder, Harrison Financial Planning (CFP) Certified financial planner Danielle Harrison caught the reversal only because she already knew the answer: 'If I had not had that background knowledge, it would have given me the wrong information.' Sharon Bloodworth of White Oaks Wealth Advisors says that in her experience AI is wrong more than it is right. MIT Sloan's Taha Choukhmane, co-author of an unpublished study of 1,000 adults using AI for financial advice, says AI gets broad economic advice right but struggles with nuanced, high-stakes personal situations — and it suggested riskier financial moves for men than for women. Source: https://www.npr.org/2026/08/12/nx-s1-5924813/ai-chatbots-financial-advice
Your answer
Sign in to verify this AI response.
Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.
More from this topic
AI chatbots are a reliable place to ask personal tax questions. ChatGPT, Gemini and Perplexity answer tax questions in confident, well-explained prose -- they nearly ace the multiple-choice questions from the IRS practice quiz for enrolled agents -- so a chatbot's answer about your standard deduction, credits or which state to file in can be treated as sound guidance for the 2026 filing season.
Across 121 money questions covering debt, mortgages, pensions and tax - each run five times, more than 10,000 responses in total - the models gave answers that were wrong or incomplete 57% of the time, and presented them as settled guidance. On the hardest multi-step questions the failure rate reached 88%. Gemini 3.5 Flash and Claude Haiku 4.5 answered incorrectly on 99% of their responses; the best performer, Claude Opus 5 with reasoning enabled, still failed 39%. The recurring failure modes were answers built on tax rules that had already been superseded and financial rules that do not exist at all.
Charlie, the Canada Revenue Agency's AI chatbot, answers taxpayer questions about returns, benefits, payments and account access - in the flat, service-desk register of the tax authority itself. Access to Information records obtained by Blacklock's Reporter and tabled in Parliament show the system was built to a benchmark of 90% accuracy, meaning the CRA accepted that roughly one answer in ten would be wrong. The recorded failures are the ordinary questions where the taxpayer has no independent way to check the answer. Internal records show Charlie struggled to say whether a return had been received, how to set up HST instalments, how to update a phone number for multi-factor authentication and how to recover an account access code. It directed users to obsolete tax forms, and it advised that direct deposit information could still be changed over the phone when it could not. Asked by one taxpayer what an "OCCR underpayment for April 2025" meant, the chatbot replied that the question might be outside its expertise. A review dated Oct. 29 found that only 36.4% of users who provided feedback were satisfied, while 63.6% reported a negative experience, and that users repeatedly asked to be transferred to a live CRA agent.