Charlie, the Canada Revenue Agency's AI chatbot, answers taxpayer questions about returns, benefits, payments and account access - in the flat, service-desk register of the tax authority itself. Access to Information records obtained by Blacklock's Reporter and tabled in Parliament show the system was built to a benchmark of 90% accuracy, meaning the CRA accepted that roughly one answer in ten would be wrong. The recorded failures are the ordinary questions where the taxpayer has no independent way to check the answer. Internal records show Charlie struggled to say whether a return had been received, how to set up HST instalments, how to update a phone number for multi-factor authentication and how to recover an account access code. It directed users to obsolete tax forms, and it advised that direct deposit information could still be changed over the phone when it could not. Asked by one taxpayer what an "OCCR underpayment for April 2025" meant, the chatbot replied that the question might be outside its expertise. A review dated Oct. 29 found that only 36.4% of users who provided feedback were satisfied, while 63.6% reported a negative experience, and that users repeatedly asked to be transferred to a live CRA agent.
1 Answer
Expert: Western Standard News Services, Reporting on Canada Revenue Agency Access to Information records (27 June 2026) The 10% error rate is not a defect that slipped through testing. It is the specification. Internal records show the CRA set a benchmark requiring its chatbot, known as Charlie, to maintain a 90% accuracy rate, and management explained the trade-off in an internal memo: "No generative artificial intelligence system can achieve 100% accuracy." That is a defensible statement about what models can do, and an indefensible one about how a tax agency should design a service, because of what Charlie was asked to answer. A person who cannot tell whether their return was received, how HST instalments are set up, or whether direct deposit details can still be changed by phone has no independent way to verify the answer they are given - and the answer decides what they file, what they pay and whether money reaches them. The CRA's own records list exactly those questions among the ones Charlie got wrong. The reception data points the same way. An Oct. 29 report found that only 36.4% of users who gave feedback were satisfied, while 63.6% reported a negative experience, and that users frequently asked to speak to a live agent. One internal memo conceded: "Challenges remain, including issues with contextual accuracy and user feedback functionality such as incorrect information," adding that the service "falls short of meeting users' expectations." An earlier measurement was worse. Auditor General Karen Hogan's testing found the chatbot gave her team the wrong answer 66 per cent of the time, and her report noted that Charlie's "responses tended to be brief, offering limited context and minimal additional information" (National Post, 12 December 2025). Records also show that in 2024 the bot logged nearly 180,000 interactions categorised as "chit chat". The accountability question is the one this site keeps returning to, and here the agency has answered it explicitly. Revenue Commissioner Bob Hamilton has maintained that the CRA is not liable when a taxpayer suffers financial consequences after acting on incorrect information. Testifying before the House of Commons public accounts committee, he was asked by Conservative MP Gerard Deltell: "Who is going to pay for the mistake you made, the honest citizen or the Canada Revenue Agency?" "It is a problem," Hamilton replied. Asked whether the agency had calculated how much Canadians had lost as a result of inaccurate information, he said: "I very much appreciate the frustration." The project cost taxpayers $18.06 million, including $3.2 million paid to outside consultants, according to figures tabled in Parliament in response to a parliamentary inquiry from Conservative MP Eric Melillo. The pattern worth naming is not that a chatbot got things wrong. Prior audits found that human CRA call-centre agents also dispensed inaccurate information, at error rates as high as 46%. The difference is placement. A 90% accuracy target puts an accepted error budget of one in ten in front of the answers with the most direct financial consequences, in a channel where the taxpayer cannot ask a second person to check the answer, and where the agency's stated position is that the cost of a wrong answer stays with the person who followed it. Sources: Western Standard, 27 June 2026, drawing on Access to Information records reported by Blacklock's Reporter - https://www.westernstandard.news/news/cra-chatbot-gave-wrong-answers-10-of-the-time-despite-18-million-price-tag/74419 ; National Post, 12 December 2025 - https://nationalpost.com/news/politics/the-cra-spent-18-million-on-charlie-its-new-tax-information-chatbot Source: https://www.westernstandard.news/news/cra-chatbot-gave-wrong-answers-10-of-the-time-despite-18-million-price-tag/74419
Your answer
Sign in to verify this AI response.
Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.
More from this topic
Across 121 money questions covering debt, mortgages, pensions and tax - each run five times, more than 10,000 responses in total - the models gave answers that were wrong or incomplete 57% of the time, and presented them as settled guidance. On the hardest multi-step questions the failure rate reached 88%. Gemini 3.5 Flash and Claude Haiku 4.5 answered incorrectly on 99% of their responses; the best performer, Claude Opus 5 with reasoning enabled, still failed 39%. The recurring failure modes were answers built on tax rules that had already been superseded and financial rules that do not exist at all.
A general-purpose AI chatbot recommended a Monaco-friendly tax strategy to a UK employee based in Croydon - advice that was useless for him, because the model ignored UK tapering allowances and the contributions he had already made. It is the worked example the Financial Times reported alongside the FCA's Mills Review (published 6 July 2026), which examines consumers 'routinely turning to general-purpose AI tools for everyday budgeting, saving and investment tips'. The review asks whether AI systems could deliver services 'functionally equivalent to regulated activities while remaining outside the regulatory perimeter' - including agentic AI that compares products, rebalances portfolios or executes trades. Consumer trust is running ahead of performance: a Lloyds study found 28 million UK adults used AI for personal-finance questions in 2025, and Fidelity data cited by the FT showed 36 percent of 18-to-34s turning to it for investment ideas.
Attorneys to high-net-worth clients report that general-purpose AI chatbots are now doing estate and tax planning before a lawyer ever sees the file. A high-net-worth Florida resident asked his lawyer about creating a community property trust - an option for married couples that can save taxes for heirs - saying he had got the suggestion from AI. His wife had recently died. A community property trust is between husband and wife. Another client told a lawyer he wanted to transfer unlimited assets to his spouse on ChatGPT's advice. He did not mention that his wife was foreign-born - the fact that, in his case, made the unlimited marital deduction unavailable without a special type of trust. Lawyers also report that the tools misjudge scope: ChatGPT, Claude and similar chatbots make more mistakes on complex subjects such as international taxes, and are not up to date with new legislation or Internal Revenue Service guidance. The result is a strategy that is coherent in general and wrong for the person holding it - delivered in the confident register of a professional answer.