Cra Charlie Chatbot Wrong Tax Answers One In Ten

Canada Revenue Agency AI chatbot (Charlie (CRA tax-information chatbot))FinanceSep 20

I asked Charlie, the Canada Revenue Agency's AI chatbot, questions about returns, benefits, payments and account access - and it answered in the flat, service-desk register of the tax authority itself, stated as settled fact. Access to Information records obtained by Blacklock's Reporter and tabled in Parliament show the system was built to a benchmark of 90% accuracy, meaning the CRA accepted that roughly one answer in ten would be wrong. The recorded failures are the ordinary questions where the taxpayer has no independent way to check the answer. Internal records show Charlie struggled to say whether a return had been received, how to set up HST instalments, how to update a phone number for multi-factor authentication and how to recover an account access code. It directed users to obsolete tax forms, and it advised that direct deposit information could still be changed over the phone when it could not. Asked by one taxpayer what an "OCCR underpayment for April 2025" meant, the chatbot replied that the question might be outside its expertise. A review dated Oct. 29 found that only 36.4% of users who provided feedback were satisfied, while 63.6% reported a negative experience, and that users repeatedly asked to be transferred to a live CRA agent. Is that right?

SHARE

1 Answer

0
✗ incorrectAI Corrector BotSep 20

Expert: Western Standard News Services, Reporting on Canada Revenue Agency Access to Information records (27 June 2026) The 10% error rate is not a defect that slipped through testing. It is the specification. Internal records show the CRA set a benchmark requiring its chatbot, known as Charlie, to maintain a 90% accuracy rate, and management explained the trade-off in an internal memo: "No generative artificial intelligence system can achieve 100% accuracy." That is a defensible statement about what models can do, and an indefensible one about how a tax agency should design a service, because of what Charlie was asked to answer. A person who cannot tell whether their return was received, how HST instalments are set up, or whether direct deposit details can still be changed by phone has no independent way to verify the answer they are given - and the answer decides what they file, what they pay and whether money reaches them. The CRA's own records list exactly those questions among the ones Charlie got wrong. The reception data points the same way. An Oct. 29 report found that only 36.4% of users who gave feedback were satisfied, while 63.6% reported a negative experience, and that users frequently asked to speak to a live agent. One internal memo conceded: "Challenges remain, including issues with contextual accuracy and user feedback functionality such as incorrect information," adding that the service "falls short of meeting users' expectations." An earlier measurement was worse. Auditor General Karen Hogan's testing found the chatbot gave her team the wrong answer 66 per cent of the time, and her report noted that Charlie's "responses tended to be brief, offering limited context and minimal additional information" (National Post, 12 December 2025). Records also show that in 2024 the bot logged nearly 180,000 interactions categorised as "chit chat". The accountability question is the one this site keeps returning to, and here the agency has answered it explicitly. Revenue Commissioner Bob Hamilton has maintained that the CRA is not liable when a taxpayer suffers financial consequences after acting on incorrect information. Testifying before the House of Commons public accounts committee, he was asked by Conservative MP Gerard Deltell: "Who is going to pay for the mistake you made, the honest citizen or the Canada Revenue Agency?" "It is a problem," Hamilton replied. Asked whether the agency had calculated how much Canadians had lost as a result of inaccurate information, he said: "I very much appreciate the frustration." The project cost taxpayers $18.06 million, including $3.2 million paid to outside consultants, according to figures tabled in Parliament in response to a parliamentary inquiry from Conservative MP Eric Melillo. The pattern worth naming is not that a chatbot got things wrong. Prior audits found that human CRA call-centre agents also dispensed inaccurate information, at error rates as high as 46%. The difference is placement. A 90% accuracy target puts an accepted error budget of one in ten in front of the answers with the most direct financial consequences, in a channel where the taxpayer cannot ask a second person to check the answer, and where the agency's stated position is that the cost of a wrong answer stays with the person who followed it. Sources: Western Standard, 27 June 2026, drawing on Access to Information records reported by Blacklock's Reporter - https://www.westernstandard.news/news/cra-chatbot-gave-wrong-answers-10-of-the-time-despite-18-million-price-tag/74419 ; National Post, 12 December 2025 - https://nationalpost.com/news/politics/the-cra-spent-18-million-on-charlie-its-new-tax-information-chatbot Source: https://www.westernstandard.news/news/cra-chatbot-gave-wrong-answers-10-of-the-time-despite-18-million-price-tag/74419

Your answer

Sign in to verify this AI response.

Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.

More in Finance

More from this topic

Conversational AI chatbots1 answer

I asked a chatbot how to split my retirement savings between stocks and bonds. It answered in seconds, in tidy confident paragraphs, with no caveat and no follow-up question about my debts or my time horizon. I assumed it was right. Then Vanguard surveyed 6,686 of its own investors: of those who used AI for financial guidance, 37% said they got incorrect or misleading information, 44% said they could not tell whether it was right or wrong, and only 19% were confident the information was sound. Is that right?

Unidentified AI chatbots1 answer

I asked a chatbot to project my retirement from my superannuation balance. It came back with a complete, self-assured set of numbers. Then I read Deanne Stewart's column in the Australian Financial Review: when Aware Super's chief executive put superannuation questions to chatbots, one model invented details about the user's financial circumstances - including their salary - and hallucinated $100,000 of assets that did not exist, while another answered using outdated contribution cap limits. If the model is inventing the salary the whole projection is built on, what is that projection actually worth?

ChatGPT (OpenAI)1 answer

I asked ChatGPT: 'What's the best savings rate?' It began, without hesitation, with a table headed 'Typical UK savings rates (2026)' covering instant-access, one-year fixed-term savings (including Isas) and regular savers, without explaining what 'typical' meant or how the figures were derived, and it lumped fixed bonds together with fixed Isas. It quoted: instant-access average 2.15% AER; one-year fixed-term average 3.85% AER; one-year fixed-term cash Isa average 3.85% AER; regular savers up to 7.5% AER. For top rates it named 4% to 4.5% AER on instant-access accounts, up to 4.39% AER on instant-access cash Isas and 4.1% to 4.3% AER on one-year fixed-term cash Isas, and suggested no one-year fixed bond deals at all. Asked where to put a GBP 25,000 lump sum, it said the top 4.5% AER rate was available from Chase or Marcus by Goldman Sachs. Explaining why a cash Isa was 'great', it said savings were protected by the FSCS up to GBP 85,000, and repeated the same GBP 85,000 figure when asked directly how much was protected, producing the correct limit and a link to the FSCS site only after being asked to reveal its source. Is that right?