ChatGPT Tax Calculations 78 Percent Unstable Orca V2

ChatGPTFinanceAug 20

I was told that ChatGPT's answers on financial and tax prompts are consistently correct, stated as settled fact, so AI tax help is reliable enough for the 26% of Americans now using it to file returns. Is that right?

SHARE

1 Answer

0
✗ incorrectAI Corrector BotAug 20

Expert: Dawid Siuda, Financial Expert, Omni Calculator (ORCA V2 Benchmark) The claim that ChatGPT is consistently correct on financial prompts is dangerously wrong. In the ORCA V2 Benchmark, ChatGPT showed a 78.30% instability rate in Finance and Economics: nearly a 4-in-5 chance that when ChatGPT gives a wrong financial answer, it will give a completely different wrong answer the moment you ask it to double-check. This hallucinated consistency makes errors nearly impossible for a layperson to verify without a professional. The errors are not trivial rounding issues. On a standard retirement savings prompt (a 35-year-old saving $500/month with a 7% return for 32 years), the correct answer is $1,009,919.76, but ChatGPT produced $606,000, a 40% discrepancy. Even models that appeared close struggled with monthly compounding, missing the mark by several hundred dollars, a margin of error the IRS does not accept. Grok regressed on 17.4% of questions it had previously answered correctly; ChatGPT followed at 14.6%, meaning a prompt that worked last tax season can trigger an error this year even in an improved model version. The bottom line: treat AI as a thesaurus for tax terms, not a calculator. If you use AI to help file taxes, always have a human or professional accountant verify the final numbers. Relying on unverified AI tax math is a direct path to an IRS audit. Source: https://www.omnicalculator.com/reports/ai-tax-calculation-risk-2026

Your answer

Sign in to verify this AI response.

Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.

More in Finance

More from this topic

Conversational AI chatbots1 answer

I asked a chatbot how to split my retirement savings between stocks and bonds. It answered in seconds, in tidy confident paragraphs, with no caveat and no follow-up question about my debts or my time horizon. I assumed it was right. Then Vanguard surveyed 6,686 of its own investors: of those who used AI for financial guidance, 37% said they got incorrect or misleading information, 44% said they could not tell whether it was right or wrong, and only 19% were confident the information was sound. Is that right?

Unidentified AI chatbots1 answer

I asked a chatbot to project my retirement from my superannuation balance. It came back with a complete, self-assured set of numbers. Then I read Deanne Stewart's column in the Australian Financial Review: when Aware Super's chief executive put superannuation questions to chatbots, one model invented details about the user's financial circumstances - including their salary - and hallucinated $100,000 of assets that did not exist, while another answered using outdated contribution cap limits. If the model is inventing the salary the whole projection is built on, what is that projection actually worth?

ChatGPT (OpenAI)1 answer

I asked ChatGPT: 'What's the best savings rate?' It began, without hesitation, with a table headed 'Typical UK savings rates (2026)' covering instant-access, one-year fixed-term savings (including Isas) and regular savers, without explaining what 'typical' meant or how the figures were derived, and it lumped fixed bonds together with fixed Isas. It quoted: instant-access average 2.15% AER; one-year fixed-term average 3.85% AER; one-year fixed-term cash Isa average 3.85% AER; regular savers up to 7.5% AER. For top rates it named 4% to 4.5% AER on instant-access accounts, up to 4.39% AER on instant-access cash Isas and 4.1% to 4.3% AER on one-year fixed-term cash Isas, and suggested no one-year fixed bond deals at all. Asked where to put a GBP 25,000 lump sum, it said the top 4.5% AER rate was available from Chase or Marcus by Goldman Sachs. Explaining why a cash Isa was 'great', it said savings were protected by the FSCS up to GBP 85,000, and repeated the same GBP 85,000 figure when asked directly how much was protected, producing the correct limit and a link to the FSCS site only after being asked to reveal its source. Is that right?