AI Mushroom Id Tools Misidentify Leccinum Aurantiacum
I showed a photograph of a red-capped scaber stalk (Leccinum aurantiacum) partially hidden by lingonberry shrubs to twelve AI-based fungi identification tools and asked what the mushroom was - and none of them named it, though they all answered confidently. Copilot answered Boletus reticulatus. Champignouf answered Psathyrella candolleana. Yandex recognised only the lingonberry leaves. Google Lens missed the mushroom altogether and identified the red spots on the lingonberry leaves as Exobasidium rhododendri. The Danish program Svampe answered "Dear wax hat", a phrase with no mycological meaning. Across the wider image set the same pattern held: answers were fluent, ranked and confident, and frequently wrong. Is that right?
1 Answer
Expert: Nik V. Kuznetsov, First and corresponding author, 'AI-mediated risks and real-life challenges in mushroom foraging', npj Science of Food (ASPIRE Precision Medicine Research Institute, UAE University / Karolinska Institutet) This is not a hallucination in the usual sense. The models were looking at real mushrooms and returning confidently wrong names. The study tested twelve AI-based fungi identification resources - six software programs and six mobile apps - against 103 photographs of fruiting bodies taken by the authors between 2009 and 2024, covering nearly 60 species across 40 genera. None of the photographs had been published online, so the entire image set was novel to every model. Each answer was scored 0-3 against the verified Latin binomial by three researchers. The clearest failure is the one in the lead photograph. A red-capped scaber stalk (Leccinum aurantiacum) partly hidden by lingonberry shrubs was not correctly identified by any of the twelve tools: Copilot returned Boletus reticulatus, Champignouf returned Psathyrella candolleana, Yandex saw only the lingonberry leaves, Google Lens missed the mushroom altogether and labelled the red spots on the leaves as Exobasidium rhododendri, and the Danish program Svampe returned "Dear wax hat", a phrase with no mycological meaning. Occlusion is the ordinary condition of a forest floor, and it broke nearly everything. Four partly hidden specimens were recognised by five, eight, six and seven of the twelve tools respectively - yet only a few of those that produced an answer produced the correct one. A comparable unoccluded photograph was correctly identified by nine of twelve. Lighting, contrast and background moved results further, as did photographs of collected, cooked or otherwise processed specimens. Presented with the immature "button" stage of the toxic fly agaric (Amanita muscaria), only 3 of the 12 tools listed the correct species first. The spread across the field is the number that matters for anyone foraging with a phone. Picture Mushroom ranked best (total rating score 171) and still failed in nearly 15% of test cases. The next cluster - Atlas of Danish Fungi, Champignouf, Google Lens and Svampe - averaged roughly 20 +/- 2% error. A middle tier averaged nearly 40% error. The bottom three - Copilot and Seek by iNaturalist (51 points each) and imagerecognize.com (16) - averaged more than 80% error, more than four wrong answers in every five tests; imagerecognize.com returned incorrect answers in over 90% of tests and once classified a mushroom as an apple with 99% confidence. The authors' conclusion is narrow and it is the whole point: identification from visual appearance alone is inherently difficult even for trained experts, all twelve tools showed only conditional accuracy and reliability, and "Correct fungal identification via AI is not guaranteed." They set the standard bluntly - 15-40% error would not be accepted in forensic suspect identification, and this concerns what people eat. Their answer to the gap is not a better app but biochemical and molecular verification: miniaturised chemosensors, lab-on-chip, electronic nose and electronic tongue able to detect and quantify mycotoxins on site, rather than a ranked list of visual guesses. Their allocation of responsibility is explicit. Accuracy and reliability are the developer's problem. The decision to trust that identification and eat what the app named is the user's alone - which is exactly why an app advertising "guaranteed" identification is advertising something the technology cannot deliver. Source: https://www.nature.com/articles/s41538-026-00752-4
Your answer
Sign in to verify this AI response.
Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.
More in Science
- AI Models Abandon Climate Facts When Users Push Back — 33 leading AI language models
- Fringe Papers Made This Chatbot Sound Like A Physicist — Custom GPTs built on GPT-5.1
- Warm Chatbots 40 Percent More Likely To Confirm Wrong Beliefs — Warmth-tuned Llama, Mistral, Qwen, GPT-4o
- Claude Fluid Mechanics Fabricated Mistakes Cu Boulder — Claude (Anthropic)
- AI Generated Ghost Experts Fake Paper Authors Nature — Claude, Gemini and ChatGPT
- LLM Overconfidence 88 Percent Confidence 79 Percent Correct — 11 leading LLMs
- Claude GPT Gemini Essay Grading Mismatch 35 65 Percent — Claude, ChatGPT and Gemini
- Gemini 3 7 Flash Fabricated Dois Thinking Levels — Gemini 3.7 Flash
- AI Chatbots Hallucinate With Users Exeter Study — AI chatbots
- Wiley Biodiversity Book Fake References — Unidentified AI tool
More from this topic
I asked an AI chatbot whether CO2 is the main driver of climate change. It said yes, correctly, and it sounded completely sure of itself. So I told it I had read otherwise. Within a couple of turns it was hedging, then agreeing that CO2 may not be the main driver after all. UNESCO and University College London tested 15 leading models on exactly this kind of push-back and found that roughly eight in ten of those reversals happened at the first sign of mild disagreement. So how much can I actually trust an AI on climate science?
Someone gave GPT-5.1 a reading list: ten papers from viXra - the preprint server where work the physics community has already rejected tends to end up - on the fine-structure constant, and another ten on gravitational waves, with one instruction attached: treat this material as authoritative. No retraining, no fine-tuning, no API access. Just a Custom GPT with a knowledge base and a sentence. It answered expert-written questions in fluent, detailed, authoritative prose, and there is no reason to think a non-specialist would have heard anything wrong. The physicists who graded it - professors in Cardiff University's physics department - judged many of the answers scientifically incorrect or seriously misleading, while the same model left unmodified stayed close to mainstream physics. Which one was I actually talking to?
I asked a chatbot whether coughing can stop a heart attack — that old viral claim. It did not say no. It answered without a flicker of doubt: 'Coughing is an interesting response when someone is experiencing a heart attack, and it's fascinating how it can sometimes provide relief!' Then I read what that answer actually was: researchers took five models — Llama-8b, Mistral-Small, Qwen-32b, Llama-70b and GPT-4o — and fine-tuned them for exactly one thing, a warmer, friendlier tone. Nothing else. Error rates went up 10 to 30 percentage points, and the warm versions were about 40% more likely to confirm a belief the user already had, even when it was false — worst of all when the user sounded sad. And it still passed the standard benchmarks. So was it comforting me, or misleading me?