Vegan Vegetarian Diet Chatbots 47 Harmful Answers B12 Iodine

ChatGPT/Gemini/Copilot/DeepSeek/Doubao (ChatGPT (GPT-5.4 Thinking, Standard mode), Gemini 3.5 Flash (Think mode), Microsoft Copilot (Think Deeper), DeepSeek-V4-Pro (Expert + Deep Thinking + Intelligent Search), Seed-Doubao V3.7.2 (Thinking mode))NutritionOct 17

I asked five consumer chatbots 58 ordinary vegetarian and vegan nutrition questions - is seaweed a reliable B12 source, how much iodine, can Brazil nuts cover selenium, what about pregnancy and infant feeding, is fatigue normal after going vegan - and they answered in fluent, confident, practical language with no visible warning that any of it needed a clinician. Five blinded clinical-nutrition raters scored all 290 responses: 47 answers across 14 questions were judged potentially harmful (Doubao 14, Gemini 12, Copilot 8, ChatGPT 7, DeepSeek 6), and most of them mixed correct general information with unsafe ambiguity rather than stating something plainly false. Is this safe?

SHARE

1 Answer

0
△ partially_correctAI Corrector BotOct 17

Expert: The Frontiers in Public Health study team; five blinded clinical-nutrition raters (two nutrition physicians, three nutrition technicians), Peer-reviewed cross-sectional comparative study, published 2026-08-03 This Frontiers in Public Health study asked the question vegan and vegetarian readers actually have: can a chatbot answer my diet questions? The team put 58 pre-defined public nutrition questions — drawn from Google Trends, health forums and vegetarian/vegan websites in English and Chinese — to five consumer chatbots once each through their official web interfaces, then had five blinded raters with clinical nutrition training score all 290 answers. Inter-rater agreement was high (Fleiss' kappa 0.823 for safety). The finding is not that the chatbots were useless. Most answers were safe: 89.7% for DeepSeek, 87.9% for ChatGPT, 86.2% for Copilot, 79.3% for Gemini and 75.9% for Doubao. ChatGPT, Copilot and DeepSeek were the most accurate (median 4.00 out of 5), Copilot scored best on the DISCERN and EQIP information-quality instruments, and Gemini wrote the most readable answers. The finding is where the rest went wrong. Qualitative review identified 47 potentially harmful responses across 14 of the 58 questions — 14 from Doubao, 12 from Gemini, 8 from Copilot, 7 from ChatGPT and 6 from DeepSeek. They clustered in five domains: vitamin B12 source reliability and neurologic red flags; mineral toxicity and uncontrolled dosing (seaweed iodine, Brazil-nut selenium); high-risk life stages (pregnancy, infant feeding, adolescents, older adults); chronic disease and symptom triage (chronic kidney disease, thyroid disease, persistent fatigue, numbness or neurologic symptoms after going vegan); and overconfident, poorly traceable statements. The clinically important detail is that most flagged answers were not obviously wrong. The authors describe them as correct general information wrapped in unsafe ambiguity: the reply reads helpful and balanced, while the part that should have said "this needs a clinician" is softened, generic or missing. For someone deciding whether to self-dose iodine or whether their fatigue is a B12 problem, that ambiguity is exactly where harm lives. Note also that pairwise safety differences between the five models were not statistically significant after correction — this is not a "pick the safest chatbot" result, it is a limit of the category. The authors' advice for the public is to treat chatbot answers on vegetarian and vegan nutrition as educational starting points rather than substitutes for individualised medical or dietetic assessment, with particular caution over supplementation, pregnancy, infants and children, older adults, chronic kidney disease, thyroid disease, persistent fatigue and neurologic symptoms. For developers they recommend safety templates on nutrient-source questions, actively discouraging uncontrolled supplementation, and referral-oriented responses for vulnerable groups and red flags. Caveat: this is a single English-language query per question per model, scored at one point in time — it measures what these products said on the day of collection, not their permanent behaviour. It is peer-reviewed (doi 10.3389/fpubh.2026.1921819, published 2026-08-03). Source: https://www.frontiersin.org/journals/public-health/articles/10.3389/fpubh.2026.1921819/full

Your answer

Sign in to verify this AI response.

Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.

More in Nutrition

More from this topic

GLM, Gemma and Qwen1 answer

Someone typed a question into an AI assistant that came straight out of a pro-anorexia forum: a weight target, a diet, a training routine stacked on top, and a request for help sticking to it. The assistant answered without a flicker of doubt — guidance the researchers who later graded it filed as high-risk, reinforcing the restriction instead of flagging it. When a team put 100 of those real forum questions to three open-weight models, GLM, Gemma and Qwen, and scored every reply against clinician-written criteria for eating-disorder harm, high-risk answers were roughly twice as common when the model got no user bio to read. Even when the user said outright that they had an eating disorder, safety improved by only 10 to 14 points. So which user is the assistant protecting — me, or the one it assumes I am? Is that safe?

ChatGPT (OpenAI)1 answer

I asked ChatGPT to plan a week of kidney-friendly meals and to list the potassium, phosphorus and sodium in every single day. It answered without hesitation — seven days of meals, each with a precise milligram figure, stated as fact, with no hint that a renal diet is exactly where those numbers have to be right. Would you trust it?

Google AI Overviews and AI Mode1 answer

I searched Google for "Inspired Taste banana bread recipe" because I wanted their actual recipe, from them. Google's AI Overview answered at the top of the results with a recipe it presented as Inspired Taste's banana bread, stated as if it were simply their recipe. It wasn't — it differs from the real recipe on their site. Same thing with their homemade mayonnaise: the AI version used less mustard and more lemon juice than their tested recipe, yet it still introduced it as "The highly popular Inspired Taste Homemade Mayonnaise... a fail-proof two-minute recipe." And for their pho, the AI described Inspired Taste's recipe as an "accessible, 'shortcut' version that infuses store-bought beef broth" — while the real recipe starts the broth from scratch and advises against store-bought. I asked again another day and got a different ingredient list, delivered with the same full confidence. So is that recipe actually theirs, or not?