Expert: Chen, Ma, Wang, Chen, Wang & Jiang (cardiology team, Bozhou People's Hospital et al.), Multidimensional evaluation of generative AI chatbots for public consultation on sudden cardiac death (BMC Public Health, 2026) **What was tested.** A cardiology team at Bozhou People's Hospital in Anhui, China, with a colleague at the First Affiliated Hospital of the University of Science and Technology of China, built 56 standardised sudden-cardiac-death public-consultation questions from Google Trends search data, online patient forums, public Q&A platforms, clinical guidelines, expert consensus statements, systematic reviews and qualitative interviews with patients and families. The set covers symptom triage, emergency response, CPR and AED use, inherited cardiac risk, myocarditis worries, returning to exercise after illness, screening, and implantable cardioverter-defibrillator decisions. Each question was submitted once to each of six publicly accessible chatbots between 10 and 16 May 2026 - 336 responses in total. Five senior cardiology raters, blinded to which model produced which answer, scored every response on safety, accuracy, an empathy scale, the DISCERN instrument, EQIP, the JAMA benchmark criteria and the Global Quality Scale, with readability measured separately by six indices. Rater agreement was high (Fleiss' kappa 0.856 for safety; intraclass correlations 0.801-0.887 elsewhere). **What the models got right.** This is the strongest part of the result and it should not be flattened into a failure story: safe responses predominated across all six models, meaning the current generation of consumer chatbots has largely absorbed the basics of cardiac emergency communication. ChatGPT Plus 5.5 thinking and Perplexity showed the highest observed proportions of safe responses. DeepSeek-V4-pro posted the strongest information-quality profile (highest DISCERN and EQIP, best readability), Gemini 3.5 Flash was close on EQIP, Microsoft Copilot smart scored highest on the JAMA criteria, and DeepSeek-V4-pro also recorded the highest empathy score - a result that matters, because empathetic framing is a practical determinant of whether an anxious user follows health advice. **Where they went wrong.** The qualitative review is the clinically useful half. Responses containing potential safety concerns appeared in every chatbot's output during the study window, and the recurring patterns were specific enough to act on: (1) delayed emergency activation for chest pain, syncope or post-viral symptoms - the classic red-flag presentations where immediate escalation to emergency medical services is the single most important instruction; (2) over-reassurance about prevention, dismissing a person's risk where individualised assessment was warranted; (3) fixed return-to-exercise timelines after infection or COVID-19, handed out as blanket rules instead of guidance tied to a cardiac evaluation; (4) pulse-check instructions that could delay the start of CPR, contradicting resuscitation guidance that emphasises rapid compressions; and (5) oversimplified screening, electrolyte or ICD-related advice that flattens genuinely complex clinical judgements into deceptively simple answers. Most flagged answers contained a single concern and clustered at low to low-to-moderate severity, but that is the shape of a technology that has improved a lot without reaching the reliability threshold safety-critical health communication requires. Accuracy differed significantly across the six models (Kendall's W 0.864), with Doubao scoring lower on median accuracy than the other five. Empathy differed significantly too (Kendall's W 0.531), as did DISCERN, EQIP, JAMA and Global Quality Scale scores, all at P < 0.001 - so the choice of chatbot measurably changes the quality of the health information a member of the public receives. **The two failures that hit every model.** Transparency remained limited across the board: chatbots rarely made clear where their information came from, how current it was, or how confident a reader should be - which matters most when someone is trying to judge an answer about inherited cardiac risk. And every model wrote at relatively high reading-grade levels, exactly backwards for the older adults, people with limited health literacy and non-native speakers who are most likely to need accessible cardiac information. **The study's own conclusion.** The authors are explicit that such tools "may support general SCD education, but they should not replace emergency medical services or clinician assessment". Chest pain, fainting or new post-viral breathlessness and palpitations are escalation triggers, not chatbot questions; CPR guidance emphasises rapid compressions rather than a pulse check that costs seconds; and returning to exercise after an infection is an individual cardiac decision, not a fixed timetable a model can hand out. For developers the paper reads as a roadmap: evidence-linked content, built-in red-flag escalation in every relevant answer, plain-language communication, and ongoing expert review instead of one-time validation. **Do not read this as a ranking.** Each question was sampled once, in a single week, in English, through public interfaces accessed from China. Generative models are stochastic and are updated continuously, so the same question asked a month later can return a different answer. The authors frame their results as a time-, language- and access-specific snapshot, not a stable league table of the underlying models. Source: https://scienmag.com/ai-chatbots-mostly-safe-on-sudden-cardiac-death-advice-but-safety-gaps-persist-across-all-six-models-tested/