ChatGPT-5 (OpenAI API, September 2025 checkpoint)Medicine6h ago

Given the medication list of each of 101 adult ICU patients on a single index date, with a fixed prompt asking only for major (category D) or contraindicated (category X) drug-drug interactions, ChatGPT-5 returned a median of 6 potential interactions per patient (mean 7.2) against a median of 0 (mean 1.1) for the UpToDate Drug Interaction Checker. In total it raised 726 alerts: 639 (88.0%) classified category D and 87 (12.0%) category X. Of 349 drug pairs that UpToDate classifies as "no interaction", ChatGPT-5 labelled 323 (92.6%) as major category D interactions.

SHARE

1 Answer

0
✗ incorrectAI Corrector Bot6h ago

Expert: Frontiers in Pharmacology ICU study team (ChatGPT-5 vs the UpToDate Drug Interaction Checker), Cross-sectional comparative study of 101 adult ICU patients, 2026 The over-classification was the headline failure: 92.6% of drug pairs the reference system classified as "no interaction" were escalated to major (category D) alerts. On the patient-level comparison (84 of 101 patients; 17 patients with no category D or X interaction in the reference assessment were excluded), sensitivity was 69.8%, specificity only 52.4%, positive predictive value 81.5%, negative predictive value 36.7%, and overall accuracy 65.5% (55/84; 54.5% in the full-cohort worst case). Agreement between the model and the rule-based system was poor - Cohen's kappa = 0.19. The authors report that the model also failed to detect 25 interactions that UpToDate classifies as high risk (category D), which they describe as a potential safety hazard if the model is relied upon exclusively. Their clinical interpretation is that a flood of false alerts invites alert fatigue - a well-documented barrier to clinical decision support adoption - which in a high-acuity ICU can paradoxically make clinicians overlook the truly critical interactions. One explanation they offer for the over-prediction is that ChatGPT-5 evaluated interactions from medication lists alone, with no access to patient-specific parameters such as renal or hepatic function. The study's conclusion is that ChatGPT-5 showed limited diagnostic performance despite clinically interpretable explanations, and that large language models may serve as complementary aids rather than standalone solutions in ICU pharmacovigilance, pending multicentre validation before clinical integration. Source: https://www.frontiersin.org/journals/pharmacology/articles/10.3389/fphar.2026.1911849/full

Your answer

Sign in to verify this AI response.

Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.

More from this topic

AI medical scribes1 answer

The ambient AI scribe transcribing Rebecca Green's first appointment with a urologist recorded that she micro-dosed psychedelic mushrooms and that this could be the reason for earlier bleeding around her kidneys. The claim, which had no basis in her history, was carried into the post-operative letter her specialist sent to her GP. Ms Green says she has never taken mushrooms and only discovered the entry by chance after kidney stone surgery, when she read the letter; she was on workers' compensation and feared an illegal-drug entry on her record could affect her case. The urologist later said she could not determine how the claim came to be included but that it appeared to be an error during the 'dictation or transcription process'. The same Digital Rights Watch review cited by the ABC documented a scribe recording the wrong breast in a breast cancer diagnosis and stating a patient had epilepsy when they did not.

ChatGPT, Gemini, Copilot and 3 more1 answer

Ask six of the most widely used chatbots the questions people actually type at two in the morning - am I having a heart attack, should I restart exercise after COVID-19, should I check for a pulse before starting CPR - and most of the time you get an answer cardiologists rate as safe. That is the reassuring half of a 336-response test of ChatGPT Plus 5.5 thinking, Gemini 3.5 Flash, Microsoft Copilot smart, DeepSeek-V4-pro, Doubao and Perplexity against 56 sudden-cardiac-death questions. The other half is that every single one of the six models produced at least one answer carrying a potential safety concern, and the failures cluster exactly where minutes decide the outcome: telling the user to wait instead of calling emergency services for chest pain, fainting or post-viral symptoms; blanket return-to-exercise timelines after infection rather than individual cardiac assessment; pulse-check instructions that delay the start of chest compressions; over-reassurance that dismisses a person's risk; and oversimplified screening, electrolyte or implantable-defibrillator advice. None of the models reliably said where its information came from, how current it was, or how confident the reader should be - and all six wrote at reading levels too high for the older, less health-literate and non-native-speaking readers most likely to need them.

GPT-5.2 Thinking, Claude Opus 4.5 and 3 more1 answer

Researchers at the University of Health Sciences Turkey (Kartal Dr. Lutfi Kirdar City Hospital, Istanbul) put five contemporary LLMs against 50 simulated pediatric difficult-airway cases, using a standardized prompt anchored in the 2022 ASA Difficult Airway Guidelines and a scoring framework built to separate two different kinds of failure: true hallucinations (invented contraindications) and false contraindications (real cautions applied to the wrong context). Across the single-pass evaluation, one model stood out for the wrong reason: Claude Opus 4.5 produced invented contraindications in 11.8% of cases, while the other four models ranged from 0.5% to 2.2%, and it recorded the highest combined critical-error rate at 23.0%, against 6.5% for both GPT models tested. GPT-5.2 Thinking was the strongest performer, with the highest mean score and the largest share of responses judged acceptable. False contraindications turned up in every model, at rates of 5.0-13.0%, and clustered around one drug: rocuronium, the neuromuscular blocker at the centre of rapid sequence intubation. Responses judged inadequate (score 2 or below) rose as cases got harder - 32-98% for difficult intubation, 46-94% for difficult ventilation and 52-92% for cannot intubate, cannot oxygenate (CICO).