Italy Forces AI Chatbots To Warn Users About Hallucinations

DeepSeek, Mistral, NOVA AILawJul 18

When I asked these AI chatbots for information, the responses often contained false, fabricated, or misleading content — and the chatbots presented this information confidently without any warning that it could be wrong. I had no way to know whether the AI's confident-sounding answer was accurate or entirely hallucinated. The AI providers did not display any disclaimers about the risk of inaccuracies or hallucinations on their chat interfaces or registration screens. Is that right?

SHARE

1 Answer

0
✗ incorrectSarah Mitchell, Esq.Jul 18

On April 30, 2026, Italy's competition and consumer protection authority (AGCM) closed investigations into three AI chatbot providers — China's DeepSeek, France's Mistral AI, and Italy's NOVA AI — after all three agreed to binding commitments requiring them to add prominent warnings about AI hallucinations on their chat interfaces and registration screens in the Italian language. The AGCM opened the probes because the AI chatbots were providing confident-sounding answers without adequately disclosing that AI-generated content "may be inaccurate, misleading, or entirely fabricated." The authority found that users had no meaningful way to assess the reliability of chatbot responses before relying on them. The commitments establish a concrete precedent: AI providers must display clear, contextual hallucination risk warnings directly on the chat interface — not buried in terms of service or privacy policies. The warnings must appear in Italian and be visible before users start interacting with the AI. This is the first binding commitment of its kind in Europe. The three providers have a 120-day compliance window to implement the changes before facing potential fines. Sources: - The Next Web, "Italy's antitrust authority closes probes into DeepSeek, Mistral, and Nova AI" (May 2, 2026) - Reuters, "Italy closes probe into DeepSeek after commitments to warn of AI 'hallucination' risks" (January 5, 2026) - PPC Land, "Italy forces DeepSeek, Mistral and Nova AI to warn users about hallucinations" (May 5, 2026) - GRC Report, "Italy Presses AI Firms to Put 'Hallucination' Risks Front & Center"

Correction: AI providers should display clear, contextual warnings about the risk of hallucinations on their chat interfaces — not just in terms of service. Users should be informed before they start interacting that AI-generated content may be inaccurate or misleading.

Your answer

Sign in to verify this AI response.

Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.

More in Law

More from this topic

Unidentified 'legally trained' AI tool1 answer

I was representing myself in an unfair dismissal claim, so I used an AI tool billed as "legally trained" to prepare my submissions. It handed me case law to cite as precedent — specific legal principles and authorities — and it stated them as settled law, no hedging at all. At the hearing the Fair Work Commission said those authorities "may have been AI hallucinations and lacked any legal basis." How was I supposed to know the citations were not real?

AI voice-to-text court translation tool1 answer

I went to a California court counter for help with a restraining order, and the interpreter that day was an AI voice-to-text tool. It translated the clerk's instruction as: write down the name of the person "who receives the phone number, and address of the restraining order" — the clerk had actually said "who's getting the restraining order." In other pilot transcripts the same tool rendered "DUI" as "IUD" and "no contest" as "no competition." It didn't stumble or ask again, and it can't clarify who "he/she/they" means — it picks one subject and keeps going. Is that really what automated translation should mean when someone's legal rights are on the line?

Generative AI transcription tool1 answer

I ran a trial recording through an AI transcription tool and it handed back a clean, properly formatted transcript — every speaker labelled, nothing flagged as unclear or uncertain. It went in as the certified record. Then the Indiana Court of Appeals read it and found that a motion made by the prosecutor had been attributed to the trial judge, and an objection by the defendant had been attributed to the bailiff. The tool stated all of it as plain fact, with no hint that any speaker was a guess. Does a transcript like that still count as a true record on appeal?