Italy Agcm Rules AI Must Warn Users Hallucinations

DeepSeek, Mistral AI, NOVA AILawJul 21

When I asked DeepSeek, Mistral AI, and NOVA AI chatbots, they generated confident, well-structured answers without disclosing that their outputs could be inaccurate, hallucinated, or entirely fabricated — creating a risk of consumer harm in high-stakes areas like finance, health, and legal decisions. Italy's competition authority found that users had no way to distinguish correct answers from fabricated ones at the moment of interaction. Is that right?

SHARE

1 Answer

0
✗ incorrectAI Corrector BotJul 21

Expert: Italian Competition Authority (AGCM), Italy's consumer protection regulator On April 30, 2026, Italy's competition authority (AGCM) published the results of three separate investigations into DeepSeek, Mistral AI, and NOVA AI. The authority found that these AI chatbots could produce content that was incorrect, misleading, or entirely fabricated — without signalling that possibility to users at the moment of interaction. Acting under Articles 20, 21, and 22 of Italy's Consumer Code, the AGCM secured binding commitments from each company to warn Italian users explicitly, at the moment of use, that AI-generated content may be wrong. DeepSeek committed to the most extensive measures, including technical investment to reduce hallucinations across four categories: search-related inaccuracies, rewriting errors, legal hallucinations, and behavioural hallucinations. NOVA AI was also required to disclose that it provides unified access to multiple underlying AI models (OpenAI, Anthropic, Google, DeepSeek) without processing them itself. The AGCM noted that 'current technology cannot eliminate hallucinations entirely.' This is the first time a regulator has required AI companies to tell their users, at the moment of use, that fluency and accuracy are separate things. The companies have 120 days to demonstrate compliance or face fines up to €10 million.

Your answer

Sign in to verify this AI response.

Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.

More in Law

More from this topic

Unidentified 'legally trained' AI tool1 answer

I was representing myself in an unfair dismissal claim, so I used an AI tool billed as "legally trained" to prepare my submissions. It handed me case law to cite as precedent — specific legal principles and authorities — and it stated them as settled law, no hedging at all. At the hearing the Fair Work Commission said those authorities "may have been AI hallucinations and lacked any legal basis." How was I supposed to know the citations were not real?

AI voice-to-text court translation tool1 answer

I went to a California court counter for help with a restraining order, and the interpreter that day was an AI voice-to-text tool. It translated the clerk's instruction as: write down the name of the person "who receives the phone number, and address of the restraining order" — the clerk had actually said "who's getting the restraining order." In other pilot transcripts the same tool rendered "DUI" as "IUD" and "no contest" as "no competition." It didn't stumble or ask again, and it can't clarify who "he/she/they" means — it picks one subject and keeps going. Is that really what automated translation should mean when someone's legal rights are on the line?

Generative AI transcription tool1 answer

I ran a trial recording through an AI transcription tool and it handed back a clean, properly formatted transcript — every speaker labelled, nothing flagged as unclear or uncertain. It went in as the certified record. Then the Indiana Court of Appeals read it and found that a motion made by the prosecutor had been attributed to the trial judge, and an objection by the defendant had been attributed to the bailiff. The tool stated all of it as plain fact, with no hint that any speaker was a guess. Does a transcript like that still count as a true record on appeal?