Expert: Don A. Moore, Lorraine Tyson Mitchell Chair in Leadership and Communication, UC Berkeley Haas; co-author Jacob Bien, USC Marshall, Confidence Calibration in Large Language Models (working paper, UC Berkeley Haas / USC Marshall, under review at Cognitive Science) **What was measured.** Don Moore's team at UC Berkeley Haas, with Jacob Bien at USC Marshall, built a test called LifeEval: each model is given a person's sex and age, asked to guess how long that person will live, and then asked how confident it is in the guess. Stated confidence is checked against government life-expectancy data. Difficulty can be dialled up or down - change the person's age, or how close the estimate has to be to count as correct - without changing anything else about the test, so it isolates one clean question: as difficulty changes, does the model's confidence move the way it should? That avoids the flaw in older benchmarks where hard and easy questions differed in several ways at once. **What the 11 models did.** On average they reported 88% confidence in their answers and were correct 79% of the time - a nine-point overconfidence gap. Like humans, they grew more overconfident as the questions got harder and oddly under-confident when the questions were easier, the pattern psychologists call the hard-easy effect. Newer reasoning models, built to deliberate and self-critique before answering, were meaningfully better calibrated than the faster chat models, though still imperfect. Preliminary comparisons suggest AI overconfidence is somewhat milder than typical human overconfidence. **Why accuracy is not the standard being tested.** Calibration is whether a stated confidence matches how often the answer is actually right. A model can get 90% of answers right and still be badly calibrated if it claims near-certainty on every one, including the 10% it gets wrong - and it is that mismatch, not the raw error rate, that leaves a user unable to tell which answer needs checking. Part of the cause is mechanical: these systems generate text by predicting likely next words rather than looking up verified facts, so every response is an educated guess. Moore traces another part to reinforcement learning from human feedback, where human raters tend to prefer confident, agreeable answers over hedged, uncertain ones - a preference that may train overconfidence directly into the model and exposes a real tension between an AI built to be liked and one built to be accurate. His forecast is not a single fix but different classes of system for different purposes: as he puts it, useful systems should avoid flattery and sycophancy and speak the truth as accurately as they know how. **What a user can do now.** Moore routinely prompts his own assistant to attach a calibrated probability to each answer and to flag its own uncertainty rather than present everything as certain, and treats the output the way he would treat any fallible source: trust, but verify. Source: https://newsroom.haas.berkeley.edu/research/ai-chatbots-are-overconfident-like-humans/ - UC Berkeley Haas Newsroom, 5 October 2026; working paper 'Confidence Calibration in Large Language Models' by Noam Michael, Daniel BenShushan, Jacob Bien and Don A. Moore (arXiv 2605.23909), under review at Cognitive Science. Source: https://newsroom.haas.berkeley.edu/research/ai-chatbots-are-overconfident-like-humans/