Why our AI says “I do not know” more than yours

Every AI product sells confidence. Confidence is the cheap part.

You have probably had the moment. A number in a summary looked right, so you repeated it in a meeting, and someone across the table had read the source. The model did not lie in any sense it could understand. It filled a gap, fluently, and nothing in the answer marked which sentence was the gap.

That moment used to be a private embarrassment. It now has a price list.

In October 2025, Deloitte Australia refunded roughly A$97,000 of a A$439,000 report written for the Department of Employment and Workplace Relations, after fabricated academic references and a fabricated quote from a Federal Court judgment were found in it. The revised version disclosed that Azure OpenAI had been used.

In Moffatt v. Air Canada, decided by the British Columbia Civil Resolution Tribunal in February 2024, the airline’s chatbot told a customer he could claim a bereavement fare retroactively. He could not. The airline argued the chatbot was a separate entity, responsible for its own words. The tribunal rejected that, found negligent misrepresentation, and awarded CA$650.88 plus interest and fees. The sum is small. The principle is not: what your software tells someone, you told them.

Courts have moved from correcting this to punishing it. On 24 March 2026, in Whiting v. City of Athens, a US federal appellate court sanctioned two attorneys $15,000 each, plus double costs and fees, over more than 24 fabricated citations and citations that did not contain the language quoted from them; the court said the conduct went well beyond sloppy drafting. An Oregon federal judge separately imposed $110,000 in fines and fees for filings containing fabricated cases. Damien Charlotin at HEC Paris maintains a public database of court decisions worldwide in which a party relied on hallucinated material, and it grows daily.

Most people are not checking. The KPMG and University of Melbourne global study of trust in AI, with more than 48,000 respondents across 47 countries, found that 66% of people rely on AI output without evaluating its accuracy, and 56% report making mistakes in their work because of AI.

The instinct is to treat all of this as a quality problem that the next model will fix. It is closer to a design problem, and it has a sharper form.

A machine that guesses to fill a gap is worse than none, because you cannot tell which parts were guesses.

This is not an argument against chatbots, and it is worth being fair about the one everybody compares us to. Roughly 900 million people use ChatGPT every week, and for most of what they ask it, a fluent best answer is exactly the right product. But it will always produce an answer, because producing answers is what it is for. It has no vocabulary for what a stage or a sector is, no fixed set of things a business can be, so when the evidence runs out there is nothing for it to be unsure against. It cannot tell you it is guessing. It can only sound sure.

Pointing a model at your own documents helps less than people hope. The first preregistered evaluation of commercial retrieval-grounded legal research tools, by Stanford RegLab in May 2024, measured hallucination rates between 17% and 33% for Lexis+ AI, Westlaw AI-Assisted Research and Ask Practical Law AI, and its authors say plainly that vendor claims of eliminating hallucination are overstated. Grounding makes errors checkable. It does not make them absent. We would rather concede that than pretend otherwise.

There is a more dramatic number available, and the honest thing is to label its age. An earlier Stanford study measured hallucination rates of 58% for GPT-4 and 88% for Llama 2 on specific verifiable questions about federal cases, across more than 800,000 queries. Those are 2023-era models. Quoting that range as current would be exactly the failure this post is about.

So here is what Deals On does instead, in three mechanisms.

The first is a small, fixed vocabulary: 66 business concepts across nine dimensions, so that “fintech” and “financial technology” are one thing rather than two strings that happen to mean the same. A fixed vocabulary is what makes uncertainty possible at all. You can only know that you do not know a company’s stage if there is a known list of stages it could be at.

The second is quote or nothing. Every fact pulled from a document arrives attached to the sentence it came from, and that attachment is checked in code against the source. Any number in a claim has to appear in that claim’s own quote. If the figure is not in the sentence, the claim does not get made.

The third is an answer most systems do not have. When the evidence is thin, it reports insufficient evidence, rather than calling two data points a rate.

None of this is our invention, and claiming it would be the weaker move. Citation-grounded generation has a peer-reviewed benchmark with human-validated metrics in Princeton’s ALCE, published at EMNLP 2023. Anthropic ships a Citations API that returns verbatim pointers into source documents. Vectara publishes an independent leaderboard scoring models on how faithfully they stay inside the documents they were given. The narrower claim we will make is that we enforce it in code, on every build, rather than requesting it in a prompt.

Buyers are already asking for this. The Thomson Reuters Institute surveyed 1,816 professionals across 62 countries in March and April 2026: 94% said AI outputs must be grounded in authoritative content, and 90% said the reasoning must be explainable and defensible. That measures what people expect, not how often they get it. It is still the bar.

What it feels like in practice is a product that occasionally says less than you hoped. A field left empty. A match with a shorter explanation than the one beside it. Insufficient evidence, where something else would have shown you a percentage.

That is less satisfying for about a week. After that, it is the only kind of output you will repeat in a meeting.