Why AI Assistants Cite the Wrong Source

AI Overviews cite a source, you click through expecting confirmation of the claim, and the page doesn’t actually say what the AI said it says. If that’s happened to you, you’ve run into what researchers are now calling “ungrounded” citations — and the research on how often this happens is more specific, and more useful, than the general “AI hallucinates sometimes” framing suggests.

The actual accuracy numbers, and their catch

Research using the SimpleQA benchmark — a set of factual questions with clearly verifiable answers — tested AI Overviews across 4,326 real Google searches and found them roughly 85% accurate under Gemini 2, improving to around 91% after the Gemini 3 upgrade. Those numbers sound reassuring on their own. The catch, from the same research: more than half of the sampled “correct” answers were classified as ungrounded — meaning the cited source didn’t actually support the specific claim the AI attributed to it, even though the claim itself happened to be true.

That’s a distinction worth sitting with. Accuracy and grounding are two different things. An answer can be factually correct while citing a source that never said it — which means the citation is functioning more as a plausible-looking reference than as actual evidence, some meaningful portion of the time.

Why this happens, mechanically

A few separate pieces of research point at the same underlying mechanism from different angles:

• Retrieval and citation aren’t the same step. Research on ChatGPT’s search behavior (from Zyppy, circulated through SEO industry channels) found that ChatGPT cites only around 15% of the pages it actually retrieves during a search — the other 85% get read but never referenced in the visible answer. Retrieval happening doesn’t mean citation follows.
• “Attribution gaps” are a documented, named failure mode. Academic research on chat assistants (published on arXiv, covering web search credibility and response groundedness) describes cases where a response gets generated without actually fetching external pages at all, or fetches many pages but only cites a small subset — meaning the visible citation doesn’t reliably reflect what the model actually used to construct its answer.
• Even specialized, purpose-built RAG systems aren’t immune. A Stanford RegLab study on legal AI research tools — systems built specifically around retrieval-augmented generation for legal research, a use case where accuracy matters enormously — found hallucination rates of 17-33%, despite those tools being explicitly designed around grounding answers in real, retrieved documents.

Retrieval-augmented generation reduces hallucination. It doesn’t eliminate it, and it introduces a new failure mode of its own: a citation that looks like evidence but isn’t actually connected to the claim it’s attached to.

What actually makes content more likely to get cited correctly

None of the research above gives a guaranteed formula, and I’d be skeptical of anyone claiming one does. But a few patterns are consistent with how retrieval-based systems are described as working, and with what’s held up across the AI-search content I’ve reviewed:

• State claims in a form that can be lifted cleanly. A specific, self-contained sentence stating a fact directly is easier for a retrieval system to extract and attribute correctly than the same fact buried in a longer, more conversational paragraph that requires inference to isolate.
• Keep the claim and its source close together. If a stat and its attribution are separated by several paragraphs, a system extracting a snippet is more likely to grab the number without the context needed to cite it accurately.
• Avoid claims that require synthesis to be true. A claim that’s only accurate when read alongside a caveat two paragraphs later is exactly the kind of thing that produces an ungrounded citation — the system lifts the claim, misses the caveat.
• Don’t assume being retrieved means being represented correctly. Given that ChatGPT cites roughly 15% of what it retrieves, and that ungrounded citations affect a real share of even “correct” AI Overview answers, there’s a meaningful gap between “the AI found my page” and “the AI represented my page’s claim accurately.” That gap isn’t something content alone can fully close — it’s a property of how these systems currently work.

Why this problem is genuinely hard to fully solve

It’s worth understanding why grounding remains imperfect even in systems explicitly designed around retrieval, rather than assuming it’s simply a bug that will get patched away soon. Retrieval-augmented generation works by fetching relevant passages and feeding them to a language model as context — but the model still generates its final answer through the same underlying text-generation process it always uses, which means it can still blend, paraphrase, or subtly misattribute what it was given, even with the correct source material sitting directly in front of it. Better retrieval improves the odds of finding the right source. It doesn’t fully guarantee the generation step will represent that source accurately once it’s found — those are two separate points of failure, and current systems have measurably improved the first without fully solving the second.

What I’d actually take from this

The honest conclusion isn’t “here’s how to guarantee accurate AI citations” — nobody, including the researchers studying this, has that. It’s that AI-generated citations deserve the same skepticism you’d apply to an uncredited claim, both as a reader and as a site owner watching your own content get referenced. If you’re monitoring AI Overview citations of your own site, checking whether the actual claim matches what your page says — not just whether your URL shows up — is worth the extra step. And if you’re a reader relying on an AI Overview’s citation as verification of a fact, the research here suggests that’s a meaningfully weaker guarantee than it currently feels like.