Citations that can be wrong: grounding a RAG answer so the claim and the source actually match

Why does a RAG system cite a real document and still say something it does not support?

Because a citation is a second claim, and almost nothing in a standard RAG pipeline checks it.

The generation step produces an answer and a set of source markers. The answer is conditioned on the retrieved passages, which makes it usually consistent with them. The markers are produced by the same generation process — they are text the model emitted, not a computed link back to the span that supports the sentence. When the two drift apart, nothing in the pipeline notices, and the interface presents the result with the visual grammar of a footnoted document, which is the most trust-inducing format in professional writing.

That is the problem in one line: the citation UI promises verification the system never performed.

This is engineering guidance grounded in production delivery of retrieval systems. It is not legal advice. Where a regulated decision depends on the output, the obligations discussed at the end are yours and your counsel's to apply.

Four different things people call a "citation hallucination"

They have different causes and different fixes, and treating them as one problem is why teams add a reranker and are surprised when nothing improves.

1. The source does not exist. The classic fabricated reference — a plausible document id, a real-sounding case name, a page number for a page that was never written. In a closed-corpus RAG system this is the rarest failure, because the model is choosing among passages you handed it. It is common in systems that let the model cite from parametric memory as well as from retrieval, and the fix is structural: citations may only be emitted as identifiers of retrieved chunks, validated against the retrieval set before rendering. A citation that does not resolve to a chunk id in this request is dropped, not shown.

2. The document is real, but it does not support the claim. The dominant failure in practice, and the one everyone under-tests. The retrieved passage is topically adjacent — it mentions the same entity, the same regulation, the same product — but the specific proposition in the sentence is not in it. Reviewers accept these at a high rate, because checking requires reading the source properly and the citation is already there implying someone did.

3. The document supports the claim, but not any more. A superseded policy, a rate that changed, a deadline that moved. The retrieval was correct on similarity and wrong on time. Corpora accumulate this silently, because nothing about an outdated document looks outdated to a vector search.

4. The claim is right and the marker is on the wrong sentence. Answer-level citations attached to a multi-claim paragraph, where two of the four assertions come from the cited chunk and the other two came from somewhere else, or from nowhere. Reads as fully sourced. Is not.

Faithfulness scores measure the generator, not the truth

The standard groundedness pipeline — decompose the answer into atomic claims, run an entailment check of each claim against the retrieved context, score the proportion supported — is a genuinely useful instrument, and it is measuring one specific thing: whether the generator stayed inside the passages it was given.

It says nothing about whether those passages were right. A system can score 0.95 faithfulness and produce a wrong answer, faithfully grounded in a stale or incorrect document. Failure 3 above is invisible to it by construction, and failure 2 is only caught to the extent the entailment check is stricter than the human reading it.

Two further cautions about the instrument itself, because it is widely deployed as though it were a measurement rather than an estimate:

• The pipeline is brittle in its middle. Claim extraction is itself a model call. When decomposition produces vague or compound claims, the entailment step grades something other than what the answer asserted, and the score stays confidently in range.
• LLM-as-judge is uncalibrated unless you calibrate it. The common frameworks judge with a model and report a number without an established baseline accuracy, without inter-judge agreement, and with the judge's own biases unmeasured. Independent evaluation has found that frameworks in this family struggle to distinguish factually wrong context from correct context at all. That is not an argument against using them — it is an argument for holding out a human-labelled set and periodically checking your judge against it, the same way you would validate any other classifier you did not train.

I have deliberately not quoted a headline percentage for how much any technique reduces citation errors. The numbers circulating are mostly from vendor posts and secondary summaries, on corpora unlike yours, and reproducing them in a post about citation discipline would be a joke at my own expense.

What to build instead

Five things, in the order they pay off.

Make citations structural, not generated. The generator selects among chunk identifiers it was given; the rendering layer resolves those identifiers and drops anything that does not resolve. This eliminates failure 1 entirely — not reduces it, eliminates it — and it costs a validation pass. Any architecture where the model can emit a free-text reference is one that will occasionally invent one.

Cite at claim level, not answer level. Sub-sentence or per-claim attribution is more work in the prompt and in the UI, and it is what makes failure 4 visible rather than absorbed. If a sentence contains an assertion no chunk supports, that has to be representable in your output format. If your schema only allows citations on paragraphs, unsupported claims have nowhere to show up.

Verify entailment separately from generation, with a different model. After generation, check each claim against the specific passage it cites — a small cross-encoder or NLI model is well suited and much cheaper than the generator. Below a calibrated threshold, do not silently keep the citation: either retract it or surface the claim as unverified. The important word is calibrated: pick the threshold against a labelled set from your own corpus, and revisit it when the corpus changes. This is the direct attack on failure 2, and it is the highest-value component in the list.

Give time its own treatment. Similarity search will not solve staleness. Carry effective dates and supersession relationships as metadata, filter or down-weight on them at query time, and make the document's date visible in the citation. For anything regulatory, an explicit "as of" in the answer is worth more than another point of retrieval accuracy. This is the only real defence against failure 3, and it is a corpus-management problem wearing a retrieval costume.

Ship the refusal path. The most under-built component in production RAG: a system that can say the corpus does not support an answer to this. It requires a threshold, a distinct response mode, and — the hard part — an interface and a stakeholder expectation that treat refusal as correct behaviour rather than as a failed query. Systems without it convert every retrieval miss into a confident, well-formatted, wrong answer. Refusal rate is a metric to watch in both directions: zero is a bad sign.

Test it the way you would test isolation

The pattern we keep returning to: a property nobody can falsify is a belief, not a property. The tests that earn their place here are adversarial by construction.

Build a small evaluation set from your own corpus, with deliberate traps rather than representative questions: a question whose answer is only in a superseded document; a question whose answer is genuinely absent, where the correct output is refusal; a question where the topically nearest chunk contains a subtly different proposition than the one asked about. Then assert the behaviour, not the score — this query must refuse, this citation must resolve to chunk X, this claim must be flagged unverified.

Ordinary evaluation sets are built from questions the corpus answers well, which is why they pass. And run the check in CI on the pipeline, not once during a proof of concept: chunking changes, embedding model changes, and a prompt edit all move grounding behaviour, and none of them announces it.

The same discipline shows up in the tenant boundary — assert the negative deliberately, then make the assertion something CI can run.

Where this stops being an engineering preference

Once a retrieval system informs decisions about people, the citation record stops being a UX nicety and becomes the evidence layer.

Article 86 of the AI Act gives a person adversely affected by a decision made on the basis of an Annex III high-risk system the right to a clear and meaningful explanation of the system's role and the main elements of the decision. Article 26(6) requires deployers to retain the automatically generated logs. An answer that cannot say which passages drove it cannot support either — and neither can be retrofitted, because the information was never recorded. We wrote up the full obligation set in Article 26 read as an engineering backlog.

Put plainly: the same structured record that lets you debug a retrieval system is the record that answers a regulator. Teams that build it for the first reason are ready for the second by accident, and teams that build it for compliance alone usually build something that satisfies neither.

Where we stand

We default to claim-level citations resolved against the retrieval set, dated sources, and a refusal path, because the alternative is a system whose errors are indistinguishable from its successes at a glance — which makes it unreviewable, and an unreviewable system is one we cannot honestly hand to a client in a regulated setting.

The related pieces: multi-tenant isolation for the boundary underneath retrieval, private LLM inference for where the model runs, and our security and compliance pages for the rest. If you have a RAG system in front of users and no answer to "how do you know the citation is right", that is a short conversation to start.