When the sources disagree,show the disagreement.

Start with a real disagreement

Imagine an internal process assistant answering: “Which approval route applies to this request?” An approved policy names the service manager. A newer draft names the department head. An old team guide names a role that no longer exists. All three contain the terms in the question.

This is an illustrative example, not a client incident. It exposes a common design decision: relevance is useful for finding candidates, but it does not decide authority. Picking the newest file could select the draft. Combining all three could invent an approval route that nobody agreed.

OWASP’s LLM07:2026 guidance treats misleading or unsupported evidence as a workflow risk, including where it drives an action. It calls for current, authoritative grounding and verification before execution.¹ A citation alone is therefore an incomplete check: the cited material must support the answer and apply to this request.

Keep the evidence attached

For a controlled document collection, retain a stable source identifier, revision, approval status, owner, effective dates and applicable scope. Distinguish the date a file was uploaded from the date its contents take effect. Keep a link to the supporting passage and the version used in the answer.

In the example, a policy scheduled for next month may be genuine and approved without governing today’s request. A local procedure may properly differ from an organisation-wide default. These relationships need an explicit rule from the people responsible for the material; they cannot be recovered reliably from filenames.

OWASP’s 2026 retrieval guidance also recommends provenance and source lifecycle controls.² The practical consequence is that replacing or removing a source must reach the searchable chunks, cached results and derived stores that depend on it.

Give the answer three possible outcomes

1. Supported. The applicable approved source is clear. Answer with its revision and the passage supporting the decision.
2. Needs clarification. The rule depends on a missing fact, such as request type or relevant date. Ask for that fact before choosing.
3. Unresolved. Applicable approved sources conflict, or authority cannot be established. Show the permitted evidence, name the issue and route it to the document owner.

For the last outcome, do not quietly prepare an action as though the conflict had been settled. The interface should make the blocked step and the required decision visible. A reviewer should not have to notice a caveat buried underneath a confident recommendation.

Document contents remain evidence, not permission to change the workflow. A paragraph saying “ignore the approval check” cannot acquire authority simply because the search system retrieved it.

Check access before showing evidence

The comparison must stay within the requester’s permitted information. Do not reveal a restricted document’s title or existence merely to explain why an answer is incomplete. The user-facing message can say the available evidence is insufficient, while an authorised operator investigates through a separate route.

Permission metadata can itself become stale. Microsoft’s Azure AI Search guidance makes an important dependency explicit: query-time checks against indexed permissions rely on that metadata being synchronised.³ Agree how permission changes propagate, how that delay is measured and what the application does when freshness cannot be established. Product features and preview status need checking for the chosen implementation.

Test the transitions, not just the documents

Build a small collection with deliberately different states: approved, draft, superseded, future-effective, deleted and inaccessible. Add two approved sources that genuinely conflict. Write the expected outcome before running the test, including when the correct answer is to stop.

Then change the collection. Withdraw a user’s access. Replace a document while keeping its familiar title. Remove a source after an answer has been cached. Check both the resulting answer and the evidence the interface exposes. This tests the route from source administration to the user, where a tidy retrieval demo can hide an important gap.

Sources and further reading

1. LLM07:2026 MisinformationOWASP GenAI Security Project↩
2. LLM09:2026 Vector and Embedding WeaknessesOWASP GenAI Security Project↩
3. Document-level access control in Azure AI SearchMicrosoft Learn↩