AI data readiness includes the document you have just revoked

The revealing test for an enterprise AI search service is a document that was available yesterday and restricted this morning. Can a user who has lost access still retrieve a passage, see it in a cached answer or discover it through another query route?

For the data platform lead, that question changes the meaning of readiness. Clean text and a functioning index are necessary inputs, but the service also needs to preserve the organisation’s decisions about use, authority and access as they change. Otherwise the AI project creates a parallel information system whose rules drift from the source.

Choose the information the task can rely on

Begin with a specific task and a bounded collection. An assistant answering service-policy questions needs the applicable policy, its effective date and the exceptions relevant to the user. Uploading every old presentation and discussion thread can make retrieval broader while making authoritative answers harder to identify.

Ask the data owner which version governs the decision. Preserve the source identifier, version or update marker, ownership and validity information through ingestion. Where a document is split into chunks, each chunk needs enough context to connect it to the right source and interpretation. A paragraph detached from its heading or qualification can support the wrong answer even when copied accurately.

Separate document quality from task suitability. A complete contract may still require interpretation that the proposed service cannot provide reliably. A well-written procedure may describe a retired system. Test whether the collection contains what the user needs, whether conflicting material is distinguishable and how the service handles an answer it cannot establish.

The commercial benefit depends on this discipline. More indexed material increases the maintenance surface and may increase the work required to evaluate answers. A smaller, authoritative collection can be a better starting point when it supports a valuable, recurring task.

Decide where access is actually enforced

Microsoft’s Azure AI Search security-filter pattern illustrates a consequential implementation detail. For solutions using that pattern rather than built-in ACL support, principal identifiers are stored as strings and used to filter results. Microsoft explains that the string itself does not authenticate or authorise the caller; the security filter must be applied to every query.

The application therefore needs a trustworthy route from the authenticated user to the permitted principals and the query it submits. A browser-supplied group identifier must not become an unverified entitlement. Test the actual application path, including direct search, follow-up questions and any alternative endpoints exposed to users.

Also distinguish the ingestion identity from the querying user. A connector may need broad source access to build the index. That does not mean every person using the assistant should inherit the connector’s view. The retrieval step must select material the current user is entitled to receive before it is passed into answer generation.

In a hypothetical pilot, an employee can read a project document on Monday. On Tuesday, the source owner removes their access, but the index still carries Monday’s permission data. A successful sign-in on Tuesday proves only the employee’s identity. It does not prove that the retrieved material reflects Tuesday’s authorisation.

Set a tolerable propagation interval for this collection and test it. Include a failed update, a group-membership change and a document moved into a more restricted location. Decide what the service does when permission freshness cannot be established. Sensitive material may need to be withheld until the state is known, even if that temporarily reduces answer coverage.

Deletion and freshness need separate mechanisms

Include derived stores in the deletion workflow across copies, holds and restores. Removing a source document does not establish what happened to its indexed passages or cached answers.

Change detection is not the same as deletion detection. For Azure Storage indexers, Microsoft documents automatic change detection using timestamps, while deletion requires a configured strategy. Its guidance warns that documents deleted before a deletion policy was established can remain in the index even after that policy is added and the indexer reset. For native blob soft delete, the indexer must process the blob while it is still in the soft-deleted state.

This is why deleting a source file cannot be the only acceptance test. Follow its identifier into indexed documents, derived chunks, embeddings, caches and stored conversation content. Identify which stores can reproduce its content and what removal or restricted-access mechanism applies to each. A document may be removed from retrieval while an earlier answer still contains the relevant passage.

Design the lifecycle before importing the full collection. Keep a mapping from source records to derived records, observe failed synchronisation and test recovery after a missed deletion event. Where retained records have a legitimate purpose, define their restricted use and retention rather than describing the entire data flow as erased.

Freshness needs a business meaning too. The most recent ingestion time says when the pipeline ran, not necessarily when the information became valid. A newly uploaded copy of an obsolete policy is still obsolete. Test a policy replacement through retrieval, answer generation and citation so the user can distinguish the current instruction from historical context.

Approve a data path that can keep working

Use representative users to test permitted retrieval, denied retrieval, changed permission, updated content and removal. Inspect the passages actually supplied to the model as well as the final answer. A plausible response can conceal an inappropriate source selection.

Assign responsibility for the ongoing path: source quality, connector operation, access mapping, failed updates and derived-store cleanup. The platform team cannot repair a disputed policy meaning, and the policy owner cannot repair a stalled indexer. The handoff needs to be explicit enough for a failure to reach the right team.

Start production with the collection whose behaviour you can demonstrate, and expand as those controls hold under change. Complete the boundary with the AI provider’s product-specific data controls. Internal permission-aware retrieval and the provider’s handling of submitted information answer different parts of the same deployment decision.