Enterprise AI governance should follow the authority a system is given

The internal assistant began as a way to find policy documents. The next proposal lets it update a customer record. The same model and interface may remain, but the business has changed what the system is authorised to do.

For an enterprise AI lead, that change is the centre of governance. Approval should describe a bounded capability: its users, information, permitted actions and operating conditions. A generic approval for “AI” is too broad to explain what the organisation has accepted.

Existing access becomes part of the AI decision

Microsoft says Microsoft 365 Copilot surfaces organisational data that a user has permission to view. That product boundary makes an existing access decision newly important: information can become easier to find without the underlying permissions changing.

The governance implication is to examine whether those permissions still express the intended business policy. A pilot should not treat a technically permitted answer as automatically appropriate for the workflow. Data owners and identity teams need a role where the use case exposes old access assumptions.

The same reasoning applies to an action tool. If a system can invoke an existing business API, the team must examine the authority carried by that connection, not just the quality of the model’s wording. Separate the ability to propose a change from the ability to execute it.

Govern change at the level that changes the consequence

Not every prompt edit deserves a committee meeting. A new data source, user population or action permission may deserve more scrutiny than a cosmetic interface change. Define those triggers so teams can move quickly within an agreed boundary and recognise when they have crossed it.

Keep a bounded route for low-consequence experimentation, with appropriate data and access restrictions. Governance that makes every experiment equally expensive encourages avoidance; governance that treats every successful demo as production approval misses the operating risk. The useful middle ground is a clear progression of authority supported by evidence.

Give data owners a decision they can make

A data owner needs to know which information the workflow uses, who receives the result and what happens next. That makes it possible to approve a defined use, require a narrower dataset or identify a missing access control. Record those conditions with the use case so a later expansion does not quietly inherit the original approval.

Use a framework without outsourcing the judgment

NIST’s AI Risk Management Framework is voluntary guidance for incorporating trustworthiness into AI design, use and evaluation. NIST also provides a generative-AI profile and notes that AI RMF 1.0 is being revised. It provides a useful common vocabulary for reviewing the deployment.

Our suggested operating model translates governance into five decisions: permission to experiment, permission to use particular data, permission to deploy, permission to expand the system’s authority and permission to continue operating after a material change. Each needs an owner and an evidence requirement.

Agree what would make the pilot fail

A pilot becomes difficult to assess when its success criteria are invented after the demonstration. Define the tasks the system must perform and what an unacceptable answer or action looks like. Include normal cases, awkward inputs and situations where it should decline or escalate.

For a hypothetical internal policy assistant, a useful test might require answers to refer to an authorised, current policy document. Reviewers would check whether the cited passage supports the answer, rather than counting the presence of a link as success.

Choose thresholds that fit the consequence of error. An internal brainstorming tool may tolerate mistakes that would be unacceptable in a system executing transactions. Document who accepts the remaining limitations and how users will encounter them in the workflow.

Make intervention practical

“Human oversight” is incomplete unless the person has enough context, time and authority to intervene. Identify which actions require approval, which can proceed automatically and which are prohibited. Test what happens when the approver is absent or the system cannot explain its proposed action.

Give the operating team a way to restrict or stop the service without relying on the same failing workflow. Record how affected work is recovered and who communicates with users. Treat these as operational requirements to test, rather than phrases to add to a policy.

After deployment, assign responsibility for changes in the underlying model, data sources and connected tools. A release that improves one task can still alter behaviour elsewhere; the organisation needs a decision about when evaluation must be repeated.

The first useful governance record

Start with one live or proposed use case. Record its owner, permitted scope, evidence, known limitations, intervention route and next review trigger. The record is valuable when it changes a deployment decision. Merely counting how many records exist will not show whether the organisation is controlling the risks it has identified.

Start with a use-case inventory that supports real decisions, then apply the evaluation method to the tasks that matter. The operating model earns its place when it helps a team decide what can proceed, what needs a narrower scope and what evidence would change that answer.