Enterprise AI governance: make the controls reach the work

Quick brief

Enterprise AI governance needs an accountable chain from the business decision and its applicable obligations to the permissions, evidence and recovery mechanisms in the running service. Prove that chain through consequential workflows, then expand with explicit decision rights, supplier responsibilities and tested operating controls.

Full analysis

Revised 18 October 2026.

A supplier-review assistant has a current policy, an approved model, and access to the purchasing system. It prepares a convincing recommendation. Someone then asks it to change the supplier’s bank details.

The organization needs to establish which part of the system decides whether that action is permitted, who can approve it, and what evidence they must see. It must also account for the possibility that the underlying record changes before the update completes. The policy may describe the required decision, but the running workflow still needs to enforce it.

Start an enterprise AI governance review by following one consequential piece of work from the original request to the destination record. The review should identify where authority is granted, facts may become stale, people intervene and the organization needs to explain what happened. Those are the points where controls need to operate.

Download the printable framework map.

Turn the board’s questions into evidence

A board needs a different view from a developer investigating a failed tool call. It still needs answers that can be traced to the work. An inventory of model names says little about which decisions the organization has delegated or which customers could be affected. A policy can establish human oversight while leaving the reviewer with neither the information nor the authority to intervene.

Bring a small evidence pack to that conversation. It should expose uncertainty and ownership, rather than make every application look equally mature.

Question	Evidence to bring	What it does not establish
Where can AI take consequential action?	Workflow catalog linked to actual operations, identities, destinations and authorization modes	A catalog does not prove permissions match its description
Who owns an incorrect decision?	Business owner, operational stop authority and incident route	Committee participation does not assign execution responsibility
How are affected people considered?	Use-specific evaluations, relevant group analysis where appropriate, complaints and review arrangements	Aggregate accuracy does not establish fairness
Can a person intervene effectively?	Demonstrated rejection, correction, escalation and stop paths, with reviewer workload	An Approve button does not prove effective oversight
Which obligations apply?	Legal entity, intended use, jurisdiction, role and obligation-to-control map	A generic certificate does not classify this use
Can we reconstruct an action?	Connected source, approval, execution and destination evidence from a completed case	A large archive does not prove authorization

An application awaiting source-owner approval should appear as unresolved even if its model evaluation is strong. A drafting service may be ready while its payment-changing extension remains blocked. Reporting at the workflow level makes that distinction possible.

Why a documented control can disappear in operation

A policy can be well intentioned and still have no effective path into the service. Release incentives may favor visible features while source maintenance and evidence capture remain unfunded. A periodic committee can review the approved design without seeing a changed tool permission or newly enabled supplier feature. These are conditions to investigate, not assumptions about every leadership team.

Separate the policy’s author, the person implementing its control and the person verifying its operation. If a release depends on an approval check, engineering needs an acceptance test, and operations needs a failure signal. Governance needs evidence from that path rather than a second assurance that the policy was circulated. A small review team cannot compensate for unlimited uncontrolled release paths by reading more questionnaires.

Data handling makes this concrete. A rule prohibiting unauthorized disclosure should become an access decision on the actual source and destination, with separation of tenants and purposes. A rule limiting training use needs controls on the training data path and relevant supplier terms/configuration. Test cross-customer retrieval and unauthorized export at the boundary. Redaction can minimize exposure, but a text filter does not replace source access enforcement or prove every identifier was removed.

Give the workflow an accountable owner

For the supplier example, name the business owner responsible for the outcome and the operational owner who can stop the service. Security, legal, privacy, procurement, and engineering contribute different judgments; their participation does not remove the need for someone to own the complete workflow.

Write a short decision record before selecting the technology. State which cases the service may handle, the evidence required, the actions it may propose or execute, and the conditions that require a person. Include the owner of the source policy and the person responsible for a failed or disputed action. A model inventory alone does not contain those answers.

Use the decision record to classify the actual use. Drafting a supplier summary, recommending an exception, and changing payment instructions create different consequences even when they share a model. The review should reflect that difference. A low-impact drafting tool should not inherit an expensive approval process merely because another application can move money; equally, a drafting approval should not silently become permission to execute.

The NIST AI Risk Management Framework gives organizations the Govern, Map, Measure, and Manage functions for organizing risk work. Its Generative AI Profile, published in July 2024, adds suggested actions for generative AI risks. These are useful voluntary resources. Adopting their vocabulary does not establish that a particular workflow is safe or legally compliant.

Find the AI the organization did not build

The responsibility chain extends beyond the team running a model. A model developer supplies a component; an application vendor decides how it is used; an integrator connects enterprise data and operations; the deploying organization chooses a business purpose and the people who will rely on the result. One company may perform several functions. These commercial labels do not, by themselves, settle its legal role.

Embedded SaaS features make discovery harder. An approved purchasing application can acquire a summarizer, a recommended-action panel or an automated agent after its original procurement review. An employee can paste records into an unregistered service. Asking only which foundation models the organization hosts will miss both paths.

Start discovery with business processes and information flows, drawing on procurement’s contract changes and renewal notices alongside the enabled features and release notes maintained by application owners. Identity and security teams can contribute authorized integrations and relevant usage evidence within established privacy and monitoring rules. Process owners supply another view: where recommendations actually enter their work, including services bought by a business unit. Reconcile these incomplete accounts into a workflow register and assign unresolved entries an owner.

Record the difference between a feature being available, enabled, used and allowed to act. Otherwise, an inventory becomes either a list of speculative capabilities or a false assurance that everything in the tenant is controlled. Repeat discovery when suppliers, configurations and business uses change.

Make supplier answers change a release decision

“We use responsible AI” does not tell a buyer whether records are retained for training, whether an agent can change payment instructions, or whether evidence can be exported after an incident. Procurement needs evidence relevant to the implementation.

Area	Evidence to request	Enterprise decision
Architecture	Components, subprocessors, data destinations and processing boundaries	Which intended records may enter those paths
Service changes	Version identification, notice, evaluation scope and migration or rollback arrangements	Which changes trigger local retesting
Data use	Contractual terms, operational settings, retention and deletion exceptions	Whether actual configuration meets the use’s requirements
Tool authority	Operations, identity propagation and authorization controls	Which capabilities to enable or withhold
Evaluation	Relevant tasks, failure cases, denominators, conditions and limitations	What still needs testing with enterprise cases
Incident support	Notification, investigation assistance and evidence availability	Whether a failure can be investigated and contained
Exit	Export formats, dependency ownership, revocation and data return	How to stop or replace the service

Keep unanswered questions with the release record. A supplier’s refusal to disclose an internal detail does not automatically prohibit every use, but it leaves a risk for a named owner to accept, mitigate or decline. A contract can allocate responsibilities and remedies. It cannot make an untested control operate or transfer every statutory obligation away from the deploying enterprise.

Map obligations to uses, roles and evidence

Regulatory work belongs alongside architecture because intended use affects what the service must support. The relevant legal and sector specialists should assess it. An internal label of “high risk” is not a legal determination, and a low internal rating does not displace an applicable rule.

For the EU AI Act, distinguish a general-purpose model from the system around it, and distinguish provider from deployer. Under Article 25, putting a name or trademark on a high-risk system, substantially modifying it while it remains high-risk, or changing a system’s intended purpose so that it becomes high-risk can shift responsibility to the provider of that high-risk system. Whether an organization becomes a general-purpose AI model provider is a separate Chapter V question. Ordinary prompting or API use does not by itself determine that role. Examine actual activities, agreements and the system boundary.

Under the amended Article 113 timetable, Chapter III Sections 1–3, except Article 6(5), apply from 2 December 2027 for the Annex III high-risk route and 2 August 2028 for the Article 6(1) product route. The amendment is Regulation (EU) 2026/1744, which entered into force on 27 July 2026. The Commission’s Service Desk reproduces consolidated text; the Official Journal acts are the authoritative publications. Other provisions already apply, with transitions and exceptions requiring separate assessment. The added prohibitions in Article 5(1)(ba) and (bb), and Article 5(1a) and (1b), apply from 2 December 2026. Future deadlines do not postpone controls needed for today’s operation. The full EU AI Act analysis examines the classifications, roles and evidence requirements in greater detail.

Join each applicable requirement to an artifact, an enforcement or operating point, and a responsible person. Documentation, operational logs, oversight and retention are different concerns. Do not turn one documentation retention period into a rule to keep every prompt. Existing sector, privacy and security controls may supply evidence; record what they cover and what remains unresolved.

Context	Primary reference and scope	Design question
US banking model risk	Interagency guidance from the Federal Reserve, OCC and FDIC, transmitted through SR 26-2, April 2026, supersedes SR 11-7 and SR 21-8; nonbinding risk-based guidance expected to be most relevant to banking organizations above $30 billion in total assets. Its attachment, footnote 3, expressly excludes generative and agentic AI models	Which quantitative models enter the model-risk program, and how does the bank’s broader governance cover generative or agentic components outside this guidance’s scope?
Credit decisions	Regulation B, §1002.9, includes notification and specific-reason requirements, with provisions differing by application context	Can the service support the applicable notice and actual reasons, rather than generate a plausible explanation afterward?
Insurance	The NAIC model bulletin describes a risk-proportionate written AI systems program; it is not automatically law in every state	Which actual state instruments apply to underwriting, claims and suppliers?
Clinical decision support	FDA’s final CDS guidance of 29 January 2026, superseding 6 January, interprets four cumulative statutory criteria for certain non-device CDS	What is the intended use, does each criterion hold, and can the professional independently review the recommendation’s basis?
New York City employment	Local Law 144 guidance covers defined automated employment decision tools, with audit, publication and notice requirements	Does this use meet the definition, and are the required evidence and notices maintained?

For CDS, independent review is one part of the test. The guidance's four-criteria discussion also addresses the exclusion of specified image and signal analysis, the medical information handled, and recommendations to a healthcare professional. All four criteria must hold for the statutory exclusion being discussed. The guidance explains FDA’s interpretation; it does not create a blanket exemption for software with a clinician in the loop.

This selected map shows why “our AI policy covers it” is an inadequate answer. Within one sector, drafting, deciding a credit application and acting on an account can create different obligations. Keep the classification rationale, specialist approval, assumptions and changes that would reopen the assessment.

Classify the intended use before designing its evidence

Article 6 has distinct product and Annex III routes, with conditions and exceptions. A department name is insufficient. Under Annex III, natural-person creditworthiness and life/health-insurance risk assessment or pricing have specified categories; financial-fraud detection has a particular credit-category exception. Emergency healthcare triage is specified too. These provisions do not make every banking or hospital application high-risk.

Consider hypothetical services sharing a platform: one summarizes corporate financial statements, one evaluates an individual’s creditworthiness and one flags payment fraud. Their shared model or gateway does not give them an identical legal classification. Record each purpose, affected population and downstream use, then assess the actual provision and any exception. An advisory label or approval rate cannot, by itself, determine the legal result.

For deploying organizations, Article 26 addresses use according to instructions, competent, supported oversight, relevant input data to the extent controlled, monitoring and specified risk/incident responses. Log obligations concern records under the deployer’s control, with stated retention conditions. These duties differ from provider responsibilities. Article 27 fundamental-rights impact assessment has its own actor/use scope; it is not automatically required for every enterprise deployment. That scope includes private deployers of the high-risk creditworthiness and life/health-insurance systems specified in Annex III points 5(b) and 5(c), as well as the public bodies and private public-service providers covered by Article 27.

Make that assessment actionable. A service changed from preparing an analyst’s summary to evaluating a person’s eligibility needs its classification and release assessment reopened before the team relies on the old approval. If provider instructions specify input conditions, assign who checks them and what happens outside those conditions. A classification record disconnected from change management can become stale despite sound original reasoning.

Assemble a technical file that explains a released system

Annex IV specifies applicable documentation including purpose and versions, interactions and architecture, development and data, oversight, performance, risk management and changes. It is more than a model card. The following is a proposed engineering organization of evidence, not an exhaustive legal template.

Evidence question	Illustrative artifact	Connection to operation
What is approved?	Purpose, excluded uses, populations and release boundary	Live enabled operations
What contributes?	Architecture, supplier dependencies and data flows	Component/release references
What was evaluated?	Source lineage, evaluation design and limitations	Evaluated configuration and subsequent changes
What can people control?	Correction, escalation, stop and reviewer design	Observed tests and operating capacity
What risks remain?	Assessment, mitigation and residual-risk owner	Control tests, monitoring and incident route
What reopens review?	Change rules, approval and release history	Model, prompt, retrieval, policy, tool and purpose changes

In the supplier example, imagine release A only proposes a change and release B adds commitment. The file should show the new operation, identity, approval, data path, failure behavior and evaluation. Reusing release A’s approval because the model is unchanged would miss the architectural difference. Equally, a cosmetic edit need not receive the same review as a new payment capability. Define change categories, escalation and the evidence needed to distinguish them.

Keep approved release and live configuration connected. A supplier update, prompt edit or replacement retrieval generation can affect behavior even when the visible application version stays the same. Compare the change with the approved boundary. A content digest identifies bytes; it does not establish approval, applicability or adequacy.

Choose the conformity route and manage supplier dependencies

Article 43 specifies internal control under Annex VI for Annex III points 2–8. Biometrics under point 1 has separate conditions for internal control and notified-body involvement. Systems covered by Annex I Section A follow the relevant product-law procedure with the AI requirements integrated as specified. Critical infrastructure under Annex III point 2 follows the internal-control route unless it is also covered by the Annex I Section A product-law route just described; serious consequences alone do not require a notified body.

A substantial modification can require a new assessment; specified predetermined documented learning changes are treated differently. Tie change assessment to the released system rather than inventing a universal prompt-change or fine-tuning threshold. Ask which purpose, requirement or approved boundary changes and retain the qualified decision.

An upstream general-purpose model is another dependency. Article 53 addresses provider documentation, downstream information, copyright policy and training-content summary, with specified exceptions. Systemic-risk obligations are separate. Identify what supplier information is available, what system questions it answers and what remains for the enterprise to evaluate. Upstream compliance does not make the complete application compliant, and downstream buyers should not assume entitlement to every internal training artifact.

Put that dependency in the plan: who requests information, what is missing, can the intended use proceed without it, and which changes require notice? Supplier and enterprise evidence should meet at a defined system boundary. Procurement, engineering and legal need that agreement before an unavailable document becomes a late surprise.

Preserve sector update and outcome responsibilities

The banking map distinguishes quantitative model-risk guidance from generative and agentic components outside its scope. A composite workflow can contain both. Inventory those components and map broader controls to the complete process, including sources, rule changes, applicable adverse-action reasons and destination permissions. Exclusion from one nonbinding guidance does not establish that no other obligation or governance work applies.

For healthcare, intended-use/device assessment precedes a claim that an update is permitted. FDA’s August 2025 final PCCP guidance describes planned modifications, development/validation/implementation methodology and impact assessment within a reviewed predetermined change control plan. It is not blanket permission to modify any clinical model without further assessment.

A health system’s operating plan should identify the installed version, clinical role, local evaluation, update notice, clinical owner, fallback and appropriate outcome monitoring. Assess a changed alert threshold, input or care pathway for its actual consequences. A drift score alone cannot establish whether clinicians receive useful alerts or patients benefit. These are proposed operating questions, not new FDA requirements invented for every hospital application.

For insurance, connect applicable state expectations to actual underwriting, pricing and claims uses. Can data, supplier and evaluation records explain the decision and support investigation of a dispute? A model bulletin can guide the program; the force and scope of a state’s instrument need separate assessment. An adoption count is less useful than knowing which rule applies to this product and decision.

Colorado illustrates why a model bulletin and a state insurance requirement belong in separate entries. In August 2025, its Division of Insurance announced amended Regulation 10-1-1, effective 15 October 2025, covering specified insurers’ use of external consumer data and information sources and models using those sources. The announced scope included individual life insurance, private passenger automobile insurance and health benefit plans. For the operating map, identify the insurance line, the external data and model dependency, the responsible owner and the governing instrument. Check the current rule and any later guidance before setting a reporting or testing deadline; the general Colorado automated-decision law is a separate assessment.

Put transparency into the actual user experience

Article 50 assigns different transparency duties to specified actors and uses, with exceptions. It addresses AI interaction notices, provider marking of certain synthetic outputs and deployer disclosures for specified uses such as deepfakes. It does not require an identical label on every generated sentence or classify every conversational service as high-risk. Check the particular provision, exception and transition that applies. Under Article 111(4), providers of systems generating synthetic audio, images, video or text that were placed on the market before 2 August 2026 must take the necessary steps to comply with Article 50(2) by 2 December 2026.

For an enterprise interface, turn that assessment into observable behavior. If this service must inform a person that they are interacting with AI, identify where and when the notice appears, who maintains it and how it works in voice or an embedded channel. Do not assume a notice on the main website reaches someone opening a message or using a different interface.

Test the actual journey, including the first interaction, smaller screens, assistive access and handoff to a person where offered. For generated media, retain the necessary provenance/marking and disclosure behavior through export or redistribution as applicable. These are implementation tests for the assessed obligation, not a declaration that one footer satisfies every transparency rule.

Penalties are a reason to classify the breach precisely

Article 99 distinguishes prohibited-practice violations, specified other obligations and incorrect or misleading information. The headline ceilings include €35 million/7%, €15 million/3% and €7.5 million/1%, respectively, with different application rules. SMEs, including start-ups, receive the lower amount-or-percentage ceiling across those tiers; small mid-cap enterprises receive that treatment for the €15 million and €7.5 million tiers. These are not automatic penalties for every AI error. General-purpose model providers also have a separate Article 101 regime.

An enterprise should assess the applicable actor, obligation, timing and potential response with counsel. Operational restriction, remediation and disputed decisions can matter independently of a headline fine. The engineering contribution is to identify the affected system and cases, preserve appropriate evidence and demonstrate the control and recovery behavior. A financial maximum cannot tell a delivery team which operation to stop.

Keep jurisdiction changes out of a universal checklist

The US state and local landscape needs its own maintained map. The Colorado Attorney General's current account says SB 26-189 repealed and re-enacted the earlier provisions as an automated decision-making technology framework, effective 1 January 2027. Rulemaking remains underway in October 2026. Copying a 2025 Colorado summary into a release checklist would miss that change.

Do not assume another state’s law follows the same definition, role allocation, threshold or remedy. Record the entity and affected person’s relevant location, the decision being materially influenced, the actor’s role, applicable effective dates and the specialist’s rationale. Employment, consumer-credit, privacy and professional obligations also require examination even where a general AI statute does not apply. A state-by-state register should retain the actual instrument and revision, rather than a traffic-light color detached from its source.

California’s employment automated-decision-system regulations took effect on 1 October 2025 within its civil-rights framework. They address potential employment discrimination involving AI and other automated systems. That is a different legal route from Colorado’s general framework or New York City’s defined-tool audit regime. A nationwide hiring workflow needs a use-specific map rather than one reused legal label.

Illinois’ employment AI provisions in Public Act 103-0804 likewise operate through its Human Rights Act, including discrimination and employee-notice provisions. For an actual rollout, resolve the people and employment decisions covered, required notice behavior and current implementing requirements with the responsible specialist. Do not infer that another jurisdiction’s notice or independent-audit process satisfies these provisions.

Reuse controls through explicit mappings. An evidence store can support several investigations, but access and retention may differ. A notice component can serve several workflows, but wording and triggering conditions need their own rules. A review route may support an appeal, correction or human reconsideration, yet those are different processes with different owners. Test each promised behavior against the use and obligation that requires it.

Separate a recommendation from permission to act

In this illustrative purchasing workflow, the assistant can assemble documents and prepare a proposed change. A separate control checks whether the initiating identity may request that action, whether the destination supplier is in scope, and whether the required approval is valid. The model’s explanation is input to the review; it cannot grant itself additional authority.

Define an approval precisely enough to survive execution. An approval to update one supplier’s contact address should not authorize a change to bank details or a different supplier. Bind the approval to the relevant action and material values, give it an expiry where appropriate, and invalidate it when those values change. Keep credentials outside generated text and expose only the capabilities the workflow needs.

A good interface also shows the human what will happen. Present the destination, proposed change, supporting evidence, and material uncertainty together. Asking someone to click Approve after reading a polished paragraph is a weak control if they cannot inspect the actual update. A reviewer needs the ability and time to reject, correct, or escalate it.

Some bounded actions can operate under a standing authorization rather than a fresh click. That is an operating decision with a defined scope, accountable owner, monitoring, and revocation. The important distinction is whether the authorization exists and is enforced, not whether every action has a ceremonial human confirmation.

Choose an autonomy mode with explicit decision rights

Three modes provide useful operating vocabulary. They describe who may authorize a consequential step, rather than three certifications of safety. A workflow can use different modes for different operations.

Mode	Service authority	Human responsibility	Evidence before expansion
Advisory	Retrieve, analyze and propose; no protected update	Evaluate the proposal and separately authorize action	Recommendation quality, applicable sources and separation from execution
Review before commitment	Prepare an action awaiting scoped approval	Inspect material values and evidence; approve, correct or reject	Approval binding, changed-case rejection and reviewer capacity
Bounded standing authorization	Execute only preauthorized operations and cases	Define the envelope, monitor, revoke and handle exceptions	Enforced limits, exception routing, stop tests and recovery

Bounded authorization does not mean “no humans.” People remain responsible for scope, release and operation. Conversely, adding a person to every transaction can create an unmanageable queue and superficial approvals. Test oversight at expected workload. Include people who understand the consequences for affected customers or employees, alongside the software’s owners.

This configuration sketch uses invented identifiers and no bank details. It avoids universal confidence cutoffs or arbitrary retention constants.

{
  "workflow": "supplier-bank-change-example",
  "mode": "review_before_commit",
  "proposal_operation": "prepare_bank_change",
  "protected_operation": "commit_bank_change",
  "approval_binding": [
    "supplier_id", "operation", "material_payload_digest"
  ],
  "preconditions": [
    "requester_authorized", "approval_valid", "source_applicable",
    "supplier_revision_matches", "destination_permitted"
  ],
  "unknown_destination_result": "reconcile_before_retry",
  "invalid_precondition": "withhold_commit_and_return_for_review"
}

A configuration file expresses a contract; it does not enforce one. The application must derive material values consistently, authenticate the approval’s issuer, verify its scope and validity, and check them at the protected operation. The destination must support the concurrency and duplicate-prevention mechanisms being relied on. If it cannot reject an outdated supplier revision atomically, document the remaining race and reduce or redesign the permitted action.

Assign the decisions behind those checks. The business owner defines allowed cases; security owns relevant identity and capability mechanisms; the source owner maintains policy; engineering implements and tests the boundary; operations can stop and investigate it. Legal or compliance assesses applicable obligations. A named person remains accountable for release. Splitting work among specialists should make responsibilities precise, rather than leave the spaces between them unowned.

Keep the source and the approval current together

The supplier policy can change while a recommendation is waiting. A document may be approved but not yet effective, valid for a different business unit, or superseded by a correction. “Most recently uploaded” is an inadequate rule for deciding which source applies.

Keep the source identifier, owner, approval status, applicability, version, and effective interval with the material used in the decision. Those fields require a maintained source process. A database column called current does not make the underlying policy current if nobody updates it when the owner issues a replacement.

Recheck material conditions at the action boundary. Suppose a recommendation was prepared under policy version 7 and version 8 becomes effective before execution. The workflow needs a defined response: reevaluate under the new policy, seek a renewed approval, or stop for review. Continuing silently is a design choice too, and one the business owner should have to defend.

The Version Drift framework turns that problem into a concrete test of the connection between the authoritative source, the version used, and the action being attempted. An existing content service may already support the necessary records; establish what it can enforce before adding another storage product.

Treat connected tools as part of the control boundary

An assistant’s tool access changes what a mistaken or manipulated answer can do. Read-only access to a supplier record is different from permission to alter it. Review the actual operations, identities, destinations, and data paths; a connector’s product name tells you little about its effective authority.

The MCP authorization specification addresses authorization for supported protocol interactions. It does not decide whether a purchasing manager may approve a particular bank-detail change. Transport access and business authority must meet at the application’s action boundary.

The MIT AI Risk Repository and MITRE ATLAS serve different parts of that review. The former organizes a broad range of AI risks, while ATLAS catalogs adversary tactics and techniques involving AI. Use the risk inventory to identify a concern, then develop a threat-informed test of how an attacker might exploit this particular workflow.

For the supplier case, include a document containing instructions to send records to an external address. The content may be relevant evidence about the supplier, but it must not become a new instruction that expands the assistant’s permissions. Test the attempted destination action as well as the model’s visible refusal. A reassuring answer is insufficient if a tool call already sent the material.

Turn threat categories into workflow tests

Follow information and authority through the application. These are proposed tests, not claims that attacks occurred in the illustrative service. Several failures can also arise from ordinary mistakes; test the control rather than relying on an inferred attacker intention.

Failure	Concrete path	Control to examine	Observable result
Goal hijacking	An attachment instructs the assistant to send records elsewhere	Content cannot grant capabilities or change destination policy	The external send is rejected at the operation boundary
Tool misuse	A read task becomes a bank-detail update	Separate read, propose and commit permissions	The read identity cannot commit
Delegation abuse	A helper receives broader authority than its initiating case	Scoped delegation and authenticated authority propagation	A delegated call cannot exceed the case’s scope
Memory poisoning	An untrusted note becomes a persistent policy exception	Provenance, write permissions, review and invalidation	The note cannot silently become an approved rule
Inter-agent contamination	Another agent’s summary asserts approval	Separate assertions from trusted approval records	Execution requires the authentic record
Stale authorization	Approved values change before commitment	Binding, expiry, revocation and revision checks	The changed action is withheld and recorded
Shadow integration	A newly enabled feature creates another action path	Recurring integration discovery and review	The capability cannot inherit an unrelated approval

An integration gateway can centralize checks and revocation. It also becomes a dependency and can miss actions performed through another path. An application library has different deployment and coverage tradeoffs. Choose according to the protected operation, then check for bypass through another credential or interface. A product diagram does not prove every action passes the control.

Read a vulnerability report as a chain of boundaries

The September 2025 EchoLeak case study analyzes CVE-2025-32711 in Microsoft 365 Copilot: a crafted email could cross trust boundaries and exfiltrate information through a chain involving prompt-injection defenses, links, fetched images and a permitted proxy path. It is a study of a particular vulnerability and bypass chain, not evidence that millions of users were attacked or that this remains an unpatched current condition.

Its useful lesson for this design is to test the whole disclosure path. A prompt classifier’s decision, a model’s visible response, a rendered link and the browser’s or tool’s network request are different points of control. A system can appear to refuse an instruction while a later component still requests an attacker-controlled resource. Conversely, permission to reach one service does not establish that every indirect path through it is appropriate.

For persistent context, separate an untrusted observation from an approved rule and retain its provenance. Restrict who can write durable exceptions, provide correction and invalidation, and test whether a poisoned note influences a later case. For delegated work, propagate the permitted case scope and confirm the receiving tool’s actual identity and operation. A second agent’s confident statement is not an authorization record.

Make evidence useful without recording everything

A reviewer should be able to reconstruct the request, applicable evidence and policy versions, proposed action, authorization, execution attempt, and destination result. Keep stable identifiers that connect those events. Record a human override with its reason where that is necessary to understand the decision.

This does not require retaining every prompt, private message, or hidden model reasoning. Collect the evidence needed for the business purpose and investigation, with access restrictions, retention periods, and deletion arrangements. A giant log containing unnecessary personal data creates another problem for the governance program to manage.

Give evidence an operational use. If an approval refers to a different supplier identifier than the attempted action, the mismatch should stop or redirect the attempt under a defined rule. If a destination request times out, record that the result is uncertain. Do not convert a transport timeout into proof that nothing happened and retry a consequential action blindly.

The destination may have accepted the first request before the connection failed. Use the destination’s supported duplicate-prevention and reconciliation mechanisms, then obtain a confirmed state or escalate the uncertainty. Recovery belongs in the design, alongside the successful path.

Design an audit path that survives ordinary failure

Four evidence layers separate the questions an investigation must answer. Identity identifies the initiating person and executing service. Inputs identify the case revision and applicable sources. Processing identifies the relevant configuration, proposal and authorization decisions. Outcome identifies what the destination confirmed, including uncertainty and recovery. Processing evidence describes observable steps, rather than purporting to expose hidden model reasoning.

This invented event records an uncertain result. References point to separately protected records; their presence does not prove those records are authentic or complete.

{
  "event_id": "example-event-42",
  "case_id": "example-case-17",
  "attempt_id": "example-attempt-2",
  "identity": {
    "requester_ref": "example-person-8",
    "executor_ref": "example-service-3"
  },
  "inputs": {
    "supplier_revision": "example-revision-12",
    "policy_version": "example-policy-8",
    "evidence_manifest_ref": "example-manifest-6"
  },
  "processing": {
    "release_ref": "example-release-4",
    "approval_ref": "example-approval-9",
    "material_payload_ref": "example-payload-11"
  },
  "outcome": {
    "state": "unknown",
    "reason": "destination_timeout",
    "next_step": "reconcile_destination_state"
  }
}

An audit pipeline has reliability requirements of its own. Where local state and evidence share a transactional database, an outbox can record an event in the same transaction for later delivery. It does not make an external supplier-system call part of that transaction. The external operation still needs an attempt identifier, confirmation and reconciliation.

Queue workers can validate schemas, redact, deduplicate delivery, route records and alert on material gaps. Retries need limits and an operator path for records that cannot be processed. Reconcile expected case events with delivered events: a healthy queue can coexist with a producer that never emitted the record. Monitor capture failures, delivery lag and unresolved reconciliation separately.

Decide what happens when capture fails. An operation requiring a durable authorization record should not proceed because a dashboard might notice its absence later. Losing a nonessential performance sample need not stop the business. Make that distinction in the contract and test both paths, including who may authorize an exceptional continuation.

Separate storage and access by purpose. Investigators may need protected evidence while operators need a redacted view. Documentation can have a different retention basis from execution events or temporary diagnostics. Route relevant security signals to existing incident tools with the case identifiers needed to join records, without copying sensitive payloads everywhere.

Integrity controls can make alteration detectable under stated assumptions. They cannot prove a fabricated input was true, that a missing event existed or that an answer was justified. Test access, deletion, restoration and export as well as write throughput. An archive is useful only if an authorized investigator can recover and interpret its records.

Storage tiers should serve a defined investigation

View	Typical use	Design tradeoff
Operational	Recent incidents, queues and control failures	Fast access and alerting without unrestricted sensitive payloads
Investigation	Case reconstruction, sampling and disputes	Stable joins, source references and controlled evidence access
Archive	Longer-term documented purposes and holds	Retrieval time, preservation and deletion requirements

These are functional tiers, not universal day/year retention bands. Set periods by record type, applicable requirements, purpose and holds. Test whether an investigator can find a case by decision, source version, identity, attempt and outcome, then recover the linked evidence. An immutable object store alone does not make evidence complete, legally admissible or exempt from privacy obligations.

Independent challenge and affected-party perspectives also need owners. Someone should test manipulation and attempted bypass after material changes, while the appropriate customer, employee or patient-safety function examines the consequences of error and ineffective review. These responsibilities can extend an existing accountability model; it is unnecessary to declare RACI unusable merely because the workflow contains AI.

Test the controls with deliberate changes

Before expanding the workflow, rehearse a small set of synthetic cases with the people responsible for its operation. Start with a valid request and confirmed update. Then vary one material condition at a time: an unauthorized requester, a changed supplier identifier, an expired approval, a policy replacement, a revoked capability, an untrusted instruction in a document, and an uncertain destination result.

Specify the expected outcome before running each case. An expired approval should not be accepted merely because the requested change looks reasonable. A source conflict should remain visible. A revoked permission should prevent the protected operation. Where the system must stop, check that it actually stops at the destination boundary.

Record the configuration and evidence from each test, including failures and unresolved results. Repeat tests after material model, policy, tool, or permission changes. The appropriate depth depends on the consequences and variability of the workflow; a handful of clean demonstrations does not establish a low failure rate.

Human oversight needs testing too. Can the reviewer understand the proposed change, see conflicting evidence, interrupt execution, and recover from an error? The human-oversight article examines that practical difference between assigning a person and giving that person an effective control.

A clinical validation example shows why supplier evidence is insufficient

Wong and colleagues’ 2021 external validation of the Epic Sepsis Model examined 38,455 adult hospitalizations at Michigan Medicine between December 2018 and October 2019. They reported a hospitalization-level area under the receiver operating characteristic curve of 0.63 (95% confidence interval, 0.62–0.64). That is a finding about the studied model, cohort and evaluation, not a current performance estimate for every Epic installation or subsequent version.

At the study’s selected ESM score threshold of 6, hospitalization-level sensitivity was 33% and positive predictive value was 12%. A score at or above that threshold occurred in 18% of hospitalizations. Those results make the operating question concrete: how many alerts must clinicians review, which cases will be missed, and what useful action can follow? They do not establish that a different approval screen or governance committee would have improved the model.

The governance lesson is narrower and useful: supplier availability does not replace evaluation in the intended setting. Before releasing a clinical alert, the owner needs to understand the evaluated population, outcome definition, threshold, alert burden and the action clinicians are expected to take. Assigning a clinician to review the result is insufficient if workload or missing context makes that review ineffective. This historical study also does not establish that a particular governance committee, prompt control or approval interface would have prevented its observed limitations.

Use a control map to close the gaps

I use the Control Tower Reasoning System (CTRS) to connect three parts of the operating design. Decision Velocity measures elapsed time to an accepted outcome alongside quality, review effort and cost. The Trust Layer keeps the applicable source and governing permissions connected to the decision as policies and configurations change; the Version Drift framework addresses the change-over-time part of that work. Governed agent orchestration constrains the tools, delegated authority and destinations through which a proposal can become an action. To put these parts into operation, the workflow owner needs a control map: what runs, where it is enforced, who owns it, what proves it operated and what happens when it cannot complete.

For an approval check, for example, the owner may be the purchasing process owner; the enforcement point is the update service; the evidence is the approval identifier and matched action scope; the failure response is to withhold the update and return the case for review. That is more useful than a row saying “human in the loop: yes.”

The map also reveals controls that need no model at all. Permission checks, expiry comparisons, destination restrictions, and duplicate handling should use dependable application mechanisms where available. Asking the model to remember a prohibition is a poor substitute for preventing the prohibited operation.

Start with one workflow and an explicit expansion gate

Treat an initial 90-day plan as a sequence of deliverables, not a promise that every organization can finish in 90 days. First establish the owner, scope, baseline, source responsibilities, and decision rights. Then build and test the action boundary and evidence path in an isolated environment. Finally, run a limited pilot with monitoring, incident handling, and a practiced stop procedure.

Expansion should require the owner to review accepted outcomes, unresolved cases, material control failures, review effort, and cost. Define the tolerances before evaluating the pilot. If source ownership remains unresolved or a revoked permission can still be used, a favorable response-time chart does not close that gap.

Governance will add work. For a simple drafting use, an elaborate action-control system may be disproportionate. For a service that changes consequential records, the organization needs to know who authorized the change and how to stop or explain it. The design should match the work closely enough that people can operate it, rather than building a parallel reporting process nobody can keep current.

Give the initial plan deliverables and gates

Illustrative phase	Lead and contributors	Deliverable	Gate
Days 1–20: discover and classify	Business owner with procurement, application owners and specialists	Workflow register, supplier chain, intended use, obligations and unresolved questions	Protected actions and affected people are understood; unknowns have owners
Days 21–40: define rights and evidence	Process owner with security, source owners and engineering	Decision-rights matrix, source lifecycle, action contract and evidence schema	Every consequential operation has an authorization and failure rule
Days 41–65: implement and challenge	Engineering and operations with reviewers	Isolated implementation, synthetic boundary tests and reconstruction exercise	Changed cases, revoked permissions, injection and uncertain results behave as specified
Days 66–90: limited operation	Business and operational owners	Bounded pilot, exception monitoring, workload, cost and recovery exercise	Expansion records outcomes, remaining risks and funded responsibilities

The schedule depends on conditions the team may not control. A supplier that cannot support the required authorization boundary, or a missing authoritative policy source, can require a redesign before implementation proceeds. Access, validation and legal assessment also need their own time. Expose those constraints rather than treating the calendar as proof of readiness.

The final exercise should include more than a clean demonstration. Reconstruct who requested a completed decision, which source and case versions applied, what authorized the action and what the destination confirmed. Then repeat with a changed condition. Ask the on-call team to stop new work, identify affected queued cases and reconcile an uncertain result. An unread runbook is not equivalent to that exercise.

Establish a decision function, not just another committee

There is a case for an executive decision function spanning business ownership, architecture and risk. A Chief Decision Officer is one organizational option. The useful part is the mandate: maintain the decision catalog, resolve ownership gaps, fund controls, adjudicate exceptions and require evidence before expansion. A title without those rights would add another reporting destination.

An existing executive or forum may already be able to perform that work. Give it a remit and escalation route. Leave ordinary decisions with responsible business owners, with specialist participation proportionate to the use. Reserve central review for material exceptions, disputed ownership and changes crossing business boundaries. Otherwise, every harmless drafting change acquires another queue.

Use measures that expose the work. Coverage means identified consequential workflows whose actual execution paths have mapped controls, with inventory gaps stated. Override measures need a denominator and outcome: a high rate might reveal a weak service, an effective reviewer or a different case mix. Review time should include queueing and rework. Compare accepted outcomes, unresolved cases, control failures and total operating cost under comparable workloads.

Treat decision velocity as a question to measure, not an automatic governance benefit. A maintained source process can reduce clarification; a poorly designed approval chain can delay every case. Measure elapsed time from a valid request to an accepted outcome alongside quality, corrections and safeguards for affected people. Faster authorization of the wrong action is not an improvement.

Maturity should describe demonstrated capability. An inventory is an early deliverable; a tested action boundary is another; a maintained service that survives change is further along. Buying a platform does not move every workflow to the same level. Review evidence per workflow, assign the next missing capability and fund its maintenance.

That is the standard for a review: choose one completed case, trace it through the controls, and then show what happens when a material condition changes. The policy and the running service should tell the same story.

Share or cite this article

LinkedInEmail

Ajith Vallath Prabhakar. “Enterprise AI governance: make the controls reach the work.” ajithp.com, 14 December 2025. https://ajithp.com/2025/12/14/enterprise-ai-governance-framework/