Making human oversight work for AI agents

Listen or watch the overview · 4 minutes 32 seconds

Generated narration with NotebookLM, paired with the article’s original diagrams and checked editorial visuals. The written article contains the full argument, qualifications, and primary references.

Audio overview

Video overview

Contents

1. Effective oversight needs a testable definition
2. The reviewer needs evidence, authority, and a workable job
3. Design for specific risks and consequences
4. Human bias needs controls of its own
5. Operationalize oversight before adding approval screens
6. A coding platform needs more than mandatory sign-off
7. Other decisions expose different weaknesses
8. Verify the capability through the actual workflow
9. Governance must maintain the conditions for oversight
10. Start with one decision the organization is prepared to defend
11. Frequently asked questions
12. References and evidence notes

A pull request arrives with a clean test run and a short explanation: the coding agent has refactored a cache to improve performance. A developer reviews the changes and approves the merge. During the refactor, the cache key lost the tenant identifier. The tests pass because the agent’s tests use one tenant.

In this hypothetical example, effective review depends on whether the developer could examine the tenant-isolation assumption and evidence that challenged it. The approval also needs to apply to the revision that was merged, with a clear boundary for any later release changes.

The approval log records the decision, but it does not establish whether the reviewer could examine those conditions or whether the system respected the decision.

Agentic systems make this gap more consequential. An agent can choose sources, interpret instructions, change files, call tools, and hand work to other agents before presenting a result. A person cannot reasonably replay every internal operation. Effective oversight therefore depends on choosing where human judgment matters, making those decisions examinable, and ensuring that the person’s decision changes what happens.

Before relying on a human reviewer, the organization needs evidence that the person can perform the job and that the system will honor their decision. That evidence has to cover the interface and working conditions as well as execution. An oversight label leaves those questions open.

Effective oversight needs a testable definition

For this article, I use the following working definition:

Effective human oversight is a demonstrated ability, within a specified workflow and operating conditions, for a suitably qualified and authorized person to assess relevant evidence, apply contextual judgment, and have an appropriate decision enforced in time to influence the intended outcome.

Sterz and colleagues’ FAccT 2024 analysis identifies causal power, access to relevant information, self-control, and intentions appropriate to the oversight role as conditions for effectiveness. ^[1] The working definition here translates that conceptual account into obligations a team can examine in a running workflow. “Demonstrated” requires evidence beyond a documented procedure; “operating conditions” includes workload, available time, evidence quality, and access to other experts. Enforcement connects the person’s decision to the action the system actually takes. The design and measurement practices that follow are a synthesis to validate in the intended setting.

A review process shown to work for a small documentation change has not established its effectiveness for an authentication rewrite. The same limit applies to timing: a supervisor who can stop a long-running batch process may be unable to prevent an external message sent milliseconds after an alert. Specify the kinds of work and conditions covered by the evidence before extending the arrangement to other decisions.

HITL and HOTL describe different arrangements for that influence. HITL also appears in discussions of data labeling and model training; this article concerns oversight of operational decisions and actions. Terminology varies across research and products, so the definitions below make the distinction explicit.

A coding workflow can use both. Repository exploration may proceed under standing authorization, while a production deployment waits for approval. A supervisor can interrupt an unexpected sequence of changes before it reaches that approval gate. The organization’s responsibility is to define and maintain these boundaries, including who may change them.

The distinction also depends on what is being automated. Parasuraman, Sheridan, and Wickens separated information acquisition, analysis, decision selection, and action implementation in their model of automation. A human can select the final action while relying heavily on automated information gathering and analysis. Retaining the last click leaves those earlier dependencies intact. ^[2]

There is a real limit to the argument for human involvement. People do not reliably improve every AI-supported task. Vaccaro, Almaatouq, and Malone’s 2024 meta-analysis covered 106 experiments and 370 effect sizes. Human–AI combinations performed better than humans alone on average, but worse than the better of the human or AI working alone. The results varied substantially by task. Improvement over an unaided person and improvement over the strongest standalone performer are different claims. ^[3]

That finding does not justify removing legally required human decisions or excluding people affected by a decision. It does require teams to state why human judgment is needed and test whether the arrangement supports that purpose. An accountable decision, a better technical answer, and an opportunity to contest an outcome are distinct objectives. A workflow may need all three, with different evidence for each.

The reviewer needs evidence, authority, and a workable job

A useful oversight design starts with what the person must decide. “Review this output” leaves too much unresolved. In the cache example, the decision might be whether the proposed change preserves tenant isolation and is suitable for integration under specified release conditions. That narrower statement determines the evidence and expertise required.

Visibility must follow that decision. The reviewer needs access to the relevant requirement, changed behavior, current artifact, supporting tests, and known uncertainty. An explanation can help them locate the issue, but the ability to inspect the underlying evidence matters when the explanation is incomplete or wrong.

Phaedra Boinodiris’s discussion of agentic governance identifies visibility, authority, measurement, and feedback as requirements for substantive human oversight. These are useful design questions; the discussion is a conceptual argument, rather than an empirical effectiveness standard. ^[4] In practice, several system properties make those requirements possible:

These properties support oversight in different ways. Explainability does not establish reliability, and a detailed activity log does not establish that a supervisor can recognize a dangerous sequence. Each capability has to be connected to the judgment expected from the person.

Research on explanations makes the distinction useful. Bansal and colleagues found that explanations increased acceptance of AI recommendations in their studied tasks, including incorrect recommendations, without further improving complementary team performance. Vasconcelos and colleagues found that explanations could reduce overreliance when they made verification worthwhile in controlled maze tasks. The relevant question is what the explanation enables someone to check, at what cost. ^{[5] [6]}

A review screen should therefore lead with the proposed consequential action and the evidence needed to judge it. For a payment, that includes the recipient and destination, not merely an invoice summary. For a coding change, it includes changed security assumptions, not merely a list of edited files. Where uncertainty cannot be estimated credibly, the interface should state the missing evidence instead of displaying an invented confidence score.

The evidence also needs a provenance boundary. A retrieved document is evidence to assess; instructions embedded inside it do not acquire authority over the reviewer or the agent. A generated rationale should be identified as generated. It must not be presented as a faithful record of hidden model reasoning. The useful audit record concerns inspectable inputs, actions, artifacts, explanations shown, and decisions made.

Authority then has to be usable. The reviewer needs a legitimate right to ask for more information, reject the proposal, hold work, seek another expert, and escalate a policy problem. If the organization penalizes every delay, assigns responsibility without access, or routes exceptions to a team that cannot change the outcome, the nominal authority is weak.

Competence is specific to the decision. Familiarity with a programming language does not establish competence to assess a tenant-isolation change. A customer-service supervisor may need policy knowledge and a way to obtain the customer’s missing circumstances. Decisions affecting employment or access to services may require legal, operational, accessibility, or other domain expertise. Adding more technical detail cannot substitute for context that the system never collected.

Staffing belongs in this design. A queue of reviews, their difficulty, and the time available determine whether careful judgment is possible. Assign an owner and cover for absences, and give reviewers a route to decline work outside their competence. Training should include realistic misleading proposals and correct proposals that look unusual; otherwise it teaches people to recognize a demonstration rather than perform the job.

Design for specific risks and consequences

Oversight can fail through bad judgment, an unavailable person, a late intervention, or an action that ignores a good decision. Those failures call for different controls. A risk register that groups them all under “human error” prevents a useful diagnosis.

The following catalog connects foreseeable failures to controls and evidence a team can evaluate in its own workflow.

Risk severity depends on the consequence and the ability to recover. A mistaken draft can be corrected before release. A confidential message cannot be made unread after delivery. Recovery procedures matter, but a post-event correction must not be reported as prevention.

Risk can accumulate across actions that look acceptable individually. A series of small purchases can exceed an intended budget, or separate data queries can reveal information that no single response discloses. Standing authorization needs cumulative limits and a way to recognize relevant changes in context, so the review can address the emerging consequence of the sequence.

More intervention can introduce its own harm. A randomized clinical trial of a nearly hard-stop drug-interaction alert improved the targeted prescribing response but was stopped early after four patients experienced delays in appropriate treatment. That is evidence from a particular clinical system, not an estimated effect for AI agents. It establishes why testing should count harmful delays as well as prevented errors. ^[7]

Human bias needs controls of its own

Human review adds a second decision-maker with their own assumptions and incentives. A system can supply excellent evidence while the person anchors on the first recommendation, searches only for confirmation, or overrides a correct result because they distrust the model. These mechanisms should be tested separately because an intervention that reduces one can worsen another.

Automation bias concerns inappropriate reliance on automated advice. In the coding example, a developer may treat a clean agent-generated test run as sufficient evidence. Anchoring is broader: the initial “performance refactor” description can set the frame for the whole review, so the developer never looks for a security change. Confirmation bias can work in the opposite direction as well. A reviewer already convinced that the cache is safe may use the agent’s explanation to support that belief.

Give the reviewer a useful way to examine the assumption that matters. For selected consequential decisions, ask for an initial assessment before displaying the recommendation. Artifact review may need a different design: show the diff, affected security requirements, and test evidence before the agent’s summary, then ask the developer to assess the relevant boundary. Surface material changes and contradictory evidence. Where these steps impose substantial delay or duplicate the full task, test whether their cost is justified.

Buçinca, Malaya, and Gajos tested cognitive forcing interventions with 199 participants. Designs that required more engagement reduced overreliance compared with simpler explanation-based approaches, but stronger interventions also received less favorable subjective evaluations. Their results support deliberate design of engagement, with attention to burden and differences among users. They do not establish that inserting friction into every approval improves enterprise safety. ^[8]

Overconfidence needs a different check. A reviewer who feels confident after reading a fluent explanation may be less accurate than they think. Measure confidence alongside independently assessed correctness in validation exercises, and give feedback on specific misses. Confidence alone should not determine whether a person receives more authority or less review support.

Algorithm aversion and harmful overrides also belong in the analysis. A person can reject good advice after seeing the system make an error, or replace a correct recommendation with a familiar but inferior choice. Dietvorst, Simmons, and Massey’s forecasting studies found that allowing limited modification could improve willingness to use an algorithm. That finding does not justify unrestricted overrides or establish that every modification helps. It suggests testing forms of control that preserve useful human agency without abandoning a well-performing method. ^[9]

Human prejudice requires attention beyond trust in AI. A reviewer can apply different standards to comparable cases, introduce irrelevant personal characteristics, or repeat an institutional pattern that the model already reflects. Structured reasons can make those decisions easier to examine, but documentation alone will not remove discrimination. Where lawful and appropriate, assess outcomes and error patterns across relevant groups, examine the legitimacy of the decision criteria, and provide a route for affected people to supply missing information or challenge a result.

Repeated interaction can change the reviewer as well as the decision. Glickman and Sharot’s experiments, involving 1,401 participants across perceptual, emotional, and social judgment tasks, found that interaction with biased AI could amplify human biases. Their tasks do not establish the size of that effect in enterprise review. ^[10] They do give a reason to test reviewers over repeated exposure and keep an independent source of corrective feedback, rather than train the next evaluation solely on previous human approvals.

Make the required verification specific enough to inspect. For tenant isolation, the reviewer can identify the test and changed behavior that support the requirement, with an option to record that the evidence is insufficient. A separate display of failed checks and changes outside the declared scope can challenge a reassuring summary. Asking the same agent to defend its answer is weaker evidence: a plausible follow-up explanation may simply repeat the original assumption.

Organizational pressure can suppress otherwise competent judgment. Two reviewers in the same reporting chain may share an incentive to clear the queue or defer to a senior colleague. Protect reasoned disagreement, provide escalation outside the immediate delivery chain where needed, and examine whether reviewers can use that route without penalty. After an incident, assess the information and time available at approval; judging the reviewer solely by the eventual outcome invites hindsight bias and obscures failures in system design.

Fatigue and habituation affect the conditions in which these mechanisms operate, but they should not become an all-purpose explanation for every miss. Ancker and colleagues’ clinical-alert study associated complexity and repeated alerts with lower acceptance, without finding an effect of workload itself or desensitization over time for a newly deployed alert. ^[11] Yu and colleagues’ 2026 study followed 400 repeat reviewers through 11,429 reviews of agent-submitted pull requests over seven months: approval rose from 30.1% to 36.8%, while inline comments fell 22%. The authors interpret the pattern as habituation under growing workload. The records establish changes in review behavior, rather than directly measuring attention or proving that more defects shipped. ^[12]

The operational response is to examine local performance over time: repeated requests, difficult cases, queue growth, late responses, and missed problems. Reduce interruptions that add little judgment while checking whether removed prompts had a protective function. Monitor unflagged work as well as alerts; a quieter dashboard can mean either better filtering or weaker detection.

Skill maintenance needs similar care. Shen and Tamkin’s 2026 randomized study of 52 developers learning an unfamiliar Python library found lower immediate comprehension in the AI-assisted condition, with a particular weakness in debugging. The experiment evaluated one learning session with a chat assistant; it does not establish long-term skill loss or the effect of an agentic coding workflow. ^[13] Preserve independent debugging exercises, assess reviewer competence periodically, and test whether people can resume work when the agent is unavailable. These practices need evidence that they maintain the skills the oversight role requires.

Operationalize oversight before adding approval screens

The control design should be written down before the interface is built. Otherwise a team can optimize the screen while leaving the decision rights and execution behavior unresolved.

Start with a decision specification for one consequential action. Record the outcome being sought, the unacceptable consequences, the evidence required, and the point after which prevention is impossible. Identify the person responsible for the judgment and the person responsible for the system enforcing it. In a small team those may be the same individual, but the responsibilities remain different.

Then divide activity into explicit permissions. Some work can proceed within standing authorization, some must wait for a human decision, and some should be prohibited regardless of a routine approver’s willingness. State what changes invalidate that authorization: a different destination, a broader data scope, a new privilege, a material code change, or a cumulative limit. A reviewer should not have to infer this policy from a stream of prompts.

The review package should preserve the evidence actually presented. For the coding workflow, that may include the reviewed revision, applicable requirement, test results and their provenance, material changes identified, relevant limitations, and the decision with any conditions. A link to a document that can later change is insufficient when the claim depends on what the reviewer saw at the time. Preserve the relevant version or an appropriately protected record that can reconstruct it.

Collect only what the review and its audit purpose require. Copying every source into an unrestricted log can create a new data exposure. Record design must reconcile evidentiary value with access restrictions, retention obligations, confidentiality, and the rights of affected people.

Execution needs its own checks. Approval of a specific payment should not authorize a later change to the beneficiary account. Approval of a patch should not automatically cover a new revision or an unrelated release artifact. Define how the system binds the judgment to the material action and how it responds when that binding no longer holds. Enforce that boundary in the execution path, using the relevant artifact identity, action parameters, permissions, and validity conditions; a prompt asking the agent to remember the approval’s scope is insufficient enforcement. Test retries and replay as well as the first attempt.

Delegation must preserve the boundary. A supervising agent’s permission to investigate a problem should not silently give a sub-agent permission to alter a production record. Record material delegated actions and relevant failed checks, and preserve them when work is summarized for review. Otherwise an apparently complete final proposal can conceal side effects that have already occurred or evidence that was discarded during a handoff.

Pause and cancellation also need precise meanings. A button may stop new scheduling while a queued external action still executes. Test the full path, including delegated agents, retries, and in-flight work. If an action cannot be recalled, the control should expose that limit and place preventive review earlier.

For HOTL, the human needs enough time to regain context and act. A useful design check is:

Detection, notification, understanding, decision, and enforcement must fit within the remaining opportunity to prevent the specified consequence, with a suitable margin.

Measure the full timing distribution, including slow responses and missed notifications. Averages can conceal the cases in which the intervention window closes. If a supervisor cannot act in time, a staged release or hold before commitment can create an intervention opportunity. Other options include moving the decision earlier or narrowing the permitted activity. An earlier plan approval needs explicit scope and later checks for material deviations; it cannot authorize every detail of an unfolding trajectory.

Chen and colleagues’ 2026 preprint compared four oversight strategies with 48 participants across 192 live sessions on researcher-built websites. Users blocked 29 of 138 problematic actions that reached execution, about 21%, with no reliable difference in intervention across strategies. Strategy affected the occurrence of problematic actions upstream. The analysis identified timing problems and reduced runtime scrutiny after upfront plan approval. The authors selected a lightweight model partly because it produced problematic actions reliably enough to study; plan-based configurations also changed the agent’s instructions. With controlled sites and no autonomous baseline, the findings identify failure mechanisms whose prevalence needs testing in the intended deployment. ^[14]

Finally, specify what happens when the operating conditions fail. An unavailable reviewer, missing evidence, broken monitor, or overloaded queue needs an agreed response. That may mean holding work, switching to a narrower service, or escalating to a staffed team. “Fail closed” is not automatically the safest response when holding a legitimate action can itself cause harm. The fallback needs the same consequence analysis as normal operation.

A coding platform needs more than mandatory sign-off

Return to the hypothetical organization building an AI coding platform. Its agents can inspect repositories, write changes, generate tests, and submit pull requests. Every pull request requires human approval. Management initially treats that requirement as the primary safety control.

For the cache refactor, the human is being asked to assess a security property embedded in a performance change. If the platform presents only the agent’s summary, the review depends on the same framing that produced the defect. If the tests were generated from the same mistaken assumption, passing them adds little evidence about cross-tenant behavior.

The organization should make tenant isolation an explicit acceptance condition for the relevant component. The review package can connect the changed cache key to that requirement, show the relevant revision, and include a test in which two tenants access otherwise similar records. A reviewer with the necessary system knowledge assesses whether the test and implementation address the requirement. Security-sensitive cases can require an additional qualified reviewer, but adding another approval is useful only if that person brings an appropriate and sufficiently independent examination.

Independence concerns the source of the expected result and the possibility of a shared mistake; a generated test can still provide useful evidence. An existing cross-tenant regression test, an independently maintained security requirement, and a separate review of the changed access boundary answer different questions. Running a second model over the same summary may add less independence than the team expects.

The platform also needs to honor the decision. GitHub’s protected-branch documentation describes options for dismissing stale approvals after changes, requiring approval of the most recent reviewable push, and restricting merges with checks. Those settings have different semantics, and bypass permissions matter. ^[15] A team should verify its chosen configuration with a changed revision and the actual roles allowed to merge. A green policy page is not evidence that the release path cannot bypass it.

After integration, the release should be traceable to the reviewed and tested state. A later build, configuration change, or deployment destination can introduce a different risk even if the pull request was handled correctly. Keep the human decision’s scope clear: approval to merge is not necessarily approval to deploy to every environment.

The empirical security evidence also deserves a concession. Perry and colleagues found less secure code and greater confidence in security among participants with an AI assistant in their bounded security tasks. The study used an older Codex model. Sandoval and colleagues’ study of 58 student programmers working on a particular C task found a much smaller security impact. ^{[16] [17]} Neither result supplies a vulnerability rate for a current production coding agent. The mixed findings strengthen the case for testing the organization’s own workflow rather than importing a favorable or alarming percentage.

Follow the tenant-isolation requirement through the whole path. Test whether the reviewer notices the change when the summary omits it, whether independent tests expose the defect, and whether a later material edit invalidates approval. Exercise privileged merge routes and compare the release with the tested revision. The resulting record shows which parts of the arrangement actually protected the requirement and where a correction is needed.

This is also where excessive HITL can become counterproductive. If the same specialist must approve formatting changes, routine dependency metadata, cache refactors, and access-control changes in one undifferentiated queue, the organization has spent scarce expertise poorly. Standing authorization for a tightly defined low-consequence class may be appropriate, backed by sampling and enforced boundaries. It should be earned through evidence about that class and revised when its assumptions change.

Other decisions expose different weaknesses

The coding example emphasizes technical assumptions and artifact identity. A payment decision separates the quality of the human judgment from the enforcement of a legitimate approval.

Suppose an agent prepares a supplier payment. The reviewer checks the invoice, confirms the supplier, independently verifies a changed bank account through an approved channel, and authorizes payment to that destination. The reviewer has exercised a useful judgment. If the bank details can change between approval and submission without renewed authorization, that good judgment no longer controls the outcome.

The appropriate design preserves the recipient, destination, amount, and relevant conditions of approval, then checks them at execution. A material change holds the payment for renewed review. Validation should include both a fraudulent proposal caught by the person and a legitimate proposal altered after approval. Reporting only the first test would miss an execution failure.

A customer-service decision has a different weakness. An agent recommends restricting an account because the recorded behavior meets a policy rule. The records may be accurate while the recommendation remains inappropriate: an accessibility need, an unresolved identity correction, or circumstances that the policy permits a supervisor to consider may be absent from the data.

The reviewer needs a way to obtain that context and the authority to act on it. Where policy permits exceptions, the interface should support a reasoned exception or escalation. Repeated overrides of a harmful policy should trigger a policy review, so each supervisor does not have to improvise a remedy.

The affected customer is a source of evidence as well as the subject of the decision. An accessible route to correct information and seek reconsideration can expose failures that internal monitoring misses. A nominal appeal sent to a person who sees only the original recommendation recreates the same oversight problem after the event.

All three cases use the same design discipline, but the required expertise and proof differ. Code review depends on technical behavior and release state. Payment review depends on independently verified destination and authorization integrity. Customer decisions depend on relevant circumstances, legitimate criteria, and a meaningful route to challenge the outcome.

Verify the capability through the actual workflow

A team should state the oversight claim before choosing its metrics. “A reviewer can prevent an unauthorized destination change before payment submission” is testable. “Our system has human oversight” is too broad to evaluate.

The evaluation needs a reference judgment independent of the proposal under test. Some cases have a clear technical answer, such as whether a tenant can access another tenant’s cached record. Others require qualified adjudication, documented policy, or input from affected people. Legitimate disagreement should remain visible rather than being converted into a convenient binary label. Where the correct outcome is uncertain, report that uncertainty.

Include correct proposals as well as subtle errors. A process that rejects everything can catch every bad proposal while making the service unusable. Schemmer and colleagues’ work on appropriate reliance distinguishes responses to correct AI advice from preserving a correct initial human judgment when advice is wrong. Those measures need an initial judgment, a correctness reference, and relevant disagreement cases. They cannot be reconstructed from approval timestamps alone. ^[18]

For operational oversight, the measurement set should cover the full path:

Use a defined case as the unit for these rates, with separate results for materially different decisions and consequences. Fix the denominators before testing, and report counts as well as percentages. Severity should accompany the rate: combining a low-impact drafting error and an irreversible disclosure into one average can make an unacceptable result appear tolerable.

An illustrative HOTL exercise shows why the stages matter. Suppose independent evaluators place 100 material problems into a controlled test. Monitoring exposes 80 to reviewers. Reviewers recognize 60, and intervention prevents the specified consequence in time for 45. End-to-end prevention is 45 of 100 in that exercise. Reporting only the 60 recognized among the 80 exposed cases would omit both monitoring misses and late enforcement. These hypothetical counts illustrate the accounting and do not establish an acceptance threshold.

Sampling production work requires more care because the true number of problems is unknown. Independently inspect a sample of unflagged activity and record how the sample was selected. An investigation limited to alerts cannot estimate detection coverage. Severe incidents and near misses deserve separate analysis even when they are too rare for a stable percentage.

Test with representative people, case difficulty, error prevalence, concurrent work, and time pressure. A short experiment full of obvious errors can encourage vigilance that normal operations do not sustain. Conversely, rare critical errors may require deliberately enriched test sets; report that enrichment and avoid treating the observed rate as a production forecast.

Comparisons should answer a deployment question. Depending on the setting, compare the proposed design with the existing workflow, a different evidence display, a narrower autonomy boundary, or a staffed escalation path. An autonomous comparison may be useful in a safe test environment, but legal duties and unacceptable consequences can rule out a live autonomous trial. Keep model behavior stable where possible so an interface comparison is not silently also a model comparison.

End-to-end prevention does not establish how much protection the human added. An automated check may stop a defect that the reviewer missed. Record which controls detected and prevented each problem. To estimate the added effect of review in a safe experiment, keep the other controls comparable and vary the human-review arrangement. Where the design cannot isolate that contribution, report the performance of the combined workflow without assigning the whole benefit to the reviewer.

Report sample sizes, uncertainty, and repeated observations from the same reviewers. A small exercise with no misses supports only a limited claim. Pre-specify acceptance conditions for the intended scope, including disqualifying severe failures and acceptable burden. If the process fails, the available decisions include changing the interface, adding expertise, reducing the scope, moving the review earlier, or withholding deployment.

A repeatable test for the coding case

The cache example can become a controlled evaluation in an isolated repository and staging environment containing synthetic tenant records. Begin with an independently checked requirement: one tenant must not receive another tenant’s cached data. Preserve the repository version, test data, agent configuration, review interface, role permissions, and release path so the exercise can be repeated after a change.

Use multiple variants of the defect and correct changes with similar surface features. Have evaluators establish the expected behavior before assigning cases to reviewers, and avoid showing reviewers the defect labels. The following cases test different claims; they should not be collapsed into one pass rate.

For this example, the team can set prevention of every seeded cross-tenant disclosure as a release condition for the controlled test, together with a stated limit on unnecessary blocks of correct work. That is a chosen criterion for this security property. Passing a finite set of seeded cases cannot establish that production risk is zero.

Repeat the exercise with representative reviewers and queue conditions, and compare meaningful alternatives such as a summary-first display and an evidence-first display. Randomize or counterbalance assignments where feasible so learning the defect does not masquerade as an interface improvement. Report results by reviewer and case type, uncertainty, and any exclusions. Use the outcome to change a specific control before rerunning affected cases.

Feedback should then reach the people who can change the system. A missed assumption may require a new test or evidence display. Repeated unnecessary overrides may indicate poor calibration, unclear policy, or an unsuitable model. Unavailable reviewers may require staffing or reduced service scope. Recording these events without assigning corrective action leaves the oversight design unchanged.

Governance must maintain the conditions for oversight

The final approver cannot own every condition on which their decision depends. Governance needs to allocate responsibility for defining permitted activity, providing qualified reviewers, maintaining enforcement, evaluating effectiveness, and accepting residual risk.

Ben Green’s analysis of human-oversight policies argues for institutional scrutiny beyond requiring an individual to review algorithmic outputs. Elish’s account of “moral crumple zones” examines how responsibility can fall on people with limited control over automated systems. These are policy and sociotechnical analyses, not controlled demonstrations that every human review process fails. Their practical warning is that accountability must reflect the control a person actually possesses. ^{[19] [20]}

Maintain an oversight record that can support a deployment decision. It should identify the use and consequence, decision rights, evidence available, required competence and cover, autonomy boundaries, intervention limits, evaluation results, known failures, and the owner of corrective action. Record the conditions that require reassessment, such as a new model, changed tools, a new population, higher volume, altered permissions, or a different policy.

The NIST Generative AI Profile provides voluntary risk-management actions relevant to human–AI configuration, evaluation, monitoring, and feedback. It supports this lifecycle approach without prescribing one staffing ratio or approval frequency. ^[21] Secure-development practices remain necessary in the coding example; human review should sit alongside maintained engineering controls rather than justify omitting them. ^[22]

Legal duties need a separate applicability check

Operational effectiveness and legal compliance overlap, but they are different claims. A strong test result cannot remove a statutory safeguard, and an approval screen cannot establish compliance. The following is a scoped orientation as of October 15, 2026; applicability depends on the actual use, jurisdiction, exceptions, and commencement provisions.

Under the EU AI Act, Article 14 addresses the design of human oversight for high-risk systems, including the ability to understand capabilities and limitations, recognize automation bias, interpret output, and disregard, override, or interrupt the system where appropriate. Article 26 assigns relevant deployer responsibilities, including people with the necessary competence, training, authority, and support. ^{[23] [24]} An ordinary coding assistant is not automatically high-risk simply because it produces code; classification depends on the Act’s rules and the intended use.

The timeline must also be current. The Commission’s July 27, 2026 notice states that the AI Omnibus entered into force and that Annex III high-risk rules apply from December 2, 2027, with the relevant Annex I product-related high-risk rules from August 2, 2028. ^[25] Those dates should not be generalized to every provision of the Act. Organizations need a provision-specific schedule rather than a single “AI Act deadline.”

EU GDPR Article 22 concerns decisions based solely on automated processing that produce legal or similarly significant effects, subject to its scope, exceptions, and safeguards. The endorsed guidance explains that human involvement must be meaningful, with the authority and competence to change the decision. A routine rubber stamp does not automatically remove a decision from that analysis. ^[26] Personal-data duties also extend beyond Article 22, including to the data exposed through oversight interfaces and records.

The UK framework must be considered separately. Section 80 of the Data (Use and Access) Act 2025 commenced on February 5, 2026, under SI 2026/82, subject to saving provisions for certain earlier decisions. The amended UK GDPR’s Articles 22A–22D address solely automated significant decisions and safeguards concerning information, representations, human intervention, and contesting decisions. ^[27] Map the decision and data to those provisions, including the distinct treatment of special-category data, rather than assuming the EU’s Article 22 structure describes the UK position.

Colorado’s SB26-189, signed May 14, 2026, repeals and reenacts the earlier AI provisions and applies to consequential decisions made on or after January 1, 2027. For covered uses, its definition of meaningful human review is unusually concrete: an authorized, trained reviewer must consider relevant available primary evidence, avoid defaulting to the system’s output, and have sufficient information about its use, limitations, inputs, and principal factors. The Act also provides for review and reconsideration following covered adverse outcomes, qualified by commercial reasonableness. Its definitions and exemptions matter. ^[28]

California’s ADMT regulations make human involvement relevant to the definition of substantially replacing human decision-making. The reviewer must understand how to use the output, examine it alongside other relevant information, and have authority to make or change the decision. The human-appeal exception to providing an opt-out has additional conditions. Article 11 compliance begins January 1, 2027, for covered significant-decision uses. ^[29] These provisions and Colorado’s materially-influence test need separate scope assessments; attaching a human approval to a workflow does not settle either assessment.

New York City’s Local Law 144 illustrates a different obligation. Covered automated employment decision tools are subject to bias-audit, public-information, and notice requirements. Human sign-off does not itself satisfy those duties, and this is not a general law requiring approval of every agent action. ^[30]

The personal-decision regimes are most directly relevant to the customer-service example when its outcome meets the applicable legal threshold. The coding platform needs its own classification and broader legal assessment; ordinary code review does not automatically fall within these specific rules. For practitioners, legal review should produce concrete requirements for intervention, information, records, challenge routes, and commencement. Record the source and date of the interpretation so a legal change can trigger a corresponding workflow change.

Start with one decision the organization is prepared to defend

The fastest useful starting point is one consequential decision already in the workflow. Write down what the human is expected to judge, then run a controlled case in which the proposal looks plausible but violates that condition. Observe the evidence the reviewer receives, the judgment they make, and the action that follows. Include a correct proposal too, so the exercise can expose unnecessary interference.

Before expanding the arrangement, the team should be able to answer these questions with artifacts and results:

1. What decision belongs to the human, and which consequence is that judgment intended to influence?
2. What relevant evidence and context can the person inspect, including information that challenges the proposal?
3. Who has the competence, time, authority, and organizational support to make the decision?
4. How does execution honor that decision when the artifact, destination, permissions, or circumstances change?
5. What testing demonstrates detection, appropriate judgment, timely enforcement, and acceptable burden under representative conditions?
6. Who examines failures, affected people’s challenges, and human bias, and who changes the system in response?

Some workflows will support bounded autonomy with supervision; others will need an earlier human decision or a narrower scope. Where the evidence, expertise, or intervention window is inadequate, the proposed arrangement should not be deployed in that form.

For the cache refactor, a defensible record contains the tenant-isolation requirement, the reviewed revision, evidence that challenged the single-tenant assumption, the qualified decision, and the released artifact. An approval timestamp establishes only one part of that record.

Frequently asked questions

References and evidence notes

This article draws on primary research, official legal materials, and engineering documentation available through October 15, 2026. It is a targeted research synthesis, not a preregistered systematic review or an independently validated oversight standard. The three organizational examples and the 100-case measurement exercise are hypothetical. Design recommendations require validation in their intended setting. Preprints are identified below.

1. Sterz, S., Baum, K., Biewer, S., Hermanns, H., Lauber-Rönsberg, A., Meinel, P., and Langer, M. (2024). On the Quest for Effectiveness in Human Oversight: Interdisciplinary Perspectives. FAccT, 2495–2507. DOI, conference PDF. Interdisciplinary conceptual account, rather than an experimental effectiveness estimate.
2. Parasuraman, R., Sheridan, T. B., and Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics, Part A. DOI. Conceptual model distinguishing stages and levels of automation.
3. Vaccaro, M., Almaatouq, A., and Malone, T. W. (2024). When combinations of humans and AI are useful: A systematic review and meta-analysis. Nature Human Behaviour. DOI and article. Meta-analysis; heterogeneous tasks and experimental settings.
4. Boinodiris, P., IBM Technology. Human in the Loop Is Not a Governance Strategy for Agentic AI. Video. Conceptual discussion of visibility, authority, measurement, feedback, and contextual expertise.
5. Bansal, G., et al. (2021). Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance. CHI. Paper. Controlled task experiments; explanations and acceptance are not equivalent to effective judgment.
6. Vasconcelos, H., et al. (2023). Explanations Can Reduce Overreliance on AI Systems During Decision-Making. Proceedings of the ACM on Human-Computer Interaction. Paper. Controlled maze tasks; verification cost and incentives matter.
7. Strom, B. L., et al. (2010). Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Archives of Internal Medicine. PubMed. Clinical evidence of a control’s benefits and treatment-delay harms; not an agent-system effect estimate.
8. Buçinca, Z., Malaya, M. B., and Gajos, K. Z. (2021). To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making. Proceedings of the ACM on Human-Computer Interaction. DOI. Controlled experiment, 199 participants.
9. Dietvorst, B. J., Simmons, J. P., and Massey, C. (2018). Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them. Management Science. DOI. Forecasting studies; bounded modification and willingness to use advice.
10. Glickman, M., and Sharot, T. (2025; published online December 18, 2024). How human–AI feedback loops alter human perceptual, emotional and social judgements. Nature Human Behaviour, 9, 345–359. DOI and full text. Controlled experiments on bias amplification; not a production-agent effect estimate.
11. Ancker, J. S., et al. (2017). Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Medical Informatics and Decision Making. Full text. Observational clinical evidence; does not establish a universal workload effect.
12. Yu, H., et al. (2026). Habituation at the Gate: Rising Approval and Declining Scrutiny in Human Review of AI Agent Code. Author manuscript. Observational review records do not directly establish attention or defect outcomes. The arXiv manuscript is the version cited here.
13. Shen, J. H., and Tamkin, A. (2026). How AI Impacts Skill Formation. Preprint. Randomized unfamiliar-library learning task; immediate comprehension rather than long-term skill retention.
14. Chen, C., et al. (2026). Comparing Human Oversight Strategies for Computer-Use Agents, version 2, September 12. Preprint. 48 participants, 192 live sessions on researcher-built websites; no autonomous baseline.
15. GitHub. About protected branches. Documentation. Product capabilities and configuration constraints; not empirical assurance of review quality.
16. Perry, N., et al. (2023). Do Users Write More Insecure Code with AI Assistants? ACM CCS. Paper. Security tasks with an older Codex assistant; limited transfer to current systems.
17. Sandoval, G., et al. (2023). Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants. USENIX Security. Paper and presentation. A specific C-programming task with student participants; contrasting security evidence.
18. Schemmer, M., et al. (2023). Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations. IUI. Paper. Operational measures for human responses to AI advice.
19. Green, B. (2022). The flaws of policies requiring human oversight of government algorithms. Computer Law & Security Review. DOI. Policy analysis and institutional-oversight argument.
20. Elish, M. C. (2019). Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction. Engaging Science, Technology, and Society. DOI. Sociotechnical analysis of responsibility and control.
21. NIST (2024). Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, AI 600-1. Official PDF. Voluntary risk-management guidance.
22. NIST (2022). Secure Software Development Framework (SSDF) Version 1.1, SP 800-218. Final publication. Secure-development guidance; distinguish from later drafts.
23. European Commission, AI Act Service Desk. Article 14: Human oversight. Official text.
24. European Commission, AI Act Service Desk. Article 26: Obligations of deployers of high-risk AI systems. Official text.
25. European Commission (July 27, 2026). AI Omnibus enters into force. Official notice. Commencement dates must be mapped to the relevant provisions.
26. Article 29 Data Protection Working Party (2018). Guidelines on Automated individual decision-making and Profiling for the purposes of Regulation 2016/679, WP251rev.01, endorsed by the European Data Protection Board. Official guidance.
27. United Kingdom. Data (Use and Access) Act 2025, section 80; Commencement No. 6 and Transitional and Saving Provisions Regulations 2026, SI 2026/82, regulations 2(j) and 5. Act, commencement instrument, ICO explanation of amended safeguards.
28. Colorado General Assembly (2026). SB26-189: Automated Decision-Making Technology. Signed-act record, signed Act PDF. See sections 6-1-1701(15), 6-1-1702, 6-1-1705, and the Act’s section 5; covered uses and exemptions require case-specific assessment.
29. California Privacy Protection Agency (2025). Approved CCPA regulations: ADMT definitions and Article 11. Approved text, sections 7001(e), 7200, and 7221; official adoption and commencement notice. Scope and exceptions require case-specific assessment.
30. New York City Department of Consumer and Worker Protection. Automated Employment Decision Tools. Official requirements. Local Law 144; scoped employment-tool obligations.

Related reading

• Enterprise AI Has a Measurement Problem
• The Architecture Gap: Why Enterprise AI Governance Fails
• Enterprise AI Version Drift: The Hidden Risk & How to Fix It