A kill switch you have never tested is not a kill switch

On Oct 25, ComplexDiscovery, reporting Reuters, described the questions US bank supervisors are now putting to lenders about AI agents. What keeps an agent inside its authority? Do people stay involved in key decisions? How would the bank shut one down? The formal rulebook has not caught up: the April 17 model risk guidance, SR 26-2, says generative and agentic AI “are not within the scope of this guidance,” and the regulators’ promised request for information was still unpublished “as of Oct. 24.”

The spending side shows the same gap. In Deloitte’s 3Q 2026 CFO Signals survey, released Oct 21, “Seventy-four percent” of CFOs said they would need “several business days to several weeks” to pause AI use that went over budget. TechTarget’s coverage adds that “53%” review AI usage costs after the fact, not in real time.

Be precise about what these reports establish. Supervisors asking questions is not a finding that any bank failed to stop an agent, and Deloitte measured how fast finance can pause spend, not how fast engineering can halt a running workflow. But both point at one engineering problem: at most firms, the answer to “how do you stop it?” is a policy document. A policy says who may decide to stop. It says nothing about the tasks already in flight, the credentials already issued, or the side effects already queued downstream.

We have argued before that governance has to be designed in. This post is about the mechanics of the stop path itself.

A single off switch fails in predictable ways

The default design is one global flag that the agent runtime checks. We have found it fails in four ways.

• It stops too much. A global flag halts healthy workflows along with the broken one, so people hesitate to pull it. A switch nobody wants to pull is a switch nobody tests.
• It stops too little. If the flag is checked only at the start of an agent loop, a run already in progress keeps calling tools with a credential it already holds. Anything handed to a downstream system, such as a payment file, is not reversed by the flag.
• It loses state. Killing the process drops in-flight tasks. Afterward, nobody can say which actions completed, which half-completed and which never started.
• It depends on things that are not there at 2 AM. A deploy, a config service, or the one engineer with access.

Stop in layers, narrowest first

Treat stopping as a set of layers, each with a known blast radius, meaning the scope of what it affects. Take an agent in AML alert review that enriches alerts, drafts case narratives and can close low-risk alerts. If the narratives start citing the wrong transactions, you want to stop the closing action now, not halt the whole investigations function.

1. Revoke the tool credential. Agents should call narrow, typed tools with scoped credentials, so you can cut one capability, “close alert,” while read-only enrichment keeps working. It takes effect on the next call, with no deploy.
2. Disable the workflow step. In an explicit workflow with bounded agent steps, the orchestrator skips the narrative step and routes its inputs to analysts. Deterministic steps keep running.
3. Pin or roll back the config version. Prompts, model version, tool list and confidence thresholds are versioned together, so rollback restores the last configuration that passed its evals. This handles the most common case: behavior changed after a recent change.
4. Drain in-flight tasks to a human queue. Every task not yet committed is checkpointed and handed to a reviewer with its evidence attached, rather than retried blindly or dropped.

The global switch still exists as the outermost layer. It is the last resort, not the only option. Written as policy-as-config, the stop path looks something like this (illustrative, not a product API):

# illustrative stop policy, not a real API
workflow: aml-alert-review
stop_layers:
  - layer: tool
    action: revoke_credential
    target: case_mgmt.close_alert
    effective: next_call
  - layer: step
    action: disable_step
    target: narrative_drafting
    fallback: route_to_queue(analyst_review)
  - layer: version
    action: pin_config
    target: last_eval_passed
  - layer: workflow
    action: drain_in_flight
    checkpoint: true
    queue: analyst_review
slo:
  time_to_stop: measured_per_layer
  drill: scheduled

Time-to-stop is a service level objective

A service level objective (SLO) is a target you measure continuously, not a promise in a document. Measure time-to-stop from the moment someone decides to stop until the last side effect is blocked and every in-flight task is accounted for. Then rehearse it the way you rehearse incident response: a scheduled drill, first in staging, then in production on a low-risk workflow.

Each drill should produce four numbers:

1. Time to effect, per layer. Credential revocation should be fastest; a full drain will take longer. Know both.
2. Leakage. Side effects that completed after the stop decision. The target is zero.
3. Accounting. The share of in-flight tasks with a known final state: completed, rolled back or queued for a person.
4. Time to resume. From fix to re-enabled, with re-enablement gated on the eval set passing.

If a layer has never been exercised, assume it does not work. That is how every other production control is treated, and agents should be no different.

Log enough to replay the stop

A stop you cannot explain will not satisfy a supervisor. Trace every model call, tool call and decision with its inputs, outputs, config version and credential scope. Log the stop event itself: who triggered it, which layer, when, and what was in flight.

With that trace, a stopped run can be replayed against the rolled-back version to confirm the fix. It also turns “do people stay involved in key decisions?” from an assertion into evidence: the queue shows which cases went to an analyst and what the analyst decided.

Where this lives in RapidCanvas

The Operational Harness holds the controls each layer depends on: permissions, human approval for critical actions, versioning, rollback, monitoring and audit trails across the workflow lifecycle. Evaluation & AI Ops traces model and tool calls so a stopped run can be replayed and the rolled-back version re-tested before it goes live again. Our field AI engineers design the stop path with your team when a financial services workflow is built, not after the pilot.

The question to ask next

Before your next supervisory conversation, ask your team one thing: when did we last stop this agent on purpose, and how long did it take? If nobody can answer with a timestamp, you have a policy, not a kill switch.

Book an AI workshop to map and rehearse the stop path for your first regulated workflow.