Insights

The Bank of England Says the Control in Your AI Policy Doesn't Work

By Antony Coppellotti, Founder and CTO, Gordion Solutions. September 2026.

1. The regulator blinked

On 30 June, at the ECB's forum in Sintra, the Bank of England's Deputy Governor for Financial Stability said something supervisors do not usually say out loud. "Relying on a human in the loop for all agent actions is unlikely to be realistic," Sarah Breeden told the room, adding that the frameworks were not designed with autonomous agents in mind and that more sophisticated governance may be needed.

Read that again. Human in the loop is the control every AI policy in every regulated firm leans on. It is the line in the board paper that makes the risk committee comfortable. The Bank has now said, in a set-piece speech, that it does not scale.

This was not a slip. The Bank had previously insisted existing supervision was adequate. In February its own roundtables with banks and insurers recorded that traditional model validation would not be sustainable as agentic systems proliferated, and that the human-in-the-loop concept was being challenged by agentic AI. Four months later the Deputy Governor said it herself, and reached for the alternative a central banker would: guardrails analogous to circuit breakers or kill switches that would limit or stop trading market-wide.

A market-wide kill switch is what you propose when you have concluded the firm-level control has failed. The interesting question is why it failed, and what a firm should have built instead.

2. Why the human was never in the loop

Human in the loop assumes three things, and agentic systems break all three.

Throughput. An agent that plans, calls tools and chains actions makes hundreds of decisions an hour. No reviewer can evaluate hundreds of decisions an hour. So the firm quietly redefines "review" as "sample", then as "spot check", then as "we get an alert if something looks odd". The control on the policy document does not change. The control in production has already gone.

Attention. Where humans do review every action, they stop reviewing. Lisanne Bainbridge named this in Ironies of Automation in 1983, and aviation, radiology and process control have documented the same curve since: a reviewer who approves the machine ninety-nine times in a row approves it the hundredth time without looking. The loop becomes a signature. The person is accountable for a decision they did not make.

Timing. In a multi-agent system the human sits at the end of a chain. By the time an action reaches a person, upstream agents have already acted on the same intermediate results, and downstream ones are waiting on the approval to proceed. The reviewer is not a gate. They are a bottleneck the architecture has learned to route around.

None of this is an argument against oversight. It is an argument that oversight was placed in the wrong layer. The human was put at the point of action, where the volume is highest and the context is thinnest. The controls that work sit at the boundaries: what the agent is permitted to touch, how much it may do before it must stop, and what it must record before it acts at all.

That is the runtime problem, and it is the subject of the next section.

3. What replaces the human

The phrase now in circulation is "human on the loop". It is better than "in", but as a slogan it is no more of a control than the one it replaces. What it should mean, in a regulated firm, is that the person is removed from the point of action and the controls are moved to the boundaries. I set out four of those in an earlier note on securing agents. They are numbered by distance from the agent, B1 nearest and B4 outermost, and an action passes through them in that order on its way to the world. Authority runs the other way: policy is written at B4, to meet regulation, industry standards and the firm's own security posture, and is enforced inward at B3 and B2. Nothing is enforced at B1.

B1, reasoning. Nothing the model emits is trusted. Not its plan, not its tool call, not its claimed justification. The reviewer's job of "does this look sensible" is replaced by a rule: model output is a proposal, and proposals do not execute. This sounds obvious and is almost universally violated, because the convenient way to build an agent is to let the model call tools directly.

B2, tools. Every tool the agent can reach is scoped to the least it needs, and the scope is enforced outside the model. An agent that can read a customer record cannot also amend it unless that is the job. Credentials are issued per task and expire. This is the control that does most of the work the human was supposed to do: it makes the catastrophic action impossible rather than merely disapproved.

B3, runtime. Deterministic code brackets the reasoning. It holds the budget (how many actions, how much spend, how long), enforces the sequence, and refuses to advance the state until the previous step has been recorded. The runaway inference loop that runs up a serious bill and persists nothing is a B3 failure, and no human reviewer would have caught it, because there was nothing to review. The loop never produced an action.

B4, oversight. This is where the human actually lives. Not approving actions, but setting the policy the other three boundaries enforce, reading the intent log after the fact, and holding the kill switch. A kill switch that works at the firm level, tested, with a named owner, before the market needs one. Breeden's circuit breaker is the right instinct at the wrong altitude. If every firm has a working stop, the market rarely needs its own.

The point is that a human reviewing actions is a control on the model. The boundaries are controls on the system. Models drift, get injected, and reason badly. The system can be made deterministic in the places that matter, and determinism is what a supervisor can audit.

4. What the supervisor will ask for

The Bank has not written agent-specific rules, and I doubt it will soon. It will do what it did with model risk: apply SS1/23 and ask harder questions. From the roundtables and the Consortium minutes, the shape of those questions is already visible.

Scope. Consortium members discussed how SS1/23 expectations apply to AI systems and their components, including generative models, prompts and retrieval layers. That means the prompt is a model artefact. Version it, review it, and be able to show the change history. Most firms cannot.

Correlated behaviour. Breeden's central worry was many agents responding identically to the same trigger and amplifying volatility in stress. Expect to be asked what you have done to test it. If your answer is that your agents use the same foundation model as every competitor's, with similar prompts and similar data, you have described the problem rather than a control.

Reversibility. The FSB's June consultation noted that overriding or remediating agent actions can be difficult or impossible for humans. A supervisor will want to know which of your agent's actions are reversible, which are not, and what sits between the agent and the irreversible ones. That is a B2 and B3 question with a B4 owner.

The named person. Under SM&CR someone already holds accountability for the platform. The question that will land on them is not "can you explain the model" but "can you explain the runtime". What the agent can touch, what stops it, what is logged, and who turns it off. If the honest answer is "the vendor", that will not survive a supervisory visit.

None of this requires new regulation. It requires the existing framework to stop pretending the loop had a human in it.

5. The loop was never the control

Breeden asked whether guardrails are needed, and reached for a market-wide kill switch. The honest answer is that any firm running agents in production already needed the guardrails, and the human was never one of them. Human in the loop survived as long as it did because it was cheap to write down and nobody was counting the actions. Agents count. When a Deputy Governor says the control is not realistic, the useful response is not to wait for the rules but to look at where the human actually sits in your architecture. If the answer is "approving actions", the loop is empty. Move them to the boundary, give them the policy and the stop, and make the rest of the system something a supervisor can read.

Sources

Antony Coppellotti is CTO and Founder of Gordion Solutions, which builds software and AI solutions in regulated financial services.

If your AI policy still says "human in the loop" and your agents are already in production, take the AI Readiness to see where the gap is. Get in touch with any questions on this article, or anything else.

Also published on LinkedIn, September 2026.