Special Relativity: Governing AI at the Speed of Agents

Part II of III. We draw AI governance as a pit crew. Agents never come into the pits.

I. Nine seconds

In April, an AI coding agent at a small software company called PocketOS was working on a routine task in a staging environment. It hit a credential mismatch. Rather than stopping, it found a way through to production, decided a storage volume needed deleting, and deleted it. There was no confirmation step. It took nine seconds. The production database went with it, and so did the backups, which lived on the same volume. The firm’s car-rental clients lost roughly three months of bookings before some of the data was recovered.

Asked afterwards to explain itself, the agent produced a line no risk committee should forget. It had, it said, “violated every principle I was given.”

The principles existed. They had been given. What was missing was anything that could enforce them during the nine seconds that mattered, and a credential that should never have reached production in the first place.

II. The pit crew

In Part I, The Risk of Everything, I wrote about the twelve-role AI governance chart circulating on LinkedIn, drawn as a Formula 1 pit crew around a car, and about the seat it leaves empty. This week I want to look at the picture itself.

A pit stop is a remarkable piece of coordination. Around twenty people, each with one job, a little over two seconds, no ambiguity about who does what. The car leaves the box better than it arrived. It is an appealing image for AI governance, and the chart uses it well: the board sets direction, the CISO watches for attacks, Internal Audit tests the controls, accountability is shared.

But the image carries an assumption that matters more than any role on the chart. A pit crew only works because the car stops.

III. Governance at the gate

Most AI governance today is gate governance. A use case is proposed, assessed, approved, deployed and then audited. The crew does its work while the car is stationary: at design review, at the risk committee, at the annual audit. Then the car rejoins the race and the crew waits for the next stop.

That model fitted the AI we had. A credit scoring model was trained, validated and frozen. It did one thing until a human changed it. Inspecting it at the gate was enough.

Agents change this. An agent plans, calls tools, spends money, writes to production systems and talks to customers, continuously and at machine speed. It does not wait for the next committee.

Einstein’s special relativity taught us that time runs differently depending on how fast you are moving. Governance now lives inside that asymmetry. For an agent, nine seconds is long enough to destroy a company. For a committee, nine seconds is not long enough to open the agenda. Two clocks, one organisation, and our controls are set to the slow one.

Split image: racing teal data streaks around a stopwatch on one side, a still boardroom with a calendar and cold coffee on the other — Machine time and committee time.

I have described the shift as Prompts to Loops to Loop Governance. We have reasonably mature governance for prompts. We are rapidly building loops. We have very little governance for the loop itself.

The cautionary tales share a shape. In July 2025, during SaaStr founder Jason Lemkin’s twelve-day experiment with Replit’s coding agent, the agent deleted a live database holding records on 1,206 executives, and did so during an explicit code freeze. Lemkin says it then hid what it had done. The freeze was a gate rule: declared, documented, understood. The agent drove through it.

An agent never comes into the pits. Governance has to ride in the car.

IV. Bainbridge’s irony, forty years on

In 1983 the cognitive psychologist Lisanne Bainbridge published a short paper called Ironies of Automation. Her insight was that automating a system does not remove the human from it. It leaves the human with the hardest jobs of all: monitoring a process they no longer operate, and intervening quickly in exactly the rare moments the automation cannot handle.

Every agentic deployment now reruns her experiment at scale. We hand the routine work to the agent and keep a “human in the loop” for oversight. Then we give that human a dashboard, a queue of approvals and no realistic way to intervene inside nine seconds. Bainbridge would recognise the arrangement immediately.

Enterprises are starting to sense the problem. Gartner predicted in June 2025 that over 40% of agentic AI projects will be cancelled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls. KPMG’s quarterly pulse survey, reported by CFO Dive in January, showed the share of firms deploying agents falling from 42% to 26% between the third and fourth quarters of 2025. Cost and value are part of that retreat, as I argued in Part I. But so is a simpler worry: when an agent acts across systems at speed, it is not clear who is holding the wheel.

V. Three seats the chart cannot draw

Drawn honestly for agents, the chart needs three more positions. None of them fit a pit-lane diagram, because none of them wait for the car to stop.

The runtime owner. The chart has a system owner, but a system owner owns a system. Agents act across systems, borrowing credentials and tools from several at once. Someone must be accountable for the fleet while it runs: live monitoring, spend ceilings, tool permissions, limits on blast radius and the kill-switch. Today that work falls between the CTO, the CISO and a dozen system owners.
The delegation authority. When an agent acts, whose authority is it exercising? A human employee works within a mandate: what they may decide alone, what they must escalate, what they may never do. An agent needs an equivalent mandate, written, versioned and revocable, and someone has to own it. PocketOS is what happens when the mandate lives in a prompt and the credentials say something else.
The referee. One governance writer made the point well in a post the same morning: frameworks name who participates, but rarely say what happens when those people disagree. The chart does have a CEO and a board at the top, so a tie-break exists in theory. The real question is speed. Legal says stop, the business says ship, risk says escalate. If resolving that takes a month and the agent runs every second, the escalation path is decorative.

VI. From org chart to operating system

The answer is not a bigger org chart. It is moving governance from people at the gate into controls in the loop.

That is the design idea behind TrustOS: seven layers that run alongside the agent, so that policy is enforced at the moment of action rather than reviewed after it. In practice, runtime governance means identity and authority are checked before a tool is called, spend and scope are bounded before a transaction completes, and irreversible actions such as deleting a volume require a second key the agent does not hold.

Even the security agencies are moving in this direction. On 16 October, NIST, the NSA and CISA issued voluntary joint guidance on incident readiness and response for organisations training and evaluating highly capable AI agents on long tasks with limited human supervision. It is written for frontier labs, though it notes it may also apply to deployers, and its organising idea travels: an incident is an agent taking “significant uninstructed actions” that compromise the controls its operators imposed. Writing in BleepingComputer, a security architect at BeyondTrust argued that the first question in any agent incident is whose credentials were used, and that the answer has to exist before anything goes wrong. That is the delegation authority, seen from the investigator’s chair.

The test I apply to any framework is simple. For each control, ask whether it is enforceable at runtime or only reviewable afterwards. A control that can only be reviewed afterwards does not govern an agent. It documents what the agent did.

People still matter, and so do the twelve roles. But their work changes from inspecting the car to designing the rules the car carries with it, and to settling, quickly, the disputes those rules cannot settle on their own.

VII. Where the published record lands

As in Part I, these are my usual panel’s public positions, not responses to this essay.

Yoshua Bengio has arrived at the same architecture from the safety side. He has written that he is deeply concerned by how unrestrained agentic systems already behave, and founded LawZero in 2025 to build a non-agentic “Scientist AI” that sits beside agents and asks of each proposed move: “is this proposed action from the AI agent likely to cause harm? If so, reject that action.” That is runtime governance, stated about as plainly as it can be.

Dario Amodei, in his January essay The Adolescence of Technology, locates the danger in the combination of intelligence and agency with poor controllability, and argues for targeted guardrails, starting with transparency, rather than blanket pauses. Controllability is the operative word. A gate cannot control what happens between gates.

Andrew Ng keeps the upside in view. Writing in The Batch, he describes “many green shoots” in businesses applying agentic workflows. The purpose of runtime governance is not to slow those down. It is to make it safe to let them run.

Mo Gawdat would remind us that the agent is rarely the real villain. “The dystopia is super intelligent AI is reporting to stupid leaders,” he said on The Diary of a CEO. PocketOS did not need a wiser model. It needed someone to have decided, in advance, that no agent deletes production data without a second key.

The sharpest pushback comes from the policy world. At the Paris AI Action Summit, US Vice President JD Vance argued that “the AI future is not going to be won by hand-wringing about safety.” Plenty of builders agree, and they have a point about committees. My answer is that runtime controls are the faster option. Gate governance makes every change wait for a meeting. Runtime governance lets you approve an envelope once and let the agent move at full speed inside it, with the guardrails travelling at the same speed as the agent.

VIII. A prediction

Within two years, the defining agentic incident at a large enterprise will not involve a rogue model or a malicious attacker. It will be an agent doing what its credentials permitted, at a speed and scale no committee anticipated, while the governance crew waited in the pit lane for a car that never came in.

IX. The five-minute drill

Instead of a list of questions, try one exercise this week. Pick the agent in your organisation with the widest access. Without warning, ask the person you believe owns it to stop it, completely, within five minutes, and to tell you afterwards everything it did in the hour before.

Time them. If they cannot do it, or if three people each assume someone else owns it, you have learnt more about your AI governance than any chart can show you.

Next week, in Part III, The Arrow of Time, I turn to the decisions that, once made, can never be undone.