The judgement layer is not a layer. It is a control.

The World Economic Forum is right that the missing layer of AI governance is the human one. But a layer you cannot instrument is a layer you cannot enforce. Here is what it takes to make human judgement measurable, and what fourteen jurisdictions are actually doing about it.

On Tuesday the World Economic Forum published a piece, drawing on its Centre for AI Excellence and its Agent Capability and Authorization Profile (ACAP) work, arguing that regulation governs markets, risk management governs organisations, and neither governs the moment a clinician decides whether to trust a risk score on a Tuesday afternoon. Trust, the piece argues, operates at the level of judgement, and nothing in the EU AI Act, NIST’s RMF or ACAP was designed to answer the only question that matters at that moment: should I rely on this system, here, for this decision?

I agree with the diagnosis. I have spent the last eighteen months arguing something adjacent: that most agentic AI failures happen between the rules, in the gap between what a framework permits and what a tired human at the gate actually does. The WEF piece is a welcome mainstream statement of that position, and its three in-the-moment questions (Limit it, Inspect integrity, Trace transparently, the LIT subset of its nine-step CALIBRATE discipline) are a usable checklist for a person under time pressure.

But I want to push on the piece where it is soft, because the softness is exactly where enterprises will fail. The WEF piece frames judgement as a practice: a discipline of habits, embedded in workflow, exercised daily. That is true and insufficient. A practice that cannot be measured cannot be enforced, and a practice that cannot be enforced will decay under load in a predictable, well-documented curve. The human layer is not missing because nobody thought of it. It is missing because nobody has been forced to prove it works.

This piece is my attempt to say what proving it would take. I have run the argument past my usual panel of AI voices, grounded in what they have actually said in public this year rather than what I wish they had said, and I have ledgered what fourteen jurisdictions across my patch and beyond have put on paper about human oversight as of this week. The short version: everyone names the human layer. Almost nobody instruments it.

Epistemic note. Regulatory dates, framework contents and research findings below are sourced and current to 3 September 2026. Panel positions are paraphrased from public statements and attributed as positions, not verbatim quotes, unless marked. The reliance model, the decay instrument and the TrustOS mapping are my own constructions and should be read as proposals, not standards.

I. Where the WEF piece stops, and why that is the dangerous place to stop

The WEF piece’s core move is a three-level stack: regulation at the market level, risk management at the organisational level, judgement at the level of the situation. Its evidence for the top of the stack is the set of industries that already run AI inside critical systems: grid balancing, payment fraud, bank credit and market risk. These systems are trusted, it argues, because the humans around them apply judgement consistently, embedded in workflow rather than left to willpower.

Here is the problem. Every one of those examples is a domain where the human layer is itself heavily instrumented. A payment network does not trust its fraud analysts because they have good habits. It measures their override rate, their false-positive tolerance, their queue latency, and it recalibrates the model against their decisions weekly. Grid operators sit inside control rooms built on sixty years of human-factors engineering, with alarm hierarchies designed specifically to defeat the attentional failures that automation induces. The bank’s model-risk function exists precisely because regulators stopped believing that “a qualified person reviewed it” was evidence of anything.

So the lesson from high-trust industries is not “build a discipline of judgement.” It is “build the telemetry that tells you whether the discipline is still happening.” The WEF piece gestures at organisational design. The gap between organisational design and control design is the whole game.

I call this the enforceability test, and I have applied it to every framework I have reviewed since the IMDA agentic work: for each human-oversight obligation, name the signal that would tell an auditor it had stopped working, and the threshold at which someone is paged. If no such signal exists, the obligation is a wish.

II. The reliance decision, stated properly

Let me make the “should I trust this system, here” question technical, because the informality is where the rubber-stamping hides.

Every act of reliance is a decision under uncertainty with asymmetric costs. A human reviewer receives an output ŷ with (ideally) a stated confidence c. She can accept it, verify it at cost V, or reject and escalate. The rational policy depends on three things she almost never has: the true calibration of the model in this slice of the distribution, the cost of an undetected error L, and the reversibility of the action.

// The reliance decision, one output at a time
Accept unverified  iff   (1 − p) · L  <  V
where  p = P(ŷ correct | context, c)   // the model's *conditional* accuracy here, not its benchmark score
       L = expected loss of an uncaught error   // scaled by irreversibility
       V = cost of verification   // time, expertise, attention
// Calibration condition the human is implicitly assuming:
P(correct | c) ≈ c   // almost never true out of distribution
// Value of the human layer is bounded by *independence*:
Value(oversight) ∝ 1 − ρ(error_model, error_human)
// if the human's errors are correlated with the model's, dual review adds paperwork, not safety

Three things fall out of writing it down.

First, the WEF “Inspect integrity” question is really a question about p: what data, what period, what assumptions, and therefore how far this input sits from the training distribution. A system that cannot expose its own out-of-distribution signal is asking the human to estimate p from fluency, which is the single worst available proxy. Fluent systems, as the piece rightly notes, do not announce errors; they persuade.

Second, “Limit it” is a statement about L and reversibility. The right place to bound a system is not where it is inaccurate but where an error is expensive and hard to undo. This is the logic Singapore’s agentic framework adopts when it asks organisations to classify actions by scope and reversibility before setting approval checkpoints, and it is the logic I built into TrustOS as an action-boundary control rather than a use-case whitelist.

Third, and this is the one that should worry every board: the value of the human layer collapses as the correlation term ρ rises. The moment the reviewer’s judgement is shaped by the model’s output, which is what automation bias is, dual review stops being two independent checks and becomes one check with two signatures. The EU AI Act’s Article 14(5) dual-reviewer provision assumes independence. A May 2026 analysis from Veritas JPS argues, correctly in my view, that no regulator has tested whether that assumption holds for co-exposed reviewers, and that the Act names automation bias as a risk but mandates only “awareness” of it, the weakest countermeasure the behavioural literature offers.

III. The decay curve nobody is plotting

The human layer does not fail suddenly. It decays. Three decades of human-factors research, from Parasuraman and Manzey’s attentional integration through Bainbridge’s ironies of automation, describe the same pathway: the reviewer trusts an accurate system, stops verifying, and approves by default while the audit log continues to record a human decision. Adnan Masood’s July essay grades an eleven-stage version of that pathway against documented cases from Robodebt to the American chatbot litigation. The International AI Safety Report 2026 confirms the effect persists with current AI tools and is a strong function of the human preference for mental shortcuts.

The numbers are not subtle. In the 2023 Radiology mammography study, when the AI was wrong, inexperienced radiologists’ accuracy fell from roughly 80% to under 20%; radiologists with fifteen or more years of experience fell from 82% to 45.5%. Tenure slows the onset. It does not prevent it. And in the US insurance denial cases that were later appealed, reporting suggests review times measured in seconds and reversal rates on appeal approaching 90%. A human “in the loop” for 1.2 seconds is not oversight. It is a signature-generating process.

Instrument 01 / Override-rate decay under load: what a healthy human layer looks like on telemetry, and what a rubber stamp looks like

Illustrative, not empirical. The diagnostic is the divergence between observed override rate and the model’s independently measured error rate. A human layer that overrides 0.4% of outputs from a model that is wrong 8% of the time is not exercising judgement; it is laundering the model’s error rate through a person. Seeded-error canaries (known-bad outputs injected into the queue) give you a direct detection rate, which is the only honest measure of whether the person is looking.

This is what I mean by a control rather than a layer. The instrument above is buildable today, in any queue-based workflow, with four fields: decision ID, model confidence, human action, and a canary flag. Track override rate against measured model error. Inject known-bad outputs and record the detection rate. Alert when the two series decouple. Singapore’s updated agentic framework already names override rates and response times as audit indicators. I have not yet seen a regulator anywhere set a threshold on them.

IV. Mapping LIT to controls: where judgement lives in TrustOS

The WEF piece is careful to say the judgement layer is not a substitute for regulation but what makes regulation operational. I would put it more architecturally. In TrustOS, my seven-layer agentic governance stack, the human layer is not a single tier. It is a set of controls that cut across tiers, each of which can be tested. Here is how its three in-the-moment questions decompose into things you can build and audit.

• Limit it: Action-boundary registry: each agent action classed by scope, reversibility and blast radius; approval checkpoints keyed to class, not to use case. Refusal behaviours tested as a capability, not assumed. Autonomy tiers (L0 recommend → L4 act-and-report) with promotion criteria and demotion triggers.
• Inspect integrity: Data lineage surfaced at decision time: training window, source provenance, out-of-distribution score, drift since last validation. Calibration monitored per slice, not per model. Confidence displayed as a calibrated interval, never as a bare number.
• Trace transparently: Objective function disclosed to the reviewer in plain language (what the system is optimising for, what it is penalised for). Provenance graph for every output: inputs, tools called, intermediate states, which agent acted under whose authorisation. Override audit trail with stated reason.
• (WEF omits): Reviewer-state controls: sustainable review volume, rotation, decision-fatigue caps, and cognitive forcing functions (justify-before-accept on high-L actions). This is the control that turns the WEF “practice” into something that survives a bad Tuesday.
• (WEF omits): Independence testing: correlated-bias checks for dual reviewers; canary detection rate; override-rate vs error-rate divergence alarm. The enforceability test lives here.

Notice what the two additions have in common. They are both about the human as a component with a failure mode, not the human as the safe default. Every framework I have reviewed treats the human as the answer to the risk. Almost none treats the human as a sensor whose readings drift, need recalibrating and occasionally need to be taken offline. That inversion is the whole contribution of sixty years of human-factors engineering, and AI governance has largely not absorbed it.

It also shows why “prompts → loops → loop governance” is the right frame for where we are. Prompts were a judgement problem for one person at one keyboard. Loops, where agents call agents, are a judgement problem for a system, and the human’s position in the loop is a design decision with measurable consequences. Loop governance is the discipline of deciding, per loop, where the human sits, what she sees, and what evidence would show she has stopped looking.

V. The panel weighs in

I circulate every long-form piece through the same set of voices before publishing. Their positions below are drawn from what they have argued publicly in 2026, and I have let them disagree with me where they would.

Dario Amodei (Anthropic)

Amodei’s July proposal for an FAA-style federal agency with day-one power to block a model’s release is a market-layer intervention, and he would say so. His argument is that the frontier is the small class of models where the human layer cannot be relied on at all, because the capability gap is too wide for any reviewer to grade the output. On my piece, the Amodei critique would be: instrument the human layer for the enterprise tier, certainly, but do not mistake it for a defence at the frontier. I accept that. The enforceability test is for the deployment perimeter, not for the lab.

Demis Hassabis (Google DeepMind)

Hassabis wants a FINRA-for-AI: an industry-funded, federally overseen standards body starting with voluntary pre-release review, hardening into market-access rules, and he wants it before year end. He has also said we are in “the foothills of the singularity” and that “we’ve essentially found a way to make sand think.” The Hassabis position is that standards bodies can carry the operational detail that legislatures cannot. My reply: then the standard should specify override-rate telemetry and canary detection as conformance evidence. A standards body that certifies process documents rather than instrument readings is FINRA without the audit.

Sam Altman (OpenAI)

Altman, in the Financial Times, pushed an IAEA-for-AI: a US-led international forum certifying countries, companies and standards, using frontier access as leverage. It is the most geopolitical of the three lab proposals and the least interested in the exam room. The Altman frame would treat the WEF piece as beside the point at the scale that matters. I think that is exactly backwards for the 14 markets I cover: the exam room and the credit desk are where AI harm is actually accruing in 2026, and no IAEA will ever reach them.

Yoshua Bengio (LawZero)

Bengio’s bet, through LawZero, is non-agency: build “Scientist AI” that explains and predicts but does not pursue goals, and use it to check agentic systems. On this piece he is my strongest ally and my sharpest critic at once. Ally, because a non-agentic verifier is precisely the independent second reader that drives ρ toward zero. Critic, because he would say that betting on tired humans as the independent check is already a losing position, and that the honest answer to the decay curve is to stop asking humans to be the sensor.

Geoffrey Hinton (University of Toronto)

Hinton’s public position has moved from “we cannot control something smarter than us” to arguing that the only durable safety property is a system that genuinely cares about human outcomes, his “maternal instinct” framing. Applied here, the Hinton critique is that the WEF authors and I are both engineering around a system that does not care, and that no telemetry fixes an objective-function problem. Fair. But we deploy the systems we have, and the ones we have need instruments.

Ethan Mollick (Wharton)

Mollick’s jagged-frontier work is the empirical spine of “Inspect integrity”: the same model is superhuman on one task and confidently wrong on an adjacent one, and users cannot see the boundary. His 2026 writing on skill atrophy and the “cyborg vs centaur” division of labour is the best practical guide I know to reviewer-state design. He would tell me the decay instrument is necessary but that the deeper fix is keeping reviewers doing enough unaided work to retain the expertise that makes their override meaningful. I agree, and I would add it to Layer 4 above.

Fei-Fei Li (Stanford HAI / World Labs)

Li’s human-centred AI framing would endorse the WEF piece almost without amendment, and would push back on my language of “the human as sensor” as reducing dignity to a component. I hear that. My defence is that treating the human as infallible is the less dignified position, because it sets her up to carry blame for a system that was designed to exhaust her. Instrumenting her workload is a form of respect.

Kai-Fu Lee & Mo Gawdat (Sinovation / ex-Google X)

Lee’s long-standing view is that the Chinese deployment model, iterate in production under state-defined red lines, will out-learn the Western compliance model. Gawdat’s is that the human layer is doomed by incentive: no organisation optimising for throughput will voluntarily keep a slow, expensive reviewer in the loop. Both are describing the decay curve from the outside. The answer to Lee is that China’s own 2026 rules (Section VI) are converging on oversight obligations. The answer to Gawdat is that the only reviewer who survives an incentive audit is one whose value can be shown on a dashboard. Hence the instrument.

VI. What countries are actually doing about the human layer

Everyone names human oversight. Here is what fourteen jurisdictions have on paper as of this week, ledgered against the only question I care about: does the obligation come with a signal, a threshold and a consequence, or is it a wish?

Read the right-hand column top to bottom and the pattern is stark. Fourteen jurisdictions; fourteen mentions of human oversight; one (Singapore) that names the telemetry; zero that set a threshold. The EU, which is the most detailed, has just bought itself fifteen months to write harmonised standards for Article 14. That is the window. If those standards specify override audit trails, independence testing and a divergence alarm, the human layer becomes a control in the single largest AI market. If they specify awareness training and a documented process, we will have codified the rubber stamp.

VII. Why the enterprise will get this wrong by default

Deloitte’s 2026 State of AI in the Enterprise found that 74% of companies plan to deploy agents within two years while 21% report a mature model for governing them. The gap will not be closed by frameworks. It will be closed, or not, by whether the human layer survives its first contact with a throughput target.

This is the Frozen Workforce problem in a new coat. In my earlier writing I used the phrase for organisations that have adopted AI faster than they have re-designed work, leaving humans in roles that are nominally unchanged and functionally hollowed. The reviewer whose job is to approve four hundred agent actions per shift is the purest case. She is on the org chart as the control. She is, on telemetry, a latency.

And this is where the WEF optimism about “governance as growth strategy” needs a harder edge. The piece is right that organisations that know where to delegate and where to keep humans will move faster than those that do not. But knowing where to keep humans is not the hard part. The hard part is keeping them awake once they are there, in a system that rewards them for every second they save. Ghost GDP, the productivity that shows up in the output metrics but not in the value delivered, is very often exactly this: throughput bought by converting a reviewer into a signature.

HX = CX + EX applies with full force. The customer experience of an agentic system is bounded by the employee experience of the person overseeing it. A reviewer who is exhausted, uninformed about the model’s failure modes, and measured on queue depth will produce a customer outcome that looks fine on every dashboard until it does not. Robodebt, the Amazon recruiting model and the US insurance-denial cases are all instances of this pattern, and none of them involved a broken rule.

VIII. What to do on Monday

For the leaders in my fourteen markets who will read this and ask what changes, five things, in order of how fast you can do them.

1. Log the four fields. Decision ID, model confidence, human action, override reason. If your agentic workflow cannot emit these today, it is not a governed loop; it is a pipe.
2. Plot the divergence. Override rate against independently measured model error, per queue, per week. Set an alarm at a ratio, not a number. Page a human when they decouple.
3. Seed canaries. Inject known-bad outputs at a rate the reviewers do not know. Detection rate is your only honest measure of whether anyone is looking. Publish it internally. Tie it to the reviewer’s workload, not to the reviewer.
4. Test independence before you rely on dual review. If two reviewers see the same model output, assume correlated error until you have measured it. Bengio’s non-agentic verifier is the long-run answer; a second model with a different training lineage is a reasonable stopgap.
5. Class actions by reversibility and set autonomy tiers. This is the WEF “Limit it” made structural, and it is the part of Singapore’s framework worth copying wholesale. Promote tiers on evidence; demote on divergence.

None of this requires new regulation. All of it requires treating the human in the loop as a component with a measurable failure mode, which is the one thing the WEF framing, for all its strengths, does not quite say out loud.

IX. The Fork, again

I keep returning to the same two futures. In one, the human layer is real: instrumented, defended, occasionally embarrassing when the canary rate drops, and therefore trustworthy. In the other, the human layer is a compliance artefact, a row on the org chart that lets everyone above it sleep, until the day the audit shows 1.2-second reviews and 90% reversals. The first future is Long-AND: agents and humans, throughput and judgement, with the instruments to prove both. The second is Short-OR dressed up as Long-AND.

The WEF piece ends with the line that AI’s biggest risk is not that it replaces us but that it persuades us to surrender judgement without noticing. I would sharpen it by one word. The risk is not that we surrender judgement without noticing. It is that we surrender it without measuring. The first is a human failing. The second is a design choice, and design choices can be audited.

The missing layer of AI governance is the human one. Agreed. Now show me the reading.

Sources and provenance

1. World Economic Forum, Centre for AI Excellence, “The missing layer of AI governance is the human one,” 1 Sep 2026, https://www.weforum.org/stories/artificial-intelligence/the-missing-layer-of-ai-governance-is-the-human-one/ ; WEF, “AI Agents in Action: A Playbook for Trusted Adoption, Authorization and Scaling” (ACAP).
2. Regulation (EU) 2026/1744 (Digital Omnibus on AI), OJ 24 Jul 2026; CSA research note, 1 Aug 2026; Gibson Dunn, Cooley, DLA Piper client alerts, May–Jul 2026.
3. IMDA, Model AI Governance Framework for Agentic AI, 22 Jan 2026; v1.5, 20 May 2026 (updated 5 Jun 2026); Baker McKenzie, Latham & Watkins, Covington summaries, Jan–Jul 2026.
4. South Korea AI Basic Act, in force 22 Jan 2026; Cooley, OneTrust, US ITA analyses.
5. Axios, “AI godfathers converge on regulations,” 16 Jul 2026; Axios on Hassabis proposal, 14 Jul 2026.
6. International AI Safety Report 2026, section on automation bias (arXiv 2602.21012).
7. Veritas JPS, “Automation bias in AI human oversight under the EU AI Act,” May 2026.
8. A. Masood, “The unbearable lightness of clicking approve,” Jul 2026; T. Pan, “The HITL rubber stamp problem,” Apr 2026 (mammography and insurance-denial figures as reported therein).
9. Parasuraman & Manzey (2010); Bainbridge (1983); Green & Kak (2021) on flaws of human-oversight policy.
10. Deloitte, State of AI in the Enterprise 2026, via TechTarget.
11. Jurisdiction summaries: StationX, BD Emerson, RAIL, Regulations.ai, AIUnpacking global trackers, Jul–Aug 2026. Verify against primary texts before relying on any date.

Luke Soon writes at GenesisHumanExperience.com. Views are his own and not those of PwC. Frameworks referenced (TrustOS, HX = CX + EX, The Fork, Prompts → Loops → Loop Governance, Frozen Workforce, Ghost GDP, Long-AND not Short-OR, the enforceability test) are developed across Genesis, Synthesis and prior essays.