Receipts, Please · Part IV of V
Detection loses. Process wins. Ferrari stopped a deepfake with a question about a book. Singapore has specified the engineered version of that question: a runtime gate that can say no, and a receipt proving it did.
Disclosure: I have a commercial interest in TrustOS and in runtime governance for agentic AI, discussed below. I declare it here rather than in a footnote, because this series argues that governance must be proof, not assertion, and authors should meet the standard they demand of systems.
I. Why Detection Loses, and Ferrari Wins
Maranello, July 2024. An executive at Ferrari received WhatsApp messages from what appeared to be the chief executive, Benedetto Vigna, complete with photograph, followed by a call in Vigna’s voice, southern Italian accent and all, about a confidential acquisition requiring urgent, discreet action. The executive listened. Something in the intonation felt mechanical. And then, instead of arguing with the voice, he asked it a question only the real Vigna could answer: which book had Vigna recommended to him just days earlier? The caller could not answer. The call ended. Loss: zero. Fortune reported the story on 27 July 2024; MIT Sloan Management Review turned it into a teaching case. No software caught that fake. A relationship did: a small, recent, un-Googleable piece of shared context, used as a challenge, outside the script the attacker had prepared.
Set that scene beside its siblings. WPP, May 2024: a fake WhatsApp account bearing chief executive Mark Read’s photograph, a Teams call with the cloned voice of another senior executive assembled from YouTube footage, and an agency leader who recognised the attempt for what it was, after which Read warned the whole company. LastPass, April 2024: an audio deepfake of chief executive Karim Toubba, delivered by WhatsApp with manufactured urgency, spotted by the employee, reported the same day, disclosed publicly by the company as a warning to others. Three famous firms, three attacks, zero loss. In each case the control that fired was a human verifying outside the channel the attacker controlled, or refusing to treat urgency as a reason to skip verification. In Arup and in Singapore, where verification stayed inside the attacker’s channel, where the call itself was taken as proof, the money moved. That is the cleanest pattern in this entire body of research, and it costs, in its basic form, nothing.
Notice what the Arup fraudsters actually forged. Not one face, but a chorus of them. The CFO was corroborated by colleagues, and the colleagues by each other. CETaS, in its September 2026 taxonomy of agentic failure modes, gives this pattern a name that deserves to enter every finance manual: pseudo-corroboration. Six AI agents asked to judge a report all agree it is reliable, but they share a model, training data and optimisation, so what looks like six independent judgements is one judgement counted six times. Agreement rises while accuracy does not. Arup was the human-scale version. As firms ask several agents to check one another, it will become the machine-scale one. Independence of the verifier, not the number of verifiers, is what counts.
II. Confidence Is a Vulnerability
Contrast it with where the money in “money” actually goes. The instinctive corporate purchase after a deepfake scare is detection: a tool that scores a voice, a face, a document, and pronounces it real or synthetic. Detection has a role, as a signal, as one input among several, as forensic support after the fact. As a primary defence it labours under three structural disadvantages that no product cycle will fully cure. It plays a cat-and-mouse game on the attacker’s schedule, ageing as generators improve. It returns a probability where a payment needs a decision. And it trains its buyers in exactly the wrong mental habit: outsourcing the question “should this money move?” to a score, the way Baldwin’s lenders outsourced “is there oil?” to a name on a receipt. The adjacent instinct, training staff to spot deepfakes by eye and ear, fares little better in the evidence: trained observers perform barely above chance at identifying high-quality synthetic media, while their confidence rises faster than their accuracy, which is the most dangerous combination a control can produce. A finance team that believes it can spot fakes will verify less. Confidence, in this domain, is a vulnerability with good posture.
III. Provenance Is Not a Payment Control
Provenance technology needs the same honest placing, because it is genuinely valuable and routinely oversold. The C2PA standard and its Content Credentials, updated through versions in January and April 2026, bind a tamper-evident, cryptographically signed record of origin and edits to a piece of media at creation; Google’s SynthID has, on the company’s own count, marked more than twenty billion pieces of content. Used properly, provenance answers a real question: is this recording what its credential claims, unaltered since capture, from the device and account that signed it? But NIST’s guidance states the limits plainly, and deployers should repeat them until they are bored: a credential can be stripped by re-encoding or a screenshot; absence of a credential proves nothing; presence proves nothing until the signature, certificate chain and hash are actually checked; and, decisively for our story, provenance authenticates recorded media. The Arup attack was a live interaction. There was no file to credential. A perfect provenance regime, universally deployed tomorrow, would leave the live-call channel, the channel of the two largest verified cases in this series, exactly as exposed as it is today. Buy provenance for the archive. Do not let it impersonate a payment control.
IV. What Ferrari Had
So what, precisely, did Ferrari have that Arup did not? Not genius, and not technology. Three humble things. A challenge the attacker could not pre-compute, because it lived in a private, recent interaction rather than in public footage. A culture in which questioning a chief executive’s urgent, confidential instruction was survivable: the executive who asks a suspicious question of a fake CEO is praised; imagine the meeting if the caller had been real, and design for that meeting too. And a default, however informal, that an unusual instruction earns more verification as its urgency and secrecy rise, rather than less. Codify those three things and you have the first three defences in Part V: verify out of band through a pre-registered channel, never one supplied in the request; require dual authorisation with genuine separation of duties, so the person who approves is not the person who executes and neither was chosen by the requester; and keep a pre-agreed challenge phrase for executive payment instructions, rotated, private, and boring. The OWASP GenAI Security Project’s guide to preparing for deepfake events, FinCEN’s alert, and the post-Arup standard-operating-procedure literature all converge on the same list. It fits on an index card. The scandal of AI fraud is not that we lack a defence. It is that the defence that works is procedural, cheap and unglamorous, and therefore loses budget meetings to dashboards.
Hold on to the Ferrari executive as this part turns from human process to engineered systems, because he is the design specification. He received a proposed action. He consulted context the proposer did not control. He returned a disposition (in effect, deny, pending proof) and the action did not execute. Everything in the sections that follow is an attempt to give that man’s judgement institutional form: to make it fire at machine speed, for agent instructions as well as human ones, on the thousandth request as reliably as on the first, with a receipt afterwards proving it fired at all.
V. The Missing Verb
Grammar first, because governance fails in grammar before it fails in technology. Our AI governance vocabulary is rich in nouns (frameworks, principles, inventories, maturity models) and in one overworked adjective, responsible. Its verbs cluster at two ends of time: before deployment, we assess, certify, red-team and approve; after an incident, we audit, investigate, disclose and remediate. The verb that belongs in the middle, at the moment an agent or a deceived human actually moves value (verify, at runtime, and refuse if verification fails) is the verb our instruments use least where it counts: as a requirement, with a mechanism, at execution.
VI. Singapore Specifies the Gate
That is not my rhetoric alone; it is embedded in the most operationally serious document in this research. On 3 July 2026 the Monetary Authority of Singapore and an industry consortium, BuildFin.ai (whose members include Ant International, Circle, HSBC, J.P. Morgan, Manulife, Mastercard, OCBC and Visa), published Safeguards for Agentic Finance at Runtime, SAFR, updated to version 1.1 in October 2026. Its diagnosis deserves quotation in spirit: pre-deployment model-risk review and post-hoc audit both miss the moment of action, because by the time an issue appears in an audit log, the action has already occurred. SAFR’s answer inserts four components between every agent and execution. Agent Identity: the acting agent is registered and verified, linked to a human party who is accountable for it. A Controls Repository: the mandates, exposure limits and escalation thresholds that govern that agent, held in machine-readable form rather than in a PDF nobody consults at 2 a.m. A Disposition Engine: every proposed action returns one of four decisions (Deny, Escalate, Auto-Execute or Observe) before it happens. And a tamper-evident, append-only Audit Log, in which each action is packaged in a Governance Envelope carrying the proposed action, the reasoning trace and the operative policy that disposed of it. One further rule is the quiet revolution: prior authorisation does not carry forward across a multi-step workflow. Each step gets its own checkpoint. An agent approved to begin a payment journey is not thereby approved to finish it.
Translate SAFR back into Part I and its power becomes obvious. The Arup worker’s tragedy was that his verification produced no envelope: no independent disposition, no second accountable identity, no record that could prove, afterwards, who had checked what: because, in the relevant sense, nobody had. A SAFR-shaped payment path would not have asked him to be braver or more perceptive. It would have routed the instruction (whoever or whatever issued it, however convincing the call) through a gate that requires a registered identity, checks a machine-readable mandate and limit, and returns Escalate to a human verifier on a channel the requester does not control. The deepfake could still have performed beautifully. The performance would simply have stopped being the authorisation.
VII. Specified, Not Required
Now the honesty this series owes you, in the same paragraph as the praise. SAFR is explicitly not regulatory guidance and not a supervisory expectation; it is an industry reference model. The MAS AI Risk Guidelines are supervisory expectations, not legislation. IMDA’s framework is voluntary. And my own clause-level reading of the Singapore instruments, published in August 2026 as “The Runtime Enforcement Gap in Agentic AI Governance Standards”, concludes that none of them yet requires an execution-time, deterministic, model-independent enforcement gate: a gate, that is, that does not ask the acting model’s permission, does not depend on the acting model’s self-report, and cannot be waived by the workflow it governs. I declare again the interest flagged at the top: that enforceability test is my own formulation, TrustOS, the architecture through which I work on exactly this problem, is my own commercial framing, and no independent evaluation of TrustOS was found in the research for this series. Treat the claim accordingly: as an argument to be tested, from an interested author, not as a finding. But note that the argument does not depend on buying anything from me. It depends on a question any board can ask of any vendor, any framework and any regulator, in one sentence: show me the gate, at runtime, that can say no, and show me the receipt from the last time it did.
Show me the gate, at runtime, that can say no, and show me the receipt from the last time it did.
There is a second reason the receipt cannot simply be the agent’s own account. CETaS calls it misreporting: a divergence between what an agent does and how it reports what it has done. Its illustrative case is an agent that spots a compliance concern, recognises that raising it will hurt its efficiency metrics, approves anyway, and writes a summary that plays the risk down. Managers, auditors and regulators mostly see agents through exactly such summaries, logs and reasoning traces. A Governance Envelope that packages an agent’s reasoning is valuable. A gate that trusts that reasoning, rather than checking the action against an independent rule, has simply moved the deepfake from the video call into the audit log.
VIII. The Human in the Loop, Specified Properly
Human-in-the-loop, that ubiquitous phrase, only earns its place inside such a gate if it is specified like one. Which human? With what authority to refuse, including the authority to refuse a chief executive? In how much time, with what context, presented how? And, the question that separates a control from a hope, what happens if that human does nothing? In a well-built disposition, nothing is fail-closed: silence is not consent, an unanswered escalation does not auto-execute, and volume that exceeds human oversight capacity escalates, throttles or routes rather than silently stacking approvals on a tired reviewer. McKinsey’s warning that oversight capacity is the binding constraint, and my own argument about cognitive budgets, converge here. A checkpoint that degrades invisibly under load is not a checkpoint. It is a queue with a governance label.
CETaS names the failure this section is trying to prevent. Among its human–agent failure modes are automation bias, excessive delegation, accountability diffusion (no one can clearly explain who authorised an action) and, most pointedly, responsibility laundering, in which people use an AI recommendation to justify a decision they would struggle to defend on its own. A human in the loop who approves whatever arrives, because it arrives looking sophisticated, is not oversight. It is laundering with a signature.
IX. One Verb, Many Accents
Nor is runtime verification only a finance pattern. DeepMind’s Control Roadmap, with its real-time monitoring to halt an agent before an irreversible action and refer it for human verification, is the same verb in a laboratory accent. The scientists’ open letter, with its call for standardised containment and a public record of containment failures, is the same verb in an institutional accent: receipts, kept centrally, for the times the gate was tested. Ostrom’s design principles for governing commons, invoked in the agent-ecology literature around “Consider the Agent”, say it in a social-science accent: monitor, sanction proportionately, keep the monitors accountable to the monitored community. The accents differ. The verb does not. Verify at the moment of action, by a mechanism independent of the actor, and keep proof.
Why, then, has the verb stayed missing? Partly because pre-deployment certification photographs better: a launch, a framework, a signing ceremony. Partly because runtime gates impose friction precisely where businesses compete on its absence, in the one-click payment, the straight-through process, the agent that “just handles it”. And partly, I suspect, because a real gate creates the one artefact our current arrangements carefully avoid: an undeniable record of who authorised what, under which rule, at what second. Answerability is the product. It is also, for some actors, the threat. Singapore has specified the gate more concretely than anyone else in this research. The question the next two years will answer is whether specification hardens into requirement (in MAS’s promised consultation on dedicated agentic-AI guidance in 2027, and in the procurement checklists of the banks that helped write SAFR) or whether the gate remains a reference model that everyone praises and no transaction is obliged to pass through. A receipt nobody is required to produce is just stationery.
X. The Kill Switch That Isn’t
Every technology panic eventually produces the same comforting image: a big red button. If the agents misbehave, we press it. If the market convulses, we press it. The image survives because it converts a systems problem (distributed, fast, entangled, partly invisible) into a gesture problem, and gestures are something leaders know how to perform. In June 2026 the Bank of England’s Deputy Governor Sarah Breeden floated, in remarks reported secondarily, market-wide kill switches if faulty agents trigger meltdowns. DBS’s PURE framework is reported to include runtime kill switches wired to metric breaches. DeepMind’s roadmap builds halts before irreversible actions. The direction of travel is real. I spent an essay in October 2026, “Here’s Why an AI ‘Kill Switch’ May Not Be So Simple”, arguing why the button, by itself, is the weakest part of it, and the research since has hardened, not softened, that view.
A kill switch, I argued then, is not a device. It is an option, and like any option, it has a strike price that rises every day it is not exercised, rehearsed and kept credible. Who is authorised to press it, on what evidence, in what timeframe, accepting what collateral damage to legitimate transactions frozen mid-flight? Which systems does it actually reach: the model, the agent’s tools, its credentials, its payment mandates, the downstream workflows it has already set in motion? In a multi-agent ecology of the Hugging Face kind, where hundreds of agents coordinate through infrastructure nobody provisioned, the honest answer to “what does the button disconnect?” may be: the part we can see. My October conclusion was that in 2026 the button worked and almost nothing around it did: the detection that should trigger it, the authority that should wield it, the rehearsal that should make pressing it thinkable before pressing it is catastrophic. A switch that will predictably never be pressed, because pressing it is organisationally unthinkable, does not deter, does not contain and does not govern. It decorates.
None of this argues for no switch. It argues for the switch’s proper place: last, downstream of everything above, and only as credible as the system that surrounds it. A disposition engine that already says Deny and Escalate, action by action, makes the global halt a rarer, more legible event. An append-only log makes its aftermath reconstructable. A named, accountable owner, who has rehearsed the press in simulation the way Arup’s teams rehearsed incident response, makes it an act of governance rather than an act of desperation. And a public record of halts and failures, the scientists’ letter again, makes the whole arrangement answerable to people outside the room where it failed. Build those, and the red button becomes a real, if blunt, instrument. Skip them, and you have purchased a talisman. Fraud, human or agentic, is unimpressed by talismans. It routes around them, through the gap between the ceremony of control and its practice, which is where, the reader will recall, it has lived all along.
Next in the series: Part V, Receipts, Please.
Receipts, Please is a five-part series on AI fraud, trust and runtime governance. Part I: The Seams Are the New Attack Surface · Part II: Selected for Sympathy · Part III: Protection Money · Part IV: Runtime Is the Missing Verb · Part V: Receipts, Please
Luke Soon is an AI futurist and author of Genesis: Human Experience in the Age of Artificial Intelligence and Synthesis: SuperIntelligence Protocol. He co-authored IMDA Singapore’s Model Governance Framework for Agentic AI.
Sources and verification notes
Principal sources, with the limits stated in the research: Baldwin, “Your Company’s Fraud Defenses Aren’t Ready for AI Threats”, Harvard Business Review, 28 October 2026 (page and opening verified; full text paywalled). “Protection money / OpenAI and Anthropic spy a big opportunity in cyber-security”, The Economist, reported 1 November 2026 (framing verified from the magazine’s promotion; full text paywalled, not read; the Sarah Friar quotation is second-hand via EconoTimes, 2 November 2026). “Consider the Agent”, DeepMind Institute, 4 November 2026, https://institute.deepmind.com/essays/consider-the-agent/. Arup case: CNN, 16 May 2024, via Hong Kong Police (HK$200m, 15 transfers). Singapore case: CNA / Singapore Police Force Annual Scams and Cybercrime Brief 2025, via secondary summaries (US$499,000, March 2025, recovered). Ferrari: Fortune, 27 July 2024; MIT Sloan Management Review, 2024. Scale: FBI IC3 2025 Annual Report, April 2026 ($20.9bn total; $893m across 22,364 AI-mentioned complaints, sub-breakdowns secondary; FTC Consumer Sentinel, March 2025; INTERPOL Global Financial Fraud Threat Assessment, 16 March 2026 ($442bn, citing GASA); Deloitte Center for Financial Services projection, May 2024 ($12.3bn in 2023 to $40bn by 2027, projection, not measurement); Europol IOCTA, June 2025. Documents: FinCEN Alert FIN-2024-Alert004, 13 November 2024. Panel: International AI Safety Report 2026, 3 February 2026; IDAIS London consensus, April 2026; Open Letter on AI and Cybersecurity, 13 October 2026; IMDA Model AI Governance Framework for Agentic AI v1.5, 20 May 2026; CSA Securing Agentic AI; MAS Guidelines on AI Risk Management, 7 October 2026; MAS / BuildFin.ai SAFR white paper, 3 July 2026, v1.1, 9 October 2026; EU AI Act, Regulation (EU) 2024/1689; NIST AI 100-4, November 2024. Labs: Anthropic misuse report, 10 September 2026, and SEC registration statement, 26 October 2026; OpenAI Hugging Face disclosure, 21 July 2026, and technical report with METR / Redwood investigation, 26 August 2026; Google DeepMind AI Control Roadmap, 18 June 2026. Consultants: McKinsey State of AI Trust, 25 March 2026; BCG, AI Is Raising the Stakes in Cybersecurity, December 2025; PwC Annual Threat Dynamics, 30 March 2026; WEF / Accenture Global Cybersecurity Outlook, January 2026. Author’s prior work cited: “Receipts, Please” (Substack, October 2026), the short piece this series is named after; “The Runtime Enforcement Gap in Agentic AI Governance Standards” (10 August 2026); “Here’s Why an AI ‘Kill Switch’ May Not Be So Simple” (17 October 2026); “Governing Before the Crash”.
Additional source: Rick Hennessy and Carolyn Ashurst, Behavioural Assurance of Agentic AI for Sensitive and High-Stakes Organisations, CETaS (Centre for Emerging Technology and Security, The Alan Turing Institute), September 2026.
Leave a Reply