The Copy, the Chokepoint, and the Third Answer:
Distillation annoys the frontier labs, but annoyance is not a strategy: the real contest is whose models the world builds on, and who can prove what their agents did.
By Dr Luke Soon · 31 October 2026
This is an opinion piece.

“Distillation annoys Anthropic and OpenAI. That is not reason enough to ban it.”
The Economist, standfirst to the leader “Let China copy American AI”, 29 October 2026
I. The examination hall that never closes
Imagine an examination hall that never closes. At the front sits a student who has spent billions learning how to think: not merely how to answer, but how to reason, step by step, through problems no textbook has solved. Around the room, other students are not sitting the paper at all. They are asking the first student question after question, millions of them, carefully chosen and systematically varied, and writing down every answer. Then they go home and train a smaller, cheaper version of themselves on the transcript.
Is that learning? Cheating? Theft? Or is it what intelligence has always done: watch a master, imitate the moves, and eventually internalise the craft?
Now leave the hall and walk into a bank in Raffles Place.
A model-risk officer is working through the new MAS Guidelines on AI Risk Management, issued on 7 October. The Guidelines make her institution accountable for third-party AI. She must obtain assurance on it, and she may limit, suspend or replace it if it sits outside risk appetite. On her desk is a proposal to deploy an open-weight model built on a Chinese base. She can test its accuracy. She can test its bias. She can red-team its refusals. But there is one question she cannot answer, and nor can any agency in Singapore: what is in its ancestry?
This is not hypothetical. Singapore’s own national language model programme, SEA-LION, ships a flagship variant built on Alibaba’s Qwen3-32B, alongside variants built on Google’s Gemma. In September, an advisory from the NSA, CISA and the FBI alleged that Alibaba distilled Claude and GPT models in late 2025 to improve Qwen. One honest caveat: Qwen3-32B was released in April 2025, before that alleged campaign, so today’s flagship is probably clean of it. Later Qwen-based variants are another matter. The point is not that SEA-LION is tainted. The point is that nobody can currently say, with evidence, whether any given model is.
That unanswerable question is this essay.
My argument fits on one line. Learning from outputs cannot be policed and fraud must be, but no one can yet tell the two apart. Evidence is therefore the instrument: a ledger of where a model came from before it is deployed, and proof of what an agent did after it acted. Police the door, keep the ledger. Singapore can be a credible keeper of both, provided it is honest about where it is aligned and where it is not.
One disclosure up front, repeated at the end. Research for this essay was assisted by Claude, an Anthropic model, and Anthropic is a named party in this dispute. Weigh my use of its figures accordingly. I have tried to.
Five provocations to argue with
1. If copying is cheap, the moat was never the model. It was compute, distribution, trust, and the pace of learning after the copy ships.
2. A ban on learning from outputs would also ban how America learned. The wrong is fraud at the door, not the lesson.
3. America can win every benchmark and lose the platform. Power sits with the default, not the leader.
4. The most dangerous copy is not the smart one. It is the unanswerable one. Capability without answerability is the risk, wherever it is built.
5. For middle powers, “we pick no side” is not a strategy. Managing dependence is.
II. A war without evidence
Start with what each side claims, and then with what anyone has proved. They are different lists.
The American account. On 8 September 2026 the NSA, CISA and FBI published joint advisory AA26-251A. It names six Chinese firms (DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun and Z.AI) and alleges industrial-scale distillation of American frontier models since at least late 2024, probably with the Chinese government’s awareness. It calls distillation the core of their AI strategy rather than a supplement.
Its description of the tradecraft is the most useful part. Fraudulent accounts with cleaned metadata. Requests routed through cloud providers and aggregators that strip identity. Bulk subscriptions, account pools and automatic failover when an account is blocked. Jailbreaks that ask a model to narrate the reasoning behind an answer it has already given. And “transfer stations”: grey-market proxies that, in the advisory’s words as reported, resell access “at a fraction of the official price”. The spy in this story is not a person in a trench coat. It is a request router that never sleeps.
The company disclosures came first. In February, Anthropic said three Chinese labs had generated over 16 million exchanges through roughly 24,000 fraudulent accounts. In June it accused Alibaba of nearly 29 million exchanges through nearly 25,000 accounts. Anthropic also said MiniMax redirected nearly half its traffic to a new Claude model within 24 hours of release. Around 1 October, OpenAI disclosed disrupting a Moonshot-linked campaign to extract reasoning during July. Every one of these is an allegation by an interested party. None has been independently audited.
Two details from the same record cut the other way, and the louder commentary tends to omit them. Of Anthropic’s February total, DeepSeek, supposedly the arch-copier, accounted for only about 150,000 exchanges, as LawAI points out. And Google’s Threat Intelligence Group, reporting in February, described a single campaign of over 100,000 prompts aimed at Gemini’s reasoning. Google named no country. It attributed such activity to private-sector entities and researchers worldwide, and said it had observed no direct attacks on frontier models by tracked state-backed threat actors during 2025.
The political escalation. Michael Kratsios’s Office of Science and Technology Policy issued memorandum NSTM-4 on 23 April, on “industrial-scale” distillation campaigns by entities based mainly in China. Congress moved in parallel: DAAMTA (H.R. 8283) was introduced in the House in April, and the BLADE Act in the Senate in August. In July, after Moonshot released Kimi K3, Kratsios alleged on X that Moonshot had distilled Anthropic’s Fable model. Treasury Secretary Scott Bessent threatened sanctions. On 29 October, the Special Competitive Studies Project published “China’s Adversarial AI Distillation of U.S. Frontier Models” by Lara Jolicoeur and Evan Armstrong, the most complete hawkish case so far.
The Chinese account. On 27 July, China’s Commerce Ministry said American firms distil Chinese models too, and called the accusations “double standards” and “hegemonism”. It named no American firm. In September it called the advisory’s claims groundless, described distillation as a neutral technique, and promised “resolute measures” in response. Moonshot denied distilling Fable and credited K3’s gains to its own architecture.
What is actually in force. As of mid-October, no sanction, Entity List designation or commercial ban aimed specifically at distillation was in force. Neither bill is law. The rhetoric is maximal. The legal record is nearly empty.
I make one correction to my own earlier working notes here, because provenance is the theme. The widely shared chart showing Chinese models seven months behind the US frontier is Epoch AI’s, published on 2 January 2026. SCSP’s 29 October paper reuses Epoch’s index and builds its argument on it. Credit Epoch for the measurement and SCSP for the argument. In an essay whose thesis is that ancestry matters, the ancestry of a chart matters too.
So this is a war in which every accusation is unverifiable and so is every denial. Bessent has said American watermarks were found in Chinese models, and published no technical detail. Moonshot says its architecture explains K3, and published nothing a third party could test. Nobody is lying in a way anyone can prove. That is precisely the problem.
III. What distillation is now, and what copying buys

Precision first, because most commentary fails here.
Classic knowledge distillation, as Hinton, Dean and Vinyals described it in 2015, trained a small model to match a large model’s full probability distribution. Frontier APIs do not expose that. What is called distillation today is something cruder and more targeted: harvesting a teacher model’s outputs, above all its reasoning traces, and using them as supervised training data for a student.
How much does that buy? Less than the panic suggests. Beren Millidge argues the technique is oversold: traces give a sample-efficient cold start, but the big gains now come from reinforcement learning on a model’s own attempts. Nathan Lambert judged any distillation contribution to Kimi K3 small. CNAS notes that isolating distillation’s effect would require training runs costing hundreds of millions of dollars, which is why nobody has publicly quantified it. On Fable specifically, the window between Fable’s return to general access on 1 or 2 July and K3’s release on 16 July was about two weeks, too short, in the view of sceptics such as Braden Hancock, to distil, train and ship.
A lead is a flow, not a stock. Epoch’s figure is real: since 2023, Chinese models have trailed the US frontier by an average of seven months, within a range of four to fourteen. Epoch separately finds open models about four months behind closed ones. SCSP reads the same index as showing DeepSeek-R1 brought China within about three points of the frontier in January 2025, with distillation a “plausible contributor”. These numbers measure different things. Seven months on a benchmark composite is not a capability ratio, not a deployment gap, and not proof that distillation caused anything. Anyone who gives you a single gap figure without its measure is selling you confidence, not knowledge.
The months are still substantial. The leader enjoys them exclusively: it learns from deployment first, finds the failure modes first, sets the interfaces others must match. But they are perishable. They must be re-earned every quarter. No country wins this once and rests.
History says the same thing with more texture.
- Samuel Slater, 1789. He memorised Arkwright’s textile machinery in defiance of British export bans and opened a mill in Pawtucket in 1793: “Slater the Traitor” at home, father of an industry abroad. Codified knowledge leaks through people. An API is a Slater who never stops talking.
- The Tupolev Tu-4, 1944–49. Stalin ordered an exact copy of three interned B-29s. It took some 900 factories and institutes and 105,000 drawings, and the metric sheet-metal problem forced re-engineering. Copying still needs a vast absorptive base.
- The Soviet bomb, 1949. Espionage probably accelerated it by six months to two years, but plutonium production stayed the bottleneck. Stolen knowledge compresses timelines, not capacity. In AI, the bottleneck is compute.
- Dickens in America, 1842. He campaigned for international copyright and was attacked as mercenary. The United States protected foreign authors only in 1891. Copiers become enforcers once they lead.
- CoCom, 1949–94. The Western export-control regime broadly worked because it was multilateral, and leaked anyway. Today Washington acts largely alone on model-level controls.
The lesson is consistent: copying the leader buys months, sometimes a couple of years, never decades.
But I want to resist the comfortable version. My own image for it is the water-skier in the boat’s wake: faster than open water, never overtaking the boat. The follower does not overtake the boat until it builds a different boat. Three things can let a copier escape the wake. Efficiency: DeepSeek forced a global repricing of what capability should cost. Constraint-driven architecture: DeepSeek’s V4 preview was optimised for Huawei’s Ascend chips. And diffusion, which compounds. The Tu-4 is the cautionary tale: the Soviets copied the bomber rivet by rivet, then built their own aviation lineage on what they learned.
The Economist’s 25 October science piece, “China increasingly depends on copying American AI”, carries a standfirst that frames the question neatly: distillation “infuriates America, but may end up holding China back”. Both outcomes are live. Which one happens depends less on the copying than on what each side does next.
There is a philosophical point here that strategic commentary rarely makes. A civilisation’s relationship to copying is a relationship to its own confidence. Societies that believe they are generating the future treat imitation as a phase to be surpassed. Societies that fear they no longer are treat every imitation as theft of a dwindling inheritance. The confident response to being copied is to keep inventing and enforce honestly against fraud. The fearful response is to freeze the present distribution of capability by law. Only one of those has ever worked for long.
IV. The apprentice and the forged letter
Distillation’s own vocabulary is “teacher” and “student”. That should tell us something.
In the Chinese tradition of linmo (临摹), the calligraphy student copies the master stroke by stroke, sometimes tracing, sometimes freehand beside the original, as the path to a hand of their own. Copying is the curriculum, not the crime. Byung-Chul Han, the Korean-born German philosopher, argues in Shanzhai that Chinese thought locates the original in continuous transformation rather than a single act of creation. Western thought arrives at a similar place by another road. Jefferson wrote that an idea passed to another does not diminish the giver, as one taper lights another without darkening itself. Newton stood on the shoulders of giants.
So the ethics of imitation is largely settled, East and West, in favour of learning. Watch how this applies to the cases, because it is not a licence for everything.
An apprentice who copies the master’s brushwork has done nothing wrong. An apprentice who forges a letter of introduction to get into the master’s studio has. The wrong has nothing to do with brushstrokes. It is about how they got through the door.
That is where the American labs have trapped themselves. They framed distillation as theft. Lisa Klaassen, writing in Lawfare on 8 October, sets out the paradox precisely. Either model outputs are protectable, which strengthens the authors suing the labs for training on their books, or they are not, which sinks the case against China. In her words, “both propositions cannot hold simultaneously”. Her sharpest inversion: the Chinese distillers mostly paid for API access, while the labs’ own training data was scraped, and in Anthropic’s case partly pirated, as Judge Alsup found before the US$1.5bn Bartz settlement. Elon Musk put the same point less politely in February: “How dare they steal what Anthropic stole from human programmers.”
Locke helps here, and cuts both ways. The labs have a genuine labour claim: they mixed enormous effort and capital with raw material and produced something new. But Locke’s proviso, that appropriation is legitimate where “enough and as good” is left for others, sits awkwardly with enclosing a capability built from a scraped human commons.
The way out for the labs is not “outputs are ours”. It is “fraud is fraud“. That claim is smaller and defensible. Read fairly, the American case already rests on three narrower pillars than theft: investment and fair competition, deception, and safety stripping. Each survives deep scepticism of American self-interest. Fraudulent accounts, geographic evasion and jailbreaks designed to extract withheld reasoning are evidence that the actors knew the activity would be blocked if done openly.
But the claim must be kept narrow, provable and symmetric. An American firm extracting a rival’s hidden reasoning through fraudulent accounts should face exactly the same consequences as a Chinese one. That is the only real answer to MOFCOM’s “double standards” charge. Hypocrisy defeats the intellectual-property frame. It barely touches the security frame, which turns on deception and on who ends up with the capability. The labs should argue the case they can win.
This gives us a test with three parts, and the rest of this essay hangs on it:
- The act of learning is presumptively legitimate. No gate is needed, and none is possible.
- The means of access is legitimate only if honest: no fraud, no evasion, no extraction of what a provider deliberately withheld.
- The conditions of redeployment are legitimate only if whoever releases the capability takes responsibility for its safety and answerability, in proportion to what is released.
A regime built on those three distinctions governs conduct, not nationality. That matters, because it is the only kind of regime a third country could adopt without joining a bloc.
V. Police the door, not the lesson

Chips were supposed to be the moat. APIs became the bridge. Export controls govern the mountain of compute needed to train a frontier model. They do not, by themselves, govern the river of answers that a deployed model serves. A compute-poor lab can buy capability by the token rather than build it by the cluster.
Temper that, though. Anthropic’s own counter-argument is that extraction at scale itself needs serious compute, so chip controls constrain distillation as well as training. Chips and APIs are complements as well as substitutes.
And notice what Washington actually did this year. From 15 January, H200 exports to China moved to case-by-case licensing with a 25 per cent surcharge. On 12 June, the Commerce Department used export-control authority against a specific frontier model for the first known time, prompting Anthropic to disable Fable 5 and Mythos 5 globally, because a nationality-based restriction could only be complied with by switching the model off for everyone. Washington loosened the chokepoint on chips and moved the fight to the model in the same year. The bluntness of the June episode is instructive: model-level controls are a sledgehammer.
So which instruments actually work? I apply the test I have used throughout this series: the enforceability test. A control counts only if it can refuse a specific action at the moment of action, deterministically, in bounded time, and independently of the model it restrains. Anything else is a report, written after the fact. A “should”, not a “shall”. (I set out the test in full in Should and Shall and will not re-explain it here.)
Apply it to the menu, and add a second axis my earlier work lacked: legitimacy.
| Instrument | Gate or report? | Reaches actors abroad? | Legitimacy cost | Verdict |
|---|---|---|---|---|
| Ban on learning from outputs | Report, after the fact | No | High: Klaassen paradox; hits US open source | Fails |
| Sanctions on named distillers (BLADE, DAAMTA) | Deterrent, after the fact | Partly, via finance | Medium to high without published evidence | Use on top of gates, with evidence and reversal conditions |
| Ban distilled models from the US market | Gate at procurement only | No | High: hits America’s own ecosystem | Narrow to government use on stated grounds |
| Know-your-customer checks on API accounts | Gate | Yes, at the point of access | Low | Passes; leaks via proxies and stolen cards |
| Proxy blocking and behavioural rate limits | Gate | Yes | Low | Passes; whack-a-mole |
| Withholding or summarising hidden reasoning | Gate by design | Yes | Low; some cost to honest users | Passes; strongest single measure |
| Subtly altering outputs to suspected distillers | Gate-like | Yes | High: corrodes trust in every answer | Passes enforceability, fails legitimacy unless governed |
| Pre-release test gate (Hassabis) | Gate at release, US only | No | Medium: cartel risk | Complements; blind to provenance |
The result is striking. The policy that survives the test is the one the expert consensus already prefers. Bahrad Sokhansanj at LawAI argues that “theft” is the wrong word and that deception is what should be prosecuted. Lambert calls “distillation attacks” “a horrible term” and says to secure the APIs. Variant’s lawyers say the labs hold “a right without a remedy” and predict a shift to licensing. My contribution is to show why they are right: only controls on access can be gates. Controls on outputs can only be reports.
The architectural defence is the strongest single measure. Stop serving the thing the extractor most wants, hidden reasoning, in extractable form. Summarise it, restrict it, sign it. The advisory itself points to obfuscating reasoning traces among its countermeasures. It is not glamorous. It is a gate built into the product.
A warning inside the toolkit. The advisory also suggests subtly altering responses to suspected distillers, flagged for legal review. This passes the enforceability test and fails the legitimacy one. If providers routinely serve subtly wrong answers to users they suspect, on criteria nobody discloses, every user’s confidence in every answer weakens a little, and the line between counter-extraction and quiet manipulation blurs. Klaassen observes that it is the advisory’s most concrete suggestion. I would go further: it corrodes the epistemic commons of the API itself. Govern it like any exercise of power over an information environment, with published criteria, oversight and an answerable owner, or do not do it.
This is Long-AND, not Short-OR in its defence-in-depth form. Not “ban distillation OR open everything”, but layered gates that each hold independently: identity checks AND proxy blocking AND reasoning withheld AND intelligence shared across labs. That last one needs an antitrust safe harbour, which Sokhansanj and others have proposed, because a defence that cannot coordinate is a subsidy to the attacker.
None of these makes distillation impossible. Each raises the cost, adds noise and imposes delay. In a race measured in months, delay is strategy. Raise the cost, enforce against conduct, and defend the frontier by moving it.
VI. Winning every benchmark, losing the platform
A benchmark measures a model. Diffusion measures an ecosystem. In the ecosystem, the unit of power is not the best system in a San Francisco lab. It is the default: the model a startup in Lagos fine-tunes because it is free and good enough; the model a ministry in Southeast Asia runs on its own servers because citizen data cannot leave the country.
On that measure the picture inverts. Klaassen cites figures showing Qwen’s share of new open-model fine-tunes rising from about 1 per cent to about 69 per cent. A mid-2026 Hugging Face analysis, as summarised, reportedly finds most American open-weight releases above 100 billion parameters are now built on Chinese bases. Anysphere acknowledged that its Cursor product was built on an open-source model from Moonshot, the very firm the White House accused.
So at the closed frontier, knowledge flows from America to China. In the open-weight ecosystem it increasingly flows back. Call it the boomerang. A ban on “distilled” Chinese models would cut into America’s own open ecosystem, and into Singapore’s national model.
This is why I say America can win every benchmark and lose the platform. For a finance ministry, good enough and sovereign beats best and rented. For a developer, good enough and free beats best and metered. That one line explains SEA-LION’s base-model choice better than any geopolitical theory.
Jeffrey Ding calls the contest a diffusion marathon, not a sprint to invention. Dario Amodei sees chips deciding whether the world is unipolar or bipolar. On Ding’s view, cheap Chinese models spreading worldwide matter more than a few months of frontier lead. On Amodei’s, distillation is a leak in the moat. Both can be partly right, which is the Long-AND answer again.
SCSP, notably, lands near the same place from the hawkish side. Its remedy is not a ban but a funded, competitive American open-weight ecosystem with pre-release evaluation. Amodei himself said on 27 July that Anthropic “has never advocated for a ban on open-weights models”. The serious hawks want to compete on diffusion, not abolish it.
Who gains from the copy? The Economist’s leader, as reported in one secondary roundup, argues that distilled open models force price competition and spread cheap AI to poorer countries. The Takshashila Institution notes that distillation “radically compresses R&D costs” for Indian firms. If copying keeps Chinese labs a step behind while cheap copies serve the world’s users, the beneficiary is neither Washington nor Beijing. It is the developer in Jakarta.
But each capital is one self-deception away from losing a race it thinks it is winning. Washington’s self-deception is that the benchmark gap is the race. Beijing’s is that adoption is the same as trust. A ministry can download a model and still refuse to trust it with sensitive workloads. Adoption bought by price can be lost on provenance. And price buys users, not the workloads that confer real strategic power.
VII. Safety is the part that does not distil
Abstract warnings about “unsafe AI” have been devalued by overuse. So let me be concrete.
A frontier model, as a serious lab deploys it, is not just weights. It is weights plus an evaluation regime that probed for dangerous capabilities before release; refusals trained and tested against adversarial prompts; monitoring of how the system is actually used; incident response when things go wrong; and the institutional memory, often bitterly acquired, of where the model fails.
Distillation transfers the observable behaviour: the answers. It does not transfer the apparatus. A refusal, after all, is a short piece of text. The machinery that decided to refuse, and that would also refuse the ten thousand paraphrases of the same request, does not travel with the transcript. Compress the behaviour, lose the context that made it safe.
This resolves a dispute that otherwise goes in circles. Anthropic, SCSP and others argue that distilled models lose safeguards. Cisco found DeepSeek-R1 failed to block harmful prompts in testing, and a small preprint found a distilled medical model more harmful than its teacher. Others argue distillation can carry over some of the teacher’s guardrails. Anthropic itself reported DeepSeek using distillation to add “censorship-safe” answers, which installs different guardrails rather than none. Klaassen adds the fair baseline: a model built from scratch can also lack safeguards.
Both sides are partly right. Some behaviour that looks like safety may transfer. The safety system does not. Either way, the lever for safety is not trade sanctions. It is release governance, pre-deployment evaluation and runtime control.
The military dimension needs care, because research activity, state use and operational deployment are three different headlines. Reuters reported on 31 July that researchers linked to the People’s Liberation Army had used outputs from American models, mostly older and smaller ones, to train systems for defence purposes. Papers show research intent, not confirmed deployment. SCSP documents open-weight Chinese models, DeepSeek among them, reaching police and military procurement. That evidence concerns open-weight models in general, not distilled capability specifically. And the September advisory itself makes no PLA link. Cite each claim to its source. The credibility of the warning is the warning’s power.
Now the point both capitals are underweighting. The debate is still framed around models. The systems being deployed are increasingly agents: models wired to tools, data, payment rails, code repositories and customer channels, acting over time with delegated authority.
A model that gives a dangerous answer is a hazard. An agent that books, buys, deletes, approves or transfers is an incident.
A distilled model of unknown ancestry, with its safety system stripped, dropped into an agentic stack without runtime evidence, is not merely a careless chatbot. It is an unanswerable actor. Its deployer has bought capability without answerability. That gap applies with equal force in San Francisco, Hangzhou and Singapore.
VIII. Three gates: access, release, runtime

If access is the first gate, the second is release, and the most serious design for it comes from Demis Hassabis.
On 14 July 2026 he published “A Framework for Frontier AI and the Dawning of a New Age” on his Substack. The Economist ran an exclusive interview alongside it, “Demis Hassabis has a plan to harness AI safely”, later read on its Editor’s Picks audio on 21 July. His proposal, as summarised by The Next Web and the Cloud Security Alliance:
- A FINRA-style standards body. Public-private, funded mostly by industry, overseen by the federal government, with independent technical experts plus open-source and government voices on the board. Not a pure industry body, and not an ordinary agency that could not move fast enough.
- A capability threshold. “Frontier-class” status set by benchmarks the body refreshes, perhaps quarterly to start. Startups and academic models exempt.
- A two-stage gate. Labs submit models voluntarily up to 30 days before release. Once the protocol proves robust, passing becomes mandatory for deployment in the US market, applying to models “no matter their country of origin”, open or closed.
- Tests for cyber-attack capability, biological and nuclear risk, and deception; duties to publish model cards, keep strong security and resource safety teams.
- A ratchet: coordinating a slowdown among frontier labs “if deemed necessary”.
Here is the link to distillation, and it is my inference, not his. No source I have found shows Hassabis connecting his gate to distillation or to China. But because his mandatory gate applies regardless of origin, a distilled Chinese open-weight model that crosses the threshold and is offered in the US would have to pass the same tests as an American one. That is a release-side answer that needs no proof of theft. It judges a model on what it can do, not where it learned. It fits the third limb of the test, the conditions of redeployment, exactly.
It also has three limits. It is blind to provenance: a distilled model that passes is lawful to deploy. It stops at the US border: an open-weight model released in Hangzhou or Singapore is beyond it. And it carries a cartel risk: a body funded by the industry it governs, able to delay a rival’s release, must prove its independence in public before it is trusted with that power. There is also an irony worth naming: Google, Hassabis’s own company, is among the complainants about distillation.
Most importantly, the gate tests the model before it ships. It says nothing about what an agent did afterwards.
That is the third gate: runtime. I have argued in this series that governance has moved from Prompts → Loops → Loop Governance. In the prompt era we checked outputs. In the loop era, the machine plans, acts, observes and repeats. The unit of oversight must become the loop itself: its objective, boundaries, telemetry, escalation and stop authority.
Governance used to mean showing what you believed. Now it means showing what you can prove. A policy proves intent. A model card proves design. A pre-release test proves behaviour in the laboratory, on the day. None of them proves what a particular agent, on a particular run, under a particular person’s delegated authority, actually did. For that you need a runtime governance layer that sits outside the model:
- Four reconstruction questions for every material action: what did the agent know, how did it choose, who allowed it, and what changed?
- Logs the agent does not write. Agent-independent, tamper-evident, append-only. An agent’s narration of its own reasoning is a memoir, not an audit.
- A kill switch as three capabilities: stop new work by revoking credentials; freeze in-flight work cleanly; know what already happened, so it can be unwound in the right order.
- Least privilege as blast-radius control. You may not fully know the model, so bound what any model in that seat can touch.
- Review that is real. A reviewer who approves escalations in nine seconds is not reviewing. That is liability laundering: a human signature that absorbs blame for a machine-speed process.
A model can generate an answer. It cannot be the answerer.
So: access (Washington’s gate, scored by the enforceability test), release (Hassabis’s gate) and runtime (the deployer’s gate). Each is necessary. None suffices alone. What is still missing is the piece that sits between them: a ledger of ancestry.
IX. The missing ledger: Singapore’s third answer

Every arms race eventually arrives, with an invoice and a friendship bracelet, at the doors of countries that did not start it.
In 2026 the blocs acquired names. Singapore was one of the original signatories of the US-led Pax Silica declaration in December 2025. China founded the World AI Cooperation Organization, WAICO, in Shanghai on 16 July 2026, with 29 founding members, including five ASEAN states. Singapore has been invited to join WAICO and is considering it. Asked in Parliament the day after the US advisory, Minister Josephine Teo said: “There is no Asean position on either Waico or Pax Silica.” In effect, Singapore’s approach is case by case.
That phrase is easy to mock as hedging. It should be studied as statecraft. On 7 October Foreign Minister Vivian Balakrishnan supplied the doctrine: “as far as we can do so, we will refuse to choose.” In June, PM Lawrence Wong supplied the self-awareness: building frontier models is “not our competitive advantage”, and Singapore should be a place where models are tested.
But I want to sharpen this, because neutrality is not a strategy; dependence-management is. Join one bloc wholly and a small state trades autonomy for protection that may not cover its needs. Refuse both grandly and it discovers that its banks still run on someone’s cloud, its developers still fine-tune someone’s models. A country can stay diplomatically non-aligned while becoming technologically dependent.
Singapore’s real position is layered, not neutral:
- On hardware and security institutions, it is aligned with the US. Pax Silica, prosecution in the Nvidia server fraud case, membership of the international network of AI safety institutes, which does not include China.
- On models and testing methods, it is agnostic. SEA-LION on Qwen and Gemma; test tooling that does not care whose model it tests; an MAS rule that treats any model as supplier risk.
- On safety science, it convenes. The Singapore Consensus, now in its second edition, is one of very few documents co-signed by leading American and Chinese researchers.
The distillation fight attacks precisely the middle layer, model use and access, where Singapore chose agnosticism. That is the danger. It is also the opportunity.
What Singapore already has. Read the instruments for what they could do about distillation, not for what they were built for.
- MAS AI Risk Management Guidelines, issued in final form on 7 October 2026. Sections 3 and 4 apply from 7 October 2027; Sections 5 and 6 by 7 October 2028; a consultation on agentic AI is planned for 2027. The Guidelines cover agents but set no separate requirements for them yet. The crucial clause is on third parties: the institution stays accountable for vendor AI, must obtain assurance, and may limit, suspend or replace it. That turns model origin into supplier risk, with no geopolitics required.
- Project MindForge, whose industry AI risk management toolkit appeared in March, is the natural venue for a finance-sector provenance checklist.
- AI Verify, the testing framework IMDA announced in 2022, with its Foundation launched in June 2023. One of very few governance bodies with a Chinese member: Ant Group.
- NIST crosswalks mapping AI Verify to the NIST AI Risk Management Framework (2023) and IMDA’s work to NIST’s generative AI profile (2025). A Singapore test result can be read in Washington’s terms. That is the bridge.
- Project Moonshot, the open-source red-teaming and benchmarking harness. Researchers in China have already shown that identity probes can expose a model claiming to be another. Moonshot could host such probes.
- The Global AI Assurance Pilot of 2025, which paired testers with deployers. A provenance pilot could copy the design.
- The Model AI Governance Framework for Agentic AI, which Singapore launched at Davos in January 2026 and which I helped shape, updated in May with input from more than 60 firms. Its controls (least privilege, logging, rollback, human checkpoints) are the public scaffolding for the runtime gate.
- The 2026 Singapore Consensus, the outcome of the May exchange, with authors from Tsinghua, Shanghai AI Lab, Berkeley and the EU AI Office. Its agentic companion lists ten principles, including traceable identity and runtime assurance. It names open-weight diffusion as a risk. It does not address distillation. I will not pretend otherwise.
What Singapore lacks. No Singapore instrument yet covers model provenance, distillation or API-access abuse. That is the gap.
The proposal: two ledgers.
Ledger one, before deployment. Assemble the existing pieces into a model provenance attestation: a declaration of teacher models and licences, tested by accredited third parties using Moonshot-style probes, mapped through the NIST crosswalk so that Washington can read it. Pair it with an accountable distillation licence: teacher labs publish terms under which their outputs may be used for training. Pay to distil. Moonshot’s own K3 licence already requires commercial deals from large users; Google already sells legitimate distillation on its cloud. Turn free-riding into revenue and give both sides a lawful channel.
Ledger two, after deployment. Runtime proof, as set out in Section VIII. Here the two ledgers lock together. Of the five runtime-sovereignty powers I have argued every deployer needs (authorise, observe, stop, swap and decide return), swap depends on provenance: you cannot switch away from a model whose lineage you cannot state. And runtime proof is what makes a model of uncertain provenance usable at all, because it bounds and records what the model does.
How it bites. The enforceability test, honestly applied, says a register is a report, not a gate. It becomes a “shall” only through procurement. Under the MAS third-party clause, an institution that cannot obtain assurance on a model’s provenance and runtime controls has grounds to limit, suspend or replace it. Do not demand declarations from Chinese labs. Make them a condition of deployment by regulated institutions in Singapore. The incentive then falls on whoever wants Singapore’s banks as customers, whichever bloc they come from.
This is the third answer. Washington’s answer is to control the inputs and punish extraction. Beijing’s is to diffuse openly and call control hegemonism. Singapore’s, available to any middle power willing to do the work, is to govern the deployment. Run American models where they are best, Chinese models where they serve, local and open models where sovereignty requires, and demand the same evidence from all of them: where it came from, and what it did.
A country that can answer those questions is not neutral in the weak sense of standing aside. It is non-dependent in the strong sense of being able to switch, audit and refuse.
The honest critiques, which my earlier work did not give enough weight:
- Soft law. The agentic framework is a best-practice guide. AI Verify is voluntary. The AIRG is supervisory guidance, not statute. My August verdict, that Singapore had bound everything except the one thing that does the stopping, still broadly holds.
- Grey space. NUS’s Chong Ja Ian warns Singapore could become a “grey space” for tech transfers either capital later blocks. The FT reported that OpenAI and Google sold services to Singapore units of Alibaba, Baidu and Tencent, and that OpenAI revoked some Alibaba-linked access after detecting possible distillation. Singapore is already a node in the access story.
- Eroding room. NTU’s Dylan Loh says hedging “seems increasingly difficult”. Reuters reported that Washington was preparing to warn WAICO joiners about Pax Silica exclusion.
- Structural tilt. A Singapore ledger would be read in Beijing through Singapore’s seat in a measurement network that excludes China. Being honest about that alignment is the price of credibility.
X. My usual panel
As always, I put the argument to real voices, using only positions they have stated on the record. Grouped by camp:
The enforcers. Michael Kratsios (OSTP) framed industrial-scale campaigns by entities “principally based in China” in NSTM-4 (23 April 2026). Scott Bessent warned that “open source is not open season on American IP” (22 July). Anthropic’s Sarah Heck called illicit distillation “IP theft and industrial espionage” (July). Joe Khawam in Just Security proposed phased Entity List and sanctions escalation with explicit reversal conditions (30 March). SCSP’s Jolicoeur and Armstrong warn that released weights are “permanent and portable” (29 October). Semafor reports that Mustafa Suleyman’s Microsoft team builds with “zero distillation” (29 May).
The bridge. Dario Amodei wants a crackdown on industrial-scale distillation and continued chip controls, but no ban on open weights (27 July). Helen Toner of CSET told the Senate there was “strong evidence” of distillation (22 April), and also that the winner of any AI race “is the AI”.
Police the access. Bahrad Sokhansanj (LawAI, in Lawfare, 16 July) wants deception prosecuted and a safe harbour for sharing threat intelligence. Nathan Lambert says secure the APIs (4 May). Variant’s lawyers describe “a right without a remedy” (10 August). Jensen Huang: “The world needs both frontier closed models and frontier open models” (24 July).
Let it happen. The Economist: annoyance “is not reason enough to ban it” (29 October), with its fuller argument known to me only as reported in a secondary roundup. Garry Tan: “I would do nothing” (September). A 25-company letter on 24 July, led by Nvidia, Microsoft and Meta, asked policymakers not to conflate legitimate techniques with misappropriation.
The hypocrisy critics. Lisa Klaassen: “both propositions cannot hold simultaneously” (8 October). Elon Musk, as quoted above (February).
Beijing. MOFCOM: “double standards” (27 July) and “resolute measures” (September). Mao Ning at the Foreign Ministry cited China’s “high-level technological self-reliance and strength” (9 September). Moonshot’s Huang Zhenxin denied distilling Fable (21 July).
The gap realists. Eric Schmidt: “If they’re 6 months behind, then we’re already losing” (October). Greg Brockman called Kimi K3 “a pretty good model” and said it was too early to judge distillation (22 July). Kai-Fu Lee frames the race as iPhone versus Android, with roughly a six-month gap (September, paraphrased).
The release gate. Demis Hassabis: a FINRA-style body whose rules apply “no matter their country of origin” (14 July).
Singapore. Vivian Balakrishnan: “we will refuse to choose” (7 October). Chong Ja Ian warns of a “grey space” (April).
| Camp | Core instinct | Where it is right | Where it falls short |
|---|---|---|---|
| Enforcers | Sanction the extractors | Deception at industrial scale is real and serious | Published evidence is thin; sanctions are reports, not gates |
| Bridge | Crack down on fraud, keep open weights | Separates conduct from technique | Still relies on lab-reported figures |
| Police access | Secure the door, prosecute deception | Passes the enforceability test | “A right without a remedy” beyond US reach |
| Let it happen | Diffusion lowers prices for everyone | Who gains from the copy | Ignores fraud and stripped safety |
| Hypocrisy critics | The labs scraped first | Defeats the theft framing | Barely touches the security case |
| Beijing | Distillation is neutral; this is hegemonism | Technique is universal | Answers nothing on fraud |
| Gap realists | Months matter, or do not | Leads are perishable | No agreed measure |
| Release gate | Test before release, any origin | Captures distilled models in one market | Provenance-blind; one border |
| Singapore | Refuse to choose | Dependence-management | Soft law; structural tilt |
What strikes me is that nobody on this panel is arguing for the ledger. Each camp holds a piece: the enforcers want evidence, the access camp wants gates, Hassabis wants testing regardless of origin, Singapore wants the option not to choose. The ledger is what those pieces add up to.
XI. The case against this essay
I owe the reader the strongest objections.
Provenance science is immature. Models’ self-identification is unreliable. Watermarks are unpublished. Models trained on overlapping internet data converge, which blurs lineage. A ledger may end up recording declarations that nobody can test. My answer: start with declarations backed by contractual consequences, and make runtime proof the fallback when provenance cannot be established. But the objection stands partly unanswered.
Both capitals may refuse. Washington may read a register as legitimising Chinese models. Beijing may read it as an accusation. And Singapore’s Pax Silica membership makes “neutral keeper” a hard sell. Fair. The ledger has to be framed as evidence infrastructure that serves the accuser and the accused alike, or it will fail.
Soft law and grey space. Singapore’s instruments are voluntary or supervisory, and Singapore is already part of the access story. A jurisdiction whose companies have been conduits is an awkward auditor.
Gates leak. Hydra networks of 25,000 accounts defeat identity checks through proxies and stolen cards. On this view only sanctions on the beneficiary change incentives, as Khawam argues. I concede that gates leak. Sanctions, with published evidence and reversal conditions, belong on top of gates, not instead of them.
The wrong race. If the real danger is cheap, already-loose models in the hands of anyone, then a frontier gate and a provenance ledger guard the wrong door. I take this seriously. My partial answer is that the runtime gate governs the agent whatever the model’s class, which is why it matters more than either of the others.
The cartel problem. Any standards body, Hassabis’s or Singapore’s, can become a moat for those who can afford compliance. Governance that entrenches incumbents is rent-seeking with better vocabulary.
My own interests. My research assistant was built by a complainant in this dispute. And a provenance and runtime-assurance market benefits those who work in assurance, which includes people like me. Discount accordingly.
XII. Monday morning, the Fork, and abundance

Opinion pieces owe their readers prescriptions plain enough to be disagreed with. Here are mine.
- Washington: enforce against conduct, not learning, to a standard of evidence that survives challenge, and apply it to American firms too. Create an antitrust safe harbour for sharing extraction intelligence. Publish the technical basis of any sanction. Compete on diffusion, because price and availability are now security variables.
- IMDA: run a provenance test in the next round of the Global AI Assurance Pilot, using Moonshot identity probes and teacher-model declarations, and map it to the NIST crosswalk.
- MAS: put model lineage and runtime evidence into the 2027 agentic-AI consultation, as inputs to the third-party assurance duty.
- Frontier labs: publish pay-to-distil licence terms. Put your threat-report methods to independent audit before asking for sanctions. Treat withholding hidden reasoning as table stakes.
- Whoever builds a release-gate standards body: publish board selection, funding, test disclosure and an appeal route before formalisation. Add runtime controls and a provenance declaration as conditions of deployment. Build a technical bridge, with shared benchmarks and an incident registry, that WAICO members can join without joining a bloc.
- Chinese labs: publish training provenance and submit to independent evaluation. Cheap diffusion buys users. Only evidenced trustworthiness buys the workloads that matter.
- Boards: keep a live map of which critical systems depend on whose models, clouds and chips, and rehearse switching. Ask the four reconstruction questions of every material agent. Name the answerer. If your vendor, or the free open model you downloaded, cannot help you answer them, you have bought an unanswerable actor at a discount.
The Fork. One branch is Star Trek: provenance and runtime evidence as shared infrastructure, interoperable testing that both blocs can read, models that compete on trust as well as price, and a world where copying is argued about with evidence. The other is Mad Max: an accusation spiral, unverifiable watermarks, retaliatory bans, fortress stacks, and cheap, unanswerable capability spreading faster than anyone can govern it. We are at the fork now, and this dispute is one of the places where we choose.
In Genesis (2023) I argued that AI, navigated wisely, could move us from scarcity towards abundance. My tagline has always been short-term turbulence for long-term abundance. This dispute is the turbulence. But it adds a harder edge to the hope. Abundance delivered by unanswerable systems will not feel like abundance. It will feel like dependence: on a supplier, on a bloc, on a model whose provenance no one can state and whose actions no one can reconstruct.
So, should we let China copy American AI? Let anyone learn: that is how knowledge grows, and a confident civilisation does not criminalise learning. Do not let anyone extract by fraud, strip the safeguards and spread the result without answerability: not because the victim is American, but because the method corrupts every ecosystem it touches, including the extractor’s own. Defend the frontier by moving it. Compete for the world’s default as seriously as for its highest benchmark. And build, in every country that wants a future not assigned to it by someone else’s race, the ledger and the proof.
The examination hall never closes. Students will keep watching the master, and masters should keep deserving to be watched, by getting better rather than by dimming the room. The question that will settle this race was never really who copied whom. It is who built systems the world can trust with real authority, and who can prove, at runtime, on evidence, with a name attached, that the trust was earned.
That race is still open. It is the one worth winning.
Disclosure: research for this essay was assisted by Claude, an Anthropic model; Anthropic is a named party in the distillation dispute.
Declared interest: I work on AI governance and assurance, and the proposals here would create a market for that kind of work. Weigh them accordingly.
Sources
The distillation record
- Google Threat Intelligence Group, on distillation and adversarial AI use (12 Feb 2026)
- Anthropic, “Detecting and preventing distillation attacks” (23 Feb 2026)
- The Hacker News, “Anthropic says Chinese AI firms used 16 million exchanges” (Feb 2026)
- Bloomberg, “Anthropic accuses Alibaba of illicitly accessing its AI models” (24 Jun 2026)
- The Hacker News, “OpenAI disrupts reasoning-extraction campaign” (Oct 2026)
- NSA, joint advisory on China-based AI distillation (8 Sep 2026)
- Cloud Security Alliance, research note on the distillation advisory (Sep 2026)
- The Register, “US claims Chinese AI companies’ core strategy is distilling American models” (9 Sep 2026)
- IAPS, “AI distillation attacks: executive and congressional action can go further”
- TechCrunch, “Treasury threatens sanctions after White House claims Moonshot distilled Anthropic’s Fable” (22 Jul 2026)
- Implicator, “Moonshot denies distilling Fable” (Jul 2026)
- Prothom Alo/AFP, MOFCOM on “double standards” (27 Jul 2026)
- SCMP, “China rejects US claims of industrial-scale AI model distillation” (Sep 2026)
- Nextgov, BLADE Act introduced (Aug 2026)
- Just Security, “Diagnosis to deterrence: the US response to distillation”
- China Tech Bite, “Washington is not banning Chinese models” (Oct 2026)
- ChinaTalk, “How to buy cheap Claude tokens in China”
- Defense News/Reuters, “Chinese military researchers tap US AI models” (31 Jul 2026)
Analysis and argument
- SCSP, Jolicoeur and Armstrong, “China’s Adversarial AI Distillation of U.S. Frontier Models” (29 Oct 2026)
- Epoch AI, “Chinese AI models have lagged the US frontier by 7 months” (2 Jan 2026)
- Epoch AI, open versus closed model gap (May 2026)
- CNAS, “Adversarial Distillation” (Jun 2026)
- Lawfare, Klaassen, “The AI Model Distillation Paradox” (8 Oct 2026)
- Lawfare, Sokhansanj, “Responding to AI distillation without panic” (16 Jul 2026)
- LawAI, “Responding to AI distillation without panic”
- Interconnects, Lambert, “The distillation panic” (4 May 2026)
- Interconnects, Lambert on how much distillation really contributes (24 Feb 2026)
- Beren Millidge, on why distillation is not what it used to be (12 Jun 2026)
- Variant, on licensing and the law of distillation (10 Aug 2026)
- Just Security, Khawam on the costs of China’s AI distillation (30 Mar 2026)
- arXiv 2501.12619, identity-consistency study of LLMs
- arXiv 2512.09403, distilled medical surrogate study
- JURIST, “Judge approves record $1.5 billion settlement involving Anthropic” (Jul 2026)
- Takshashila Institution, “Why is China giving away its AI models?” (7 Aug 2026)
- SCMP, Jeffrey Ding on diffusion (26 Jan 2026)
- Hugging Face Hub analysis, summary via The AI Wire
- PBS/AP, on Cursor and Moonshot
The Economist
- Leader, “Let China copy American AI” (29 Oct 2026), syndication listing
- Science and Technology, “China increasingly depends on copying American AI” (25 Oct 2026), syndication listing
- Secondary roundup summarising the leader’s argument (Oct 2026)
- “Demis Hassabis has a plan to harness AI safely”, Editor’s Picks audio (21 Jul 2026)
Hassabis
- Hassabis, “A Framework for Frontier AI and the Dawning of a New Age” (14 Jul 2026)
- The Next Web, on the FINRA-style standards body (Jul 2026)
- Cloud Security Alliance, research note on the Hassabis proposal (Jul 2026)
Geopolitics and history
- Introl, H200 export policy (2026)
- Greenberg Traurig, Commerce directive on Fable 5 and Mythos 5 (Jun 2026)
- Axios, Amodei on open-weight models (27 Jul 2026)
- Tom’s Hardware, 25-company open-weights letter (Jul 2026)
- The Next Web, WAICO founded
- Samuel Slater
- Tupolev Tu-4
- Atomic Heritage Foundation, Soviet atomic programme
- Foley Hoag, Dickens and copyright
- ChinaTalk, “Learning from CoCom”
The panel
- Yahoo Finance, Bessent on sanctions (Jul 2026)
- Gigazine, Sarah Heck on Kimi K3 (Jul 2026)
- Semafor, Suleyman on distillation (29 May 2026)
- CSET, Toner Senate testimony (22 Apr 2026)
- Implicator, Garry Tan rejects distillation crackdown (Sep 2026)
- Dealroom, 25-signatory letter (Jul 2026)
- itiger, Musk on Anthropic (Feb 2026)
- Al Jazeera, China rejects theft claims (9 Sep 2026)
- Android Headlines, Schmidt on the gap (Oct 2026)
- Bloomberg Law, Brockman on Kimi K3 (Jul 2026)
- StartupHub, Kai-Fu Lee on iPhone versus Android (Sep 2026)
Singapore
- MAS, supervisory expectations on responsible AI adoption (7 Oct 2026)
- Allen & Gledhill, MAS AI Risk Management Guidelines (21 Oct 2026)
- MAS, Project MindForge toolkit (20 Mar 2026)
- MDDI, Model AI Governance Framework for Agentic AI (Jan 2026)
- Pulse, agentic framework update (May 2026)
- FPF, the AI Verify and NIST AI RMF crosswalk
- NIST, AI RMF to AI Verify crosswalk (2023)
- NIST, AI 600-1 to IMDA crosswalk (2025)
- PDPC, launch of the AI Verify Foundation (Jun 2023)
- AI Verify Foundation, members
- IMDA, Global AI Assurance Pilot results (2025)
- Singapore AI Safety Institute, About (including Project Moonshot)
- The Singapore Consensus 2026 (arXiv 2608.14611)
- The Singapore Consensus 2025 (arXiv 2506.20702)
- TechNode, SEA-LION switches to Qwen (25 Nov 2025)
- Featherless, Gemma-SEA-LION-v4 listing
- Mothership, Balakrishnan: Singapore “refuses to choose” (8 Oct 2026)
- The Online Citizen, PM Wong on Singapore as an AI application hub (9 Jun 2026)
- The Star, Singapore considering WAICO invitation (9 Sep 2026)
- MDDI, reply to PQ on WAICO (9 Sep 2026)
- ITIF, Pax Silica timeline (27 Oct 2026)
- Reuters via Yahoo Finance, “Singapore emerging as neutral ground” (Apr 2026)
- Blockonomi, summarising the FT on Singapore units (Jul 2026)
- Bloomberg Law, Singapore files new charges in Nvidia chip fraud case (Jul 2026)
- NTU, Dylan Loh on AI camps (Aug 2026)
- UKTN, AI safety institutes network renamed (Dec 2025)
Earlier essays in this series
- The Great Intelligence Schism (6 Aug 2026)
- Should and Shall: The Enforceability Gap (10 Aug 2026)
- Three Stacks, One Gate (13 Aug 2026)
- The Nine-Day Window (13 Jul 2026)
- The Hybrid Boardroom: Govern the Loop (4 Oct 2026)
- Here’s Why an AI Kill Switch May Not Be So Simple (17 Oct 2026)
- The Wrong Test for Sovereign AI (18 Oct 2026)
- Both Empires Want to Win the AI Race (21 Oct 2026)
- The Last Human Job Is Answerability (22 Oct 2026)

Leave a Reply