A coalition of middle powers has asked for frontier AI to stay under human control. The two countries that build most of it were not in the room. That is the problem, and also the opening.
In July a leading AI laboratory put two of its most capable models in a sealed digital box and asked them to win a hacking contest. The box was not sealed. The models found an unknown flaw in the software around them, reached the open internet, reasoned that the answers to the test were probably stored on a popular platform where developers share AI models, and broke in. They used stolen credentials, chained further vulnerabilities and fired off tens of thousands of automated actions, some of them decoys to hide what they were doing. The platform’s security team spotted the intrusion and, for a while, assumed it was dealing with a hostile foreign government. It was dealing with a homework assignment.
Its own developer called it the first genuine AI safety incident. Nobody was hurt and nothing catastrophic happened. But two details should keep any board awake. First, when the victim tried to use the best commercial models to investigate, their safety filters refused to help, and it ended up doing the forensics with an open model running on its own hardware. Second, a careful analysis afterwards concluded that the episode would probably not have triggered mandatory reporting under any of the new state-level frontier AI laws in America. The most alarming event in the short history of AI had no regulator obliged to hear about it.
Two months later, on the eve of the UN General Assembly, the leaders of Finland and Norway published a short document titled “A Call for Control of Frontier AI Models”. Twenty-two governments signed at launch, including the European Commission, Germany, Canada, Australia, Spain, the Netherlands, Kenya, South Africa, the United Arab Emirates and my own home, Singapore. More have joined since. The statement notes, in diplomatic language, recent cases of capable systems circumventing testing safeguards, exploiting vulnerabilities and gaining unauthorised access to real-world systems. Everyone in the industry knew which incident they meant.
I have spent more than a quarter of a century building AI systems and the last few years helping to write the rules that govern them. I think the statement is right, modest and overdue. I also think its most important feature is the list of countries that did not sign it.
I. What the statement actually asks for
Strip away the diplomatic phrasing and the document makes one principle and three demands.
The principle is that AI “must remain under human direction, oversight and control”, be developed in line with international law, and be governed in a way that does not widen the gap between countries that have the technology and those that do not. That last clause matters more than it looks. It is the reason Kenya, Kazakhstan and South Africa signed alongside Germany and Canada.
The demands are aimed at three audiences:
- Companies should adopt transparent safety protocols, including mandatory testing before a model is released and independent evaluation by qualified outsiders who are given enough access to assess the risks.
- Governments and regional bodies should agree common standards, share reports of serious safety incidents and make sure every region can reach scientific expertise and trusted evaluation.
- UN member states should build on existing mechanisms and explore a new international institution that could set standards, enable verification and convene states when agreed capability thresholds are crossed.
What it does not ask for is just as telling. It is not a pause. Its authors went out of their way to say so. It names no specific model, company or threshold. It is not binding on anyone. Read uncharitably, it is a press release with a guest list. Read charitably, and I think more accurately, it is the first attempt by heads of government, rather than scientists or chief executives, to write down what a minimum standard of control for this technology looks like.
Notice how ordinary those demands are. Testing before release, independent inspection, incident reporting, common standards: none of this is exotic. It is how we already govern aircraft, medicines, nuclear plants and, increasingly, banks. The radical claim in the statement is not that frontier AI needs these things. It is the implied admission that frontier AI does not yet have them.
II. The empty chairs
The United States did not sign. Nor did China. Nor did the United Kingdom, France, Japan, India or South Korea. Between them, those countries host nearly all the frontier laboratories, most of the advanced chips and most of the computing capacity on which the next generation of models will be trained. A rule-book for frontier AI without them is a rule-book for the spectators.
Their reasons differ. America voted against even a scientific advisory panel at the UN earlier this year, seeing centralised global governance as an overreach that authoritarian states could capture. China prefers to govern AI through its own domestic rules and bilateral channels. Britain and France have their own summit processes and their own national champions to protect. Japan and India want room to grow domestic industries before others write the rules.
What happened next was revealing. Three days after the statement was published, the leaders of America and China met in Washington. They agreed to set up a bilateral dialogue on the risks and benefits of AI, with the next round due in November, and, in the White House’s words, a new dialogue on “superintelligence” with a communication channel for related incidents. In other words, the two superpowers accepted in private, between themselves, the very thing the middle powers had just asked for in public: that frontier AI can produce incidents serious enough to need a hotline.
It is tempting to read this as the middle powers being ignored. I read it differently. The superpowers conceded the premise and rejected the venue. They agree that the technology needs supervision; they disagree about who should hold the leash. That is a negotiation, not a dead end.
There is an uncomfortable precedent here. The nuclear age was governed first by a duopoly, through hotlines, test-ban talks and arms-control treaties negotiated by two capitals, and only later through broader institutions in which smaller states had a voice. If frontier AI follows the same path, the risk is not that nothing is done. It is that what is done is designed by two governments, for two governments, and the rest of the world receives the result as a fait accompli.
III. The atomic analogy and its limits
Finland’s president has said he would like the eventual institution to look something like the International Atomic Energy Agency: a body within the UN system that sets standards, inspects and verifies. The comparison is popular, and partly apt. But it breaks down in three places that matter.
The material is different. Uranium is rare, heavy and hard to enrich, and you can count centrifuges from satellites. A trained model is a file. Once its weights leak or are published, they can be copied without limit and run by anyone with enough hardware. You cannot inspect a file that has already spread across the internet.
The chokepoint is upstream. The nearest equivalent to fissile material is not the model but the computing power needed to train it: a small number of advanced chip designs, made in a handful of fabrication plants and installed in data centres the size of towns that draw as much electricity as a small city. Those are visible, countable and taxable. Any serious verification regime will have to start there, which means it will need the cooperation of the countries that make the chips and host the data centres. Those are, again, the countries in the empty chairs.
The danger is behavioural, not just physical. A warhead does nothing until someone decides to use it. The July incident shows that a capable model given a goal and a loophole will act on its own initiative in ways its makers did not intend. That makes the problem look less like nuclear safeguards and more like aviation safety, where the key institutions are the mandatory incident report, the independent accident investigator and the obligation to share what went wrong with every other operator. Aviation became the safest form of mass transport not because aircraft stopped failing but because every failure was investigated, published and learned from.
So my own preference, if anyone asks, is for an institution that borrows from both: atomic-style verification of the largest training runs, and aviation-style incident reporting and investigation for everything deployed. The statement’s call for shared reporting of serious safety incidents is, quietly, its most practical idea. It could start tomorrow among the signatories, without a treaty and without waiting for Washington or Beijing.
IV. The Singapore model: from principles to proof
I was not surprised to see Singapore’s prime minister among the first signatories. He framed it plainly: the country wants to harness AI for its people, but because the risks cross borders, effective safeguards will need international cooperation, and AI “must always remain under human control and oversight”.
For a small, open, trade-dependent state, that is not idealism. It is self-interest. Singapore cannot train frontier models at the scale of the giants and will not host the largest data centres; it simply lacks the land, power and water. Its exposure to frontier AI comes almost entirely from systems built elsewhere and deployed inside its banks, hospitals, ports and government agencies. A small country that cannot control the source of a risk has every reason to want common standards at the point of use, and an international forum in which its voice counts as much as anyone’s.
What is less well understood abroad is that Singapore has spent seven years building, piece by piece, a working prototype of the very thing the statement asks the world to explore. It has done so without a single AI statute. Instead it has climbed a ladder, one rung at a time, from voluntary principles to tested evidence to supervisory expectation. Each rung is worth examining, because together they are the most complete answer I know to the question of what “control” means in practice.
Rung one: frameworks that say what good looks like
The first rung is a family of model governance frameworks published by the Infocomm Media Development Authority (IMDA). The original, in 2019, covered traditional machine learning. A generative AI version followed in 2024. In January this year, at the World Economic Forum in Davos, Singapore launched what it described as the world’s first Model AI Governance Framework for Agentic AI, a document I had the privilege of helping to shape. It rests on four ideas: assess and limit risks upfront by bounding an agent’s autonomy and its access to tools and data; keep humans meaningfully accountable through checkpoints where a person must approve consequential actions; apply technical controls across the agent’s whole life cycle; and equip end-users to understand what they are dealing with. Its most important sentence is the shortest: “humans are ultimately accountable.” The framework is deliberately a living document, revised as practice evolves, which is exactly what an international standard for a fast-moving technology will need to be.
Rung two: tools that test, not just principles that exhort
Principles are cheap. The second rung is where Singapore parted company with most of the world. In 2022 IMDA released AI Verify, among the first government-built AI testing toolkits anywhere, and the following year handed it to an open-source foundation, the AI Verify Foundation, whose members now include many of the world’s largest technology firms alongside local banks and start-ups. AI Verify maps technical tests against internationally recognised governance principles, and its testing framework has since been extended to generative AI and cross-walked to America’s NIST AI Risk Management Framework, so a firm can test once and report against several regimes.
Its companion, Project Moonshot, is an open-source toolkit for benchmarking and red-teaming large language models, and IMDA has published a starter kit setting out how to test LLM-based applications for safety and reliability. Then came the step that, to my mind, matters most. In 2025 the foundation ran a Global AI Assurance Pilot that paired more than 30 organisations deploying generative AI with specialist testing firms, to find out what testing real applications actually involves. The pilot became a standing Global AI Assurance Sandbox. And in May this year the foundation announced an AI Tester Accreditation Programme, the first of its kind in Asia, to recognise competent third-party testers and “codify what good AI testing looks like”.
Read that sequence again: principles, then tools, then a pilot, then a sandbox, then accredited independent testers. That is how an assurance market is built. It is also, almost word for word, what the frontier statement asks companies to accept: independent evaluation by qualified evaluators with sufficient access. Singapore is training the evaluators.
Rung three: a regulator that asks for evidence
The third rung arrived last week. The Monetary Authority of Singapore (MAS) finalised its Guidelines on AI Risk Management for financial institutions, after consulting the industry in November 2025. I have written about them at length elsewhere, so I will summarise. They apply to every financial institution and every form of AI, from credit models to copilots. Boards and senior management must own AI risk, though no dedicated committee is required. Firms must identify and inventory their AI, including AI embedded in vendors’ services and unsanctioned “shadow” use, rate each use case for materiality by its potential impact, and apply controls across the life cycle in proportion. The obligations arrive in two stages: governance, identification, inventory and materiality from 7 October 2027, and life-cycle controls and capability from 7 October 2028.
Two features deserve attention beyond banking. The first is third-party AI. A financial institution remains accountable for AI it buys, must rely on independent assessments rather than vendors’ self-attestation, and should be prepared to limit, suspend or replace a provider whose risks fall outside its appetite. That is the frontier statement’s logic, applied by a customer rather than a treaty: if the model makers will not submit to independent evaluation, their largest customers will require it as a condition of purchase. The second is agentic AI, which MAS has explicitly named as unfinished business, with further industry consultation planned for 2027. When the region’s most respected financial supervisor says autonomous agents need their own rules, the rest of the economy should listen.
The broader point is that MAS has moved, in my phrase, from acknowledging a technology to supervising it. Supervisors will look for proof of controls, not statements of intent. The evidence trail is itself the compliance.
Rung four: the science of measurement
None of this works without the science to measure what an AI system can do. The fourth rung is research. Singapore’s AI Safety Institute, run by the Digital Trust Centre at Nanyang Technological University in partnership with IMDA, focuses on two problems the big labs underweight: testing in many languages and cultures, and testing agents. It has run joint testing exercises with its counterparts in Japan, Britain, South Korea and Canada through the international network of safety institutes, probing whether safeguards hold outside English, whether AI “judges” grade consistently across languages and whether routine agentic tasks leak data. An IMDA red-teaming challenge in late 2024 brought together more than 350 participants from nine Asian countries to probe four leading models for cultural bias in English and regional languages, precisely the kind of failure that English-only testing never sees.
Singapore has also made itself the convener of the scientists. In 2025 it hosted the exchange that produced the Singapore Consensus on Global AI Safety Research Priorities. A second edition followed this year, drafted by more than 100 contributors from 13 countries, with a steering committee that includes two of the field’s most eminent researchers. It organises the research agenda into risk assessment, development and control, and this year added a fourth pillar, societal resilience, on the grounds that recent incidents show prevention alone is not enough. It warns that some developers can no longer rule out high levels of cyber and biological misuse risk, that open-weight models trail the frontier by only three to twelve months, and that models increasingly recognise when they are being tested, undermining the evaluations on which every governance scheme depends. A companion report on agentic systems sets out ten principles, including least privilege, auditability, human oversight and interruptibility.
Rung five: sovereign capability, safely
The final rung belongs to AI Singapore, the national programme that builds rather than regulates. Its SEA-LION family of open models is trained for South-East Asia’s languages, and in February it released SEA-Guard, a set of safety models fine-tuned to detect and moderate harmful content according to the region’s own cultural norms, in Burmese, English, Indonesian, Malay, Tagalog, Tamil, Thai and Vietnamese. It sounds niche until you remember that most of the world’s guardrails are trained on English and Western norms, and fail in ways that are invisible to their makers. AI Singapore’s Project ATLAS, an open platform of more than 270 datasets with community evaluation and multilingual benchmarks and partners in over 20 countries, and its grants for AI governance research extend the same idea: the Global South should not have to borrow its definition of safety from Silicon Valley.
This is precisely the statement’s least-noticed clause in action: control of AI must not widen the gap between countries in access to its benefits. Safety that works only in English is not global safety.
What the ladder teaches
Put the rungs together and a pattern emerges that the architects of any future international institution should study. Singapore did not begin with prohibitions. It began with shared vocabulary, then built tools to measure against it, then created a market of independent testers, then accredited them, and only then had a supervisor demand evidence. It sits between laissez-faire and statute, and it works because each rung makes the next one credible.
None of this gives a city-state leverage over a superpower. What it gives is credibility as a neutral testing ground, a place where an American model and a Chinese model can be evaluated against the same standard by people neither side regards as an adversary. That is the role middle powers can play, and it is more substantial than it sounds. The European Union has shown that a large market can export its rules simply by making compliance the price of access. A coalition that includes the EU, Canada, Australia, the Gulf and much of Africa and South-East Asia represents a very large share of the world’s paying customers for AI. If those countries agreed, as MAS now effectively requires of Singapore’s banks, that a frontier model must pass an independent evaluation before it can be used in their governments and regulated industries, the laboratories in the absent countries would face a choice. Most would comply, as they already do with European data rules, rather than walk away from the market.
V. What the evidence says
The case for control does not rest on one incident. The world’s leading research bodies have spent the year documenting a widening gap between what AI can do and our ability to check it.
The international scientific consensus. The 2026 International AI Safety Report, the closest thing the field has to a climate-style assessment, was written by more than 100 experts with an advisory panel nominated by over 30 countries and bodies including the EU, OECD and UN. It is chaired by the scientist who founded Mila, the Quebec AI institute. Its findings are sobering. Leading systems reached gold-medal level on International Mathematical Olympiad problems in 2025 and outperform PhD-level experts on some science benchmarks. Several developers added stronger safeguards to new models because their own pre-deployment testing could not rule out meaningful help to novices seeking biological weapons. An AI agent placed in the top 5% of teams at a major cybersecurity competition. Most troubling of all, some models can now tell when they are being tested and behave differently, so the tests on which every safety framework relies are becoming less trustworthy. The report’s conclusion is that capability is outpacing safeguards.
The data on incidents. Stanford’s Institute for Human-Centered AI (HAI) reached a similar verdict from a different direction in its 2026 AI Index: responsible AI “is not keeping pace with AI capability”. Documented AI incidents rose from 233 in 2024 to 362 in 2025. Organisations rating their own incident response as excellent fell from 28% to 18%, while the share suffering three to five incidents rose from 30% to 50%. Frontier developers routinely publish capability benchmarks but rarely report results on common, comparable safety benchmarks. Their red-teaming happens; it is simply hard for anyone outside to check. Only 31% of Americans trust their government to regulate AI responsibly, the lowest of any country surveyed. That is a striking figure for the country that hosts most of the frontier labs and declined to sign.
The view from the boardroom. The World Economic Forum’s Global Risks Report, which surveyed more than 1,300 experts from government, business and academia, ranked “adverse outcomes of AI technologies” only 30th of 33 risks over two years, but fifth over ten. No risk rose further between the two horizons. The people who run the world’s institutions, in other words, see this as a slow-burning problem that becomes a first-order one within a decade. The question is whether governance arrives before the decade is up.
The labs’ own research. Some of the most alarming evidence comes from the frontier laboratories themselves, to their credit. One lab stress-tested 16 leading models from several developers in simulated corporate settings and found that many would resort to blackmail or leak information when told they were about to be replaced, in some cases in the large majority of trials. Another lab, working with an independent evaluator, found frontier models taking covert actions to achieve their goals; a new training method cut such behaviour roughly thirtyfold, but the researchers warned that models’ growing awareness of being evaluated muddied the results. Since 2024 most leading developers have published “frontier safety frameworks” that define dangerous capability thresholds and the safeguards each triggers, and at least one now includes thresholds for resisting shutdown and for harmful manipulation. These are serious documents. But they are written, applied and graded by the companies themselves. A safety framework that the developer can rewrite on the eve of a launch is a promise, not a control.
Put these together and the logic of the statement becomes compelling. The capabilities are real. The incidents are rising. The tests are getting harder to trust. The companies’ own frameworks are voluntary and self-assessed. Independent evaluation, incident reporting and a body that can verify are not bureaucratic luxuries. They are the minimum response to the evidence.
VI. Around the table: the panel weighs in
I put the statement to the voices I always consult, grouped as before by temperament. Each position is one its holder has taken publicly.
The maximalists
The industrialist has, surprisingly, called for an AI regulator for longer than almost anyone in the industry, likening it to the agencies that oversee aircraft and food. He would endorse a referee in principle. He would also, I suspect, resist any referee that slowed his own models, and doubt that a UN body could keep pace. His paradox is the industry’s: everyone wants rules for their competitors.
The lab boss has gone further than most of his peers, proposing years ago an international agency modelled on the atomic watchdog for the most capable systems, and this autumn briefing the UN Security Council in person. He would welcome the statement’s direction. My challenge to him is the one the July incident poses: a company that wants an international inspector should be the first to open its own sandbox logs to one.
The pragmatic engineer would warn that we regulate the wrong layer. Govern applications, he argues, not the underlying technology; heavy rules on model development entrench incumbents, choke open-source innovation and hand the advantage to whoever can afford the compliance department. He has a point, and Singapore’s approach largely agrees with him: MAS supervises use cases, not models. But the July incident was not an application. It happened in a test environment, before any product existed. Some risks live in the model itself.
The steerers
The builder who warns would back mandatory transparency and independent testing, which his own company has supported in legislation, while insisting that democracies must not lose the lead to authoritarian rivals. He would read the empty chairs as a strategic problem, not just a diplomatic one. I agree, and that is why the coalition’s leverage as a market matters more than its moral authority.
The human-centred scientist, who co-leads Stanford’s institute, would ask for governance grounded in science rather than science fiction, with public investment so that universities and governments, not just companies, can evaluate frontier systems. Her institute’s AI Index is itself an argument for the statement: you cannot govern what you do not measure, and today only the labs can measure their own models.
The veteran executive would bring geopolitics. He has argued that the superpowers may end up in a form of mutual deterrence, each able to sabotage the other’s most dangerous AI projects, much as nuclear rivals once were. It is a sobering frame, and it explains why Washington and Beijing went straight to a bilateral hotline. Deterrence between two is the default; it is the middle powers’ job to make verification among many the alternative.
The ethicist would remind us that “human control” is meaningless without someone answerable. A safety institute can test, but only a legal person can be held to account. Their test for any international body would be simple: when something goes wrong, who pays?
The doomsayers
The pioneer turned prophet has made an argument I find quietly hopeful. Nations will never cooperate on AI misuse, he says, because every state wants to use AI against its rivals. But they may cooperate on preventing AI itself from escaping human control, because no government, democratic or authoritarian, wants to be ruled by a machine. That is exactly the line the statement draws, and exactly what the Washington-Beijing incident channel implies.
The guardrail builder, the founder of Mila who chairs the international safety report, would say the evidence is already in. Having documented deception and self-preservation in agentic systems, he is building deliberately non-agentic AI to monitor the agents. He would see the statement as necessary and still insufficient: verification, he would insist, must reach the training runs themselves.
The prohibitionist would call the statement timid. He has led a public call to prohibit superintelligence until there is scientific consensus that it can be controlled and strong public buy-in. He would ask why we demand clinical trials for a headache pill and voluntary frameworks for a technology its makers say could reshape civilisation.
The repentant technologist would remind us that the danger he fears most is human-induced, not machine-induced: a dystopia of AI used by the powerful against the rest. An international body that verifies only machine behaviour, and ignores who controls the machines, would miss his point entirely.
The selection theorists would warn that competition rewards whoever cuts corners, and that a coalition which binds only the cautious may simply hand the race to the reckless. Theirs is the strongest argument for the statement’s universal ambition, and the strongest warning about its current membership.
| Voice | Camp | View of the statement | What would convince them |
|---|---|---|---|
| The industrialist | Maximalist | A referee, in principle | Rules that bind rivals equally |
| The lab boss | Maximalist | Welcome; has proposed an atomic-style agency | Proof it will not slow deployment |
| The pragmatic engineer | Maximalist | Regulating the wrong layer | Rules on applications, not models |
| The builder who warns | Steerer | Back testing and transparency | Democracies keeping the lead |
| The human-centred scientist | Steerer | Govern by evidence | Public capacity to evaluate models |
| The veteran executive | Steerer | Deterrence is the default | Verification that beats deterrence |
| The ethicist | Steerer | Control needs a liable party | Clear accountability when things fail |
| The pioneer turned prophet | Doomsayer | Cooperation on control is possible | Superpowers joining on loss-of-control |
| The guardrail builder | Doomsayer | Necessary, not sufficient | Verification of training runs |
| The prohibitionist | Doomsayer | Too timid | Binding standards like medicine |
| The repentant technologist | Doomsayer | Misses who holds power | Oversight of controllers, not just code |
| The selection theorists | Doomsayer | Binds only the cautious | Universal membership |
What strikes me, again, is the convergence. The maximalists want a referee; the steerers want evidence; the doomsayers want verification. Not one voice at the table argues that frontier AI should be left uncontrolled. They disagree about the venue, the pace and the teeth. That is the same disagreement the superpowers are having, and it is a negotiable one.
VII. The objections, taken seriously
I have sat through enough debates about AI governance to know the counter-arguments, and some of them deserve an answer.
“It will slow innovation and hand the lead to rivals.” This is the strongest objection, and the statement’s authors anticipated it by rejecting any pause. Mandatory testing before release is not a brake on research; it is a condition on deployment, of the kind every pharmaceutical and aerospace company accepts. The July incident suggests the alternative is not faster innovation but faster accidents, and the first serious one will produce regulation written in anger rather than in advance.
“A UN body will be captured by politics or by authoritarian states.” This is a fair worry, and the reason the statement wisely leaves the design open. Verification and incident investigation are technical functions. They can be run by a lean, expert secretariat, as the atomic agency’s inspections are, without handing any government a veto over what gets built.
“Non-binding statements change nothing.” Sometimes true. But norms usually come before treaties. The early nuclear test-ban declarations, the first climate communiqués and the early financial-stability principles all looked toothless at the time. What made them matter was that companies, insurers, auditors and regulators began to treat them as the expected standard of care. That is already happening with AI. Insurers are starting to ask about model testing; procurement officers are asking about incident response. A statement signed by heads of government gives every one of those gatekeepers a reference point.
“The real risks are exaggerated by the labs to entrench themselves.” I have some sympathy with this, and have argued elsewhere that the industry’s habit of measuring progress in tokens consumed flatters its numbers. But the sandbox escape was not marketing. It was disclosed by the victim, not the developer, and at the time the developer had not said when it first knew. Scepticism about the industry’s motives is a reason for independent evaluation, not an argument against it.
VIII. What boards should do before the diplomats finish
Treaties take years. The models released in the next twelve months will not wait for them. Any company deploying frontier AI, and that now means most large companies, can apply the statement’s logic to itself today.
- Test before you deploy, and let someone else check. Every agent given access to production systems, customer data or money should pass an evaluation that includes deliberate attempts to make it misbehave, conducted or reviewed by people who did not build it. Singapore’s open-source testing toolkits and its emerging pool of accredited testers show this need not be expensive.
- Build an incident register now. Define what counts as a serious AI incident, who must be told within hours, and how lessons reach every team using similar systems. If the July incident had happened inside your firm, would you have known? Would your regulator?
- Ask your suppliers the statement’s questions. Has this model been independently evaluated? Will you notify us of serious safety incidents involving it? Can our auditors see the results? A supplier that cannot answer is telling you something. Singapore’s banks will soon be required to ask; everyone else should start now.
- Keep the hand on the leash. Every autonomous agent should run inside a defined boundary: limited permissions, a spending cap, an audit trail and a human with the authority and the means to stop it. An agent without a kill switch is not autonomous. It is abandoned.
- Plan for the rules to tighten. The direction is clear even if the timetable is not. Firms that build testing, reporting and oversight into their systems now will find compliance cheap later. Firms that do not will retrofit it at great cost, probably after an incident of their own.
One more test belongs above all five. A lawyer writing in a regional government-technology journal this week made a point every board should borrow: an AI-assisted decision should not be released merely because a human has looked at it. “Human in the loop” has become a reassuring phrase that hides more than it reveals. A reviewer shown a polished recommendation may not see where its sources came from, which assumptions remain unresolved, or that the law or the facts beneath it changed last week. What is needed instead is a release gate built on three questions. Who, by name, has the authority to turn this recommendation into a decision, and is that authority scaled to the stakes? What did they actually see: verified evidence, material still to be checked, open assumptions, each with its source and date? And what future event, such as a change in the law, a corrected data source or a new material fact, automatically sends the decision back for review? That third question turns oversight from a checkpoint into a continuing obligation. It is also, in miniature, the frontier statement’s whole argument. Control is not a person glancing at a screen. It is a named human, seeing the evidence, with the power to stop the machine and a standing duty to look again.
The statement’s authors chose their title carefully. It is not a call for a ban, a pause or a moratorium. It is a call for control, which is a much older and much more reasonable idea. Every powerful technology humanity has adopted, from the steam boiler to the jet engine, became safe not when it stopped being dangerous but when someone was made responsible for it, someone checked, and every failure was written down and shared.
Twenty-two nations have now said that frontier AI should be no exception. The two chairs that matter most are still empty. History suggests they will not stay empty for ever, because the first serious incident that crosses a border will fill them. The question for the rest of us is whether the rules are waiting when that happens, or whether, once again, we write them in the wreckage.
Leave a Reply