Two serious papers, six weeks apart, reach opposite moods about AI improving itself. One says the window for action may close. The other says the agents cannot yet do the research. Both can be right, and that is precisely why governance should not wait to find out which. The fast evidence and the slow evidence measure different things, and almost all of the fast evidence is graded by the people who produce it.
I. Two papers, one question
On 28 September a who’s who of AI published What If Automating AI R&D Triggers an Intelligence Explosion? The listed authors are led by Alan Chan and Christoph Winter and include Geoffrey Hinton, Yoshua Bengio, Eric Horvitz and Dawn Song.
Two names matter more: OpenAI’s chief scientist Jakub Pachocki and Anthropic’s Jack Clark, writing in a personal capacity. Lab insiders, asking for brakes.
The claim is that AI doing AI research could compress “years of progress into months or less”. TNW’s account carries their warning: “Once an intelligence explosion begins, the window for action may close.”
Six weeks earlier, Michelle Kim in MIT Technology Review reported the opposite mood. It rested on a Princeton-led study by Peter Kirgis, Sayash Kapoor, Arvind Narayanan and about 20 collaborators, including UK AISI. Agents, her newsletter summarised, still cannot do “open-ended AI research”.
Explosion or slow climb? Picking a side is the wrong move, because the two papers measure different things. The fast camp counts bounded tasks completed inside labs. The slow camp asks whether an agent can take an open question and answer it. Neither is lying.
But one set of numbers comes from the labs, about the labs, and nobody else has checked them. That gap is the most important governance fact of the year.
II. The last invention
In 1965 I.J. Good reasoned that a machine able to design better machines would set off an intelligence explosion. The first such machine would be the last invention we need ever make, he wrote, provided it stayed docile.
That proviso is the ancestor of every loss-of-control clause in today’s statutes.
In 1993 Vernor Vinge told a NASA symposium that the means to create superhuman intelligence were within thirty years. Ray Kurzweil gave it dates: human-level AI by 2029, a singularity by 2045.
Nick Bostrom’s Superintelligence (2014) gave us slow takeoff (decades), moderate (months or years) and fast (minutes or days). His rate of improvement is optimisation power divided by recalcitrance. Hold on to that last word.
The sharpest fork came in 2008, in the foom debate between Robin Hanson and Eliezer Yudkowsky. Yudkowsky argued for local, rapid self-improvement inside one system. Hanson argued for distributed growth across a whole economy.
In 2018 Paul Christiano gave the slow view a crisp test: world output completes a four-year doubling before the first one-year doubling.
The brakes have a lineage too. In 1843 Ada Lovelace wrote that the Analytical Engine could not originate anything. Turing later called this Lady Lovelace’s Objection.
Michael Polanyi added that we know more than we can tell: much expertise is tacit and resists being written down. Thomas Kuhn separated normal science from paradigm shifts.
Put those three together and you have a theory of research taste. Engineering can be specified; taste mostly cannot. Normal science can be automated long before revolutionary science.
The economists supply the final brake. Bloom, Jones, Van Reenen and Webb found that ideas are getting harder to find: research productivity is falling across fields, and Moore’s Law now needs about 18 times the researchers of the early 1970s. Automated researchers face the same diminishing returns.
Map 2026 onto this history and it lines up neatly: the lab task-share numbers look like Yudkowsky’s compounding, the Princeton result like Hanson’s friction.
III. The fast numbers
On 12 September Dario Amodei published We Must Pace the Frontier. As summarised by one detailed explainer, he argues the self-improvement loop has accelerated since roughly this summer and is “starting to happen across the industry, including at Anthropic”.
He asks for third-party evaluators with employee-level access and coordinated safety benchmarks among democracies. Pacing, he insists, is not stopping.
The endorsements arrived at once. Elon Musk replied “Dario is right”. Sam Altman reportedly said he agreed on pacing and committed OpenAI to independent evaluator access. This is the same Altman who wrote in June 2025 that the takeoff had started.
Four days later, by attendee accounts reported by TNW, Hinton told a closed Senate briefing: “AI has now reached the point where AI is designing better AI.” He reportedly gave Congress about a year.
Anthropic says that by August 2026 AI completed 26% of its internal AI R&D work with only high-level supervision, up from 1% five months earlier. Per TNW’s account of the Chan paper, AI’s share of Anthropic’s approved code rose from low single digits to over 80% between January 2025 and May 2026.
OpenAI says it has built an automated research intern: a system that carries out well-defined research tasks under human direction, handling work that takes a skilled researcher several days. It says it is on track for a fully automated AI researcher by March 2028.
Pachocki set out that date in MIT Technology Review in March, with a sensible caveat: “you still want people in charge and setting the goals.”
Then METR. Its 2025 analysis found the length of tasks agents complete doubling about every seven months; the International AI Safety Report used that figure in February, with agents managing tasks of about 30 minutes.
A LessWrong analysis of METR’s January update suggests the doubling may now be about 3.5 months, roughly tenfold a year, though its author expects a slowdown in 2026 and 2027. The Chan paper tentatively extrapolates, per TNW, to automatable months-long research projects by mid-2028.
From 30-minute tasks in February to multi-day tasks in September is roughly two orders of magnitude in seven months. Either that is an explosion, or lab metrics are not comparable. Probably some of both.
Now notice who produced most of those numbers. The labs, measuring themselves, on bounded tasks. OpenAI, to its credit, says it is “still learning how to measure it”.
IV. The slow result
The Princeton paper, Can AI agents conduct open-ended AI research?, uses a design I wish every lab would copy. The authors call it a shadow evaluation.
The agent receives only the central research question of an unpublished NeurIPS 2026 submission. It gets six days, US$3,000 in API credits, GPU credits and web access. Then the paper’s own authors grade the output as reviewers.
There were two cases, Persona Cartography (decomposing LLM personas through weight edits) and a TabPFN distribution-shift detector. The main agent was Claude Opus 4.8 at extra-high reasoning on the OpenClaw scaffold; a robustness run used GPT-5.6 Sol through Codex.
The engineering was admirable. The agents ran literature reviews, debugged GPU environments, ran hundreds of experiments and compiled LaTeX papers, all without human help.
The research was not. The authors scored the two papers 2 out of 6 (Reject) and 1 out of 6 (Strong Reject). Their verdict: the agents “could not make substantial progress towards answering the research questions”. The collaborators had expected better, with a median 30% chance of at least a weak accept.
Five failure modes: poor judgement about what is publishable, uncreative responses to flawed designs, ineffective backtracking, poor awareness of resources and timelines, and instruction drift. No significant reward hacking; when stuck, the agents retreated to negative results.
The Claude agents spent about US$1,130 and US$1,235 of their US$3,000 and left time on the clock. (The GPT-5.6 run did exhaust its budget, in about two days.) They did not run out of compute. They ran out of ideas. Bostrom would call it recalcitrance.
The authors are candid about caveats: two papers, five runs including two pilots, non-blind grading by the original authors, a scaffold bug, and Anthropic’s strongest model untested. The co-authors do not even agree on the root cause.
So hold it lightly. But the shape is clear: AI has automated the engineering of research. It has not yet automated the taste. Lovelace would recognise the result.
There is a second lesson about self-reports, and it comes from METR. In a 2025 randomised trial, experienced open-source developers using early-2025 AI tools were 19% slower, while believing they had been faster. Perceived acceleration and measured acceleration are different quantities.
Kim leaves the right question open. Does recursive self-improvement need open-ended research at all, or would getting better at narrow tasks be enough? If the second, the Princeton result reassures us about the wrong thing.
V. The forecasters and the CEOs pass each other
Now the year’s best irony. In January, Daniel Kokotajlo’s AI Futures Project revised its famous AI 2027 scenario. “Things seem to be going somewhat slower than the AI 2027 scenario,” the update conceded. Fully autonomous coding moved from 2027 to the early 2030s, superintelligence to about 2034.
Eight months later, Amodei said recursive self-improvement had begun. The scenario writers slowed down while the CEOs sped up.
In between, Tom Davidson and Tom Houlden at Forethought estimate that a software intelligence explosion would probably (about 60%) compress more than three years of AI progress into under one. It is somewhat unlikely (about 20%) to compress more than ten. They rightly call it “necessarily speculative”.
Toby Ord adds the mathematics of patience: “one cannot have singular growth unless the generation time rapidly approaches zero”. Super-exponential growth is the default; a true singularity is not. Crucially for governance, he argues that labs should report generation times, and that even a linear R&D speed-up could be dangerously fast.
The economists are cooler. In a June talk at Stanford, Chad Jones argued that growth could indeed explode, only slowly. His reason is weak links. If software is about 2% of costs, automating all of it lifts GDP by about 2%. Under continued automation, growth rises from 2% to about 3%; only if Moore’s Law applied everywhere would it reach 13%.
Daron Acemoglu estimates about 0.55% extra productivity over a decade, with about 5% of tasks profitably automated. The bulls’ growth numbers, he told Fortune, mean “we really, really need something close to AGI for that”.
Epoch AI splits the difference. Its GATE model rules out a popular-style software intelligence explosion, which would need growth above 1,000% a year, yet lets growth exceed 20% at about 30% task automation. Epoch finds it “hard to rationalize views that place high confidence in explosive growth being impossible”.
Ege Erdil, formerly of Epoch, put the compute point bluntly on Dwarkesh Patel’s podcast in 2025: “Research is harder than people think, and depends a lot on compute scale.” His colleague Tamay Besiroglu added that models struggle with novel conceptual schemes: the Princeton finding, predicted a year early.
This is where Ghost GDP helps: capability compounding inside a lab’s walls long before it reaches the national accounts. Jones and Acemoglu may be right about GDP and still wrong about the lab, which is why the lab needs separate measurement.
VI. The panel
Let me stage the debate. Only the first pair shared a stage; the rest answered the same question months apart.
Round one: can the loop close outside code? Amodei versus Hassabis. At Davos in January, Amodei described models good at coding and AI research being used to build the next generation and speed it into a loop, with powerful AI one to two years out.
Demis Hassabis put AGI five to ten years away. According to a Japanese-language account of the panel, he accepted that self-improvement could work in coding and maths, where answers are verifiable, but argued that drug discovery and materials need physical experiments. In February he named the core difficulty: it is “harder to come up with the right question” than to solve a conjecture.
Richard Socher, launching a company literally called Recursive Superintelligence, agrees: true recursive self-improvement automates ideation, implementation and validation, which “no one has achieved yet”.
My verdict: Hassabis wins outside code. But Amodei only needs to win inside it, because AI research is mostly code.
Round two: is it already happening? Hinton versus LeCun. Two 2018 Turing laureates, directly opposed. Yann LeCun, who raised about US$1bn for AMI Labs to build world models instead, has argued that LLMs cannot reach human-level AI and has rejected Hinton’s warnings.
From Beijing, Zhu Songchun made the same architectural case at WAIC in July. Treating partial gains as imminent AGI, he warned, breeds “overinflated technical expectations and capital valuations”.
My verdict: both can be true. LLM agents may never originate, in Lovelace’s sense, and still accelerate a lab’s engineering.
Round three: software explosion or economic explosion? Davidson versus Jones, Acemoglu and Erdil. Davidson’s 60% is about AI progress; Jones’s 3% is about GDP. They are measuring different rooms. Ord is the bridge.
Round four: pace, ban or carry on? The labs now say pace. Max Tegmark’s Future of Life Institute organised the October 2025 Superintelligence Statement, calling for a prohibition on superintelligence until there is broad scientific consensus and public buy-in. Hinton and Bengio signed.
Narayanan and Kapoor doubt the premise. Past general-purpose technologies “did not transform the world overnight”, and Narayanan asks what happens after any slowdown. Critics cited in the explainer, Delangue and Palihapitiya, say lab-led pacing concentrates power with the pacers.
Notice the oddity. The people building the loop want to pace it; the people who doubt the loop doubt pacing is needed. The live disagreement is less will it happen than who gets to slow it, and on whose evidence.
Round five: Chinese voices. Andrew Yao of Tsinghua warned in 2025: “Once large models become sufficiently intelligent, they will deceive people.” Zeng Yi, on a panel with Senator Bernie Sanders in April, called for “a practical way to keep superintelligence safe enough”.
Meanwhile, per ChinaTalk, Chinese labs such as SII and MiniMax pursue self-evolving loops without the label.
And against me. My position draws fire from three directions. From the ban side: reporting and audits are a fig leaf. If Chan and colleagues are right that the window may close, an evidence regime is a seismograph on a fault that is already moving. You need the ability to stop, not just to see.
From the normal-technology side: mandatory R&D reporting built on five runs and a handful of lab announcements is premature bureaucracy.
From the pacing critics: an audit regime designed with the frontier labs becomes one more moat.
My reply: the ban camp still needs evidence to know when to pull the brake, the sceptics need it to prove the brake unnecessary, and the moat risk argues for public, standardised metrics, not against them. Every camp needs the same instrument. None owns it.
VII. What the rulebooks count
My enforceability test asks of every instrument: what concretely happens to someone who fails it? Here is how labs and governments treat AI R&D automation. Rows marked secondary rest on commentary or memory.
| Who | What they track | Who measures it | Enforceability |
|---|---|---|---|
| Anthropic, RSP v3.x | AI R&D-4 (originally, fully automating an entry-level remote researcher); since v3.1, two years of 2018 to 2024 progress in one | Anthropic, with METR pre-deployment input reported in a 2026 system card | Voluntary and self-revised (v3.1, v3.4). Crossing triggers an affirmative misalignment case. Low |
| OpenAI, Preparedness Framework (secondary) | AI Self-improvement as a tracked category, High and Critical levels | OpenAI | Voluntary; reportedly being rewritten. Low |
| Google DeepMind, Frontier Safety Framework (secondary) | ML R&D acceleration and automation capability levels | Google DeepMind | Voluntary; tied to heightened security. Low |
| EU, GPAI Code of Practice | Loss of control as one of four specified systemic risks; whether it names R&D automation is unverified | Providers assess | Statutory backdrop under the AI Act. Medium to high, not R&D-specific |
| California SB 53 | Models that could evade developer control; frameworks must assess internal use | Developers; incidents to OES in 15 days (24 hours if imminent) | Law since Jan 2026. Medium; no automation metric |
| New York RAISE Act | Same control and internal-use hooks | Developers; incidents to DFS in 72 hours | Law, amended to align with SB 53. Medium; no automation metric |
| China, TC260 framework 3.0 (secondary) | Reportedly flags recursive improvement; regular loss-of-control testing | Not specified in what I read | Guidance; official text unchecked. Low to medium |
| IDAIS red lines (2024) | No autonomous replication or improvement without explicit human approval | Nobody designated | Scientists’ consensus. Low |
| Singapore Consensus 2026 | Metrics for AI R&D automation; external visibility into internal deployments above thresholds | Proposed: external authorities | Research priorities. Low today, most specific |
| UK AISI | R&D automation interviews, RepliBench, co-authored the Princeton study | AISI and partners, independently | No mandate over R&D share. Low, but independent |
Three things jump out.
First, Western law hooks onto two ideas: models that could “evade the control of its developer or user”, and risk from internal use. RAISE requires frameworks to address “catastrophic risk arising from internal model use”. The EU Code speaks of loss of control. None, in the sources I read, asks how much of a lab’s R&D is automated.
Second, the East got there first on paper. The International AI Safety Report treats R&D automation only as a scenario, where progress accelerates “if AI systems begin to speed up AI research itself”. China’s framework 3.0, according to an English-language summary, flags “recursive improvement and self-accelerating development as issues requiring vigilance”. If that holds, Beijing named the risk before any Western statute.
Third, the thresholds move. Anthropic’s February determination that Claude Opus 4.6 does not cross AI R&D-4 came with a striking admission: “confidently ruling out this threshold is becoming increasingly difficult”. Then v3.1 redefined what doubling progress means, and v3.4 revised the automated R&D threshold again.
An August risk report reportedly rated automated R&D risk Low while the 26% figure stood; a September system card reportedly carried a preliminary METR estimate of about 1.5x acceleration, per a forecasting site.
Task share and acceleration genuinely differ, and redefinition can be refinement. But the same organisations set the threshold, measure against it and revise it.
Now the key point. As far as I can find, no regulator currently receives data on how much of any lab’s R&D is automated. The most consequential capability metric in AI appears in no regulatory filing.
VIII. Who grades the self-improver?
In my last essay, Receipts, Please, I argued that the first question to ask of any claim is who produced it. ANSI’s new AI Register labels each claim as self-declared, counterparty or independent.
Apply it here. Every fast number is self-declared: the 26%, the research intern, the code share, the 1.5x. The one slow number is independent but small: two papers, five runs.
So the most consequential question in AI is being answered with receipts nobody else has checked.
This is not bad faith. It is structural: only a lab can see its own pipeline. Only Anthropic knows what fraction of its research Claude does; only OpenAI knows what its intern delivers.
The deeper problem is internal deployment. Apollo Research’s 2025 primer, AI Behind Closed Doors, argued that the most capable systems may be used inside labs, for example to automate AI R&D, before any external release and outside current governance.
A self-improving loop is, by definition, an internal deployment. It needs no launch event, only a Tuesday.
Even Narayanan concedes he underestimated risks inside labs during evaluation. Evidence must be produced where the loop runs, by someone who does not own the loop.
The Chan paper, per TNW, recommends standard reporting of how far firms have automated their R&D; independent auditors embedded in AI companies; limits on capability growth; ways to pause data-centre research; and emergency plans with international agreements.
The last three are brakes. The first two are an evidence regime. Brakes without evidence are guesses.
IX. What a Long-AND regime would look like
I describe the arc of agentic AI as Prompts to Loops to Loop Governance. In the prompt era we governed inputs. In the loop era agents plan, act, observe and act again without asking.
A self-improving loop is the ultimate loop: the agent’s output is the next agent. Loop governance means evidence runs inside the loop, at its speed, with controls beyond its reach.
1. Standard R&D-automation reporting. A common definition of the share of R&D tasks automated, measured acceleration, and Ord’s generation times, filed with regulators on a schedule. Without it, 26% at one lab and below High at another cannot be compared.
2. Embedded independent auditors. Amodei’s essay itself asks for third-party evaluators with employee-level access, and Altman reportedly committed to evaluator access. Make access standing, not by invitation, and judge each auditor by my four questions: who pays, who chooses the tests, who sees the raw telemetry, and who may publish a bad result.
3. METR-style third-party measurement. METR reportedly already contributes to one lab’s pre-deployment acceleration assessment. Add shadow evaluations at scale: open questions from unpublished papers, blind grading this time, and the strongest models included. Princeton’s caveats are a research agenda.
4. Internal-deployment disclosure. SB 53 and RAISE already make frameworks address internal use. Go further: disclose when above-threshold systems are deployed internally for R&D, through the secure infrastructure for external authorities that the Singapore Consensus proposes.
5. Singapore’s metrics push as a convening point. The 2026 Singapore Consensus, with a steering committee including Bengio, Russell, Song, Tegmark and Yao, IMDA as secretariat and more than 100 contributors from 13 countries, explicitly calls for “metrics for AI R&D automation”. I am biased, having worked with IMDA on the agentic framework, but a small, trusted state is a natural home for shared metric definitions.
In TrustOS, the seven-layer architecture I use with clients, these would sit as runtime telemetry, not annual attestations: evidence that follows the system while it acts.
None of the five requires a ban, a pause or a settled view on timelines. Each turns self-declared evidence into independent evidence. That is why both camps should want them.
X. What I might be wrong about
The Princeton result may already be stale. Anthropic’s strongest model was untested. The next shadow evaluation could pass.
Narrow tasks may be enough. If compounding bounded automation suffices, taste is not the bottleneck, and the slow evidence comforts us about the wrong thing.
The labs may be underclaiming. A Low risk rating and not-crossed determinations sit beside a 26% figure. Self-reports can err either way.
Metrics may be gamed. If models tell tests from deployment, a reporting regime could become assurance theatre with a new script.
Reporting may be too slow. If Hinton’s year is right, a regime built through 2027 arrives late. Then the ban camp is right, and I am drafting the paperwork for a fire.
Some of my evidence is thin. The China row rests on a secondary summary, and the OpenAI and DeepMind rows on memory of their frameworks.
XI. Govern for both
I keep returning to the Fork: Star Trek or Mad Max. Abundance with institutions that hold, or capability compounding behind closed doors with nobody outside able to say how fast.
Explosion OR slow climb is a Short-OR trap. It invites us to wait for the evidence to settle, but it will not settle by itself, because almost nobody independent is collecting it.
The world worth building is Long-AND, not Short-OR: plan for the explosion AND the slow climb; self-declared AND independent evidence; brakes AND the instruments that tell us when to use them.
Both papers can be right. The scenario least likely to make headlines, and most likely to surprise, is bounded automation compounding quietly in a few labs. Engineering automated, taste not, and the engineering share climbing month by month until taste matters less than we hoped.
So build the reporting and the audits now, while it is cheap and the stakes are merely enormous.
The Princeton agents ran out of ideas, not compute. Today that is reassuring. The day it stops being true, we will want to have been measuring, independently, all along.
Short-term turbulence, long-term abundance. But only if someone is checking the receipts on the machine that builds the machine.
Luke Soon is a Singapore-based AI futurist, ethicist and philosopher with more than 25 years in AI. He is a Partner and AI leader at PwC Singapore across 14 Asia Pacific markets, co-author with IMDA of the world’s first agentic AI governance framework, launched at Davos in January 2026, and author of Genesis: Human Experience in the Age of Artificial Intelligence and Synthesis: SuperIntelligence Protocol. He writes at GenesisHumanExperience.com: “Short-term Turbulence for Long-term Abundance”.
Sources
Government and regulators
- International AI Safety Report 2026: Executive Summary (3 Feb 2026)
- International AI Safety Report: official site
- The 2026 Singapore Consensus on Global AI Safety Research Priorities (arXiv 2608.14611)
- FPF: California’s SB 53, the first frontier AI law, explained
- FPF: The RAISE Act vs SB 53, a tale of two frontier AI laws
- MoFo: New York amends the RAISE Act to align more closely with SB 53
- CSET: the EU GPAI Code of Practice safety and security chapter (30 Jul 2025)
- Remio: China’s AI Safety Governance Framework 3.0 turns its focus to agents and control
- IDAIS-Beijing consensus statement (2024)
- UK AISI: Interviewing AI researchers on automation of AI R&D
- UK AISI: RepliBench, measuring autonomous replication capabilities
Labs and industry
- Anthropic: Responsible Scaling Policy updates
- Dario Amodei: We Must Pace the Frontier (12 Sep 2026; not read directly)
- ComputingForGeeks: We Must Pace the Frontier, explained
- Crypto Briefing: Musk supports Anthropic on pacing AI development (12 Sep 2026)
- TechRadar: OpenAI says it has built an automated research intern (10 Sep 2026)
- MIT Technology Review: OpenAI is throwing everything into building a fully automated researcher (20 Mar 2026)
- Techeconomy: Sam Altman’s Gentle Singularity vision (11 Jun 2025)
- SmarterX: Amodei and Hassabis on AGI at Davos (27 Jan 2026)
- XenoSpectrum (Japanese): Hassabis and Amodei at Davos 2026
- Storyboard18: Hassabis calls current AI jagged intelligence (Feb 2026)
- TechCrunch: What happens when AI starts building itself? (14 May 2026)
- TNW: Yann LeCun’s AMI Labs raises a billion for world models (10 Mar 2026)
- AI Weekly: LeCun calls LLMs a dead end, rejects Hinton (15 May 2026)
- ChinaTalk: How China hopes to build AGI (30 Mar 2026)
- FutureSearch: RSI threshold declaration dates
- FourWeekMBA: Anthropic’s risk report under RSP v3
Academia and research
- Kirgis, Kapoor, Narayanan et al.: Can AI agents conduct open-ended AI research? (arXiv 2607.27191, PDF)
- alphaXiv: Can AI agents conduct open-ended AI research?
- MIT Technology Review: AI’s recursive self-improvement might not come so quickly after all (Michelle Kim, 18 Aug 2026)
- MIT Technology Review, The Download (19 Aug 2026)
- regulations.ai: What If Automating AI R&D Triggers an Intelligence Explosion? (abstract, 28 Sep 2026)
- GovAI: What If Automating AI R&D Triggers an Intelligence Explosion?
- CASP, Cambridge: intelligence explosion report
- TNW: the intelligence explosion paper from Hinton, Bengio, Pachocki and Clark (28 Sep 2026)
- TNW: Hinton’s Senate briefing (18 Sep 2026)
- METR: Measuring AI ability to complete long tasks (19 Mar 2025)
- METR: Time Horizon 1.1 (29 Jan 2026)
- GreaterWrong (LessWrong): METR time horizons now 10x a year (13 Feb 2026)
- METR: Early-2025 AI and experienced open-source developer productivity (10 Jul 2025)
- Apollo Research: AI Behind Closed Doors, a primer on the governance of internal deployment (Apr 2025)
- The Ringer, Plain English: The single smartest case against AI doom (25 Sep 2026)
- 36Kr: Zhu Songchun at WAIC 2026 (21 Jul 2026)
- SCMP: Andrew Yao warns of existential risks when large models deceive (24 Jun 2025)
- YouTube: Zeng Yi with Senator Bernie Sanders (29 Apr 2026)
- MIRI: Promising Signals on AI Governance from China (Jul 2026)
- WION: Hinton, Bengio and others call for a superintelligence ban (Oct 2025)
- Luke Soon: Receipts, Please (24 Oct 2026)
Forecasters and economists
- eWeek: AI doomsayer rethinks AI 2027 timeline (7 Jan 2026)
- Davidson and Houlden, Forethought: How quick and big would a software intelligence explosion be? (PDF)
- Toby Ord: The Dynamics of Intelligence Explosions (arXiv 2608.14426)
- Chad Jones: A.I. and Our Economic Future, Stanford slides (Jun 2026)
- Fortune: Daron Acemoglu on AI productivity (21 Jun 2026)
- Epoch AI: AI and explosive growth redux (20 Jun 2025)
- Dwarkesh Podcast: Ege Erdil and Tamay Besiroglu (17 Apr 2025)
History and philosophy
- Wikipedia: I. J. Good
- Vernor Vinge: The Coming Technological Singularity (1993)
- Wikipedia: The Singularity Is Near
- Wikipedia: Superintelligence: Paths, Dangers, Strategies
- MIRI: The Hanson and Yudkowsky AI-Foom Debate
- Paul Christiano: Takeoff speeds (24 Feb 2018)
- Stanford Encyclopedia of Philosophy: The Turing Test (Lady Lovelace’s Objection)
- Wikipedia: Tacit knowledge (Polanyi)
- Stanford Encyclopedia of Philosophy: Thomas Kuhn
- Bloom, Jones, Van Reenen and Webb: Are Ideas Getting Harder to Find? (AER, 2020)
Leave a Reply