Ball published a post distinguishing weak from strong recursion and arguing current evidence supports only the weak form.
OpenAI Uses AI to Build Models, But Not Autonomous Self-Improvement
The distinction between human-assisted AI development and fully autonomous self-improvement has direct implications for AI safety governance. Ball argued that conflating the two risks both unnecessary alarm and strategic disadvantage.
When stories like this move, I write about them in an email each morning.
One free email each morning, around 8 AM US Eastern.
Check your email.
I have sent you a confirmation link. One click and you are in.
Not in your inbox? Look in spam or promotions. If it's there, mark it "Not spam" so future editions land in your inbox.
The full picture
Sam Altman confirmed OpenAI uses its own AI to accelerate model development, but said the company is not running fully autonomous recursive self-improvement where a model designs its own successor without human oversight. Altman said OpenAI could push further toward that end of the spectrum but does not believe it can do so safely at present. Separately, Dean Ball drew a conceptual distinction between 'weak recursion,' where AI is a key input into producing better AI (which he characterizes as historically normal), and 'strong recursion,' where AI autonomously plans and executes its successor's development with no human in the loop. Ball argued that current evidence, including a METR report, supports only the weak form, since humans still specify the tasks. He called strong recursion a genuine near-term risk but argued the evidence for it is thinner than sometimes supposed, and said banning weak recursion would effectively ban AI improvement.
How it developed
Altman confirmed OpenAI uses AI to accelerate model development but not autonomous recursive self-improvement without human oversight.
Sources
Related
- Anthropic's embedded METR safety evaluation
- Dean Ball's OpenAI Strategic Futures blog
- Nick Cammarata's interpretability reversal
- Anthropic's Fable 5.2 and Mythos 5.2
BackgroundWhen AI doesn't do what we meant
Get the daily email
When stories like this move, I write about them in an email each morning.
One free email each morning, around 8 AM US Eastern.
Check your email.
I have sent you a confirmation link. One click and you are in.
Not in your inbox? Look in spam or promotions. If it's there, mark it "Not spam" so future editions land in your inbox.