Updated todayNew5 sources

MIT team's weight-space programming reproduced; Gemma transfer limited

The gist

A frontier model editing another model's weights without training is a departure from standard fine-tuning pipelines, and independent reproduction confirms the core Qwen result. The limited transfer to Gemma raises questions about how broadly the technique generalizes across architectures.

When stories like this move, I write about them in an email each morning.

One free email each morning, around 8 AM US Eastern.

The full picture

MIT CSAIL researchers, led by Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab, demonstrated that GPT-6 Astra can directly inspect and edit another model's weights without gradient-based training, raising Qwen3.5-0.8B's MATH-500 accuracy from 7.9% to 48.1% by modifying only 131,072 weights (0.016% of total parameters). The technique, which they call Weight-Space Programming, worked by identifying two layers generating a repetitive signal that caused the smaller model to loop until hitting its context limit, then subtracting that signal directly from the weights. Alex Krentsel independently reproduced the MIT results and reported one additional finding beyond the original post. A follow-up experiment applying the same approach to Gemma 3 270M IT showed only marginal improvement, with MATH-500 accuracy rising from a 6.8% baseline to a range of 7.0% to 7.7%, indicating the technique does not transfer automatically across model architectures.

How it developed

3 November 2026

Latent Space covers the work, citing precise benchmark figures

2 November 2026

Follow-up experiment on Gemma 3 270M IT shows only marginal MATH-500 improvement

Sources

Dean Ball on X