On November 2, MIT CSAIL researchers Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab published a technical blog post titled “Sparks of Weight-Space Programming.” In it, OpenAI’s GPT-6 Astra reads and modifies a neural network’s weights directly, with no optimizer and no gradient descent. The authors call the approach “weight-space programming.”
The most closely watched result involves Qwen3.5-0.8B, an open-source small model from Alibaba. The model has 24 layers and roughly 800 million weights. Astra subtracted one “looping direction” from each of the MLPs in layers 15 and 19, changing 131,072 weights in total, or 0.016% of the model, with no retraining at all.
For now the material is only a blog post. It has not been peer reviewed, and the authors themselves describe the results as very early.
Eight experiments, with wins and failures across reading, repairing and writing
The authors split the tasks into three types.
Reading weights. Looking only at the weights of PPO policy networks, Astra worked out which of four grid games each one had been trained on. Given the weights of three DDPMs, it determined that they generate circles, triangles and squares respectively, though it drew the filled shapes as hollow outlines. On a 7.16-million-parameter character-level Transformer, it looked at only the middle four layers, identified the training corpora of three models and found all of the copy heads.
Repairing weights. In the Kinetix environment, a small cart-climbing-a-hill policy had failed in all 256 episodes. After Astra changed 3.5% of its weights, it passed all 1,024 episodes.
Writing weights from scratch. On three MinAtar games, Astra was given no data, no gradients and no tools, and had to write a small network in a single pass. Its scores against PPO:
| Game | PPO, 100M steps | PPO, 500M steps | Astra, written in one pass |
|---|---|---|---|
| Breakout | 15.8 | 25.3 | 517.4 |
| Seaquest | 5.6 | 88.9 | 53.1 |
| Asterix | 25.8 | 55.2 | 53.2 |
All three beat PPO at 100 million steps, but against the 500-million-step runs only Breakout still comes out ahead.
The blog also records a failure. Asked to tell which of two Gemma 3 270M models had been fine-tuned to be sycophantic, Astra tried four times looking at weights alone and never got it. Once it was allowed to inspect attention patterns, it got all three pairs right.
How much of the 6x is just fixing the loop
The Qwen baseline is unusual: 93% of its answers hit the token cap, with the model circling in its reasoning and never finishing an answer. With a budget of 50 forward passes for Astra, the numbers become:
| Metric | Original model | After editing |
|---|---|---|
| MATH-500 | 7.9% | 48.1% |
| GSM8K | 13.8% | 67.1% |
| IFEval | 21.0% | 62.3% |
| Hit the token cap | 93.0% | 33.1% |
The control group says more. Only 8.5% of Gemma 3 270M’s answers hit the cap, and its base score is 6.8. With 5, 10, 25 and 50 forward passes, Astra produced scores of 7.0, 7.6, 7.7 and 7.0, which is essentially no movement. The authors concede that the gain comes mainly from fixing the looping, and that editing cannot give a model abilities it did not already have.
This is where outside skepticism has concentrated. Latent Space’s AI News rounded up community discussion, in which some commenters argued that a 7.9% baseline is abnormally low, that the edit is closer to the single-direction ablation described by Arditi et al., and that because Astra does its analysis in a Python sandbox, the claim of “natively reading and writing weights” is somewhat overstated. Latent Space itself also flagged the “50 steps” wording as possibly exaggerated. These criticisms mostly come from discussion threads, and no systematic third-party replication has appeared yet.
“exciting but still limited”
That is the authors’ own assessment of the capability, and they attribute the limits to context length and the efficiency of the harness it runs in.
Placing it on the self-improvement line
Last month, Epoch AI’s InnovationEval tested a different direction: frontier models working out post-training methods on their own in an offline sandbox. The best result recovered only 15% of the improvement reported in the reference paper. That route runs through ideation, writing code and running training.
Weight-space programming skips the training step, which is why some people have raised it in discussions of recursive self-improvement. Taken together, what can be confirmed today is still narrow: a model can find something fixable in a small network with an obvious flaw, and there is no evidence yet for large models or models without an obvious defect. Applications such as auditing, zero-data tuning and model merging remain prospects in the blog, with no experiments behind them.
The next thing to watch is replication: swap in a small model that does not loop, or a math task with a normal baseline, and see whether Astra can still push the score up.
Sources: the authors’ MIT CSAIL technical blog, Machine Heart (Jiqizhixin), CocoLoop, Latent Space AI News; Qwen and Gemma scores follow the authors’ published experiment tables, and InnovationEval results follow Epoch AI’s report.