I wrote about Weight-Space Programming and the MIT team’s numbers yesterday. Three rebuttals appeared within a day, and the most interesting one comes off a laptop.
Simon Willison reproduced it, locally
Willison repeated the post’s first experiment: four small networks trained on simplified versions of Breakout, Pong, Pac-Man and Freeway, with one question. Which game do these weights belong to? No tools, no code execution, a single response. Exactly the task where GPT-6 Astra scored four out of four.
Qwen 3.8 27B in LM Studio, 17 GB on his own machine:
| Game | Result | Time | Confidence |
|---|---|---|---|
| Pac-Man | correct | 46 min 12 s | 72% |
| Pong | correct | 1 h 4 min 54 s | 75% |
| Breakout | correct | 52 min 25 s | 88% |
| Freeway | wrong | – | – |
Kimi K3.1 Pro through OpenRouter got Breakout in 9 minutes 41 seconds and 19,493 reasoning tokens. The prompts sit in the authors’ GitHub repository, weights included as plain floating point numbers.
Willison’s verdict on the reasoning traces is disarming. He thinks an experienced human with a notepad and a pencil could probably get there too, given enough patience.
Honnibal takes the abstract apart
Matthew Honnibal, author of spaCy, calls the post misleading. His complaint is not about the experiments but about how they were bundled. The abstract opens with “just by looking at the raw weights” and then hands you the Qwen result. Read together, it suggests the maths jump happened without tools.
Honnibal sorted all eight experiments into a table. Only numbers 1, 2 and 4 work from raw weights with no tools, on networks of 17,000 to 172,000 weights. Every experiment involving a real language model writes Python. The Qwen experiment, the headline one, also gets to run the model on up to 50 batches of practice problems.
Sean Goedecke arrives at the same place from another direction: what Astra does are the steps a human ML engineer would take anyway.
The finding holds, the headline does not
Honnibal says himself that models from a year or two ago would have failed all of these tasks. That part survives. What does not survive is the reading in which a frontier model glanced at some numbers and made another model six times better at maths.
My own piece yesterday named the repair-not-new-capability side of the Qwen result but said nothing about tools. That was the gap, and Honnibal’s table fills it. And the fact that a 17 GB model on a laptop solves the same weight-reading task three times out of four does not shrink the finding. It moves it. The capability looks more widely available than anyone assumed, and a lot less magical.