Local Qwen 3.8 27B Model Identifies Games From Raw Network Weights in Willison Test

Willison was responding to a study titled "Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights", by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. The authors said OpenAI's GPT-6 Astra could identify what a network had been trained to do from its weights, write small game-playing networks from scratch, and repair a Qwen3.5-0.8B model by editing 0.016% of its weights. According to the study, that edit raised the model's MATH-500 score from 7.9% to 48.1%.

The Test

Willison focused on the study's first experiment. The researchers had trained four small reinforcement learning policy networks on simplified 10x10 grid versions of Breakout, Pong, Pac-Man and Freeway. They then gave the weights to a language model as plain floating-point numbers and asked, without allowing tools, which game each network had learned. The prompts are published in a public repository, and apart from the weights they are identical.

Willison ran the prompts himself. Kimi K3.1 Pro, accessed through OpenRouter, correctly identified Breakout after 9 minutes 41 seconds and 19,493 reasoning tokens. He then ran a quantised 17GB version of Qwen 3.8 27B on his laptop through LM Studio. For it to work, he had to raise the context window above the 8,192-token default and extend the timeout.

The local model described the Breakout network as a paddle on the bottom row keeping a ball in play against bricks, with 88% stated confidence, after 52 minutes 25 seconds. It identified Pong after 1 hour 4 minutes 54 seconds, with 75% confidence. For Pac-Man, it described a maze game with reward pellets, with 72% confidence, after 46 minutes 12 seconds and 43,506 reasoning tokens. It got Freeway wrong, calling it a Pac-Man-like grid game. Willison said that miss did not surprise him, since Freeway, a 1981 Atari 2600 title, is the most obscure of the four.

Debate Over the Claims

Willison said the original study mixes two questions: what models can do with weights unaided, and how well models with access to code can modify other models. He argued the two should be judged separately. He pointed to critiques from spaCy author Matthew Honnibal, who called the paper misleading for conflating them, and from engineer Sean Goedecke, who argued that the Qwen repair amounted to steps a human machine learning researcher would take.

Willison called the no-tools reading result the more interesting part of the work. He said the reasoning traces were dense notes about individual weight values and grid positions, and that a capable person with pencil and paper could probably reach the same conclusions given enough effort. He described the results as impressive but said some commentary had gone too far, writing that he is "not ready to declare that the age of source code is over."

The test matters for anyone weighing claims about frontier-only capabilities. It suggests that inspecting small networks directly, at least for toy tasks, is within reach of open models that run on local hardware. It also shows how slow that still is.