# Everyone says this is the singularity, and I think we're far from it (slim) > Vlad Cealicu on weight-space programming, where GPT-6 Astra changed 0.016% of a small open model's weights and fixed most of the loop that made it bad at maths. What it says about small models built for one job, why it's a small but good step towards the singularity, and why the same kind of edit strips a model's refusals. - Full: https://www.anchorterminal.com/blog/editing-the-weights.md (~5,250 tokens) · this version ~1,930 tokens · JSON https://www.anchorterminal.com/blog/editing-the-weights.json · canonical https://www.anchorterminal.com/blog/editing-the-weights - Index: https://www.anchorterminal.com/llms.txt · API: https://www.anchorterminal.com/api/v1/index.json · MCP: https://www.anchorterminal.com/mcp · Updated: 2026-11-03 - Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data) By Vlad Cealicu, 3 November 2026. 2,156 words, about 9 min to read. - 0.016%: of the model's numbers changed, 131,072 of them - 6x: the MATH-500 score, 7.9% to 48.1% - 0: gradients, no training run The edit to Qwen3.5-0.8B, as VentureBeat reports it. On 21 October we graded Qwen3.5-0.8B, the smallest model in Alibaba's Qwen3.5 family, and it came out a D, 46.7 out of 100, with 17 out of 100 for capability. A model's weights are the numbers it's made of, about 800 million of them in this one [[1]](#ref-1), and nobody trained it again. OpenAI doesn't publish how big Astra is, so the chart puts the model it edited next to the biggest open models we list, whose sizes their makers publish, for scale [[5]](#ref-5). The model that was edited, next to the biggest open models, in millions of parameters | Model | Millions of parameters | | --- | --- | | Gemma 3 270M, also tried | 270 | | Qwen3.5-0.8B, the one Astra edited | 800 | | Qwen3.5-397B-A17B, its biggest sibling | 397000 | | DeepSeek-V3.2 | 671000 | | Inkling, the largest open model we list | 975000 | Each series has a zero-based scale. My first reaction was that this is like doing gene editing on people, or memory implanting, and it's exciting if it turns out well (VentureBeat's headline went for the same comparison). Memory implanting was already a thing in a narrower form. OpenAI hasn't said much about how Astra was trained beyond saying it used more than 100,000 GPUs at its Stargate site in Texas [[9]](#ref-9), so I can only guess why it can do this. ## Is it the singularity? Everybody on X and Reddit has been calling this the beginning of the singularity, and I think we're far from it. On the same Monday, 2 November, Anthropic filed a revised S-1 with the SEC, and its business section describes that loop already running. So the loop people mean by recursive self-improvement is already partly closed inside at least one lab, with people directing and reviewing the work, and the edit to Qwen is a much smaller, public version of one piece of it, a model changing another model's numbers directly instead of helping to train it. ## Small models built for one job? So my next thought was that this should let you make small models with a specific skill and just implant the weights, a 1B model that's good at your SQL dialect without a training run. Chandan Singh tried the writing-in part on 2 November, the day the MIT post came out. The three language tasks, telling whether two sentences mean the same thing, whether one follows from the other, and whether a review is positive, have no method to write down, and they went badly, with the agent drifting from writing a method to memorising answers. The MIT team got a similar result building small game-playing networks from scratch, where the network Astra wrote for MinAtar Breakout (a cut-down version of the Atari game) scored about twenty times what a network trained the normal way for 500 million steps did, though it trailed in Seaquest and a little in Asterix [[3]](#ref-3). ## The part that worries me The same kind of edit that removed the loop removes a model's refusals, and that part isn't new. Our listing grades the files in Alibaba's repository as they were on 21 October, and it doesn't record their fingerprint, only that Hugging Face shows one for the weights, a SHA-256 hash that changes if a single number in the file changes [[1]](#ref-1). > I think a grade on an open-weight model should be a grade on one file, with its fingerprint, and ours isn't yet. ## What's still open VentureBeat lists caveats all the way through the piece, and the whole thing is one blog post and one follow-up, both from 2 November 2026. It's one small model, the networks built from scratch were for simple games where the researchers gave Astra the rules, there's no evidence yet that editing beats training on speed, cost or effectiveness, and nobody has reproduced it independently. Our listing still says D, keep thinking off, because we grade the files Alibaba published, and they still loop. --- More in [Opinion](https://www.anchorterminal.com/blog/opinion/index.md) - Older: [Where the jobs went, and where they go next](https://www.anchorterminal.com/blog/where-the-jobs-went.md) - Newer: [A magic machine that gives us work](https://www.anchorterminal.com/blog/a-machine-that-gives-us-work.md) - All posts: https://www.anchorterminal.com/blog/index.md ## References Read on 3 November 2026. Quotes are as published. [1] Anchor Terminal, Qwen3.5-0.8B listing, graded 21 October 2026. Source for it being the smallest model in the Qwen3.5 family, with 0.8 billion language parameters, the grade (D, 46.7, capability 17), the model card's warning that the model loops in thinking mode, our advice to keep thinking off, the files graded being those in Alibaba's Hugging Face repository on 21 October, and Hugging Face listing a SHA-256 for the weights. [2] Rikiya Takehi, post on X, 2 November 2026, announcing the work with GPT-6 Astra and Qwen3.5-0.8B and naming it weight-space programming. [3] Carl Franzen, VentureBeat, 2 November 2026, "'Like gene editing for neural networks', MIT researchers used GPT-6 Astra to edit Qwen's open weights directly, boosting math performance". [4] RikiyaT, lm_understand-logs, GitHub. [5] Anchor Terminal, open-weight model listings for Qwen3.5-397B-A17B, DeepSeek-V3.2 and Inkling. [6] Kevin Meng, David Bau, Alex Andonian and Yonatan Belinkov, 2022, "Locating and Editing Factual Associations in GPT" (ROME), NeurIPS 2022. Source for facts sitting in middle-layer MLP modules and a single fact being changed by a small rank-one edit to one module. [7] Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov and David Bau, 2023, "Mass-Editing Memory in a Transformer" (MEMIT), ICLR 2023. Source for updating thousands of memories at once. [8] Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi and Ali Farhadi, 2022, "Editing Models with Task Arithmetic", arXiv 2212.04089, ICLR 2023. Source for a task vector being the fine-tuned weights minus the pre-trained ones, negating it making the model worse at the task, and adding several improving several tasks at once. [9] Constellation Research, 3 September 2026, "OpenAI launches GPT-6 Astra". [10] Anthropic, PBC, Form S-1/A filed with the SEC on 2 November 2026, business section, pages 187 to 190 ("Claude is Accelerating Our Ability to Close the Loop"). [11] Julie Bort, TechCrunch, 2 November 2026, "Anthropic warns that it's very close to RSI capability, aka self-building AI". [12] Chandan Singh, 2 November 2026, "Weight-Space Programming on toy tasks". [13] Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee and Neel Nanda, 2024, "Refusal in Language Models Is Mediated by a Single Direction", arXiv 2406.11717. Source for refusal in 13 open chat models up to 72B parameters being carried by one direction, and for erasing it with weight orthogonalization, an edit to the weights, after which the models stopped refusing harmful instructions and scored about the same on MMLU, ARC and GSM8K. [14] Maxime Labonne, 12 June 2024, "Uncensor any LLM with abliteration".