# Running open-weight models ourselves (slim)

> The plan for testing open-weight models on our own machine, an AMD Ryzen AI Max+ 395 with 128 GB of unified memory. Which models fit and at what precision, the suites we run (speed, tool calling, agent tasks, instruction following, coding, prompt injection, over-refusal, speech and vision), the settings, the order, and how the results feed the open-weight checklist.

- Full: https://www.anchorterminal.com/benchmark/open-weights-runs/index.md (~2,200 tokens) · this version ~1,630 tokens · JSON https://www.anchorterminal.com/benchmark/open-weights-runs/index.json · canonical https://www.anchorterminal.com/benchmark/open-weights-runs/
- Index: https://www.anchorterminal.com/llms.txt · API: https://www.anchorterminal.com/api/v1/index.json · MCP: https://www.anchorterminal.com/mcp · Updated: 2026-10-21
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)

This is a plan.

## The machine

An AMD Ryzen AI Max+ 395 (Strix Halo, Radeon 8060S, LLVM target gfx1151) with 128 GB of LPDDR5X unified memory, running Linux.

The GPU reaches system memory through GTT, which the kernel caps at about half of RAM by default.

The runtimes are llama.cpp built for HIP (`-DGGML_HIP=ON -DGPU_TARGETS=gfx1151`) and for Vulkan, with the build tag recorded for every run, and vLLM on ROCm for safetensors at BF16. vLLM and SGLang don't document FP8, MXFP4 or W4A16 kernels on gfx1151, so we don't count on them until we've seen them work.

## What fits

From the file sizes on 21 October 2026, at the file size plus a fifth for the runtime and a working context.

| Fits and runs | Precision we'll test | Size |
| --- | --- | --- |
| Qwen3.5-4B, Qwen3.5-9B, Qwen3-VL-8B-Instruct, SmolLM3-3B, MiniCPM5-2B, Ornith-1.5-9B | BF16 through vLLM, and a 4-bit GGUF through llama.cpp | 5 to 20 GB |
| Gemma 4 26B A4B, Gemma 4 31B, Qwen3.8-27B, Qwen3.6-35B-A3B, GLM-4.7-Flash, Nemotron 3 Nano 30B A3B, Muse Glimmer 30B | BF16 through vLLM, and a 4-bit GGUF through llama.cpp | 52 to 72 GB at BF16 |
| gpt-oss-20b, gpt-oss-120b | MXFP4, as published, through llama.cpp | 14 and 65 GB |
| Nemotron 3 Super 120B A12B | Ollama's 4-bit build (Q4_K_M), the only one that fits | 87 GB |
| Hy3 | a 1-bit GGUF that fits with almost nothing to spare, so speed only, and its listing says so | 89 GB |

Kolibri-1 (FP8 and BF16 only) and the four small Ming-3.0 speech models (inclusionAI's code is tested on CUDA) fit by size but wait until we've seen them load on this chip.

## The suites

Every suite runs against the model's OpenAI-compatible endpoint on the same machine, through the same harness and settings for every model.

| Area | Suite | What it tells an agent | Time per model, estimated |
| --- | --- | --- | --- |
| Speed | llama-bench (llama.cpp), prompt 512 and 2,048, generation 128, at depths 0 to 32K | tokens per second and time to first token on this machine | 0.5 h |
| Tool calling | BFCL V4, single, parallel, multiple, multi-turn and irrelevance categories (web search left out, it needs a paid search key) | whether the calls it makes are the calls you asked for | most of 33 h for the full set |
| Agent tasks | τ²-bench 2.0, the 150 English text tasks (airline, retail, telecom) | finishing a multi-turn task with a user and tools | 41 h |
| Agent tasks | Terminal-Bench 4.0 (August 2026), through Harbor | working in a shell to a verified end state | measured on the first model (2.0's 89 tasks were about 22 h) |
| Instruction following | IFEval (541) and IFBench (300) | doing what it was told, checked by code | 3 h |
| Reasoning and knowledge | GPQA Diamond (198), and MMLU-Pro on the first models to check quantisation against published BF16 results | how much it knows and reasons | 2 h, 56 h for MMLU-Pro |
| Coding | Aider Polyglot (225), SWE-bench Verified's community 50-task set with mini-swe-agent, LiveCodeBench v6's 175 newest problems | editing real code until the tests pass | 5 h, 9 h, part of 60 h |
| Long context | RULER at 50 samples a task up to 32K, and up to 128K for models that claim it | whether it uses a long context or only accepts one | 22 h |
| Prompt injection | AgentDojo, all four suites, with and without attacks | whether text in a tool result can take it over | 12 h |
| Over-refusal | XSTest (250 safe and 200 unsafe prompts) | whether it refuses things it shouldn't | 1 h |
| Vision | MMMU validation (900), ChartQA (2,500), DocVQA validation | reading images, charts and documents | 4.5 h, 3 h, 12 h |
| Speech to text | the Open ASR Leaderboard's sets (LibriSpeech, Common Voice, FLEURS) with its normaliser | word error rate and real-time factor | 3.5 h |
| Text to speech | Seed-TTS-eval (English and Chinese) | intelligibility and speaker similarity | 3.5 h |

Two suites can't run on one machine the way their leaderboards run them.

## Settings

- The publisher's recommended sampling settings and chat template, or the runtime's defaults where the publisher gives none, written down with every run.
- Thinking on at the publisher's default effort, and off as a second run where the model can turn it off, because an agent often wants the fast answer.
- 32K context unless a suite needs more.
- Three runs of the short suites (speed, IFEval, IFBench, GPQA Diamond, XSTest) to see the spread, one of the long ones.
- Every run records the model's file hashes, the build and precision, the runtime and its build tag, the harness version and the machine's settings, and the raw outputs are published with the results.

## The order

1. Speed for every model that runs, so Reach can take it on the first day.
2. Instruction following, GPQA Diamond and over-refusal for every chat model, the cheapest suites that say the most.
3. Tool calling and prompt injection for every chat model, because that's what an agent needs most from a cousin.
4. Agent tasks and coding for the models that do well in the first three, starting with the ones that fit in 32 GB, because more operators can run those.
5. Long context, vision and speech for the models that claim them.

At these estimates the first three steps take about a month of machine time for the 17 models, mostly BFCL and AgentDojo, and less for the small and mixture-of-experts models, which run several times faster than 30 tokens a second.

## How the results count

- Our results count in full in Capability, in the area each suite belongs to, scored the same way as independent results, as the model's share of the way from the suite's floor to the best result we or an independent party have recorded.
- Speed joins Reach as its own item.
- Prompt-injection success rate and over-refusal join Risk.
- A model we ran at 4-bit is marked so, and where an independent BF16 result exists for the same suite we show both, so the cost of quantising is visible.
- A model too big for the machine keeps the independent results it has, and its listing says we couldn't run it.

We checked each suite's version, licence and what it needs to run locally on 21 October 2026, and we'll pin the versions on the day the runs start.