# An open model on a £3,879 box scored 98 on our test. Claude scored 100 (slim)

> Vlad Cealicu on what eleven open-weight models scored on Anchor Terminal's own agent test, run on a Minisforum MS-S1 Max with 128 GB of memory. The best scored 98, a 14.4 GB model scored 92, and Claude Haiku, Sonnet and Fable each scored 100 by hand. What that means for choosing a model to run yourself and for anyone whose API an agent calls, and what the lab's harness does now.

- Full: https://www.anchorterminal.com/blog/local-models-on-one-box-against-claude.md (~3,400 tokens) · this version ~1,680 tokens · JSON https://www.anchorterminal.com/blog/local-models-on-one-box-against-claude.json · canonical https://www.anchorterminal.com/blog/local-models-on-one-box-against-claude
- Index: https://www.anchorterminal.com/llms.txt · API: https://www.anchorterminal.com/api/v1/index.json · MCP: https://www.anchorterminal.com/mcp · Updated: 2026-11-22
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)

By Vlad Cealicu*, 22 November 2026. 1,723 words, about 7 min to read.

- 98: the Lab score of Qwen3.8-Flash-Next on the lab box, with 57 of 59 questions right
- 92: the Lab score of Gemma 4 26B A4B, from a 14.4 GB file
- 100: the Lab score Claude Haiku 5.5, Sonnet 5.5 and Fable 5.2 each got on the same 59 questions, run by hand
- £3,879: a Minisforum MS-S1 Max with 128 GB of memory and a 4 TB disk

Anchor Terminal's lab evaluation, each model's latest score [1], the hosted runs [2] and Minisforum's UK store [3], read 21 November 2026.

The lab box next to our server is a Minisforum MS-S1 Max with 128 GB of unified memory, and the same machine was £3,879 on Minisforum's UK store on 21 November 2026 [[3]](#ref-3).

Three Claude models took the same 59 questions by hand on 21 November 2026 and each scored 100 [[2]](#ref-2).

Lab score on the same 59 questions, 0 to 100

| Model | Lab score |
| --- | --- |
| Claude Fable 5.2, by hand | 100 |
| Claude Sonnet 5.5, by hand | 100 |
| Claude Haiku 5.5, by hand | 100 |
| Qwen3.8-Flash-Next | 98 |
| Gemma 4 26B A4B | 92 |
| Gemma 4 31B | 87 |
| Qwen3.6-35B-A3B | 77 |
| GLM-4.5-Air | 67 |
| gpt-oss-120b | 66 |
| Qwen3.5-9B | 66 |
| Nemotron 3 Super 120B A12B | 63 |
| Qwen3.5-4B | 62 |
| gpt-oss-20b | 61 |
| GLM-4.7-Flash | 55 |

Each series has a zero-based scale.

The test is 59 questions an agent might ask Anchor Terminal, such as what grade we give a listing or which of two model APIs has the higher score [[2]](#ref-2).

## If you're choosing a model to run yourself

| Model | Lab score | Right, of 59 | Hardest ones, of 12 | File |
| --- | --- | --- | --- | --- |
| Qwen3.8-Flash-Next | 98 | 57 | 12 | 75.8 GB |
| Gemma 4 26B A4B | 92 | 55 | 10 | 14.4 GB |
| Gemma 4 31B | 87 | 52 | 10 | 17.7 GB |
| Qwen3.6-35B-A3B | 77 | 50 | 5 | 36.9 GB |
| GLM-4.5-Air | 67 | 44 | 5 | 63.6 GB |
| gpt-oss-120b | 66 | 43 | 5 | 63.4 GB |
| Qwen3.5-9B | 66 | 44 | 3 | 9.5 GB |
| Nemotron 3 Super 120B A12B | 63 | 41 | 5 | 79.0 GB |
| Qwen3.5-4B | 62 | 41 | 4 | 4.5 GB |
| gpt-oss-20b | 61 | 40 | 4 | 12.1 GB |
| GLM-4.7-Flash | 55 | 38 | 3 | 31.8 GB |

Scores and questions right are from the evaluation's page [[1]](#ref-1), and file sizes from the lab's model list [[4]](#ref-4).

If what your agent does is look one thing up through an MCP server, almost any of these will do it.

The models split on work with several steps.

I wouldn't choose by the size of the file or the name on it.

One question on the lab box, from the fastest model to the slowest

| Who | Time | Outcome |
| --- | --- | --- |
| gpt-oss-20b | 48 s | Lab score 61 |
| gpt-oss-120b | 55 s | Lab score 66 |
| Qwen3.6-35B-A3B | 56 s | Lab score 77 |
| Qwen3.5-4B | 69 s | Lab score 62 |
| Qwen3.8-Flash-Next | 71 s | Lab score 98 |
| Gemma 4 26B A4B | 91 s | Lab score 92 |
| GLM-4.7-Flash | 102 s | Lab score 55 |
| Qwen3.5-9B | 2.0 min | Lab score 66 |
| GLM-4.5-Air | 2.3 min | Lab score 67 |
| Nemotron 3 Super 120B A12B | 4.8 min | Lab score 63 |
| Gemma 4 31B | 5.3 min | Lab score 87 |

Time on a log scale.

What you give up is time.

What you get for it is that nothing leaves the machine.

## If agents call your API

The model on the other end of your API won't always be a frontier one.

I don't think MCP is where this ends up.

What decided the hard questions was how much the site let a model ask for in one go.

The other thing is to say no clearly.

## How Claude took it

Claude isn't a model the box can run, so it took the test by hand.

## The harder set

A test where three models score 100 can't tell me when it's time to change model, so there's a second set of 66 questions, kept beside the normal one and never part of the Lab score [[2]](#ref-2).

| Model | Right in each pass, of 66 | Right, passes together | Score | Right in every pass |
| --- | --- | --- | --- | --- |
| Claude Fable 5.2 | 66, 66, 66 | 66 | 100 | 66 |
| Claude Sonnet 5.5 | 62, 64, 64 | 65 | 98 | 59 |
| Claude Haiku 5.5 | 64, 62, 55 | 63 | 95 | 53 |

Three passes each by hand, from the harness's notes [[2]](#ref-2).

What cost Haiku and Sonnet their questions was a count over 60 listings that Haiku left blank in all three passes, rows counted that the question ruled out, a deduction left off a grade and a licence read from the wrong part of a file [[2]](#ref-2).

## What the harness does now

The scores above are from the first run of the eleven models.

A model is told how many replies, calls and fetches a question gets, and it gets more of each, 32 replies and 200 calls and 200 fetches where it had 16 and 40 [[5]](#ref-5).

On the four questions about something that doesn't exist, being told so counts as the lookup, and the eleven saved runs are marked again from their own records, so no model runs twice for it [[6]](#ref-6).

The lab is also learning to test plain HTTP APIs the way it tests MCP servers, with a model writing a small program against an API from what the API says about itself [[7]](#ref-7).

No local model has taken the harder set yet.

---

More in [From the data](https://www.anchorterminal.com/blog/from-the-data/index.md) · Previous from the data [What to build that lasts.

- Older: [We test a model on our own site before it tests anyone else's tools](https://www.anchorterminal.com/blog/we-test-a-model-on-our-own-site-first.md)
- Newer: [A small file took an open model from 35% to 51% on hard coding tasks. What a LoRA is, and what it changes outside code](https://www.anchorterminal.com/blog/what-a-lora-is-and-what-it-changes.md)
- All posts: https://www.anchorterminal.com/blog/index.md

Anchor Terminal grades the tools AI agents run on, with a source for every score.

## References

Read on 21 and 22 November 2026.

[1] Anchor Terminal, Lab score, open-weight models on our own evaluation.

[2] Anchor Terminal, anchorlab, README.md and DESIGN.md.

[3] Minisforum, MS-S1 Max, UK store.

[4] Anchor Terminal, anchorlab, MODELS.md.

[5] Anchor Terminal, anchorlab, pull request 35. Source for a model being told its limits, the replies, calls, fetches, tokens and context it gets, the repaired build and the reasons a question was lost.

[6] Anchor Terminal, anchorlab, pull request 34. Source for being told a thing isn't there counting as the lookup, and the saved runs being marked again.

[7] Anchor Terminal, anchorlab, pull request 37. Source for the lab testing a plain HTTP API from its OpenAPI document.
