Benchmark
The lab: we run the tools and the models ourselves
The lab is one machine we run ourselves, beside the site's server. On it open-weight models try the tools we list and answer questions about this site, and what they find is published with its date. Every other score on the site comes from reading what a vendor publishes.
What it does
A fixed set of questions, run the same way for every model, so two scores can be put side by side. Open-weight models are the first, and more will follow.
15 models scored so far. Qwen3.8-Flash-Next leads on 98 of 100.
The evaluations Tool testsThe tools we list, tried by a modelA model is given only what a server says about its tools and writes programs to call each one. The lab's gateway counts and times every call, and what it found is on the listing, under Lab.
102 listings tried so far. The results move a listing's Performance and Task success.
How it moves a gradeThe setup
An AMD Ryzen AI Max+ 395 with 128 GB, running plain Debian. Every model runs on it through llama.cpp with its publisher's own sampling settings, from a build pinned by file hash.
A program a model writes runs with no network. What it calls goes through the lab's gateway, which counts and times every call and holds it to the same limits as the model's own calls.
A tool's descriptions and answers are a stranger's text, the model has just read them, and the program the model wrote is run. The site checks the box's results one field at a time before it shows any.
What the lab found on each listing, as JSON: /api/v1/lab.json. The models' scores: /api/v1/lab-evals.json.