# Check an LLM's output before it is sent or stored

> Check an LLM's output before it is sent or stored: 4 companies fully do it, 16 with a limit, 30 carry the tag. Every listing in Decision models with the reason and a source.

- Canonical: https://www.anchorterminal.com/niches/decision-models.verify-output/
- Markdown: https://www.anchorterminal.com/niches/decision-models.verify-output/index.md (~2,000 tokens)
- Slim: https://www.anchorterminal.com/niches/decision-models.verify-output/index.min.md (~430 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/niches/decision-models.verify-output/index.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-15

Four or more companies' docs show a full fit.

| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |
| --- | --- | --- | --- | --- |
| 4 | 16 | 30 | 12 | 5 |

## What counts as done

A supported or unsupported probability per response.

Measured, for our own tests: the probabilities match held-out human labels at least as well as an LLM judge on the caller's set, at under a tenth of the judge's cost.

How agents ask for it: is every claim in this answer supported by the context; does this draft follow policy; is this retrieved passage relevant; which of these two replies is better

## Does the job

The docs show it meets the test above, and nothing we read says otherwise.

- **Celeris-1 Decision (Marqo Inc)**: predicate and score questions over a response plus context, 512 questions a request, $0.004 per million input tokens, far under a judge's cost. No independent accuracy measurement against human labels was found.
- **Clef (Cloudflare, Inc.)**: Arize: 85.2 percent balanced accuracy on RAGTruth, within noise of an Opus 5 judge at 86.6 percent and $14.30 per 1,000, at $0.223 per 1,000; 91.5 percent on the Phoenix evaluator suites; wording-stable.
- **OpenAI Decisions API (OpenAI OpCo, LLC)**: Arize: 81.5 percent on RAGTruth with intervals touching Jev's, 90.0 percent on the Phoenix suites, calibration error 0.042, wording-stable, $0.090 per 1,000 against $14.30 for an Opus 5 judge.
- **Jev (TypeSafe AI, Inc.)**: Arize: 85.5 percent on RAGTruth with calibration error 0.023, within noise of an Opus 5 judge at 86.6 percent and $14.30, at $0.045 per 1,000; wording-stable (0.93 both ways); a verify_llm_outputs cookbook exists.

## Does the job, with a limit

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

- **Decider (Mark Marosi (Mapika))**: noul and score questions over a context, with the README warning that calibration on hard items is the weak axis (ECE 0.15 to 0.31 on JevBench hard). No measurement against an LLM judge was found.
- **Decision 3.0 (vLLM Semantic Router project)**: noul and score questions over context and answer in one call. No calibration figures on the cards and no measurement against an LLM judge; issue 6367 crashed the batched /v1/decisions route on a rejected state.
- **Drex Max v1 (Nace AI, Inc.)**: noul and score over context and answer; vendor's Decision Index and Artificial Analysis scores only, no judge comparison. Licence cap.
- **elephant-1 (Baseten Labs, Inc.)**: noul and score over context and answer, with 16 percent of training on judging and rubric grading. No comparison against an LLM judge on human labels was found.
- **Granite Switch 4.1 (International Business Machines Corporation)**: Possible as a two-label or multi-label logprob read with the state as a chat turn; no typed answer, no calibration figure and no judge comparison.
- **Kev (Jared Palmer)**: Arize: 76.8 percent on RAGTruth against the 86.6 percent judge, 88.7 percent on the Phoenix suites, calibration error 0.055, and a coin flip on reversed wording.
- **JEV-27B-VL (AutoTrust AI Lab)**: The card's agent-judge result (73.2 percent macro, AUROC 0.91) and multimodal judge tables are AutoTrust's own; one question per request and no human-label comparison.
- **joey-1 (Baseten Labs, Inc.)**: noul and score over context and answer, with 16 percent of training on judging and rubric grading. No comparison against an LLM judge on human labels was found.
- **Liquid d1 (Liquid AI, Inc.)**: Arize: 84.8 percent on RAGTruth, within noise of the 86.6 percent judge at $0.034 per 1,000, but calibration error 0.051 and the reversed-wording flip to 0.17 AUC.
- **mammoth-1 (Baseten)**: noul and score over context and answer, with 16 percent of training on judging and rubric grading. No comparison against an LLM judge on human labels was found.
- **Microsoft-Decision-1 (Microsoft)**: Microsoft says its Copilot team found it competitive with GPT-5.6 Luna for judging responses; no independent comparison on human labels was found.
- **Drex 1.5 (Nace AI, Inc.)**: noul and score over context and answer, 512 questions a request; no calibration figure and no comparison against an LLM judge on human labels.
- **pplx-decider (Perplexity AI, Inc.)**: Grouped questions change each other's answers, so independent candidates must go one per request, which multiplies calls; the docs say grouping cost up to 4.8 points on ranking tasks. No judge comparison.
- **pplx-decider-flash (Perplexity AI, Inc.)**: One question per request, so no grouping effects, but 10.74 points behind the 27B on the index and the docs say to keep Flash only if it meets your accuracy bar on your own labelled sample.
- **RightNow System Zero (RightNow AI, Inc.)**: boolean and score over context and answer, per-question refusals, uncalibrated probabilities and per-question fees; no human-label comparison.
- **ScaleDown (Carbon ScaleDown, Inc.)**: noul and score per question, each its own call; 'constrained, calibrated decode' is claimed without a figure, and no judge comparison exists.
- **SPEV (Sprag AI, Inc.)**: predicate and score over text, with many questions per request; Sprag's figures cover latency against OpenAI, not accuracy against human labels.
- **kettei (Sqwish Ltd, trading as Sqwish Labs)**: A groundedness example and an evaluations endpoint that reports accuracy, log loss and ECE on your own dataset; no independent comparison against an LLM judge.
- **Vela 2.0 (vLLM Semantic Router project (with KR Labs))**: A 'halu' span head marks unsupported claims with offsets and probabilities and RAGTruth F1 is published; no comparison against an LLM judge on the caller's labels and Choice confidence isn't calibrated by the usage guide's own note.

- not this job: Laya (Convai Innovations Pvt. Ltd.): Arize: 59.8 percent balanced accuracy on RAGTruth against an Opus 5 judge at 86.6 percent, and AUC 0.64 falling to 0.36 with the question reversed.
- not this job: GLiClass (Knowledgator Engineering Ltd.): No state-plus-question shape for a response against its context; GLiClass scores label strings against one text. Not aimed at this use case.
- not this job: Strands Decider 2B (Amazon Web Services, Inc.): Arize: 69.0 percent on RAGTruth against the 86.6 percent judge, and AUC 0.75 falling to 0.26 with the question reversed.

1 listings in the category were added after this reading and aren't marked yet.

## What stops agents finishing

- reversed question wording inverts the answer on several open models
- grouped candidates change each other's answers, so one candidate per request multiplies calls
- context over 32k on long RAG bundles

## Around it

Before: inference.llm.generate, vector-databases.retrieve. After: chat.send-message, human-review, evals.record-score.

## Where this comes from

Weight 12 of 100 in Decision models: 5 of 41 accounts; the Arize RAGTruth and Phoenix evaluator runs, the OpenRouter verify step and the AWS judge and rerank desks

As data: https://www.anchorterminal.com/api/v1/use-cases/decision-models.verify-output.json, or the MCP tool get_use_case with id decision-models.verify-output. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp
