# Replace an expensive model call with a small tuned model

> Replace an expensive model call with a small tuned model: 0 companies fully do it, 7 with a limit, 13 carry the tag. Every listing in Fine-tuning with the reason and a source.

- Canonical: https://www.anchorterminal.com/niches/fine-tuning.distil-to-small/
- Markdown: https://www.anchorterminal.com/niches/fine-tuning.distil-to-small/index.md (~1,200 tokens)
- Slim: https://www.anchorterminal.com/niches/fine-tuning.distil-to-small/index.min.md (~330 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/niches/fine-tuning.distil-to-small/index.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-15

No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.

| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |
| --- | --- | --- | --- | --- |
| 0 | 7 | 13 | 12 | 6 |

## What counts as done

Teacher outputs or traces can be turned into a training set in the documented format, the tuned small model is served at a published per-token price below the model it replaces, and the docs show how to evaluate it against the teacher on held-out cases before switching.

How agents ask for it: make a cheaper model that does this step as well; distil our judge into a small model; train a small model on GPT outputs for this task

## Does the job, with a limit

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

- **Amazon Bedrock model customisation (Amazon Web Services, Inc.)**: Distillation generates teacher responses from your prompts and tunes the student, and on-demand serving bills per use, but only Nova models and Llama 3.3 70B deploy on demand, each in one Region; others need Provisioned Throughput.
- **Microsoft Foundry fine-tuning (Azure OpenAI) (Microsoft Corporation)**: Distillation is documented as SFT on teacher-generated data, but Standard and Global Standard serving add an hourly hosting fee to base-model token rates, and fee-free Developer deployments are removed after 24 hours.
- **Fireworks AI Fine-tuning (Fireworks.ai, Inc.)**: A distillation cookbook exists and serverless LoRA serves adapters per token at the base price, but only on GLM 5.3 Flash with three adapters per account; adapters on other bases need an on-demand deployment.
- **Nebius Token Factory fine-tuning (Nebius B.V.)**: SFT on chat JSONL suits teacher traces and acc-02 plans precisely this, but tuned models deploy only to dedicated endpoints (custom weights in beta on request) and no per-token price for a tuned model is published.
- **Tinker (Thinking Machines Labs, Inc.)**: Cookbook recipes cover on- and off-policy distillation, but per-token serving prices and an evaluation-against-teacher guide sit in the docs, which were not read this time.
- **Together AI Fine-tuning (Together Computer, Inc.)**: Conversational JSONL with loss on assistant turns fits teacher traces, but tuned models are served on dedicated endpoints billed per minute even when idle; acc-19 and acc-45 describe serverless LoRA the current docs no longer offer.
- **Vertex AI Gemini tuning (Google LLC)**: SFT on Flash-Lite from teacher outputs is served per token at the base price in the global region (1.5 times elsewhere for Gemini 3), below a Pro teacher, but no guide to evaluating the student against the teacher was found.

- not this job: Axolotl (Axolotl AI): Trains a student from chat datasets, but serves nothing, so the per-token price and side-by-side switch the success needs lie outside the tool.
- not this job: Unsloth (Unsloth): It can train a student on teacher traces, but it serves nothing, so the per-token price and swap-over the success needs lie outside the tool.

## What stops agents finishing

- tuned model served only on a dedicated endpoint, so it costs more than the call it replaces
- traces need cleaning the service does not do

## Around it

Before: agent-observability.coding-agent-sessions, agent-observability.offline-eval-regression. After: inference.fast-small-calls, open-weights.sub-task-cousin.

## Where this comes from

Weight 12 of 100 in Fine-tuning: 6 of 47 accounts: an agent picking and tuning a replacement model, a judge distilled on Token Factory, a Fireworks tune for consistent output, an email sorter, a forum design for a trace-trained coding adapter and one vendor benchmark

As data: https://www.anchorterminal.com/api/v1/use-cases/fine-tuning.distil-to-small.json, or the MCP tool get_use_case with id fine-tuning.distil-to-small. Draft, read 2026-11-15 from public docs and first-hand accounts, not yet reviewed by a person.

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp
