{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/fine-tuning.distil-to-small/",
    "json": "https://www.anchorterminal.com/niches/fine-tuning.distil-to-small/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/fine-tuning.distil-to-small/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/fine-tuning.distil-to-small/index.min.md"
  },
  "markdown": "No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 0 | 7 | 13 | 12 | 6 |\n\n## What counts as done\n\nTeacher outputs or traces can be turned into a training set in the documented format, the tuned small model is served at a published per-token price below the model it replaces, and the docs show how to evaluate it against the teacher on held-out cases before switching.\n\nHow agents ask for it: make a cheaper model that does this step as well; distil our judge into a small model; train a small model on GPT outputs for this task\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **Amazon Bedrock model customisation (Amazon Web Services, Inc.)**: Distillation generates teacher responses from your prompts and tunes the student, and on-demand serving bills per use, but only Nova models and Llama 3.3 70B deploy on demand, each in one Region; others need Provisioned Throughput.\n- **Microsoft Foundry fine-tuning (Azure OpenAI) (Microsoft Corporation)**: Distillation is documented as SFT on teacher-generated data, but Standard and Global Standard serving add an hourly hosting fee to base-model token rates, and fee-free Developer deployments are removed after 24 hours.\n- **Fireworks AI Fine-tuning (Fireworks.ai, Inc.)**: A distillation cookbook exists and serverless LoRA serves adapters per token at the base price, but only on GLM 5.3 Flash with three adapters per account; adapters on other bases need an on-demand deployment.\n- **Nebius Token Factory fine-tuning (Nebius B.V.)**: SFT on chat JSONL suits teacher traces and acc-02 plans precisely this, but tuned models deploy only to dedicated endpoints (custom weights in beta on request) and no per-token price for a tuned model is published.\n- **Tinker (Thinking Machines Labs, Inc.)**: Cookbook recipes cover on- and off-policy distillation, but per-token serving prices and an evaluation-against-teacher guide sit in the docs, which were not read this time.\n- **Together AI Fine-tuning (Together Computer, Inc.)**: Conversational JSONL with loss on assistant turns fits teacher traces, but tuned models are served on dedicated endpoints billed per minute even when idle; acc-19 and acc-45 describe serverless LoRA the current docs no longer offer.\n- **Vertex AI Gemini tuning (Google LLC)**: SFT on Flash-Lite from teacher outputs is served per token at the base price in the global region (1.5 times elsewhere for Gemini 3), below a Pro teacher, but no guide to evaluating the student against the teacher was found.\n\n- not this job: Axolotl (Axolotl AI): Trains a student from chat datasets, but serves nothing, so the per-token price and side-by-side switch the success needs lie outside the tool.\n- not this job: Unsloth (Unsloth): It can train a student on teacher traces, but it serves nothing, so the per-token price and swap-over the success needs lie outside the tool.\n\n## What stops agents finishing\n\n- tuned model served only on a dedicated endpoint, so it costs more than the call it replaces\n- traces need cleaning the service does not do\n\n## Around it\n\nBefore: agent-observability.coding-agent-sessions, agent-observability.offline-eval-regression. After: inference.fast-small-calls, open-weights.sub-task-cousin.\n\n## Where this comes from\n\nWeight 12 of 100 in Fine-tuning: 6 of 47 accounts: an agent picking and tuning a replacement model, a judge distilled on Token Factory, a Fireworks tune for consistent output, an email sorter, a forum design for a trace-trained coding adapter and one vendor benchmark\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/fine-tuning.distil-to-small.json, or the MCP tool get_use_case with id fine-tuning.distil-to-small. Draft, read 2026-11-15 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Replace an expensive model call with a small tuned model",
        "url": ""
      }
    ],
    "description": "Replace an expensive model call with a small tuned model: 0 companies fully do it, 7 with a limit, 13 carry the tag. Every listing in Fine-tuning with the reason and a source.",
    "facts": null,
    "h1": "Replace an expensive model call with a small tuned model",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/fine-tuning.distil-to-small/",
    "published": "",
    "section": "agents",
    "title": "Replace an expensive model call with a small tuned model | Anchor…",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/fine-tuning.distil-to-small/"
  },
  "tokens": {
    "markdown": 1200,
    "slim": 330
  },
  "version": 1
}
