{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/fine-tuning.rl-custom-loop/",
    "json": "https://www.anchorterminal.com/niches/fine-tuning.rl-custom-loop/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/fine-tuning.rl-custom-loop/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/fine-tuning.rl-custom-loop/index.min.md"
  },
  "markdown": "No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 0 | 7 | 11 | 15 | 8 |\n\n## What counts as done\n\nThe service or library accepts a custom reward function or a hand-written training loop, documents which base models RL runs on, renders tool calls in the model's own format, and returns a checkpoint that can be sampled from during and after training.\n\nHow agents ask for it: run GRPO with my reward function; RL fine-tune this model on our task; write a custom post-training loop\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **Amazon Bedrock model customisation (Amazon Web Services, Inc.)**: Reinforcement fine-tuning scores responses with a reward function in Lambda on Nova 2 Lite, gpt-oss-20B and Qwen3 32B, but there is no hand-written loop, no sampling mid-training in the pages read, and tool-call rendering is not documented.\n- **Axolotl (Axolotl AI)**: GRPO takes a custom reward module with vLLM serving for rollouts, and checkpoints save to disk, but no per-model RL support table or tool-call rendering for agent RL was found in the pages read.\n- **Microsoft Foundry fine-tuning (Azure OpenAI) (Microsoft Corporation)**: Managed RFT with model graders runs on the models the table marks, and Programmed Training exposes training primitives and custom losses, but Programmed Training is preview and needs explicit access approval.\n- **Fireworks AI Fine-tuning (Fireworks.ai, Inc.)**: The Training API takes custom loops and losses (forward_backward_custom, optim_step), GRPO and agentic RL recipes, and checkpoints sampled mid-run; but the per-model method matrix is an interactive widget, and tool-call rendering is not documented in the pages read.\n- **Tinker (Thinking Machines Labs, Inc.)**: A hand-written loop with create_lora_training_client, RL recipes with verifiable rewards and sampling from saved weights mid-run, and acc-35 and acc-36 report it working; but acc-38 found 17 of 200 tool-call headers malformed and acc-37 a renderer failing under concurrency.\n- **Unsloth (Unsloth)**: GRPO takes custom reward functions and verifiers with documented examples, but Studio's train page is SFT only (acc-40), the preference trainers lack the GRPO memory work (acc-41), and tool-call rendering for RL is not documented.\n- **Vertex AI Gemini tuning (Google LLC)**: RLFT on Gemini 3.5 Flash and 3.1 Flash-Lite takes up to 16 weighted rewards, including a custom Python evaluate function or a Cloud Run endpoint, with step checkpoints; but it is Pre-GA, there is no hand-written loop, and tool-call formatting is not documented.\n\n- not this job: Nebius Token Factory fine-tuning (Nebius B.V.): Reinforcement fine-tuning is a limited professional service behind a request-access form; no reward function or training-loop API is documented.\n- not this job: Together AI Fine-tuning (Together Computer, Inc.): Managed fine-tuning has SFT and DPO; the docs send RL and custom training loops to post-training, a separate product billed per GPU-minute and not this listing.\n\n## What stops agents finishing\n\n- RL offered only through a request-access programme\n- renderer emits malformed tool-call headers, so rewards are computed on broken calls\n- studio or UI has SFT only\n\n## Around it\n\nBefore: code-sandboxes.run-generated-code. After: agent-observability.offline-eval-regression, fine-tuning.export-and-serve.\n\n## Where this comes from\n\nWeight 15 of 100 in Fine-tuning: 8 of 47 accounts: Tinker users on Hacker News and in cookbook issues (GRPO, on-policy distillation, tool-call rendering), a GRPO write-up, and two Unsloth requests for RL and preference trainers\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/fine-tuning.rl-custom-loop.json, or the MCP tool get_use_case with id fine-tuning.rl-custom-loop. Draft, read 2026-11-15 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Train with reinforcement learning and a custom reward",
        "url": ""
      }
    ],
    "description": "Train with reinforcement learning and a custom reward: 0 companies fully do it, 7 with a limit, 11 carry the tag. Every listing in Fine-tuning with the reason and a source.",
    "facts": null,
    "h1": "Train with reinforcement learning and a custom reward",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/fine-tuning.rl-custom-loop/",
    "published": "",
    "section": "agents",
    "title": "Train with reinforcement learning and a custom reward | Anchor…",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/fine-tuning.rl-custom-loop/"
  },
  "tokens": {
    "markdown": 1200,
    "slim": 330
  },
  "version": 1
}
