# Train with reinforcement learning and a custom reward

> Train with reinforcement learning and a custom reward: 0 companies fully do it, 7 with a limit, 11 carry the tag. Every listing in Fine-tuning with the reason and a source.

- Canonical: https://www.anchorterminal.com/niches/fine-tuning.rl-custom-loop/
- Markdown: https://www.anchorterminal.com/niches/fine-tuning.rl-custom-loop/index.md (~1,200 tokens)
- Slim: https://www.anchorterminal.com/niches/fine-tuning.rl-custom-loop/index.min.md (~330 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/niches/fine-tuning.rl-custom-loop/index.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-15

No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.

| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |
| --- | --- | --- | --- | --- |
| 0 | 7 | 11 | 15 | 8 |

## What counts as done

The service or library accepts a custom reward function or a hand-written training loop, documents which base models RL runs on, renders tool calls in the model's own format, and returns a checkpoint that can be sampled from during and after training.

How agents ask for it: run GRPO with my reward function; RL fine-tune this model on our task; write a custom post-training loop

## Does the job, with a limit

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

- **Amazon Bedrock model customisation (Amazon Web Services, Inc.)**: Reinforcement fine-tuning scores responses with a reward function in Lambda on Nova 2 Lite, gpt-oss-20B and Qwen3 32B, but there is no hand-written loop, no sampling mid-training in the pages read, and tool-call rendering is not documented.
- **Axolotl (Axolotl AI)**: GRPO takes a custom reward module with vLLM serving for rollouts, and checkpoints save to disk, but no per-model RL support table or tool-call rendering for agent RL was found in the pages read.
- **Microsoft Foundry fine-tuning (Azure OpenAI) (Microsoft Corporation)**: Managed RFT with model graders runs on the models the table marks, and Programmed Training exposes training primitives and custom losses, but Programmed Training is preview and needs explicit access approval.
- **Fireworks AI Fine-tuning (Fireworks.ai, Inc.)**: The Training API takes custom loops and losses (forward_backward_custom, optim_step), GRPO and agentic RL recipes, and checkpoints sampled mid-run; but the per-model method matrix is an interactive widget, and tool-call rendering is not documented in the pages read.
- **Tinker (Thinking Machines Labs, Inc.)**: A hand-written loop with create_lora_training_client, RL recipes with verifiable rewards and sampling from saved weights mid-run, and acc-35 and acc-36 report it working; but acc-38 found 17 of 200 tool-call headers malformed and acc-37 a renderer failing under concurrency.
- **Unsloth (Unsloth)**: GRPO takes custom reward functions and verifiers with documented examples, but Studio's train page is SFT only (acc-40), the preference trainers lack the GRPO memory work (acc-41), and tool-call rendering for RL is not documented.
- **Vertex AI Gemini tuning (Google LLC)**: RLFT on Gemini 3.5 Flash and 3.1 Flash-Lite takes up to 16 weighted rewards, including a custom Python evaluate function or a Cloud Run endpoint, with step checkpoints; but it is Pre-GA, there is no hand-written loop, and tool-call formatting is not documented.

- not this job: Nebius Token Factory fine-tuning (Nebius B.V.): Reinforcement fine-tuning is a limited professional service behind a request-access form; no reward function or training-loop API is documented.
- not this job: Together AI Fine-tuning (Together Computer, Inc.): Managed fine-tuning has SFT and DPO; the docs send RL and custom training loops to post-training, a separate product billed per GPU-minute and not this listing.

## What stops agents finishing

- RL offered only through a request-access programme
- renderer emits malformed tool-call headers, so rewards are computed on broken calls
- studio or UI has SFT only

## Around it

Before: code-sandboxes.run-generated-code. After: agent-observability.offline-eval-regression, fine-tuning.export-and-serve.

## Where this comes from

Weight 15 of 100 in Fine-tuning: 8 of 47 accounts: Tinker users on Hacker News and in cookbook issues (GRPO, on-policy distillation, tool-call rendering), a GRPO write-up, and two Unsloth requests for RL and preference trainers

As data: https://www.anchorterminal.com/api/v1/use-cases/fine-tuning.rl-custom-loop.json, or the MCP tool get_use_case with id fine-tuning.rl-custom-loop. Draft, read 2026-11-15 from public docs and first-hand accounts, not yet reviewed by a person.

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp
