{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/local-ai.coding-agent-backend/",
    "json": "https://www.anchorterminal.com/niches/local-ai.coding-agent-backend/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/local-ai.coding-agent-backend/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/local-ai.coding-agent-backend/index.min.md"
  },
  "markdown": "One company's docs show a full fit.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 1 | 16 | 49 | 26 | 21 |\n\n## What counts as done\n\nThe harness runs against the local endpoint with every tool call parsed into structured calls.\n\nMeasured, for our own tests: the harness completes a multi-step edit task (read, edit, run a command, re-read) with no manual retries and no 4xx or 5xx from the runner, at 20 tokens per second or more on the owner's hardware.\n\nHow agents ask for it: point Claude Code at a local model; run OpenCode against my Ollama; use qwen3-coder on my Mac instead of paying for a subscription; offload small coding tasks to a local model\n\n## Does the job\n\nThe docs show it meets the test above, and nothing we read says otherwise.\n\n- **vLLM (The Linux Foundation (vLLM is a PyTorch Foundation project))**: A Claude Code integration page gives the ANTHROPIC_BASE_URL setup against the Anthropic Messages implementation, with --enable-auto-tool-choice and a parser per model. Responses and chat routes are documented, and Codex has its own page.\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **Berth (Trevin Tindall (T-Crypt on GitHub))**: The preview gateway proxies /v1/chat/completions, /v1/completions and /v1/embeddings to the engine that owns the card, with KV snapshots so an agent context survives a switch. No Anthropic or Responses route; Linux and NVIDIA for full support.\n- **Docker Model Runner (Docker, Inc.)**: An account used the Anthropic-compatible route on a Mac, and the README shows OpenAI routes. Another got HTTP 500 whenever the model made two tool calls in one turn, and the maintainer said Model Runner is being deprecated for llmman.\n- **Foundry Local (Microsoft Corporation)**: The REST reference documents OpenAI chat completions with functions and tools and a supportsToolCalling flag per model; no Anthropic or Responses route. An issue shows tool calling failing on the QNN NPU while CUDA works.\n- **Core (ZMJ, Inc., doing business as Ghost, 325 9th Street, San Francisco, CA 94103, United States)**: The FAQ claims an OpenAI Responses-compatible endpoint for OpenCode, OpenClaw and Codex, but publishes no address, authentication, model names or limits, and the device had not shipped: batch 1 was due 31 October and stock reads sold_out.\n- **HyperQwen (syv.ai (Copenhagen, CVR DK44671247))**: vLLM's OpenAI API with a Qwen tool parser and 64k to 240k context at 127 tokens a second on an RTX 3090. One model, Linux with a 24 GB NVIDIA card, a 30-minute first start, and no harness docs.\n- **Jan (Menlo Research Pte Ltd)**: OpenAI-compatible server on 127.0.0.1:1337/v1 over llama.cpp, so OpenCode-style clients work. No Anthropic Messages or Responses route is documented for Claude Code or Codex, and no harness page exists.\n- **KoboldCpp (LostRuins (Concedo))**: The OpenAPI file lists /v1/messages, /v1/responses and /v1/chat/completions, so every harness shape exists. No harness docs page, and one account's coding-agent run on a P40 stalled on parsing before it worked.\n- **Lemonade (Advanced Micro Devices, Inc.)**: A Claude Code docs page (lemonade launch claude) plus /v1/responses and chat routes with tool_calls. An open issue on Strix Halo shows every prompt past about 33k tokens crashing with 36 tool definitions loaded.\n- **llama.cpp (ggml.ai, part of Hugging Face since 2026)**: Server README documents /v1/messages, /v1/responses and tool use with --jinja; accounts run Claude Code, pi and Codex against it. No per-harness docs page, and streamed tool-call bugs with OpenCode and Qwen recurred before fixes.\n- **LM Studio (Element Labs, Inc.)**: OpenAI chat, Responses and Anthropic Messages routes are documented and one account runs OpenCode daily. Others saw Claude Code lose its place, Codex's namespace MCP tools dropped on /v1/responses, and a Qwen fix-and-break loop.\n- **LocalAI (Ettore Di Giacinto and the LocalAI team)**: OpenAI chat, Anthropic Messages and Open Responses APIs are served across backends, with grammar-constrained tool calls. No harness docs page was found, and tool calls on vLLM backends depend on a parser setting.\n- **MLX LM (Apple Inc.)**: One OpenAI-style chat completions route with a 512-token default max_tokens and no Anthropic or Responses route, so Claude Code and Codex have nothing to call. The server parses tool_calls in source, but SERVER.md leaves tools out.\n- **Ollama (Ollama Inc.)**: Docs pages for Claude Code (Anthropic route, ollama launch claude), Codex (wire_api responses) and OpenCode (OpenAI chat). Default context is 4k below 24 GiB VRAM and silently truncates until raised to 64k, which one account hit.\n- **PrivateGPT (PriBAI Technology Spain S.L.)**: Claude Code and OpenCode pages with a compatibility table (tools, file tools, sub-agents) and private-gpt run claude-code. It is a layer over a separate OpenAI-compatible model server, and the last release is 1.0.1 of June 2026.\n- **TextGen (oobabooga)**: The API doc calls it a drop-in for OpenAI and Anthropic including Messages, with tool_calls and ctx_size on load. --api must be set and a model loaded first; no harness page, and no release since v4.9 of May 2026.\n- **Underdog (Conway Research)**: The Underdog app documents no API. The 27B model card gives a Splash serve command for an OpenAI-compatible endpoint and 130 tokens a second on Python coding on an M5 Max, but that is Splash's route, with no Anthropic shape or harness docs.\n\n- not this job: AnythingLLM (Mintplex Labs, Inc.): Not a runner. /v1/openai/chat/completions chats with a workspace through the configured LLM provider; no Anthropic or Responses route and no harness docs.\n- not this job: BitRun (Abhishek Khanra (IshekKhal on GitHub)): Six ternary models of 70M to 8B with context windows of 768 to 8,192 tokens behind one chat completions route; no Anthropic or Responses route and no tool calling.\n- not this job: GPT4All (Nomic, Inc.): The API has models, completions and chat completions only, with no tool, Anthropic or Responses route, and must be switched on in the desktop app.\n- not this job: Khoj (Khoj Inc.): A personal assistant over notes; no model-serving API shape for a coding harness.\n- not this job: LocalGhost (LocalGhost.ai Ltd (company number 17213100, registered in England and Wales, incorporated 12 May 2026)): The README says there is no API, MCP server or agent surface; the box answers only its enrolled phone.\n- not this job: Open WebUI (Open WebUI Inc.): A chat front end over configured model connections, not a model runner. Its Anthropic and OpenAI routes forward to the backend; no harness setup is documented.\n- not this job: screenpipe (Negentropy Labs, Inc. (dba Screenpipe)): Not a model runner; it gives coding agents screen memory over MCP.\n\n## What stops agents finishing\n\n- model emits the tool call as plain text or inside reasoning_content, so the harness loop stalls\n- runner has no Anthropic Messages or OpenAI Responses route for the harness\n- default 4k context silently truncates the harness's system prompt and tool list\n- generation below about 15 tokens per second, so a task that takes Claude minutes takes an hour\n\n## Around it\n\nBefore: open-weights.local-coding-agent, agent-harnesses.local-model. After: agent-harnesses.pair-in-terminal.\n\n## Where this comes from\n\nWeight 26 of 100 in Local AI: 22 of 50 counted accounts; the single most common thing people reported doing with a local runner in the Hacker News comments and GitHub issues read\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/local-ai.coding-agent-backend.json, or the MCP tool get_use_case with id local-ai.coding-agent-backend. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Serve an open model to a coding harness on the owner's machine",
        "url": ""
      }
    ],
    "description": "Serve an open model to a coding harness on the owner's machine: 1 companies fully do it, 16 with a limit, 49 carry the tag. Every listing in Local AI with the reason and a source.",
    "facts": null,
    "h1": "Serve an open model to a coding harness on the owner's machine",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/local-ai.coding-agent-backend/",
    "published": "",
    "section": "agents",
    "title": "Serve an open model to a coding harness on the owner's machine |…",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/local-ai.coding-agent-backend/"
  },
  "tokens": {
    "markdown": 2150,
    "slim": 380
  },
  "version": 1
}
