Niches · Local AI

Serve an open model to a coding harness on the owner's machine

One company's docs show a full fit.

1companies, full fit
16companies, fit with a limit
49companies carry the tag
26share of the category's work
21first-hand accounts

What counts as done

The harness runs against the local endpoint with every tool call parsed into structured calls.

Measured, for our own tests: the harness completes a multi-step edit task (read, edit, run a command, re-read) with no manual retries and no 4xx or 5xx from the runner, at 20 tokens per second or more on the owner's hardware. No docs we read state these figures, so they don't count against a listing here.

How agents ask for it: point Claude Code at a local model, run OpenCode against my Ollama, use qwen3-coder on my Mac instead of paying for a subscription, offload small coding tasks to a local model

Does the job 1

The docs show it meets the test above, and nothing we read says otherwise.

ListingWhy
vLLM D
The Linux Foundation (vLLM is a PyTorch Foundation project)
A Claude Code integration page gives the ANTHROPIC_BASE_URL setup against the Anthropic Messages implementation, with --enable-auto-tool-choice and a parser per model. Responses and chat routes are documented, and Codex has its own page. source

Does the job, with a limit 16

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
Berth B
Trevin Tindall (T-Crypt on GitHub)
The preview gateway proxies /v1/chat/completions, /v1/completions and /v1/embeddings to the engine that owns the card, with KV snapshots so an agent context survives a switch. No Anthropic or Responses route; Linux and NVIDIA for full support. source
Docker Model Runner C
Docker, Inc.
An account used the Anthropic-compatible route on a Mac, and the README shows OpenAI routes. Another got HTTP 500 whenever the model made two tool calls in one turn, and the maintainer said Model Runner is being deprecated for llmman. source
Foundry Local B
Microsoft Corporation
The REST reference documents OpenAI chat completions with functions and tools and a supportsToolCalling flag per model; no Anthropic or Responses route. An issue shows tool calling failing on the QNN NPU while CUDA works. source
Core F
ZMJ, Inc., doing business as Ghost, 325 9th Street, San Francisco, CA 94103, United States
The FAQ claims an OpenAI Responses-compatible endpoint for OpenCode, OpenClaw and Codex, but publishes no address, authentication, model names or limits, and the device had not shipped: batch 1 was due 31 October and stock reads sold_out. source
HyperQwen C
syv.ai (Copenhagen, CVR DK44671247)
vLLM's OpenAI API with a Qwen tool parser and 64k to 240k context at 127 tokens a second on an RTX 3090. One model, Linux with a 24 GB NVIDIA card, a 30-minute first start, and no harness docs. source
Jan C
Menlo Research Pte Ltd
OpenAI-compatible server on 127.0.0.1:1337/v1 over llama.cpp, so OpenCode-style clients work. No Anthropic Messages or Responses route is documented for Claude Code or Codex, and no harness page exists. source
KoboldCpp B
LostRuins (Concedo)
The OpenAPI file lists /v1/messages, /v1/responses and /v1/chat/completions, so every harness shape exists. No harness docs page, and one account's coding-agent run on a P40 stalled on parsing before it worked. source
Lemonade B
Advanced Micro Devices, Inc.
A Claude Code docs page (lemonade launch claude) plus /v1/responses and chat routes with tool_calls. An open issue on Strix Halo shows every prompt past about 33k tokens crashing with 36 tool definitions loaded. source
llama.cpp D
ggml.ai, part of Hugging Face since 2026
Server README documents /v1/messages, /v1/responses and tool use with --jinja; accounts run Claude Code, pi and Codex against it. No per-harness docs page, and streamed tool-call bugs with OpenCode and Qwen recurred before fixes. source
LM Studio C
Element Labs, Inc.
OpenAI chat, Responses and Anthropic Messages routes are documented and one account runs OpenCode daily. Others saw Claude Code lose its place, Codex's namespace MCP tools dropped on /v1/responses, and a Qwen fix-and-break loop. source
LocalAI B
Ettore Di Giacinto and the LocalAI team
OpenAI chat, Anthropic Messages and Open Responses APIs are served across backends, with grammar-constrained tool calls. No harness docs page was found, and tool calls on vLLM backends depend on a parser setting. source
MLX LM C
Apple Inc.
One OpenAI-style chat completions route with a 512-token default max_tokens and no Anthropic or Responses route, so Claude Code and Codex have nothing to call. The server parses tool_calls in source, but SERVER.md leaves tools out. source
Ollama D
Ollama Inc.
Docs pages for Claude Code (Anthropic route, ollama launch claude), Codex (wire_api responses) and OpenCode (OpenAI chat). Default context is 4k below 24 GiB VRAM and silently truncates until raised to 64k, which one account hit. source
PrivateGPT C
PriBAI Technology Spain S.L.
Claude Code and OpenCode pages with a compatibility table (tools, file tools, sub-agents) and private-gpt run claude-code. It is a layer over a separate OpenAI-compatible model server, and the last release is 1.0.1 of June 2026. source
TextGen E
oobabooga
The API doc calls it a drop-in for OpenAI and Anthropic including Messages, with tool_calls and ctx_size on load. --api must be set and a model loaded first; no harness page, and no release since v4.9 of May 2026. source
Underdog F
Conway Research
The Underdog app documents no API. The 27B model card gives a Splash serve command for an OpenAI-compatible endpoint and 130 tokens a second on Python coding on an M5 Max, but that is Splash's route, with no Anthropic shape or harness docs. source

Doesn't do this job 7

Listings in the category that do something else, with the reason.

ListingWhy
AnythingLLM E
Mintplex Labs, Inc.
Not a runner. /v1/openai/chat/completions chats with a workspace through the configured LLM provider; no Anthropic or Responses route and no harness docs. source
BitRun E
Abhishek Khanra (IshekKhal on GitHub)
Six ternary models of 70M to 8B with context windows of 768 to 8,192 tokens behind one chat completions route; no Anthropic or Responses route and no tool calling. source
GPT4All F
Nomic, Inc.
The API has models, completions and chat completions only, with no tool, Anthropic or Responses route, and must be switched on in the desktop app. source
Khoj E
Khoj Inc.
A personal assistant over notes; no model-serving API shape for a coding harness. source
LocalGhost F
LocalGhost.ai Ltd (company number 17213100, registered in England and Wales, incorporated 12 May 2026)
The README says there is no API, MCP server or agent surface; the box answers only its enrolled phone. source
Open WebUI D
Open WebUI Inc.
A chat front end over configured model connections, not a model runner. Its Anthropic and OpenAI routes forward to the backend; no harness setup is documented. source
screenpipe C
Negentropy Labs, Inc. (dba Screenpipe)
Not a model runner; it gives coding agents screen memory over MCP. source

What stops agents finishing

  • model emits the tool call as plain text or inside reasoning_content, so the harness loop stalls
  • runner has no Anthropic Messages or OpenAI Responses route for the harness
  • default 4k context silently truncates the harness's system prompt and tool list
  • generation below about 15 tokens per second, so a task that takes Claude minutes takes an hour

Around it

Before: open-weights.local-coding-agent, agent-harnesses.local-model. After: agent-harnesses.pair-in-terminal.

Where this comes from

Weight 26 of 100 in Local AI: 22 of 50 counted accounts; the single most common thing people reported doing with a local runner in the Hacker News comments and GitHub issues read

The same as data: /api/v1/use-cases/local-ai.coding-agent-backend.json, or the MCP tool get_use_case with id local-ai.coding-agent-backend. Other jobs in this category: /api/v1/niches?category=local-ai.

Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.