# Run a non-coding agent with tools or MCP servers against a local model

> Run a non-coding agent with tools or MCP servers against a local model: 5 companies fully do it, 11 with a limit, 101 carry the tag. Every listing in Local AI with the reason and a source.

- Canonical: https://www.anchorterminal.com/niches/local-ai.agent-tool-loop/
- Markdown: https://www.anchorterminal.com/niches/local-ai.agent-tool-loop/index.md (~2,000 tokens)
- Slim: https://www.anchorterminal.com/niches/local-ai.agent-tool-loop/index.min.md (~430 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/niches/local-ai.agent-tool-loop/index.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-15

Four or more companies' docs show a full fit.

| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |
| --- | --- | --- | --- | --- |
| 5 | 11 | 101 | 15 | 11 |

## What counts as done

The runner returns tool calls in the structured tool_calls field and handles two or more calls in one turn.

Measured, for our own tests: across ten turns every tool call comes back in that field with arguments that validate against the schema, and the conversation is never left in a state the client cannot continue.

How agents ask for it: let my local model call a calculator tool; connect an MCP server to a local model; build a private assistant that runs Python in a sandbox; give the model file tools so it can fix formatting in my headers

## Does the job

The docs show it meets the test above, and nothing we read says otherwise.

- **KoboldCpp (LostRuins (Concedo))**: MCP tool calling since 1.106 takes a Claude Desktop mcp.json with HTTP and STDIO servers, tool toggles and optional manual approval. An account had Devstral reformat headers through an MCP file-tools server.
- **LocalAI (Ettore Di Giacinto and the LocalAI team)**: Functions map to grammars on llama.cpp with a parallel tool calls section and streamed reasoning separated from tool output. MCP connects HTTP and stdio servers with server-side execution limits.
- **PrivateGPT (PriBAI Technology Spain S.L.)**: Built-in server tools, custom tools returned as Anthropic tool_use blocks, client tools (bash, text editor) and MCP connectors are documented with request shapes.
- **TextGen (oobabooga)**: Tool calls come back as finish_reason tool_calls with structured names and arguments; Qwen and Mistral call several in parallel. MCP servers over HTTP and stdio (mcp.json) are documented in the tutorial.
- **vLLM (The Linux Foundation (vLLM is a PyTorch Foundation project))**: Tool parsers per model family, named and required tool choice guaranteed parsable, parallel_tool_calls true by default, streaming parsers, and a Responses example with MCP tools.

## Does the job, with a limit

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

- **AnythingLLM (Mintplex Labs, Inc.)**: Agents take MCP servers over StdIO, SSE and Streamable HTTP on Docker and Desktop. An open issue shows the schema helper dropping a parameter named definitions or $defs while leaving it required, so the model guesses its shape.
- **Berth (Trevin Tindall (T-Crypt on GitHub))**: Tool calls pass through to the engine untouched; Berth adds queuing and cold starts of up to minutes, with 503 and Retry-After while an engine loads.
- **Foundry Local (Microsoft Corporation)**: Tool support differs by hardware variant: on a Snapdragon X2 NPU the model answered in plain text with no tool call and ToolChoice Required failed everywhere, with no fix at reading.
- **HyperQwen (syv.ai (Copenhagen, CVR DK44671247))**: Tool calls go through vLLM's parser with a strict mode; the README notes a tool schema's maxLength drops decode to about 7 tokens a second without the bundled xgrammar fix.
- **Jan (Menlo Research Pte Ltd)**: Stdio MCP servers with per-call approval and saved permissions; local models need tool calling enabled per model. An open issue on 0.8.8 shows a truncated tool call saved without arguments breaking the whole thread.
- **Lemonade (Advanced Micro Devices, Inc.)**: Chat completions return tool_calls and document streamed reassembly; tool_choice objects are treated as auto on llama.cpp models. The MCP endpoint exposes Lemonade as a server, not a client. See the 33k crash issue.
- **llama.cpp (ggml.ai, part of Hugging Face since 2026)**: Function calling has native and generic parsers, parallel_tool_calls off by default, and stdio MCP servers. One issue shows a tool call returned inside reasoning_content on Vulkan until a build fix.
- **LM Studio (Element Labs, Inc.)**: Tool use docs distinguish native and default support and warn that small models produce calls LM Studio cannot parse into tool_calls. MCP via API takes ephemeral servers or mcp.json with allowed_tools.
- **MLX LM (Apple Inc.)**: server.py builds tool_calls from the model's tool-calling template when the tokenizer supports it, but the documented request fields do not include tools, and SERVER.md calls the server unfit for production.
- **Ollama (Ollama Inc.)**: Tool calling returns structured tool_calls, documents parallel calls and streaming reassembly. An issue shows qwen2.5-coder:7b writing the call as text and qwen3 calling a missing tool; the maintainer advised a better tool-use model.
- **Open WebUI (Open WebUI Inc.)**: Native MCP since 0.6.31 is Streamable HTTP only with OAuth 2.1, admin-added and scoped by group; stdio needs mcpo. Accounts use web search, code calling and mcpo tools with local 27B models.

## Couldn't check

The docs were closed to us, or silent. These carry no mark either way.

- **Core (ZMJ, Inc., doing business as Ghost, 325 9th Street, San Francisco, CA 94103, United States)**: No API or tool documentation exists on ghost.ai; the setup guide covers pairing and Wi-Fi only.
- **Underdog (Conway Research)**: The model card says nothing about tool calling and underdog.ai refuses Claude-named crawlers in robots.txt, so the app docs were not read.

- not this job: BitRun (Abhishek Khanra (IshekKhal on GitHub)): No tools parameter documented; research checkpoints the README says to check before relying on.
- not this job: Docker Model Runner (Docker, Inc.): The maintainer states one tool call per turn is an upstream grammar limit and two calls return a 500, so the two-calls-in-one-turn criterion cannot be met.
- not this job: GPT4All (Nomic, Inc.): No tools parameter on the API. An account reports JSON errors with remote models and the project asks whether development has stopped.
- not this job: Khoj (Khoj Inc.): The functions list has built-in online search, code execution and image tools inside Khoj; no MCP or custom tool connection appears in the docs navigation.
- not this job: LocalGhost (LocalGhost.ai Ltd (company number 17213100, registered in England and Wales, incorporated 12 May 2026)): No agent surface.
- not this job: screenpipe (Negentropy Labs, Inc. (dba Screenpipe)): An MCP server, not a client or runner.

## What stops agents finishing

- server returns 500 when the model makes two tool calls in one turn
- a truncated tool call is saved without arguments and every later message in the thread fails
- tool calling works on one hardware backend (CUDA) and not another (NPU)
- weak tool-use models such as 7B coder variants write the call as JSON text and the client never sees it

## Around it

Before: decision-models.pick-tool-or-skill. After: agent-memory.store-and-confirm, search.fetch-url.

## Where this comes from

Weight 15 of 100 in Local AI: 11 of 50 accounts, most of them GitHub issues where the failure is the use case: tool calls parsed wrongly, dropped, or breaking the thread

As data: https://www.anchorterminal.com/api/v1/use-cases/local-ai.agent-tool-loop.json, or the MCP tool get_use_case with id local-ai.agent-tool-loop. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp
