Run a non-coding agent with tools or MCP servers against a local model
Four or more companies' docs show a full fit.
What counts as done
The runner returns tool calls in the structured tool_calls field and handles two or more calls in one turn.
Measured, for our own tests: across ten turns every tool call comes back in that field with arguments that validate against the schema, and the conversation is never left in a state the client cannot continue. No docs we read state these figures, so they don't count against a listing here.
How agents ask for it: let my local model call a calculator tool
, connect an MCP server to a local model
, build a private assistant that runs Python in a sandbox
, give the model file tools so it can fix formatting in my headers
Does the job 5
The docs show it meets the test above, and nothing we read says otherwise.
| Listing | Why |
|---|---|
| KoboldCpp B LostRuins (Concedo) | MCP tool calling since 1.106 takes a Claude Desktop mcp.json with HTTP and STDIO servers, tool toggles and optional manual approval. An account had Devstral reformat headers through an MCP file-tools server. source |
| LocalAI B Ettore Di Giacinto and the LocalAI team | Functions map to grammars on llama.cpp with a parallel tool calls section and streamed reasoning separated from tool output. MCP connects HTTP and stdio servers with server-side execution limits. source |
| PrivateGPT C PriBAI Technology Spain S.L. | Built-in server tools, custom tools returned as Anthropic tool_use blocks, client tools (bash, text editor) and MCP connectors are documented with request shapes. source |
| TextGen E oobabooga | Tool calls come back as finish_reason tool_calls with structured names and arguments; Qwen and Mistral call several in parallel. MCP servers over HTTP and stdio (mcp.json) are documented in the tutorial. source |
| vLLM D The Linux Foundation (vLLM is a PyTorch Foundation project) | Tool parsers per model family, named and required tool choice guaranteed parsable, parallel_tool_calls true by default, streaming parsers, and a Responses example with MCP tools. source |
Does the job, with a limit 11
It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.
| Listing | Why |
|---|---|
| AnythingLLM E Mintplex Labs, Inc. | Agents take MCP servers over StdIO, SSE and Streamable HTTP on Docker and Desktop. An open issue shows the schema helper dropping a parameter named definitions or $defs while leaving it required, so the model guesses its shape. source |
| Berth B Trevin Tindall (T-Crypt on GitHub) | Tool calls pass through to the engine untouched; Berth adds queuing and cold starts of up to minutes, with 503 and Retry-After while an engine loads. source |
| Foundry Local B Microsoft Corporation | Tool support differs by hardware variant: on a Snapdragon X2 NPU the model answered in plain text with no tool call and ToolChoice Required failed everywhere, with no fix at reading. source |
| HyperQwen C syv.ai (Copenhagen, CVR DK44671247) | Tool calls go through vLLM's parser with a strict mode; the README notes a tool schema's maxLength drops decode to about 7 tokens a second without the bundled xgrammar fix. source |
| Jan C Menlo Research Pte Ltd | Stdio MCP servers with per-call approval and saved permissions; local models need tool calling enabled per model. An open issue on 0.8.8 shows a truncated tool call saved without arguments breaking the whole thread. source |
| Lemonade B Advanced Micro Devices, Inc. | Chat completions return tool_calls and document streamed reassembly; tool_choice objects are treated as auto on llama.cpp models. The MCP endpoint exposes Lemonade as a server, not a client. See the 33k crash issue. source |
| llama.cpp D ggml.ai, part of Hugging Face since 2026 | Function calling has native and generic parsers, parallel_tool_calls off by default, and stdio MCP servers. One issue shows a tool call returned inside reasoning_content on Vulkan until a build fix. source |
| LM Studio C Element Labs, Inc. | Tool use docs distinguish native and default support and warn that small models produce calls LM Studio cannot parse into tool_calls. MCP via API takes ephemeral servers or mcp.json with allowed_tools. source |
| MLX LM C Apple Inc. | server.py builds tool_calls from the model's tool-calling template when the tokenizer supports it, but the documented request fields do not include tools, and SERVER.md calls the server unfit for production. source |
| Ollama D Ollama Inc. | Tool calling returns structured tool_calls, documents parallel calls and streaming reassembly. An issue shows qwen2.5-coder:7b writing the call as text and qwen3 calling a missing tool; the maintainer advised a better tool-use model. source |
| Open WebUI D Open WebUI Inc. | Native MCP since 0.6.31 is Streamable HTTP only with OAuth 2.1, admin-added and scoped by group; stdio needs mcpo. Accounts use web search, code calling and mcpo tools with local 27B models. source |
Couldn't check 2
The docs were closed to us, or silent. These carry no mark either way.
| Listing | Why |
|---|---|
| Core F ZMJ, Inc., doing business as Ghost, 325 9th Street, San Francisco, CA 94103, United States | No API or tool documentation exists on ghost.ai; the setup guide covers pairing and Wi-Fi only. |
| Underdog F Conway Research | The model card says nothing about tool calling and underdog.ai refuses Claude-named crawlers in robots.txt, so the app docs were not read. |
Doesn't do this job 6
Listings in the category that do something else, with the reason.
| Listing | Why |
|---|---|
| BitRun E Abhishek Khanra (IshekKhal on GitHub) | No tools parameter documented; research checkpoints the README says to check before relying on. source |
| Docker Model Runner C Docker, Inc. | The maintainer states one tool call per turn is an upstream grammar limit and two calls return a 500, so the two-calls-in-one-turn criterion cannot be met. source |
| GPT4All F Nomic, Inc. | No tools parameter on the API. An account reports JSON errors with remote models and the project asks whether development has stopped. source |
| Khoj E Khoj Inc. | The functions list has built-in online search, code execution and image tools inside Khoj; no MCP or custom tool connection appears in the docs navigation. source |
| LocalGhost F LocalGhost.ai Ltd (company number 17213100, registered in England and Wales, incorporated 12 May 2026) | No agent surface. source |
| screenpipe C Negentropy Labs, Inc. (dba Screenpipe) | An MCP server, not a client or runner. source |
What stops agents finishing
- server returns 500 when the model makes two tool calls in one turn
- a truncated tool call is saved without arguments and every later message in the thread fails
- tool calling works on one hardware backend (CUDA) and not another (NPU)
- weak tool-use models such as 7B coder variants write the call as JSON text and the client never sees it
Around it
Before: decision-models.pick-tool-or-skill. After: agent-memory.store-and-confirm, search.fetch-url.
Where this comes from
Weight 15 of 100 in Local AI: 11 of 50 accounts, most of them GitHub issues where the failure is the use case: tool calls parsed wrongly, dropped, or breaking the thread
The same as data: /api/v1/use-cases/local-ai.agent-tool-loop.json, or the MCP tool get_use_case with id local-ai.agent-tool-loop. Other jobs in this category: /api/v1/niches?category=local-ai.
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.