Niches · Local AI

Run a non-coding agent with tools or MCP servers against a local model

Four or more companies' docs show a full fit.

5companies, full fit
11companies, fit with a limit
101companies carry the tag
15share of the category's work
11first-hand accounts

What counts as done

The runner returns tool calls in the structured tool_calls field and handles two or more calls in one turn.

Measured, for our own tests: across ten turns every tool call comes back in that field with arguments that validate against the schema, and the conversation is never left in a state the client cannot continue. No docs we read state these figures, so they don't count against a listing here.

How agents ask for it: let my local model call a calculator tool, connect an MCP server to a local model, build a private assistant that runs Python in a sandbox, give the model file tools so it can fix formatting in my headers

Does the job 5

The docs show it meets the test above, and nothing we read says otherwise.

ListingWhy
KoboldCpp B
LostRuins (Concedo)
MCP tool calling since 1.106 takes a Claude Desktop mcp.json with HTTP and STDIO servers, tool toggles and optional manual approval. An account had Devstral reformat headers through an MCP file-tools server. source
LocalAI B
Ettore Di Giacinto and the LocalAI team
Functions map to grammars on llama.cpp with a parallel tool calls section and streamed reasoning separated from tool output. MCP connects HTTP and stdio servers with server-side execution limits. source
PrivateGPT C
PriBAI Technology Spain S.L.
Built-in server tools, custom tools returned as Anthropic tool_use blocks, client tools (bash, text editor) and MCP connectors are documented with request shapes. source
TextGen E
oobabooga
Tool calls come back as finish_reason tool_calls with structured names and arguments; Qwen and Mistral call several in parallel. MCP servers over HTTP and stdio (mcp.json) are documented in the tutorial. source
vLLM D
The Linux Foundation (vLLM is a PyTorch Foundation project)
Tool parsers per model family, named and required tool choice guaranteed parsable, parallel_tool_calls true by default, streaming parsers, and a Responses example with MCP tools. source

Does the job, with a limit 11

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
AnythingLLM E
Mintplex Labs, Inc.
Agents take MCP servers over StdIO, SSE and Streamable HTTP on Docker and Desktop. An open issue shows the schema helper dropping a parameter named definitions or $defs while leaving it required, so the model guesses its shape. source
Berth B
Trevin Tindall (T-Crypt on GitHub)
Tool calls pass through to the engine untouched; Berth adds queuing and cold starts of up to minutes, with 503 and Retry-After while an engine loads. source
Foundry Local B
Microsoft Corporation
Tool support differs by hardware variant: on a Snapdragon X2 NPU the model answered in plain text with no tool call and ToolChoice Required failed everywhere, with no fix at reading. source
HyperQwen C
syv.ai (Copenhagen, CVR DK44671247)
Tool calls go through vLLM's parser with a strict mode; the README notes a tool schema's maxLength drops decode to about 7 tokens a second without the bundled xgrammar fix. source
Jan C
Menlo Research Pte Ltd
Stdio MCP servers with per-call approval and saved permissions; local models need tool calling enabled per model. An open issue on 0.8.8 shows a truncated tool call saved without arguments breaking the whole thread. source
Lemonade B
Advanced Micro Devices, Inc.
Chat completions return tool_calls and document streamed reassembly; tool_choice objects are treated as auto on llama.cpp models. The MCP endpoint exposes Lemonade as a server, not a client. See the 33k crash issue. source
llama.cpp D
ggml.ai, part of Hugging Face since 2026
Function calling has native and generic parsers, parallel_tool_calls off by default, and stdio MCP servers. One issue shows a tool call returned inside reasoning_content on Vulkan until a build fix. source
LM Studio C
Element Labs, Inc.
Tool use docs distinguish native and default support and warn that small models produce calls LM Studio cannot parse into tool_calls. MCP via API takes ephemeral servers or mcp.json with allowed_tools. source
MLX LM C
Apple Inc.
server.py builds tool_calls from the model's tool-calling template when the tokenizer supports it, but the documented request fields do not include tools, and SERVER.md calls the server unfit for production. source
Ollama D
Ollama Inc.
Tool calling returns structured tool_calls, documents parallel calls and streaming reassembly. An issue shows qwen2.5-coder:7b writing the call as text and qwen3 calling a missing tool; the maintainer advised a better tool-use model. source
Open WebUI D
Open WebUI Inc.
Native MCP since 0.6.31 is Streamable HTTP only with OAuth 2.1, admin-added and scoped by group; stdio needs mcpo. Accounts use web search, code calling and mcpo tools with local 27B models. source

Couldn't check 2

The docs were closed to us, or silent. These carry no mark either way.

ListingWhy
Core F
ZMJ, Inc., doing business as Ghost, 325 9th Street, San Francisco, CA 94103, United States
No API or tool documentation exists on ghost.ai; the setup guide covers pairing and Wi-Fi only.
Underdog F
Conway Research
The model card says nothing about tool calling and underdog.ai refuses Claude-named crawlers in robots.txt, so the app docs were not read.

Doesn't do this job 6

Listings in the category that do something else, with the reason.

ListingWhy
BitRun E
Abhishek Khanra (IshekKhal on GitHub)
No tools parameter documented; research checkpoints the README says to check before relying on. source
Docker Model Runner C
Docker, Inc.
The maintainer states one tool call per turn is an upstream grammar limit and two calls return a 500, so the two-calls-in-one-turn criterion cannot be met. source
GPT4All F
Nomic, Inc.
No tools parameter on the API. An account reports JSON errors with remote models and the project asks whether development has stopped. source
Khoj E
Khoj Inc.
The functions list has built-in online search, code execution and image tools inside Khoj; no MCP or custom tool connection appears in the docs navigation. source
LocalGhost F
LocalGhost.ai Ltd (company number 17213100, registered in England and Wales, incorporated 12 May 2026)
No agent surface. source
screenpipe C
Negentropy Labs, Inc. (dba Screenpipe)
An MCP server, not a client or runner. source

What stops agents finishing

  • server returns 500 when the model makes two tool calls in one turn
  • a truncated tool call is saved without arguments and every later message in the thread fails
  • tool calling works on one hardware backend (CUDA) and not another (NPU)
  • weak tool-use models such as 7B coder variants write the call as JSON text and the client never sees it

Around it

Before: decision-models.pick-tool-or-skill. After: agent-memory.store-and-confirm, search.fetch-url.

Where this comes from

Weight 15 of 100 in Local AI: 11 of 50 accounts, most of them GitHub issues where the failure is the use case: tool calls parsed wrongly, dropped, or breaking the thread

The same as data: /api/v1/use-cases/local-ai.agent-tool-loop.json, or the MCP tool get_use_case with id local-ai.agent-tool-loop. Other jobs in this category: /api/v1/niches?category=local-ai.

Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.