{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/local-ai.agent-tool-loop/",
    "json": "https://www.anchorterminal.com/niches/local-ai.agent-tool-loop/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/local-ai.agent-tool-loop/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/local-ai.agent-tool-loop/index.min.md"
  },
  "markdown": "Four or more companies' docs show a full fit.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 5 | 11 | 101 | 15 | 11 |\n\n## What counts as done\n\nThe runner returns tool calls in the structured tool_calls field and handles two or more calls in one turn.\n\nMeasured, for our own tests: across ten turns every tool call comes back in that field with arguments that validate against the schema, and the conversation is never left in a state the client cannot continue.\n\nHow agents ask for it: let my local model call a calculator tool; connect an MCP server to a local model; build a private assistant that runs Python in a sandbox; give the model file tools so it can fix formatting in my headers\n\n## Does the job\n\nThe docs show it meets the test above, and nothing we read says otherwise.\n\n- **KoboldCpp (LostRuins (Concedo))**: MCP tool calling since 1.106 takes a Claude Desktop mcp.json with HTTP and STDIO servers, tool toggles and optional manual approval. An account had Devstral reformat headers through an MCP file-tools server.\n- **LocalAI (Ettore Di Giacinto and the LocalAI team)**: Functions map to grammars on llama.cpp with a parallel tool calls section and streamed reasoning separated from tool output. MCP connects HTTP and stdio servers with server-side execution limits.\n- **PrivateGPT (PriBAI Technology Spain S.L.)**: Built-in server tools, custom tools returned as Anthropic tool_use blocks, client tools (bash, text editor) and MCP connectors are documented with request shapes.\n- **TextGen (oobabooga)**: Tool calls come back as finish_reason tool_calls with structured names and arguments; Qwen and Mistral call several in parallel. MCP servers over HTTP and stdio (mcp.json) are documented in the tutorial.\n- **vLLM (The Linux Foundation (vLLM is a PyTorch Foundation project))**: Tool parsers per model family, named and required tool choice guaranteed parsable, parallel_tool_calls true by default, streaming parsers, and a Responses example with MCP tools.\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **AnythingLLM (Mintplex Labs, Inc.)**: Agents take MCP servers over StdIO, SSE and Streamable HTTP on Docker and Desktop. An open issue shows the schema helper dropping a parameter named definitions or $defs while leaving it required, so the model guesses its shape.\n- **Berth (Trevin Tindall (T-Crypt on GitHub))**: Tool calls pass through to the engine untouched; Berth adds queuing and cold starts of up to minutes, with 503 and Retry-After while an engine loads.\n- **Foundry Local (Microsoft Corporation)**: Tool support differs by hardware variant: on a Snapdragon X2 NPU the model answered in plain text with no tool call and ToolChoice Required failed everywhere, with no fix at reading.\n- **HyperQwen (syv.ai (Copenhagen, CVR DK44671247))**: Tool calls go through vLLM's parser with a strict mode; the README notes a tool schema's maxLength drops decode to about 7 tokens a second without the bundled xgrammar fix.\n- **Jan (Menlo Research Pte Ltd)**: Stdio MCP servers with per-call approval and saved permissions; local models need tool calling enabled per model. An open issue on 0.8.8 shows a truncated tool call saved without arguments breaking the whole thread.\n- **Lemonade (Advanced Micro Devices, Inc.)**: Chat completions return tool_calls and document streamed reassembly; tool_choice objects are treated as auto on llama.cpp models. The MCP endpoint exposes Lemonade as a server, not a client. See the 33k crash issue.\n- **llama.cpp (ggml.ai, part of Hugging Face since 2026)**: Function calling has native and generic parsers, parallel_tool_calls off by default, and stdio MCP servers. One issue shows a tool call returned inside reasoning_content on Vulkan until a build fix.\n- **LM Studio (Element Labs, Inc.)**: Tool use docs distinguish native and default support and warn that small models produce calls LM Studio cannot parse into tool_calls. MCP via API takes ephemeral servers or mcp.json with allowed_tools.\n- **MLX LM (Apple Inc.)**: server.py builds tool_calls from the model's tool-calling template when the tokenizer supports it, but the documented request fields do not include tools, and SERVER.md calls the server unfit for production.\n- **Ollama (Ollama Inc.)**: Tool calling returns structured tool_calls, documents parallel calls and streaming reassembly. An issue shows qwen2.5-coder:7b writing the call as text and qwen3 calling a missing tool; the maintainer advised a better tool-use model.\n- **Open WebUI (Open WebUI Inc.)**: Native MCP since 0.6.31 is Streamable HTTP only with OAuth 2.1, admin-added and scoped by group; stdio needs mcpo. Accounts use web search, code calling and mcpo tools with local 27B models.\n\n## Couldn't check\n\nThe docs were closed to us, or silent. These carry no mark either way.\n\n- **Core (ZMJ, Inc., doing business as Ghost, 325 9th Street, San Francisco, CA 94103, United States)**: No API or tool documentation exists on ghost.ai; the setup guide covers pairing and Wi-Fi only.\n- **Underdog (Conway Research)**: The model card says nothing about tool calling and underdog.ai refuses Claude-named crawlers in robots.txt, so the app docs were not read.\n\n- not this job: BitRun (Abhishek Khanra (IshekKhal on GitHub)): No tools parameter documented; research checkpoints the README says to check before relying on.\n- not this job: Docker Model Runner (Docker, Inc.): The maintainer states one tool call per turn is an upstream grammar limit and two calls return a 500, so the two-calls-in-one-turn criterion cannot be met.\n- not this job: GPT4All (Nomic, Inc.): No tools parameter on the API. An account reports JSON errors with remote models and the project asks whether development has stopped.\n- not this job: Khoj (Khoj Inc.): The functions list has built-in online search, code execution and image tools inside Khoj; no MCP or custom tool connection appears in the docs navigation.\n- not this job: LocalGhost (LocalGhost.ai Ltd (company number 17213100, registered in England and Wales, incorporated 12 May 2026)): No agent surface.\n- not this job: screenpipe (Negentropy Labs, Inc. (dba Screenpipe)): An MCP server, not a client or runner.\n\n## What stops agents finishing\n\n- server returns 500 when the model makes two tool calls in one turn\n- a truncated tool call is saved without arguments and every later message in the thread fails\n- tool calling works on one hardware backend (CUDA) and not another (NPU)\n- weak tool-use models such as 7B coder variants write the call as JSON text and the client never sees it\n\n## Around it\n\nBefore: decision-models.pick-tool-or-skill. After: agent-memory.store-and-confirm, search.fetch-url.\n\n## Where this comes from\n\nWeight 15 of 100 in Local AI: 11 of 50 accounts, most of them GitHub issues where the failure is the use case: tool calls parsed wrongly, dropped, or breaking the thread\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/local-ai.agent-tool-loop.json, or the MCP tool get_use_case with id local-ai.agent-tool-loop. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Run a non-coding agent with tools or MCP servers against a local model",
        "url": ""
      }
    ],
    "description": "Run a non-coding agent with tools or MCP servers against a local model: 5 companies fully do it, 11 with a limit, 101 carry the tag. Every listing in Local AI with the reason and a source.",
    "facts": null,
    "h1": "Run a non-coding agent with tools or MCP servers against a local model",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/local-ai.agent-tool-loop/",
    "published": "",
    "section": "agents",
    "title": "Run a non-coding agent with tools or MCP servers against a local…",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/local-ai.agent-tool-loop/"
  },
  "tokens": {
    "markdown": 2000,
    "slim": 430
  },
  "version": 1
}
