# A local assistant that calls tools in under 3 seconds

> A local assistant that calls tools in under 3 seconds: 2 companies fully do it, 16 with a limit, 90 carry the tag. Every listing in Open-weight models with the reason and a source.

- Canonical: https://www.anchorterminal.com/niches/open-weights.tool-calling-assistant/
- Markdown: https://www.anchorterminal.com/niches/open-weights.tool-calling-assistant/index.md (~5,200 tokens)
- Slim: https://www.anchorterminal.com/niches/open-weights.tool-calling-assistant/index.min.md (~930 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/niches/open-weights.tool-calling-assistant/index.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-15

Two or three companies' docs show a full fit.

| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |
| --- | --- | --- | --- | --- |
| 2 | 16 | 90 | 10 | 7 |

## What counts as done

The model returns tool calls with JSON arguments, running on an 8 GB GPU.

Measured, for our own tests: at least 95 percent of 30 test commands produce the right tool call with valid JSON arguments, median latency under 3 seconds on an 8 GB GPU, and zero empty replies.

How agents ask for it: run Home Assistant Assist on a local model; a voice assistant that turns the lights off; route a spoken question to the right tool; an 8 GB GPU model that never misses a tool call

## Does the job

The docs show it meets the test above, and nothing we read says otherwise.

- **Granite 4.2 3B (International Business Machines Corporation (per IBM's naming guidance, as read for granite-guardian))**: Tool calling with OpenAI schemas is documented, enable_thinking false gives direct answers, the Q4_K_M is 2.2 GB on Ollama, and acc-26 measured 100 percent routing at 0.54 seconds mean on a 16 GB GPU.
- **Qwen3.5-0.8B (Alibaba Cloud (named in the licence's copyright line))**: Card names qwen3_coder for vLLM and SGLang and Ollama declares tools, but nothing measures accuracy on 10 to 60 tools at this size; only the template shows the format.
- **Qwen3.5-2B (Alibaba Cloud (named in the licence's copyright line))**: Card names qwen3_coder for vLLM and SGLang and Ollama declares tools, but nothing measures accuracy on 10 to 60 tools at this size; only the template shows the format.

## Does the job, with a limit

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

- **Agents-A1-4B (Shanghai Artificial Intelligence Laboratory (the InternAgent team there, per the card's recommended system prompt and the template's default prompt); the GitHub LICENSE leaves the copyright line as the Apache template)**: Native tool calling with the qwen3_coder parser and a 2.7 GB GGUF fit an 8 GB GPU, but thinking defaults on, no Ollama tag exists and no test of 30 commands was found.
- **Gemma 4 12B IT (Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author))**: acc-24 ran Assist on an RTX 3060 Ti 8 GB: 100 percent tool-call success but 3.6 seconds median, above the 3 second target; the 3-bit quant returned empty 9 of 33 times, IQ4_XS none.
- **Gemma 4 26B A4B IT (Google LLC (RDAP registrant of gemma.ai; the card names Google DeepMind as author))**: Google's Q4_0 is 14.4 GB and Ollama's 16 GB, which does not fit an 8 GB GPU and leaves no context on 16 GB; acc-24 chose the 12B for this reason. Tools badge present.
- **Gemma 4 E2B IT (Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author))**: Fits any 8 GB GPU with Ollama tools support, but no 30-command test exists for E2B and acc-45 rates it well below E4B on every task.
- **Gemma 4 E4B IT (Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author))**: Fits an 8 GB GPU with Ollama tools support and acc-45 passed a six-step agent task, but no 30-command tool test exists for E4B and acc-26 found some Gemma builds without tool templates.
- **GLM-4.7-Flash (JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai))**: acc-23 ran Home Assistant Assist on GLM-4.7 Flash through Ollama and it worked where Qwen failed; the 19 GB Ollama build exceeds an 8 to 16 GB GPU without offload, and Unsloth's Q3_K_XL is 13.8 GB.
- **gpt-oss-20b (OpenAI (named in USAGE_POLICY and the model card; the LICENSE file's copyright line is the unfilled Apache template))**: Fits a 16 GB GPU (acc-47 on an RTX 4080) but acc-47 found tool calling unreliable and long thinking too slow for Home Assistant; no off switch.
- **LFM2.5-2.6B (Liquid AI, Inc. (the licensor named in the LFM Open License))**: Tool use is documented and 1.6 GB fits an 8 GB GPU, but calls come back as Pythonic lists between special tokens, thinking cannot be turned off, and a discussion calls the tool-call tokens the model's biggest problem.
- **Ling-3.0-tiny (Ant Intelligence (Hang Zhou) Technology Co., Ltd (named as provider in the training-content summary); inclusionAI, Ant Group in the site footer of inclusion-ai.org)**: Fits 8 GB with ling3 parsers in vLLM and llama.cpp --jinja, but there is no Ollama library entry, only a manual import, and looping is reported.
- **MiMo-V2.6 Distill Qwen 9B (Xiaomi (the card and report name Xiaomi MiMo; no LICENSE file names a copyright holder))**: Fits 8 GB through Ollama's tools-badged tag, but tool-call reliability on llama.cpp is disputed in discussions and the card documents only SGLang's mimo reasoning parser.
- **MiniCPM-o 4.5 (OpenBMB (named as the MiniCPM-o team on the technical report; the GitHub LICENSE is the unfilled Apache template))**: The template renders tool_call tags but the card never documents tool calling or names a parser; the Ollama build shows no tools badge.
- **MiniCPM5-2B (OpenBMB (named in the GitHub LICENSE's copyright line))**: vLLM 0.29 and SGLang ship a minicpm5 parser for its XML calls and 1.6 GB fits any GPU, but thinking precedes each call, the Ollama page shows no tools badge, and discussions report a template crash on string arguments.
- **NVIDIA Nemotron 3.5 Bolt E4B 4GB (NVIDIA Corporation)**: Built for voice assistants: fits 8 GB, thinking off, gemma4 parser, BFCL-v3 67.3 at Q4_K_M (NVIDIA's run). No account runs 30 commands, and Ollama's e4b tag advertises 128K against NVIDIA's 64K.
- **Nemotron 3 Nano 4B (NVIDIA Corporation)**: Fits an 8 GB GPU with thinking off, qwen3_coder parser in vLLM and an Ollama tools badge, but BFCL v3 is 61.1 on the card and no account measures 30 commands.
- **Ornith-1.5-9B (Ornith)**: Fits an 8 GB GPU and Ollama's 9b tag has a tools badge, but thinking is on by default and lengthy, so the 3-second median needs the undocumented template switch.
- **Qwen3-30B-A3B-Instruct-2507 (Alibaba Cloud (named in the licence's copyright line))**: Non-thinking with tools in Ollama, but 19 GB needs Unsloth's expert offload to CPU to run on an 8 GB to 16 GB GPU, and no account measures latency or 30 commands.
- **Qwen3-4B-Instruct-2507 (Alibaba Cloud (named in the licence's copyright line))**: Tools are in the template and the Ollama library declares tools, but no parser is named on the card and no account measures 30 commands on an 8 GB GPU.
- **Qwen3.5-4B (Alibaba Cloud (named in the licence's copyright line))**: Parsers named and Ollama declares tools, thinking off per request; no account measures 30 commands on this size, and acc-28 shows an Ollama regression printing calls as text on the family.
- **Qwen3.5-9B (Alibaba Cloud (named in the licence's copyright line))**: acc-24 ran qwen3.5:9b in Home Assistant on an 8 GB GPU at 2.9 s median, but gives no success rate for it, and acc-26 found routing slow; parsers are named on the card.
- **Qwen3-Coder-30B-A3B-Instruct (Alibaba Cloud (named in the licence's copyright line))**: No thinking and a named parser, but Q4_K_M is 18.6 GB, so an 8 to 16 GB GPU offloads experts; no voice-pipeline account exists.
- **Qwen3-VL-2B-Instruct (Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder))**: Tools in the template and Ollama declares tools, no thinking to block, 1.1 GB at Q4; the card names no parser and no account measures 30 commands.
- **Qwen3-VL-4B-Instruct (Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder))**: Tools in the template and Ollama declares tools, no thinking to block, 2.5 GB at Q4; the card names no parser and no account measures 30 commands.
- **Qwen3-VL-8B-Instruct (Alibaba Cloud (per the pilot's RDAP check of qwen.ai; this model's files name no copyright holder))**: Tools in the template and Ollama declares tools, no thinking to block, 5.0 GB at Q4; the card names no parser and no account measures 30 commands.
- **SmolLM3-3B (Hugging Face, Inc.)**: Card names vLLM's hermes parser and thinking off, 1.9 GB fits any GPU; no account measures 30 commands, and the template renders tool results as user turns.
- **Spark-X2.6-4B (XHToken (the licence's copyright line); the Hugging Face organisation's name is SparkLLM)**: Aimed at agents with a named SGLang parser and thinking off per request on a 3.5 GB build, but the Ollama entry shows no tools badge and no account measures 30 commands.

- not this job: Apertus v1.5 8B (ETH Zurich and EPFL (the Swiss National AI Institute, named in the Acceptable Use Policy)): Nothing runs it on an Ollama or llama.cpp server: the architecture is not in llama.cpp and Ollama has no library entry.
- not this job: Apodex-1.1-mini (Apodex (the site's copyright line reads 2026 Apodex)): The smallest publisher build is 21.7 GB, beyond an 8 to 16 GB GPU, and there is no Ollama tag.
- not this job: BigBang-v1 (The Endless Frontier (endless-frontier)): The smallest build anywhere is 10 GB at IQ2_XXS from a third party and 21.9 GB at Q4; no Ollama entry and the card targets 8-GPU serving.
- not this job: Carbon-X (Hugging Face (the HuggingFaceBio organisation, named Hugging Face Biology Research on the Hub)): No tool-calling training documented; the template inherits Qwen3's tool format but the card never uses it.
- not this job: Cosmos-Reason2-2B (NVIDIA Corporation): No tool-calling documentation; the inherited Qwen3-VL template carries tools but NVIDIA never exercises it.
- not this job: Darwin-36B-Opus (VIDRAFT Inc. (named on vidraft.net); the card's citation names FINAL-Bench and VIDRAFT_LAB): The Q4_K_M build is 21.2 GB; only the 1 to 3 bit POCKET files fit 16 GB, there is no Ollama entry and tool calling is undocumented.
- not this job: DeepSeek-V3.2 (DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)): Cannot load on an 8 to 16 GB GPU; Ollama retired its deepseek-v3.2 entry on 15 July 2026.
- not this job: DeepSeek V4.1 Flash (DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)): Cannot load on an 8 to 16 GB GPU; Ollama's entry is cloud-only.
- not this job: DeepSeek-V4-Flash-0731 (DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)): Cannot load on an 8 to 16 GB GPU; Ollama retired deepseek-v4-flash on 27 August 2026.
- not this job: DiffusionGemma 26B A4B IT (Google LLC (RDAP registrant of gemma.ai; the card names Google DeepMind as author)): Nothing serves it on Ollama or released llama.cpp; the 16.8 GB third-party Q4 needs an unmerged branch.
- not this job: Gemma 4 31B IT (Google LLC (the registrant of gemma.ai; the card names Google DeepMind as author)): 17.6 GB at Q4_0 exceeds an 8 to 16 GB GPU.
- not this job: GLM-4.5-Air (JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai, per the first batch's check on 21 October 2026)): No build fits an 8 to 16 GB GPU and Ollama has no entry.
- not this job: GLM-5.3 (JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai). The licence names Z.AI): Cannot load on an 8 to 16 GB GPU; no Ollama entry.
- not this job: GLM-5.3-Flash (Z.AI Co., Ltd (named in the licence's copyright line)): Cannot load on an 8 to 16 GB GPU; Ollama's entry is cloud-only.
- not this job: gpt-oss-120b (OpenAI (named as publisher on the card; the openai-api listing's terms name OpenAI OpCo, LLC)): Cannot load on an 8 to 16 GB GPU.
- not this job: Hy3 (Tencent (named in the LICENSE file's copyright line)): Cannot load on an 8 to 16 GB GPU; no Ollama entry.
- not this job: Inkling (Thinking Machines Lab, Inc. (named on its model card page and in its training-content summary)): Cannot load on an 8 to 16 GB GPU.
- not this job: Inkling-Small (Thinking Machines Lab, Inc. (named on its model card page and in its training-content summary)): Cannot load on an 8 to 16 GB GPU; no Ollama entry.
- not this job: InternVL3.5-GPT-OSS-20B-A4B-Preview (Shanghai AI Laboratory (OpenGVLab), not named in the repository): No tool calling in the template or card.
- not this job: Kimi K3 (Moonshot AI (the licence names Moonshot AI; the Kimi Platform terms name Moonshot AI PTE. LTD.)): Cannot load locally on a small GPU; Ollama's kimi-k3 is cloud-only.
- not this job: Kolibri-1 (Aleph Alpha GmbH): No build fits an 8 to 16 GB GPU and Ollama has no entry.
- not this job: Kronos (No company. Yu Shi publishes it from the GitHub account shiyu-coder and the Hugging Face user account NeoQuasar (named ShiYu), with co-authors at Tsinghua University per the paper): No tool calling; the model takes OHLCV bars and returns forecast bars.
- not this job: Laguna M.1 (Poolside, Inc. (named in Poolside's Terms of Use; the LICENSE.md copyright line reads Poolside)): No build fits an 8 GB to 16 GB GPU.
- not this job: Laguna S 2.1 (Poolside, Inc. (named in Poolside's Terms of Use)): No build fits an 8 GB to 16 GB GPU.
- not this job: Laguna XS 2.1 (Poolside, Inc. (named in Poolside's Terms of Use)): The only quantised builds are 20 GB Q4_K_M and larger; nothing fits an 8 GB to 16 GB GPU.
- not this job: LBS-27B (inclusionAI, Ant Group (named in the site footer of inclusion-ai.org); the work is credited to authors from Wuhan University of Technology, CASIA, Zhejiang University and Ant Group): No tool calling; actions are plain-text crop instructions parsed by inclusionAI's Python.
- not this job: MiniMax-M2.5 (MiniMax AI (named in the licences; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)): No build fits an 8 GB to 16 GB GPU.
- not this job: MiniMax-M2.7 (MiniMax (named in the licence's copyright line; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)): No build fits an 8 GB to 16 GB GPU, and every reply opens a think block.
- not this job: MiniMax-M3.1-Flash (MiniMax (named in the licence's copyright line; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)): No build fits an 8 GB to 16 GB GPU, and thinking is adaptive with no off switch in the template.
- not this job: Mistral Large 4 (Mistral AI (RCS Paris 952 418 325)): No build fits an 8 GB to 16 GB GPU, and reasoning_effort has no off value.
- not this job: Muse Glimmer 30B (Meta (the verified Hugging Face organisation is "Meta Inc.", the card names Meta Superintelligence Lab, and the licence carries no copyright line)): Smallest build is 16.8 GB (18 GB in Ollama), over an 8 GB GPU, and thinking cannot be turned off.
- not this job: Nanbeige4.3-33B (BOSS Zhipin (Kanzhun Limited), unconfirmed): Smallest GGUF is 10.1 GB at IQ2_XXS; nothing at a usable bit width fits an 8 GB GPU, and there is no Ollama library entry.
- not this job: Nemotron 3.5 Lightning 30B A3B (NVIDIA Corporation): Smallest build is 18.9 GB (Ollama 19 GB), over an 8 GB to 16 GB GPU.
- not this job: Nemotron 3 Nano 30B A3B (NVIDIA Corporation): Smallest Ollama build is 18 GB, over an 8 GB to 16 GB GPU.
- not this job: Nemotron 3 Nano Omni 30B A3B Reasoning (NVIDIA Corporation): Smallest build is 18.5 GB (Ollama 21 GB), over an 8 GB to 16 GB GPU.
- not this job: Nemotron 3 Super 120B A12B (NVIDIA Corporation): No build fits an 8 GB to 16 GB GPU.
- not this job: Nemotron 3 Ultra 550B A55B (NVIDIA Corporation): No build fits an 8 GB to 16 GB GPU; Ollama lists only a cloud tag.
- not this job: occamy-1.0 (Accio (Accio-Lab)): Smallest GGUF is 11.7 GB, over an 8 GB GPU, and there is no Ollama library entry.
- not this job: Olmo 3 7B Think (The Allen Institute for Artificial Intelligence (Ai2), a 501(c)(3) nonprofit, per its Responsible Use Guidelines): Not wired for OpenAI-style tools (custom functions field, no parser) and thinking cannot be turned off.
- not this job: Ornith-1.5-35B-A3B (Ornith): Smallest build is Q4_K_M at 21.7 GB; nothing fits an 8 GB to 16 GB GPU.
- not this job: Ornith-1.5-397B (Ornith): No build fits an 8 GB to 16 GB GPU.
- not this job: Qwen3.5-122B-A10B (Alibaba Cloud (named in the licence's copyright line)): No build fits an 8 to 16 GB GPU; the smallest Unsloth quant is 34 GB.
- not this job: Qwen3.5-397B-A17B (Alibaba Cloud (named in the licence's copyright line)): No build fits an 8 to 16 GB GPU.
- not this job: Qwen3.6-27B (Alibaba Cloud (named in the licence's copyright line)): IQ4_XS is 15.4 GB before 4 GB of KV at 64K, so it does not fit an 8 to 16 GB GPU, and acc-11 saw it describe actions instead of calling tools on Ollama.
- not this job: Qwen3.6-35B-A3B (Alibaba Cloud (named in the licence's copyright line)): UD-Q4_K_XL is 22 GB, past an 8 to 16 GB GPU, and acc-23 saw the family's 35B fail tool calling on Ollama before it worked on llama.cpp.
- not this job: Qwen3.8-27B (Alibaba Cloud (named in the licence's copyright line)): UD-IQ4_XS is 14.4 GB before 4 GB of KV at 64K; too large for an 8 GB GPU and no account measures it in a voice pipeline.
- not this job: Qwen3.8-Flash-Next (Alibaba Cloud (Singapore) Private Limited, the registrant of qwen.ai. The licence names Qwen as copyright holder): No build fits an 8 to 16 GB GPU.
- not this job: Qwen3-Coder-Next (Alibaba Cloud (the Qwen team's employer; no LICENSE file names a copyright holder for this model)): No build fits an 8 to 16 GB GPU; Unsloth's smallest is 19 GB at 1 bit.
- not this job: Qwen3-Omni-30B-A3B-Instruct (Alibaba Cloud (carried from the 20 October 2026 pilot; this model's files name no copyright holder)): Tool calls are JSON in the template with no parser named, and the smallest build is 18.6 GB plus a 1.3 GB projector, past an 8 to 16 GB GPU.
- not this job: Qwen3-VL-30B-A3B-Instruct (Alibaba Cloud (per the pilot's RDAP check of qwen.ai; this model's files name no copyright holder)): 18.6 GB at Q4 plus projector does not fit an 8 to 16 GB GPU.
- not this job: Qwen3-VL-32B-Instruct (Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder)): 19.8 GB at Q4 does not fit an 8 to 16 GB GPU.
- not this job: Qwopus3.8-27B-Flash-V2 (Jackrong, an individual Hugging Face account): IQ4_XS is 15.4 GB before KV and the recommended mode is thinking at xhigh; no fit for an 8 GB GPU voice pipeline.
- not this job: Recursive Flint (Recursive Superintelligence, Inc. (the Hugging Face organisation's full name)): Thinking cannot be switched off, which the pipeline cannot tolerate, and 23 GB passes an 8 to 16 GB GPU.
- not this job: Ripple-65B Preview (Ricursive Intelligence, Inc.): Thinking cannot be turned off and nothing fits an 8 to 16 GB GPU.
- not this job: Swift 1.5 Qwen3.8-27B (UkisAI): IQ4_XS is 15.5 GB before KV; too large for an 8 GB GPU.
- not this job: Swift 1.5 Qwen3.8-Flash-Next (UkisAI): No build fits an 8 to 16 GB GPU.
- not this job: ThinkingCap (Qwen3.8 27B) (BottleCap AI (named in the licence's copyright line)): IQ4_XS is 15.5 GB before KV; too large for an 8 GB GPU.
- not this job: Baidu Unlimited-OCR-2 (Baidu, Inc.): No tool calling and no chat template.

28 listings in the category were added after this reading and aren't marked yet.

## What stops agents finishing

- Ollama template lacks tools, so the server answers 'does not support tools'
- a 3-bit quant gets stuck in hidden reasoning and returns nothing
- runtime regression prints the tool call as text

## Around it

Before: speech-to-text.transcribe. After: home-automation.execute, text-to-speech.speak.

## Where this comes from

Weight 10 of 100 in Open-weight models: 7 of 51 accounts, all from the Home Assistant forum, personal blogs and Ollama issues

As data: https://www.anchorterminal.com/api/v1/use-cases/open-weights.tool-calling-assistant.json, or the MCP tool get_use_case with id open-weights.tool-calling-assistant. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp
