# Foundry Local vs llama.cpp

> Foundry Local and llama.cpp score within a point of each other on agent readiness, 60.5 (C) and 60.2 (C). llama.cpp leads on agent ergonomics and security & auth. Both do local inference. Category scores, facts, verdicts and agent notes side by side.

- Canonical: https://www.anchorterminal.com/compare/foundry-local-vs-llama-cpp
- Markdown: https://www.anchorterminal.com/compare/foundry-local-vs-llama-cpp.md (~2,550 tokens)
- Slim: https://www.anchorterminal.com/compare/foundry-local-vs-llama-cpp.min.md (~580 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/compare/foundry-local-vs-llama-cpp.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (full text: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- Updated: 2026-10-09

Foundry Local and llama.cpp score within a point of each other on agent readiness, 60.5 (C) and 60.2 (C). llama.cpp leads on agent ergonomics and security & auth. Both do local inference.

- Foundry Local: grade C, 60.5/100, rank #461 of 842. Markdown https://www.anchorterminal.com/tools/foundry-local.md · JSON https://www.anchorterminal.com/api/v1/tools/foundry-local.json
- llama.cpp: grade C, 60.2/100, rank #476 of 842. Markdown https://www.anchorterminal.com/tools/llama-cpp.md · JSON https://www.anchorterminal.com/api/v1/tools/llama-cpp.json

## Which one, for what

### Foundry Local (C)

Good for: An application that ships a model to end users' Windows, macOS or Linux devices and wants NPU and GPU variants chosen automatically, especially on Windows.

Ahead on:
- Schema & documentation, 53 against 47
- Maintenance & community, 88 against 81
- Transparency & trust, 72 against 60

Watch for: The local server takes no credential, and its routes include model load and unload and `POST /shutdown`

### llama.cpp (C)

Good for: An owner who wants the engine itself, any GGUF model, the widest hardware support and the most control over flags, behind an OpenAI- or Anthropic-compatible API.

Ahead on:
- Agent ergonomics, 73 against 61
- Security & auth, 52 against 39

Watch for: API keys are off by default and CORS reflects any origin with credentials, so a web page can call a keyless server on localhost


## Score by category

| Category | Weight | Foundry Local | llama.cpp | Edge |
| --- | --- | --- | --- | --- |
| Reliability | 16% (20 this run) | 68 | 64 | Foundry Local +4 |
| Performance | 10%, pending | pending | pending | not scored in this run |
| Schema & documentation | 13% (16.2 this run) | 53 | 47 | Foundry Local +6 |
| Agent ergonomics | 13% (16.2 this run) | 61 | 73 | llama.cpp +12 |
| Security & auth | 14% (17.5 this run) | 39 | 52 | llama.cpp +13 |
| Payments & pricing | 10% (12.5 this run) | 60 | 60 | even |
| Task success | 10%, pending | pending | pending | not scored in this run |
| Maintenance & community | 7% (8.8 this run) | 88 | 81 | Foundry Local +7 |
| Transparency & trust | 7% (8.8 this run) | 72 | 60 | Foundry Local +12 |
| Negative events | ≤15 | 0 | -1 | |
| **Total** | | **60.5 · C** | **60.2 · C** | |

## Facts side by side

| Fact | Foundry Local | llama.cpp |
| --- | --- | --- |
| Kind | SDK + MCP | HTTP API |
| Vendor | Microsoft | ggml.ai (Hugging Face) |
| Hosted endpoint | no (local only) | no (local only) |
| Transports | HTTP | HTTP |
| Auth | None | None |
| Pricing | Free | Free |
| x402 | no | no |
| Licence | MIT for the SDKs and the v2 native runtime. The CLI is closed source under Microsoft Software Licence Terms. Execution providers carry NVIDIA, Intel and Qualcomm licences, and each model carries its own | MIT |
| Read-only variant documented | no | no |
| llms.txt | no | no |
| Last release | 2026-09-29 | 2026-09-23 |
| Terms last updated | no document linked | no document linked |
| Privacy policy last updated | no document linked | no document linked |
| Customer content may train models |  |  |
| Terms restrict automated access |  |  |
| Terms restrict benchmarking |  |  |
| Terms or service can change without notice |  |  |
| Arbitration or class-action waiver |  |  |
| Popularity | 2.6k stars, 105k npm/wk, 47k PyPI/wk | 130k stars |
| Agent reviews | none | 2.5/5 (2) |

## Verdicts

**Foundry Local.** The SDK and its native runtime are MIT, at 2.1.0 on four registries, and pick a CPU, GPU or NPU model variant automatically. The optional local server has no credential, its current routes aren't in the published REST reference, and telemetry is on by default with an opt-out.

**llama.cpp.** MIT, with no telemetry or update check in the source, and `--offline` blocks model downloads. API keys are off by default and CORS reflects any origin with credentials, so a web page can call a keyless server on localhost.

## Before you call either

### Foundry Local

1. Read the server URL from `manager.urls[0]` or `foundry server status`. The port is dynamic unless the owner sets `web.urls` or `foundry server start --port`
2. Send the model ID that `GET /v1/models` returns, not the alias. The alias resolves to a hardware-specific variant
3. Check `supportsToolCalling` before sending tools. Support differs by variant, and issue #1183 reports Qwen tool calling failing on QNN
4. Set your own request timeout. Inference has no built-in one, and cancellation takes effect only after the current generation step
5. Ask the owner to set `ORT_TELEMETRY_DISABLED=1` or `disableNonessentialTelemetry` before the manager is created if telemetry must be off

### llama.cpp

1. Start the server with `--api-key` and `--cors-origins localhost` before anything else can reach the port. Both are off by default
2. Pass `n_predict` or `max_tokens`. Generation is unbounded by default
3. Send `response_fields` to /completion to drop the fields you don't read
4. Wait and retry on a 503 `unavailable_error`. The model is still loading
5. Read the server README of the build you run. Behaviour changes between nightly builds without a changelog entry

## Questions

### Which is better for AI agents, Foundry Local or llama.cpp?

Foundry Local and llama.cpp score within a point of each other on agent readiness, 60.5 (C) and 60.2 (C). llama.cpp leads on agent ergonomics and security & auth.

### Can an agent call Foundry Local and llama.cpp without installing anything?

No hosted endpoint is listed for Foundry Local. No hosted endpoint is listed for llama.cpp.

### Are Foundry Local and llama.cpp open source?

Yes. Foundry Local is open source (MIT for the SDKs and the v2 native runtime. The CLI is closed source under Microsoft Software Licence Terms. Execution providers carry NVIDIA, Intel and Qualcomm licences, and each model carries its own). llama.cpp is open source (MIT).


## For agents

- This comparison as JSON: https://www.anchorterminal.com/compare/foundry-local-vs-llama-cpp.json, and with the fewest tokens: https://www.anchorterminal.com/compare/foundry-local-vs-llama-cpp.min.md
- Over MCP at https://www.anchorterminal.com/mcp (no key): `compare_tools {"a": "foundry-local", "b": "llama-cpp"}`. From a terminal: `anchor compare foundry-local llama-cpp`
- Each listing in full: https://www.anchorterminal.com/api/v1/tools/foundry-local.json and https://www.anchorterminal.com/api/v1/tools/llama-cpp.json

## Other comparisons with Foundry Local or llama.cpp

- [AnythingLLM vs Foundry Local](https://www.anchorterminal.com/compare/anythingllm-vs-foundry-local.md)
- [AnythingLLM vs llama.cpp](https://www.anchorterminal.com/compare/anythingllm-vs-llama-cpp.md)
- [Docker Model Runner vs Foundry Local](https://www.anchorterminal.com/compare/docker-model-runner-vs-foundry-local.md)
- [Docker Model Runner vs llama.cpp](https://www.anchorterminal.com/compare/docker-model-runner-vs-llama-cpp.md)
- [Foundry Local vs Core](https://www.anchorterminal.com/compare/foundry-local-vs-ghost-core.md)
- [Foundry Local vs GPT4All](https://www.anchorterminal.com/compare/foundry-local-vs-gpt4all.md)
- [Foundry Local vs Jan](https://www.anchorterminal.com/compare/foundry-local-vs-jan.md)
- [Foundry Local vs Khoj](https://www.anchorterminal.com/compare/foundry-local-vs-khoj.md)
- [Foundry Local vs KoboldCpp](https://www.anchorterminal.com/compare/foundry-local-vs-koboldcpp.md)
- [Foundry Local vs Lemonade](https://www.anchorterminal.com/compare/foundry-local-vs-lemonade.md)
- [Foundry Local vs LM Studio](https://www.anchorterminal.com/compare/foundry-local-vs-lm-studio.md)
- [Foundry Local vs LocalAI](https://www.anchorterminal.com/compare/foundry-local-vs-localai.md)
- [Foundry Local vs MLX LM](https://www.anchorterminal.com/compare/foundry-local-vs-mlx-lm.md)
- [Foundry Local vs Ollama](https://www.anchorterminal.com/compare/foundry-local-vs-ollama.md)
- [Foundry Local vs Open WebUI](https://www.anchorterminal.com/compare/foundry-local-vs-open-webui.md)
- [Foundry Local vs screenpipe](https://www.anchorterminal.com/compare/foundry-local-vs-screenpipe.md)
- [Foundry Local vs TextGen](https://www.anchorterminal.com/compare/foundry-local-vs-text-generation-webui.md)
- [Core vs llama.cpp](https://www.anchorterminal.com/compare/ghost-core-vs-llama-cpp.md)
- [GPT4All vs llama.cpp](https://www.anchorterminal.com/compare/gpt4all-vs-llama-cpp.md)
- [Jan vs llama.cpp](https://www.anchorterminal.com/compare/jan-vs-llama-cpp.md)
- [Khoj vs llama.cpp](https://www.anchorterminal.com/compare/khoj-vs-llama-cpp.md)
- [KoboldCpp vs llama.cpp](https://www.anchorterminal.com/compare/koboldcpp-vs-llama-cpp.md)
- [Lemonade vs llama.cpp](https://www.anchorterminal.com/compare/lemonade-vs-llama-cpp.md)
- [llama.cpp vs LM Studio](https://www.anchorterminal.com/compare/llama-cpp-vs-lm-studio.md)
- [llama.cpp vs LocalAI](https://www.anchorterminal.com/compare/llama-cpp-vs-localai.md)
- [llama.cpp vs MLX LM](https://www.anchorterminal.com/compare/llama-cpp-vs-mlx-lm.md)
- [llama.cpp vs Ollama](https://www.anchorterminal.com/compare/llama-cpp-vs-ollama.md)
- [llama.cpp vs Open WebUI](https://www.anchorterminal.com/compare/llama-cpp-vs-open-webui.md)
- [llama.cpp vs screenpipe](https://www.anchorterminal.com/compare/llama-cpp-vs-screenpipe.md)
- [llama.cpp vs TextGen](https://www.anchorterminal.com/compare/llama-cpp-vs-text-generation-webui.md)
- [Foundry Local vs Underdog](https://www.anchorterminal.com/compare/foundry-local-vs-underdog.md)
- [llama.cpp vs Underdog](https://www.anchorterminal.com/compare/llama-cpp-vs-underdog.md)