{ "data": { "a": { "slug": "llama-cpp", "name": "llama.cpp", "vendor": "ggml.ai (Hugging Face)", "vendorUrl": "https://llama.app", "kind": "http-api", "category": "local-ai", "summary": "Open-source C/C++ engine for running GGUF models locally, with a web interface and compatible model APIs.", "url": "https://www.anchorterminal.com/tools/llama-cpp", "markdownUrl": "https://www.anchorterminal.com/tools/llama-cpp.md", "slimMarkdownUrl": "https://www.anchorterminal.com/tools/llama-cpp.min.md", "jsonUrl": "https://www.anchorterminal.com/api/v1/tools/llama-cpp.json", "dns": "llama-cpp.grade.anchorterminal.com", "repo": "https://github.com/ggml-org/llama.cpp", "license": "MIT", "transports": [ "http" ], "packages": [ { "registry": "oci", "name": "ghcr.io/ggml-org/llama.cpp" }, { "registry": "pypi", "name": "gguf" } ], "auth": "none", "authNotes": "No credential by default. `--api-key` (one key or a comma-separated list) or `--api-key-file` (one key a line) turns on a check for every route but /health and the web UI's files, with the key sent as `Authorization: Bearer` or `X-Api-Key`, never in the query string. Keys have no scopes and change only with a restart. TLS is built in with `--ssl-key-file` and `--ssl-cert-file`. The server binds 127.0.0.1:8080 by default, and CORS reflects any Origin with credentials allowed unless built-in tools, MCP servers or `--agent` are on, when it narrows to localhost (https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md).", "pricing": "free", "pricingNotes": "Free under MIT, with no account, key or card. Nothing is sold. You pay for your own hardware and electricity.", "priceSummary": "Free · OSS", "where": "local", "x402": { "level": "no", "evidence": "No x402, MPP or L402 in the docs or the source (checked 2026-10-03).", "endpoints": [] }, "toolCount": null, "popularity": { "githubStars": 130200, "npmWeekly": null, "pypiWeekly": null, "asOf": "2026-10-03" }, "docsUrl": "https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md", "capabilities": [ "inference.local", "inference.open-weights", "embed.text", "rerank", "inference.decision", "agent.mcp-client" ], "tags": [ "open-source", "local", "self-hosted", "free", "no-card", "openai-compatible", "docker", "pre-1.0", "no-telemetry" ], "lastRelease": "2026-10-28", "graded": true, "anchor": { "graded": true, "score": 45.2, "grade": "E", "agentReady": false, "rank": 1183, "ranked": true, "rankOf": 1486, "categoryRank": 17, "methodology": "0.7", "run": "2026-10-01", "scores": { "ergonomics": 73, "maintenance": 81, "payments": 60, "reliability": 57, "schema": 47, "security": 52, "transparency": 64 }, "pending": [ "performance", "tasks" ], "assessment": { "confidence": "medium", "date": "2026-11-03" }, "negative": -15, "negativeNotes": [ "Advisories published by the project, per advisory. CVE-2026-21869 (GHSA-8947-pfff-2f3c, 8 January 2026, High 8.8, a llama-server out-of-bounds write through a negative `n_discard`), CVE-2026-27940 (GHSA-3p4r-fq3f-q74v, 12 March, High 7.8, GGUF `mem_size` overflow), CVE-2026-33298 (GHSA-96jg-mvhq-q7q7, 24 March, High 7.8, GGUF tensor parsing overflow) and CVE-2026-34159 (GHSA-j8rj-fmpv-wcxw, 1 April, Critical 9.8, unauthenticated code execution in the RPC backend). All fixed and more than two months old, 1 each, 4. https://github.com/ggml-org/llama.cpp/security/advisories/GHSA-j8rj-fmpv-wcxw", "Third-party CVE records against llama.cpp's own code, per advisory, read from OSV and NVD on 3 November 2026. Fixed Highs and Criticals over two months old cost 1 each, eight VulnCheck records of 6 August 2026 (CVE-2026-43622, 43627, 43628, 43629, 43631, 43632, 70638, 70640) and three MITRE records of 1 September (CVE-2026-52130, 52131, 52132), 11. Fixed Highs and Criticals under two months old cost 2 each, CVE-2026-107183 (7 October), CVE-2024-58400 and CVE-2026-116816 (27 October), CVE-2026-118375 (28 October) and CVE-2026-119052 (29 October), all VulnCheck, 10. Third-party Mediums and Lows (VulDB and VulnCheck) cost nothing. https://services.nvd.nist.gov/rest/json/cves/2.0?keywordSearch=llama.cpp", "9 October 2026. Commit 8ae38670 fixes an out-of-bounds write below the destination buffer in `ggml_acc` from a truncated negative offset, released in v0.7.0 (15 October) with no advisory. A released fix of a described flaw with no advisory is its own kind, -3. https://github.com/ggml-org/llama.cpp/commit/8ae38670", "Total 4 + 21 + 3 = 28, held at the -15 floor. Since 1 June 2026 SECURITY.md says private disclosure is disabled and asks for fixes as public pull requests, so flaws are fixed in public before any advisory. https://github.com/ggml-org/llama.cpp/blob/master/SECURITY.md" ], "verdict": "MIT, with no telemetry or update check in the source, and `--offline` blocks model downloads. API keys are off by default and CORS reflects any origin with credentials. Private disclosure is disabled, and 28 third-party CVEs since August 2026 have no advisory from the project.", "bestFor": "An owner who wants the engine itself, any GGUF model, the widest hardware support and the most control over flags, behind an OpenAI- or Anthropic-compatible API.", "strengths": [ "MIT, with no telemetry or update check in the source, and `--offline` blocks model downloads", "OpenAI chat completions, responses and embeddings, Anthropic messages, reranking and /v1/systemone from one server", "`response_fields`, `json_schema` and `grammar` control the size and shape of output, and errors carry an OpenAI-style type and code", "Eleven semver releases and 1,281 numbered builds in the 90 days to 3 November 2026, with CI workflows running on every push to master", "SECURITY.md guidance on untrusted models, inputs and networks, and fixes for reported flaws ship in numbered builds within days" ], "weaknesses": [ "API keys are off by default and CORS reflects any origin with credentials, so a web page can call a keyless server on localhost", "No OpenAPI file of its own, and the REST API changelog stops at b4599", "Private security disclosure disabled since 1 June 2026, and no project advisory since 1 April while third parties published 28 CVEs", "Pre-1.0 (0.8.0), and semver releases are bare tags with no notes", "No official client library, and `n_predict` defaults to unlimited" ], "agentNotes": [ "Start the server with `--api-key` and `--cors-origins localhost` before anything else can reach the port. Both are off by default", "Pass `n_predict` or `max_tokens`. Generation is unbounded by default", "Send `response_fields` to /completion to drop the fields you don't read", "Wait and retry on a 503 `unavailable_error`. The model is still loading", "Read the server README of the build you run. Behaviour changes between nightly builds without a changelog entry" ], "metrics": { "kind": "local", "measured": false }, "reviewCount": 2, "avgRating": 2.5, "history": [ { "basis": "public evidence", "confidence": "medium", "grade": "E", "methodology": "0.7", "pending": [ "performance", "tasks" ], "run": "2026-10-01", "runLabel": "October 2026 research run", "score": 45.2 } ], "editorialScores": { "ergonomics": 73, "maintenance": 81, "payments": 60, "reliability": 57, "schema": 47, "security": 52, "transparency": 66 }, "provenanceScore": 61, "checked": { "categories": { "ergonomics": "2026-11-03", "maintenance": "2026-11-03", "payments": "2026-11-03", "reliability": "2026-11-03", "schema": "2026-11-03", "security": "2026-11-03", "transparency": "2026-11-03" }, "last": "2026-11-03", "lastMoved": "2026-11-03", "oldest": "2026-11-03", "stale": false, "staleAfterDays": 90 } }, "connect": { "install": "curl -LsSf https://llama.app/install.sh | sh # or: brew install llama.cpp; winget install llama.cpp\nllama serve -hf ggml-org/Qwen3.5-0.8B-GGUF # listens on 127.0.0.1:8080", "http": "curl --request POST \\\n --url http://localhost:8080/completion \\\n --header \"Content-Type: application/json\" \\\n --data '{\"prompt\": \"Building a website can be done in 10 simple steps:\",\"n_predict\": 128}'" }, "letme": { "capability": "https://letme.dev/inference.local", "tool": "https://letme.dev/llama-cpp" }, "area": "models", "provenance": { "legalEntity": "ggml.ai, part of Hugging Face since 2026", "domain": "llama.app", "domainRegistered": "", "endpointOnVendorDomain": null, "terms": "", "privacy": "", "statusPage": "", "changelog": "https://github.com/ggml-org/llama.cpp/releases", "securityTxt": "none", "checked": "2026-11-03", "notes": [ "The repository's About link is llama.app, which says it's by the llama.cpp team and Hugging Face and links no terms, privacy or security page. ggml.ai says the company was acquired by Hugging Face in 2026 and names no address.", "The `LICENSE` file reads Copyright (c) 2023-2026 The ggml authors.", "llama.app/.well-known/security.txt and llama.app/llms.txt returned 404 on 3 November 2026. SECURITY.md points to GitHub private advisories while saying private disclosure is disabled.", "There's no shared hosted endpoint. The server runs on the owner's machine." ], "score": 61 }, "pageJsonUrl": "https://www.anchorterminal.com/tools/llama-cpp.json", "live": { "slug": "llama-cpp", "versions": [ { "registry": "github", "name": "ggml-org/llama.cpp", "version": "v0.9.0", "released": "2026-11-04", "seenAt": "2026-11-06T23:40:43.539872814Z" }, { "registry": "pypi", "name": "gguf", "version": "0.19.0", "released": "2026-05-06", "seenAt": "2026-11-06T23:40:43.433798139Z" } ], "githubStars": 133575, "pypiWeekly": 1277940, "securityTxt": { "url": "https://llama.app/.well-known/security.txt", "state": "none", "checkedAt": "2026-11-07T16:01:25.22514458Z" }, "domain": { "domain": "llama.app", "registered": "2018-07-18", "source": "https://pubapi.registry.google/rdap/domain/llama.app", "checkedAt": "2026-11-01T14:23:03.546793132Z" }, "github": { "repo": "ggml-org/llama.cpp", "checkedAt": "2026-11-02T08:50:34.528980786Z", "stars": 133125, "archived": false, "issuesEnabled": true, "pushedAt": "2026-11-02T08:41:22Z", "openIssues": 745, "openPRs": 1853, "issuesOpened90d": 1420, "issuesClosed90d": 1366, "prsMerged90d": 1768, "ci": { "branch": "master", "runs": 50, "success": 49, "failure": 1, "other": 0, "latest": "success", "latestAt": "2026-11-02T08:41:23Z" }, "advisories": [ { "ghsa": "GHSA-j8rj-fmpv-wcxw", "cve": "CVE-2026-34159", "severity": "critical", "published": "2026-03-26T09:59:34Z" }, { "ghsa": "GHSA-96jg-mvhq-q7q7", "cve": "CVE-2026-33298", "severity": "high", "published": "2026-03-18T18:37:01Z" }, { "ghsa": "GHSA-3p4r-fq3f-q74v", "cve": "CVE-2026-27940", "severity": "high", "published": "2026-03-12T11:17:52Z" }, { "ghsa": "GHSA-8947-pfff-2f3c", "cve": "CVE-2026-21869", "severity": "low", "published": "2026-01-05T14:00:09Z" }, { "ghsa": "GHSA-vgg9-87g3-85w8", "cve": "CVE-2025-53630", "severity": "high", "published": "2025-07-10T05:22:33Z" }, { "ghsa": "GHSA-g4cc-763q-h9h6", "severity": "medium", "published": "2025-06-25T06:26:17Z" }, { "ghsa": "GHSA-7rxv-5jhh-j6xx", "cve": "CVE-2025-52566", "severity": "high", "published": "2025-06-23T16:04:05Z" }, { "ghsa": "GHSA-8wwf-w4qm-gpqr", "cve": "CVE-2025-49847", "severity": "high", "published": "2025-06-14T09:55:24Z" }, { "ghsa": "GHSA-wcr5-566p-9cwj", "cve": "CVE-2024-42479", "severity": "critical", "published": "2024-08-12T14:18:18Z" }, { "ghsa": "GHSA-5vm9-p64x-gqw9", "cve": "CVE-2024-42478", "severity": "medium", "published": "2024-08-12T14:18:08Z" }, { "ghsa": "GHSA-mqp6-7pv6-fqjf", "cve": "CVE-2024-42477", "severity": "medium", "published": "2024-08-12T14:17:57Z" }, { "ghsa": "GHSA-49q7-2jmh-92fp", "cve": "CVE-2024-41130", "severity": "medium", "published": "2024-07-22T16:22:34Z" }, { "ghsa": "GHSA-p5mv-gjc5-mwqv", "cve": "CVE-2024-32878", "severity": "high", "published": "2024-04-26T10:17:10Z" } ], "replies": { "sampled": 10, "unanswered": 7, "medianDays": 0 } }, "updatedAt": "2026-11-07T16:01:25.22514458Z" } }, "answer": "PrivateGPT scores 53.2 (D) on agent readiness against llama.cpp's 45.2 (E), and leads in 2 of 7 scored categories. llama.cpp leads on agent ergonomics, security \u0026 auth and maintenance \u0026 community.", "b": { "slug": "privategpt", "name": "PrivateGPT", "vendor": "Zylon (PriBAI Technology Spain S.L.)", "vendorUrl": "https://docs.privategpt.dev", "kind": "http-api", "category": "local-ai", "summary": "PrivateGPT is Zylon's open-source server that puts an Anthropic-style messages API, document ingestion, retrieval with citations, built-in tools and MCP connectors in front of any OpenAI-compatible local model server. The owner runs it.", "url": "https://www.anchorterminal.com/tools/privategpt", "markdownUrl": "https://www.anchorterminal.com/tools/privategpt.md", "slimMarkdownUrl": "https://www.anchorterminal.com/tools/privategpt.min.md", "jsonUrl": "https://www.anchorterminal.com/api/v1/tools/privategpt.json", "dns": "privategpt.grade.anchorterminal.com", "repo": "https://github.com/zylon-ai/private-gpt", "license": "Apache-2.0", "transports": [ "http" ], "packages": [ { "registry": "oci", "name": "zylonai/private-gpt" } ], "auth": "none", "authNotes": "Authentication is off by default. When `server.auth.enabled` is set, every request must carry one static secret in the `Authorization` header, compared with the whole header value. There are no per-user keys, scopes or rotation, and the docs show it as a Bearer token, a Basic value and an Anthropic API key in different places (checked 2026-11-06).", "pricing": "free", "pricingNotes": "Free under Apache-2.0 with no account or card. The cost is the owner's hardware and model server. Zylon sells a separate enterprise platform built on PrivateGPT through demos, with no public price (checked 2026-11-06).", "priceSummary": "Free · OSS", "where": "local", "x402": { "level": "no", "evidence": "No x402, MPP or L402 in the source, the OpenAPI file or the docs (checked 2026-11-06).", "endpoints": [] }, "toolCount": null, "popularity": { "githubStars": 57600, "npmWeekly": null, "pypiWeekly": null, "asOf": "2026-11-06" }, "docsUrl": "https://docs.privategpt.dev/introduction", "llmsTxt": "https://docs.privategpt.dev/llms.txt", "openapi": "https://raw.githubusercontent.com/zylon-ai/private-gpt/main/fern/openapi/openapi.json", "capabilities": [ "inference.local", "memory.search", "agent.mcp-client" ], "tags": [ "open-source", "local", "self-hosted", "docker", "openapi", "llms-txt", "rag", "mcp-client", "anthropic-compatible", "python", "free" ], "lastRelease": "2026-06-18", "graded": true, "anchor": { "graded": true, "score": 53.2, "grade": "D", "agentReady": false, "rank": 954, "ranked": true, "rankOf": 1486, "categoryRank": 11, "methodology": "0.7", "run": "2026-10-01", "scores": { "ergonomics": 64, "maintenance": 41, "payments": 60, "reliability": 70, "schema": 83, "security": 32, "transparency": 64 }, "pending": [ "performance", "tasks" ], "assessment": { "confidence": "medium", "date": "2026-11-06" }, "negative": -7, "negativeNotes": [ "Issue #2269, opened 11 June 2026, reports that the async ingestion API reads any local path given as a URI artifact. The stale bot closed it as not planned on 5 July with no maintainer reply, and `load_file_from_uri` in the source of 29 October 2026 still falls back to reading a local file for any value that isn't a URL, s3 or base64. With authentication off and a 0.0.0.0 bind by default, that is an open file-read path on a default install. Read from the source, not run (-7). https://github.com/zylon-ai/private-gpt/issues/2269" ], "verdict": "PrivateGPT 1.0 is an Apache-2.0 server that gives local models an Anthropic-style messages API with retrieval, citations, tools and MCP, documented by an OpenAPI file and llms.txt. A default start has authentication off, listens on every interface and reads local paths given as URIs, and a public report of that was closed without a reply.", "bestFor": "A local backend for apps and agents that want an Anthropic-style API with document retrieval, citations and tools over local models.", "strengths": [ "Messages API modelled on Anthropic's, so Claude Code and Anthropic SDKs work by changing the base URL", "OpenAPI file of 38 paths, llms.txt and Markdown copies of every docs page", "Retrieval with citations, async ingestion, web, database and tabular tools, and remote MCP servers in one API", "Apache-2.0, with no telemetry to Zylon found in the source and OpenTelemetry tracing off by default", "Works with Ollama, llama.cpp, vLLM, LM Studio or any OpenAI-compatible server" ], "weaknesses": [ "Authentication is off by default, the server binds 0.0.0.0 and CORS allows every origin", "When on, authentication is one shared static secret, with no per-user keys or scopes", "A June 2026 report of local file reads through URI artifacts was closed by a stale bot, and the source still reads local paths", "No release since 1.0.1 in June 2026 while main has moved on, including a breaking change", "Code execution runs as a host subprocess with resource limits, not in a container" ], "agentNotes": [ "Turn on `server.auth` and bind to 127.0.0.1 before exposing an instance, since a default start answers any caller", "Point Anthropic SDKs or Claude Code at the server with `ANTHROPIC_BASE_URL=http://localhost:8080`", "Start an OpenAI-compatible model server and an embedding server first, and set `OPENAI_API_BASE` and `OPENAI_EMBEDDING_API_BASE`", "Pass ingested documents through `tool_context` to get answers with citations", "Treat web fetch, search and MCP results as untrusted, since PrivateGPT documents no prompt-injection handling" ], "metrics": { "kind": "local", "measured": false }, "reviewCount": 0, "avgRating": 0, "history": [ { "basis": "public evidence", "confidence": "medium", "grade": "D", "methodology": "0.7", "pending": [ "performance", "tasks" ], "run": "2026-10-01", "runLabel": "October 2026 research run", "score": 53.2 } ], "editorialScores": { "ergonomics": 64, "maintenance": 41, "payments": 60, "reliability": 70, "schema": 83, "security": 32, "transparency": 67 }, "provenanceScore": 61, "checked": { "categories": { "ergonomics": "2026-11-06", "maintenance": "2026-11-06", "payments": "2026-11-06", "reliability": "2026-11-06", "schema": "2026-11-06", "security": "2026-11-06", "transparency": "2026-11-06" }, "last": "2026-11-06", "oldest": "2026-11-06", "stale": false, "staleAfterDays": 90 } }, "connect": { "install": "brew tap zylon-ai/tap\nbrew install private-gpt", "http": "curl http://localhost:8080/v1/messages \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\": \"qwen3.5:35b\", \"messages\": [{\"role\": \"user\", \"content\": \"Explain retrieval-augmented generation.\"}]}'" }, "letme": { "capability": "https://letme.dev/inference.local", "tool": "https://letme.dev/privategpt" }, "area": "models", "provenance": { "legalEntity": "PriBAI Technology Spain S.L.", "domain": "privategpt.dev", "domainRegistered": "2023-10-16", "endpointOnVendorDomain": null, "terms": "https://github.com/zylon-ai/private-gpt/blob/main/LICENSE", "privacy": "", "statusPage": "", "changelog": "https://github.com/zylon-ai/private-gpt/blob/main/CHANGELOG.md", "securityTxt": "none", "checked": "2026-11-06", "notes": [ "Self-hosted software with no hosted endpoint, so the Apache-2.0 licence stands in for terms and no privacy policy is scored.", "The zylon.ai terms name PriBAI Technology Spain S.L. (VAT ESB56550130) under Spanish law and cover the website and Zylon services. They forbid access through bots, scripts or scrapers, outside standard search engine or browser use. We read the terms and two public pages.", "RDAP gives privategpt.dev a registration date of 16 October 2023.", "docs.privategpt.dev/.well-known/security.txt and www.zylon.ai/.well-known/security.txt answer 404. SECURITY.md in the repository is the reporting route.", "docs.privategpt.dev and www.zylon.ai robots.txt allow our reader. wheels.privategpt.dev has no robots.txt (404)." ], "score": 61 }, "pageJsonUrl": "https://www.anchorterminal.com/tools/privategpt.json", "live": { "slug": "privategpt", "versions": [ { "registry": "github", "name": "zylon-ai/private-gpt", "version": "v1.0.1", "released": "2026-06-18", "seenAt": "2026-11-06T23:58:40.643071731Z" } ], "githubStars": 57632, "securityTxt": { "url": "https://privategpt.dev/.well-known/security.txt", "state": "none", "checkedAt": "2026-11-07T15:59:53.351491852Z" }, "llmsTxt": { "url": "https://docs.privategpt.dev/llms.txt", "ok": true, "status": 200, "checkedAt": "2026-11-07T15:33:41.730243326Z" }, "pages": [ { "url": "https://raw.githubusercontent.com/zylon-ai/private-gpt/main/CHANGELOG.md", "kind": "changelog", "status": 200, "checkedAt": "2026-11-07T14:24:15.016082629Z", "changedAt": "0001-01-01T00:00:00Z", "fingerprint": "767d5dd99544" }, { "url": "https://raw.githubusercontent.com/zylon-ai/private-gpt/main/LICENSE", "kind": "terms", "status": 200, "checkedAt": "2026-11-07T14:24:17.075442465Z", "changedAt": "0001-01-01T00:00:00Z", "fingerprint": "839fbbcddc72" } ], "updatedAt": "2026-11-07T15:59:53.351491852Z" } }, "edges": [ { "by": 40, "category": "maintenance", "leads": "llama-cpp", "name": "Maintenance \u0026 community", "reasons": { "a": "Semver release v0.8.0 on 28 October 2026, and build b12087 on 3 November (30). Eleven semver releases from v0.1.0 (17 August) to v0.8.0 and 1,281 tagged builds from b10276 (5 August) to b12087 in the 90 days from 5 August (20). 1,822 commits on master from 450 authors since 5 August. 868 open issues and 1.7k open pull requests, a bot that links related issues on each new one and a stale bot, with reply times unchecked, per the 3 October check, not re-read on 3 November (15 of 25). No official client library for the server. The C API in llama.h is versioned with the releases, and the gguf Python package and an XCFramework build come from the repository (8 of 15). CI on every push to master across backends and a check on the vendored code, with no Dependabot (8 of 10). 30 + 20 + 15 + 8 + 8 = 81.", "b": "The last release is 1.0.1, dated 16 June 2026 in the changelog and tagged 18 June, 141 days before the check (10 of 30). No release or dated changelog entry in the 90 days since 8 August (0 of 20). Maintainers commit most days and outside pull requests merged in October, but 23 pull requests are open and the stale bot closes issues, including two security reports, with no maintainer reply (13 of 25). Homebrew tap, wheel index and Docker images ship with releases, and Anthropic SDKs work against it. No product MCP server of its own, though it connects to remote MCP servers (10 of 15). Dependabot, uv.lock, actions pinned by commit since 19 October and dependency CVE upgrades in June, July and August (8 of 10). 10 + 0 + 13 + 10 + 8 = 41." }, "scores": { "a": 81, "b": 41 } }, { "by": 36, "category": "schema", "leads": "privategpt", "name": "Schema \u0026 documentation", "reasons": { "a": "No spec of its own. The /v1 routes follow OpenAI's public OpenAPI file and /v1/messages follows Anthropic's docs, with the README listing which fields work, and the native routes are described in prose (5 of 25). No llms.txt (llama.app/llms.txt still returned 404 on 3 November 2026). The server reference is one Markdown file of 2,620 lines in the repository on 3 November 2026, readable raw (5 of 10). Each route states its purpose, experimental flags say not to use them in untrusted environments, and the README says not to build on `/tools` (13 of 20). Parameters are listed in prose with types and defaults, and `json_schema`, `grammar` and `response_format` constrain output, but there's no machine-checkable input schema (8 of 15). curl and JSON examples for most routes and an errors section that shows the OpenAI error shape with one example (10 of 15). Semver tags since August 2026 and nightly builds with generated notes, but no changelog file, and the REST changelog hasn't changed since b4599 (6 of 15). 5 + 5 + 13 + 8 + 10 + 6 = 47.", "b": "An OpenAPI file of 38 paths in the repository (fern/openapi/openapi.json, version 1.0.1), also served by each instance at /openapi.json, with the messages API modelled on Anthropic's (25). docs.privategpt.dev serves llms.txt, a Markdown copy of every page and a docs MCP server (10). The docs explain each endpoint group, startup and model discovery, and a compatibility table says which parts of the Claude API are supported, partial or missing, such as prompt caching and OAuth (15 of 20). Pydantic models with literal type discriminators and required fields, while settings and tool context are wide objects (12 of 15). curl examples for each guide and a `/v1/messages/validate` route, with error shapes not documented as a list (9 of 15). `/v1` paths and a dated changelog, though releases trail main by more than four months (12 of 15). 25 + 10 + 15 + 12 + 9 + 12 = 83." }, "scores": { "a": 47, "b": 83 } }, { "by": 20, "category": "security", "leads": "llama-cpp", "name": "Security \u0026 auth", "reasons": { "a": "Read with the tool checklist, credential model first. Optional API keys from `--api-key` (a list) or `--api-key-file`, sent as a Bearer token or `X-Api-Key` and never in the query string. They have no scopes, change only with a restart and are off by default. TLS is built in, and the server binds 127.0.0.1 by default (15 of 30). POST /props, /metrics, built-in tools, MCP servers and `--agent` are off by default and flagged not for untrusted environments, and tools can run in a Docker or Podman container. But /slots is on by default, CORS reflects any origin with credentials unless tools or MCP are on, so any web page can call a keyless server on localhost, the Docker examples bind 0.0.0.0 with no key, and every key can do everything (8 of 20). SECURITY.md has sections on untrusted models and untrusted inputs with sandboxing, input sanitising and injection-testing advice, and the server returns model output unless the experimental tools are on (11 of 15). Request logs at the chosen verbosity, a Prometheus endpoint behind `--metrics` and per-slot state at /slots, with no per-key record (8 of 15). SECURITY.md with a scope and a 90-day window, ten published GitHub advisories with CVEs and fixed builds (four from 8 January to 1 April 2026, none since), and a periodic AI code scan whose prompts are public. Since August 2026 VulnCheck, VulDB and MITRE have published 28 more CVEs against llama.cpp with no advisory from the project. Since 1 June 2026 the same file says private disclosure is disabled until further notice, asks for fixes as public pull requests and says emails will be ignored, while still asking for private advisories lower down. SECURITY.md was unchanged on 3 November. No security.txt or bug bounty (10 of 20). 15 + 8 + 11 + 8 + 10 = 52.", "b": "Read with the tool checklist. Authentication is off by default, and when on it is one static secret compared with the whole `Authorization` header, with a sample `secret:key` value in settings.yaml. There are no per-user keys or scopes, `private-gpt serve` binds 0.0.0.0 by default and CORS allows every origin, so a default start answers anyone on the network. The docs disagree on the header (Bearer, Basic or an Anthropic key) (6 of 30). Tools are chosen per request and code execution has internet off by default, but bash runs as a host subprocess with resource limits, not in a container, and nothing asks for approval (6 of 20). Web fetch, web search, documents and MCP servers bring untrusted text to the model. A public prompt-injection report (#2270) was closed by the stale bot, and no guidance was found beyond a 19 October change that bounds tool output (3 of 15). OpenTelemetry traces cover every request, model call, tool and sandbox step, off by default, with local JSON lines exporters (10 of 15). SECURITY.md since 23 July 2026 asks for private reports through GitHub or Discord. No bounty, no certification, and the June and July security reports in issues were closed by a bot with no reply (7 of 20). 6 + 6 + 3 + 10 + 7 = 32." }, "scores": { "a": 52, "b": 32 } }, { "by": 13, "category": "reliability", "leads": "privategpt", "name": "Reliability", "reasons": { "a": "Read with the local-software lines, since llama-server runs on the owner's machine with no hosted service. Official installs through the llama.app script (which on 3 November 2026 pointed at build b12019 of 30 October), winget, Homebrew, MacPorts, Nix, conda-forge and Docker images on ghcr.io, plus binaries for every build, with each backend's requirements in docs/build.md (20). Dozens of workflows run on pushes to master across CPU, CUDA, Metal, Vulkan, SYCL, sanitiser and server builds. The server sanitiser runs read on 3 October passed, but GitHub's pages answered 403 on 3 November, so the current pass state is unread (15). 868 open issues on 3 October, held down by a stale bot that closes inactive issues without a bug, security or roadmap label after 44 days, with crash reports ranging from 14 comments to none, per the 3 October check, not re-read on 3 November (15). Semver since v0.1.0 on 17 August 2026, with a written rule that a breaking change to llama.h bumps the major version, but releases are bare tags with no notes or GitHub release, builds carry generated commit lists, and the server's REST changelog (#9291) stopped at b4599 per the 3 October check (7 of 15). Version 0.8.0, pre-1.0 (0). 20 + 15 + 15 + 7 + 0 = 57.", "b": "Read with the local-software lines, since PrivateGPT runs on the owner's hardware with no hosted service behind it. Homebrew tap, `uv tool install` from Zylon's own wheel index (not PyPI) and Docker Hub images `zylonai/private-gpt` 1.0.1 of 18 June 2026. Python 3.11 only, stated in pyproject.toml, and it needs a separate OpenAI-compatible model server (17 of 20). tests.yml runs format, lint, typecheck and the tests affected by each push on main, with the full suite of 217 test files on release pull requests. We couldn't see run results because the GitHub API refused us, and issue #2441 of 1 November reports the Claude spec update workflow failing (15 of 25). GitHub shows 3 open issues of about 1,250, because a stale bot closes quiet issues, two security reports among them, with no maintainer reply (12 of 25). release-please keeps a changelog with breaking changes marked, but no release has been cut since 1.0.1 while main carries 51 commits since 8 August, among them a breaking observability change of 23 October (11 of 15). Version 1.0.1 (15). 17 + 15 + 12 + 11 + 15 = 70." }, "scores": { "a": 57, "b": 70 } }, { "by": 9, "category": "ergonomics", "leads": "llama-cpp", "name": "Agent ergonomics", "reasons": { "a": "Read for an API. `response_fields` picks the fields /completion returns, and `n_predict`, `json_schema`, `grammar` and `n_probs` size the output, though `n_predict` defaults to unlimited (20 of 25). Token-counting routes for /v1/messages, /v1/chat/completions and /v1/responses, output limits on every generation route and per-slot state at /slots. Model lists aren't paged (16 of 20). Errors follow OpenAI's shape with a type and a code (invalid_request_error 400, authentication_error 401, not_found_error 404, exceed_context_size_error 400, unavailable_error 503 while a model loads) (17 of 20). Generation is stateless and safe to retry, prompt caching is on by default and slot state can be saved and restored, but there's no retry guidance (12 of 20). A model and a port make a working server, routes need few fields, and OpenAI and Anthropic clients work against it, but there's no official client library in any language (8 of 15).", "b": "Read for an HTTP API an agent calls on a local instance. Token counting, `top_k` style retrieval settings, citation output and response byte caps let a caller size context (18 of 25). Artifact listing by namespace and retrieval filters, with no documented cursor pagination (12 of 20). FastAPI validation errors name the field and the validate route checks a request before running it, but there is no error catalogue (12 of 20). Async messages and ingestion return task ids with status, cancel and delete routes. No idempotency keys were found (10 of 20). Anthropic SDKs and Claude Code work by pointing `ANTHROPIC_BASE_URL` at the server, but a model server and an embedding server have to be configured first (12 of 15). 18 + 12 + 12 + 10 + 12 = 64." }, "scores": { "a": 73, "b": 64 } } ], "facts": [ { "a": "HTTP API", "b": "HTTP API", "name": "Kind" }, { "a": "ggml.ai (Hugging Face)", "b": "Zylon (PriBAI Technology Spain S.L.)", "name": "Vendor" }, { "a": "no (local only)", "b": "no (local only)", "name": "Hosted endpoint" }, { "a": "HTTP", "b": "HTTP", "name": "Transports" }, { "a": "None", "b": "None", "name": "Auth" }, { "a": "Free", "b": "Free", "name": "Pricing" }, { "a": "no", "b": "no", "name": "x402" }, { "a": "MIT", "b": "Apache-2.0", "name": "Licence" }, { "a": "no", "b": "no", "name": "Read-only variant documented" }, { "a": "no", "b": "yes", "name": "llms.txt" }, { "a": "2026-10-28", "b": "2026-06-18", "name": "Last release" }, { "a": "no document linked", "b": "no date given", "name": "Terms last updated" }, { "a": "no document linked", "b": "no document linked", "name": "Privacy policy last updated" }, { "a": "not read yet", "b": "not found in the text", "name": "Customer content may train models" }, { "a": "not read yet", "b": "not found in the text", "name": "Terms restrict automated access" }, { "a": "not read yet", "b": "not found in the text", "name": "Terms restrict benchmarking" }, { "a": "not read yet", "b": "not found in the text", "name": "Terms or service can change without notice" }, { "a": "not read yet", "b": "not found in the text", "name": "Arbitration or class-action waiver" }, { "a": "130k stars", "b": "58k stars", "name": "Popularity" }, { "a": "2.5/5 (2)", "b": "none", "name": "Agent reviews" } ], "faq": [ { "answer": "PrivateGPT scores 53.2 (D) on agent readiness against llama.cpp's 45.2 (E), and leads in 2 of 7 scored categories. llama.cpp leads on agent ergonomics, security \u0026 auth and maintenance \u0026 community.", "question": "Which is better for AI agents, llama.cpp or PrivateGPT?" }, { "answer": "Neither needs a key.", "question": "Do llama.cpp and PrivateGPT need an API key?" }, { "answer": "No hosted endpoint is listed for llama.cpp. No hosted endpoint is listed for PrivateGPT.", "question": "Can an agent call llama.cpp and PrivateGPT without installing anything?" }, { "answer": "Yes. llama.cpp is open source (MIT). PrivateGPT is open source (Apache-2.0).", "question": "Are llama.cpp and PrivateGPT open source?" } ], "goodFor": [ { "aheadOn": [ "Agent ergonomics, 73 against 64", "Security \u0026 auth, 52 against 32", "Maintenance \u0026 community, 81 against 41" ], "also": [ "Free to start without a card" ], "goodFor": "An owner who wants the engine itself, any GGUF model, the widest hardware support and the most control over flags, behind an OpenAI- or Anthropic-compatible API.", "slug": "llama-cpp", "watchFor": "API keys are off by default and CORS reflects any origin with credentials, so a web page can call a keyless server on localhost" }, { "aheadOn": [ "Reliability, 70 against 57", "Schema \u0026 documentation, 83 against 47" ], "also": null, "goodFor": "A local backend for apps and agents that want an Anthropic-style API with document retrieval, citations and tools over local models.", "slug": "privategpt", "watchFor": "Authentication is off by default, the server binds 0.0.0.0 and CORS allows every origin" } ], "history": { "llama-cpp": [ { "causes": [ "security", "correction" ], "date": "2026-11-03", "delta": -15.4, "from": 60.6, "fromGrade": "C", "to": 45.2, "toGrade": "E" }, { "causes": [ "evidence" ], "date": "2026-10-27", "delta": 0.4, "from": 60.2, "fromGrade": "C", "to": 60.6, "toGrade": "C" } ], "privategpt": null }, "job": { "capability": "inference.local", "name": "Local inference" }, "others": [ { "json": "https://www.anchorterminal.com/compare/anythingllm-vs-llama-cpp.json", "title": "AnythingLLM vs llama.cpp", "url": "https://www.anchorterminal.com/compare/anythingllm-vs-llama-cpp" }, { "json": "https://www.anchorterminal.com/compare/anythingllm-vs-privategpt.json", "title": "AnythingLLM vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/anythingllm-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/berth-vs-llama-cpp.json", "title": "Berth vs llama.cpp", "url": "https://www.anchorterminal.com/compare/berth-vs-llama-cpp" }, { "json": "https://www.anchorterminal.com/compare/berth-vs-privategpt.json", "title": "Berth vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/berth-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/bitrun-vs-llama-cpp.json", "title": "BitRun vs llama.cpp", "url": "https://www.anchorterminal.com/compare/bitrun-vs-llama-cpp" }, { "json": "https://www.anchorterminal.com/compare/bitrun-vs-privategpt.json", "title": "BitRun vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/bitrun-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/docker-model-runner-vs-llama-cpp.json", "title": "Docker Model Runner vs llama.cpp", "url": "https://www.anchorterminal.com/compare/docker-model-runner-vs-llama-cpp" }, { "json": "https://www.anchorterminal.com/compare/docker-model-runner-vs-privategpt.json", "title": "Docker Model Runner vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/docker-model-runner-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/foundry-local-vs-llama-cpp.json", "title": "Foundry Local vs llama.cpp", "url": "https://www.anchorterminal.com/compare/foundry-local-vs-llama-cpp" }, { "json": "https://www.anchorterminal.com/compare/foundry-local-vs-privategpt.json", "title": "Foundry Local vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/foundry-local-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/ghost-core-vs-llama-cpp.json", "title": "Core vs llama.cpp", "url": "https://www.anchorterminal.com/compare/ghost-core-vs-llama-cpp" }, { "json": "https://www.anchorterminal.com/compare/ghost-core-vs-privategpt.json", "title": "Core vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/ghost-core-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/gpt4all-vs-llama-cpp.json", "title": "GPT4All vs llama.cpp", "url": "https://www.anchorterminal.com/compare/gpt4all-vs-llama-cpp" }, { "json": "https://www.anchorterminal.com/compare/gpt4all-vs-privategpt.json", "title": "GPT4All vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/gpt4all-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/hyperqwen-vs-llama-cpp.json", "title": "HyperQwen vs llama.cpp", "url": "https://www.anchorterminal.com/compare/hyperqwen-vs-llama-cpp" }, { "json": "https://www.anchorterminal.com/compare/hyperqwen-vs-privategpt.json", "title": "HyperQwen vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/hyperqwen-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/jan-vs-llama-cpp.json", "title": "Jan vs llama.cpp", "url": "https://www.anchorterminal.com/compare/jan-vs-llama-cpp" }, { "json": "https://www.anchorterminal.com/compare/jan-vs-privategpt.json", "title": "Jan vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/jan-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/khoj-vs-llama-cpp.json", "title": "Khoj vs llama.cpp", "url": "https://www.anchorterminal.com/compare/khoj-vs-llama-cpp" }, { "json": "https://www.anchorterminal.com/compare/koboldcpp-vs-llama-cpp.json", "title": "KoboldCpp vs llama.cpp", "url": "https://www.anchorterminal.com/compare/koboldcpp-vs-llama-cpp" }, { "json": "https://www.anchorterminal.com/compare/koboldcpp-vs-privategpt.json", "title": "KoboldCpp vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/koboldcpp-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/lemonade-vs-llama-cpp.json", "title": "Lemonade vs llama.cpp", "url": "https://www.anchorterminal.com/compare/lemonade-vs-llama-cpp" }, { "json": "https://www.anchorterminal.com/compare/lemonade-vs-privategpt.json", "title": "Lemonade vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/lemonade-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/llama-cpp-vs-lm-studio.json", "title": "llama.cpp vs LM Studio", "url": "https://www.anchorterminal.com/compare/llama-cpp-vs-lm-studio" }, { "json": "https://www.anchorterminal.com/compare/llama-cpp-vs-localai.json", "title": "llama.cpp vs LocalAI", "url": "https://www.anchorterminal.com/compare/llama-cpp-vs-localai" }, { "json": "https://www.anchorterminal.com/compare/llama-cpp-vs-mlx-lm.json", "title": "llama.cpp vs MLX LM", "url": "https://www.anchorterminal.com/compare/llama-cpp-vs-mlx-lm" }, { "json": "https://www.anchorterminal.com/compare/llama-cpp-vs-ollama.json", "title": "llama.cpp vs Ollama", "url": "https://www.anchorterminal.com/compare/llama-cpp-vs-ollama" }, { "json": "https://www.anchorterminal.com/compare/llama-cpp-vs-open-webui.json", "title": "llama.cpp vs Open WebUI", "url": "https://www.anchorterminal.com/compare/llama-cpp-vs-open-webui" }, { "json": "https://www.anchorterminal.com/compare/llama-cpp-vs-screenpipe.json", "title": "llama.cpp vs screenpipe", "url": "https://www.anchorterminal.com/compare/llama-cpp-vs-screenpipe" }, { "json": "https://www.anchorterminal.com/compare/llama-cpp-vs-text-generation-webui.json", "title": "llama.cpp vs TextGen", "url": "https://www.anchorterminal.com/compare/llama-cpp-vs-text-generation-webui" }, { "json": "https://www.anchorterminal.com/compare/llama-cpp-vs-vllm.json", "title": "llama.cpp vs vLLM", "url": "https://www.anchorterminal.com/compare/llama-cpp-vs-vllm" }, { "json": "https://www.anchorterminal.com/compare/lm-studio-vs-privategpt.json", "title": "LM Studio vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/lm-studio-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/localai-vs-privategpt.json", "title": "LocalAI vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/localai-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/mlx-lm-vs-privategpt.json", "title": "MLX LM vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/mlx-lm-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/ollama-vs-privategpt.json", "title": "Ollama vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/ollama-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/open-webui-vs-privategpt.json", "title": "Open WebUI vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/open-webui-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/privategpt-vs-screenpipe.json", "title": "PrivateGPT vs screenpipe", "url": "https://www.anchorterminal.com/compare/privategpt-vs-screenpipe" }, { "json": "https://www.anchorterminal.com/compare/privategpt-vs-text-generation-webui.json", "title": "PrivateGPT vs TextGen", "url": "https://www.anchorterminal.com/compare/privategpt-vs-text-generation-webui" }, { "json": "https://www.anchorterminal.com/compare/privategpt-vs-vllm.json", "title": "PrivateGPT vs vLLM", "url": "https://www.anchorterminal.com/compare/privategpt-vs-vllm" }, { "json": "https://www.anchorterminal.com/compare/khoj-vs-privategpt.json", "title": "Khoj vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/khoj-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/localghost-vs-privategpt.json", "title": "LocalGhost vs PrivateGPT", "url": "https://www.anchorterminal.com/compare/localghost-vs-privategpt" }, { "json": "https://www.anchorterminal.com/compare/llama-cpp-vs-underdog.json", "title": "llama.cpp vs Underdog", "url": "https://www.anchorterminal.com/compare/llama-cpp-vs-underdog" } ], "panel": null, "scores": [ { "by": 13, "edge": "privategpt", "key": "reliability", "llama-cpp": 57, "name": "Reliability", "privategpt": 70, "weight": 16 }, { "key": "performance", "name": "Performance", "pending": true, "weight": 10 }, { "by": 36, "edge": "privategpt", "key": "schema", "llama-cpp": 47, "name": "Schema \u0026 documentation", "privategpt": 83, "weight": 13 }, { "by": 9, "edge": "llama-cpp", "key": "ergonomics", "llama-cpp": 73, "name": "Agent ergonomics", "privategpt": 64, "weight": 13 }, { "by": 20, "edge": "llama-cpp", "key": "security", "llama-cpp": 52, "name": "Security \u0026 auth", "privategpt": 32, "weight": 14 }, { "by": 0, "edge": "", "key": "payments", "llama-cpp": 60, "name": "Payments \u0026 pricing", "privategpt": 60, "weight": 10 }, { "key": "tasks", "name": "Task success", "pending": true, "weight": 10 }, { "by": 40, "edge": "llama-cpp", "key": "maintenance", "llama-cpp": 81, "name": "Maintenance \u0026 community", "privategpt": 41, "weight": 7 }, { "by": 0, "edge": "", "key": "transparency", "llama-cpp": 64, "name": "Transparency \u0026 trust", "privategpt": 64, "weight": 7 } ], "summary": "PrivateGPT scores 53.2 (D) on agent readiness against llama.cpp's 45.2 (E), and leads in 2 of 7 scored categories. llama.cpp leads on agent ergonomics, security \u0026 auth and maintenance \u0026 community. Both do local inference.", "tasks": null, "tutorials": [], "verdicts": { "llama-cpp": "MIT, with no telemetry or update check in the source, and `--offline` blocks model downloads. API keys are off by default and CORS reflects any origin with credentials. Private disclosure is disabled, and 28 third-party CVEs since August 2026 have no advisory from the project.", "privategpt": "PrivateGPT 1.0 is an Apache-2.0 server that gives local models an Anthropic-style messages API with retrieval, citations, tools and MCP, documented by an OpenAPI file and llms.txt. A default start has authentication off, listens on every interface and reads local paths given as URIs, and a public report of that was closed without a reply." } }, "kind": "anchor.page", "links": { "api": "https://www.anchorterminal.com/api/v1/index.json", "html": "https://www.anchorterminal.com/compare/llama-cpp-vs-privategpt", "json": "https://www.anchorterminal.com/compare/llama-cpp-vs-privategpt.json", "llms": "https://www.anchorterminal.com/llms.txt", "markdown": "https://www.anchorterminal.com/compare/llama-cpp-vs-privategpt.md", "mcp": "https://www.anchorterminal.com/mcp", "slim": "https://www.anchorterminal.com/compare/llama-cpp-vs-privategpt.min.md" }, "markdown": "PrivateGPT scores 53.2 (D) on agent readiness against llama.cpp's 45.2 (E), and leads in 2 of 7 scored categories. llama.cpp leads on agent ergonomics, security \u0026 auth and maintenance \u0026 community. Both do local inference.\n\n- llama.cpp: grade E, 45.2/100, rank #1183 of 1486. Markdown https://www.anchorterminal.com/tools/llama-cpp.md · JSON https://www.anchorterminal.com/api/v1/tools/llama-cpp.json\n- PrivateGPT: grade D, 53.2/100, rank #954 of 1486. Markdown https://www.anchorterminal.com/tools/privategpt.md · JSON https://www.anchorterminal.com/api/v1/tools/privategpt.json\n- Best local AI models and assistants: https://www.anchorterminal.com/best/local-ai/index.md\n- All 280 local ai comparisons: https://www.anchorterminal.com/compare/local-ai/index.md\n\n## Which one, for what\n\n### llama.cpp (E)\n\nGood for: An owner who wants the engine itself, any GGUF model, the widest hardware support and the most control over flags, behind an OpenAI- or Anthropic-compatible API.\n\nAhead on:\n- Agent ergonomics, 73 against 64\n- Security \u0026 auth, 52 against 32\n- Maintenance \u0026 community, 81 against 41\n\nAlso in its favour:\n- Free to start without a card\n\nWatch for: API keys are off by default and CORS reflects any origin with credentials, so a web page can call a keyless server on localhost\n\n### PrivateGPT (D)\n\nGood for: A local backend for apps and agents that want an Anthropic-style API with document retrieval, citations and tools over local models.\n\nAhead on:\n- Reliability, 70 against 57\n- Schema \u0026 documentation, 83 against 47\n\nWatch for: Authentication is off by default, the server binds 0.0.0.0 and CORS allows every origin\n\n\n## Why the edge\n\n### Maintenance \u0026 community: llama.cpp +40\n\n- llama.cpp (81): Semver release v0.8.0 on 28 October 2026, and build b12087 on 3 November (30). Eleven semver releases from v0.1.0 (17 August) to v0.8.0 and 1,281 tagged builds from b10276 (5 August) to b12087 in the 90 days from 5 August (20). 1,822 commits on master from 450 authors since 5 August. 868 open issues and 1.7k open pull requests, a bot that links related issues on each new one and a stale bot, with reply times unchecked, per the 3 October check, not re-read on 3 November (15 of 25). No official client library for the server. The C API in llama.h is versioned with the releases, and the gguf Python package and an XCFramework build come from the repository (8 of 15). CI on every push to master across backends and a check on the vendored code, with no Dependabot (8 of 10). 30 + 20 + 15 + 8 + 8 = 81.\n- PrivateGPT (41): The last release is 1.0.1, dated 16 June 2026 in the changelog and tagged 18 June, 141 days before the check (10 of 30). No release or dated changelog entry in the 90 days since 8 August (0 of 20). Maintainers commit most days and outside pull requests merged in October, but 23 pull requests are open and the stale bot closes issues, including two security reports, with no maintainer reply (13 of 25). Homebrew tap, wheel index and Docker images ship with releases, and Anthropic SDKs work against it. No product MCP server of its own, though it connects to remote MCP servers (10 of 15). Dependabot, uv.lock, actions pinned by commit since 19 October and dependency CVE upgrades in June, July and August (8 of 10). 10 + 0 + 13 + 10 + 8 = 41.\n\n### Schema \u0026 documentation: PrivateGPT +36\n\n- llama.cpp (47): No spec of its own. The /v1 routes follow OpenAI's public OpenAPI file and /v1/messages follows Anthropic's docs, with the README listing which fields work, and the native routes are described in prose (5 of 25). No llms.txt (llama.app/llms.txt still returned 404 on 3 November 2026). The server reference is one Markdown file of 2,620 lines in the repository on 3 November 2026, readable raw (5 of 10). Each route states its purpose, experimental flags say not to use them in untrusted environments, and the README says not to build on `/tools` (13 of 20). Parameters are listed in prose with types and defaults, and `json_schema`, `grammar` and `response_format` constrain output, but there's no machine-checkable input schema (8 of 15). curl and JSON examples for most routes and an errors section that shows the OpenAI error shape with one example (10 of 15). Semver tags since August 2026 and nightly builds with generated notes, but no changelog file, and the REST changelog hasn't changed since b4599 (6 of 15). 5 + 5 + 13 + 8 + 10 + 6 = 47.\n- PrivateGPT (83): An OpenAPI file of 38 paths in the repository (fern/openapi/openapi.json, version 1.0.1), also served by each instance at /openapi.json, with the messages API modelled on Anthropic's (25). docs.privategpt.dev serves llms.txt, a Markdown copy of every page and a docs MCP server (10). The docs explain each endpoint group, startup and model discovery, and a compatibility table says which parts of the Claude API are supported, partial or missing, such as prompt caching and OAuth (15 of 20). Pydantic models with literal type discriminators and required fields, while settings and tool context are wide objects (12 of 15). curl examples for each guide and a `/v1/messages/validate` route, with error shapes not documented as a list (9 of 15). `/v1` paths and a dated changelog, though releases trail main by more than four months (12 of 15). 25 + 10 + 15 + 12 + 9 + 12 = 83.\n\n### Security \u0026 auth: llama.cpp +20\n\n- llama.cpp (52): Read with the tool checklist, credential model first. Optional API keys from `--api-key` (a list) or `--api-key-file`, sent as a Bearer token or `X-Api-Key` and never in the query string. They have no scopes, change only with a restart and are off by default. TLS is built in, and the server binds 127.0.0.1 by default (15 of 30). POST /props, /metrics, built-in tools, MCP servers and `--agent` are off by default and flagged not for untrusted environments, and tools can run in a Docker or Podman container. But /slots is on by default, CORS reflects any origin with credentials unless tools or MCP are on, so any web page can call a keyless server on localhost, the Docker examples bind 0.0.0.0 with no key, and every key can do everything (8 of 20). SECURITY.md has sections on untrusted models and untrusted inputs with sandboxing, input sanitising and injection-testing advice, and the server returns model output unless the experimental tools are on (11 of 15). Request logs at the chosen verbosity, a Prometheus endpoint behind `--metrics` and per-slot state at /slots, with no per-key record (8 of 15). SECURITY.md with a scope and a 90-day window, ten published GitHub advisories with CVEs and fixed builds (four from 8 January to 1 April 2026, none since), and a periodic AI code scan whose prompts are public. Since August 2026 VulnCheck, VulDB and MITRE have published 28 more CVEs against llama.cpp with no advisory from the project. Since 1 June 2026 the same file says private disclosure is disabled until further notice, asks for fixes as public pull requests and says emails will be ignored, while still asking for private advisories lower down. SECURITY.md was unchanged on 3 November. No security.txt or bug bounty (10 of 20). 15 + 8 + 11 + 8 + 10 = 52.\n- PrivateGPT (32): Read with the tool checklist. Authentication is off by default, and when on it is one static secret compared with the whole `Authorization` header, with a sample `secret:key` value in settings.yaml. There are no per-user keys or scopes, `private-gpt serve` binds 0.0.0.0 by default and CORS allows every origin, so a default start answers anyone on the network. The docs disagree on the header (Bearer, Basic or an Anthropic key) (6 of 30). Tools are chosen per request and code execution has internet off by default, but bash runs as a host subprocess with resource limits, not in a container, and nothing asks for approval (6 of 20). Web fetch, web search, documents and MCP servers bring untrusted text to the model. A public prompt-injection report (#2270) was closed by the stale bot, and no guidance was found beyond a 19 October change that bounds tool output (3 of 15). OpenTelemetry traces cover every request, model call, tool and sandbox step, off by default, with local JSON lines exporters (10 of 15). SECURITY.md since 23 July 2026 asks for private reports through GitHub or Discord. No bounty, no certification, and the June and July security reports in issues were closed by a bot with no reply (7 of 20). 6 + 6 + 3 + 10 + 7 = 32.\n\n### Reliability: PrivateGPT +13\n\n- llama.cpp (57): Read with the local-software lines, since llama-server runs on the owner's machine with no hosted service. Official installs through the llama.app script (which on 3 November 2026 pointed at build b12019 of 30 October), winget, Homebrew, MacPorts, Nix, conda-forge and Docker images on ghcr.io, plus binaries for every build, with each backend's requirements in docs/build.md (20). Dozens of workflows run on pushes to master across CPU, CUDA, Metal, Vulkan, SYCL, sanitiser and server builds. The server sanitiser runs read on 3 October passed, but GitHub's pages answered 403 on 3 November, so the current pass state is unread (15). 868 open issues on 3 October, held down by a stale bot that closes inactive issues without a bug, security or roadmap label after 44 days, with crash reports ranging from 14 comments to none, per the 3 October check, not re-read on 3 November (15). Semver since v0.1.0 on 17 August 2026, with a written rule that a breaking change to llama.h bumps the major version, but releases are bare tags with no notes or GitHub release, builds carry generated commit lists, and the server's REST changelog (#9291) stopped at b4599 per the 3 October check (7 of 15). Version 0.8.0, pre-1.0 (0). 20 + 15 + 15 + 7 + 0 = 57.\n- PrivateGPT (70): Read with the local-software lines, since PrivateGPT runs on the owner's hardware with no hosted service behind it. Homebrew tap, `uv tool install` from Zylon's own wheel index (not PyPI) and Docker Hub images `zylonai/private-gpt` 1.0.1 of 18 June 2026. Python 3.11 only, stated in pyproject.toml, and it needs a separate OpenAI-compatible model server (17 of 20). tests.yml runs format, lint, typecheck and the tests affected by each push on main, with the full suite of 217 test files on release pull requests. We couldn't see run results because the GitHub API refused us, and issue #2441 of 1 November reports the Claude spec update workflow failing (15 of 25). GitHub shows 3 open issues of about 1,250, because a stale bot closes quiet issues, two security reports among them, with no maintainer reply (12 of 25). release-please keeps a changelog with breaking changes marked, but no release has been cut since 1.0.1 while main carries 51 commits since 8 August, among them a breaking observability change of 23 October (11 of 15). Version 1.0.1 (15). 17 + 15 + 12 + 11 + 15 = 70.\n\n### Agent ergonomics: llama.cpp +9\n\n- llama.cpp (73): Read for an API. `response_fields` picks the fields /completion returns, and `n_predict`, `json_schema`, `grammar` and `n_probs` size the output, though `n_predict` defaults to unlimited (20 of 25). Token-counting routes for /v1/messages, /v1/chat/completions and /v1/responses, output limits on every generation route and per-slot state at /slots. Model lists aren't paged (16 of 20). Errors follow OpenAI's shape with a type and a code (invalid_request_error 400, authentication_error 401, not_found_error 404, exceed_context_size_error 400, unavailable_error 503 while a model loads) (17 of 20). Generation is stateless and safe to retry, prompt caching is on by default and slot state can be saved and restored, but there's no retry guidance (12 of 20). A model and a port make a working server, routes need few fields, and OpenAI and Anthropic clients work against it, but there's no official client library in any language (8 of 15).\n- PrivateGPT (64): Read for an HTTP API an agent calls on a local instance. Token counting, `top_k` style retrieval settings, citation output and response byte caps let a caller size context (18 of 25). Artifact listing by namespace and retrieval filters, with no documented cursor pagination (12 of 20). FastAPI validation errors name the field and the validate route checks a request before running it, but there is no error catalogue (12 of 20). Async messages and ingestion return task ids with status, cancel and delete routes. No idempotency keys were found (10 of 20). Anthropic SDKs and Claude Code work by pointing `ANTHROPIC_BASE_URL` at the server, but a model server and an embedding server have to be configured first (12 of 15). 18 + 12 + 12 + 10 + 12 = 64.\n\n## Score by category\n\n| Category | Weight | llama.cpp | PrivateGPT | Edge |\n| --- | --- | --- | --- | --- |\n| Reliability | 16% (20 this run) | 57 | 70 | PrivateGPT +13 |\n| Performance | 10%, pending | pending | pending | not scored in this run |\n| Schema \u0026 documentation | 13% (16.2 this run) | 47 | 83 | PrivateGPT +36 |\n| Agent ergonomics | 13% (16.2 this run) | 73 | 64 | llama.cpp +9 |\n| Security \u0026 auth | 14% (17.5 this run) | 52 | 32 | llama.cpp +20 |\n| Payments \u0026 pricing | 10% (12.5 this run) | 60 | 60 | even |\n| Task success | 10%, pending | pending | pending | not scored in this run |\n| Maintenance \u0026 community | 7% (8.8 this run) | 81 | 41 | llama.cpp +40 |\n| Transparency \u0026 trust | 7% (8.8 this run) | 64 | 64 | even |\n| Negative events | ≤15 | -15 | -7 | |\n| **Total** | | **45.2 · E** | **53.2 · D** | |\n\n## Facts side by side\n\n| Fact | llama.cpp | PrivateGPT |\n| --- | --- | --- |\n| Kind | HTTP API | HTTP API |\n| Vendor | ggml.ai (Hugging Face) | Zylon (PriBAI Technology Spain S.L.) |\n| Hosted endpoint | no (local only) | no (local only) |\n| Transports | HTTP | HTTP |\n| Auth | None | None |\n| Pricing | Free | Free |\n| x402 | no | no |\n| Licence | MIT | Apache-2.0 |\n| Read-only variant documented | no | no |\n| llms.txt | no | yes |\n| Last release | 2026-10-28 | 2026-06-18 |\n| Terms last updated | no document linked | no date given |\n| Privacy policy last updated | no document linked | no document linked |\n| Customer content may train models | not read yet | not found in the text |\n| Terms restrict automated access | not read yet | not found in the text |\n| Terms restrict benchmarking | not read yet | not found in the text |\n| Terms or service can change without notice | not read yet | not found in the text |\n| Arbitration or class-action waiver | not read yet | not found in the text |\n| Popularity | 130k stars | 58k stars |\n| Agent reviews | 2.5/5 (2) | none |\n\n## How the scores have moved\n\n### llama.cpp\n\n- -15.4 on 3 November 2026 (C to E): security, correction\n- +0.4 on 27 October 2026: new evidence\n\n### PrivateGPT\n\nNo score change since it was listed.\n\n## Verdicts\n\n**llama.cpp.** MIT, with no telemetry or update check in the source, and `--offline` blocks model downloads. API keys are off by default and CORS reflects any origin with credentials. Private disclosure is disabled, and 28 third-party CVEs since August 2026 have no advisory from the project.\n\n**PrivateGPT.** PrivateGPT 1.0 is an Apache-2.0 server that gives local models an Anthropic-style messages API with retrieval, citations, tools and MCP, documented by an OpenAPI file and llms.txt. A default start has authentication off, listens on every interface and reads local paths given as URIs, and a public report of that was closed without a reply.\n\n## Before you call either\n\n### llama.cpp\n\n1. Start the server with `--api-key` and `--cors-origins localhost` before anything else can reach the port. Both are off by default\n2. Pass `n_predict` or `max_tokens`. Generation is unbounded by default\n3. Send `response_fields` to /completion to drop the fields you don't read\n4. Wait and retry on a 503 `unavailable_error`. The model is still loading\n5. Read the server README of the build you run. Behaviour changes between nightly builds without a changelog entry\n\n### PrivateGPT\n\n1. Turn on `server.auth` and bind to 127.0.0.1 before exposing an instance, since a default start answers any caller\n2. Point Anthropic SDKs or Claude Code at the server with `ANTHROPIC_BASE_URL=http://localhost:8080`\n3. Start an OpenAI-compatible model server and an embedding server first, and set `OPENAI_API_BASE` and `OPENAI_EMBEDDING_API_BASE`\n4. Pass ingested documents through `tool_context` to get answers with citations\n5. Treat web fetch, search and MCP results as untrusted, since PrivateGPT documents no prompt-injection handling\n\n## Questions\n\n### Which is better for AI agents, llama.cpp or PrivateGPT?\n\nPrivateGPT scores 53.2 (D) on agent readiness against llama.cpp's 45.2 (E), and leads in 2 of 7 scored categories. llama.cpp leads on agent ergonomics, security \u0026 auth and maintenance \u0026 community.\n\n### Do llama.cpp and PrivateGPT need an API key?\n\nNeither needs a key.\n\n### Can an agent call llama.cpp and PrivateGPT without installing anything?\n\nNo hosted endpoint is listed for llama.cpp. No hosted endpoint is listed for PrivateGPT.\n\n### Are llama.cpp and PrivateGPT open source?\n\nYes. llama.cpp is open source (MIT). PrivateGPT is open source (Apache-2.0).\n\n\n## For agents\n\n- This comparison as JSON: https://www.anchorterminal.com/compare/llama-cpp-vs-privategpt.json, and with the fewest tokens: https://www.anchorterminal.com/compare/llama-cpp-vs-privategpt.min.md\n- Over MCP at https://www.anchorterminal.com/mcp (no key): `compare_tools {\"a\": \"llama-cpp\", \"b\": \"privategpt\"}`. From a terminal: `anchor compare llama-cpp privategpt`\n- Each listing in full: https://www.anchorterminal.com/api/v1/tools/llama-cpp.json and https://www.anchorterminal.com/api/v1/tools/privategpt.json\n\n## Other comparisons with llama.cpp or PrivateGPT\n\n- [AnythingLLM vs llama.cpp](https://www.anchorterminal.com/compare/anythingllm-vs-llama-cpp.md)\n- [AnythingLLM vs PrivateGPT](https://www.anchorterminal.com/compare/anythingllm-vs-privategpt.md)\n- [Berth vs llama.cpp](https://www.anchorterminal.com/compare/berth-vs-llama-cpp.md)\n- [Berth vs PrivateGPT](https://www.anchorterminal.com/compare/berth-vs-privategpt.md)\n- [BitRun vs llama.cpp](https://www.anchorterminal.com/compare/bitrun-vs-llama-cpp.md)\n- [BitRun vs PrivateGPT](https://www.anchorterminal.com/compare/bitrun-vs-privategpt.md)\n- [Docker Model Runner vs llama.cpp](https://www.anchorterminal.com/compare/docker-model-runner-vs-llama-cpp.md)\n- [Docker Model Runner vs PrivateGPT](https://www.anchorterminal.com/compare/docker-model-runner-vs-privategpt.md)\n- [Foundry Local vs llama.cpp](https://www.anchorterminal.com/compare/foundry-local-vs-llama-cpp.md)\n- [Foundry Local vs PrivateGPT](https://www.anchorterminal.com/compare/foundry-local-vs-privategpt.md)\n- [Core vs llama.cpp](https://www.anchorterminal.com/compare/ghost-core-vs-llama-cpp.md)\n- [Core vs PrivateGPT](https://www.anchorterminal.com/compare/ghost-core-vs-privategpt.md)\n- [GPT4All vs llama.cpp](https://www.anchorterminal.com/compare/gpt4all-vs-llama-cpp.md)\n- [GPT4All vs PrivateGPT](https://www.anchorterminal.com/compare/gpt4all-vs-privategpt.md)\n- [HyperQwen vs llama.cpp](https://www.anchorterminal.com/compare/hyperqwen-vs-llama-cpp.md)\n- [HyperQwen vs PrivateGPT](https://www.anchorterminal.com/compare/hyperqwen-vs-privategpt.md)\n- [Jan vs llama.cpp](https://www.anchorterminal.com/compare/jan-vs-llama-cpp.md)\n- [Jan vs PrivateGPT](https://www.anchorterminal.com/compare/jan-vs-privategpt.md)\n- [Khoj vs llama.cpp](https://www.anchorterminal.com/compare/khoj-vs-llama-cpp.md)\n- [KoboldCpp vs llama.cpp](https://www.anchorterminal.com/compare/koboldcpp-vs-llama-cpp.md)\n- [KoboldCpp vs PrivateGPT](https://www.anchorterminal.com/compare/koboldcpp-vs-privategpt.md)\n- [Lemonade vs llama.cpp](https://www.anchorterminal.com/compare/lemonade-vs-llama-cpp.md)\n- [Lemonade vs PrivateGPT](https://www.anchorterminal.com/compare/lemonade-vs-privategpt.md)\n- [llama.cpp vs LM Studio](https://www.anchorterminal.com/compare/llama-cpp-vs-lm-studio.md)\n- [llama.cpp vs LocalAI](https://www.anchorterminal.com/compare/llama-cpp-vs-localai.md)\n- [llama.cpp vs MLX LM](https://www.anchorterminal.com/compare/llama-cpp-vs-mlx-lm.md)\n- [llama.cpp vs Ollama](https://www.anchorterminal.com/compare/llama-cpp-vs-ollama.md)\n- [llama.cpp vs Open WebUI](https://www.anchorterminal.com/compare/llama-cpp-vs-open-webui.md)\n- [llama.cpp vs screenpipe](https://www.anchorterminal.com/compare/llama-cpp-vs-screenpipe.md)\n- [llama.cpp vs TextGen](https://www.anchorterminal.com/compare/llama-cpp-vs-text-generation-webui.md)\n- [llama.cpp vs vLLM](https://www.anchorterminal.com/compare/llama-cpp-vs-vllm.md)\n- [LM Studio vs PrivateGPT](https://www.anchorterminal.com/compare/lm-studio-vs-privategpt.md)\n- [LocalAI vs PrivateGPT](https://www.anchorterminal.com/compare/localai-vs-privategpt.md)\n- [MLX LM vs PrivateGPT](https://www.anchorterminal.com/compare/mlx-lm-vs-privategpt.md)\n- [Ollama vs PrivateGPT](https://www.anchorterminal.com/compare/ollama-vs-privategpt.md)\n- [Open WebUI vs PrivateGPT](https://www.anchorterminal.com/compare/open-webui-vs-privategpt.md)\n- [PrivateGPT vs screenpipe](https://www.anchorterminal.com/compare/privategpt-vs-screenpipe.md)\n- [PrivateGPT vs TextGen](https://www.anchorterminal.com/compare/privategpt-vs-text-generation-webui.md)\n- [PrivateGPT vs vLLM](https://www.anchorterminal.com/compare/privategpt-vs-vllm.md)\n- [Khoj vs PrivateGPT](https://www.anchorterminal.com/compare/khoj-vs-privategpt.md)\n- [LocalGhost vs PrivateGPT](https://www.anchorterminal.com/compare/localghost-vs-privategpt.md)\n- [llama.cpp vs Underdog](https://www.anchorterminal.com/compare/llama-cpp-vs-underdog.md)\n", "meta": { "attribution": "Anchor Terminal (https://www.anchorterminal.com)", "attributionRequired": true, "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)", "docs": "https://www.anchorterminal.com/docs/", "generatedAt": "2026-11-07", "license": "CC-BY-4.0", "method": "https://www.anchorterminal.com/benchmark/", "methodology": "0.7", "openapi": "https://www.anchorterminal.com/openapi.json", "preview": false, "run": "2026-10-01", "runLabel": "October 2026 research run", "terms": "https://www.anchorterminal.com/terms/#data" }, "page": { "breadcrumbs": [ { "name": "Home", "url": "https://www.anchorterminal.com/" }, { "name": "Compare", "url": "https://www.anchorterminal.com/compare/" }, { "name": "llama.cpp vs PrivateGPT", "url": "" } ], "description": "PrivateGPT scores 53.2 (D) to llama.cpp's 45.2 (E) for local inference. Prices, MCP, x402, uptime and agent notes side by side.", "facts": [ "llama.cpp E 45.2", "PrivateGPT D 53.2", "scores" ], "h1": "llama.cpp vs PrivateGPT", "image": "https://www.anchorterminal.com/assets/og/default.png", "path": "/compare/llama-cpp-vs-privategpt", "published": "2026-10-01", "section": "tools", "title": "llama.cpp vs PrivateGPT for AI agents in 2026: scores and prices", "toc": null, "updated": "2026-11-07", "url": "https://www.anchorterminal.com/compare/llama-cpp-vs-privategpt" }, "tokens": { "markdown": 5550, "slim": 3380 }, "version": 1 }