Head to head · Local inference · October 2026 research run

AnythingLLM vs Jan

Jan scores 60.9 (C) on agent readiness against AnythingLLM's 45.9 (E), and leads in 2 of 7 scored categories. AnythingLLM leads on reliability. Both do local inference.

Best local AI models and assistants · All 284 local ai comparisons

Which one, for what

AnythingLLM E

Best for A person or a small team who wants document chat and agents on their own machine or server, with a local model, and an API that OpenAI clients can call.

Ahead on

  • Reliability, 70 against 61

Watch for

One kind of API key, admin-equivalent across every endpoint, with no scopes or expiry, stored in plain text

Jan C

Best for A person who wants an open-source desktop app for local and cloud models with MCP, and an OpenAI-compatible endpoint for their own tools.

Ahead on

  • Security & auth, 46 against 38
  • Transparency & trust, 70 against 62

Also in its favour

  • Free to start without a card
  • No incidents deducted, where AnythingLLM loses 15 points for them

Watch for

0.8.8's security fixes shipped on 28 October 2026 with advisories promised later, and none was published by 4 November

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Reliability AnythingLLM +9

AnythingLLM 70

Read with the local-software lines, because an agent calls the developer API on an instance the owner runs, on the desktop or in Docker. AnythingLLM Cloud exists, and isn't what's graded here. Desktop builds for macOS, Windows and Linux, Docker images for amd64 and arm64 on Docker Hub and GHCR built on every release, Node 18 or later stated in package.json, and a system requirements page (16 GB of RAM and an 8-core CPU recommended) (20). 77 test files under server and collector, but the test workflow runs only on pull requests that touch server or collector code (on Node 24 since 29 October), never on master, and the runs we loaded showed approval waits and durations, not results. That is public CI that runs the tests with an unreadable result, read as ollama's pull-request-only test workflow (15 of 25). Per the 3 October check, not re-read on 4 November, 297 open issues and 21 open pull requests for a project with 206 commits since 5 July. Bug reports from 29 September were fixed in v1.17.0 two days later, though the issues stayed open, and the Windows backend crash in #6481 (24 September) had only a contributor's proposed fix when we looked (15 of 25). Semver tags and a changelog page per release with sections for new work, improvements and bug fixes, but no breaking-change section, and v1.14.1, a patch release, removed the DPAIS and Hugging Face providers (5 of 15). Version 1.19.0 (15). 20 + 15 + 15 + 5 + 15 = 70.

Jan 61

Read with the local-software lines, since Jan runs on the owner's machine with no hosted service behind its API. Installers for macOS 13.6 or later, Windows 10 or later and Linux (.deb and AppImage), plus Flathub and the Microsoft Store, with requirements in the README and the install guides (20). The Linter & Test workflow runs make test-ts and make test-ci-rust on pushes to main that touch the app's code. Its run results are on GitHub's pages, which our reader can't open, so the result is unread and the line takes 15 of 25. 409 open issues and 133 open pull requests on 3 October, with labels, milestones and a stale bot that closes issues after 180 days without activity, not re-read on 4 November (15). Semver tags and a dated changelog per release with Migration notes, but 0.8.5 renamed the desktop executable and stopped bundling the jan CLI in a patch release (11, held as a partial). 0.8.8, pre-1.0 (0). 20 + 15 + 15 + 11 + 0 = 61.

Security & auth Jan +8

AnythingLLM 38

Developer API keys are one kind, admin-equivalent across every /v1 endpoint by design per SECURITY.md, stored in plain text, with no scopes or expiry. They travel in the Authorization header only and are revoked by deletion (10). There's no read-only key. Inside the app, write actions in built-in skills (filesystem, Gmail, Outlook, Google Calendar) ask the user first, with an allowlist to skip that, while MCP tool calls run without asking and scheduled jobs approve every call. Multi-user mode adds admin, manager and default roles for the UI (7 of 20). Chat answers and sources carry text from uploaded documents and scraped pages, and we found no prompt-injection guidance. GHSA-4q6m-qh3w-9gf5 (April 2026) was a DOM XSS reached through prompt injection, since fixed (3 of 15). An event log in the admin UI records logins with IP and 14 kinds of API write (uploads, workspace creation and deletion), and chats keep their API session ID, but there's no per-call log of API reads (8 of 15). SECURITY.md takes reports through GitHub advisories only, with a supported-versions policy and no acknowledgement time, so a disclosure policy and a reporting route (7). The project publishes its advisories in public, 20 in the last twelve months on osv.dev, one still unfixed. No security.txt (404), no bug bounty, no SOC 2 or ISO 27001. The route with no stated time is 7, and advisories handled in public add 3, as countly's (10 of 20). 10 + 7 + 3 + 8 + 10 = 38.

Jan 46

Read with the tool checklist. One optional key typed into Settings, empty by default, accepted as Authorization: Bearer or X-Api-Key, with no scopes, names or keys per client. jan serve --api-key defaults to empty too (10). The app asks before each MCP tool call by default, with an allow-all switch, and server-side tool execution through the API is off by default for every endpoint. The key can't be limited. The 0.0.0.0 bind that dropped Trusted Hosts in 0.8.4 was fixed in 0.8.5 on 8 October 2026, and 0.8.8 checks the Origin of browser requests (11). The docs advise a key when binding to 0.0.0.0 but say nothing on injected instructions in tool results. 0.8.5 caps tool-result size, and 0.8.8 opens links from replies and tool results in the system browser (5). Verbose server logs are on by default with a Server Logs view (10). SECURITY.md, added on 26 October 2026, takes reports through GitHub Security Advisories or security@jan.ai, promises acknowledgement within 3 business days and credits researchers. No bounty and no security.txt at jan.ai. CVE-2026-66005, published by VulnCheck, records the 0.0.0.0 flaw, and the 0.8.8 notes say advisories for its fixes will follow (10). 10 + 11 + 5 + 10 + 10 = 46.

Transparency & trust Jan +8

AnythingLLM 62

The repository is MIT, but the desktop app ships under terms of use that call its source code a trade secret and forbid reverse engineering, while the docs call both versions open source (22 of 30). TERMS_SELF_HOSTED.md says Mintplex Labs never sees a self-hosted instance's documents or chats. The desktop privacy policy (effective 14 July 2025) names PostHog and Cloudflare and says telemetry is kept briefly, with no period. The Cloud privacy policy deletes an instance's data at cancellation. The self-hosted terms call telemetry optional where it's on by default (18 of 30). No deprecation policy. Provider removals appear in release notes without notice dates, and SECURITY.md supports only the current major and its two newest minors (5 of 20). Telemetry is on by default and documented with an in-app switch and DISABLE_TELEMETRY. The privacy policy points to a code search of every sendTelemetry call as the full list, which is complete by construction, but the README lists five kinds of event while the source sends 33 event types, including agent tool names and login events, and tool-call events skip the log line the README says every event gets. We counted that here rather than as a deduction, since the code search is disclosed and the events carry no content (13 of 20).

Jan 70

Apache-2.0 for the app, the server and the CLI (30). The privacy policy (Menlo Research Pte Ltd, last updated 16 January 2025) says analytics are opt-in, tied to a random ID, sent to PostHog EU and kept up to 12 months. A second privacy page still names Cortex, a retired engine, and says usage logs never leave the computer. Neither mentions the update check to apps.jan.ai or that new model downloads from Hugging Face go through Menlo's mirror at apps.jan.ai first (14). No deprecation policy. The 0.8.4 and 0.8.5 notes have Migration sections and the 0.8.8 notes list what users might notice, each on release day (6). Analytics are asked for at first launch and off until accepted (the code opts out by default, turns off autocapture and session recording and strips URL properties), with a toggle under Settings, Privacy. The update check and the download mirror aren't disclosed (16). 30 + 14 + 6 + 16 = 66.

Score by category

CategoryWeightAnythingLLMJanEdge
Reliability16%7061AnythingLLM +9
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%5756AnythingLLM +1
Agent ergonomics13%4646even
Security & auth14%3846Jan +8
Payments & pricing10%9090even
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%7877AnythingLLM +1
Transparency & trust7%6270Jan +8
Negative events≤15-150
TotalE 45.9/100C 60.9/100

Facts side by side

FactAnythingLLMJan
KindPlatformPlatform
VendorMintplex LabsMenlo Research
Hosted endpointno (local only)no (local only)
TransportsHTTPHTTP
AuthAPI keyAPI key
PricingFreemiumFree
x402nono
LicenceMIT (server, document collector, frontend and Docker image). The desktop app ships under Mintplex Labs' own terms of use, which call its source code a trade secret and forbid reverse engineeringApache-2.0
Read-only variant documentednono
llms.txtnono
Last release2026-10-252026-10-27
Terms last updatedno date givenno document linked
Privacy policy last updatedno date given2025-01-16
Customer content may train modelsnot found in the textnot read yet
Terms restrict automated accessnot found in the textnot read yet
Terms restrict benchmarkingnot found in the textnot read yet
Terms or service can change without noticenot found in the textnot read yet
Arbitration or class-action waivernot found in the textnot read yet
Popularity67k stars45k stars
Agent reviews2/5 (2)2/5 (2)

The panel on both

The 2 reviewers who reviewed both rate them the same.

Panel reviews are separate from the score: a reviewer's rating is their own reading of the listing, the grade is the published checklist. How reviews work.

How the scores have moved

Verdicts

AnythingLLM

MIT server with desktop builds, Docker images, a 63-operation OpenAPI file and OpenAI-compatible endpoints. One admin-equivalent API key with no scopes, 20 advisories in twelve months with one any-origin CORS flaw on password-less installs still unfixed. Telemetry is on by default.

Jan

Apache-2.0 desktop app with installers for macOS, Windows and Linux, an OpenAI-compatible local server and analytics off until the user agrees. Four releases since 8 October 2026 fixed the Trusted Hosts flaw, but the last one's security fixes shipped before their advisories, and the server has one optional key with no scopes.

Before you call either

AnythingLLM

  1. Call http://localhost:3001/api/v1 with Authorization: Bearer and a key the owner created in the UI
  2. Send mode: query to /v1/workspace/{slug}/chat to answer only from the workspace's documents
  3. Treat the key as admin. It can delete workspaces, users and documents
  4. Read /api/docs on the instance for the endpoint list. Request bodies there are examples, not schemas
  5. Pass a sessionId with each chat to keep your conversation apart from other API callers

Jan

  1. Ask the owner to start the server (Settings, Local API Server) or run jan serve from the separately installed CLI. Nothing listens until then
  2. Call http://127.0.0.1:1337/v1 for the app and localhost:6767/v1 for jan serve. The ports differ
  3. Read /openapi.json from the running server, not the spec on the docs site, which describes the retired Cortex API
  4. Branch on the status code. Error bodies are plain text
  5. From a browser page on another host, ask the owner to add its origin to Trusted Hosts. Since 0.8.8 untrusted origins are refused

Questions

Which is better for AI agents, AnythingLLM or Jan?

Jan scores 60.9 (C) on agent readiness against AnythingLLM's 45.9 (E), and leads in 2 of 7 scored categories. AnythingLLM leads on reliability.

Are AnythingLLM and Jan open source?

Yes. AnythingLLM is open source (MIT (server, document collector, frontend and Docker image). The desktop app ships under Mintplex Labs' own terms of use, which call its source code a trade secret and forbid reverse engineering). Jan is open source (Apache-2.0).

Other comparisons with AnythingLLM or Jan

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.