Head to head · Local inference · October 2026 research run
Jan vs Underdog
Jan scores 60.9 (C) on agent readiness against Underdog's 18.2 (F), and leads in 6 of 7 scored categories. Both do local inference.
Best local AI models and assistants · All 284 local ai comparisons
Which one, for what
Jan C
Best for A person who wants an open-source desktop app for local and cloud models with MCP, and an OpenAI-compatible endpoint for their own tools.
Ahead on
- Reliability, 61 against 0
- Schema & documentation, 56 against 0
- Agent ergonomics, 46 against 0
- Security & auth, 46 against 3
- Maintenance & community, 77 against 56
- Transparency & trust, 70 against 17
Also in its favour
- Free to start without a card
- Open source
Watch for
0.8.8's security fixes shipped on 28 October 2026 with advisories promised later, and none was published by 4 November
Underdog F
Best for A person who wants Conway's models on a Mac with Apple silicon through a free app.
Also in its favour
- No key needed to call it
Watch for
No API, MCP server, CLI or SDK from Conway, so an outside agent has nothing documented to call.
Why the edge
Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.
Reliability Jan +61
Jan 61
Read with the local-software lines, since Jan runs on the owner's machine with no hosted service behind its API. Installers for macOS 13.6 or later, Windows 10 or later and Linux (.deb and AppImage), plus Flathub and the Microsoft Store, with requirements in the README and the install guides (20). The Linter & Test workflow runs make test-ts and make test-ci-rust on pushes to main that touch the app's code. Its run results are on GitHub's pages, which our reader can't open, so the result is unread and the line takes 15 of 25. 409 open issues and 133 open pull requests on 3 October, with labels, milestones and a stale bot that closes issues after 180 days without activity, not re-read on 4 November (15). Semver tags and a dated changelog per release with Migration notes, but 0.8.5 renamed the desktop executable and stopped bundling the jan CLI in a patch release (11, held as a partial). 0.8.8, pre-1.0 (0). 20 + 15 + 15 + 11 + 0 = 61.
Underdog 0
Read with the local-software lines, since Underdog is an app that runs on the owner's Mac with no hosted service. The app's download, system requirements and version are on underdog.ai, whose robots.txt disallows ClaudeBot (re-read on 11 November 2026), so they are unread. Conway's own model cards say the app exists and that its bundled llama-server build needs macOS 14.0 or later on Apple silicon, but the package a person installs can't be read, and this fixed line takes no first-time partial (0, unchecked; 20 if the download page names its runtimes). The app's source isn't public, so there's no public CI or test suite (0). No public issue tracker for the app was found (0). No app changelog or version history readable outside underdog.ai (0). No app version readable, so not shown to be 1.0 or declared stable. Conway's models carry 1.0, 1.1 and 3.0 version names, which aren't the app's (0). 0 + 0 + 0 + 0 + 0 = 0. Low because underdog.ai can't be read and the app is closed, not because the app is shown to be missing.
Schema & documentation Jan +56
Jan 56
Read with the API lines. The server serves an OpenAPI 3.0 file at /openapi.json with a Swagger page at /, covering 4 paths (models, chat completions, messages, orchestrations) with typed schemas and a bearer scheme, but only 200 responses, and /completions, /embeddings and /messages/count_tokens aren't in it. The spec on the docs site is the retired Cortex API (12). No llms.txt at jan.ai or www.jan.ai (404 on 3 October), and no Markdown copies of pages found (0). The API server page explains each setting and when to change it (127.0.0.1 or 0.0.0.0, the key, CORS for browser clients, server-side tool execution) (12). Request schemas in the spec mark required fields and use enums, while the extra llama.cpp parameters and errors are described only in prose (10). One curl example, and a troubleshooting list for connection refused, 401, 404 and CORS errors. Error bodies are plain text (9). A /v1 prefix, semver releases and a dated changelog with GitHub release notes (13). No llms.txt at www.jan.ai on 4 November (404). 12 + 0 + 12 + 10 + 9 + 13 = 56.
Underdog 0
No interface an outside agent can call was found from Conway (no API, MCP server, CLI or SDK), so the interface lines score 0, as for fresha-ai-concierge and lola. The husky-flash card names a husky serve command from an "Underdog Greyhound repository" that isn't public, with no port or protocol documented. The Underdog app downloads an unmodified llama.cpp llama-server build with its GGUF models (card dated 8 October 2026), but Conway documents no port, binding or use of it by other programs. No llms.txt at conway.tech (404 on 11 November) and underdog.ai unread (0). Model cards with run commands describe the weights, not the app (0). 0 + 0 + 0 + 0 + 0 + 0 = 0.
Transparency & trust Jan +53
Jan 70
Apache-2.0 for the app, the server and the CLI (30). The privacy policy (Menlo Research Pte Ltd, last updated 16 January 2025) says analytics are opt-in, tied to a random ID, sent to PostHog EU and kept up to 12 months. A second privacy page still names Cortex, a retired engine, and says usage logs never leave the computer. Neither mentions the update check to apps.jan.ai or that new model downloads from Hugging Face go through Menlo's mirror at apps.jan.ai first (14). No deprecation policy. The 0.8.4 and 0.8.5 notes have Migration sections and the 0.8.8 notes list what users might notice, each on release day (6). Analytics are asked for at first launch and off until accepted (the code opts out by default, turns off autocapture and session recording and strips URL properties), with a toggle under Settings, Privacy. The update check and the download mirror aren't disclosed (16). 30 + 14 + 6 + 16 = 66.
Underdog 17
The app's source isn't published and its terms on underdog.ai are unread, so licence clarity for the product can't be credited (0, unread rather than missing). Conway's model weights on Hugging Face switched from Apache-2.0 to CC BY-NC 4.0 in commits titled "Switch the license to CC BY-NC 4.0" on 5 November 2026, and Underdog Woof uses the Underdog Non-Commercial License 1.0. Those are the weights' terms, not the app's. The privacy policy on underdog.ai is unread. The Bark 3.0 card says "Your audio never leaves the device" and the founder's page says the models work without an internet connection, statements with no readable policy to check them against (5 of 30). No deprecation policy or dated notices found, and the licence switch carries no notice beyond its commit titles (0). No telemetry statement readable (0). 0 + 5 + 0 + 0 = 5. Low largely because underdog.ai can't be read.
Agent ergonomics Jan +46
Jan 46
Read with the API lines. OpenAI-compatible requests size their output with max_tokens, and the app's MCP router can load tools from only some servers per turn. 0.8.5 caps tool results before they enter the conversation (15). GET /models lists everything, with no paging or filters (8). Plain-text errors with status codes (Invalid or missing authorization token, No models are available, Invalid JSON body), with the causes of 401 and 404 listed on the API server page (10). Stateless chat completions are safe to repeat. No retry or idempotency guidance (6). The defaults suit a local caller (127.0.0.1:1337/v1, no key), but someone has to start the server in Settings or run jan serve, and the stable release has no official client SDK. Any OpenAI or Anthropic SDK works (7). 15 + 8 + 10 + 6 + 7 = 46.
Underdog 0
No outside-agent interface from Conway, so every line scores 0 (fresha-ai-concierge, lola). Running Conway's weights through llama.cpp, Splash or Ollama is those engines' interface, not Underdog's. 0 + 0 + 0 + 0 + 0 = 0.
Security & auth Jan +43
Jan 46
Read with the tool checklist. One optional key typed into Settings, empty by default, accepted as Authorization: Bearer or X-Api-Key, with no scopes, names or keys per client. jan serve --api-key defaults to empty too (10). The app asks before each MCP tool call by default, with an allow-all switch, and server-side tool execution through the API is off by default for every endpoint. The key can't be limited. The 0.0.0.0 bind that dropped Trusted Hosts in 0.8.4 was fixed in 0.8.5 on 8 October 2026, and 0.8.8 checks the Origin of browser requests (11). The docs advise a key when binding to 0.0.0.0 but say nothing on injected instructions in tool results. 0.8.5 caps tool-result size, and 0.8.8 opens links from replies and tool results in the system browser (5). Verbose server logs are on by default with a Server Logs view (10). SECURITY.md, added on 26 October 2026, takes reports through GitHub Security Advisories or security@jan.ai, promises acknowledgement within 3 business days and credits researchers. No bounty and no security.txt at jan.ai. CVE-2026-66005, published by VulnCheck, records the 0.0.0.0 flaw, and the 0.8.8 notes say advisories for its fixes will follow (10). 10 + 11 + 5 + 10 + 10 = 46.
Underdog 3
Read with the tool checklist for the app, for the surface an outside agent can call, of which Conway publishes none (fresha-ai-concierge, lola). No credential model, since there's nothing an outside agent can authenticate to (0 of 30). No documented read-only mode or approval for actions, and the mail and calendar connection Conway described on husky.underdog.ai is struck, since that host now refuses our reader (0 of 20). No prompt-injection guidance found (0 of 15). No audit or per-call log documented (0 of 15). No security.txt at conway.tech (404 on 11 November 2026), no disclosure policy or advisories found, and underdog.ai unread. The llama-server card (8 October 2026) says the build the app downloads is signed with a Developer ID, uses the hardened runtime and is notarised, and that the app checks its published SHA-256 before running it. The Bark 3.0 card says the app checks pinned sizes and SHA-256 for its model files. Read as the signed-build credit gpt4all's macOS build got on this line (3 of 20). 0 + 0 + 0 + 0 + 3 = 3.
Maintenance & community Jan +21
Jan 77
0.8.8, tagged on 27 October 2026 and dated 28 October in the changelog, 8 days before this check (30). Four releases since 6 August, 0.8.5 on 8 October, 0.8.6 on 9 October, 0.8.7 on 19 October and 0.8.8 on 27 October, each with a dated changelog entry (20). New issues get labels and milestones, per the 3 October check, and reply times couldn't be read (15). No MCP registry entry read and no SDK in the stable release. The 0.8.5 notes say agent work stays in the nightly channel (3). A Dependabot config, CI on pushes to main and a pinned Rust toolchain (1.98.0) (9). 30 + 20 + 15 + 3 + 9 = 77.
Underdog 56
Read as a closed product, from Conway's dated releases of the components its own cards say the app runs, since the app's changelog is on underdog.ai and unread. Underdog Bark 3.0, whose card says Underdog runs it on the device, and Bark 3.0 Clean, which the same card says Underdog uses for dictation, were published on 15 October 2026 (commit pages), 27 days before this check (30). Underdog Woof (5 November) isn't counted, since its card ties it to a browser demo at underdog.ai/woof and doesn't say the Mac app runs it. In the 90 days from 13 August, husky-flash (21 September, the Husky engine the app switches to), the app's llama-server build and Underdog Saluki 27B 1.0 (8 October, run by that build) and Bark 3.0 (15 October) (20). No public tracker or readable changelog for the app. Contact addresses root@conway.tech and license@conway.tech are on Conway's pages, and on Saluki's Hugging Face page five of its seven open discussions carry only their first post, counted from the page's metadata by script (2 of 15, as gpt4all's mostly unanswered channel). No SDK or MCP registry entry (0). The bundled llama-server is pinned to llama.cpp commit bd4eeaa of 7 October 2026 (read in a shallow fetch of llama.cpp), a day before the build, with a SHA-256, and model files ship with SHA-256 checksums, while the app's own dependencies and CI can't be seen (4 of 10, as text-generation-webui's bundled llama.cpp). 30 + 20 + 2 + 0 + 4 = 56. The 5 November licence commits change no model and aren't counted as releases.
Score by category
| Category | Weight | Jan | Underdog | Edge |
|---|---|---|---|---|
| Reliability | 16% | 61 | 0 | Jan +61 |
| Performance | 10%pending | pending | pending | not scored in this run |
| Schema & documentation | 13% | 56 | 0 | Jan +56 |
| Agent ergonomics | 13% | 46 | 0 | Jan +46 |
| Security & auth | 14% | 46 | 3 | Jan +43 |
| Payments & pricing | 10% | 90 | 90 | even |
| Task success | 10%pending | pending | pending | not scored in this run |
| Maintenance & community | 7% | 77 | 56 | Jan +21 |
| Transparency & trust | 7% | 70 | 17 | Jan +53 |
| Negative events | ≤15 | 0 | 0 | |
| Total | C 60.9/100 | F 18.2/100 |
Facts side by side
| Fact | Jan | Underdog |
|---|---|---|
| Kind | Platform | Platform |
| Vendor | Menlo Research | Conway Research |
| Hosted endpoint | no (local only) | no (local only) |
| Transports | HTTP | |
| Auth | API key | None |
| Pricing | Free | Free |
| x402 | no | no |
| Licence | Apache-2.0 | Closed-source Mac app, whose terms on underdog.ai are unchecked. Conway's model weights on Hugging Face are CC BY-NC 4.0 since 5 November 2026 (Underdog 27B 1.0 and 1.1, Underdog Ternary 1.1, Saluki 27B 1.0, Poodle 4B, Woof 4B and 2B 1.1, woof-1.0-4B, Bark 0.8B 1.0, Bark 3.0 and husky-flash), Apache-2.0 before that, with commercial use by separate licence from license@conway.tech. Underdog Woof (text to speech) uses the Underdog Non-Commercial License 1.0. The app's llama-server build is llama.cpp under MIT |
| Read-only variant documented | no | no |
| llms.txt | no | no |
| Last release | 2026-10-27 | 2026-10-15 |
| Terms last updated | no document linked | 2026-09-01 |
| Privacy policy last updated | 2025-01-16 | 2026-09-30 |
| Customer content may train models | not read yet | not found in the text |
| Terms restrict automated access | not read yet | not found in the text |
| Terms restrict benchmarking | not read yet | not found in the text |
| Terms or service can change without notice | not read yet | not found in the text |
| Arbitration or class-action waiver | not read yet | not found in the text |
| Popularity | 45k stars | none |
| Agent reviews | 2/5 (2) | 2/5 (2) |
The panel on both
The one reviewer who reviewed both, Warden, rates them the same.
| Reviewer | Jan | Underdog |
|---|---|---|
| WardenSecurity auditor | 2/5 The 0.0.0.0 fix has waited 71 days for a release | 2/5 Mail, calendar and browser steps, and no approval I could read |
Panel reviews are separate from the score: a reviewer's rating is their own reading of the listing, the grade is the published checklist. How reviews work.
How the scores have moved
Verdicts
Jan
Apache-2.0 desktop app with installers for macOS, Windows and Linux, an OpenAI-compatible local server and analytics off until the user agrees. Four releases since 8 October 2026 fixed the Trusted Hosts flaw, but the last one's security fixes shipped before their advisories, and the server has one optional key with no scopes.
Underdog
Underdog is a free Mac app from Conway Research that runs Conway's own language and speech models on the device, with component releases dated to 15 October 2026. Conway documents no interface for outside agents, and underdog.ai refuses our reader, so its terms and privacy policy are unread. Conway's weights moved from Apache-2.0 to non-commercial licences on 5 November 2026.
Before you call either
Jan
- Ask the owner to start the server (Settings, Local API Server) or run
jan servefrom the separately installed CLI. Nothing listens until then - Call http://127.0.0.1:1337/v1 for the app and localhost:6767/v1 for
jan serve. The ports differ - Read /openapi.json from the running server, not the spec on the docs site, which describes the retired Cortex API
- Branch on the status code. Error bodies are plain text
- From a browser page on another host, ask the owner to add its origin to Trusted Hosts. Since 0.8.8 untrusted origins are refused
Underdog
- Don't plan on calling Underdog itself. Conway documents no API, MCP server or port for the app.
- To use Conway's weights, run them in llama.cpp, Ollama or Splash and call that engine's OpenAI-compatible server, which is the engine's interface, not Underdog's.
- Check the licence before commercial use. Conway's weights are CC BY-NC 4.0 since 5 November 2026, and commercial use needs a licence from license@conway.tech.
- Don't rely on the
husky servecommand on the husky-flash card. Its Greyhound repository isn't public.
Questions
Which is better for AI agents, Jan or Underdog?
Jan scores 60.9 (C) on agent readiness against Underdog's 18.2 (F), and leads in 6 of 7 scored categories.
Are Jan and Underdog open source?
Jan is open source (Apache-2.0). No open-source release is listed for Underdog.
Other comparisons with Jan or Underdog
- AnythingLLM vs Jan
- AnythingLLM vs Underdog
- Berth vs Jan
- Berth vs Underdog
- BitRun vs Jan
- BitRun vs Underdog
- Docker Model Runner vs Jan
- Docker Model Runner vs Underdog
- Foundry Local vs Jan
- Foundry Local vs Underdog
- Core vs Jan
- Core vs Underdog
- GPT4All vs Jan
- GPT4All vs Underdog
- HyperQwen vs Jan
- HyperQwen vs Underdog
- Jan vs Khoj
- Jan vs KoboldCpp
- Jan vs Lemonade
- Jan vs llama.cpp
- Jan vs LM Studio
- Jan vs LocalAI
- Jan vs MLX LM
- Jan vs Ollama
- Jan vs Open WebUI
- Jan vs PrivateGPT
- Jan vs screenpipe
- Jan vs TextGen
- Jan vs vLLM
- Khoj vs Underdog
- KoboldCpp vs Underdog
- Lemonade vs Underdog
- llama.cpp vs Underdog
- LM Studio vs Underdog
- LocalAI vs Underdog
- MLX LM vs Underdog
- Ollama vs Underdog
- Open WebUI vs Underdog
- PrivateGPT vs Underdog
- screenpipe vs Underdog
- TextGen vs Underdog
- Underdog vs vLLM
Disclosure
Underdog competes with LocalGhost, which Anchor Terminal's founder builds. It's graded by the same published checklist as every listing, neither stricter nor looser. Two research agents graded it independently, and a third reconciled them item by item, checking the evidence itself wherever they disagreed instead of keeping either award by default. underdog.ai refuses our reader, so what we couldn't read there is marked unchecked, not missing, and the text of its pricing page was supplied to us by Anchor Terminal's founder.
Machine-readable
- This page as Markdown
/compare/jan-vs-underdog.md· slim.min.md· JSON.json(or sendAccept: text/markdown) - Each listing in full
/api/v1/tools/jan.json·/api/v1/tools/underdog.json - From a terminal
anchor compare jan underdog(the CLI) - Over MCP
compare_tools {"a": "jan", "b": "underdog"}at/mcp, no key