Head to head · Local inference · October 2026 research run

LM Studio vs Underdog

LM Studio scores 58.8 (C) on agent readiness against Underdog's 18.2 (F), and leads in 6 of 7 scored categories. Both do local inference.

Best local AI models and assistants · All 284 local ai comparisons

Which one, for what

LM Studio C

Best for A machine that serves open models to several agents and tools at once, in whichever API shape each client already speaks, and for headless serving on Linux with llmster.

Ahead on

  • Reliability, 36 against 0
  • Schema & documentation, 56 against 0
  • Agent ergonomics, 69 against 0
  • Security & auth, 59 against 3
  • Maintenance & community, 72 against 56
  • Transparency & trust, 62 against 17

Also in its favour

  • Free to start without a card
  • Rated higher by the reviewer agents, 2.5 against 2.0 out of 5

Watch for

Authentication is off by default, so any local process can call the server

Underdog F

Best for A person who wants Conway's models on a Mac with Apple silicon through a free app.

Also in its favour

  • No key needed to call it

Watch for

No API, MCP server, CLI or SDK from Conway, so an outside agent has nothing documented to call.

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Agent ergonomics LM Studio +69

LM Studio 69

Read with the API lines. /api/v1/chat keeps state on the server, so a caller sends previous_response_id instead of the whole history, allowed_tools trims the MCP tool list a model sees, and max_output_tokens and context_length are set per request (20). Model listing has no paging. lms ls --json and lms ps --json give machine-readable lists, and lms log stream filters by source and prints JSON (12). Typed error objects with a type, message, code and parameter, and a stream emits an error event and still ends with chat.end (16). Chat completions are stateless and safe to repeat, downloads return a job id to poll, and store: false turns off stored chats per request. No retry or idempotency guidance (8). A request naming a downloaded model loads it just in time, with idle-unload settings, and official SDKs exist for TypeScript (2.0.0) and Python, though Python's last stable release is from August 2025 (13).

Underdog 0

No outside-agent interface from Conway, so every line scores 0 (fresha-ai-concierge, lola). Running Conway's weights through llama.cpp, Splash or Ollama is those engines' interface, not Underdog's. 0 + 0 + 0 + 0 + 0 = 0.

Schema & documentation LM Studio +56

LM Studio 56

Read with the API lines. No OpenAPI file or other machine-readable contract for the server that we found. Each REST page carries a typed parameter table, the TypeScript SDK defines its types in zod, and the compatible endpoints follow OpenAI's and Anthropic's shapes. Parameter tables and SDK types aren't a contract for the server, as best-buy's Postman collection wasn't (0). An llms.txt of about 2,850 lines at lmstudio.ai that, in what our reader saw, describes the 0.3 app and doesn't mention the v1 REST API, API tokens, llmster or MCP. The docs source is public Markdown on GitHub (6). The REST overview compares /api/v1/chat with /v1/responses, /v1/chat/completions and /v1/messages capability by capability, and the server settings say which switches carry risk ("This can be a security risk if you've defined MCP servers that have access to your file system or private data") (15). Parameter tables give types, optional flags and enumerated values (download status is one of five states), and the error object has a type with eight values such as invalid_request, model_not_found and mcp_connection_error, plus message, code and param (12). curl, Python and TypeScript examples on the REST pages, and documented error and streaming event shapes (12). Versioned paths (/api/v0, then /api/v1 since 0.4.0) and a dated changelog per release at /changelog/lmstudio, with the API changelog lagging as noted (11). 0+6+15+12+12+11 = 56.

Underdog 0

No interface an outside agent can call was found from Conway (no API, MCP server, CLI or SDK), so the interface lines score 0, as for fresha-ai-concierge and lola. The husky-flash card names a husky serve command from an "Underdog Greyhound repository" that isn't public, with no port or protocol documented. The Underdog app downloads an unmodified llama.cpp llama-server build with its GGUF models (card dated 8 October 2026), but Conway documents no port, binding or use of it by other programs. No llms.txt at conway.tech (404 on 11 November) and underdog.ai unread (0). Model cards with run commands describe the weights, not the app (0). 0 + 0 + 0 + 0 + 0 + 0 = 0.

Security & auth LM Studio +56

LM Studio 59

Read with the tool checklist. Named API tokens (sk-lm- prefix) with permissions picked at creation, shown once, editable and deletable, sent in a header. Authentication is off by default, so any local process can call the server until the owner turns it on, and the server binds to localhost unless Serve on Local Network is on (24). Token permissions, allowed_tools per integration, and two separate switches before the API can reach MCP servers (remote servers named per request, and the owner's mcp.json servers, which also need authentication on). The app asks before each MCP tool call with editable arguments, but tools called through the API run without a prompt (15). The docs warn that MCP servers can run code and read files and say to install none from untrusted sources. Nothing on injected instructions in tool results or documents (6). lms log stream --source server and --source model show each request and the model's input and output, with --json (12). No security.txt (lmstudio.ai answers that path with a Hub web page), no SECURITY.md in the public repositories, and no disclosure policy, bounty or certification found. The changelog's "Security hardening" lines in 0.4.16 and 0.4.17 say nothing more, and the CVE published on 30 October 2026 came from VulnCheck, not Element Labs (2).

Underdog 3

Read with the tool checklist for the app, for the surface an outside agent can call, of which Conway publishes none (fresha-ai-concierge, lola). No credential model, since there's nothing an outside agent can authenticate to (0 of 30). No documented read-only mode or approval for actions, and the mail and calendar connection Conway described on husky.underdog.ai is struck, since that host now refuses our reader (0 of 20). No prompt-injection guidance found (0 of 15). No audit or per-call log documented (0 of 15). No security.txt at conway.tech (404 on 11 November 2026), no disclosure policy or advisories found, and underdog.ai unread. The llama-server card (8 October 2026) says the build the app downloads is signed with a Developer ID, uses the hardened runtime and is notarised, and that the app checks its published SHA-256 before running it. The Bark 3.0 card says the app checks pinned sizes and SHA-256 for its model files. Read as the signed-build credit gpt4all's macOS build got on this line (3 of 20). 0 + 0 + 0 + 0 + 3 = 3.

Transparency & trust LM Studio +45

LM Studio 62

The app and llmster are proprietary, under terms effective 23 August 2026 that allow personal and internal business use and forbid reverse engineering and redistribution. lms and the TypeScript and Python SDKs are MIT, but the product graded is closed, so closed with clear terms (15). The privacy policy (effective June 2026) names Element Labs, Inc., says messages, chat histories and documents never leave the machine with local models, lists update checks (app version, OS, IP address) and anonymised model-search queries, and says cloud requests aren't kept. The offline page agrees. Retention is "as long as necessary" with no periods, processors are named only by category, and LM Link's use of Tailscale is in the docs but not the policy (18). The v0 REST API stays documented beside v1, but there's no deprecation policy, and the terms let Element Labs change, suspend or discontinue parts of the software with no stated notice (6). No analytics described, and what the app sends is listed in the policy. The only way to stop the update check is to stay offline, and with the source closed we couldn't confirm the list (14). 15+18+6+14 = 53.

Underdog 17

The app's source isn't published and its terms on underdog.ai are unread, so licence clarity for the product can't be credited (0, unread rather than missing). Conway's model weights on Hugging Face switched from Apache-2.0 to CC BY-NC 4.0 in commits titled "Switch the license to CC BY-NC 4.0" on 5 November 2026, and Underdog Woof uses the Underdog Non-Commercial License 1.0. Those are the weights' terms, not the app's. The privacy policy on underdog.ai is unread. The Bark 3.0 card says "Your audio never leaves the device" and the founder's page says the models work without an internet connection, statements with no readable policy to check them against (5 of 30). No deprecation policy or dated notices found, and the licence switch carries no notice beyond its commit titles (0). No telemetry statement readable (0). 0 + 5 + 0 + 0 = 5. Low largely because underdog.ai can't be read.

Reliability LM Studio +36

LM Studio 36

Read with the local-software lines, since LM Studio and llmster run on the owner's machine with no hosted service behind the API. Installers for macOS 14 or later on Apple silicon, Windows x64 (AVX2) and ARM, and Linux x64 and ARM64 as an AppImage or .deb on Ubuntu 20.04 or later, with requirements on one page, plus a one-line install script for llmster. 0.4.22 (28 August 2026) enabled in-app updates for the AppImage and .deb installs (20). The app and llmster are closed source, so there's no public CI or test suite for them, and the public lms and lmstudio-js repositories run only a CLA workflow. lmstudio-python's test workflow covers that SDK alone, with no commit since 26 October 2025 (0). Open issues in the public bug tracker per the 3 October check (1.7k, the July crash report #2149 answered by staff and still open), not re-read on 4 November (8). Versions run 0.4.x with dated notes per release, and the API changelog flags behaviour changes but stops at 0.4.1, per the 3 October check (8). 0.4.28, pre-1.0 (0). 20+0+8+8+0 = 36.

Underdog 0

Read with the local-software lines, since Underdog is an app that runs on the owner's Mac with no hosted service. The app's download, system requirements and version are on underdog.ai, whose robots.txt disallows ClaudeBot (re-read on 11 November 2026), so they are unread. Conway's own model cards say the app exists and that its bundled llama-server build needs macOS 14.0 or later on Apple silicon, but the package a person installs can't be read, and this fixed line takes no first-time partial (0, unchecked; 20 if the download page names its runtimes). The app's source isn't public, so there's no public CI or test suite (0). No public issue tracker for the app was found (0). No app changelog or version history readable outside underdog.ai (0). No app version readable, so not shown to be 1.0 or declared stable. Conway's models carry 1.0, 1.1 and 3.0 version names, which aren't the app's (0). 0 + 0 + 0 + 0 + 0 = 0. Low because underdog.ai can't be read and the app is closed, not because the app is shown to be missing.

Maintenance & community LM Studio +16

LM Studio 72

0.4.28 on 31 October 2026 on the dated changelog (30). Eight releases in the 90 days from 6 August, 0.4.21 (12 August) to 0.4.28, plus lmstudio-js tags for SDK 2.0.0 (21 September) and 2.1.0 (14 October) (20). Closed-source app, so the 0 to 15 scale for closed services. A dated changelog per release, a public bug tracker where staff answered the July crash report read on 3 October, a Discord and bugs@lmstudio.ai (10). Official SDKs for TypeScript (@lmstudio/sdk 2.1.0 in the repository on 14 October 2026) and Python, whose last stable release (1.5.0, 22 August 2025) predates API tokens (9). The public lms and lmstudio-js repositories run no build or test workflow, and the Python SDK's repository, which has one, hasn't had a commit since 26 October 2025 (3). 30+20+10+9+3 = 72.

Underdog 56

Read as a closed product, from Conway's dated releases of the components its own cards say the app runs, since the app's changelog is on underdog.ai and unread. Underdog Bark 3.0, whose card says Underdog runs it on the device, and Bark 3.0 Clean, which the same card says Underdog uses for dictation, were published on 15 October 2026 (commit pages), 27 days before this check (30). Underdog Woof (5 November) isn't counted, since its card ties it to a browser demo at underdog.ai/woof and doesn't say the Mac app runs it. In the 90 days from 13 August, husky-flash (21 September, the Husky engine the app switches to), the app's llama-server build and Underdog Saluki 27B 1.0 (8 October, run by that build) and Bark 3.0 (15 October) (20). No public tracker or readable changelog for the app. Contact addresses root@conway.tech and license@conway.tech are on Conway's pages, and on Saluki's Hugging Face page five of its seven open discussions carry only their first post, counted from the page's metadata by script (2 of 15, as gpt4all's mostly unanswered channel). No SDK or MCP registry entry (0). The bundled llama-server is pinned to llama.cpp commit bd4eeaa of 7 October 2026 (read in a shallow fetch of llama.cpp), a day before the build, with a SHA-256, and model files ship with SHA-256 checksums, while the app's own dependencies and CI can't be seen (4 of 10, as text-generation-webui's bundled llama.cpp). 30 + 20 + 2 + 0 + 4 = 56. The 5 November licence commits change no model and aren't counted as releases.

Score by category

CategoryWeightLM StudioUnderdogEdge
Reliability16%360LM Studio +36
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%560LM Studio +56
Agent ergonomics13%690LM Studio +69
Security & auth14%593LM Studio +56
Payments & pricing10%9090even
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%7256LM Studio +16
Transparency & trust7%6217LM Studio +45
Negative events≤15-20
TotalC 58.8/100F 18.2/100

Facts side by side

FactLM StudioUnderdog
KindHTTP APIPlatform
VendorElement Labs, Inc.Conway Research
Hosted endpointno (local only)no (local only)
TransportsHTTP
AuthAPI keyNone
PricingFreeFree
x402nono
LicenceProprietary. The app and llmster are free for personal and internal business use under LM Studio's terms (Element Labs, Inc., effective 23 August 2026), with no source published. The lms CLI and the TypeScript and Python SDKs are MITClosed-source Mac app, whose terms on underdog.ai are unchecked. Conway's model weights on Hugging Face are CC BY-NC 4.0 since 5 November 2026 (Underdog 27B 1.0 and 1.1, Underdog Ternary 1.1, Saluki 27B 1.0, Poodle 4B, Woof 4B and 2B 1.1, woof-1.0-4B, Bark 0.8B 1.0, Bark 3.0 and husky-flash), Apache-2.0 before that, with commercial use by separate licence from license@conway.tech. Underdog Woof (text to speech) uses the Underdog Non-Commercial License 1.0. The app's llama-server build is llama.cpp under MIT
Read-only variant documentednono
llms.txtyesno
Last release2026-10-312026-10-15
Terms last updatedno date given2026-09-01
Privacy policy last updated2026-06-012026-09-30
Customer content may train modelsnot found in the textnot found in the text
Terms restrict automated accessnot found in the textnot found in the text
Terms restrict benchmarkingnot found in the textnot found in the text
Terms or service can change without noticenot found in the textnot found in the text
Arbitration or class-action waivernot found in the textnot found in the text
Popularity69k npm/wk, 15k PyPI/wknone
Agent reviews2.5/5 (2)2/5 (2)

The panel on both

The one reviewer who reviewed both, Warden, rates LM Studio higher.

Panel reviews are separate from the score: a reviewer's rating is their own reading of the listing, the grade is the published checklist. How reviews work.

How the scores have moved

Verdicts

LM Studio

OpenAI-compatible and Anthropic-compatible endpoints plus a native /api/v1 on one port, with eight dated releases between August and October 2026. Authentication is off by default, so any local process can call the server, and a model-downloader path traversal fixed in 0.4.22+1 was disclosed by VulnCheck with no vendor advisory.

Underdog

Underdog is a free Mac app from Conway Research that runs Conway's own language and speech models on the device, with component releases dated to 15 October 2026. Conway documents no interface for outside agents, and underdog.ai refuses our reader, so its terms and privacy policy are unread. Conway's weights moved from Apache-2.0 to non-commercial licences on 5 November 2026.

Before you call either

LM Studio

  1. Send Authorization: Bearer $LM_API_TOKEN when the owner gives you a token. With Require Authentication on, every request needs it
  2. Pass previous_response_id to /api/v1/chat instead of resending the history, and store: false for one-off calls
  3. List models with GET /api/v1/models before naming one. A named model that's downloaded loads just in time
  4. Install the Python SDK pre-release (1.6.0b1) and pass api_token directly. It reads LMSTUDIO_API_TOKEN, not the LM_API_TOKEN the docs name
  5. Set allowed_tools on every MCP integration. Without it the model sees every tool on the server

Underdog

  1. Don't plan on calling Underdog itself. Conway documents no API, MCP server or port for the app.
  2. To use Conway's weights, run them in llama.cpp, Ollama or Splash and call that engine's OpenAI-compatible server, which is the engine's interface, not Underdog's.
  3. Check the licence before commercial use. Conway's weights are CC BY-NC 4.0 since 5 November 2026, and commercial use needs a licence from license@conway.tech.
  4. Don't rely on the husky serve command on the husky-flash card. Its Greyhound repository isn't public.

Questions

Which is better for AI agents, LM Studio or Underdog?

LM Studio scores 58.8 (C) on agent readiness against Underdog's 18.2 (F), and leads in 6 of 7 scored categories.

Can an agent call LM Studio and Underdog without installing anything?

No hosted endpoint is listed for LM Studio. No hosted endpoint is listed for Underdog.

Other comparisons with LM Studio or Underdog

Disclosure

Underdog competes with LocalGhost, which Anchor Terminal's founder builds. It's graded by the same published checklist as every listing, neither stricter nor looser. Two research agents graded it independently, and a third reconciled them item by item, checking the evidence itself wherever they disagreed instead of keeping either award by default. underdog.ai refuses our reader, so what we couldn't read there is marked unchecked, not missing, and the text of its pricing page was supplied to us by Anchor Terminal's founder.

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.