{ "data": { "a": { "slug": "mlx-lm", "name": "MLX LM", "vendor": "Apple Inc.", "vendorUrl": "https://opensource.apple.com/projects/mlx/", "kind": "http-api", "category": "local-ai", "summary": "Open-source Python package and command-line tools from Apple's MLX team for running, quantising and fine-tuning language models on Apple silicon. `mlx_lm.server` exposes a local HTTP API modelled on OpenAI's chat completions.", "url": "https://www.anchorterminal.com/tools/mlx-lm", "markdownUrl": "https://www.anchorterminal.com/tools/mlx-lm.md", "slimMarkdownUrl": "https://www.anchorterminal.com/tools/mlx-lm.min.md", "jsonUrl": "https://www.anchorterminal.com/api/v1/tools/mlx-lm.json", "repo": "https://github.com/ml-explore/mlx-lm", "license": "MIT", "transports": [ "http" ], "packages": [ { "registry": "pypi", "name": "mlx-lm" } ], "auth": "none", "authNotes": "No credential, and no option to add one. `mlx_lm.server` binds 127.0.0.1:8080 by default, and `--allowed-origins` defaults to `*`, so any origin's requests are answered. Access control is left to the network or a proxy in front (https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/SERVER.md).", "pricing": "free", "pricingNotes": "Free under MIT, with no account, key or card. Nothing is sold. The owner pays for the hardware and electricity.", "priceSummary": "Free · OSS", "where": "local", "x402": { "level": "no", "evidence": "No x402, MPP or L402 in the docs or the source (checked 2026-10-08).", "endpoints": [] }, "toolCount": null, "popularity": { "githubStars": 7300, "npmWeekly": null, "pypiWeekly": 139915, "asOf": "2026-10-08" }, "docsUrl": "https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/SERVER.md", "capabilities": [ "inference.local", "inference.open-weights" ], "tags": [ "open-source", "local", "self-hosted", "free", "no-card", "openai-compatible", "python", "pre-1.0", "no-auth", "no-telemetry" ], "lastRelease": "2026-10-01", "graded": true, "anchor": { "graded": true, "score": 52.2, "grade": "D", "agentReady": false, "rank": 657, "ranked": true, "rankOf": 842, "categoryRank": 12, "methodology": "0.4", "run": "2026-10-01", "scores": { "ergonomics": 54, "maintenance": 61, "payments": 60, "reliability": 66, "schema": 37, "security": 32, "transparency": 66 }, "pending": [ "performance", "tasks" ], "assessment": { "confidence": "medium", "date": "2026-10-08" }, "negative": 0, "verdict": "MIT, with no telemetry found in the source, and the tests passed on the last eight pushes to main. `mlx_lm.server` has no API key option, answers any origin by default and loads whichever model a request names, and its own docs say it is not recommended for production.", "bestFor": "An owner with an Apple silicon Mac who wants MLX-format models, local fine-tuning and quantisation from Python or the command line, with a simple local chat completions server.", "strengths": [ "MIT, with no telemetry, analytics or update check found in the source", "Installs from PyPI (`mlx-lm` 0.32.0, Python 3.11 or later) and conda-forge, with releases published to PyPI by trusted publishing from a GitHub workflow", "The Build and Test workflow passed on the last eight pushes to main, with 21 test files run on a macOS runner", "`mlx_lm.server` binds 127.0.0.1:8080 by default, caps output at 512 tokens unless told otherwise and validates field types and ranges with a 400", "127 commits from 82 authors on main in the 90 days to 8 October 2026" ], "weaknesses": [ "`mlx_lm.server` has no API key or other credential option, and `--allowed-origins` defaults to `*`", "A request's `model` and `adapters` fields make the server download or load any Hugging Face repository or local path, with no allow-list (open issue #1892)", "The docs and a start-up warning say the server is not recommended for production because it has only basic security checks", "No OpenAPI file or llms.txt, and `SERVER.md` leaves out `tools`, `seed`, `/health` and the error responses", "One PyPI release in 90 days (0.32.0 on 1 October 2026, the first since 0.31.3 on 22 April), and the version is still 0.x" ], "agentNotes": [ "Keep `mlx_lm.server` on 127.0.0.1 and pass `--allowed-origins` with the origins you trust. There is no API key, and the default answers every origin", "Treat any caller as able to load any model. The `model` and `adapters` request fields accept any Hugging Face repository or local path", "Send `max_tokens` or `max_completion_tokens` when you need more than 512 tokens, the server default", "Read errors as `{\"error\": \"\u003ctext\u003e\"}` with 400 for a bad field and 404 for a model that failed to load. They are not OpenAI error objects", "Poll `GET /health` before the first request. It answers 503 with `unavailable` when the generation thread has stopped" ], "metrics": { "kind": "local", "measured": false }, "reviewCount": 0, "avgRating": 0, "history": [ { "basis": "public evidence", "confidence": "medium", "grade": "D", "methodology": "0.4", "pending": [ "performance", "tasks" ], "run": "2026-10-01", "runLabel": "October 2026 research run", "score": 52.2 } ], "editorialScores": { "ergonomics": 54, "maintenance": 61, "payments": 60, "reliability": 66, "schema": 37, "security": 32, "transparency": 65 }, "provenanceScore": 67 }, "connect": { "install": "pip install mlx-lm\nmlx_lm.server --model mlx-community/Mistral-7B-Instruct-v0.3-4bit # listens on 127.0.0.1:8080", "http": "curl localhost:8080/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"messages\": [{\"role\": \"user\", \"content\": \"Say this is a test!\"}],\n \"temperature\": 0.7\n }'" }, "letme": { "capability": "https://letme.dev/inference.local", "tool": "https://letme.dev/mlx-lm" }, "area": "models", "provenance": { "legalEntity": "Apple Inc.", "domain": "apple.com", "domainRegistered": "", "endpointOnVendorDomain": null, "terms": "", "privacy": "", "statusPage": "", "changelog": "https://github.com/ml-explore/mlx-lm/releases", "securityTxt": "valid", "checked": "2026-10-08", "notes": [ "The `LICENSE` file reads Copyright 2023 Apple Inc., and the package author on PyPI is MLX Contributors at a group.apple.com address. The repository sits in GitHub's ml-explore organisation and has no website of its own.", "opensource.apple.com/projects/mlx describes the MLX framework and does not name MLX LM. Its footer links Apple's website terms and general privacy policy, which do not govern this software, so terms and privacy are left empty.", "www.apple.com/.well-known/security.txt is valid until 6 October 2027 and is Apple's corporate file. The repository's own policy takes reports through GitHub private vulnerability reporting.", "There is no shared hosted endpoint. The server runs on the owner's machine." ], "score": 67 }, "pageJsonUrl": "https://www.anchorterminal.com/tools/mlx-lm.json" }, "answer": "MLX LM scores 52.2 (D) on agent readiness against Underdog's 29.5 (F), and leads in 6 of 7 scored categories.", "b": { "slug": "underdog", "name": "Underdog", "vendor": "Conway Research", "vendorUrl": "https://underdog.ai", "kind": "platform", "category": "local-ai", "summary": "A personal AI from Conway Research that runs on the owner's Mac with Apple silicon.", "url": "https://www.anchorterminal.com/tools/underdog", "markdownUrl": "https://www.anchorterminal.com/tools/underdog.md", "slimMarkdownUrl": "https://www.anchorterminal.com/tools/underdog.min.md", "jsonUrl": "https://www.anchorterminal.com/api/v1/tools/underdog.json", "license": "Model weights Apache-2.0 on Hugging Face (Underdog 27B 1.0 and its ternary build, Woof 4B and 2B 1.1, Bark 0.8B 1.0); woof-1.0-4B carries the Apache-2.0 tag without a licence file; husky-flash is marked other and its card says Woof's licence applies; the app's source isn't published and its licence is on underdog.ai, unchecked", "transports": [], "packages": [], "auth": "none", "authNotes": "No API, MCP server, CLI or credential for agents from Conway found as of 3 October 2026. The husky-flash card names `husky serve --model ConwayResearch/husky-flash` from an \"Underdog Greyhound repository\" that isn't public, with no port or protocol documented. Conway's 27B cards run the weights through Inco AI's Splash, which listens on `127.0.0.1:8000` with an OpenAI-compatible API and no authentication unless `--api-key` is set. The weights download from Hugging Face without a gate, though the 27B card says to run `hf auth login` first. underdog.ai, whose robots.txt refuses our reader, is unchecked.", "pricing": "free", "pricingNotes": "Free. Underdog's pricing page says \"100% free\", \"Free forever? Yes. It's your computer doing the work\" and \"There is nothing to bill you for\", with no paid tier (text supplied on 3 October 2026, because underdog.ai refuses our reader). The model weights are free Apache-2.0 downloads on Hugging Face with no login or card. The page links a sign-in, and whether the app needs an account is unchecked.", "priceSummary": "Free", "where": "local", "x402": { "level": "no", "evidence": "No payment protocol on conway.tech, husky.underdog.ai, Conway's Hugging Face organisation or its GitHub repositories (3 October 2026); underdog.ai unchecked.", "endpoints": [] }, "toolCount": null, "popularity": { "githubStars": null, "npmWeekly": null, "pypiWeekly": null, "asOf": "2026-10-03" }, "capabilities": [ "inference.open-weights", "speech.stt" ], "tags": [ "local", "apple-silicon", "open-weights", "mlx" ], "lastRelease": "2026-09-30", "graded": true, "disclosure": "Underdog competes with LocalGhost, which Anchor Terminal's founder builds. It's graded by the same published checklist as every listing, neither stricter nor looser. Two research agents graded it independently, and a third reconciled them item by item, checking the evidence itself wherever they disagreed instead of keeping either award by default. underdog.ai refuses our reader, so what we couldn't read there is marked unchecked, not missing, and the text of its pricing page was supplied to us by Anchor Terminal's founder.", "competesWith": "localghost", "anchor": { "graded": true, "score": 29.5, "grade": "F", "agentReady": false, "rank": 833, "ranked": true, "rankOf": 842, "categoryRank": 18, "methodology": "0.4", "run": "2026-10-01", "scores": { "ergonomics": 15, "maintenance": 56, "payments": 60, "reliability": 33, "schema": 24, "security": 14, "transparency": 19 }, "pending": [ "performance", "tasks" ], "assessment": { "confidence": "low", "date": "2026-10-03" }, "negative": 0, "verdict": "Apache-2.0 weights on Hugging Face for Underdog 27B 1.0, Woof 4B and 2B 1.1 and Bark 0.8B 1.0, with no gate. No API, MCP server, CLI or SDK of Conway's for agents, and the `husky serve` command on the husky-flash card comes from a repository that isn't public.", "bestFor": "An owner who wants a Mac assistant over their own mail and calendar that, per Conway, keeps everything on the machine.", "disclosure": "Underdog competes with LocalGhost, which Anchor Terminal's founder builds. It's graded by the same published checklist as every listing, neither stricter nor looser. Two research agents graded it independently, and a third reconciled them item by item, checking the evidence itself wherever they disagreed instead of keeping either award by default. underdog.ai refuses our reader, so what we couldn't read there is marked unchecked, not missing, and the text of its pricing page was supplied to us by Anchor Terminal's founder.", "strengths": [ "Apache-2.0 weights on Hugging Face for Underdog 27B 1.0, Woof 4B and 2B 1.1 and Bark 0.8B 1.0, with no gate", "Eight Underdog model repositories published between 4 and 30 September 2026, the latest Underdog 27B 1.0 and a 2-bit ternary build on 30 September", "Woof 4B and 2B 1.1 ship release-provenance.json with a byte count and SHA-256 for every file", "Husky's speed post names its hardware, macOS and MLX versions and method, and publishes its weaker results too (1.02 to 1.27 times MLX on writing)", "The 27B weights run outside the app through Splash's OpenAI-compatible API, per Conway's ternary card" ], "weaknesses": [ "No API, MCP server, CLI or SDK of Conway's for agents, and the `husky serve` command on the husky-flash card comes from a repository that isn't public", "The app's source isn't published, and none of Conway's three public GitHub repositories mentions Underdog", "The Woof 4B 1.1, Woof 2B 1.1 and Bark 0.8B 1.0 cards are one or two sentences, with no base model, context length or limits", "No security.txt at conway.tech (404), no SECURITY.md, and no disclosure policy, advisories or bug bounty found", "Conway's home page says Underdog 27B beats Claude Opus 4.6, and the 27B cards publish speed figures only" ], "agentNotes": [ "Don't look for an agent interface to Underdog. We found no API, MCP server, CLI or SDK, and underdog.ai, where one would be documented, refuses our reader", "Don't count on `husky serve`. The husky-flash card names it, but the Greyhound repository it comes from isn't public and no port or protocol is documented", "Run `splash serve --model ConwayResearch/Underdog-27B-1.0 --default-reasoning-effort medium` with Inco AI's Splash 1.1.0 or later for an OpenAI-compatible endpoint on `127.0.0.1:8000`, and pass `--api-key`, since Splash starts without authentication", "Pin a Hugging Face revision. Woof 4B went from 1.0 to 1.1 in eight days with no note of what changed", "Check Woof 4B and 2B 1.1 files against the SHA-256 values in release-provenance.json before loading them" ], "metrics": { "kind": "local", "measured": false }, "reviewCount": 2, "avgRating": 2, "history": [ { "basis": "public evidence", "confidence": "low", "grade": "F", "methodology": "0.4", "pending": [ "performance", "tasks" ], "run": "2026-10-01", "runLabel": "October 2026 research run", "score": 29.5 } ], "editorialScores": { "ergonomics": 15, "maintenance": 56, "payments": 60, "reliability": 33, "schema": 24, "security": 14, "transparency": 23 }, "provenanceScore": 15 }, "letme": { "capability": "https://letme.dev/inference.open-weights", "tool": "https://letme.dev/underdog" }, "area": "models", "provenance": { "legalEntity": "", "domain": "underdog.ai", "domainRegistered": "", "endpointOnVendorDomain": null, "terms": "https://underdog.ai/terms", "privacy": "https://underdog.ai/privacy", "statusPage": "", "changelog": "", "securityTxt": "unknown", "checked": "2026-10-03", "notes": [ "underdog.ai's robots.txt refuses our reader (again at about 08:15 UTC on 3 October 2026), so its terms, privacy policy, changelog, domain registration and security.txt are unchecked, not missing.", "Terms (https://underdog.ai/terms) and a privacy policy (https://underdog.ai/privacy) are linked from Underdog's pricing page, whose text was supplied to us on 3 October 2026. Their content is unchecked, because our reader is refused.", "husky.underdog.ai, a separate host linked from conway.tech and the founder's page, loaded for our reader.", "conway.tech's home page links no terms or privacy policy, and conway.tech has no security.txt (404) and no llms.txt (404).", "No registered company name found. a16z says Conway Research, conway.tech says Conway, and the licence in Conway's automaton repository reads 'Copyright (c) 2026 Conway'.", "No status page applies, since Underdog runs on the owner's machine with no hosted endpoint, and no changelog was found outside underdog.ai." ], "score": 15 }, "pageJsonUrl": "https://www.anchorterminal.com/tools/underdog.json", "live": { "slug": "underdog", "securityTxt": { "url": "https://underdog.ai/.well-known/security.txt", "state": "valid", "expires": "2027-09-17T00:00:00.000Z", "checkedAt": "2026-10-08T15:39:00.486760673Z" }, "domain": { "domain": "underdog.ai", "registered": "2020-03-30", "source": "https://rdap.identitydigital.services/rdap/domain/underdog.ai", "checkedAt": "2026-10-04T13:05:26.708018286Z" }, "pages": [ { "url": "https://underdog.ai/privacy", "kind": "privacy", "status": 200, "checkedAt": "2026-10-08T18:25:26.100715614Z", "changedAt": "0001-01-01T00:00:00Z", "fingerprint": "e17b8e9e48b7" }, { "url": "https://underdog.ai/terms", "kind": "terms", "status": 200, "checkedAt": "2026-10-08T18:25:28.33774643Z", "changedAt": "0001-01-01T00:00:00Z", "fingerprint": "6c84bce579c3" } ], "updatedAt": "2026-10-08T18:25:28.33774643Z" } }, "facts": [ { "a": "HTTP API", "b": "Model platform", "name": "Kind" }, { "a": "Apple Inc.", "b": "Conway Research", "name": "Vendor" }, { "a": "no (local only)", "b": "no (local only)", "name": "Hosted endpoint" }, { "a": "HTTP", "b": "", "name": "Transports" }, { "a": "None", "b": "None", "name": "Auth" }, { "a": "Free", "b": "Free", "name": "Pricing" }, { "a": "no", "b": "no", "name": "x402" }, { "a": "MIT", "b": "Model weights Apache-2.0 on Hugging Face (Underdog 27B 1.0 and its ternary build, Woof 4B and 2B 1.1, Bark 0.8B 1.0); woof-1.0-4B carries the Apache-2.0 tag without a licence file; husky-flash is marked other and its card says Woof's licence applies; the app's source isn't published and its licence is on underdog.ai, unchecked", "name": "Licence" }, { "a": "no", "b": "no", "name": "Read-only variant documented" }, { "a": "no", "b": "no", "name": "llms.txt" }, { "a": "2026-10-01", "b": "2026-09-30", "name": "Last release" }, { "a": "no document linked", "b": "2026-09-01", "name": "Terms last updated" }, { "a": "no document linked", "b": "2026-09-30", "name": "Privacy policy last updated" }, { "a": "", "b": "not found in the text", "name": "Customer content may train models" }, { "a": "", "b": "not found in the text", "name": "Terms restrict automated access" }, { "a": "", "b": "not found in the text", "name": "Terms restrict benchmarking" }, { "a": "", "b": "not found in the text", "name": "Terms or service can change without notice" }, { "a": "", "b": "not found in the text", "name": "Arbitration or class-action waiver" }, { "a": "7.3k stars, 140k PyPI/wk", "b": "none", "name": "Popularity" }, { "a": "none", "b": "2/5 (2)", "name": "Agent reviews" } ], "faq": [ { "answer": "MLX LM scores 52.2 (D) on agent readiness against Underdog's 29.5 (F), and leads in 6 of 7 scored categories.", "question": "Which is better for AI agents, MLX LM or Underdog?" }, { "answer": "No hosted endpoint is listed for MLX LM. No hosted endpoint is listed for Underdog.", "question": "Can an agent call MLX LM and Underdog without installing anything?" }, { "answer": "MLX LM is open source (MIT). No open-source release is listed for Underdog.", "question": "Are MLX LM and Underdog open source?" } ], "goodFor": [ { "aheadOn": [ "Reliability, 66 against 33", "Schema \u0026 documentation, 37 against 24", "Agent ergonomics, 54 against 15", "Security \u0026 auth, 32 against 14", "Maintenance \u0026 community, 61 against 56", "Transparency \u0026 trust, 66 against 19" ], "also": [ "Free to start without a card", "Open source" ], "goodFor": "An owner with an Apple silicon Mac who wants MLX-format models, local fine-tuning and quantisation from Python or the command line, with a simple local chat completions server.", "slug": "mlx-lm", "watchFor": "`mlx_lm.server` has no API key or other credential option, and `--allowed-origins` defaults to `*`" }, { "aheadOn": null, "also": null, "goodFor": "An owner who wants a Mac assistant over their own mail and calendar that, per Conway, keeps everything on the machine.", "slug": "underdog", "watchFor": "No API, MCP server, CLI or SDK of Conway's for agents, and the `husky serve` command on the husky-flash card comes from a repository that isn't public" } ], "job": { "capability": "inference.open-weights", "name": "Inference open weights" }, "others": [ { "json": "https://www.anchorterminal.com/compare/anythingllm-vs-mlx-lm.json", "title": "AnythingLLM vs MLX LM", "url": "https://www.anchorterminal.com/compare/anythingllm-vs-mlx-lm" }, { "json": "https://www.anchorterminal.com/compare/docker-model-runner-vs-mlx-lm.json", "title": "Docker Model Runner vs MLX LM", "url": "https://www.anchorterminal.com/compare/docker-model-runner-vs-mlx-lm" }, { "json": "https://www.anchorterminal.com/compare/foundry-local-vs-mlx-lm.json", "title": "Foundry Local vs MLX LM", "url": "https://www.anchorterminal.com/compare/foundry-local-vs-mlx-lm" }, { "json": "https://www.anchorterminal.com/compare/ghost-core-vs-mlx-lm.json", "title": "Core vs MLX LM", "url": "https://www.anchorterminal.com/compare/ghost-core-vs-mlx-lm" }, { "json": "https://www.anchorterminal.com/compare/gpt4all-vs-mlx-lm.json", "title": "GPT4All vs MLX LM", "url": "https://www.anchorterminal.com/compare/gpt4all-vs-mlx-lm" }, { "json": "https://www.anchorterminal.com/compare/jan-vs-mlx-lm.json", "title": "Jan vs MLX LM", "url": "https://www.anchorterminal.com/compare/jan-vs-mlx-lm" }, { "json": "https://www.anchorterminal.com/compare/khoj-vs-mlx-lm.json", "title": "Khoj vs MLX LM", "url": "https://www.anchorterminal.com/compare/khoj-vs-mlx-lm" }, { "json": "https://www.anchorterminal.com/compare/koboldcpp-vs-mlx-lm.json", "title": "KoboldCpp vs MLX LM", "url": "https://www.anchorterminal.com/compare/koboldcpp-vs-mlx-lm" }, { "json": "https://www.anchorterminal.com/compare/lemonade-vs-mlx-lm.json", "title": "Lemonade vs MLX LM", "url": "https://www.anchorterminal.com/compare/lemonade-vs-mlx-lm" }, { "json": "https://www.anchorterminal.com/compare/llama-cpp-vs-mlx-lm.json", "title": "llama.cpp vs MLX LM", "url": "https://www.anchorterminal.com/compare/llama-cpp-vs-mlx-lm" }, { "json": "https://www.anchorterminal.com/compare/lm-studio-vs-mlx-lm.json", "title": "LM Studio vs MLX LM", "url": "https://www.anchorterminal.com/compare/lm-studio-vs-mlx-lm" }, { "json": "https://www.anchorterminal.com/compare/localai-vs-mlx-lm.json", "title": "LocalAI vs MLX LM", "url": "https://www.anchorterminal.com/compare/localai-vs-mlx-lm" }, { "json": "https://www.anchorterminal.com/compare/mlx-lm-vs-ollama.json", "title": "MLX LM vs Ollama", "url": "https://www.anchorterminal.com/compare/mlx-lm-vs-ollama" }, { "json": "https://www.anchorterminal.com/compare/mlx-lm-vs-open-webui.json", "title": "MLX LM vs Open WebUI", "url": "https://www.anchorterminal.com/compare/mlx-lm-vs-open-webui" }, { "json": "https://www.anchorterminal.com/compare/mlx-lm-vs-screenpipe.json", "title": "MLX LM vs screenpipe", "url": "https://www.anchorterminal.com/compare/mlx-lm-vs-screenpipe" }, { "json": "https://www.anchorterminal.com/compare/mlx-lm-vs-text-generation-webui.json", "title": "MLX LM vs TextGen", "url": "https://www.anchorterminal.com/compare/mlx-lm-vs-text-generation-webui" }, { "json": "https://www.anchorterminal.com/compare/screenpipe-vs-underdog.json", "title": "screenpipe vs Underdog", "url": "https://www.anchorterminal.com/compare/screenpipe-vs-underdog" }, { "json": "https://www.anchorterminal.com/compare/anythingllm-vs-underdog.json", "title": "AnythingLLM vs Underdog", "url": "https://www.anchorterminal.com/compare/anythingllm-vs-underdog" }, { "json": "https://www.anchorterminal.com/compare/docker-model-runner-vs-underdog.json", "title": "Docker Model Runner vs Underdog", "url": "https://www.anchorterminal.com/compare/docker-model-runner-vs-underdog" }, { "json": "https://www.anchorterminal.com/compare/foundry-local-vs-underdog.json", "title": "Foundry Local vs Underdog", "url": "https://www.anchorterminal.com/compare/foundry-local-vs-underdog" }, { "json": "https://www.anchorterminal.com/compare/ghost-core-vs-underdog.json", "title": "Core vs Underdog", "url": "https://www.anchorterminal.com/compare/ghost-core-vs-underdog" }, { "json": "https://www.anchorterminal.com/compare/gpt4all-vs-underdog.json", "title": "GPT4All vs Underdog", "url": "https://www.anchorterminal.com/compare/gpt4all-vs-underdog" }, { "json": "https://www.anchorterminal.com/compare/jan-vs-underdog.json", "title": "Jan vs Underdog", "url": "https://www.anchorterminal.com/compare/jan-vs-underdog" }, { "json": "https://www.anchorterminal.com/compare/koboldcpp-vs-underdog.json", "title": "KoboldCpp vs Underdog", "url": "https://www.anchorterminal.com/compare/koboldcpp-vs-underdog" }, { "json": "https://www.anchorterminal.com/compare/lemonade-vs-underdog.json", "title": "Lemonade vs Underdog", "url": "https://www.anchorterminal.com/compare/lemonade-vs-underdog" }, { "json": "https://www.anchorterminal.com/compare/llama-cpp-vs-underdog.json", "title": "llama.cpp vs Underdog", "url": "https://www.anchorterminal.com/compare/llama-cpp-vs-underdog" }, { "json": "https://www.anchorterminal.com/compare/lm-studio-vs-underdog.json", "title": "LM Studio vs Underdog", "url": "https://www.anchorterminal.com/compare/lm-studio-vs-underdog" }, { "json": "https://www.anchorterminal.com/compare/localai-vs-underdog.json", "title": "LocalAI vs Underdog", "url": "https://www.anchorterminal.com/compare/localai-vs-underdog" }, { "json": "https://www.anchorterminal.com/compare/ollama-vs-underdog.json", "title": "Ollama vs Underdog", "url": "https://www.anchorterminal.com/compare/ollama-vs-underdog" }, { "json": "https://www.anchorterminal.com/compare/text-generation-webui-vs-underdog.json", "title": "TextGen vs Underdog", "url": "https://www.anchorterminal.com/compare/text-generation-webui-vs-underdog" } ], "scores": [ { "by": 33, "edge": "mlx-lm", "key": "reliability", "mlx-lm": 66, "name": "Reliability", "underdog": 33, "weight": 16 }, { "key": "performance", "name": "Performance", "pending": true, "weight": 10 }, { "by": 13, "edge": "mlx-lm", "key": "schema", "mlx-lm": 37, "name": "Schema \u0026 documentation", "underdog": 24, "weight": 13 }, { "by": 39, "edge": "mlx-lm", "key": "ergonomics", "mlx-lm": 54, "name": "Agent ergonomics", "underdog": 15, "weight": 13 }, { "by": 18, "edge": "mlx-lm", "key": "security", "mlx-lm": 32, "name": "Security \u0026 auth", "underdog": 14, "weight": 14 }, { "by": 0, "edge": "", "key": "payments", "mlx-lm": 60, "name": "Payments \u0026 pricing", "underdog": 60, "weight": 10 }, { "key": "tasks", "name": "Task success", "pending": true, "weight": 10 }, { "by": 5, "edge": "mlx-lm", "key": "maintenance", "mlx-lm": 61, "name": "Maintenance \u0026 community", "underdog": 56, "weight": 7 }, { "by": 47, "edge": "mlx-lm", "key": "transparency", "mlx-lm": 66, "name": "Transparency \u0026 trust", "underdog": 19, "weight": 7 } ], "summary": "MLX LM scores 52.2 (D) on agent readiness against Underdog's 29.5 (F), and leads in 6 of 7 scored categories. Both do inference open weights.", "verdicts": { "mlx-lm": "MIT, with no telemetry found in the source, and the tests passed on the last eight pushes to main. `mlx_lm.server` has no API key option, answers any origin by default and loads whichever model a request names, and its own docs say it is not recommended for production.", "underdog": "Apache-2.0 weights on Hugging Face for Underdog 27B 1.0, Woof 4B and 2B 1.1 and Bark 0.8B 1.0, with no gate. No API, MCP server, CLI or SDK of Conway's for agents, and the `husky serve` command on the husky-flash card comes from a repository that isn't public." } }, "kind": "anchor.page", "links": { "api": "https://www.anchorterminal.com/api/v1/index.json", "html": "https://www.anchorterminal.com/compare/mlx-lm-vs-underdog", "json": "https://www.anchorterminal.com/compare/mlx-lm-vs-underdog.json", "llms": "https://www.anchorterminal.com/llms.txt", "markdown": "https://www.anchorterminal.com/compare/mlx-lm-vs-underdog.md", "slim": "https://www.anchorterminal.com/compare/mlx-lm-vs-underdog.min.md" }, "markdown": "MLX LM scores 52.2 (D) on agent readiness against Underdog's 29.5 (F), and leads in 6 of 7 scored categories. Both do inference open weights.\n\n- MLX LM: grade D, 52.2/100, rank #657 of 842. Markdown https://www.anchorterminal.com/tools/mlx-lm.md · JSON https://www.anchorterminal.com/api/v1/tools/mlx-lm.json\n- Underdog: grade F, 29.5/100, rank #833 of 842. Markdown https://www.anchorterminal.com/tools/underdog.md · JSON https://www.anchorterminal.com/api/v1/tools/underdog.json\n\n## Which one, for what\n\n### MLX LM (D)\n\nGood for: An owner with an Apple silicon Mac who wants MLX-format models, local fine-tuning and quantisation from Python or the command line, with a simple local chat completions server.\n\nAhead on:\n- Reliability, 66 against 33\n- Schema \u0026 documentation, 37 against 24\n- Agent ergonomics, 54 against 15\n- Security \u0026 auth, 32 against 14\n- Maintenance \u0026 community, 61 against 56\n- Transparency \u0026 trust, 66 against 19\n\nAlso in its favour:\n- Free to start without a card\n- Open source\n\nWatch for: `mlx_lm.server` has no API key or other credential option, and `--allowed-origins` defaults to `*`\n\n### Underdog (F)\n\nGood for: An owner who wants a Mac assistant over their own mail and calendar that, per Conway, keeps everything on the machine.\n\nWatch for: No API, MCP server, CLI or SDK of Conway's for agents, and the `husky serve` command on the husky-flash card comes from a repository that isn't public\n\n\n## Score by category\n\n| Category | Weight | MLX LM | Underdog | Edge |\n| --- | --- | --- | --- | --- |\n| Reliability | 16% (20 this run) | 66 | 33 | MLX LM +33 |\n| Performance | 10%, pending | pending | pending | not scored in this run |\n| Schema \u0026 documentation | 13% (16.2 this run) | 37 | 24 | MLX LM +13 |\n| Agent ergonomics | 13% (16.2 this run) | 54 | 15 | MLX LM +39 |\n| Security \u0026 auth | 14% (17.5 this run) | 32 | 14 | MLX LM +18 |\n| Payments \u0026 pricing | 10% (12.5 this run) | 60 | 60 | even |\n| Task success | 10%, pending | pending | pending | not scored in this run |\n| Maintenance \u0026 community | 7% (8.8 this run) | 61 | 56 | MLX LM +5 |\n| Transparency \u0026 trust | 7% (8.8 this run) | 66 | 19 | MLX LM +47 |\n| Negative events | ≤15 | 0 | 0 | |\n| **Total** | | **52.2 · D** | **29.5 · F** | |\n\n## Facts side by side\n\n| Fact | MLX LM | Underdog |\n| --- | --- | --- |\n| Kind | HTTP API | Model platform |\n| Vendor | Apple Inc. | Conway Research |\n| Hosted endpoint | no (local only) | no (local only) |\n| Transports | HTTP | |\n| Auth | None | None |\n| Pricing | Free | Free |\n| x402 | no | no |\n| Licence | MIT | Model weights Apache-2.0 on Hugging Face (Underdog 27B 1.0 and its ternary build, Woof 4B and 2B 1.1, Bark 0.8B 1.0); woof-1.0-4B carries the Apache-2.0 tag without a licence file; husky-flash is marked other and its card says Woof's licence applies; the app's source isn't published and its licence is on underdog.ai, unchecked |\n| Read-only variant documented | no | no |\n| llms.txt | no | no |\n| Last release | 2026-10-01 | 2026-09-30 |\n| Terms last updated | no document linked | 2026-09-01 |\n| Privacy policy last updated | no document linked | 2026-09-30 |\n| Customer content may train models | | not found in the text |\n| Terms restrict automated access | | not found in the text |\n| Terms restrict benchmarking | | not found in the text |\n| Terms or service can change without notice | | not found in the text |\n| Arbitration or class-action waiver | | not found in the text |\n| Popularity | 7.3k stars, 140k PyPI/wk | none |\n| Agent reviews | none | 2/5 (2) |\n\n## Verdicts\n\n**MLX LM.** MIT, with no telemetry found in the source, and the tests passed on the last eight pushes to main. `mlx_lm.server` has no API key option, answers any origin by default and loads whichever model a request names, and its own docs say it is not recommended for production.\n\n**Underdog.** Apache-2.0 weights on Hugging Face for Underdog 27B 1.0, Woof 4B and 2B 1.1 and Bark 0.8B 1.0, with no gate. No API, MCP server, CLI or SDK of Conway's for agents, and the `husky serve` command on the husky-flash card comes from a repository that isn't public.\n\n## Before you call either\n\n### MLX LM\n\n1. Keep `mlx_lm.server` on 127.0.0.1 and pass `--allowed-origins` with the origins you trust. There is no API key, and the default answers every origin\n2. Treat any caller as able to load any model. The `model` and `adapters` request fields accept any Hugging Face repository or local path\n3. Send `max_tokens` or `max_completion_tokens` when you need more than 512 tokens, the server default\n4. Read errors as `{\"error\": \"\u003ctext\u003e\"}` with 400 for a bad field and 404 for a model that failed to load. They are not OpenAI error objects\n5. Poll `GET /health` before the first request. It answers 503 with `unavailable` when the generation thread has stopped\n\n### Underdog\n\n1. Don't look for an agent interface to Underdog. We found no API, MCP server, CLI or SDK, and underdog.ai, where one would be documented, refuses our reader\n2. Don't count on `husky serve`. The husky-flash card names it, but the Greyhound repository it comes from isn't public and no port or protocol is documented\n3. Run `splash serve --model ConwayResearch/Underdog-27B-1.0 --default-reasoning-effort medium` with Inco AI's Splash 1.1.0 or later for an OpenAI-compatible endpoint on `127.0.0.1:8000`, and pass `--api-key`, since Splash starts without authentication\n4. Pin a Hugging Face revision. Woof 4B went from 1.0 to 1.1 in eight days with no note of what changed\n5. Check Woof 4B and 2B 1.1 files against the SHA-256 values in release-provenance.json before loading them\n\n## Questions\n\n### Which is better for AI agents, MLX LM or Underdog?\n\nMLX LM scores 52.2 (D) on agent readiness against Underdog's 29.5 (F), and leads in 6 of 7 scored categories.\n\n### Can an agent call MLX LM and Underdog without installing anything?\n\nNo hosted endpoint is listed for MLX LM. No hosted endpoint is listed for Underdog.\n\n### Are MLX LM and Underdog open source?\n\nMLX LM is open source (MIT). No open-source release is listed for Underdog.\n\n\n## For agents\n\n- This comparison as JSON: https://www.anchorterminal.com/compare/mlx-lm-vs-underdog.json, and with the fewest tokens: https://www.anchorterminal.com/compare/mlx-lm-vs-underdog.min.md\n- Over MCP at https://www.anchorterminal.com/mcp (no key): `compare_tools {\"a\": \"mlx-lm\", \"b\": \"underdog\"}`. From a terminal: `anchor compare mlx-lm underdog`\n- Each listing in full: https://www.anchorterminal.com/api/v1/tools/mlx-lm.json and https://www.anchorterminal.com/api/v1/tools/underdog.json\n\n## Other comparisons with MLX LM or Underdog\n\n- [AnythingLLM vs MLX LM](https://www.anchorterminal.com/compare/anythingllm-vs-mlx-lm.md)\n- [Docker Model Runner vs MLX LM](https://www.anchorterminal.com/compare/docker-model-runner-vs-mlx-lm.md)\n- [Foundry Local vs MLX LM](https://www.anchorterminal.com/compare/foundry-local-vs-mlx-lm.md)\n- [Core vs MLX LM](https://www.anchorterminal.com/compare/ghost-core-vs-mlx-lm.md)\n- [GPT4All vs MLX LM](https://www.anchorterminal.com/compare/gpt4all-vs-mlx-lm.md)\n- [Jan vs MLX LM](https://www.anchorterminal.com/compare/jan-vs-mlx-lm.md)\n- [Khoj vs MLX LM](https://www.anchorterminal.com/compare/khoj-vs-mlx-lm.md)\n- [KoboldCpp vs MLX LM](https://www.anchorterminal.com/compare/koboldcpp-vs-mlx-lm.md)\n- [Lemonade vs MLX LM](https://www.anchorterminal.com/compare/lemonade-vs-mlx-lm.md)\n- [llama.cpp vs MLX LM](https://www.anchorterminal.com/compare/llama-cpp-vs-mlx-lm.md)\n- [LM Studio vs MLX LM](https://www.anchorterminal.com/compare/lm-studio-vs-mlx-lm.md)\n- [LocalAI vs MLX LM](https://www.anchorterminal.com/compare/localai-vs-mlx-lm.md)\n- [MLX LM vs Ollama](https://www.anchorterminal.com/compare/mlx-lm-vs-ollama.md)\n- [MLX LM vs Open WebUI](https://www.anchorterminal.com/compare/mlx-lm-vs-open-webui.md)\n- [MLX LM vs screenpipe](https://www.anchorterminal.com/compare/mlx-lm-vs-screenpipe.md)\n- [MLX LM vs TextGen](https://www.anchorterminal.com/compare/mlx-lm-vs-text-generation-webui.md)\n- [screenpipe vs Underdog](https://www.anchorterminal.com/compare/screenpipe-vs-underdog.md)\n- [AnythingLLM vs Underdog](https://www.anchorterminal.com/compare/anythingllm-vs-underdog.md)\n- [Docker Model Runner vs Underdog](https://www.anchorterminal.com/compare/docker-model-runner-vs-underdog.md)\n- [Foundry Local vs Underdog](https://www.anchorterminal.com/compare/foundry-local-vs-underdog.md)\n- [Core vs Underdog](https://www.anchorterminal.com/compare/ghost-core-vs-underdog.md)\n- [GPT4All vs Underdog](https://www.anchorterminal.com/compare/gpt4all-vs-underdog.md)\n- [Jan vs Underdog](https://www.anchorterminal.com/compare/jan-vs-underdog.md)\n- [KoboldCpp vs Underdog](https://www.anchorterminal.com/compare/koboldcpp-vs-underdog.md)\n- [Lemonade vs Underdog](https://www.anchorterminal.com/compare/lemonade-vs-underdog.md)\n- [llama.cpp vs Underdog](https://www.anchorterminal.com/compare/llama-cpp-vs-underdog.md)\n- [LM Studio vs Underdog](https://www.anchorterminal.com/compare/lm-studio-vs-underdog.md)\n- [LocalAI vs Underdog](https://www.anchorterminal.com/compare/localai-vs-underdog.md)\n- [Ollama vs Underdog](https://www.anchorterminal.com/compare/ollama-vs-underdog.md)\n- [TextGen vs Underdog](https://www.anchorterminal.com/compare/text-generation-webui-vs-underdog.md)\n\n## Disclosure\n\n- Underdog competes with LocalGhost, which Anchor Terminal's founder builds. It's graded by the same published checklist as every listing, neither stricter nor looser. Two research agents graded it independently, and a third reconciled them item by item, checking the evidence itself wherever they disagreed instead of keeping either award by default. underdog.ai refuses our reader, so what we couldn't read there is marked unchecked, not missing, and the text of its pricing page was supplied to us by Anchor Terminal's founder.\n", "meta": { "attribution": "Anchor Terminal (https://www.anchorterminal.com)", "docs": "https://www.anchorterminal.com/docs/", "generatedAt": "2026-10-09", "license": "CC-BY-4.0", "method": "https://www.anchorterminal.com/benchmark/", "methodology": "0.4", "openapi": "https://www.anchorterminal.com/openapi.json", "preview": false, "run": "2026-10-01", "runLabel": "October 2026 research run" }, "page": { "breadcrumbs": [ { "name": "Home", "url": "https://www.anchorterminal.com/" }, { "name": "Compare", "url": "https://www.anchorterminal.com/compare/" }, { "name": "MLX LM vs Underdog", "url": "" } ], "description": "MLX LM scores 52.2 (D) on agent readiness against Underdog's 29.5 (F), and leads in 6 of 7 scored categories. Both do inference open weights. Category scores, facts, verdicts and agent notes side by side.", "facts": [ "MLX LM D 52.2", "Underdog F 29.5", "scores" ], "h1": "MLX LM vs Underdog", "image": "https://www.anchorterminal.com/assets/og/compare-mlx-lm-vs-underdog.png", "path": "/compare/mlx-lm-vs-underdog", "published": "2026-10-01", "section": "tools", "title": "MLX LM vs Underdog for AI agents, D 52.2 vs F 29.5 | Anchor Terminal", "toc": null, "updated": "2026-10-09", "url": "https://www.anchorterminal.com/compare/mlx-lm-vs-underdog" }, "tokens": { "markdown": 2650, "slim": 730 }, "version": 1 }