Category · Models & inference
Local AI: models and assistants that run on your own hardware
Software that runs models on hardware the owner keeps, a laptop, a desktop or a home server. Model runners with a local API, chat apps, and personal assistants that work from the owner's own files, mail and records, with no cloud account needed. Compared on what models they run, what hardware they need, what leaves the machine, what an agent can call and the licence.
Capability keys inference.local · inference.open-weights · memory.user · memory.search · agent.mcp-client · All tools
letme.dev/inference.local picks the top-graded tool in this list and says how to call it direct; calling through letme comes later.
The same listing from the live API. Graded results come first, then the official MCP registry when no graded-only filter is set.
https://www.anchorterminal.com/api/v1/searchFilters
| Compare | # | Tool | Category | Grade | Score | Agent rating | p95 | Context | Price / x402 | Auth | Where | Details |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 133 | LocalAIEttore Di Giacinto and the LocalAI team · HTTP API | Local AI | B | 68 | 3.0 (2) | n/a | n/a | Free · OSS | OAuth or key | Local | ||
|
Open-source engine in Go, MIT licensed, that runs models on the owner's hardware behind OpenAI-, Anthropic-, Ollama- and ElevenLabs-compatible APIs on port 8080. Top strength MIT and Go, with Docker images for CUDA 12 and 13, ROCm, Intel oneAPI, Vulkan, Jetson and CPU, Linux binaries and a macOS app Top weakness No authentication by default. Loopback, LAN and VPN binds answer every caller, and keys set by environment variable grant full admin |
||||||||||||
| 233 | screenpipeNegentropy Labs, Inc. (dba Screenpipe) · Model platform | Local AI | C | 61.1 | 2.0 (2) | n/a | n/a | $21 / mo | API key | Local | ||
|
Desktop app and CLI from Negentropy Labs, Inc. (Screenpipe, YC S26) that records the owner's screen and audio continuously on macOS, Windows and Linux. Top strength 33 MCP tools with typed JSON Schemas, every one annotated, 21 marked read-only and Top weakness 33 tools at roughly 5,600 to 8,300 tokens of definitions with no toolsets, and the MCP docs page describes 2 of them |
||||||||||||
| 253 | llama.cppggml.ai (Hugging Face) · HTTP API | Local AI | C | 60.2 | 2.5 (2) | n/a | n/a | Free · OSS | None | Local | ||
|
Open-source C/C++ engine for running GGUF models locally, with a web interface and compatible model APIs. Top strength MIT, with no telemetry or update check in the source, and Top weakness API keys are off by default and CORS reflects any origin with credentials, so a web page can call a keyless server on localhost |
||||||||||||
| 287 | LM StudioElement Labs, Inc. · HTTP API | Local AI | C | 57.9 | 2.5 (2) | n/a | n/a | Free | API key | Local | ||
|
Desktop app and headless daemon from Element Labs for running open-weight models on the owner's machine with llama.cpp and MLX, plus the Splash engine on Apple silicon M3 or newer since 0.4.25. Top strength OpenAI-compatible chat completions, responses, completions and embeddings, Anthropic-compatible /v1/messages and a native /api/v1, all on one port Top weakness Authentication is off by default, so any local process can call the server |
||||||||||||
| 302 | OllamaOllama Inc. · HTTP API | Local AI | C | 56.6 | 2.5 (2) | n/a | n/a | $20 / mo | None | Local | ||
|
Open-source model runner for macOS, Windows and Linux, with a local API and a library of downloadable models. Top strength An OpenAPI 3.1 file for the 15 native operations and llms.txt with 68 links to Markdown pages Top weakness No credential on the local API, and any caller that reaches it can pull, push, create and delete models |
||||||||||||
| 330 | AnythingLLMMintplex Labs · Model platform | Local AI | D | 53.6 | 2.0 (2) | n/a | n/a | $50 / mo | API key | Local | ||
|
Open-source app for chatting with documents using local or hosted models. Available as a desktop app or a self-hosted server. Top strength MIT server with desktop builds for macOS, Windows and Linux and Docker images for amd64 and arm64 Top weakness One kind of API key, admin-equivalent across every endpoint, with no scopes or expiry, stored in plain text |
||||||||||||
| 345 | Open WebUIOpen WebUI Inc. · Model platform | Local AI | D | 52 | 2.5 (2) | n/a | n/a | Free | API key | Local | ||
|
Self-hosted web interface for chatting with models, from Open WebUI Inc., with a Python (FastAPI) back end and a Svelte front end. Top strength Five releases in the 90 days to 3 October 2026, each with a dated changelog entry that warns of database migrations Top weakness API keys are off by default, and each user gets one key with no scopes or expiry |
||||||||||||
| 349 | JanMenlo Research · Model platform | Local AI | D | 51.4 | 2.0 (2) | n/a | n/a | Free · OSS | API key | Local | ||
|
Open-source desktop app for running models locally or connecting to cloud models with the user's API keys. Top strength Apache-2.0, with installers for macOS, Windows and Linux plus Flathub and the Microsoft Store Top weakness No release since 0.8.4 on 23 July 2026, while a security fix waits on main |
||||||||||||
| 396 | LocalGhostLocalGhost · Model platform | Local AI | E | 45.8 | none | n/a | n/a | Free · OSS | Token | Local | ||
|
Pre-release, open-source personal AI server that runs on hardware its owner keeps, started in London in December 2025. Top strength MIT for the server and the Android app, with no telemetry, analytics or crash-reporting library in either Top weakness No API, MCP server or SDK for agents, and every caller but the paired phone gets the same 503 as a box that is down |
||||||||||||
| 426 | KhojKhoj Inc. · Model platform | Local AI | E | 38.8 | 1.0 (2) | n/a | n/a | Free · OSS | OAuth or key | Local | ||
|
Open-source personal AI application with a Python server and a web interface. Top strength AGPL-3.0-or-later, with the server, web app and Obsidian, Emacs and desktop clients in one public repository Top weakness No tagged release since 2.0.0-beta.28 on 26 March 2026 and no commit since 2 August |
||||||||||||
| 438 | GPT4AllNomic, Inc. · Model platform | Local AI | F | 36.3 | 1.0 (2) | n/a | n/a | Free · OSS | None | Local | ||
|
Desktop app from Nomic that runs GGUF models on Windows, macOS and Linux through Nomic's fork of llama.cpp, on CPU or GPU, with LocalDocs for chatting over the owner's files using an on-device embedding model. Top strength MIT, with installers for Windows x64 and ARM64, macOS 12.6 or later and Linux, and published minimum and recommended hardware Top weakness No release since 24 February 2025 and no commit to main since 27 May 2025 |
||||||||||||
| 444 | UnderdogConway Research · Model platform | Local AI | F | 29.9 | 2.0 (2) | n/a | n/a | Free | None | Local | ||
|
A personal AI from Conway Research that runs on the owner's Mac with Apple silicon. Top strength Apache-2.0 weights on Hugging Face for Underdog 27B 1.0, Woof 4B and 2B 1.1 and Bark 0.8B 1.0, with no gate Top weakness No API, MCP server, CLI or SDK of Conway's for agents, and the |
||||||||||||
Nothing matches these filters. .
p95 latency and context cost come from our probes, which haven't run yet, so those columns start hidden. Grades run from AA to F, and agent-ready means BB or better. Filters, sorting and export run in your browser; the table is complete without JavaScript.
How we test this category
In this run, public evidence against the published checklist, read as software the owner runs on their own hardware (the local-software lines for Reliability and the self-hosted rule for Payments). When the task suites run, the same small open model, prompts and documents on the same machine through each listing's local API or MCP server. We check the setup steps, tokens per second, memory use, whether answers cite the right file and what a network monitor sees leave the machine. This test hasn't run yet, so Task success is pending and the grades here come from the categories assessed from public evidence.
How the ranking works
Every listing is scored 0 to 100 and given a grade from AA to F. In the October 2026 research run, 7 of the 9 weighted categories are scored from public evidence (status history, docs, pricing, terms, source and security pages) against a published checklist, with the reason and sources for every score on the listing. Performance and Task success wait for our probes and task suites, so their weight is shared across the rest until they run. Negative events deduct up to 15 points. Read the methodology.
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.