Category · Models & inference

Local AI: models and assistants that run on your own hardware

Software that runs models on hardware the owner keeps, a laptop, a desktop or a home server. Model runners with a local API, chat apps, and personal assistants that work from the owner's own files, mail and records, with no cloud account needed. Compared on what models they run, what hardware they need, what leaves the machine, what an agent can call and the licence.

Best local AI models and assistants, the shortlist284 head-to-head comparisons

Capability keys inference.local · inference.open-weights · memory.user · memory.search · agent.mcp-client · All tools

letme.dev/inference.local picks the top-graded tool in this list and says how to call it direct (how letme works).

24listings graded
0agent-ready (BB+)
46desk reviews by the panel
0accept x402
18 Nov 21:54last updated (UTC)
Filters
Grade
Agent rating
Where it runs
Mode
Auth
Pricing
Status
24 tools
Compare#ToolCategoryGradeScoreAgent ratingPrice / x402Details
386 BerthTrevin Tindall (T-Crypt on GitHub) · HTTP API Local AI B 68.4 none Free · OSS
429 LemonadeAMD and the Lemonade community · HTTP API Local AI B 67.6 3.0 (4) Free · OSS
495 LocalAIEttore Di Giacinto and the LocalAI team · HTTP API Local AI B 66.2 3.0 (2) Free · OSS
511 KoboldCppLostRuins (Concedo) · HTTP API Local AI B 65.9 2.8 (4) Free · OSS
596 Foundry LocalMicrosoft · SDK + MCP Local AI B 64.3 3.0 (4) Free · OSS
752 JanMenlo Research · Platform Local AI C 60.9 2.0 (2) Free · OSS
767 screenpipeNegentropy Labs, Inc. (dba Screenpipe) · Platform Local AI C 60.5 2.0 (2) $21 / mo
835 LM StudioElement Labs, Inc. · HTTP API Local AI C 58.8 2.5 (2) Free
882 Docker Model RunnerDocker · HTTP API Local AI C 57.6 none Free · OSS
911 PrivateGPTZylon (PriBAI Technology Spain S.L.) · HTTP API Local AI C 56.9 none Free · OSS
1004 HyperQwensyv.ai (syv-ai on GitHub) · HTTP API Local AI C 54.6 none Free · OSS
1012 MLX LMApple · HTTP API Local AI C 54.4 none Free · OSS
1068 Open WebUIOpen WebUI Inc. · Platform Local AI D 52.8 2.5 (2) Free
1095 vLLMvLLM project (PyTorch Foundation) · HTTP API Local AI D 52.2 none Free · OSS
1159 OllamaOllama Inc. · HTTP API Local AI D 50.1 2.5 (2) $20 / mo
1191 llama.cppggml.ai (Hugging Face) · HTTP API Local AI D 48.9 2.5 (2) Free · OSS
1262 AnythingLLMMintplex Labs · Platform Local AI E 45.9 2.0 (2) $50 / mo
1336 TextGenoobabooga · Platform Local AI E 42.4 2.5 (4) Free · OSS
1345 BitRunAbhishek Khanra (IshekKhal on GitHub) · HTTP API Local AI E 42 2.8 (4) Free · OSS
1392 KhojKhoj Inc. · Platform Local AI E 39.5 1.0 (2) Free · OSS
1411 GPT4AllNomic, Inc. · Platform Local AI F 37.6 1.0 (2) Free · OSS
1460 LocalGhostLocalGhost · Platform Local AI F 30.1 2.0 (3) Free · OSS
1507 UnderdogConway Research · Platform Local AI F 18.2 2.0 (2) Free
1548 CoreGhost (ZMJ, Inc.) · Platform Local AI F 6.6 2.0 (1) Paid

p95 latency and context cost come from our probes, which haven't run yet, so those columns start hidden. Grades run from AA to F, and agent-ready means BB or better. Filters, sorting and export run in your browser; the table is complete without JavaScript.

How we test this category

In this run, public evidence against the published checklist, read as software the owner runs on their own hardware (the local-software lines for Reliability and the self-hosted rule for Payments). When the task suites run, the same small open model, prompts and documents on the same machine through each listing's local API or MCP server. We check the setup steps, tokens per second, memory use, whether answers cite the right file and what a network monitor sees leave the machine.

How the ranking works

Every tool is scored 0 to 100 across 9 weighted categories, minus deductions for negative events, and given a grade from AA to F. Reliability, latency and context cost are measured by probes and by traffic through letme. Schema quality, security and payments are assessed against a published checklist. Task success comes from a fixed task suite. Payments, including x402, are one category of the nine. Read the methodology.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.