# Local AI: models and assistants that run on your own hardware

> 12 local AI listings ranked by the Anchor benchmark. Leader LocalAI (B). Software that runs models on hardware the owner keeps, a laptop, a desktop or a home server. Model runners with a local API, chat apps, and personal assistants that work from the owner's own files, mail and records, with no cloud account needed. Compared on what models they run, what hardware they need, what leaves the machine, what an agent can call and the licence.

- Canonical: https://www.anchorterminal.com/categories/local-ai
- Markdown: https://www.anchorterminal.com/categories/local-ai.md (~3,200 tokens)
- Slim: https://www.anchorterminal.com/categories/local-ai.min.md (~580 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/categories/local-ai.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (full text: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- Updated: 2026-10-04

Software that runs models on hardware the owner keeps, a laptop, a desktop or a home server. Model runners with a local API, chat apps, and personal assistants that work from the owner's own files, mail and records, with no cloud account needed. Compared on what models they run, what hardware they need, what leaves the machine, what an agent can call and the licence.

- Tools ranked: 12 · agent-ready (BB or better): 0 · accept x402: 0 · hosted endpoints: 0 · desk reviews by the panel: 22
- JSON: https://www.anchorterminal.com/api/v1/tools.json (list) · https://www.anchorterminal.com/api/v1/rankings.json (ranked) · https://www.anchorterminal.com/api/v1/x402.json (payable) · https://www.anchorterminal.com/api/v1/capabilities.json (by capability)
- Grades run AA, A, BB, B, C, D, E, F · methodology: https://www.anchorterminal.com/benchmark/

- Capabilities in this category: inference.local, inference.open-weights, memory.user, memory.search, agent.mcp-client
- https://letme.dev/inference.local picks the top-graded tool in this list and says how to call it direct; calling through letme comes later (https://www.anchorterminal.com/letme/index.md)

## Ranking

| # | Tool | Vendor | Kind | Category | Grade | Score | Confidence | x402 | Auth | Where | Reviews | Page |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 133 | LocalAI | Ettore Di Giacinto and the LocalAI team | HTTP API | Local AI | B | 68 | medium | no | OAuth or key | local | 3/5 (2) | https://www.anchorterminal.com/tools/localai.md |
| 233 | screenpipe | Negentropy Labs, Inc. (dba Screenpipe) | Model platform | Local AI | C | 61.1 | medium | no | API key | local | 2/5 (2) | https://www.anchorterminal.com/tools/screenpipe.md |
| 253 | llama.cpp | ggml.ai (Hugging Face) | HTTP API | Local AI | C | 60.2 | medium | no | None | local | 2.5/5 (2) | https://www.anchorterminal.com/tools/llama-cpp.md |
| 287 | LM Studio | Element Labs, Inc. | HTTP API | Local AI | C | 57.9 | medium | no | API key | local | 2.5/5 (2) | https://www.anchorterminal.com/tools/lm-studio.md |
| 302 | Ollama | Ollama Inc. | HTTP API | Local AI | C | 56.6 | medium | no | None | local | 2.5/5 (2) | https://www.anchorterminal.com/tools/ollama.md |
| 330 | AnythingLLM | Mintplex Labs | Model platform | Local AI | D | 53.6 | medium | no | API key | local | 2/5 (2) | https://www.anchorterminal.com/tools/anythingllm.md |
| 345 | Open WebUI | Open WebUI Inc. | Model platform | Local AI | D | 52 | medium | no | API key | local | 2.5/5 (2) | https://www.anchorterminal.com/tools/open-webui.md |
| 349 | Jan | Menlo Research | Model platform | Local AI | D | 51.4 | medium | no | API key | local | 2/5 (2) | https://www.anchorterminal.com/tools/jan.md |
| 396 | LocalGhost | LocalGhost | Model platform | Local AI | E | 45.8 | medium | no | Token | local | none | https://www.anchorterminal.com/tools/localghost.md |
| 426 | Khoj | Khoj Inc. | Model platform | Local AI | E | 38.8 | medium | no | OAuth or key | local | 1/5 (2) | https://www.anchorterminal.com/tools/khoj.md |
| 438 | GPT4All | Nomic, Inc. | Model platform | Local AI | F | 36.3 | high | no | None | local | 1/5 (2) | https://www.anchorterminal.com/tools/gpt4all.md |
| 444 | Underdog | Conway Research | Model platform | Local AI | F | 29.9 | low | no | None | local | 2/5 (2) | https://www.anchorterminal.com/tools/underdog.md |

Scores are from public evidence against the published checklist (https://www.anchorterminal.com/benchmark/), with Performance and Task success pending. p95 latency and context cost come from our probes, which haven't run yet.

## Summaries

### 133. LocalAI, B (68)

Open-source engine in Go, MIT licensed, that runs models on the owner's hardware behind OpenAI-, Anthropic-, Ollama- and ElevenLabs-compatible APIs on port 8080. MIT and Go, with Docker images for CUDA 12 and 13, ROCm, Intel oneAPI, Vulkan, Jetson and CPU, Linux binaries and a macOS app. No authentication by default. Loopback, LAN and VPN binds answer every caller, and keys set by environment variable grant full admin.

- Page: https://www.anchorterminal.com/tools/localai · Markdown: https://www.anchorterminal.com/tools/localai.md · JSON: https://www.anchorterminal.com/api/v1/tools/localai.json
- Capabilities: inference.local, inference.open-weights, agent.mcp-client, embed.text, rerank, speech.stt, speech.tts, voice.speech-to-speech, image.generate, video.generate, guard.pii, finetune.sft, db.vector

### 233. screenpipe, C (61.1)

Desktop app and CLI from Negentropy Labs, Inc. (Screenpipe, YC S26) that records the owner's screen and audio continuously on macOS, Windows and Linux. 33 MCP tools with typed JSON Schemas, every one annotated, 21 marked read-only and `merge-speakers` marked destructive. 33 tools at roughly 5,600 to 8,300 tokens of definitions with no toolsets, and the MCP docs page describes 2 of them.

- Page: https://www.anchorterminal.com/tools/screenpipe · Markdown: https://www.anchorterminal.com/tools/screenpipe.md · JSON: https://www.anchorterminal.com/api/v1/tools/screenpipe.json
- Capabilities: memory.user, memory.search, agent.mcp-client, inference.local, speech.stt, speech.diarisation

### 253. llama.cpp, C (60.2)

Open-source C/C++ engine for running GGUF models locally, with a web interface and compatible model APIs. MIT, with no telemetry or update check in the source, and `--offline` blocks model downloads. API keys are off by default and CORS reflects any origin with credentials, so a web page can call a keyless server on localhost.

- Page: https://www.anchorterminal.com/tools/llama-cpp · Markdown: https://www.anchorterminal.com/tools/llama-cpp.md · JSON: https://www.anchorterminal.com/api/v1/tools/llama-cpp.json
- Capabilities: inference.local, inference.open-weights, embed.text, rerank, inference.decision, agent.mcp-client

### 287. LM Studio, C (57.9)

Desktop app and headless daemon from Element Labs for running open-weight models on the owner's machine with llama.cpp and MLX, plus the Splash engine on Apple silicon M3 or newer since 0.4.25. OpenAI-compatible chat completions, responses, completions and embeddings, Anthropic-compatible /v1/messages and a native /api/v1, all on one port. Authentication is off by default, so any local process can call the server.

- Page: https://www.anchorterminal.com/tools/lm-studio · Markdown: https://www.anchorterminal.com/tools/lm-studio.md · JSON: https://www.anchorterminal.com/api/v1/tools/lm-studio.json
- Capabilities: inference.local, inference.open-weights, agent.mcp-client, embed.text

### 302. Ollama, C (56.6)

Open-source model runner for macOS, Windows and Linux, with a local API and a library of downloadable models. An OpenAPI 3.1 file for the 15 native operations and llms.txt with 68 links to Markdown pages. No credential on the local API, and any caller that reaches it can pull, push, create and delete models.

- Page: https://www.anchorterminal.com/tools/ollama · Markdown: https://www.anchorterminal.com/tools/ollama.md · JSON: https://www.anchorterminal.com/api/v1/tools/ollama.json
- Capabilities: inference.local, inference.open-weights, inference.llm, embed.text, inference.decision, web.search, web.fetch

### 330. AnythingLLM, D (53.6)

Open-source app for chatting with documents using local or hosted models. Available as a desktop app or a self-hosted server. MIT server with desktop builds for macOS, Windows and Linux and Docker images for amd64 and arm64. One kind of API key, admin-equivalent across every endpoint, with no scopes or expiry, stored in plain text.

- Page: https://www.anchorterminal.com/tools/anythingllm · Markdown: https://www.anchorterminal.com/tools/anythingllm.md · JSON: https://www.anchorterminal.com/api/v1/tools/anythingllm.json
- Capabilities: inference.local, memory.search, agent.mcp-client, memory.user, inference.open-weights

### 345. Open WebUI, D (52)

Self-hosted web interface for chatting with models, from Open WebUI Inc., with a Python (FastAPI) back end and a Svelte front end. Five releases in the 90 days to 3 October 2026, each with a dated changelog entry that warns of database migrations. API keys are off by default, and each user gets one key with no scopes or expiry.

- Page: https://www.anchorterminal.com/tools/open-webui · Markdown: https://www.anchorterminal.com/tools/open-webui.md · JSON: https://www.anchorterminal.com/api/v1/tools/open-webui.json
- Capabilities: inference.local, agent.mcp-client, memory.user, knowledge.search

### 349. Jan, D (51.4)

Open-source desktop app for running models locally or connecting to cloud models with the user's API keys. Apache-2.0, with installers for macOS, Windows and Linux plus Flathub and the Microsoft Store. No release since 0.8.4 on 23 July 2026, while a security fix waits on main.

- Page: https://www.anchorterminal.com/tools/jan · Markdown: https://www.anchorterminal.com/tools/jan.md · JSON: https://www.anchorterminal.com/api/v1/tools/jan.json
- Capabilities: inference.local, inference.open-weights, agent.mcp-client

### 396. LocalGhost, E (45.8)

Pre-release, open-source personal AI server that runs on hardware its owner keeps, started in London in December 2025. The server and Android app are MIT-licensed, with no telemetry libraries found. There is no agent API, MCP server or SDK; unpaired callers receive HTTP 503.

- Page: https://www.anchorterminal.com/tools/localghost · Markdown: https://www.anchorterminal.com/tools/localghost.md · JSON: https://www.anchorterminal.com/api/v1/tools/localghost.json
- Capabilities: memory.store, memory.search, memory.user, memory.delete

### 426. Khoj, E (38.8)

Open-source personal AI application with a Python server and a web interface. AGPL-3.0-or-later, with the server, web app and Obsidian, Emacs and desktop clients in one public repository. No tagged release since 2.0.0-beta.28 on 26 March 2026 and no commit since 2 August.

- Page: https://www.anchorterminal.com/tools/khoj · Markdown: https://www.anchorterminal.com/tools/khoj.md · JSON: https://www.anchorterminal.com/api/v1/tools/khoj.json
- Capabilities: memory.search, memory.user, inference.local, agent.mcp-client

### 438. GPT4All, F (36.3)

Desktop app from Nomic that runs GGUF models on Windows, macOS and Linux through Nomic's fork of llama.cpp, on CPU or GPU, with LocalDocs for chatting over the owner's files using an on-device embedding model. MIT, with installers for Windows x64 and ARM64, macOS 12.6 or later and Linux, and published minimum and recommended hardware. No release since 24 February 2025 and no commit to main since 27 May 2025.

- Page: https://www.anchorterminal.com/tools/gpt4all · Markdown: https://www.anchorterminal.com/tools/gpt4all.md · JSON: https://www.anchorterminal.com/api/v1/tools/gpt4all.json
- Capabilities: inference.local, inference.open-weights, memory.search, embed.text

### 444. Underdog, F (29.9)

A personal AI from Conway Research that runs on the owner's Mac with Apple silicon. Apache-2.0 weights on Hugging Face for Underdog 27B 1.0, Woof 4B and 2B 1.1 and Bark 0.8B 1.0, with no gate. No API, MCP server, CLI or SDK of Conway's for agents, and the `husky serve` command on the husky-flash card comes from a repository that isn't public.

- Page: https://www.anchorterminal.com/tools/underdog · Markdown: https://www.anchorterminal.com/tools/underdog.md · JSON: https://www.anchorterminal.com/api/v1/tools/underdog.json
- Capabilities: inference.open-weights, speech.stt

## How we test this category

In this run, public evidence against the published checklist, read as software the owner runs on their own hardware (the local-software lines for Reliability and the self-hosted rule for Payments). When the task suites run, the same small open model, prompts and documents on the same machine through each listing's local API or MCP server. We check the setup steps, tokens per second, memory use, whether answers cite the right file and what a network monitor sees leave the machine. This test hasn't run yet, so Task success is pending and the grades here come from the categories assessed from public evidence.