Docker Model Runner vs Jan

Docker Model Runner

CI passes on the main branch, and Docker has published two security advisories with CVEs and fixed versions for the project. The API takes no credential, so any client or container that reaches it can pull, delete and run models, and the documentation has no OpenAPI file or error reference.

Jan

Apache-2.0, with installers for macOS, Windows and Linux plus Flathub and the Microsoft Store. No release since 0.8.4 on 23 July 2026, while a security fix waits on main.

Docker Model Runner

1. Use base URL http://localhost:12434/engines/v1 for OpenAI clients and http://localhost:12434 for Anthropic and Ollama clients. Any API key value is accepted
2. In Docker Desktop, run docker desktop enable model-runner --tcp 12434 first. Host-side TCP is off by default
3. From a container, call http://model-runner.docker.internal on Docker Desktop or http://172.17.0.1:12434 on Docker Engine
4. Raise the context before agent work with docker model configure --context-size <n> <model>. The llama.cpp default is 4,096 tokens
5. Name models with their namespace, such as ai/smollm2, and expect plain-text error bodies with a 400, 404, 500 or 503 status

Jan

1. Ask the owner to start the server (Settings, Local API Server) or run jan serve. Nothing listens until then
2. Call http://127.0.0.1:1337/v1 for the app and localhost:6767/v1 for jan serve. The ports differ
3. Read /openapi.json from the running server, not the spec on the docs site, which describes the retired Cortex API
4. Branch on the status code. Error bodies are plain text
5. Keep the bind at 127.0.0.1 on 0.8.4. Trusted Hosts is ignored on 0.0.0.0 until the next release