Docker Model Runner vs MLX LM

Docker Model Runner

CI passes on the main branch, and Docker has published two security advisories with CVEs and fixed versions for the project. The API takes no credential, so any client or container that reaches it can pull, delete and run models, and the documentation has no OpenAPI file or error reference.

MLX LM

MIT, with no telemetry found in the source, and the tests passed on the last eight pushes to main. mlx_lm.server has no API key option, answers any origin by default and loads whichever model a request names, and its own docs say it is not recommended for production.

Docker Model Runner

1. Use base URL http://localhost:12434/engines/v1 for OpenAI clients and http://localhost:12434 for Anthropic and Ollama clients. Any API key value is accepted
2. In Docker Desktop, run docker desktop enable model-runner --tcp 12434 first. Host-side TCP is off by default
3. From a container, call http://model-runner.docker.internal on Docker Desktop or http://172.17.0.1:12434 on Docker Engine
4. Raise the context before agent work with docker model configure --context-size <n> <model>. The llama.cpp default is 4,096 tokens
5. Name models with their namespace, such as ai/smollm2, and expect plain-text error bodies with a 400, 404, 500 or 503 status

MLX LM

1. Keep mlx_lm.server on 127.0.0.1 and pass --allowed-origins with the origins you trust. There is no API key, and the default answers every origin
2. Treat any caller as able to load any model. The model and adapters request fields accept any Hugging Face repository or local path
3. Send max_tokens or max_completion_tokens when you need more than 512 tokens, the server default
4. Read errors as {"error": "<text>"} with 400 for a bad field and 404 for a model that failed to load. They are not OpenAI error objects
5. Poll GET /health before the first request. It answers 503 with unavailable when the generation thread has stopped