KoboldCpp vs MLX LM

KoboldCpp

One file runs text, image, speech and music models behind a published OpenAPI 3.0.3 document, with eight releases in 90 days. The server listens on every interface with no password by default, and --password leaves the image routes open.

MLX LM

MIT, with no telemetry found in the source, and the tests passed on the last eight pushes to main. mlx_lm.server has no API key option, answers any origin by default and loads whichever model a request names, and its own docs say it is not recommended for production.

KoboldCpp

1. Start with --host 127.0.0.1 and --password. The default listens on every interface with no key
2. Send the password as Authorization: Bearer <password>. It is not read from the query string
3. Treat 503 as both busy and rate limited. The server never sends 429 or Retry-After, and the wait in seconds is in detail.msg
4. Pass max_length or max_tokens. The default is 2,048 tokens unless --defaultgenamt changes it
5. Send a genkey with each generation so /api/extra/generate/check and /api/extra/abort act on your request and not another caller's

MLX LM

1. Keep mlx_lm.server on 127.0.0.1 and pass --allowed-origins with the origins you trust. There is no API key, and the default answers every origin
2. Treat any caller as able to load any model. The model and adapters request fields accept any Hugging Face repository or local path
3. Send max_tokens or max_completion_tokens when you need more than 512 tokens, the server default
4. Read errors as {"error": "<text>"} with 400 for a bad field and 404 for a model that failed to load. They are not OpenAI error objects
5. Poll GET /health before the first request. It answers 503 with unavailable when the generation thread has stopped