Foundry Local vs MLX LM

Foundry Local

The SDK and its native runtime are MIT, at 2.1.0 on four registries, and pick a CPU, GPU or NPU model variant automatically. The optional local server has no credential, its current routes aren't in the published REST reference, and telemetry is on by default with an opt-out.

MLX LM

MIT, with no telemetry found in the source, and the tests passed on the last eight pushes to main. mlx_lm.server has no API key option, answers any origin by default and loads whichever model a request names, and its own docs say it is not recommended for production.

Foundry Local

1. Read the server URL from manager.urls[0] or foundry server status. The port is dynamic unless the owner sets web.urls or foundry server start --port
2. Send the model ID that GET /v1/models returns, not the alias. The alias resolves to a hardware-specific variant
3. Check supportsToolCalling before sending tools. Support differs by variant, and issue #1183 reports Qwen tool calling failing on QNN
4. Set your own request timeout. Inference has no built-in one, and cancellation takes effect only after the current generation step
5. Ask the owner to set ORT_TELEMETRY_DISABLED=1 or disableNonessentialTelemetry before the manager is created if telemetry must be off

MLX LM

1. Keep mlx_lm.server on 127.0.0.1 and pass --allowed-origins with the origins you trust. There is no API key, and the default answers every origin
2. Treat any caller as able to load any model. The model and adapters request fields accept any Hugging Face repository or local path
3. Send max_tokens or max_completion_tokens when you need more than 512 tokens, the server default
4. Read errors as {"error": "<text>"} with 400 for a bad field and 404 for a model that failed to load. They are not OpenAI error objects
5. Poll GET /health before the first request. It answers 503 with unavailable when the generation thread has stopped