llama.cpp vs TextGen

llama.cpp

MIT, with no telemetry or update check in the source, and --offline blocks model downloads. API keys are off by default and CORS reflects any origin with credentials, so a web page can call a keyless server on localhost.

TextGen

AGPL-3.0 with no telemetry, and a local API on 127.0.0.1:5000 that checks the Host header, limits CORS to localhost and separates an admin key from the caller's key. No release since v4.9 on 20 May 2026, no code commits since 31 May, no test suite, and ten security advisories in the year, all fixed.

llama.cpp

1. Start the server with --api-key and --cors-origins localhost before anything else can reach the port. Both are off by default
2. Pass n_predict or max_tokens. Generation is unbounded by default
3. Send response_fields to /completion to drop the fields you don't read
4. Wait and retry on a 503 unavailable_error. The model is still loading
5. Read the server README of the build you run. Behaviour changes between nightly builds without a changelog entry

TextGen

1. Ask the owner to launch with --api. Nothing listens on port 5000 without it, and a model must be loaded first
2. Call http://127.0.0.1:5000/v1. Any Host header other than localhost or 127.0.0.1 gets 400 Invalid host header unless --listen is set
3. Send the key as Authorization: Bearer on OpenAI routes and as x-api-key on /v1/messages. Model loading needs the admin key
4. Read http://127.0.0.1:5000/docs or modules/api/typing.py for parameters. max_tokens defaults to 512 on chat completions
5. Run tool calls yourself. The API returns finish_reason: "tool_calls" and executes nothing on the server
6. Use the repository name oobabooga/textgen. The old text-generation-webui URL redirects