Ollama vs TextGen

Ollama

An OpenAPI 3.1 file for the 15 native operations and llms.txt with 68 links to Markdown pages. No credential on the local API, and any caller that reaches it can pull, push, create and delete models.

TextGen

AGPL-3.0 with no telemetry, and a local API on 127.0.0.1:5000 that checks the Host header, limits CORS to localhost and separates an admin key from the caller's key. No release since v4.9 on 20 May 2026, no code commits since 31 May, no test suite, and ten security advisories in the year, all fixed.

Ollama

1. Send "stream": false for one JSON body. The native routes stream NDJSON by default
2. Set OLLAMA_CONTEXT_LENGTH=64000 or options.num_ctx before agent work. The default is 4k below 24 GiB of VRAM
3. Back off on a 503. It means the queue (512 by default) is full
4. Put an authenticating proxy in front before binding past 127.0.0.1. The server checks no credential
5. Expect model names with a cloud tag to run on Ollama's servers. They need ollama signin and fail with OLLAMA_NO_CLOUD=1

TextGen

1. Ask the owner to launch with --api. Nothing listens on port 5000 without it, and a model must be loaded first
2. Call http://127.0.0.1:5000/v1. Any Host header other than localhost or 127.0.0.1 gets 400 Invalid host header unless --listen is set
3. Send the key as Authorization: Bearer on OpenAI routes and as x-api-key on /v1/messages. Model loading needs the admin key
4. Read http://127.0.0.1:5000/docs or modules/api/typing.py for parameters. max_tokens defaults to 512 on chat completions
5. Run tool calls yourself. The API returns finish_reason: "tool_calls" and executes nothing on the server
6. Use the repository name oobabooga/textgen. The old text-generation-webui URL redirects