# Keep a 32k to 128k token agent session without silent truncation or a crash

> Keep a 32k to 128k token agent session without silent truncation or a crash: 1 companies fully do it, 10 with a limit, 26 carry the tag. Every listing in Local AI with the reason and a source.

- Canonical: https://www.anchorterminal.com/niches/local-ai.long-context-agent-session/
- Markdown: https://www.anchorterminal.com/niches/local-ai.long-context-agent-session/index.md (~1,650 tokens)
- Slim: https://www.anchorterminal.com/niches/local-ai.long-context-agent-session/index.min.md (~380 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/niches/local-ai.long-context-agent-session/index.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-15

One company's docs show a full fit.

| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |
| --- | --- | --- | --- | --- |
| 1 | 10 | 26 | 8 | 8 |

## What counts as done

A 60k token prompt is accepted whole and the server reports the active context length through its API.

Measured, for our own tests: the response shows no sign of a truncated prefix, and the same tool works at 60k as at 5k.

How agents ask for it: set the context to 64k for Claude Code; why did the model forget the start of the task; run a 100k token repository read locally; stop the model looping once the conversation is long

## Does the job

The docs show it meets the test above, and nothing we read says otherwise.

- **HyperQwen (syv.ai (Copenhagen, CVR DK44671247))**: CTX=long gives 150k and CTX=huge 240k with measured decode rates and cold time-to-first-token at 66k and 100k. vLLM sizes the KV pool at start and the README states the pool size per arm.

## Does the job, with a limit

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

- **Docker Model Runner (Docker, Inc.)**: Context is configured per model, but the account shows the stored value not always applied at load, with the packaged 128k default used instead.
- **Jan (Menlo Research Pte Ltd)**: Context size is a per-model setting and Fit to Hardware caps it where enabled. Nothing documents the default, what happens past it, or an API field reporting the active value.
- **KoboldCpp (LostRuins (Concedo))**: --contextsize sets the window and /api/extra/true_max_context_length reports it over the API. Smart Context and context shift drop the oldest half when full rather than rejecting, so long sessions lose their prefix silently.
- **Lemonade (Advanced Micro Devices, Inc.)**: ctx_size is set per load and /v1/models reports the active context_length. The Strix Halo issue shows a compute crash past 33k while 262k was advertised, with no maintainer reply when read.
- **llama.cpp (ggml.ai, part of Hugging Face since 2026)**: --ctx-size defaults to the model's trained length, /props and /slots expose n_ctx, responses carry a truncated flag. One account got 128k on 16 GB; another hit a tool loop past 30k until a parser fix.
- **MLX LM (Apple Inc.)**: A rotating fixed-size KV cache keeps generation going by discarding older tokens, which is silent truncation of the prefix; the quantised cache disables batching. No API field reports the active context.
- **Ollama (Ollama Inc.)**: OLLAMA_CONTEXT_LENGTH or num_ctx sets 64k and ollama ps shows the allocated context. The default is 4k below 24 GiB and prompts past it are truncated with no error, per two accounts.
- **PrivateGPT (PriBAI Technology Spain S.L.)**: context_window must be set explicitly in the model profile to avoid overflow; the backend decides the rest.
- **TextGen (oobabooga)**: ctx_size is set per load (32768 in the example) and --parallel N gives llama.cpp slots. No field reports the active context and nothing says whether over-length prompts are rejected.
- **vLLM (The Linux Foundation (vLLM is a PyTorch Foundation project))**: --max-model-len fixes the window and the KV pool is sized at start, with a Context Extension feature page. Whether an over-length request is rejected or truncated was not read.

## Couldn't check

The docs were closed to us, or silent. These carry no mark either way.

- **Foundry Local (Microsoft Corporation)**: No context length setting or report appears in the REST, CLI or SDK reference pages read.
- **Core (ZMJ, Inc., doing business as Ghost, 325 9th Street, San Francisco, CA 94103, United States)**: No context lengths are stated for any listed model.
- **GPT4All (Nomic, Inc.)**: Context Length is a setting, but the docs state no default, no behaviour past it and no API field for the active value.
- **LM Studio (Element Labs, Inc.)**: The context length page (/docs/app/advanced/context-length) returns 404 and the developer docs read do not state the default or whether the API reports the active value.
- **LocalAI (Ettore Di Giacinto and the LocalAI team)**: Per-model context size configuration and its default were not found in the pages read.
- **Underdog (Conway Research)**: No context length is stated on the card.

- not this job: AnythingLLM (Mintplex Labs, Inc.): Context length belongs to the configured LLM provider.
- not this job: Berth (Trevin Tindall (T-Crypt on GitHub)): Context belongs to the engine; Berth only saves and restores its KV slots across switches.
- not this job: BitRun (Abhishek Khanra (IshekKhal on GitHub)): The largest catalogue context is 8,192 tokens.
- not this job: Khoj (Khoj Inc.): Context handling belongs to the configured model server.
- not this job: LocalGhost (LocalGhost.ai Ltd (company number 17213100, registered in England and Wales, incorporated 12 May 2026)): No agent sessions.
- not this job: Open WebUI (Open WebUI Inc.): Context length belongs to the connected backend; the interface sets none of its own.
- not this job: screenpipe (Negentropy Labs, Inc. (dba Screenpipe)): Not a model server.

## What stops agents finishing

- default context of 2k or 4k with prompts silently truncated rather than rejected
- hardware backend crashes with a compute error past about 33k tokens while advertising 262k
- model repeats the same tool call once the context is long

## Around it

Before: local-ai.fit-model-to-hardware. After: local-ai.coding-agent-backend.

## Where this comes from

Weight 8 of 100 in Local AI: 8 of 50 accounts hit a context limit: the 2k or 4k default, a crash past 33k, or tool loops past 30k

As data: https://www.anchorterminal.com/api/v1/use-cases/local-ai.long-context-agent-session.json, or the MCP tool get_use_case with id local-ai.long-context-agent-session. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp
