Keep a 32k to 128k token agent session without silent truncation or a crash
One company's docs show a full fit.
What counts as done
A 60k token prompt is accepted whole and the server reports the active context length through its API.
Measured, for our own tests: the response shows no sign of a truncated prefix, and the same tool works at 60k as at 5k. No docs we read state these figures, so they don't count against a listing here.
How agents ask for it: set the context to 64k for Claude Code
, why did the model forget the start of the task
, run a 100k token repository read locally
, stop the model looping once the conversation is long
Does the job 1
The docs show it meets the test above, and nothing we read says otherwise.
Does the job, with a limit 10
It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.
| Listing | Why |
|---|---|
| Docker Model Runner C Docker, Inc. | Context is configured per model, but the account shows the stored value not always applied at load, with the packaged 128k default used instead. source |
| Jan C Menlo Research Pte Ltd | Context size is a per-model setting and Fit to Hardware caps it where enabled. Nothing documents the default, what happens past it, or an API field reporting the active value. source |
| KoboldCpp B LostRuins (Concedo) | --contextsize sets the window and /api/extra/true_max_context_length reports it over the API. Smart Context and context shift drop the oldest half when full rather than rejecting, so long sessions lose their prefix silently. source |
| Lemonade B Advanced Micro Devices, Inc. | ctx_size is set per load and /v1/models reports the active context_length. The Strix Halo issue shows a compute crash past 33k while 262k was advertised, with no maintainer reply when read. source |
| llama.cpp D ggml.ai, part of Hugging Face since 2026 | --ctx-size defaults to the model's trained length, /props and /slots expose n_ctx, responses carry a truncated flag. One account got 128k on 16 GB; another hit a tool loop past 30k until a parser fix. source |
| MLX LM C Apple Inc. | A rotating fixed-size KV cache keeps generation going by discarding older tokens, which is silent truncation of the prefix; the quantised cache disables batching. No API field reports the active context. source |
| Ollama D Ollama Inc. | OLLAMA_CONTEXT_LENGTH or num_ctx sets 64k and ollama ps shows the allocated context. The default is 4k below 24 GiB and prompts past it are truncated with no error, per two accounts. source |
| PrivateGPT C PriBAI Technology Spain S.L. | context_window must be set explicitly in the model profile to avoid overflow; the backend decides the rest. source |
| TextGen E oobabooga | ctx_size is set per load (32768 in the example) and --parallel N gives llama.cpp slots. No field reports the active context and nothing says whether over-length prompts are rejected. source |
| vLLM D The Linux Foundation (vLLM is a PyTorch Foundation project) | --max-model-len fixes the window and the KV pool is sized at start, with a Context Extension feature page. Whether an over-length request is rejected or truncated was not read. source |
Couldn't check 6
The docs were closed to us, or silent. These carry no mark either way.
| Listing | Why |
|---|---|
| Foundry Local B Microsoft Corporation | No context length setting or report appears in the REST, CLI or SDK reference pages read. |
| Core F ZMJ, Inc., doing business as Ghost, 325 9th Street, San Francisco, CA 94103, United States | No context lengths are stated for any listed model. |
| GPT4All F Nomic, Inc. | Context Length is a setting, but the docs state no default, no behaviour past it and no API field for the active value. |
| LM Studio C Element Labs, Inc. | The context length page (/docs/app/advanced/context-length) returns 404 and the developer docs read do not state the default or whether the API reports the active value. |
| LocalAI B Ettore Di Giacinto and the LocalAI team | Per-model context size configuration and its default were not found in the pages read. |
| Underdog F Conway Research | No context length is stated on the card. |
Doesn't do this job 7
Listings in the category that do something else, with the reason.
| Listing | Why |
|---|---|
| AnythingLLM E Mintplex Labs, Inc. | Context length belongs to the configured LLM provider. source |
| Berth B Trevin Tindall (T-Crypt on GitHub) | Context belongs to the engine; Berth only saves and restores its KV slots across switches. source |
| BitRun E Abhishek Khanra (IshekKhal on GitHub) | The largest catalogue context is 8,192 tokens. source |
| Khoj E Khoj Inc. | Context handling belongs to the configured model server. source |
| LocalGhost F LocalGhost.ai Ltd (company number 17213100, registered in England and Wales, incorporated 12 May 2026) | No agent sessions. source |
| Open WebUI D Open WebUI Inc. | Context length belongs to the connected backend; the interface sets none of its own. source |
| screenpipe C Negentropy Labs, Inc. (dba Screenpipe) | Not a model server. source |
What stops agents finishing
- default context of 2k or 4k with prompts silently truncated rather than rejected
- hardware backend crashes with a compute error past about 33k tokens while advertising 262k
- model repeats the same tool call once the context is long
Around it
Before: local-ai.fit-model-to-hardware. After: local-ai.coding-agent-backend.
Where this comes from
Weight 8 of 100 in Local AI: 8 of 50 accounts hit a context limit: the 2k or 4k default, a crash past 33k, or tool loops past 30k
The same as data: /api/v1/use-cases/local-ai.long-context-agent-session.json, or the MCP tool get_use_case with id local-ai.long-context-agent-session. Other jobs in this category: /api/v1/niches?category=local-ai.
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.