{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/open-weights.private-chat-knowledge/",
    "json": "https://www.anchorterminal.com/niches/open-weights.private-chat-knowledge/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/open-weights.private-chat-knowledge/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/open-weights.private-chat-knowledge/index.min.md"
  },
  "markdown": "Four or more companies' docs show a full fit.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 8 | 19 | 90 | 8 | 4 |\n\n## What counts as done\n\nThe model answers a question from the retrieved passages given in the prompt.\n\nMeasured, for our own tests: the answer uses only the retrieved passages, is factually correct against them in 9 of 10 test questions, and arrives in under 5 seconds for a one-sentence reply.\n\nHow agents ask for it: chat over my Obsidian vault locally; a translation and chat model that stays on my laptop; answer from my wiki in one spoken sentence; a personal knowledge base with bug triage\n\n## Does the job\n\nThe docs show it meets the test above, and nothing we read says otherwise.\n\n- **Gemma 4 12B IT (Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author))**: 12B dense under Apache 2.0 with thinking off by default fits a 16 GB GPU at Q4_0 (7 GB) and 64K of context (about 1.6 GB with 8 of 48 layers global); Ollama and llama.cpp both serve it.\n- **Gemma 4 E4B IT (Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author))**: Apache 2.0, 5.15 GB Q4_0, thinking off by default, 131K context and multilingual speech and text; acc-45 ran it on a 24 GB laptop across three languages.\n- **Granite 4.2 3B (International Business Machines Corporation (per IBM's naming guidance, as read for granite-guardian))**: Apache 2.0, 2.2 GB at Q4_K_M, 131K context, enable_thinking false for direct answers and RAG named among its uses; acc-26 synthesised Wikipedia chunks into one spoken sentence with it.\n- **MiniCPM-o 4.5 (OpenBMB (named as the MiniCPM-o team on the technical report; the GitHub LICENSE is the unfilled Apache template))**: Apache 2.0 on the card, Q4_0 4.8 GB plus a 1.1 GB vision file fits a laptop, enable_thinking false gives direct answers, and it can speak the reply through OpenBMB's speech stack.\n- **NVIDIA Nemotron 3.5 Bolt E4B 4GB (NVIDIA Corporation)**: conversational_mode appends a brevity instruction, thinking defaults off, RAGTruth 77.9 and GaRAGe 40.7 at Q4_K_M (NVIDIA's runs), and 6 GB fits any laptop.\n- **Qwen3-4B-Instruct-2507 (Alibaba Cloud (named in the licence's copyright line))**: 4B Apache 2.0 instruct model with no thinking block, 262K context, 2.5 GB in Ollama; answers start at once on a laptop.\n- **Qwen3.5-4B (Alibaba Cloud (named in the licence's copyright line))**: 4B Apache 2.0 vision-language model, thinking off with enable_thinking false, 2.7 GB at Q4 and 262K context; a one-sentence answer arrives in seconds.\n- **Qwen3.5-9B (Alibaba Cloud (named in the licence's copyright line))**: 9B Apache 2.0 model with enable_thinking false per request, 6.6 GB in Ollama, 262K context; acc-25 built a Czech home assistant on it.\n- **Qwen3.6-27B (Alibaba Cloud (named in the licence's copyright line))**: Apache 2.0, enable_thinking false per request, 17 GB in Ollama with vision; acc-50 runs a personal wiki with summarisation and triage on it.\n- **Qwen3.8-27B (Alibaba Cloud (named in the licence's copyright line))**: Apache 2.0, enable_thinking false and reasoning_effort per request, 18 GB in Ollama with vision; acc-39 uses it as the think and vision model on a Mac.\n- **Qwen3-VL-4B-Instruct (Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder))**: Apache 2.0 by metadata and the GitHub LICENSE, no thinking block, 2.5 GB at Q4 with 262K context; an instruct model that answers at once.\n- **Qwen3-VL-8B-Instruct (Alibaba Cloud (per the pilot's RDAP check of qwen.ai; this model's files name no copyright holder))**: Apache 2.0 by metadata and the GitHub LICENSE, no thinking block, 5.0 GB at Q4 with 262K context; an instruct model that answers at once.\n- **Spark-X2.6-4B (XHToken (the licence's copyright line); the Hugging Face organisation's name is SparkLLM)**: 4B Apache 2.0 with a LICENSE file, thinking off per request, 3.5 GB GGUF and a 1M context; a short answer needs no reasoning block.\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **Agents-A1-4B (Shanghai Artificial Intelligence Laboratory (the InternAgent team there, per the card's recommended system prompt and the template's default prompt); the GitHub LICENSE leaves the copyright line as the Apache template)**: Apache 2.0, 2.7 GB GGUF and a thinking switch fit a laptop, but the model is tuned for long-horizon agent work and inserts an Intern-A1 system prompt with a fixed date unless you send your own.\n- **Apertus v1.5 8B (ETH Zurich and EPFL (the Swiss National AI Institute, named in the Acceptable Use Policy))**: A multilingual 8B aimed at precisely this, but the download is gated behind an Acceptable Use Policy and the architecture needs Swiss AI's forked vLLM, not a laptop runtime.\n- **Gemma 4 26B A4B IT (Google LLC (RDAP registrant of gemma.ai; the card names Google DeepMind as author))**: Fast at 3.8B active with thinking off by default, but Q4_0 is 14.4 GB and Ollama defaults to 4K context under 24 GiB VRAM; a 16 GB GPU leaves little room for 64K of retrieved text.\n- **Gemma 4 31B IT (Google LLC (the registrant of gemma.ai; the card names Google DeepMind as author))**: acc-02 uses Gemma 4 31B for chat and translation on a 128 GB laptop; at 17.6 GB Q4_0 it needs more than a 16 GB GPU, and 64K of context adds about 16 GB of cache at bf16.\n- **Gemma 4 E2B IT (Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author))**: Runs on any laptop under Apache 2.0 with thinking off, but acc-45 found E2B garbled and weak against E4B; faithfulness in 9 of 10 answers is unlikely at this size.\n- **GLM-4.7-Flash (JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai))**: MIT, thinking off per request and 3B active, but 30B total and 17.5 GB at Q4 exceed a 16 GB GPU; a 24 GB machine works.\n- **gpt-oss-20b (OpenAI (named in USAGE_POLICY and the model card; the LICENSE file's copyright line is the unfilled Apache template))**: Apache 2.0 and 12 GB on a 16 GB GPU, but every reply is preceded by an analysis channel that cannot be disabled, so a one-sentence spoken answer in under 5 seconds is uncertain.\n- **Laguna XS 2.1 (Poolside, Inc. (named in Poolside's Terms of Use))**: OpenMDW licence and thinking off suit it, but 20 GB Q4_K_M needs a 32 GB laptop rather than a 16 GB GPU.\n- **LFM2.5-2.6B (Liquid AI, Inc. (the licensor named in the LFM Open License))**: Fits a laptop and the licence file is present, but commercial use is licensed only below a 10 million dollar revenue threshold and every reply starts with reasoning.\n- **Ling-3.0-tiny (Ant Intelligence (Hang Zhou) Technology Co., Ltd (named as provider in the training-content summary); inclusionAI, Ant Group in the site footer of inclusion-ai.org)**: MIT-tagged, small and switchable, but the repository has no LICENSE file and a discussion reports loops.\n- **MiMo-V2.6 Distill Qwen 9B (Xiaomi (the card and report name Xiaomi MiMo; no LICENSE file names a copyright holder))**: MIT-tagged and small, but no LICENSE file and no copy of the base's Apache notice; thinking switches off.\n- **MiniCPM5-2B (OpenBMB (named in the GitHub LICENSE's copyright line))**: Apache 2.0 and tiny, but the model reasons before every short answer and users report it getting stuck in loops on simple questions.\n- **Muse Glimmer 30B (Meta (the verified Hugging Face organisation is \"Meta Inc.\", the card names Meta Superintelligence Lab, and the licence carries no copyright line))**: Apache 2.0 and 128K context, but the 17 GB build misses a 16 GB GPU and thinking at low strength still precedes every short answer.\n- **Nanbeige4.3-33B (BOSS Zhipin (Kanzhun Limited), unconfirmed)**: Thinking switches off, but 33B is above the use case's 31B range and only 3-bit builds (14.5 GB to 15.9 GB) fit a 16 GB GPU; without a system prompt the template injects a Chinese identity line.\n- **Nemotron 3.5 Lightning 30B A3B (NVIDIA Corporation)**: Thinking switches off and the licence is permissive, but the 18.9 GB build misses a 16 GB GPU and needs a 24 GB card or 32 GB laptop.\n- **Nemotron 3 Nano 30B A3B (NVIDIA Corporation)**: Thinking switches off, but the 18 GB build misses a 16 GB GPU, and the card says it is intended for English and code.\n- **Nemotron 3 Nano 4B (NVIDIA Corporation)**: Fits any laptop with thinking off and RULER 128K 91.1, but it is English only and no grounding or brevity result is published.\n- **Nemotron 3 Nano Omni 30B A3B Reasoning (NVIDIA Corporation)**: Instruct mode is short and fast, but 18.5 GB-plus misses a 16 GB GPU and the card states English only.\n- **occamy-1.0 (Accio (Accio-Lab))**: Thinking switches off and 3-bit builds of 14.5 GB to 16.8 GB reach a 16 GB GPU, but languages aren't stated and no grounding result is published.\n- **Ornith-1.5-35B-A3B (Ornith)**: MIT is tagged but the repository has no LICENSE file and the card's link is dead; 21.7 GB needs 24 to 32 GB.\n- **Ornith-1.5-9B (Ornith)**: Small and switchable, but the MIT licence is a tag with no LICENSE file and the card's link is dead, and default reasoning is long.\n- **Qwen3-30B-A3B-Instruct-2507 (Alibaba Cloud (named in the licence's copyright line))**: Apache 2.0, non-thinking and 262K context suit short grounded answers, but the 19 GB build misses a 16 GB GPU without offload.\n- **Qwen3.5-0.8B (Alibaba Cloud (named in the licence's copyright line))**: Below the 4B floor the use case states and scoped by the card to prototyping; Apache 2.0 and thinking off, so short answers are quick.\n- **Qwen3.5-2B (Alibaba Cloud (named in the licence's copyright line))**: Below the 4B floor the use case states and scoped by the card to prototyping; Apache 2.0 and thinking off, so short answers are quick.\n- **Qwen3.6-35B-A3B (Alibaba Cloud (named in the licence's copyright line))**: Apache 2.0 and thinking off per request, but 22 GB at 4 bits does not fit a 16 GB GPU; acc-02 reaches for it on a 128 GB laptop.\n- **Qwen3-VL-2B-Instruct (Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder))**: Below the 4B floor the use case states; Apache 2.0 by metadata, no thinking, 1.1 GB at Q4.\n- **Qwen3-VL-30B-A3B-Instruct (Alibaba Cloud (per the pilot's RDAP check of qwen.ai; this model's files name no copyright holder))**: Apache 2.0 and no thinking, but 18.6 GB at Q4 passes a 16 GB GPU; fits a 32 GB laptop.\n- **Qwen3-VL-32B-Instruct (Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder))**: Apache 2.0 and no thinking, but 19.8 GB at Q4 passes a 16 GB GPU and 64 layers of full attention cost 16 GB of KV at 64K.\n- **Qwopus3.8-27B-Flash-V2 (Jackrong, an individual Hugging Face account)**: Apache 2.0 by metadata but no LICENSE file, thinking off per request, 17 GB with vision; trained for shorter agent reasoning rather than grounded answers, and no account.\n- **SmolLM3-3B (Hugging Face, Inc.)**: Apache 2.0 by metadata, thinking off per request and 1.9 GB at Q4, but below the 4B floor and no LICENSE file in the repository.\n- **Swift 1.5 Qwen3.8-27B (UkisAI)**: Thinking off per request, 17 GB with vision, but the Swift Open License is free only below US$1M gross annual revenue; not a plain commercial licence.\n- **ThinkingCap (Qwen3.8 27B) (BottleCap AI (named in the licence's copyright line))**: Thinking off per request and 17 GB with vision, but free only for personal use and companies under 100 people and US$1M revenue; not a plain commercial licence.\n\n- not this job: Apodex-1.1-mini (Apodex (the site's copyright line reads 2026 Apodex)): 36B total parameters and a 21.7 GB smallest build exceed the use case's 4B to 31B on a 16 GB GPU, and the tune is for long-horizon agents.\n- not this job: BigBang-v1 (The Endless Frontier (endless-frontier)): 36B total and a research-tuned reasoning model with BF16 only from the publisher; outside the 4B to 31B laptop band.\n- not this job: Carbon-X (Hugging Face (the HuggingFaceBio organisation, named Hugging Face Biology Research on the Hub)): Answers questions about variants and regulatory regions from DNA, not from retrieved notes.\n- not this job: Cosmos-Reason2-2B (NVIDIA Corporation): A camera-footage reasoner, not a notes assistant; NVIDIA points users to Cosmos 3 and no longer develops it.\n- not this job: Darwin-36B-Opus (VIDRAFT Inc. (named on vidraft.net); the card's citation names FINAL-Bench and VIDRAFT_LAB): A 35B reasoning merge whose answers begin with a think trace; outside the 4B to 31B short-answer band.\n- not this job: DeepSeek-V3.2 (DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)): A 671B model; not a laptop or 16 GB GPU fit.\n- not this job: DeepSeek V4.1 Flash (DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)): 552B parameters; not a laptop model.\n- not this job: DeepSeek-V4-Flash-0731 (DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)): 284B parameters; not a laptop model.\n- not this job: DiffusionGemma 26B A4B IT (Google LLC (RDAP registrant of gemma.ai; the card names Google DeepMind as author)): Needs a 60 GB GPU in Transformers or SGLang; not a laptop or 16 GB GPU model.\n- not this job: GLM-4.5-Air (JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai, per the first batch's check on 21 October 2026)): 106B total; not a laptop or 16 GB GPU model.\n- not this job: GLM-5.3 (JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai). The licence names Z.AI): 744B parameters.\n- not this job: GLM-5.3-Flash (Z.AI Co., Ltd (named in the licence's copyright line)): 320B parameters.\n- not this job: gpt-oss-120b (OpenAI (named as publisher on the card; the openai-api listing's terms name OpenAI OpCo, LLC)): 117B parameters; not a laptop model.\n- not this job: Hy3 (Tencent (named in the LICENSE file's copyright line)): 295B parameters.\n- not this job: Inkling (Thinking Machines Lab, Inc. (named on its model card page and in its training-content summary)): 975B parameters.\n- not this job: Inkling-Small (Thinking Machines Lab, Inc. (named on its model card page and in its training-content summary)): 276B parameters.\n- not this job: InternVL3.5-GPT-OSS-20B-A4B-Preview (Shanghai AI Laboratory (OpenGVLab), not named in the repository): A vision preview of 42.5 GB in BF16; not a laptop notes model.\n- not this job: Kimi K3 (Moonshot AI (the licence names Moonshot AI; the Kimi Platform terms name Moonshot AI PTE. LTD.)): 2.8T parameters.\n- not this job: Kolibri-1 (Aleph Alpha GmbH): 78B total and 49.8 GB at Q4; outside the laptop band, though German and English chat is its aim.\n- not this job: Kronos (No company. Yu Shi publishes it from the GitHub account shiyu-coder and the Hugging Face user account NeoQuasar (named ShiYu), with co-authors at Tsinghua University per the paper): Not a language model; no text input or output.\n- not this job: Laguna M.1 (Poolside, Inc. (named in Poolside's Terms of Use; the LICENSE.md copyright line reads Poolside)): 225B model; outside the 4B to 31B laptop class.\n- not this job: Laguna S 2.1 (Poolside, Inc. (named in Poolside's Terms of Use)): 118B model; outside the 4B to 31B laptop class.\n- not this job: LBS-27B (inclusionAI, Ant Group (named in the site footer of inclusion-ai.org); the work is credited to authors from Wuhan University of Technology, CASIA, Zhejiang University and Ant Group): Trained for click targets on screenshots, not text question answering.\n- not this job: MiniMax-M2.5 (MiniMax AI (named in the licences; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)): 229B model.\n- not this job: MiniMax-M2.7 (MiniMax (named in the licence's copyright line; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)): A 229B model; nothing fits a laptop or 16 GB GPU.\n- not this job: MiniMax-M3.1-Flash (MiniMax (named in the licence's copyright line; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)): Too large for a laptop or 16 GB GPU, the machines the use case names.\n- not this job: Mistral Large 4 (Mistral AI (RCS Paris 952 418 325)): Too large for a laptop or 16 GB GPU.\n- not this job: Nemotron 3 Super 120B A12B (NVIDIA Corporation): Too large for a laptop or 16 GB GPU.\n- not this job: Nemotron 3 Ultra 550B A55B (NVIDIA Corporation): Too large for a laptop or 16 GB GPU.\n- not this job: Olmo 3 7B Think (The Allen Institute for Artificial Intelligence (Ai2), a 501(c)(3) nonprofit, per its Responsible Use Guidelines): Thinking is always on with long traces, so a one-sentence reply in under 5 seconds is not what it is built for; Ai2 has Olmo 3 7B Instruct for that.\n- not this job: Ornith-1.5-397B (Ornith): 397B model.\n- not this job: Qwen3.5-122B-A10B (Alibaba Cloud (named in the licence's copyright line)): Far above the 4B to 31B laptop or 16 GB GPU band the use case states.\n- not this job: Qwen3.5-397B-A17B (Alibaba Cloud (named in the licence's copyright line)): Far above the laptop or 16 GB GPU band.\n- not this job: Qwen3.8-Flash-Next (Alibaba Cloud (Singapore) Private Limited, the registrant of qwen.ai. The licence names Qwen as copyright holder): Far above the laptop or 16 GB GPU band.\n- not this job: Qwen3-Coder-30B-A3B-Instruct (Alibaba Cloud (named in the licence's copyright line)): A coding model; the card aims it at agentic coding and tool use, not chat over documents.\n- not this job: Qwen3-Coder-Next (Alibaba Cloud (the Qwen team's employer; no LICENSE file names a copyright holder for this model)): A coding-agent model of 80B; outside the use case's size band and aim.\n- not this job: Qwen3-Omni-30B-A3B-Instruct (Alibaba Cloud (carried from the 20 October 2026 pilot; this model's files name no copyright holder)): Above the 4B to 31B band on a 16 GB GPU once the projector and 6 GB of KV at 64K are added; built for audio-visual chat.\n- not this job: Recursive Flint (Recursive Superintelligence, Inc. (the Hugging Face organisation's full name)): A coding model with reasoning that cannot be turned off; short spoken answers in 5 seconds are not its aim.\n- not this job: Ripple-65B Preview (Ricursive Intelligence, Inc.): 65B dense with always-on reasoning; outside the 4B to 31B laptop band.\n- not this job: Swift 1.5 Qwen3.8-Flash-Next (UkisAI): Far above the laptop or 16 GB GPU band.\n- not this job: Baidu Unlimited-OCR-2 (Baidu, Inc.): Not a chat model.\n\n28 listings in the category were added after this reading and aren't marked yet.\n\n## What stops agents finishing\n\n- rambling 600-character answers where one sentence was asked\n- model refuses lawful requests the user wants done locally\n\n## Around it\n\nBefore: vector-databases.query, embeddings.embed. After: text-to-speech.speak.\n\n## Where this comes from\n\nWeight 8 of 100 in Open-weight models: 4 of 51 accounts; vendor use-case pages name RAG and chat for most listings\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/open-weights.private-chat-knowledge.json, or the MCP tool get_use_case with id open-weights.private-chat-knowledge. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Chat and answer over private notes and documents",
        "url": ""
      }
    ],
    "description": "Chat and answer over private notes and documents: 8 companies fully do it, 19 with a limit, 90 carry the tag. Every listing in Open-weight models with the reason and a source.",
    "facts": null,
    "h1": "Chat and answer over private notes and documents",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/open-weights.private-chat-knowledge/",
    "published": "",
    "section": "agents",
    "title": "Chat and answer over private notes and documents | Anchor Terminal",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/open-weights.private-chat-knowledge/"
  },
  "tokens": {
    "markdown": 5100,
    "slim": 1280
  },
  "version": 1
}
