Niches · Open-weight models

Chat and answer over private notes and documents

Four or more companies' docs show a full fit.

8companies, full fit
19companies, fit with a limit
90companies carry the tag
8share of the category's work
4first-hand accounts

What counts as done

The model answers a question from the retrieved passages given in the prompt.

Measured, for our own tests: the answer uses only the retrieved passages, is factually correct against them in 9 of 10 test questions, and arrives in under 5 seconds for a one-sentence reply. No docs we read state these figures, so they don't count against a listing here.

How agents ask for it: chat over my Obsidian vault locally, a translation and chat model that stays on my laptop, answer from my wiki in one spoken sentence, a personal knowledge base with bug triage

Does the job 13

The docs show it meets the test above, and nothing we read says otherwise.

ListingWhy
Gemma 4 12B IT BB
Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author)
12B dense under Apache 2.0 with thinking off by default fits a 16 GB GPU at Q4_0 (7 GB) and 64K of context (about 1.6 GB with 8 of 48 layers global); Ollama and llama.cpp both serve it. source
Gemma 4 E4B IT B
Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author)
Apache 2.0, 5.15 GB Q4_0, thinking off by default, 131K context and multilingual speech and text; acc-45 ran it on a 24 GB laptop across three languages. source
Granite 4.2 3B C
International Business Machines Corporation (per IBM's naming guidance, as read for granite-guardian)
Apache 2.0, 2.2 GB at Q4_K_M, 131K context, enable_thinking false for direct answers and RAG named among its uses; acc-26 synthesised Wikipedia chunks into one spoken sentence with it. source
MiniCPM-o 4.5 C
OpenBMB (named as the MiniCPM-o team on the technical report; the GitHub LICENSE is the unfilled Apache template)
Apache 2.0 on the card, Q4_0 4.8 GB plus a 1.1 GB vision file fits a laptop, enable_thinking false gives direct answers, and it can speak the reply through OpenBMB's speech stack. source
NVIDIA Nemotron 3.5 Bolt E4B 4GB C
NVIDIA Corporation
conversational_mode appends a brevity instruction, thinking defaults off, RAGTruth 77.9 and GaRAGe 40.7 at Q4_K_M (NVIDIA's runs), and 6 GB fits any laptop. source
Qwen3-4B-Instruct-2507 C
Alibaba Cloud (named in the licence's copyright line)
4B Apache 2.0 instruct model with no thinking block, 262K context, 2.5 GB in Ollama; answers start at once on a laptop. source
Qwen3.5-4B C
Alibaba Cloud (named in the licence's copyright line)
4B Apache 2.0 vision-language model, thinking off with enable_thinking false, 2.7 GB at Q4 and 262K context; a one-sentence answer arrives in seconds. source
Qwen3.5-9B C
Alibaba Cloud (named in the licence's copyright line)
9B Apache 2.0 model with enable_thinking false per request, 6.6 GB in Ollama, 262K context; acc-25 built a Czech home assistant on it. source
Qwen3.6-27B B
Alibaba Cloud (named in the licence's copyright line)
Apache 2.0, enable_thinking false per request, 17 GB in Ollama with vision; acc-50 runs a personal wiki with summarisation and triage on it. source
Qwen3.8-27B B
Alibaba Cloud (named in the licence's copyright line)
Apache 2.0, enable_thinking false and reasoning_effort per request, 18 GB in Ollama with vision; acc-39 uses it as the think and vision model on a Mac. source
Qwen3-VL-4B-Instruct D
Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder)
Apache 2.0 by metadata and the GitHub LICENSE, no thinking block, 2.5 GB at Q4 with 262K context; an instruct model that answers at once. source
Qwen3-VL-8B-Instruct C
Alibaba Cloud (per the pilot's RDAP check of qwen.ai; this model's files name no copyright holder)
Apache 2.0 by metadata and the GitHub LICENSE, no thinking block, 5.0 GB at Q4 with 262K context; an instruct model that answers at once. source
Spark-X2.6-4B C
XHToken (the licence's copyright line); the Hugging Face organisation's name is SparkLLM
4B Apache 2.0 with a LICENSE file, thinking off per request, 3.5 GB GGUF and a 1M context; a short answer needs no reasoning block. source

Does the job, with a limit 32

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
Agents-A1-4B C
Shanghai Artificial Intelligence Laboratory (the InternAgent team there, per the card's recommended system prompt and the template's default prompt); the GitHub LICENSE leaves the copyright line as the Apache template
Apache 2.0, 2.7 GB GGUF and a thinking switch fit a laptop, but the model is tuned for long-horizon agent work and inserts an Intern-A1 system prompt with a fixed date unless you send your own. source
Apertus v1.5 8B C
ETH Zurich and EPFL (the Swiss National AI Institute, named in the Acceptable Use Policy)
A multilingual 8B aimed at precisely this, but the download is gated behind an Acceptable Use Policy and the architecture needs Swiss AI's forked vLLM, not a laptop runtime. source
Gemma 4 26B A4B IT BB
Google LLC (RDAP registrant of gemma.ai; the card names Google DeepMind as author)
Fast at 3.8B active with thinking off by default, but Q4_0 is 14.4 GB and Ollama defaults to 4K context under 24 GiB VRAM; a 16 GB GPU leaves little room for 64K of retrieved text. source
Gemma 4 31B IT BB
Google LLC (the registrant of gemma.ai; the card names Google DeepMind as author)
acc-02 uses Gemma 4 31B for chat and translation on a 128 GB laptop; at 17.6 GB Q4_0 it needs more than a 16 GB GPU, and 64K of context adds about 16 GB of cache at bf16. source
Gemma 4 E2B IT C
Google LLC (RDAP registrant of gemma.ai as recorded for the first batch's Gemma 4 listings; the card names Google DeepMind as author)
Runs on any laptop under Apache 2.0 with thinking off, but acc-45 found E2B garbled and weak against E4B; faithfulness in 9 of 10 answers is unlikely at this size. source
GLM-4.7-Flash B
JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai)
MIT, thinking off per request and 3B active, but 30B total and 17.5 GB at Q4 exceed a 16 GB GPU; a 24 GB machine works. source
gpt-oss-20b B
OpenAI (named in USAGE_POLICY and the model card; the LICENSE file's copyright line is the unfilled Apache template)
Apache 2.0 and 12 GB on a 16 GB GPU, but every reply is preceded by an analysis channel that cannot be disabled, so a one-sentence spoken answer in under 5 seconds is uncertain. source
Laguna XS 2.1 C
Poolside, Inc. (named in Poolside's Terms of Use)
OpenMDW licence and thinking off suit it, but 20 GB Q4_K_M needs a 32 GB laptop rather than a 16 GB GPU. source
LFM2.5-2.6B D
Liquid AI, Inc. (the licensor named in the LFM Open License)
Fits a laptop and the licence file is present, but commercial use is licensed only below a 10 million dollar revenue threshold and every reply starts with reasoning. source
Ling-3.0-tiny C
Ant Intelligence (Hang Zhou) Technology Co., Ltd (named as provider in the training-content summary); inclusionAI, Ant Group in the site footer of inclusion-ai.org
MIT-tagged, small and switchable, but the repository has no LICENSE file and a discussion reports loops. source
MiMo-V2.6 Distill Qwen 9B C
Xiaomi (the card and report name Xiaomi MiMo; no LICENSE file names a copyright holder)
MIT-tagged and small, but no LICENSE file and no copy of the base's Apache notice; thinking switches off. source
MiniCPM5-2B C
OpenBMB (named in the GitHub LICENSE's copyright line)
Apache 2.0 and tiny, but the model reasons before every short answer and users report it getting stuck in loops on simple questions. source
Muse Glimmer 30B BB
Meta (the verified Hugging Face organisation is "Meta Inc.", the card names Meta Superintelligence Lab, and the licence carries no copyright line)
Apache 2.0 and 128K context, but the 17 GB build misses a 16 GB GPU and thinking at low strength still precedes every short answer. source
Nanbeige4.3-33B C
BOSS Zhipin (Kanzhun Limited), unconfirmed
Thinking switches off, but 33B is above the use case's 31B range and only 3-bit builds (14.5 GB to 15.9 GB) fit a 16 GB GPU; without a system prompt the template injects a Chinese identity line. source
Nemotron 3.5 Lightning 30B A3B B
NVIDIA Corporation
Thinking switches off and the licence is permissive, but the 18.9 GB build misses a 16 GB GPU and needs a 24 GB card or 32 GB laptop. source
Nemotron 3 Nano 30B A3B B
NVIDIA Corporation
Thinking switches off, but the 18 GB build misses a 16 GB GPU, and the card says it is intended for English and code. source
Nemotron 3 Nano 4B C
NVIDIA Corporation
Fits any laptop with thinking off and RULER 128K 91.1, but it is English only and no grounding or brevity result is published. source
Nemotron 3 Nano Omni 30B A3B Reasoning C
NVIDIA Corporation
Instruct mode is short and fast, but 18.5 GB-plus misses a 16 GB GPU and the card states English only. source
occamy-1.0 C
Accio (Accio-Lab)
Thinking switches off and 3-bit builds of 14.5 GB to 16.8 GB reach a 16 GB GPU, but languages aren't stated and no grounding result is published. source
Ornith-1.5-35B-A3B C
Ornith
MIT is tagged but the repository has no LICENSE file and the card's link is dead; 21.7 GB needs 24 to 32 GB. source
Ornith-1.5-9B C
Ornith
Small and switchable, but the MIT licence is a tag with no LICENSE file and the card's link is dead, and default reasoning is long. source
Qwen3-30B-A3B-Instruct-2507 C
Alibaba Cloud (named in the licence's copyright line)
Apache 2.0, non-thinking and 262K context suit short grounded answers, but the 19 GB build misses a 16 GB GPU without offload. source
Qwen3.5-0.8B D
Alibaba Cloud (named in the licence's copyright line)
Below the 4B floor the use case states and scoped by the card to prototyping; Apache 2.0 and thinking off, so short answers are quick. source
Qwen3.5-2B D
Alibaba Cloud (named in the licence's copyright line)
Below the 4B floor the use case states and scoped by the card to prototyping; Apache 2.0 and thinking off, so short answers are quick. source
Qwen3.6-35B-A3B B
Alibaba Cloud (named in the licence's copyright line)
Apache 2.0 and thinking off per request, but 22 GB at 4 bits does not fit a 16 GB GPU; acc-02 reaches for it on a 128 GB laptop. source
Qwen3-VL-2B-Instruct C
Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder)
Below the 4B floor the use case states; Apache 2.0 by metadata, no thinking, 1.1 GB at Q4. source
Qwen3-VL-30B-A3B-Instruct C
Alibaba Cloud (per the pilot's RDAP check of qwen.ai; this model's files name no copyright holder)
Apache 2.0 and no thinking, but 18.6 GB at Q4 passes a 16 GB GPU; fits a 32 GB laptop. source
Qwen3-VL-32B-Instruct C
Alibaba Cloud (per the first batch's RDAP check of qwen.ai; this model's files name no copyright holder)
Apache 2.0 and no thinking, but 19.8 GB at Q4 passes a 16 GB GPU and 64 layers of full attention cost 16 GB of KV at 64K. source
Qwopus3.8-27B-Flash-V2 C
Jackrong, an individual Hugging Face account
Apache 2.0 by metadata but no LICENSE file, thinking off per request, 17 GB with vision; trained for shorter agent reasoning rather than grounded answers, and no account. source
SmolLM3-3B C
Hugging Face, Inc.
Apache 2.0 by metadata, thinking off per request and 1.9 GB at Q4, but below the 4B floor and no LICENSE file in the repository. source
Swift 1.5 Qwen3.8-27B C
UkisAI
Thinking off per request, 17 GB with vision, but the Swift Open License is free only below US$1M gross annual revenue; not a plain commercial licence. source
ThinkingCap (Qwen3.8 27B) C
BottleCap AI (named in the licence's copyright line)
Thinking off per request and 17 GB with vision, but free only for personal use and companies under 100 people and US$1M revenue; not a plain commercial licence. source

Doesn't do this job 41

Listings in the category that do something else, with the reason.

ListingWhy
Apodex-1.1-mini C
Apodex (the site's copyright line reads 2026 Apodex)
36B total parameters and a 21.7 GB smallest build exceed the use case's 4B to 31B on a 16 GB GPU, and the tune is for long-horizon agents. source
BigBang-v1 C
The Endless Frontier (endless-frontier)
36B total and a research-tuned reasoning model with BF16 only from the publisher; outside the 4B to 31B laptop band. source
Carbon-X C
Hugging Face (the HuggingFaceBio organisation, named Hugging Face Biology Research on the Hub)
Answers questions about variants and regulatory regions from DNA, not from retrieved notes. source
Cosmos-Reason2-2B C
NVIDIA Corporation
A camera-footage reasoner, not a notes assistant; NVIDIA points users to Cosmos 3 and no longer develops it. source
Darwin-36B-Opus C
VIDRAFT Inc. (named on vidraft.net); the card's citation names FINAL-Bench and VIDRAFT_LAB
A 35B reasoning merge whose answers begin with a think trace; outside the 4B to 31B short-answer band. source
DeepSeek-V3.2 C
DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)
A 671B model; not a laptop or 16 GB GPU fit. source
DeepSeek V4.1 Flash C
DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)
552B parameters; not a laptop model. source
DeepSeek-V4-Flash-0731 C
DeepSeek (the MIT licence names DeepSeek, and DeepSeek's platform terms name Hangzhou DeepSeek Artificial Intelligence Co., Ltd.)
284B parameters; not a laptop model. source
DiffusionGemma 26B A4B IT D
Google LLC (RDAP registrant of gemma.ai; the card names Google DeepMind as author)
Needs a 60 GB GPU in Transformers or SGLang; not a laptop or 16 GB GPU model. source
GLM-4.5-Air C
JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai, per the first batch's check on 21 October 2026)
106B total; not a laptop or 16 GB GPU model. source
GLM-5.3 B
JINGSHENG HENGXING TECHNOLOGY PTE.LTD (operator of Z.ai per its terms of use, and registrant of z.ai). The licence names Z.AI
744B parameters. source
GLM-5.3-Flash B
Z.AI Co., Ltd (named in the licence's copyright line)
320B parameters. source
gpt-oss-120b BB
OpenAI (named as publisher on the card; the openai-api listing's terms name OpenAI OpCo, LLC)
117B parameters; not a laptop model. source
Hy3 B
Tencent (named in the LICENSE file's copyright line)
295B parameters. source
Inkling B
Thinking Machines Lab, Inc. (named on its model card page and in its training-content summary)
975B parameters. source
Inkling-Small B
Thinking Machines Lab, Inc. (named on its model card page and in its training-content summary)
276B parameters. source
InternVL3.5-GPT-OSS-20B-A4B-Preview D
Shanghai AI Laboratory (OpenGVLab), not named in the repository
A vision preview of 42.5 GB in BF16; not a laptop notes model. source
Kimi K3 B
Moonshot AI (the licence names Moonshot AI; the Kimi Platform terms name Moonshot AI PTE. LTD.)
2.8T parameters. source
Kolibri-1 C
Aleph Alpha GmbH
78B total and 49.8 GB at Q4; outside the laptop band, though German and English chat is its aim. source
Kronos C
No company. Yu Shi publishes it from the GitHub account shiyu-coder and the Hugging Face user account NeoQuasar (named ShiYu), with co-authors at Tsinghua University per the paper
Not a language model; no text input or output.
Laguna M.1 C
Poolside, Inc. (named in Poolside's Terms of Use; the LICENSE.md copyright line reads Poolside)
225B model; outside the 4B to 31B laptop class.
Laguna S 2.1 C
Poolside, Inc. (named in Poolside's Terms of Use)
118B model; outside the 4B to 31B laptop class.
LBS-27B C
inclusionAI, Ant Group (named in the site footer of inclusion-ai.org); the work is credited to authors from Wuhan University of Technology, CASIA, Zhejiang University and Ant Group
Trained for click targets on screenshots, not text question answering.
MiniMax-M2.5 C
MiniMax AI (named in the licences; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)
229B model.
MiniMax-M2.7 C
MiniMax (named in the licence's copyright line; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)
A 229B model; nothing fits a laptop or 16 GB GPU. source
MiniMax-M3.1-Flash C
MiniMax (named in the licence's copyright line; MiniMax's platform terms name Nanonoble Pte. Ltd., Singapore)
Too large for a laptop or 16 GB GPU, the machines the use case names. source
Mistral Large 4 C
Mistral AI (RCS Paris 952 418 325)
Too large for a laptop or 16 GB GPU. source
Nemotron 3 Super 120B A12B B
NVIDIA Corporation
Too large for a laptop or 16 GB GPU. source
Nemotron 3 Ultra 550B A55B B
NVIDIA Corporation
Too large for a laptop or 16 GB GPU. source
Olmo 3 7B Think D
The Allen Institute for Artificial Intelligence (Ai2), a 501(c)(3) nonprofit, per its Responsible Use Guidelines
Thinking is always on with long traces, so a one-sentence reply in under 5 seconds is not what it is built for; Ai2 has Olmo 3 7B Instruct for that. source
Ornith-1.5-397B C
Ornith
397B model.
Qwen3.5-122B-A10B B
Alibaba Cloud (named in the licence's copyright line)
Far above the 4B to 31B laptop or 16 GB GPU band the use case states. source
Qwen3.5-397B-A17B B
Alibaba Cloud (named in the licence's copyright line)
Far above the laptop or 16 GB GPU band. source
Qwen3.8-Flash-Next C
Alibaba Cloud (Singapore) Private Limited, the registrant of qwen.ai. The licence names Qwen as copyright holder
Far above the laptop or 16 GB GPU band. source
Qwen3-Coder-30B-A3B-Instruct C
Alibaba Cloud (named in the licence's copyright line)
A coding model; the card aims it at agentic coding and tool use, not chat over documents. source
Qwen3-Coder-Next C
Alibaba Cloud (the Qwen team's employer; no LICENSE file names a copyright holder for this model)
A coding-agent model of 80B; outside the use case's size band and aim. source
Qwen3-Omni-30B-A3B-Instruct D
Alibaba Cloud (carried from the 20 October 2026 pilot; this model's files name no copyright holder)
Above the 4B to 31B band on a 16 GB GPU once the projector and 6 GB of KV at 64K are added; built for audio-visual chat. source
Recursive Flint C
Recursive Superintelligence, Inc. (the Hugging Face organisation's full name)
A coding model with reasoning that cannot be turned off; short spoken answers in 5 seconds are not its aim. source
Ripple-65B Preview D
Ricursive Intelligence, Inc.
65B dense with always-on reasoning; outside the 4B to 31B laptop band. source
Swift 1.5 Qwen3.8-Flash-Next C
UkisAI
Far above the laptop or 16 GB GPU band. source
Baidu Unlimited-OCR-2 C
Baidu, Inc.
Not a chat model. source

28 listings in the category were added after this reading and aren't marked yet.

What stops agents finishing

  • rambling 600-character answers where one sentence was asked
  • model refuses lawful requests the user wants done locally

Around it

Before: vector-databases.query, embeddings.embed. After: text-to-speech.speak.

Where this comes from

Weight 8 of 100 in Open-weight models: 4 of 51 accounts; vendor use-case pages name RAG and chat for most listings

The same as data: /api/v1/use-cases/open-weights.private-chat-knowledge.json, or the MCP tool get_use_case with id open-weights.private-chat-knowledge. Other jobs in this category: /api/v1/niches?category=open-weights.

Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.