# Embed and search on the agent's own machine

> Embed and search on the agent's own machine: 3 companies fully do it, 9 with a limit, 114 carry the tag. Every listing in Embeddings & rerankers with the reason and a source.

- Canonical: https://www.anchorterminal.com/niches/embeddings.local-private-index/
- Markdown: https://www.anchorterminal.com/niches/embeddings.local-private-index/index.md (~2,050 tokens)
- Slim: https://www.anchorterminal.com/niches/embeddings.local-private-index/index.min.md (~530 tokens, same facts, less prose, for token-sensitive contexts)
- JSON: https://www.anchorterminal.com/niches/embeddings.local-private-index/index.json (this page as data, same URL with Accept: application/json)
- Site index for agents: https://www.anchorterminal.com/llms.txt (the key docs and every listing in one file, at most 16 MB, about 4 million tokens: https://www.anchorterminal.com/llms-full.txt)
- API: https://www.anchorterminal.com/api/v1/index.json
- MCP: https://www.anchorterminal.com/mcp (the whole catalogue as tools, no key)
- DNS: dig +short TXT <slug>.grade.anchorterminal.com (the grade as a DNS TXT record, for a sandbox with no HTTP where names still resolve; also top., best.<category>. and <a>.vs.<b>. under the zone, and the same under grade.letme.dev)
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)
- Updated: 2026-11-15

Two or three companies' docs show a full fit.

| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |
| --- | --- | --- | --- | --- |
| 3 | 9 | 114 | 21 | 19 |

## What counts as done

The model has open weights under a licence that allows the use, a documented route on a common local runtime (Ollama, llama.cpp, vLLM or Sentence Transformers) with the pooling, prompt and dtype settings stated, a stated memory footprint, and no network call after download.

Measured, for our own tests: 1,000 short chunks embedded on a 16 GB laptop CPU or integrated GPU in under 5 minutes, and load time under 60 seconds once cached.

How agents ask for it: search my files without uploading them; give my agent a local memory index; run embeddings offline on my laptop; which embedding model fits on a 16 GB Mac

## Does the job

The docs show it meets the test above, and nothing we read says otherwise.

- **EmbeddingGemma 2 (Google LLC (RDAP registrant of gemma.ai; the card names Google DeepMind as author))**: Apache 2.0, not gated, no repository code; Sentence Transformers route with prompts, shipped pooling and normalise modules, bfloat16 or float32 stated, and loadable sizes per encoder (270M text only). An account embedded 49.3M items on a laptop iGPU. Load time unmeasured.
- **Qwen3-Embedding-0.6B (Alibaba Cloud (the registrant of qwen.ai per RDAP, carried from the Qwen3.8-27B check))**: Apache 2.0 with no repository code; Ollama (639 MB), Sentence Transformers, vLLM and TEI routes with last-token pooling, left padding and the query prompt stated. Accounts run it on a 16 GB M4 Mac and in llama.cpp. Load time unmeasured.
- **Qwen3-Embedding-4B (Alibaba Cloud (named in the copyright line of the 8B sibling's Apache 2.0 licence; this repository has no LICENSE file))**: Apache 2.0 with no repository code; Ollama (2.5 GB), Sentence Transformers, vLLM and TEI routes with last-token pooling, left padding and the query prompt stated. An account runs qwen3-embedding:4b for local RAG at 2,560 dimensions. Load time unmeasured.

## Does the job, with a limit

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

- **EmbeddingGemma 300M (Google LLC (RDAP registrant of gemma.ai; the card names Google DeepMind as author))**: Ollama serves a 622 MB build with no account, Sentence Transformers is documented and float16 is ruled out, but the Hugging Face download is gated behind the Gemma Terms and an account found the licence step went to a Kaggle 404.
- **Granite Embedding Small English R2 (International Business Machines Corporation)**: Apache 2.0, 47M parameters, a Sentence Transformers route with CLS pooling and no prefixes, and no repository code, but the card states no dtype or memory footprint and there is no Ollama or llama.cpp route.
- **jina-embeddings-v5-omni-small (Jina AI GmbH, a subsidiary of Elastic N.V. (per the jina-embeddings listing's 2 October 2026 check))**: Sentence Transformers and vLLM 0.20.1 routes with last-token pooling, prefixes and bf16 defaults stated, but CC BY-NC 4.0 bars commercial agents, every route needs trust_remote_code, and an account saw cached loads take over 8 minutes.
- **Nomic Embed (Nomic, Inc.)**: The Python client's inference_mode='local' runs the model through GPT4All after an automatic download, and accounts run Nomic in Ollama, but the client defaults to the hosted API, dynamic mode sends larger jobs remote, and no dtype or footprint is stated.
- **NVIDIA NeMo Retriever Embedding and Reranking NIMs (NVIDIA Corporation)**: Runs inside the owner's network on one GPU (a nominal 24 GB profile), but it is a NIM container rather than Ollama, llama.cpp, vLLM or Sentence Transformers, needs a supported NVIDIA GPU, and production use needs an AI Enterprise licence.
- **Qwen3-Embedding-8B (Alibaba Cloud (named in the licence's copyright line))**: Apache 2.0, Ollama Q4 at 4.7 GB and Sentence Transformers, vLLM and TEI routes are documented, but the card's examples use float16, where accounts and an open issue report all-NaN vectors with no error, a named blocker.
- **Qwen3-VL-Embedding-2B (Alibaba Cloud (RDAP names Alibaba Cloud (Singapore) Private Limited as the registrant of qwen.ai))**: Apache 2.0 and a Sentence Transformers route that runs no repository code, with a bfloat16 default in the vLLM example, but the card states no memory footprint, there is no GGUF or Ollama build, and the Transformers path imports the publisher's script.
- **Qwen3-VL-Embedding-8B (Alibaba Group (Tongyi Lab, named in the copyright line of the GitHub repository's Apache 2.0 licence))**: Apache 2.0 and a Sentence Transformers route with no repository code, but only BF16 weights with no stated footprint, and an account says the 8B needs about 18 GB of RAM, so it won't fit the 16 GB laptop the use case names.
- **Voyage AI embeddings and rerankers (Voyage AI Innovations, Inc.)**: voyage-4-nano is published as open weights on Hugging Face and shares the 4-series space, and an account runs it as a memory index on an 8 GB container; its card, licence and runtime settings weren't read here.
- **ZeroEntropy zerank and zembed (ZeroEntropy, Inc.)**: zembed-1 is Apache 2.0 with a Sentence Transformers route in bfloat16, but it needs trust_remote_code, a named blocker, and the card states no memory footprint for its 4B parameters.

- not this job: Amazon Nova Multimodal Embeddings (Amazon Web Services, Inc.): Hosted on Bedrock only; no weights are published.
- not this job: Cohere Embed and Rerank (Cohere Inc.): Hosted API and cloud marketplaces only; no weights to run on the agent's machine.
- not this job: Gemini Embedding (Google LLC): Hosted API only; no weights. An account running local ChromaDB still sends every embedding call to Google.
- not this job: Jina Embeddings and Reranker (Jina AI GmbH): This listing is the hosted API; the open weights are a separate listing under CC BY-NC 4.0.
- not this job: Mistral Embed and Codestral Embed (Mistral AI (RCS Paris 952 418 325)): Hosted API only; neither embedding model has published weights.
- not this job: OpenAI embeddings (OpenAI OpCo, LLC): Hosted API only; no weights.

## What stops agents finishing

- gated download whose licence step is broken (EmbeddingGemma 300M to a Kaggle 404)
- non-commercial licence (jina-embeddings-v5-omni-small) for a commercial agent
- float16 returns NaN vectors with no error (Qwen3-Embedding-8B, EmbeddingGemma)
- route needs trust_remote_code or a forked runtime
- GPU container that needs an enterprise licence for production use (NeMo Retriever NIMs)
- "local-first" tool that defaults to a hosted API

## Around it

Before: local-ai.fit-model-to-hardware, local-ai.chat-with-own-files. After: vector-search.local-embedded, agent-memory.coding-session-memory, agent-memory.user-facts-recall.

## Where this comes from

Weight 21 of 100 in Embeddings & rerankers: 19 of 62 accounts (91 account-use-case tags, accounts counted in each use case they fit, scaled to 100 and rounded by hand); local or private running was the most repeated constraint, in Show HN tools, Ask HN local RAG and Hugging Face discussions; overlaps heavily with the RAG use case

As data: https://www.anchorterminal.com/api/v1/use-cases/embeddings.local-private-index.json, or the MCP tool get_use_case with id embeddings.local-private-index. Draft, read 2026-11-15 from public docs and first-hand accounts, not yet reviewed by a person.

---

Query this catalogue directly over MCP: https://www.anchorterminal.com/mcp (Streamable HTTP, no key). How: https://www.anchorterminal.com/agents/#mcp
