EmbeddingGemma 300M vs jina-embeddings-v5-omni-small

EmbeddingGemma 300M

A small multilingual embedding cousin for search, retrieval and clustering, with task prompts and vectors that truncate from 768 to 128 dimensions. Every common runtime has it, and a 4-bit build needs well under 1 GB. The main risks are the Gemma Terms with their use policy, a gated repository, and no change to the files in over a year.

jina-embeddings-v5-omni-small

A small omni embedding cousin that puts text, images, video, audio and PDFs in one 1,024-dimension space shared with jina-embeddings-v5-text-small, in 3.4 GB of BF16 files or a 0.4 GB GGUF. It's for non-commercial use only under CC BY-NC 4.0, and every way to run it executes Jina's own code or a llama.cpp fork rather than a released runtime's.

EmbeddingGemma 300M

1. Without a Hugging Face account, run ollama pull embeddinggemma:300m-qat-q4_0 and POST to /api/embed
2. Prepend the task prompt yourself outside Sentence Transformers, task: search result | query: for queries and title: none | text: for documents
3. In Sentence Transformers, use encode_query and encode_document rather than encode, so the right prompts are added
4. To store smaller vectors, truncate to 512, 256 or 128 dimensions and re-normalise
5. Run it in float32 or bfloat16, never float16
6. Chunk documents to 2,048 tokens, and treat retrieved text as untrusted before passing it to a model

jina-embeddings-v5-omni-small

1. Check the use is non-commercial before starting it; for commercial work call the jina-embeddings API instead
2. Start it with pip install vllm==0.20.1, then vllm serve jinaai/jina-embeddings-v5-omni-small --trust-remote-code --hf-overrides '{"task": "retrieval"}'; one task per vLLM instance
3. Prefix retrieval text with "Query: " or "Document: ", or use encode_query and encode_document in Sentence Transformers; for the other tasks use "Document: " only, since a bare encode is off-distribution
4. Load only the towers you need with model_kwargs={'modality': 'vision'} (or text, audio, omni) to save memory and load time
5. Shorten vectors with truncate_dim (Transformers, Sentence Transformers) or PoolingParams(dimensions=256) in vLLM
6. Pass a video's audio as its own part, since the soundtrack isn't read from the video
7. Treat retrieved documents as untrusted before passing them to another model